1

I'm trying to figure out if the following linear program has a closed form solution

$$ \begin{align} \min_{x \in \mathbb{R}^{n}} ||x&-\alpha||^2 \\ Ax &= b \\ x &\geq 0 \end{align} $$ where $\alpha$ is a constant vector in $\mathbb{R}^{n}$, $A$ is a $m \times n$ matrix with real coefficients, $x \in \mathbb{R}^{n}$ and $b \in \mathbb{R}^m$. Any hint on how to figure this out is also welcome.

Edit: The Lagrangian is $$ ||x-\alpha ||_2^2 -\lambda x+\mu(Ax-b) $$ and the KKT conditions are $$ 2(x-\alpha) - \lambda \mathbf{1} + \sum_i \mu_i A_i = 0 \\ \lambda \cdot x = 0 \\ x \geq 0 \\ A_i x - b_i = 0 \quad \forall i \\ \lambda \geq 0 $$ where $A_i$ is the $i$th line of matrix $A$.

RobPratt
  • 50,938
Dalamar
  • 789

1 Answers1

2

I'm not sure whether your problem has a closed form solution (it'll probably depend on $A$, $b$, $\alpha$), but let's try to work it out.

To start with, we need to fix the KKT's - yours are not correct. Note that the condition $x\ge 0$ is a shortcut for $x_i \ge 0$ for all $i$. Then, the Lagrangian is $$ L(x,\lambda,\mu) = \lVert x-\alpha\rVert^2 - 2\lambda^T x + 2\mu^T(Ax-b) $$ where $\lambda \in \mathbb{R}^n$ and $\mu \in \mathbb{R}^m$ (we use $2\lambda$ and $2\mu$ because it'll be useful in a moment). Now we can write the KKT's: \begin{align} 2(x-\alpha) - 2\lambda + 2A^T\mu &= 0 \tag{1}\label{1}\\ x &\ge 0 \tag{2}\label{2}\\ Ax - b &= 0 \tag{3}\label{3}\\ \lambda &\ge 0 \tag{4}\label{4}\\ \lambda_i x_i &= 0 \qquad \forall i=1,\dots,n. \tag{5}\label{5} \end{align}

From (\ref{1}), we get $$ x=\alpha+\lambda-A^T\mu $$ (see how the 2's simplify).

From now on, we'll assume $m<n$ and $A$ full row rank. Indeed, if $m \ge n$, your problem has either a single solution or no solutions depending on whether $b$ is in the column space of $A$ or not, respectively. Injecting the expression for $x$ into the primal feasibility condition (\ref{3}), we get \begin{align} A(\alpha+\lambda-A^T\mu) = b &\implies \mu = (AA^T)^{-1}(A\alpha + A\lambda - b) \\ &\implies x=\alpha+\lambda-A^T(AA^T)^{-1}(A\alpha + A\lambda - b) \end{align} or equivalently $$ x_i = \alpha_i+\lambda_i-A_i^T(AA^T)^{-1}(A\alpha + A\lambda - b) $$ for all $i$ and where $A_i$ is the $i$th column of $A$ - we can take the inverse of $AA^T$ because of the assumptions we made on A.

At this point, one would use the fact that (\ref{5}) implies that at least one between $x_i$ and $\lambda_i$ must be equal to zero and try to come up with the solution. Unfortunately, in our case, the term $A\lambda$ couples all the variables together and the problem is not trivial (the case $n=2$ and $m=1$ should be doable, but it'll take you some time).

To conclude, the KKT method doesn't seem to yield a closed form solution. But maybe there are other, better approaches (I couldn't come up with any, though). I hope this was helpful, anyway.

dvdgrgrtt
  • 547