Trace minimization subject to constraints

Question

I have seen in an article that

$ \min_{\mathbf{K}} \hspace{0.2cm} tr[\mathbf{K} \Sigma \mathbf{K}^T]$ s.t. $ \mathbf{KH} = \mathbf{I} $

where $\mathbf{H}$ is of full column rank yields,

$\tilde{\mathbf{K} } = (\mathbf{H}^T\Sigma^{-1}\mathbf{H})^{-1}\mathbf{H}^T\Sigma^{-1}$.

Does anyone aware of some theorem related to this result.

Does $\Sigma$ have any special characteristics other than being invertible? — copper.hat, Aug 01 '14 at 06:14
Not exactly. For simplicity you can assume its mean is zero. But don't think it is useful. — shani, Aug 01 '14 at 06:20
It is one of the results used in a thesis. I think the above solution is correct. But couldn't find a reference for that. The thesis also not giving the mathematical proof. So it must be in somewhere. — shani, Aug 01 '14 at 06:40
$\Sigma $ should be positive semidefinite. Otherwise, the problem can be unbounded. — The Pheromone Kid, Aug 01 '14 at 06:58
Yes. It is positive definite. Sorry I could not mention about it. Any suggestions for a reference? — shani, Aug 01 '14 at 07:01

score 4 · Accepted Answer · answered Aug 01 '14 at 08:52

First, let $L=(\sqrt{\Sigma})^{-1}H$ the minimization problem becomes $$ \min_{K:K\sqrt{\Sigma}\cdot L=I}{\rm tr}((K\sqrt{\Sigma})(K\sqrt{\Sigma})^T) =\min_{S:SL=I}\,{\rm tr}(SS^T) =\min_{S:SL=I}\,\Vert S\Vert^2_{HS} $$ Where $\Vert\cdot\Vert_{HS}$ is the Hilbert-Schmidt norm. This minimization problem is well-known, and its solution goes back to Penrose:"On best approximate solutions of linear matrix equations". The minimum is attained at a unique $S$ which is the pseudo-inverse of $L$. In our case : $S^{+}=(L^TL)^{-1}L^T$ because $L$ has full rank. Rolling back we see that the solution of the original minimization problem is given by $\tilde K=(H^T\Sigma^{-1}H)^{-1}H^T\Sigma^{-1}$.

Trace minimization subject to constraints

1 Answers1