1

Let $\mathbf{A}\in\mathbb{R}^{m\times n}$. I wish to solve

$$\mathrm{argmax}_{\mathbf{X}\in\mathbb{R}^{n\times k}} \left\|\mathbf{AX}\right\|_F$$

under constraint $\left\|\mathbf{X}\right\|_F=c$, with $c>0$.

We have that $\left\|\mathbf{Ax}\right\|_F$ s.t. $\left\|\mathbf{x}\right\|_2=1$ is maximized when $\mathbf{x}$ is the first right singular vector of $\mathbf{A}$ (that is, the right singular vector $\mathbf{v}_1$ corresponding to the largest singular value $\sigma_1$ of $\mathbf{A}$).

My intuition is that any rank-1 matrix of the form $\mathbf{X}=\mathbf{v}_1 \mathbf{b}^T$, with $\mathbf{b}\in\mathbb{R}^k$ and $\left\|\mathbf{b}\right\|_2=c$, solves my problem. That is, any matrix whose columns are $b_i\mathbf{v}_1$, with $\sum_{i=1}^k b^2_i=c^2$. Does it seem correct? How can I prove that?

  • You can use Lagrange multipliers and applying some matrix calculus should give the result. – obareey Apr 22 '24 at 16:00
  • Hint: Using the SVD of $A$ and unitary invariance of the Frobenius norms, you can reduce to maximizing $|\Sigma X|_F$ s.t. $|X|_F = c$ where $\Sigma$ is the matrix of singular values of $A$ with diagonal principle submatrix. – whpowell96 Apr 22 '24 at 16:07
  • I guess we cannot get rid of $V^T$, as $\left|\Sigma V^T X \right| \neq \left| \Sigma X \right|$ for a general X – Djoudjou Apr 23 '24 at 09:42

1 Answers1

3

First of all, it is equivalent to consider the objective function $\|AX\|_F^2$, which can be rewritten as $$ \|AX\|_F^2 = \operatorname{tr}((AX)^T(AX)) = \operatorname{tr}(X^TA^TAX) = \operatorname{tr}(A^TAXX^T)\\ = \operatorname{tr}[(A^TA)(XX^T)] $$ To put this another way: in terms of the Frobenius inner product $\langle P,Q\rangle = \operatorname{tr}(P^TQ)$, we are looking to maximize $\langle A^TA, XX^T\rangle$.

To put this another way, we are given a positive semidefinite matrix $P_A = A^TA$ (which is generically of rank $m$ if $n > m$ and generically invertible otherwise), and we are looking for a positive semidefinite matrix $P_X = XX^T$ (which is generically of rank $k$ if $n > k$ and generically invertible otherwise). Our goal is to find the positive semidefinite matrix $P_X$ that maximizes $\operatorname{tr}(P_AP_X) = \langle P_A,P_X \rangle$ subject to the constraint that $\operatorname{tr}(P_X) = c$ and, in the case that $n > k$, the additional constraint that the rank of $P_X$ is at most equal to $k$. Notably, your suggested maximizer is of rank-1, so this constraint on the rank of $P_X$ is irrelevant.

From here, the problem is straightforward to solve. This inequality tells us that $$ \operatorname{tr}(P_AP_X) = \langle P_A,P_X \rangle \leq \sigma_{\max}(P_A)\operatorname{Tr}(P_X) = \sigma_\max^2(A) \cdot c^2. $$ From there, it suffices to show that if each column of $X$ is a multiple of $\mathbf v_1$, then the same holds for $P_{X}$, and it follows that $$ P_AP_X = A^TA P_X = \sigma_\max^2(A) P_X, $$ so that $\operatorname{tr}(P_AP_X) = c^2\sigma_\max^2(A)$ and the upper bound is attained. Thus, $X = \mathbf v_1\mathbf b^T$ indeed maximizes the objective function.

Ben Grossmann
  • 234,171
  • 12
  • 184
  • 355