1

How to solve this \begin{aligned} &\underset{V \in \mathbb{R}^{n \times n}}{\text{minimize}} & & \sum_k \|Y_k - A_k V \|_F^2 \\ &\text{subject to} & & V^T V = I, \end{aligned} where $Y_k \in \mathbb{R}^{m \times n}$ and $A_k \in \mathbb{R}^{m \times n}$ are given. $I$ denotes an identity matrix of size $n \times n$.

At first it seemed like an Orthogonal Procrustes problem. However, I am not sure if that is the case. or is it related to generalized SVD that is not very obvious to me?

By the way, is there any known orthogonal projection onto the orthogonality constraint?

--8< ------------

My partial attempt:

Form the Lagrangian: \begin{align} L(V, \Lambda) := \sum_{k=1}^K \|Y_k - A_k V \|_F^2 + {\operatorname{Tr}}\left\{\Lambda^T \left( V^T V - I\right)\right\}. \end{align}

Taking the gradient with respect to $V$ and then set it to zero, i.e., \begin{align} \nabla_V L(V, \Lambda) := (-2)^K \sum_{k=1}^K A_k^T \left( Y_k - A_k V \right) + V \Lambda^T + V \Lambda = 0. \end{align}

But now, how to solve this?

learning
  • 743
  • 1
    $\def\bV{\mathbf{V}}$ $\def\bU{\mathbf{U}}$ $\def\bS{\mathbf{S}}$You can reduce the objective to $$ \phi -2 {\rm tr}(\bU^\intercal \bV) + {\rm tr}(\bV^\intercal \bS \bV )$$ where $\phi=\sum_k \lVert {\bf Y}_k \rVert^2_F$, $\bU =\sum_k {\bf A}_k^\intercal {\bf Y}_k$ and $\bS=\sum_k {\bf A}_k^\intercal {\bf A}_k$. If $\bV$ is orthogonal the last term becomes ${\rm tr}(\bS)$. – Ted Black Sep 20 '24 at 17:56

1 Answers1

3

The problem can be transformed into the orthogonal Procrustes problem by noting that the problem can be simplified to the case $k=1$ through stacking the matrices, because

$$‖Y_1-A_1⋅X‖_F^2 + ‖Y_2-A_2⋅X‖_F^2 = \left\| \begin{bmatrix}Y_1 \\ Y_2\end{bmatrix} -\begin{bmatrix}A_1 \\ A_2\end{bmatrix}⋅X \right\|_F^2 $$

For a given matrix $X= UΣV^T$, then the closest orthogonal matrix is $UV^T$. (see Showing that matrix $Q=UV^T$ is the nearest orthogonal matrix to $A$.) However, this is not an orthogonal projection, since this is not a linear mapping to begin with. In fact, the question doesn't make sense because the space of orthogonal matrices is not a linear space.

Hyperplane
  • 12,204
  • 1
  • 22
  • 52