$\max{\text{Tr}|AXA^{T}+BXB^{T}|}$ where $X^2=I$

Question

Suppose $A,B\in\mathbb{R}^{n\times n}$ are given matrices. What is the value for this optimization problem and can this be expressed in terms of $A$ and $B$? \begin{align*} &\max{\text{Tr}|AXA^{T}+BXB^{T}|}\\ \text{s.t }& X=X^{T},X^2=I \end{align*} The problem is easy for $\max{\text{Tr}|AXA^{T}|}$, because $\text{Tr}|AXA^{T}|\leq ||A||_2||X||_2||A^T||_2\leq ||A||^2$ (the dimension normalizer is omit,$||·||_2$ is the Spectral 2-norm). Using this method we can get an upper bound: $||AA^T||_2+||BB^{T}||_2$ but the equality may not hold.

Update: For a Hermitian or real symmetric matrix $A$, $Tr|A|=\sum_i|\lambda_i| $, $\lambda_i$ are eigenvalues of $A$

Answer 1

Note that $X$ is an orthogonal matrix. Let $U$ be the unitary factor in the polar decomposition $AXA^T+BXB^T=PU$. Then \begin{align*} \operatorname{tr}|AXA^T+BXB^T| &=\operatorname{tr}\left((AXA^T+BXB^T)U^T\right)\\ &=\operatorname{tr}\left( \begin{bmatrix}AX&BX\end{bmatrix} \begin{bmatrix}A^TU^T\\ B^TU^T\end{bmatrix} \right)\\ &\le\left\|\begin{bmatrix}AX&BX\end{bmatrix}\right\|_F \left\|\begin{bmatrix}A^TU\\ B^TU\end{bmatrix}\right\|_F \quad\text{(Cauchy-Schwarz)}\\ &=\left\|\begin{bmatrix}A&B\end{bmatrix}\right\|_F^2 \quad\text{(as $X$ and $U$ are orthogonal)}\\ &=\operatorname{tr}(AA^T+BB^T)\\ &=\operatorname{tr}|AA^T+BB^T|. \end{align*} Hence the maximum of $\operatorname{tr}|AXA^T+BXB^T|$ is attained at $X=I$ and the maximum value is $\operatorname{tr}|AA^T+BB^T|=\operatorname{tr}(AA^T+BB^T)=\|A\|_F^2+\|B\|_F^2$.

Remark. The stronger inequality $|AXA^T+BXB^T|\preceq AA^T+BB^T$ (without taking traces on both sides) is not always true. Consider $B=0$, \begin{align*} &A=\pmatrix{2&-1\\ -1&2},\ X=\pmatrix{1\\ &-1},\ AA^T=\pmatrix{5&-4\\ -4&5},\ AXA^T=\pmatrix{3\\ &-3},\\ &AA^T-|AXA^T|=AA^T-3I=\pmatrix{2&-4\\ -4&2}\text{ is not PSD.} \end{align*}

Answer 2

The trace norm also known as the Schatten 1 norm is a norm so it obeys the triangle inequality which yields the result
$\big \Vert AXA^T +BXB^T\big \Vert_{S_1}\leq \big \Vert AXA^T\big \Vert_{S_1} +\big \Vert BXB^T\big \Vert_{S_1}\leq \text{trace}\big(AA^T +BB^T\big)$

Note the right hand inequality holds since e.g.
$\big \Vert AXA^T\big \Vert_{S_1} \leq \big \Vert A^TAX\big \Vert_{S_1}=\big \Vert A^TA\big \Vert_{S_1}=\text{trace}\big(AA^T \big) $ where $AXA^T$ and $A^TAX$ have the same eigenvalues but the former is normal so
$\sum_{k=1}^n \sigma_k^{(AXA^T)} =\sum_{k=1}^n \big\vert\lambda_k^{(AXA^T)}\big\vert= \sum_{k=1}^n \big\vert\lambda_k^{(A^TAX)}\big\vert\leq \sum_{k=1}^n \sigma_k^{(A^TAX)}=\sum_{k=1}^n \sigma_k^{(A^TA)}$
where the right hand side holds since multiplication by orthogonal matrices does not change singular values. The upper bound is met with equality iff $A^TA X$ is normal (see 'deferred proof' in Kronecker Product of Normal Matrices) which occurs iff $X$ commutes with $A^TA$.