Adding trace operator to a constraint in optimization?

Question

MIN tr(ABCD)
s.t BC=I

An optimization problem with trace in the objective function, but matrix equation in the constraint. So, you see the problem right away when constructing the Lagrangian (you end up adding scalar with matrices).

I'm reading the solution and it goes like this when building the Lagrangian:

L=tr(ABCD) - tr((BC-I)*Z)

Can you explain this solution? Where did the Z come from? how does it help solving the problem mentioned above. How was tr() added to the constraint.

PS: that wasn't the exact problem, just a mock, but it conveys the idea. — John Deterious, Apr 08 '19 at 19:24
@MarkFischler No, it is a symmetric matrix. Somehow it helps. — John Deterious, Apr 08 '19 at 19:25
And the condition that $BC = I$ can't be what you meant, because this becomes finding $\min \mbox{ tr}(AD)$. — Mark Fischler, Apr 08 '19 at 19:26
@MarkFischler This is just a toy example, you are correct. But, still, regardless of that point, what's the answer? — John Deterious, Apr 08 '19 at 19:27
@JohnDeterious I have the similar question as you did, do you have the answer now? — FantasticAI, Nov 11 '22 at 20:11

score 1 · Accepted Answer · answered Apr 08 '19 at 20:14

I think, you need the trace in this setting, since the constraint is not scalar, but matrix-valued. The $Z$ is your matrix-valued Lagrange multiplier, which cannot be scalar in this setting.

I have written down a the Lagrange approach for such very general equality constraints: \begin{align} \max_{x \in X} f(x) \\ \text{s.t. g(x) = 0}, \end{align} where $f:X \rightarrow \mathbb{R}$, and $g: X \rightarrow Y$, where $X$ and $Y$ are Hilbert spaces (like $\mathbb{R}^k$, $\mathcal{L}^2$,...).

The Lagrange function is given by $$L(x, \Lambda) = f(x) + \langle \Lambda, g(x) \rangle_Y,$$

where $\Lambda \in Y$ is the Lagrange multiplier, and $\langle \cdot, \cdot \rangle_Y$ is the inner product of $Y$.

The question is: Does the set of matrices that is some superset of the image of $BC-I$ in your notation, form a Hilbert space $Y$ with the trace of the product of two matrices being their inner product; i.e. $\langle A, B \rangle_X := \mathrm{trace}(AB)$? That is at least not obvious for me.

"Z is your matrix-valued Lagrange multiplier" Makes perfect sense. Thank you. — John Deterious, Apr 08 '19 at 23:21
Does the answer explain where the trace comes to be, feeling still confused? — zyxue, Aug 28 '21 at 22:21
I find the trace explained at https://math.stackexchange.com/a/1104412/348916, it's from inner product $\langle A, B \rangle = \text{tr}(A^T B)$, e.g. $A$ is the Lagrange multiplier, and $B$ is the matrix constraint. — zyxue, Aug 29 '21 at 16:37

Adding trace operator to a constraint in optimization?

1 Answers1