Why is $\max_{c^TBc=1}c^T a a^Tc =a^TB^{-1}a$?

Question

I'm having some trouble in understanding the last step in this sequence of equalities:

$$\max_{c}\frac{c^Taa^Tc}{c^TBc}=\max_{c^TBc=1}c^T a a^Tc =a^TB^{-1}a$$

I would think that the maximum would be equal the biggest eigenvalue of the matrix $B^{-1}aa^T$

@RodrigodeAzevedo Yes. ;) – An old man in the sea. Jun 26 '18 at 11:28 — An old man in the sea., Jun 26 '18 at 11:28

A.Γ. · Accepted Answer · 2018-06-26T10:16:03.360

2

I assume $B$ is positive definite. Defining $x=B^{1/2}c$ the problem is $$ \max_{c^TBc=1}|a^Tc|^2=\max_{\|x\|=1}|a^TB^{-1/2}x|^2=\|B^{-1/2}a\|^2=a^TB^{-1}a. $$

EDIT: the eigenvalues of $B^{-1}aa^T$ are the same as the eigenvalues of the (similar) matrix $B^{-1/2}aa^TB^{-1/2}$, which has the same nonzero eigenvalues as the matrix (actually, number) $a^TB^{-1/2}B^{-1/2}a=a^TB^{-1}a$. Your conclusion is right: the maximum is the largest eigenvalue of $B^{-1}aa^T$, and it is $a^TB^{-1}a$.

edited Jun 26 '18 at 10:16

answered Jun 26 '18 at 09:42

A.Γ.

30,381

Much obliged ;) – An old man in the sea. Jun 26 '18 at 11:28
Dear A.G., by the way, how does one prove the 1st equality in my question. I thought I could do it, but it's taking a bit more time than I though. A hint should suffice. ;) – An old man in the sea. Jun 28 '18 at 16:00
Also, how does one prove the second equality in your answer? I thought I understood it... – An old man in the sea. Jun 28 '18 at 16:37
1

@Anoldmaninthesea. The 1st equality in the question follows from the fact that both numerator and the denominator are homogeneous in $c$, so if $c^TBc=\rho^2\ne 1$ it is possible to consider $\tilde c=\frac{c}{\rho}$. – A.Γ. Jun 28 '18 at 19:29
2

@Anoldmaninthesea. One way to understand the second equality is the Cauchy-Schwarz inequality $u^Tv=\langle u,v\rangle\le|u||v|$ with equality iff $u$ and $v$ are parallel. – A.Γ. Jun 28 '18 at 19:32
Thanks, I got it. ;) – An old man in the sea. Jun 28 '18 at 19:52

V. Vancak · Answer 2 · 2018-06-26T13:27:19.660

You can also use Lagrange multiplier to show it, i.e., you problem is equivalent to $$ \arg \max_{c, \lambda} \left( c'Ac - \lambda (c'Bc - 1) \right), $$ where $A = aa'$. The first order condition is $$ 2Ac - 2\lambda Bc=0, $$ i.e., $$ Ac=\lambda Bc, $$ that is a generalized eigenvalue problem. If $B$ is invertible, then $$ B^{-1}Ac=\lambda c, $$ that is the vector $c$ that satisfies the F.O.C, must satisfy this linear, which is by the definition pairs of eigenvectors and eigenvalues of $B^{-1}A$. As such, the final question is which pair to choose? So, going back to the original formulation you have in the nominator $$ c'Ac=c' \lambda_A c = \lambda_A \sum_{i=1}^nc_i^2, $$ where the pair of largest eigenvalue and eigenvector of $A$ correspond to the same pair in $B^{-1}A$.

Alternatively you can use even more basic calculus to show it. Recall that $$ \left( \frac{f(x)}{g(x)} \right)' = \frac{f'g - fg'}{ g^ 2 }, $$ hence the F.O.C is, $$ \left(\frac{c'Ac}{c'Bc} \right)' = \frac{2Ac c'Bc - 2c'AcBc}{(c'Bc)^2} = 0, $$ rewriting the LHS and multiplying both sides by $c'Bc/2$ you have $$ Ac - Bc\frac{ c'Ac }{c'Bc} = 0, $$ Note that $\frac{ c'Ac }{c'Bc}$ is a (positive) scalar, so denote it by $\lambda$, thus you'll get the generalized eigenvalue problem, i.e., $$ Ac = \lambda B c, $$ hence as previously you have $$ B^{-1}Ac = \lambda c, $$ i.e., the mximum is attained for $c$ that is eigenvector of $B^{-1} A$ and the maximum value equals its largest eigenvalue.

Why is $\max_{c^TBc=1}c^T a a^Tc =a^TB^{-1}a$?

2 Answers2