14

If $\lambda$ is the largest eigenvalue of a real symmetric $n \times n$ matrix $H$, how can I show that: $$\forall v \in \mathbb{R^n}, ||v||=1 \implies v^tHv\leq \lambda$$

Thank you.

3 Answers3

31

Step 1: All Real Symmetric Matrices can be diagonalized in the form: $ H = Q\Lambda Q^T $ So, $ {\bf v}^TH{\bf v} = {\bf v}^TQ\Lambda Q^T{\bf v} $

Step 2: Define transformed vector: $ {\bf y} = Q^T{\bf v} $.

So, $ {\bf v}^TH{\bf v} = {\bf y}^T\Lambda {\bf y} $

Step 3: Expand

$ {\bf y}^T\Lambda {\bf y} = \lambda_{\max}y_1^2 + \lambda_{2}y_2^2 + \cdots + \lambda_{\min}y_N^2 $

\begin{eqnarray} \lambda_{\max}y_1^2 + \lambda_{2}y_2^2 + \cdots + \lambda_{\min}y_N^2& \le & \lambda_{\max}y_1^2 + \lambda_{\max}y_2^2 + \cdots + \lambda_{\max}y_N^2 \\ & & =\lambda_{\max}(y_1^2 +y_2^2 + \cdots y_N^2) \\ & & =\lambda_{\max} {\bf y}^T{\bf y} \\ \implies {\bf y}^T\Lambda {\bf y} & \le & \lambda_{\max} {\bf y}^T{\bf y} \end{eqnarray}

Step 5: Since $Q^{-1} = Q^T, QQ^T = I $ \begin{eqnarray} {\bf y}^T{\bf y} &= &{\bf v}^TQQ^T{\bf v} = {\bf v}^T{\bf v} \end{eqnarray}

Step 6: Putting it all back together \begin{eqnarray} {\bf y}^T\Lambda {\bf y} & \le & \lambda_{\max} {\bf y}^T{\bf y} \\ {\bf v}^TH{\bf v} & \le & \lambda_{\max}{\bf v}^T{\bf v} \end{eqnarray}

By definition, $ {\bf v}^T{\bf v} = \|{\bf v}\|^2 $ and by definition $\|{\bf v}\| = 1$ \begin{eqnarray} {\bf v}^TH{\bf v} & \le & \lambda_{\max} \end{eqnarray} Boom!

Lifu Huang
  • 557
ssk08
  • 455
5

Hint:

Real symmetric matrices are diagonalizable.

Hint 2 (added after reading comments on posts):

A matrix is diagonalizable by a suitable choice of coordinates if and only if there is an eigenbasis. (taken from here)

Rudy the Reindeer
  • 48,301
  • 1
    The matrix $\begin{bmatrix}1& 2 \0 &0\end{bmatrix}$ is diagonalizable. You want to you orthogonal diagonalization. http://en.wikipedia.org/wiki/Symmetric_matrix#Properties – Jonas Meyer Mar 29 '11 at 12:03
  • 1
    ok, so $H$ is diagonalizable by an orthogonal matrix. Is the spectral theorem somehow related here? –  Mar 29 '11 at 13:19
  • 1
    @Rafael: According to Wikipedia, the spectral theorem gives you conditions under which a matrix is diagonalizable, so yes, I think the spectral theorem is related. – Rudy the Reindeer Mar 29 '11 at 14:13
4

Another hint along the same lines as Matt's: for which $\vec{v}$ is the LHS of your inequality maximised?

Bob Durrant
  • 386
  • 1
    not sure I understand... if $x$ is an eigenvector of $\lambda$ then $Hx=\lambda x$ and $x^tHx=x^t\lambda x=\lambda ||x||$ –  Mar 29 '11 at 12:56
  • 1
    You have a constraint on the $\mathcal{l}_{2}$ norm in your original question, so if $x$ is an eigenvector of $H$ s.t. $|x|=1$ what does this give you? You're there anyway I think. – Bob Durrant Mar 29 '11 at 13:30
  • 1
    FWIW, this is the more typical form of what you are trying to prove: Rayleigh quotient – Bob Durrant Mar 29 '11 at 13:32
  • 1
    then $x^tHx=x^t\lambda x=\lambda$ but this is only for eigenvectors of $\lambda$, and I need to show this is true for all $v\in \mathbb{R^n}$ –  Mar 29 '11 at 13:38
  • 1
    Okay, but you also know there is an orthonormal basis for $\mathbb{R}^n$ with the basis vectors the unit eigenvectors of $H$ (okay?). So you can rewrite $v$ as...? – Bob Durrant Mar 29 '11 at 13:47
  • 1
    ok I think I got it... if $Hv=\alpha v$ for some eigenvalue $\alpha$ then $v^tHv=\alpha ||v||=\alpha \leq \lambda$. Where's the mistake? Because I haven't used the fact that $H$ is symmetric. –  Mar 29 '11 at 13:48
  • 1
    For which matrices is there an orthonormal eigenvector basis? – Bob Durrant Mar 29 '11 at 13:50
  • 1
    All normal matrices? –  Mar 29 '11 at 13:53
  • 1
    ...and if all the eigenvalues are real (so that the inequality makes sense)? – Bob Durrant Mar 29 '11 at 15:21
  • 1
    The eigenvalues are real iff $H$ is Hermitian (symmetric in the real case). So the scheme for proving this is to diagonalise $H$ as $U^{T}DU$ where $U$ is unitary (Matt's hint), then use that $|Ux|=|x|=1$, then do what you did in your comment. – Bob Durrant Mar 29 '11 at 15:38