3

We know that the eigenvalues of a Hessian matrix can provide information about the curvature of a function under study.

Specifically, we know that the sign of the eigenvalues give us information whether the function is increasing or decreasing.

I am curious to know if the magnitude of eigenvalues of the Hessian of a function gives information about the steepness of curvature of the given function?

I searched alot but I cannot find any relevant sources to learn this from. Please share any sources you have. It helps me.

Kishan
  • 33
  • Well we know that the determinant of the Hessian gives us information, and the determinant is the product of the eigenvalues. – rb612 Jun 28 '18 at 13:32
  • Examine the related quadratic form. – amd Jun 28 '18 at 17:43
  • That is good interpretation but there are two concerns for me.
    1. My understanding is that each unique eigenvalue corresponds to movement in one direction. I would like to know, say we have $\lambda_1$ corresponds to independent variable $\mathit{x}_1$ then if $\lambda_1 = 0.1$ is less steep compared to when $\lambda_1 = 5$ in the direction of $\mathit{x}_1$ variable in the space that has multiple independent variables.
    2. The determinant is not a good indicator when itnot extreme points. source
     – Kishan Jun 28 '18 at 17:45
  • @amd Actually it has more than 3 independent variables and it cannot be visualized. I tried searching literature. I want to see if the value of eigenvalues can help us have a rough idea has to how the geometry in higher dimensions is, when we look at dimension at a time. (I can check on geometry that can be plotted but I am not sure generalizing from looking at lower dimensions is a good idea?) – Kishan Jun 28 '18 at 17:50
  • Where did I say anything about trying to visualize it? I’m giving you a related topic to research. The Hessian is the core of the quadratic form that is the second-order term of the Taylor series. Bilinear forms and the meaning of their eigensystems have been studied extensively. – amd Jun 28 '18 at 17:55
  • Hi amd, Thank you so much. I understood what you meant now little bit. I don't have a formal knowledge of mathematics at this level. I learn by myself usually. wikipedia's section "Geometric meaning" really helped me. But what I understand is not sufficient to put in practice. Please, can you share any source that lets me understand a toy example? I will work hard to understand it and take it from there. (the function I have is not homogenous so I want to analyze systematically if I can, step by step since I am from electrical bkgrd) – Kishan Jun 28 '18 at 18:37
  • After some search, the eigenvalues sign can change the direction of eigenvector while its magnitude only increases the length of an eigen vector. But length of eigen vector cannot tell anything useful, we only check the direction. Can we say that magnitude of eigenvalue doesn't really give us any useful information? only the sign gives useful information? – Kishan Jun 28 '18 at 18:59
  • Magnitudes of eigenvectors are mostly meaningless. Any nonzero scalar multiple of an eigenvector is also an eigenvector. The magnitude of the corresponding eigenvalue gives the amount of “stretch” in that direction. For a quadratic form, nonzero eigenvalues are the reciprocal squares of the semi-axis lengths, which are directly related to curvature. However, I’m not sure how much use any of this will be when you’re not at a critical point since the behavior of the function in a small neighborhood of a non-critical point is dominated by the gradient. – amd Jun 30 '18 at 00:11

0 Answers0