5

Given a matrix $A$ and the set of all of its eigenvectors, it is possible to find all of the matrix’s eigenvalues by solving $A\vec v = \lambda\vec v$. Given the set of all eigenvalues, it is possible to find the corresponding eigenspaces by finding $\mathrm N(A-\lambda I)$.

It is also possible to find all of the eigenvalues independently of the eigenvectors by finding the zeroes of the characteristic polynomial $\lvert A – \lambda I\rvert$.

This makes me wonder, is it possible to find all of the eigenvectors or eigenspaces independently of the eigenvalues? If so, how?

My linear algebra instructor’s assistant directed me to this video, which I found unhelpful.

Rodrigo de Azevedo
  • 23,223
gen-ℤ ready to perish
  • 6,732
  • https://mathoverflow.net/questions/107891/can-eigenvector-be-found-without-computing-the-eigenvalue – Derek Allums Apr 24 '20 at 05:16
  • In a sense, not really. If you have an eigenvector, multiplying $A$ by it recovers the eigenvalue, so whatever method you use, if it computes eigenvectors it will essentially compute the eigenvalues (as illustrated by the method linked above). It does not work the other way, after having eigenvalue one needs to solve a linear system to find the eigenvector. So it makes sense to look for them separately. – Conifold Apr 24 '20 at 06:15
  • @Conifold Could you adapt that soft explanation into a formal proof as an answer? – gen-ℤ ready to perish Apr 26 '20 at 02:48
  • I do not think you can make a formal proof of a soft claim like "find independently". One can probably prove something like the complexity of finding eigenvectors and eigenvalues is essentially the same as of finding eigenvectors alone, while the complexity of finding eigenvalues alone is strictly lower. But even then I am not sufficiently versed in the theory to properly define the right kind of complexity. – Conifold Apr 26 '20 at 02:54
  • @Conifold Why do you say that “find independently” is a soft claim? – gen-ℤ ready to perish Apr 26 '20 at 02:55
  • Well, what is a formal definition of it? That we do not "see" the eigenvalues in the process? But then one can easily rephrase the usual way of solving the characteristic equation first, and then solving for eigenvectors so as not to "see" them, but use some equivalent intermediary. When you try to make something "formal" out of "independently" you'll see that "obvious" ways simply do not work. – Conifold Apr 26 '20 at 03:02
  • Hmm, I see what you’re saying. We could say, “The algorithm for isolating the eigenvectors cannot reference the eigenvalues $\lambda$, cannot reference the zeroes of the characteristic polynomial, and cannot reference any immediately recognisable derivative thereof,” but “immediately recognisable” is highly subjective! This is a more difficult problem than I first anticipated. . . . – gen-ℤ ready to perish Apr 26 '20 at 03:07
  • Right. A more objective way is to say that there is no "easy" way to recover eigenvalues from any intermediate or final output of the algorithm ("easy" compared to the complexity of the algorithm itself). But this is (intuitively) impossible here, because they can be "easily" recovered from the final output, the eigenvectors. This is what they often do in complexity theory: show that Y can be easily found from X, hence there can be no algorithm for finding X that avoids going through Y in an essential way. – Conifold Apr 26 '20 at 03:29
  • See a similar example showing that there is no way to find the sum of divisors of an integer without essentially factoring the integer (and hence this is a "hard" problem). – Conifold Apr 26 '20 at 03:44

3 Answers3

2

Some iterative methods find eigenvectors (approximately, at least) without going through the eigenvalues.

As a simplest possible approach, given your matrix $A$, you can pick an arbitrary starting vector $\mathbf x$, and repeatedly multiply it by $A$, scaling the result down whenever it gets too big (or scaling it up whenever it gets too small). This doesn't necessarily converge (you're in trouble if $A$ is a rotation matrix, for instance), but if it converges up to a scaling factor, it converges to an eigenvector.

(For most starting $\mathbf x$, you'll get the eigenvector associated to the eigenvalue that's largest in absolute value. You can also replace $A$ by $(A - t I)^{-1}$ to find the eigenvector associated to the eigenvalue closest to $t$.)

Once you have the approximate eigenvector, you can estimate the associated eigenvalue, but that's inescapable. In any case, the eigenvector comes first in this method.

Misha Lavrov
  • 159,700
  • What do you mean by “converges up to a scaling factor”? – gen-ℤ ready to perish Apr 26 '20 at 06:23
  • The vector $\mathbf x^{(n)}$ we get after $n$ steps won't actually be converging to a specific vector, because the scaling we do is arbitrary: we only scale to prevent our numbers from getting too big or too close to $0$. (Being an eigenvector is only defined up to scaling anyway.) By "convergence up to a scaling factor" I mean getting closer and closer to the eigenspace of the dominant eigenvalue. – Misha Lavrov Apr 26 '20 at 15:13
  • So what you’re meaning is that its direction will approach that of an eigenvector, but its magnitude could be anything? – gen-ℤ ready to perish Apr 26 '20 at 17:32
  • Exactly. We could get convergence out of that by normalizing our vector somehow: since it doesn't matter how, the easiest way is probably to divide by the absolutely-largest entry at every step. (Or we could transform into a unit vector, but that takes a bit more work to compute.) – Misha Lavrov Apr 26 '20 at 17:36
  • So I have to ask, what’s so great about $\mathrm e^{\sqrt{\ln x}}$ (as in your bio)? – gen-ℤ ready to perish Apr 26 '20 at 17:38
  • It's a fun intermediate function whose growth rate is slower than any polynomial in $x$ but faster than any polynomial in $\ln x$, that you're always kind of surprised to get in asymptotic analysis. – Misha Lavrov Apr 26 '20 at 17:43
  • I don’t feel like an answer to this question requires a proof. However, if I asked a separate question asking for a proof that $A^n\mathbf x$ becomes parallel to an eigenvector as $n\to\infty$ if $A$ isn’t a rotation matrix, would you be interested in providing an answer to that as well? – gen-ℤ ready to perish Apr 26 '20 at 21:43
  • I'd suggest asking a question more along the lines of "when does this happen, and what exactly can we guarantee?" because "isn't a rotation matrix" is more of an example than exactly the right condition. I might not be the best person to answer it, but I'd give it a shot. – Misha Lavrov Apr 26 '20 at 22:38
  • You may also be interested in this recent question, though it doesn't have an answer yet: https://math.stackexchange.com/questions/3638902/power-iteration-error-for-eigenvectors – Misha Lavrov Apr 26 '20 at 22:40
1

Some words about the reference -given by the OP and that I don't know- taken from the AMM.

i) If you randomly choose $u$ and if $char(F)=0$, then, with probability $1$, your polynomial $a_0+\cdots+a_kt^k$ is the minimal polynomial of $A$ (and not only the one of $u$). cf. my post in

A better algorithm to find the minimum polynomial of a matrix?

ii) The title of this paper “Computing Eigenvalues and Eigenvectors Without Determinants" seems to be a joke; indeed, the decomposition of $A^ku$ as a linear combination of the previous calculated vectors needs the calculation of the reduced row echelon form (which is a hidden determinant).

iii) Your factorization $(t-\lambda)Q(t)$ over $F=\overline{\mathbb{Q}}$ (for example) can only be done in an approximate way; moreover, since $\lambda\in spectrum(A)$ , you obtain the eigenvalue before the eigenvector $Q(A)u$!!!

Everything looks tautological in your posts.

iv) About the Misha's good answer. It is false to say that the eigenvalue does not appear during the calculation; at each step, if we scale the result, then we obtain an approximation (at least of the absolute value) of the considered eigenvalue.

0

I am convinced that it is more likely than not that it is impossible to isolate the eigenvectors without referencing the eigenvalues $\lambda$, without referencing the zeroes of the characteristic polynomial, and without referencing any immediately recognisable derivative thereof.

Nevertheless, my instructor sent me an article that is as close to a solution as I expect is possible in that it does not require the eigenvalues from the beginning, and it is a single process that returns them both, rather than sticking together an eigenvalue-finding process and an eigenvector-finding process and calling it “one process.”

The algorithm is described by William A. McWorter, Jr., and Leroy F. Meyers in their paper “Computing Eigenvalues and Eigenvectors Without Determinants” in Mathematics Magazine, vol. 71, no. 1 (Feb. 1998):

Let $\newcommand{\u}{\vec{u}} \u$ be any nonzero vector in $\newcommand{\F}{\Bbb{F}} \F^n$ [the same field that matrix $A$ comes from]. Since $\F^n$ has finite dimension $n$, the $n+1$ vectors $\u, A\u, A^2\u, \dots, A^n\u$ are linearly dependent. Let $k$ be the smallest positive integer such that $a_0\u+a_1A\u+a_2A^2\u+\cdots+a_kA^k\u=\vec0$, for some $a_0,\dots,a_k$ in $\F$ with $a_k\neq0$. Algebraic closure ensures that the polynomial $a_0+a_1t+a_2t^2+\cdots+a_kt^k$ in $\F[t]$ is factorable as $(t-\lambda)\,Q(t)$ for some $\lambda$ in $\F$ and some polynomial $Q(t)$ in $\F[t]$. Hence $(A-\lambda I)\,Q(A)\,\u=\vec0$ [this is part of the proof, not the process]. The minimality of $k$ implies that the vector $Q(A)\,\u$ is nonzero and so is an eigenvector. . . .

If you can eyeball the factorisation $(t-\lambda)\,Q(t)$, then at least you don’t have to start with the eigenvalues.

gen-ℤ ready to perish
  • 6,732