2

I'm trying to understand the time complexity for the conjugate gradient method, when a fair amount of system details are considered.

The first thing I'm not quite sure about, is the dependence. Previous answers (here) assert it is $O(m\sqrt{k})$, where $m$ is the number of non-zero entries. Then given $d$ the density of the matrix (as defined here), this would result in a $O(dn^2\sqrt{k})$ complexity.

On the other hand, different sources (here and here) report that the $O(n^2)$ time complexity is actually that of the iteration, and since $n$ iterations are needed at most, the effective time complexity is actually $O(n^3)$ or better, $O(dn^3\sqrt{k})$, where we intend "time complexity" as "the time required to obtain the solution of the linear system, as a function of the matrix size".

On the other hand, I have found many additional confusing (for me) details. Particularly, they stated:

  • that $n$ is the number of maximum steps to find the solution in theory, yet not in practice, due to round-off errors (which would imply some even larger dependence on $n$);
  • that $n$ is the number of maximum steps to find the exact solution, but that a lot less steps are needed to achieve an approximate solution (which would explain why the additional $n$ term is occasionally discarded in the time complexity?).

So I wanted to settle this once and for all: what is the time complexity of the conjugate gradient method for solving a linear system, for a generic non-sparse matrix?

jackphen
  • 137
  • 2
    For a dense matrix with exact arithmetic CG takes at most $n$ iterations with a matrix multiply at each iteration so the (time) cost is $n^3$. If the condition number features in the complexity then this is inexact arithmetic. – copper.hat Apr 26 '20 at 03:20

0 Answers0