1

I'm reading a book related to numerical methods (Chapra, Canale) and on the topic of fixed point iteration, (copying from the book) we have to rearrange the function $\operatorname{f}(x) = 0$ so that $x$ is on the left side of the equation:

$x= {g}(x)$

Example:

Manipulation for the following

$x^2-2x+3=0$ will become $x = \frac{x^2+3}{2} $

and adding $x$ to both sides for the following

$sin(x) = 0$ will become $x = sin(x) + x$

--

Looking at them they somehow make sense, but for the first equation I would say that I could just add $x$ on both sides so we'd have an equation equal to $x$.

But now $x^2-x+3=x$ as is seems wrong; I could move the $x$ term on the left side to the right to produce a result same as above, but why should I when I already have $=x$?

I still don't understand when I should do an addition to both sides or just manipulating the placement of the term.

Thank you

Trevor
  • 13
  • 1
    Welcome to MSE. A thing about fixed-point iteration is that it doesn’t necessarily converge. Try to iterate using both your examples. Have you seen the visualisation with diagrams? https://en.m.wikipedia.org/wiki/Cobweb_plot – Benjamin Wang Jun 21 '20 at 14:11
  • 3
    It's a good question. There are many ways to rewrite an equation $f(x)=0$ to get one of the form $x=g(x)$, so that one then has an obvious fixed-point iteration. So a better question is, how do we tell when a particular rewriting gives us a "good" fixed-point iteration? Such a criterion can then be applied to the various options for rewriting. – hardmath Jun 21 '20 at 14:11
  • Trevor: I'd love to go through an example or two in illustrating my Answer. Normally I would take at least one of the examples proposed in the Question. However note that $x^2 -2x + 3 = 0$, which you consider first, has no real roots (completing the square shows $(x-1)^2 = -2$). So is there a preference you have for an example? – hardmath Jun 22 '20 at 14:43

1 Answers1

1

Here are some considerations for choosing one way rather than another to rewrite an equation into fixed-point form.

(1) Does the rewriting introduce any extra ("artifact") roots?

(2) Does the resulting function $g(x)$ serve as a "contraction mapping"?

Your book, Chapra and Canale, "Numerical Methods for Engineers"(?), will likely have some information on these ideas, but here are some quick thoughts.

The fixed-point iterations can always be defined from chosen starting point $x_0$:

$$ x_{k+1} = g(x_k), k = 0,1,2,3,\ldots $$

but whether these converge to a root of $f(x) = 0$ very much depends on how $g(x)$ is chosen and (to a lesser extent) on the choice of the starting point.

If $\{x_k\}$ converges to $x_*$ and $g(x)$ is continuous in a neighborhood of $x_*$, then $x_*$ is a solution of $x = g(x)$. Whether it is also a solution of the original equation $f(x)=0$ will depend on if the rewriting steps have introduced any artifact roots, i.e. the roots of $x=g(x)$ not satisfying the original equation $f(x)=0$. Typical ways this can happen are squaring both sides of equation or multiplying both sides by an expression that introduces a new root. Adding $x$ to both sides will not introduce an artifact root, because it is a reversible step (one can subtract $x$ from both sides).

The deeper property desired for $g(x)$ is that of being a contraction mapping in a neighborhood of the root $x_*$ we are seeking. For some historical background, see the excellent Answer there by Willie Wong. In any case the idea is that if $\{x_k\}$ converges, the terms should (eventually) be close to the limit $x_*$ and hence close to each other. The contraction mapping property guarantees that this will happen because for some constant $0\lt c \lt 1$:

$$ |x_{k+1} - x_k| = |f(x_k) - f(x_{k-1}| < c |x_k - x_{k-1}| $$

In other words the "gaps" between terms will eventually shrink by at least a factor $c$ with each iteration.

In most cases a good way to check for the contraction mapping property is by showing the function $g(x)$ is differentiable in a neighborhood of $x_*$ and has a derivative less than one in absolute value.

The Mean Value Theorem equates $f(x) - f(y)$ to $f'(z)\cdot (x-y)$, for some $z$ between $x$ and $y$. Hence if $|f'(z)|\lt 1$, we will have $|f(x)-f(y)|\lt |x-y|$. In higher dimensions (functions of more than one argument) a similar contraction analysis involving the Jacobian (rather than the ordinary derivative) can be applied.

hardmath
  • 37,715
  • 1
    I marked it as the correct answer because I was stupid not to read a few more pages (I was really just skimming, and then it occurred to me that I never really know when to add or just manipulate by moving); there is an explanation at the next page and more 3 pages after. Thank you for the help! – Trevor Jun 22 '20 at 15:24
  • Yes, digging into the details of Newton's method and its quadratic rate of convergence (more or less doubling the number of digits of accuracy with each step), I get a real appreciation for the basin of attraction in fixed-point iterations. – hardmath Jun 22 '20 at 16:32