Can someone help me understand using the Jacobian matrix with Newton's Method for finding zeros?

Question

I'm struggling to understand approximating solutions to non linear equations using a Jacobian matrix. I understand intermediate steps, but I'm unsure how everything comes together.

I want to use Newton's method, but by using the Jacobian matrix with forward differences.

For example, I have these two equations that I want to find solutions for: $$ 3y - 2x^3 + 2x^2 - 8x = 0 $$ $$ y^3 - 5x -3 = 0 $$ I'm calling those $F(X)$.

Here is the Jacobian that I think is correct:

$$F^1(x)= \begin{bmatrix}-6x^2+4x-8 & 3\\-5 & 3y^2 \end{bmatrix} $$

I'm picking a point to start at, $$[x, y] = [1, 1]$$

I'm stuck on getting the next iteration of Newton's method, here's what I'm trying:

$$x+h = x -\frac{f(x)}{f^1(x)}$$

I understand the forward difference approximation as: $$f^1(x) = \frac{f(x+h) - f(x)}{h}$$

I'm getting $h$ by, starting with $x = [1,1]$: $$F^1(x)h = -F(x)$$

I'm pretty confused by forward difference, and exactly what it's used for. I'm unsure how it plays a role in Newton's Method.

If someone could provide some guidance I'd appreciate it, thanks!

Answer 1

Let me start considering two equations for two unknown varibles. We need to solve $$f(x,y)=0 \qquad , \qquad g(x,y)=0$$ So, a round a point $(x_0,y_0)$, we have $$f(x,y)=f(x_0,y_0)+f'_x(x_0,y_0)(x-x_0)+f'_y(x_0,y_0)(y-y_0)=0$$ $$g(x,y)=g(x_0,y_0)+g'_x(x_0,y_0)(x-x_0)+g'_y(x_0,y_0)(y-y_0)=0$$ Solving for $(x-x_0)$ and $(y-y_0)$ does not make any problem as soon as the derivatives are known.

In your case, you just need to eveluate the derivatives using forward differences that is to say $$f'_x(x,y)=\frac {f(x+\Delta x,y)-f(x,y)}{\Delta x}$$ $$f'_y(x,y)=\frac {f(x,y+\Delta y)-f(x,y)}{\Delta y}$$ instead of the analytical formulae.

Applied to your case, using $x_0=y_0=1$ and $\Delta x=\Delta y=\frac 1 {100}$, you would get $f_x'(1,1)=-10.0402$ instead of $-10$ and $g_y'(1,1)=3.0301$ instead of $3$, the other derivatives being exact.