To solve a system using Newton Raphson method, we use the Jacobian. In this question Solving a set of equations with Newton-Raphson, without mentioning the invertibility of the Jacobian matrix, they have used the method.
So, to use Newton Raphson method, should the Jacobian of the system be invertible over the domain? Do we have to explicitly check the invertibility of the Jacobian?
The convergence of Newton’s method implies the existence of a root, but the function and the initial guess must satisfy some conditions to guarantee local convergence of Newton’s method. Local convergence of Newton's method (which implies the existence of a root) is guaranteed by the Kantorovich Theorem. The assumptions are a bit technical, but they are standard in the analysis of Newton's method. The form of the assumptions and precise statement of the theorem vary between sources and here I state something similar to that found in Kelley, 1995.
Assumption 1
There exist constants $\beta$, $\eta$, $\bar{r}$, and $\gamma$ with $\beta\eta\gamma \leq \frac{1}{2}$ and $x_0\in\mathbb{R}^n$ such that
- $F$ is differentiable at $x_0$, and $$ \|F'(x_0)^{-1}\| \leq \beta, \ \text{and } \|F'(x_0)^{-1}F(x_0)\|\leq \eta. $$
- $F'$ is Lipschitz continuous with Lipschitz constant $\gamma$ in a ball of radius $\bar{r} \geq r_-$ about $x_0$ where $$ r_- = \frac{1 - \sqrt{1 - 2\beta\eta\gamma}}{\beta\gamma}. $$
In English, these assumptions state, roughly:
The root of the linear approximation to $F$ ant $x_0$ is unique and the root of the linear approximation at $x_0$ is not too far away from $x_0$.
The higher derivatives of $F$ are sufficiently well-behaved in a neighborhood about $x_0$ so that the first point remains true when we do the same procedure at the root of the linear approximation.
Indeed, it is remarkable that the existence of a root and precise error estimates of Newton’s method and various modifications can be guaranteed by simple estimates on $F$ and its first few derivatives.
The statement of the theorem is then
Theorem (Kantorovich)
Let Assumption 1 hold and let $\beta\eta\gamma \leq \frac{1}{2}$ and define $\mathcal{B}_0 = \{x\in\mathbb{R}^n:\|x-x_0\|\leq r_-\}$. Then there is a unique root $x^*$ of $F$ in $\mathcal{B}_0$, the Newton iteration with $x_0$ as the initial iterate converges to $x^*$, and the iterates $x_k \in \mathcal{B}_0$ for all $k$. Furthermore, $x^*$ is the unique root of $F$ in the ball of radius $$ r = \min\left\{\bar{r}, \frac{1+\sqrt{1 - 2\beta\eta\gamma}}{\beta\gamma}\right\} $$ about $x_0$ and the errors $e_k$ satisfy the estimate $$ \|e_k\| \leq \frac{(2\beta\eta\gamma)^{2^k}}{2^k \beta\gamma}. $$ That is, Newton's method converges quadratically.
In practice, you can rarely guarantee a priori that these assumptions are met, but they work to provide a quantitative measure of what "local" means for the local convergence of Newton's method and how this locality is influenced by, e.g., near-singularity of $F'$ or large second derivatives. The most practical ways of maximizing your chances of converging are to reduce the conditioning of your problem via rescaling or something or to simply have better initial guesses. Obviously "just guess closer to the correct answer" is somewhat circular advice, but if the system of equations correspond to a problem for which there is some existing domain knowledge then this can be leveraged for better initialization. For example, initializing at parameter values from literature or values predicted by reduced models or models with missing physics can be surprisingly effective. There are some fields where this is frowned upon (CFD comes to mind) for biasing results, but it is effective if you are having problems even converging in the first place.
Kelley, C. T., Iterative methods for linear and nonlinear equations, Frontiers in Applied Mathematics. 16. Philadelphia, PA: SIAM, Society for Industrial and Applied Mathematics. xiii, 165 p. (1995). ZBL0832.65046.
