2

Using a second-degree multivariable Taylor polynomial for $x=(x_{1},...,x_{n})$ it follows that

$$f(x+\Delta x)\approx f(x)+\nabla f(x)\Delta x+\frac{1}{2}\Delta x^{T}H(x)\Delta x$$

with $\nabla f$ the gradient and $H(x)$ the Hessian of $f$.

What is the general form of the third degree polynomial?

My attempt would be something like $\frac{1}{6}\Delta x^{T}I(x)\Delta x^{2}$, such that (e.g., for $n=2$)

$$I(x)\Delta x^{2}=\begin{bmatrix}\frac{\partial^{3}f(x)}{\partial x_{1}^{3}}&3\frac{\partial f}{\partial x_{1}}\frac{\partial^{2}f}{\partial x_{2}^{2}}\\3\frac{\partial f}{\partial x_{2}}\frac{\partial^{2}f}{\partial x_{1}^{2}}&\frac{\partial^{3}f(x)}{\partial x_{2}^{3}}\end{bmatrix}\begin{bmatrix}\Delta x^{2}_{1}\\\Delta x^{2}_{2}\end{bmatrix}$$

and

$$\frac{1}{6}\Delta xI(x)\Delta x^{2}=\frac{1}{6}\frac{\partial^{3}f(x)}{\partial x_{1}^{3}}\Delta x_{1}^{3}+\frac{1}{6}\frac{\partial^{3}f(x)}{\partial x_{2}^{3}}\Delta x_{2}^{3}+\frac{1}{2}\frac{\partial f}{\partial x_{1}}\frac{\partial^{2}f}{\partial x_{2}^{2}}\Delta x_{1}\Delta x_{2}^{2}+\frac{1}{2}\frac{\partial f}{\partial x_{2}}\frac{\partial^{1}f}{\partial x_{1}^{2}}\Delta x_{2}\Delta x_{1}^{2}$$

; however, I know this to be wrong unfortunately.

1 Answers1

2

Higher order terms of the taylor polynomial cannot be captured by vectors ($\nabla f(x)$) or matrices ($Hf(x)$), because one needs to deal with general multilinear maps. See this answer for the fully general case. If you follow the notation there, then the third order term in the expansion of $f(a+h)$ about the point $a$ is (say assuming $V=\Bbb{R}^n,W=\Bbb{R}$) described by a trilinear-map $(D^3f)_a: \Bbb{R}^n\times\Bbb{R}^n\times\Bbb{R}^n\to\Bbb{R}$, \begin{align} \frac{1}{3!}(D^3f)_a[h,h,h]&=\frac{1}{6}(D^3f)_a\left[\sum_{i=1}^nh_ie_i,\sum_{j=1}^nh_je_j,\sum_{k=1}^nh_ke_k\right]\\ &=\frac{1}{6}\sum_{i,j,k=1}^n\frac{\partial^3f}{\partial x_i\partial x_j\partial x_k}(a) h_ih_jh_k \end{align} Note that not all these terms are "independent" because higher order partial derivatives for a sufficiently differentiable function are symmetric, so there's other ways of "regrouping" this sum (eg which is what you might find in Wikipedia).

Calvin Khor
  • 36,192
  • 6
  • 47
  • 102
peek-a-boo
  • 65,833
  • Thanks a lot! Maybe it is a different question, but in case of single variable Newton method, the error propagation for finding the root $f=0$ is defined by $\frac{(D^{2}f){a}}{(Df){a}}$, is it true that in this case $\frac{(D^{3}f){a}}{(D^{2}f){a}}$ doe the same for this multi variable setting of the root $(Df)_{a}=0$? –  Sep 02 '21 at 13:28
  • 1
    @user297530 If you're looking at the $k^{th}$ order Taylor polynomial, and if we assume $D^{k+1}f$ is bounded in a neighborhood of $a$, then we can bound the error using $D^{k+1}f$ (this is called the Lagrange form of the remainder). – peek-a-boo Sep 02 '21 at 13:33
  • 1
    oh I see you're talking about Newton's method for approximating roots... but I'm not sure what this directly has to do with Taylor's theorem. In the higher dimensional case, you can't divide by multilinear maps (we can only divide things by numbers). Anyway, you should probably ask a separate question for that. – peek-a-boo Sep 02 '21 at 13:35
  • From my understanding, Newton's method is a direct result of Taylor's theorem; however, in multi variable setting the error propagation becomes difficult to understand. I have uploaded this in a new question. –  Sep 02 '21 at 14:41