0

I would like to preface my question by confessing that I come from a Physics background, so I apologize for any abuse of notation.

Given a 1st order ODE

$$ y' = f(x, y) $$

we can use Euler's Method to find an approximate solution:

$$ y_{n+1} = y_n + f(x_n, y_n) \,dx $$

Assuming both $f$ and $y$ are both analytic, if locally

$$ y = y_0 +y'_0 dx + \frac12y''_0(dx)^2 + \cdots +\frac{1}{n!}y^{(n)}_0(dx)^n + \cdots$$

and since

$$ y'' = \frac{d}{dx}f(x, y) = \frac{\partial f}{\partial y}y' + \frac{\partial f}{\partial x} $$

could one better approximate a solution to the ODE by adding a second order correction to Euler's Method of the form:

$$ y_{n+1} = y_n +y'_n * dx + \frac12y''_n(dx)^2 = y_n +f(x_n,y_n)dx + \frac12[\frac{\partial f}{\partial y}(x_n,y_n)*f(x_n,y_n) + \frac{\partial f}{\partial x}(x_n,y_n)](dx)^2 $$

and generally, could Euler's method provide even better approximations if correction terms of higher order are added (assuming we can easily determine the partial derivatives of $f$)?

Rodrigo de Azevedo
  • 23,223
Andrew Murphy
  • 265
  • 1
    I suggest looking at https://en.wikipedia.org/wiki/Heun%27s_method – irchans Jun 30 '20 at 15:04
  • Hello irchans, from the page that you linked, both Heun's Method and the Runge-Kutta method do not explicitly include the partial derivatives of $ f $ , as my correction term does. – Andrew Murphy Jun 30 '20 at 15:15
  • Yes, I like how you incorporated that. I saw something related at work a few months ago. Nice idea. – irchans Jun 30 '20 at 22:30

2 Answers2

3

Yes, one can do that, this is called the Taylor series method. Most efficiently you do this with methods of algorithmic differentiation, that is, you capture the right side as computational graph, equip each node with space for the Taylor series coefficients of that node, and populate these coefficients with increasing degree. Straight evaluation gives $y'$, augmenting the input by the linear term allows to augment all nodes, giving $y''$ in the end, then augment the input to a quadratic polynomial etc.

One rather old package that does this efficiently is the TADIFF part of the FADBAD, now both FADBAD++ package with applications to certified error bounds of ODE solutions

Local examples for the Taylor series method are https://math.stackexchange.com/a/2953714/115115 and https://math.stackexchange.com/a/3663073/115115, while https://math.stackexchange.com/a/2499684/115115 explores the Taylor arithmetic aspect of a simple example.

More frequently you will encounter the first derivative of the ODE function, as part of a Newton-like method in implicit methods, in determining the split in splitting methods, which leads to Rosenbrock and similar methods. The latter can be seen as an amalgamation of the Taylor series and Runge-Kutta approaches.

Lutz Lehmann
  • 131,652
1

If we have information on higher order derivatives, then it is certainly possible to get higher order approximations as you have shown. However, this is usually not the case in practice and so methods like Euler's are developed in the spirit of approximating $y$ with as little knowledge as possible.

I think that this post is also worth looking at, where someone describes how Richardson's extrapolation could be used to achieve a more accurate version of Euler's method while still only requiring knowledge of $y'$.

Will
  • 91