Confusion regarding order of error/truncation notation for iterative methods

Question

I am confused as to the meaning of the $O(h)$ notation used to denote error associated with an iterative algorithm (for example RKF45 has an local error of order $O(h^5)$)

The general template for deriving these solver methods seems to be something like this $f(x+h) = f(x) + g(x,h) + i(x, h^2) ...+z(x,h^{p-1}) + O(h^p)$ Then we say $O(h^p)$ (ie the highest power and beyond) are small enough to be neglected.

All this makes sense as long as $|h| < 1$ since $h^{p-1} > h^p$. So my question is, are we restricted to take step sizes $h<1$. What happens if we take $|h|>1$. Will and $O(h^4)$ solver perform better than $O(h^5)$ solver

EDIT:

I am asking this because for one of my problems, $O(h^5)$ took big steps (larger than 1 sec, still giving really low error) compared to $O(h^4)$.

The $O(h^5)$ method will outperform the $O(h^4)$ only for sufficiently small $h$... If you imagine the errors behaving like $ 100000 h^5$ and $ h^4$, you'll get what I mean. — PierreCarre, Nov 05 '19 at 20:01
@PierreCarre see what you mean. The constant associated with $O(h)$ would indeed make a difference. So, is it safe to say that M4 will be better/have less error compared to M5 given a large enough step-size h? I am asking this because for one of my problems, ode45 took big steps (larger than 1 sec, still giving really low error). I am wondering if an $O(h)$ method will perform better. — User 10482, Nov 05 '19 at 20:24
It is hard to draw very general conclusions, as the constant in front of $h^k$ can vary greatly from one particular equation to the other. I would say that in general it is not a good idea to use very high order methods, especially if you are uncertain on the regularity of the solution. RK4 is the highest order I personally use. — PierreCarre, Nov 05 '19 at 21:30
Similar questions: Truncation error with growing step size, Runge-Kutta methods: step size greater than 1, Euler's Method Error Term (Big O Notation) — Lutz Lehmann, Apr 01 '20 at 22:34
Note that ode45, using one of the Dormand-Prince 45 variants, is a 4th order method that uses a 5th order step to estimate the local error. — Lutz Lehmann, Apr 01 '20 at 22:37
@User10482, your algorithm uses iterations together with a step-size h, correct? — Davi Barreira, Apr 08 '20 at 18:38
It moves in time steps of h; computing result at each and moving on to the next.@DaviBarreira — User 10482, Apr 08 '20 at 18:49
Just so is clear, the final solution is reached only after "n" steps of size "h". And the error is comparing only the result in the final step. Is that correct? — Davi Barreira, Apr 08 '20 at 19:12
Could you please name the methods you use? You might have a wrong idea on what their order is and how they work internally, if you did not implement everything yourself. — Lutz Lehmann, Apr 09 '20 at 05:09

Lutz Lehmann · Answer 1 · 2020-04-09T11:26:02.023

First, your question has a trivial answer, there is no general step size boundary at $1$, if you have an ODE $y'(t)=f(y(t))$ and consider another function $u(s)=y(sT)$, then this new function satisfies $u'(s)=Ty'(sT)=Tf(u(s))$. The error of the numerical solution however is independent of the scaling of the time (if the integration interval is scaled likewise), by varying $T$ you can create instances of the problem that have numerically small and large step sizes for the same sequence of $y$ values. Think about a solar system simulation set up to use days/astronomical units and then rescale to have years/lightyears or seconds/meters, the time step will remain about the same for the same relative error tolerance, but the number that represents it will be drastically different.

Secondly, about truncation errors and expansions. The mentioned methods are designed for variable step sizes, having a marching method of order 4 and an error estimating embedded method of order 5 for RKF45 or DoPri45.

A one-step method for $y'=f(y)$ (autonomous for simplicity, not really required) can be summarized as $$ y_{n+1}=y_n+h_n\Phi_f(y_n,h_n) $$

The local truncation error has order $p$ if for an exact solution $y$ of the ODE one gets $$ y(x_n+h_n)=y(x_n)+h_n\Phi_f(y(x_n),h_n)+h_n^{p+1}R_p(y(x_n),h_n) $$ The expression $R_p$ is in first order a polynomial expression in $f$ and the derivatives of $f$ up to order $p$. If one considers some bounded region around the exact solution close to $y(x_n)$, this expression is bounded by some constant $M_p(y(x_n))$.

The adaptive step size controller now attempts to estimate the truncation error, for instance using the embedded error estimator, and regulate the step size so that $M_p(y(x_n))h_n^p$ or $M_p(y_n)h_n^p$ is about constant, at a level given by the tolerances, say $C\varepsilon$, where $ε$ is the desired global error.

Then the local errors sum up to the global error $$ e(x_n)=Cε\sum_{k=0}^{n-1} h_ke^{L(x_n-x_k)}\approx\frac{e^{L(x_n-x_0)}-1}{L}Cε $$ where $L$ is a Lipschitz constant of $f$. The factor $e^{L(x_n-k_k)}$ is an upper bound for the magnification of the local truncation error as part of the global error at $x=x_n$, this is similar to the Grönwall lemma in the case of a linear differential inequality.

In all that means that the step sizes used mainly depend on the size of $f$ and its derivatives, and then also on the method as that determines the concrete shape and coefficients inside $R_p$. For different methods of different orders there can be some variability in the magnitudes which can make their step size selections at the larger end of the range of sensible error tolerances rather unrelated.

Confusion regarding order of error/truncation notation for iterative methods

1 Answers1