Parameter Second Derivative of Solution to Coupled ODEs

Question

Problem Statement

I am treating a system of ordinary differential equations (ODE) that can be written as $$ \frac{\mathrm d\vec{x}(t)}{\mathrm dt} = A \, \vec{\theta}(\vec{x}(t), u(t), t), $$ with initial conditions $\vec{x}(t=0) = \vec{0}$. Here, $\vec{x}(t) \in \mathbb{R}^N$ and $u(t) \in \mathbb{R}$. Additionally, $A \in \mathbb{R}^{N \times J}$ is constant in time and $\vec{\theta}(\vec{x}(t), u(t), t) \in \mathbb{R}^{J}$ is often called a “feature mapping” where a set of scalar-valued candidate functions $\phi_j$ are used to define $$ \vec{\theta}(\vec{x}(t), u(t), t) = \begin{bmatrix} \phi_1(\vec{x}(t), u(t), t) & \cdots &\phi_J(\vec{x}(t), u(t), t) \end{bmatrix}^T. $$ Based on a previous question, define $A_{n,j}$ to be the entry in the $i$-th row and $j$-th column of $A$. Then, also define some sensitivity vector $\vec{s}(t)\equiv \frac{\partial \vec{x}(t)}{\partial A_{n,j}}$. To find $\vec{s}(t)$, solve $$ \frac{\mathrm d\vec{s}(t)}{\mathrm dt}=\vec{e}_n\vec{\theta}(\vec{x}(t), u(t), t)_j + A J^{(1)} \vec{s}(t) $$ with initial condition $\vec{s}(t=0) = \vec{0}$. In this equation, $\vec{e}_n \in \mathbb{R}^N$ is a zero vector with a single 1 in the $n$-th entry. Also, $$ J^{(1)} \equiv \begin{bmatrix} \dfrac{\partial \phi_1}{\partial x_1} & \cdots & \dfrac{\partial \phi_1}{\partial x_N} \\ \vdots & \ddots & \vdots \\ \dfrac{\partial \phi_J}{\partial x_1} & \cdots & \dfrac{\partial \phi_J}{\partial x_N} \end{bmatrix}. $$

Question

What is the governing equation for $\vec{a}(t)$ where $\vec{a}(t) \equiv \begin{bmatrix} \dfrac{\partial^2 x(t)_1}{\partial A_{n,j}^2} & \dfrac{\partial^2 x(t)_2}{\partial A_{n,j}^2} & \cdots & \dfrac{\partial^2 x(t)_N}{\partial A_{n,j}^2} \end{bmatrix}$?

Solution Attempt

Summary

By following the procedure described in the previous question applied to the first-order sensitivity equation, I found $$ \frac{\mathrm d \vec{a}(t)}{\mathrm dt} = 2 \vec{e}_n \left(J^{(1)} \vec{s}(t)\right)_j + A\left(J^{(2)} \left(\vec{s}(t) \odot \vec{s}(t) \right) + J^{(1)} \vec{a}(t) \right) $$ with $\vec{a}(t=0)=0$. In this expression, $$ J^{(2)} \equiv \begin{bmatrix} \dfrac{\partial^2 \phi_1}{\partial x_1^2} & \cdots & \dfrac{\partial^2 \phi_1}{\partial x_N^2} \\ \vdots & \ddots & \vdots \\ \dfrac{\partial^2 \phi_J}{\partial x_1^2} & \cdots & \dfrac{\partial^2 \phi_J}{\partial x_N^2} \end{bmatrix}. $$ and $\odot$ is used to indicate the element-wise product. After testing this numerically, I think this is an incorrect equation. However, I am not sure where the error lies.

Details on Derivation

Apply the derivative w.r.t. $A_{n,j}$ to first order sensitivity equation $$ \frac{\mathrm d\vec{a}(t)}{\mathrm dt}=\frac{\partial }{\partial A_{n,j}} \left[ \vec{e}_n\vec{\theta}(\vec{x}(t), u(t), t)_j\right] + \frac{\partial }{\partial A_{n,j}} \left[ A J^{(1)} \vec{s}(t)\right] $$ Treat the first term: $$ \frac{\partial }{\partial A_{n,j}} \left[ \vec{e}_n\vec{\theta}(\vec{x}(t), u(t), t)_j\right] = \vec{e}_n \left(J^{(1)} \vec{s}(t)\right)_j. $$ Then, the second term: $$ \frac{\partial }{\partial A_{n,j}} \left[ A J^{(1)} \vec{s}(t)\right] = \vec{e}_n \left(J^{(1)} \vec{s}(t)\right)_j + A\frac{\partial }{\partial A_{n,j}} \left[ J^{(1)} \vec{s}(t)\right]. $$ By writing out the matrices and performing the operations, I was able to find $$ \frac{\partial }{\partial A_{n,j}} \left[ J^{(1)} \vec{s}(t)\right] = A\left(J^{(2)} \left(\vec{s}(t) \odot \vec{s}(t) \right) + J^{(1)} \vec{a}(t) \right). $$ Combine these to obtain the expression given in the previous subsection.

Answer 1

Hint.

Let us consider the dynamical system

$$ \dot x = \Theta(x,a) = a\theta(x) $$

with $a$ as a multiplicative parameter. Now

$$ \frac{\partial \dot x}{\partial a} = \frac{\partial \Theta}{\partial x}\frac{\partial x}{\partial a}+\frac{\partial \Theta}{\partial a} $$

or calling $\mu = \frac{\partial x}{\partial a}$

$$ \dot \mu = \frac{\partial \Theta}{\partial x}\mu+\frac{\partial \Theta}{\partial a} $$

Now

$$ \frac{\partial^2 \dot x}{\partial a^2} = \frac{\partial}{\partial a}\left(\frac{\partial \Theta}{\partial x}\mu+\frac{\partial \Theta}{\partial a}\right) = \frac{\partial^2\Theta}{\partial x\partial a}\mu + \frac{\partial \Theta}{\partial x}\frac{\partial\mu}{\partial a} $$

or calling $\xi = \frac{\partial\mu}{\partial a}$, $\dot\xi = \frac{\partial \Theta}{\partial x}\xi+\frac{\partial^2\Theta}{\partial x\partial a}\mu$ and the complete system

$$ \cases{ \dot x = a\theta(x)\\ \dot \mu = a\frac{\partial \theta}{\partial x}\mu+\theta(x)\\ \dot\xi = a\frac{\partial \theta}{\partial x}\xi+\frac{\partial\theta}{\partial x}\mu } $$

EDIT

Using the Einstein notation

$$ \dot X = F(X,A)=A\Theta=\sum_{i,j}a_{i,j}\theta_j(X) $$

then

$$ \dot{x_i} = a_{i,j}\theta_j(X) $$

and

$$ \frac{\partial \dot x_i}{\partial a_{k,l}}=\frac{\partial F_i(X,A)}{\partial x_m}\frac{\partial x_m}{\partial a_{k,l}}+\frac{\partial F_i(X,A)}{\partial a_{k,l}}=\frac{a_{i,j}\partial \theta_i(X)}{\partial x_m}\frac{\partial x_m}{\partial a_{k,l}}+\theta_l(X) $$

now calling

$$ \frac{\partial \dot x_i}{\partial a_{k,l}}=\mu^i_{k,l} $$

we have

$$ \dot{\mu}^i_{k,l} = \frac{a_{i,j}\partial \theta_i(X)}{\partial x_m}\mu^m_{k,l}+\theta_l(X) $$

etc.