0

We can derive a closed-form expression for the $n$th derivative of the sigmoid function $\sigma(x):=\frac{1}{1+\exp(-x)}$: $$\sigma^{(n)}(x)=\sum_{k=0}^n\sum_{j=1}^k(-1)^j(j+1)^n\begin{pmatrix}k\\j\\\end{pmatrix}\sigma^{k+1}(x).$$ Derivation here.

I'm curious if it's possible to find closed form expression for the optima of $\sigma^{(n)}$, which is equivalent to finding the max/min of the function evaluated at the roots of $\sigma^{(n+1)}$. I know in general it is not possible to do this for even polynomials past $n=4$, but looking at the figures below the derivation, there is a clear pattern to the structure of the derivatives.

If this is not possible, is it possible to bound them? Best I can do is $|\sigma^{(n)}(x)|\leq\sum_k |c_k|$, where $c_k$ are the coefficients of $\sigma^k(x)$ in $\sigma^{(n)}(x)$ which we can find a closed form expression of. But this isn't good enough. I'd like to find a bound such that $\sigma^{(n)}(x)=O(n!)$ so I can find a useful Lagrange error bound.

Leland Stirner
  • 755
  • 1
  • 6
  • 16

1 Answers1

1

for $n>0$:

$$\frac{d^n}{dx^n}\frac{1}{1+e^{-x}}=\sum_{k=1}^n(-1)^{k+1}\left(\sum_{j=0}^{n-k}(-1)^j\binom{n-k+1}{j}(n-k+1-j)^n\right)\frac{(e^{-x})^{n-k+1}}{(1+e^{-x})^{n-k+2}}$$

$$=\sum_{k=1}^n(-1)^{k+1}(n-k+1)!\ \mathrm{Stirling2}(n,n-k+1)\frac{(e^{-x})^{n-k+1}}{(1+e^{-x})^{n-k+2}}$$

We get this by applying General Leibniz rule (Higher Product rule) to the multiplicative inverse of $1+e^{-x}$.

IV_
  • 7,902