It turns out that
$$ I(\omega) = \frac{\omega^2\log(1/\omega)}{2} + \mathcal{O}(\omega^2) $$
as $\omega \to 0^+$. The logarithmic correction arises from the asymptotic behavior
$$ \text{[integrand]} \sim \frac{\omega^2}{\sqrt{x^2 + \omega^2} + x} \qquad \text{as } x \to 0^+. $$
Below is a more rigorous argument:
1. Denote the integrand by $f(x \mid \omega, \delta)$:
$$ f(x \mid \omega, \delta) = (1 + x)^2 \biggl( \sqrt{x^2 + \frac{\omega^2}{(1+x)^{\delta}}} - x \biggr) $$
In order to investigate the asymptotic behavior of $I(\omega)$, we first manipulate the integrand so it is easily analyzed.
\begin{align*}
f(x \mid \omega, \delta)
&= (1 + x)^2 \frac{\left( x^2 + \frac{\omega^2}{(1+x)^{\delta}} \right) - x^2}{\sqrt{x^2 + \frac{\omega^2}{(1+x)^{\delta}}} + x} \\
&= \omega^2 \frac{1}{(1+x)^{\delta-2}} \frac{1}{\sqrt{x^2 + \frac{\omega^2}{(1+x)^{\delta}}} + x}
\end{align*}
This shows that $f(x \mid \omega, \delta) \sim \frac{\omega^2}{(1+x)^{\delta-2}(2x)}$ as $\omega \to 0^+$, suggesting that the primary source of the divergence of $I''(0)$ is the behavior of $f(x \mid \omega, \delta)$ near $x = 0$ as $\omega \to 0^+$. So, we will investigate the integral of $f(x \mid \omega, \delta)$ near $x = 0$ separately from the rest.
2. Define the function $J(\omega)$ by
$$ J(\omega) = \int_{0}^{1} \frac{1}{\sqrt{x^2 + \omega^2} + x} \, \mathrm{d}x. $$
Although $J(\omega)$ admits a closed form in terms of inverse trigonometric functions and hence a detailed asymptotic expansion as $\omega \to 0^+$, let me demonstrate a more elementary approach. Indeed, let us assume $\omega \in (0, 1)$ without losing the generality. Using the inequality
$$ \max\{x, \omega\} \leq \sqrt{x^2 + \omega^2} \leq x + \omega,$$
$J(\omega)$ is bounded as
$$ \int_{0}^{1} \frac{1}{2x+\omega} \, \mathrm{d}x
\leq J(\omega)
\leq \int_{0}^{1} \frac{1}{x + \max\{x, \omega\}} \, \mathrm{d}x. $$
By evaluating both integrals, we easily find that
$$ J(\omega) = \frac{\log(1/\omega)}{2} + \mathcal{O}(1). $$
3. Finally, decompose $I(\omega)$ into the sum of three integrals $I_1(\omega)$, $I_2(\omega)$, $I_3(\omega)$, where
\begin{align*}
I_1(\omega)
&= \int_{0}^{1} \frac{\omega^2}{\sqrt{x^2 + \frac{\omega^2}{(1+x)^{\delta}}} + x} \, \mathrm{d}x, \\
I_2(\omega)
&= \int_{0}^{1} \Biggl[ f(x \mid \omega, \delta) - \frac{\omega^2}{\sqrt{x^2 + \frac{\omega^2}{(1+x)^{\delta}}} + x} \Biggr] \, \mathrm{d}x, \\
I_3(\omega)
&= \int_{1}^{\infty} f(x \mid \omega, \delta) \, \mathrm{d}x
\end{align*}
As $\omega \to 0^+$, it is not hard to check that the integrand of each of $I_2(\omega)$ and $I_3(\omega)$ is bounded by $\omega^2$ times an integrable function. Hence, $I_2(\omega) + I_3(\omega) = \mathcal{O}(\omega^2)$. On the other hand,
$$ \omega^2 J(\omega) \leq I_1(\omega) \leq \omega^2 J(\omega\cdot2^{-\delta/2}) $$
shows that
$$ I_1(\omega) = \frac{\omega^2\log(1/\omega)}{2} + \mathcal{O}(\omega^2). $$
4. Combining all the observations together, we conclude
$$ \bbox[padding:5px;border:1px dotted navy;color:navy]{I(\omega) = \frac{\omega^2\log(1/\omega)}{2} + \mathcal{O}(\omega^2)} $$