If $a=0$, the inequality
$$
\left||a+b|^p - |a|^p \right| \leq \epsilon |a|^p + C_{\epsilon} |b|^p\tag1
$$
is true with $C_\epsilon=1$. Otherwise, divide by $|a|^p$ and set $x=\frac ba$ to get
$$
\left||1+x|^p-1\right|\le\epsilon+C_\epsilon|x|^p\tag2
$$
We will let $u=x+1$, so that $(2)$ becomes
$$
\left|1-|u|^p\right|\le\epsilon+C_\epsilon|1-u|^p\tag3
$$
Case: $\boldsymbol{|u|\le1}$
In this case, $(3)$ becomes
$$
1-|u|^p\le\epsilon+C_\epsilon|1-u|^p\tag4
$$
If $|u|\ge(1-\epsilon)^{1/p}$, then $(4)$ is trivially satisfied for any $C_\epsilon$; so suppose that $|u|\lt(1-\epsilon)^{1/p}$. Then the triangle inequality says
$$
\begin{align}
|1-u|^p
&\ge(1-|u|)^p\tag5
\end{align}
$$
Therefore,
$$
\begin{align}
\frac{1-|u|^p}{|1-u|^p}
&\le\frac{1-|u|^p}{|1-|u||^p}\tag6
\end{align}
$$
Thus, for $t=|u|\in\left[0,(1-\epsilon)^{1/p}\right]$
$$
\begin{align}
\frac{\mathrm{d}}{\mathrm{d}t}\frac{1-t^p}{(1-t)^p}
&=\frac{p\left(1-t^{p-1}\right)}{(1-t)^{p+1}}\tag{7a}\\
&\le0&\text{if }p\le1\tag{7b}\\[6pt]
&\ge0&\text{if }p\ge1\tag{7c}
\end{align}
$$
For $p\le1$, $\frac{1-t^p}{(1-t)^p}$ is decreasing, so its max is $1$ at $t=0$.
For $p\ge1$, $\frac{1-t^p}{(1-t)^p}$ is increasing, so its max is $\frac\epsilon{\left(1-(1-\epsilon)^{1/p}\right)^p}$ at $t=(1-\epsilon)^{1/p}$.
Therefore, we can use
$$
C_\epsilon=\left\{\begin{array}{cl}
1&\text{if }p\le1\\
\frac\epsilon{\left(1-(1-\epsilon)^{1/p}\right)^p}&\text{if }p\gt1
\end{array}\right.\tag8
$$
Case: $\boldsymbol{|u|\ge1}$
In this case, $(3)$ becomes
$$
|u|^p-1\le\epsilon+C_\epsilon|u-1|^p\tag9
$$
If $|u|\le(1+\epsilon)^{1/p}$, then $(9)$ is trivially satisfied for any $C_\epsilon$; so suppose that $|u|\gt(1+\epsilon)^{1/p}$. Then the triangle inequality says
$$
\begin{align}
|u-1|^p
&\ge(|u|-1)^p\tag{10}
\end{align}
$$
Therefore,
$$
\begin{align}
\frac{|u|^p-1}{|u-1|^p}
&\le\frac{|u|^p-1}{||u|-1|^p}\tag{11}
\end{align}
$$
Thus, for $t=|u|\ge(1+\epsilon)^{1/p}$
$$
\begin{align}
\frac{\mathrm{d}}{\mathrm{d}t}\frac{t^p-1}{(t-1)^p}
&=\frac{p\left(1-t^{p-1}\right)}{(t-1)^{p+1}}\tag{12a}\\
&\le0&\text{if }p\ge1\tag{12b}\\[6pt]
&\ge0&\text{if }p\le1\tag{12c}
\end{align}
$$
For $p\le1$, $\frac{t^p-1}{(t-1)^p}$ is increasing, so its sup is $1$ as $t\to\infty$.
For $p\ge1$, $\frac{t^p-1}{(t-1)^p}$ is decreasing, so its max is $\frac\epsilon{\left((1+\epsilon)^{1/p}-1\right)^p}$ at $t=(1+\epsilon)^{1/p}$.
Therefore, we can use
$$
\bbox[5px,border:2px solid #C0A000]{C_\epsilon=\left\{\begin{array}{cl}
1&\text{if }p\le1\\
\frac\epsilon{\left((1+\epsilon)^{1/p}-1\right)^p}&\text{if }p\gt1
\end{array}\right.}\tag{13}
$$
Bernoulli's Inequality says that for $p\gt1$,
$$
(1+\epsilon)^{1/p}-1\le1-(1-\epsilon)^{1/p}\tag{14}
$$
so for all $u$, we can use $(13)$ for $C_\epsilon$.