Looking more closely at the case $k=4$, a computer program allowed me to conjecture that we had $$4=(x-1)^3+(y+5)^3-x^3-y^3 \;\;(1)$$ for an infinite number of pairs of integers $(x,y)$. The previous equality is equivalent to $$(2x-1)^2-5(2y+5)^2=36.$$ Now it is easy to prove that the Pell's equation $X^2-5Y^2=36$ has an infinite number of solutions $(X,Y)\in\mathbb{Z}^2$ with $X$ and $Y$ odd, it follows that the equation (1) has infinitely many solutions with $x$ and $y$ integers.
Similarly, we can show that we can write $$5=(x+1)^3+(y-2)^3-x^3-y^3,$$ for an infinite number of pairs of integers $(x,y)$. Indeed, the previous equality is equivalent to $$(2x+1)^2-8(y-1)^2=9,$$ and it's easy to prove that the Pell's equation $X^2-8Y^2=9$ has an infinite number of solutions $(X,Y)\in\mathbb{Z}^2$ with $X$ odd.
Without being certain, it seems to me that we can perhaps generalize the preceding reasoning, so as to prove that many integers of the form $k=a^3+b^3+c^3+d^3$ can be decomposed in an infinite number of ways into a sum of four cubes.
Indeed let $u=a+b$ and $v=-(c+d)$ and assume that $uv>0$ and that $uv$ is not a perfect square. Note that $k-u^3+v^3\equiv0\;(\text{mod }3)$.
The equation $$(u-x)^3-(v+y)^3+x^3+y^3=k$$ has the particular solution $(x,y)=(a,c)$, and this is equivalent to $$(2ux-u^2)^2-uv(2y+v)^2=\frac{u}{3}(4k-u^3+v^3),$$
which has the form of a Pell's equation. I haven't looked at the details, but it should be possible to prove that it has infinitely many solutions.
N.B. There may be an interesting identity (see $(\star)$ below) in connection with what I've just said.
Let $a$, $b$, $c$ and $d$ be four integers such that $(a+b)(c+d)<0$ and $-(a+b)(c+d)$ is not a perfect square.
We know that there are then two integers $x$ and $y$ such that
$$(2x+1)^2+4(a+b)(c+d)y^2=1.$$
Let
$$\left\{\begin{array}{l}
A=(a-b)x+(d^2-c^2)y+a\\
B=(b-a)x+(c^2-d^2)y+b\\
C=(c-d)x+(a^2-b^2)y+c\\
D=(d-c)x+(b^2-a^2)y+d\\
\end{array}\right.$$
then $A+B=a+b$, $C+D=c+d$, and
$$A^3+B^3+C^3+D^3=a^3+b^3+c^3+d^3\;\;(\star)$$
Here's an example:
We have $a^3+b^3+c^3+d^3=4$ with $a=-5$, $b=1$, $c=4$ and $d=4$,
$(a+b)(c+d)=-32<0$ and $32$ is not a perfect square,
we have $(2x+1)^2+4(a+b)(c+d)y^2=1$ with $x=288$ and $y=51$,
then $A^3+B^3+C^3+D^3=4$ with $A=-1733$, $B=1729$, $C=1228$ and $D=-1220$.