That's a great question!
Usually counterexamples are already easy to find for $\mathbb{Z}$-modules (also known as abelian groups), but sadly, in this case, if you look only at finitely generated abelian groups, then any short exact sequence
$$0 \to A' \to A \to A'' \to 0$$
with $A \cong A'\oplus A''$ is actually split. (It is possible to see it by revising the classification of finitely generated abelian groups and calculations of the Yoneda $\operatorname{Ext}^1_\mathbb{Z} (A'',A')$.)
So we need to look for counterexamples among abelian groups that are not finitely generated. Maybe the easiest is the following. Take an infinite direct sum of copies of $\mathbb{Z}/n\mathbb{Z}$ and consider the map
\begin{align*}
p\colon \mathbb{Z} \oplus \bigoplus_{i \ge 0} \mathbb{Z}/n\mathbb{Z} & \twoheadrightarrow \bigoplus_{i \ge 0} \mathbb{Z}/n\mathbb{Z},\\
(x,y_0,y_1,y_2,\ldots) & \mapsto (x \mod{n}, y_0, y_1, y_2, \ldots).
\end{align*}
This is not the projection, but this is a surjective homomorphism, and we have a legitimate short exact sequence
$$0 \to \mathbb{Z} \xrightarrow{x \mapsto (n x, ~ 0,0,0,\ldots)} \mathbb{Z} \oplus \bigoplus_{i \ge 0} \mathbb{Z}/n\mathbb{Z} \xrightarrow{p} \bigoplus_{i \ge 0} \mathbb{Z}/n\mathbb{Z} \to 0$$
But you can see that $p$ doesn't have a section.