For abelian groups (or even in any abelian category) you can use the splitting lemma:
Proposition 1. Let $0 \longrightarrow A' \stackrel{f}\longrightarrow A \stackrel{g}\longrightarrow A'' \longrightarrow 0$ be a short exact sequence of abelian groups. Then the following are equivalent:
(i) There exists a homomorphism $h \colon A \rightarrow A'$ such that $h \circ f = \text{id}_{A'}.$
(ii) There exists a homomorphism $j \colon A'' \rightarrow A$ such that $g \circ j = \text{id}_{A''}.$
(iii) We have an isomorphism $h \colon A \rightarrow A' \oplus A''$, such that $h \circ f$ is the natural inclusion of $A'$ into the direct sum $A' \oplus A''$ and $g \circ h^{-1}$ is the natural projection from $A' \oplus A''$ onto $A''$.
For non-abelian groups the splitting lemma does not hold in general. Consider for example the short exact sequence $1 \longrightarrow A_n \stackrel{\iota}\longrightarrow S_n \stackrel{\text{sgn}}\longrightarrow C_2 \longrightarrow 1$. We can send the generator of $C_2$ to any $2$-cycle to get (ii), but (i) and (iii) do not hold.
Let me now rephrase the splitting lemma for general groups:
Proposition 2. Let $1 \longrightarrow G' \stackrel{f}\longrightarrow G \stackrel{g}\longrightarrow G'' \longrightarrow 1$ be a short exact sequence of groups. Then the following are equivalent:
(i) There exists a homomorphism $h \colon G \rightarrow G'$ such that $h \circ f = \text{id}_{G'}.$
(ii) There exists an isomorphism $\alpha \colon G \rightarrow G' \times G''$ such that $\alpha \circ f = i_1$ and $p_2 \circ \alpha = g$. Here $i_1 \colon G' \rightarrow G' \times G''$ is the inclusion into the first factor and $p_2 \colon G' \times G'' \rightarrow G''$ is the projection onto the second factor.
You can also get another version:
Proposition 3. Let $1 \longrightarrow G' \stackrel{f}\longrightarrow G \stackrel{g}\longrightarrow G'' \longrightarrow 1$ be a short exact sequence of groups. Then the following are equivalent:
(i) There exists a homomorphism $j \colon G'' \rightarrow G$ such that $g \circ j = \text{id}_{G''}.$
(ii) There exists a homomorphism $\varphi \colon G'' \rightarrow \text{Aut}(G')$, such that $\beta \colon G' \rtimes G'' \rightarrow G$, $(a,b) \mapsto f(a)j(b)$ is an isomorhpism.