To find a finite group with non-abelian Frattini subgroup, you'll need a group with a non-abelian maximal subgroup, but that does not suffice. The dihedral group of order 16 has a maximal subgroup isomorphic to the dihedral group of order 8, but the Frattini subgroup of a non-cyclic group of order 16 has order at most 4, so is abelian (it is Klein 4 in this case).
The smallest examples have order 64. One particular example is given by a matrix group:$$G=\left\langle \begin{bmatrix} 0 & 1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & 1 \end{bmatrix},
\begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \mod 4\right\rangle \leq \operatorname{GL}(2,\mathbb{Z}/4\mathbb{Z})$$ which is a semi-direct product of $C_4$ acting on $C_4 \times C_4$. Its Frattini subgroup is isomorphic to $C_2 \times D_8$. The only other possibility for a non-abelian Frattini subgroup of a group of order 64 is $C_2 \times Q_8$.
One reason books emphasize Frattini subgroups of $p$-groups is that they have a very nice definition there: $\Phi(G) = G^p [G,G]$. Hence calculations and theorems are much easier. For solvable groups, there is still some research into embedding properties related to the Frattini subgroup, and for non-solvable groups, there are significant questions left open.
I gave an answer you might be interested in, describing the groups that can occur as Frattini subgroups. In particular, some non-abelian groups are on the list. :-)