This statement is false. For a counter example, consider the group of unipotent upper triangular matrices with coefficients in $\mathbf{F}_3$ (under matrix multiplication):
$$U(3, \mathbf{F}_3) = \left\{\begin{pmatrix} 1 & a & b \\ 0 & 1 & c \\ 0 & 0 & 1\end{pmatrix} : a, b, c \in \mathbf{F}_3\right\}.$$
Some computational details:
Write $[a, b, c]$ for the element $$\begin{pmatrix} 1 & a & b \\ 0 & 1 & c \\ 0 & 0 & 1\end{pmatrix} $$
of the group $U(3, \mathbf{F}_3)$. Then, the group law can be written as:
$$[a_1, b_1, c_1] \cdot [a_2, b_2, c_2] = [a_1 + a_2, b_1 + a_1 c_2 + b_2, c_1 + c_2].$$
From here, you may quickly verify that every element has order $3$ in $U(3, \mathbf{F}_3)$ and that it is not abelian. Thus, this group has exponent $3$ but is not abelian.