To add to the great and simple counterexample of user1551 let me add the following related result:
Theorem. Given $A\in\mathbb C^{n\times n}$ the following statements are equivalent:
- There exist $t_1,t_2\in\mathbb R$ rationally independent such that $e^{iAt_1},e^{iAt_2}$ are both unitary
- $A$ is Hermitian
So while $e^{iA}$ can be unitary without $A$ being Hermitian (as seen above), if we knew additionally that, say, $e^{iA\sqrt2}$ is unitary, then we can in fact conclude that $A$ is Hermitian. In a way this is the weakest possible formulation of a converse to "$A$ Hermitian $\Rightarrow$ $e^{iA}$ unitary". One can probably prove the above result in an elegant manner using representation theory but I'll instead opt for a more basic/elementary proof at the core of which sits the following lemma:
Lemma. Let $D,X\in\mathbb C^{n\times n}$ and $t_1,t_2\in\mathbb R$ be given such that $D$ is Hermitian and $t_1,t_2$ are rationally independent. If $[X,e^{iDt_1}]=[X,e^{it_2D}]=0$, then $[X,D]=0$.
Proof. Because $D$ is Hermitian we can apply the spectral decomposition to obtain $D=\sum_{j=1}^m d_jP_j$ for some $m\in\mathbb N$, $d_1,\ldots,d_m\in\mathbb R$ pairwise distinct, and some spectral projections $P_1,\ldots,P_m$ (i.e. $P_j^*=P_j$, $P_jP_k=\delta_{jk}P_k$, and $\sum_{j=1}^m P_j={\bf1}$). Indeed if we can show that $P_jXP_k=0$ for all $j\neq k$, then we are done due to
\begin{align*}
[X,D]&=\Big(\sum_{j=1}^mP_j\Big)[X,D]\Big(\sum_{k=1}^mP_k\Big)\\
&=\sum_{j,k=1}^mP_jXDP_k-P_jDXP_k\\
&=\sum_{j,k=1}^m(d_k-d_j)\underbrace{P_jXP_k}_{\Rightarrow j=k}=\sum_{j=1}^m(d_j-d_j)P_jXP_j=0\,.
\end{align*}
Now for any $j\neq k$ we know by assumption that
$$
P_jXP_k(e^{id_kt_1}-e^{id_jt_1})= P_j[X,e^{iDt_1}]P_k =0=P_j[X,e^{iDt_2}]P_k=P_jXP_k(e^{id_kt_2}-e^{id_jt_2})
$$
from which we can draw the desired conclusion if $e^{id_kt_1}\neq e^{id_jt_1}$ or $e^{id_kt_2}\neq e^{id_jt_2}$.
But $e^{i(d_k-d_j)t_1}=1$ and $e^{i(d_k-d_j)t_2}=1$ cannot both be true because $(d_k-d_j)t_1,(d_k-d_j)t_2\in2\pi\mathbb Z$ (together with $d_j\neq d_k$ by construction) would imply that $t_1,t_2$ are rationally dependent, a contradiction. $\square$
Proof of theorem. 2. $\Rightarrow$ 1. is the known direction. For 1. $\Rightarrow$ 2. we first re-use the argument of user1551 that $A$ has to be diagonalizable, i.e. $A=SDS^{-1}$ for some $D\in\mathbb R^{n\times n}$ diagonal.
Now we know that $e^{iAt_j}=Se^{iDt_j}S^{-1}$ is unitary for both $j=1,2$ which is equivalent to $(S^{-1})^*e^{-iDt_j}S^*Se^{iDt_j}S^{-1}={\bf1}$ $\Leftrightarrow$ $[S^*S,e^{iDt_j}]=0$. Because $t_1,t_2$ are rationally independent the previous lemma allows us to conclude that $[S^*S,D]=0$, as well. But this completes the proof:
$$
A^*=(S^{-1})^*DS^*=(S^{-1})^*\underbrace{DS^*S}_{=S^*SD}S^{-1}=(S^{-1})^*S^*SDS^{-1}=SDS^{-1}=A\tag*{$\square$}
$$