Here is one of my favorite proof for the direction ($\Rightarrow$)
Suppose $(p − 1)!= −1 [p]$ then we have : $(p − 1)! +pU =-1 \Leftrightarrow -(p-1)!-pU=1$ with $U \in \mathbb{Z}$
So we have a form like this :
$(p-1)\times (p-2)...\times k\times...2\times 1 -pU=1$
According to Bachet-Bézout theorem we have for each $k\in \{1,...,p-1\}$, $\gcd(k,p)=1$ which means that $p$ is prime. Indeed $k<p$ and $k$ cannot divide $p$ because of Bachet-Bézout relations.
For the direction ($\Leftarrow$) you can see the answer of Stella Biderman who used the properties of field and inverses.