I managed to prove that $p^4$ divides ${p^2 \choose p}-p$ by writing ${p^2\choose p}-p=p{p^2-1\choose p-1}-p$ and then using gaussian pairing(multiplying the last and first terms in the numerator of ${p^2-1\choose p-1}$)to show the congruence holds true.
However I could not advance to $p^5$ divides ${p^2 \choose p}-p$. Any help would be appreciated.