I was curious to see if a computer gets some counterexamples (after playing Galois games, and chasing discriminants). And it found some many counterexamples starting with the prime $p=17$.
I picked one with "small coefficients",
that can be maybe checked also with human tools.
We work over $\Bbb F_p$ all the time.
$$
p = 17\ ,\ F=\Bbb F_p=\Bbb F_{17}\ ,\qquad f= x^5+7x+1\in F[x]\ .
$$
The computer verdict is quick, sage code:
p = 17
F = GF(p)
R.<x> = PolynomialRing(F)
a, b = F(7), F(1)
f = x^5 + a*x + b
s = 25*b^4 + 16*a^5
print(f'Is f = {f} irreducible? {f.is_irreducible()}')
print(f'Is s = 25b^4 + 16 a^5 = {s} a square? {s.is_square()}')
Pasting the code into the sage interpreter gives:
Is f = x^5 + 7*x + 1 irreducible? True
Is s = 25b^4 + 16 a^5 = 14 a square? False
To see that $14$ is not a square, it is enough to compute all squares and see that $14$ is not in the list. The squares are
$(\pm 1)^2=1$,
$(\pm 2)^2=4$,
$(\pm 3)^2=9$,
$(\pm 4)^2=16$,
$(\pm 5)^2=8$,
$(\pm 6)^2=2$,
$(\pm 7)^2=15$,
$(\pm 8)^2=13$. And $14$ is indeed not in the list.
To see that the given polynomial is irreductible, we have to check all possible roots in $F$. No match. Then check that there is no factor of degree two. If there is one, then we get a decomposition:
$$
f = x^5 + 7x +1
=(x^2 +sx+t)\left(x^3-sx^2+ux+\frac 1t\right)\ .
$$
This leads to an algebraic system with $3$ equations, $u-s^3+t=0$,
$1/t+su-st=0$, and $s/t + tu =7$.
It is not so simple to see there is no solution with $s,t,u\in F$.
Again using computer:
sage: J = S.ideal([u - s^3 + t, 1 + s*t*u - s*t^2, s + t^2*u - 7*t])
sage: J.elimination_ideal([u,s]).gens()[0].factor()
(t^3 + t^2 + 3*t + 1)
* (t^4 - t^3 + 5*t^2 - 8*t - 8)
* (t^6 - 4*t^4 + 5*t^3 - 4*t^2 + 8*t - 2)
(Manually rearranged.)
(After the elimination of $u,s$ we get an ideal of $F[t]$, it is principal, there is one generator, its factorization above is over $F$, no linear factor.)