Strategy: See KCd's outline, which applies here mutatis mutandis. The idea for proving irreducibility is as follows. (A) Show that $f(x)$ and its reversal $\tilde{f}(x) = x^n f(1/x)$ have no common roots. (B) Argue directly that $f \tilde{f}$ has no degree $n$ factors besides the obvious ones. (C) Use (B) to argue that a factor of $f$ divides both $f$ and $\tilde{f}$, so from (A) it must be constant.
The modular conditions become apparent in step (A), which also proves the reducible cases.
Step A: Let $f_n(x) = x^n + x^4 + x^2 + x + 1$, so $\tilde{f_n}(x) = 1 + x^{n-4} + x^{n-2} + x^{n-1} + x^n$. We will show
\begin{align*}
n \equiv_5 3 &\Rightarrow \Phi_5 \mid f_n \qquad\text{(reducible)} \\
n \equiv_6 4 &\Rightarrow \Phi_6 \mid f_n \qquad\text{(reducible)} \\
\gcd(f_n, \tilde{f_n}) &= 1 \text{ otherwise}.
\end{align*}
A little algebra gives
\begin{equation}
f_n(x) = \Phi_5 + x^3 (x^{n-3} - 1) = \Phi_3 \Phi_6 + x (x^{n-1} + 1) \qquad(1)
\end{equation}
where $\Phi_d$ is a cyclotomic polynomial, so that
\begin{equation}
\tilde{f_n}(x) = x^{n-4} \Phi_5 - (x^{n-3} - 1) = x^{n-4} \Phi_3 \Phi_6 + (x^{n-1} + 1).
\end{equation}
Note that
$$ \tilde{f_n} - x^{n-4} f_n = -(x^{n-3} - 1)(x^{n-1} + 1), $$
so $$\gcd(f_n, \tilde{f_n}) = \gcd(f_n, (x^{n-3} - 1)(x^{n-1} + 1)). \qquad(2) $$
Irreducible contributions to (2) are either divisors of $x^{n-3} - 1$ or $x^{n-1} + 1$.
- If $g \mid x^{n-3} - 1$ and $g \mid f_n$, then from the first equality in (1), $g$ must be $\Phi_5$. But $\zeta_5^{n-3} - 1 = 0$ if and only if $n \equiv_5 3$.
- If $g \mid x^{n-1} + 1$ and $g \mid f_n$, then from the second equality in (1), $g$ must be $\Phi_3$ or $\Phi_6$. Now $\zeta_3^{n-1} + 1$ is never zero, whereas $\zeta_6^{n-1} + 1 = 0$ precisely when $n \equiv_3 1$ and $(n-1)/3$ is odd, which is precisely when $n \equiv_6 4$.
The claim follows.
Step B: Suppose $f_n \tilde{f_n} = k \tilde{k}$ for some $k \in \mathbb{Z}[x]$ of degree $n$. We will show $k = \pm f_n$ or $\pm \tilde{f_n}$.
Let $k(x) = k_0 + k_1 x + \cdots + k_n x^n$. Compare coefficients of $x^n$ in $f_n \tilde{f_n} = k \tilde{k}$ to get $5 = \sum_{i=0}^n k_i^2$. Hence either $k(x)$ is a sum of $5$ monomials with coefficients $\pm 1$, or $k(x)$ is a sum of two monomials with one coefficient $\pm 2$ and the other $\pm 1$. Since $f_n \tilde{f_n}$ is monic with constant coefficient $1$, we find $k_0 k_n = \pm 1$, so indeed $k(x)$ is a sum of five monomials. Replacing $k$ with $-k$ if necessary, we may assume $k_0 = k_n = 1$.
For $5 \leq n \leq 8$, the result may be checked directly, so now take $n \geq 9$. We have
\begin{align*}
f_n \tilde{f_n}
&= x^{2n} + x^{2n-1} + x^{2n-2} + x^{2n-4} + x^{n+4} + \cdots \\
k \tilde{k}
&= (x^n + \epsilon_1 x^{n_1} + \epsilon_2 x^{n_2} + \epsilon_3 x^{n_3} + 1) \\
&\ \ \ \cdot (x^n + \epsilon_3 x^{n-n_3} + \epsilon_2 x^{n-n_2} + \epsilon_1 x^{n-n_1} + 1).
\end{align*}
Comparing coefficients of $x^{2n-1}$ forces ($n_1 = n-1$ and $\epsilon_1 = 1$) or ($n_3 = 1$ and $\epsilon_3 = 1$). We may replace $k$ with $\tilde{k}$ if necessary and assume $n_3=1$ with $\epsilon_3 = 1$, so $\tilde{k}(x) = x^n + x^{n-1} + \cdots + 1$.
If $n_1 = n-1$, the coefficient of $x^{2n-1}$ would be $2$ or $0$, so $n_1 \leq n-2$. If $n-n_2 < n-2$, then the only possible contribution to $x^{2n-2}$ is from $\epsilon_1 x^{n_1} \cdot x^n$ if $n_1=n-2$ and $\epsilon_1=1$. At this stage, the coefficient of $x^{2n-3}$ is $0 = 1 + \epsilon_2 [n_2=n-3] + \epsilon_2 [n_2=3]$ (using Iverson brackets). Hence $\epsilon_2=-1$ and either $n_2=n-3$ or $n_2=3$. If $n_2=n-3$, the coefficient of $x^{2n-4}$ is $-1 \neq 1$, and if $n_2=3$, the coefficient of $x^{2n-5}$ is $-1 \neq 0$. Thus $n-n_2 \geq n-2$ after all, forcing $n_2 = 2$.
At this stage $\tilde{k} = x^n + x^{n-1} + \epsilon_2 x^{n-2} + \epsilon_1 x^{n-n_1} + 1$ for $n_1 \leq n-2$. The coefficient of $x^{2n-2}$ is $1 = \epsilon_2 + \epsilon_1 [n_1=n-2] + \epsilon_1 [n_1=n-3]$. Hence $\epsilon_2 = 1$ and $n_1 \leq n-4$. If $n_1 = 3$, then the coefficient of $x^{2n-3}$ would be $\epsilon_1 \neq 0$, so $n_1 > 3$. If $n_1 \geq 5$, then $n-n_1 \leq n-5$, and the coefficient of $x^{2n-4}$ would be $0 \neq 1$. Hence $n_1=4$, in which case the coefficient of $x^{2n-4}$ is $\epsilon_1 = 1$. That is, $k = f_n$.
Step C: Suppose that $n \not\equiv_5 3$ or $n \not\equiv_6 4$ and let $f_n = gh$ where $g, h \in \mathbb{Z}$ have non-zero constant term. Clearly $\tilde{f_n} = \tilde{g}\tilde{h}$. Let $k=g\tilde{h}$, which is of degree $n$. Now $k\tilde{k} = g\tilde{h}\tilde{g}h = f_n\tilde{f_n}$, so from (B), $k = \pm f_n, \pm \tilde{f_n}$. If $k = \pm f_n$, then $\tilde{h} \mid f_n$, so $\tilde{h} \mid \gcd(f_n, \tilde{f_n}) = 1$, and $h$ is constant. Similarly if $k = \pm \tilde{f_n}$, then $g$ is constant. Thus $f_n$ is irreducible.
