You can deal with these questions totally abstractly, no hand computations needed. I’ll answer your question as asked, but the generalization to other-degree polynomials than quartic and other characteristics than $2$ are pretty much immediate.
You need to know that $\Bbb F_{2^m}\subset\Bbb F_{2^n}$ if and only if $m\mid n$. Let’s use this fact:
Let $g(x)$ be an irreducible quartic over $\Bbb F_2$. Then any one of its roots, say $\rho$, will generate $\Bbb F_{2^4}=\Bbb F_{16}$, and all its roots are in that extension of $\Bbb F_2$. Now look at $g$ as an $\Bbb F_{2^2}=\Bbb F_4$-polynomial. The root $\rho$ of $g$ still generates $\Bbb F_{16}$, which you see does contain $\Bbb F_4$, and is a quadratic extension of this latter field. So the $\Bbb F_4$-minimal polynomial of $\rho$ is quadratic: $g$ factors over $\Bbb F_4$. (And as @JWL has remarked in a comment, $x^4+x^3+1=(x^2+\omega x+\omega)(x^2+\omega^2x+\omega^2)$, where $\omega$ is an element of $\Bbb F_4$ not in the prime field.)
On the other hand, $\Bbb F_8\not\subset\Bbb F_{16}$ because $3$ is not a divisor of $4$. You see that $\rho$ now must generate a quartic extension of $\Bbb F_8$ because $\rho$ still isn’t in $\Bbb F_{2^6}$ : $4$ isn’t a divisor of $6$. In other words, $g$ is still irreducible over $\Bbb F_8$.