If $L$ is context-free, then by the pumping lemma for CFLs, there is a pumping length $k$ such that each string of length longer than $k$ can be somehow subdivided into $uvwxy$, with $vx$ nonempty (among other conditions), such that $uv^nwx^ny$ is in the language for each $n\geq 0$. (Without loss of generality, we may choose a pumping length greater than 1 so as to avoid edge cases in what follows.)
Let $p$ be any prime greater than $k$. Then $a^p$ belongs to the language, and we should be able to subdivide it as above. Let us put $\ell=|vx|\geq 1$. Then the pumped string $uv^nwx^ny$ is equal to $a^{p+(n-1)\ell}$. But that means if we pump $n=p+1$ times, we get the string $a^{p(\ell+1)}$, which is not in the language because $p(\ell+1)$ is composite.
Hence $L$ is not context-free, because it fails the conditions of the pumping lemma for CFLs.