First recall the definition of $\|\cdot\|_\infty$ as stated in Folland's book:
$$\|f\|_\infty=\inf\{a\geq0:\mu(\{x:|f(x)|>a\})=0\}$$
If $\|f\|_\infty<\sup_x|f(x)|$, there is some $x\in A$ such that $|f(x)|>\|f\|_\infty$. But then $|f(x)|>a>\|f\|_\infty$ for some $a>0$. Hence $\mu(\{x:|f(x)|>a\}>0$, contradicting the definition of $\|\cdot\|_\infty$. Thus $\sup_x|f(x)|\leq\|f\|_\infty$.
If now $\sup_x|f(x)|<\|f\|_\infty$, then there is some $a>0$ such that $|f(x)|< a<\|f\|_\infty$ for all $x\in A$. Thus, we have
$$\mu(\{x:|f(x)|>a\}>0\})=\mu(\varnothing)=0,$$
again contradicting the definition of $\|f\|_\infty$. Thus, we must have $\sup_x|f(x)|=\|f\|_\infty$.
From here, since the map $x\mapsto x^p$ is continuous and increasing, it is clear that $\|f\|^p_\infty=\sup_x|f(x)|^p$.