Let $G$ be a finite group and let $\phi$ be an automorphism of $G$. We define an action of $\operatorname{Aut}(G)$ on the set $\operatorname{Rep}(G)$ of complex-valued representations of $G$ by ${}^\phi\rho(g) := \rho(\phi(g))$. Let $[\operatorname{Rep}(G)]$ denote the set of isomorphism classes of representations of $G$, where $\rho \cong \rho'$ iff there exists an invertible linear map $T$ such that $T^{-1}\rho(g)T = \rho'(g)$ for all $g \in G$.
If $\phi$ is an inner automorphism of $G$, then the induced action of $\phi$ on $[\operatorname{Rep}(G)]$ is trivial. Thus we get an action of the outer automorphism group $\operatorname{Out}(G)$ on $[\operatorname{Rep}(G)]$.
My question: is this action faithful, i.e. can there be outer automorphisms of $G$ that fix all isomorphism classes of representations of $G$?