Lets consider the adjoint action
$$\mathrm{Ad}: G\rightarrow G$$ $$\mathrm{Ad}_g(g')=gg'g^{-1}$$
This can equally be written as
$$(l_g\circ r_{g^{-1}})g'=\mathrm{Ad}_g(g')$$
Lets consider a Lie group, for example the $SL(n)$ Lie group of $n\times n$ matrices $M$ with $\det M=1$. It is known that the generalized Pauli elements under matrix multiplication form a subgroup of $SL(n)$. It is also known that the generalized Pauli elements without the identity constitute a basis set for the $\mathfrak{sl}_n$ general linear Lie algebra.
The Lie algebra of a Lie group $L(G)$ is isomorphic to $T_eG$ via
$$i(A)=l_{g*} A:=L^A\in L(G)$$
being an isomorphism for $A\in T_eG$ and $L(G)$ denoting the space of left-invariant vector fields.
Crucially: $L^A_e=A$, so the vector field evaluated on $e\in G$ equals $A$.
There are elements that can be defined with both, being in the Lie group and its associated Lie algebra, and those are precisely the elements of the basis set consisting of group elements spanning the Lie algebra. It is known that
$$(h_* v)(f) = v (f\circ h)$$
Lets take a basis vector for the Lie algebra $P$ which is a group element, so $P\in SL(n)$ and $P\in \mathfrak{sl}_n$.
$$\rightarrow (l_g\circ r_{g^{-1}})_* P(f)=(l_g\circ r_{g^{-1}})_* L^P_e(f)=L^P(f\circ l_g\circ r_{g^{-1}})(e)=L^P_e (f)$$
But this does not make sense since this would indicate that the adjoint representation is always like the identity for basis vectors of the Lie algebra.
So where did I go wrong?