The main trick here is to let $G$ act on the upper-half plane in $\Bbb C$, which we denote by $\Bbb H$. That way you can tell if a reduced word should be the identity or not depending on whether it fixes all elements of $\Bbb H$ or not.
The matrices in $PSL_2(\Bbb R)$ act naturally in this way as Möbius transformations (which I assume you know based on the fact that you know this is applicable to modular forms).
So we wish to show that a reduced word $S^{n_1}X^{m_1}SX^{m_2}\ldots SX^{m_k}$ is the identity iff all the exponents are zero. Since the action is free--recall a Möbius transform is the identity iff it fixes three points--this is enough.
Now if we write the matrix action in more concrete terms it is easier
$$S\cdot z= S\begin{pmatrix} z \\ 1\end{pmatrix} = \begin{pmatrix} -1 \\ z\end{pmatrix}=-{1\over z}$$
$$X\cdot z=X\begin{pmatrix} z \\ 1\end{pmatrix} = \begin{pmatrix} -1 \\ z+1\end{pmatrix}=-{1\over z+1}$$
So now we want to break up the upper half plane into pieces that are easily understood in terms of the action of $G$. Since we're using $S,X$ for generators, the usual fundamental domain isn't right, but $T^{-1}D$ works great.
So what will the pieces be? Well the first should be $A=\{\operatorname{Re}(z)>0\}$ since $S$ reverses the sign of a real part. The next is $B=\{|z+1|<1\}$ since the inside of that disk is moved around in a natural way by $X$, which is inversion through the circle $\{|z+1|=1\}$. Finally we take $C=\{\operatorname{Re}(z)<-1/2\}$. We note that since $S$ gives a value with the opposite real part to that of $z$ that
$$S(B), S(C)\subseteq A.$$
Also it is clear from a direct computation that $X$ rotates $\Bbb C$ around $\zeta_3=\exp\left({2\pi i\over 3}\right)$ so that
$$X(A), X^2(A)\subset B\cup C.$$
Finally we see that $S(\Bbb H\setminus (A\cup B\cup C))\subseteq A$ and $X, X^2$ both map $\Bbb H\setminus (A\cup B\cup C)$ into $B\cup C$.
Now if $z\in \Bbb H\setminus (A\cup B\cup C)$ we are out of luck, since any word ending in $S$ acts on this as a word ending in $X^i$ on the set $A$, which we know goes into $B\cup C$. If the word ends there, it cannot be trivial, so there must be another $S$, but then this just maps back into $A$, and then the next power of $X$ pushes it back into $B\cup C$ ad infinitum, ergo the word must not have ended with an $S$. But if it ended with $X^i$, then $X^i(z)\in B\cup C$ and we start the game all over again.
Hence the word cannot end in a power of $X$ or $S$, so it must have been the trivial word.