In the linked paper, they define the group action in question immediately before Remark 2.5.1: $$(g\cdot p)(x,y)=p(g^{-1}(x,y))$$ That is, compute $g^{-1}\cdot(x,y)$ via the usual action (see below) of $\mathrm{GL}(\mathbb{R}^2)$ on $\mathbb{R}^2$; then evaluate $p$ at the new point.
In comments to the answer to the linked question, you ask what it means to multiply a matrix by an ordered pair. Simple: treat it like a vector; that is, $$\mathcal{A}\cdot(x,y)=\mathcal{A}\begin{bmatrix}x\\y\end{bmatrix}$$ I call this the "usual action."
For example, suppose $p(x,y)=x^2+2xy$ and $g=\begin{bmatrix}1&1\\0&1\end{bmatrix}$. Then $g^{-1}=\begin{bmatrix}1&-1\\0&1\end{bmatrix}$, so that $$g^{-1}\cdot(x,y)=(x-y,y)$$ Thus \begin{align*}
(g\cdot p)(x,y)&=p(x-y,y) \\
&=(x-y)^2+2(x-y)y \\
&=x^2-y^2
\end{align*}
This operation preserves addition and multiplication, because evaluation of polynomials is a homomorphism of rings. For example, if $g$ is as above, then \begin{align*}
(g\cdot pq)(x,y)&=(pq)(x-y,y) \\
&=p(x-y,y)q(x-y,y) \\
&=(g\cdot p)(x,y)(g\cdot q)(x,y)
\end{align*}