To motivate what might be said about "degrees of freedom" in the more general $n\times n$ case, let's look at the $2\times 2$ case in some detail.
One might informally say that the degrees of freedom in the special linear group $SL(2,\mathbb{Z})$ has an intuitive sense of "the number of coordinates needed to specify an instance".
This is a firmer notion when the coordinates involved are real numbers than when, as here, the coordinates are only discrete integers. The problem is related to the possibility that a pair of integers might be coded together as a single integer, so that the counting "how many coordinates are needed" becomes muddled. In the case of real numbers we are saved by imposing a requirement that any "coding" has to involve continuous functions (in a suitably restricted domain) that are continuously invertible (decoding). This prevents a pair of real numbers from being combined into a single real number.
Acknowledging that we are walking on slippery ground, let's consider a couple of "natural ways" to parameterize $SL(2,\mathbb{Z})$. The first involves making an arbitrary choice of the two diagonal entries $a,d$:
$$ \begin{bmatrix} a & b \\ c & d \end{bmatrix} $$
Since we desire that $ad - bc = 1$, we have (in order to get an element of $SL(2,\mathbb{Z})$) only to solve:
$$ bc = ad - 1 $$
Apart from the peculiar case that $ad = 1$ (which admits an infinite number of solutions $b,c$ provided at least one of them is zero), we find that there will be only finitely many $b,c$ which "factor" $ad - 1$.
This seems to say that there are (loosely speaking) two degree of freedom, since (with the two exceptions $a = d = \pm 1$) the values $a,d$ can be chosen arbitrarily and leave a finite number of additional "choices" (about the ways $ad - 1$ will factor).
On the other hand we might begin with choosing $a$ and $b$. Now the valid choices are those pairs $a,b$ which are coprime (no common divisor greater than one). Although not every pair is satisfactory in this sense, the relative fraction of coprime pairs $a,b$ in an expanding region $[-M,M]\times [-M,M]$ as $M\to \infty$ converges to $6/\pi^2$, which is roughly $61\%$.
So again it seems that choosing $a,b$ requires two degrees of freedom. Further, as $a,b$ are coprime, there exist coefficients $c,d$ such that $ad-bc=1$. Then we can introduce an additional integer coordinate $k$ because:
$$ \det \begin{bmatrix} a & b \\ c+ak & d+bk \end{bmatrix} = 1 $$
By this reckoning we would have three degrees of freedom at our disposal!