I'm trying to determine the degrees of freedom for parameters of a tensor in shape $(J_1,\dots,J_D)$ and with rank $r$, where "rank" refers to the smallest number of rank-1 tensors whose sum recovers the original tensor, and a "rank-1 tensor" refers to a tensor that can be written as an outer product of vectors.
For the specific case of matrices ($D=2$), the answer is $r(J_1+J_2-r)$, solved in two ways:
Using linearly independent columns (by link):
- Choose $r$ columns freely (and generically) provides $rJ_1$ degrees of freedom.
- Each of the remaining $(J_2-r)$ columns is a linear combinations (so $r$ parameters) of the first $r$ columns, contributing $r(J_2-r)$ degrees of freedom.
- Put together, $rJ_1+r(J_2-r) = r(J_1+J_2-r)$.
Using Singular Value Decomposition (by link):
- Decomposed into $\mathbf{U}\Sigma\mathbf{V}^\intercal$, where:
- $\Sigma$ is a shape $(r,r)$ diagonal matrix contributing $r$ degrees of freedom.
- $\mathbf{U}$ is a shape $(J_1,r)$ orthonormal matrix, with a total number of $rJ_1$ parameters, minus $r$ for each column's unit length constraint, and minus $r(r-1)/2$ for each column pair's orthogonal constraints. In total $rJ_1 - r - r(r-1)/2$.
- $\mathbf{V}$ is similar to $\mathbf{U}$, contributing $rJ_2 - r - r(r-1)/2$ degrees of freedom.
- Put together, $r+ (rJ_1 - r - r(r-1)/2) + (rJ_2 - r - r(r-1)/2) = r(J_1+J_2-r)$.
However, I'm struggling to generalize the above two solutions to tensors. For solution 1, unlike matrices, tensors rank don't easily translate to "linearly independent vectors". For solution 2, CP decomposition looks more alike in structure but lacks orthonormality constraints, and HOSVD has a different structure and is less linked with rank.
How can we derive a similar formula for the degrees of freedom in parameters for tensors? Is there an established method or decomposition that might help simplify this calculation? (I saw some discussions about tensors degree of freedom at math.stackexchange but they are usually related to (anti)symmetries, not ranks).
