4

Here's the entire question: Let $A$ be an 8 $\times$ 5 matrix of rank 3, and let $b$ be a nonzero vector in $N(A^T)$.

a) Show that the system $Ax = b$ must be inconsistent. Gonna take a wild stab at this one... If the rank is 3, that means the dimension of the column space is 3. But $A$ has 5 columns, so they are not all linearly independent and therefore $Ax = b$ is inconsistent.

b) How many least squares solutions will the system $Ax = b$ have? Explain.

On previous problems, I found the best least squares linear fit, where the approximation of $x$ was a vector that contained sometimes regular numbers, and sometimes variables. Does this mean that there must be either 1 linear solution or infinite (because you can always find an approximation)? In the example that apparently had an infinite number of least squares solutions, it appeared that one row of $A^TA$ was a constant multiple of another row, leading to a row of zeros in reduced row echelon form. From this problem I know that $A^TA$ is a 5x5 matrix, but I don't think I can prove that any rows are a scalar multiple of other rows, so I'm guessing I have to use some other means of figuring this out.

Sorry if I sound like I have no idea what I'm talking about. Just wanted to try out the problem to my best ability before asking about it.

Lorenzo B.
  • 2,290
Chris
  • 755
  • For (a), that's not good enough, because your argument doesn't mention what $b$ is. You need to use the fact that the column space of $A$ is the set of all vectors $c$ such that $Ax=c$ is consistent. ... The vector $b$ being in the null space of $A^\top$ means $b$ is perpendicular to every vector in the column space (why?); hence, $b$ can't be in the column space, unless $b$ is zero (why?). – Christopher Carl Heckman Nov 11 '15 at 07:11
  • For (b), use the fact that $b$ is in the null space of $A^\top$ means $A^\top b = 0$, and use the fact that the normal equation is $(A^\top A)x = (A^\top b)$. – Christopher Carl Heckman Nov 11 '15 at 07:13
  • So $R(A)$ = $N(A^T)^⊥$ (just found that formula in my book), and because b is in the latter, that means it must be perpendicular to R(A). Though I can't find anything about b having to be 0 because of this. I'll keep looking, though. – Chris Nov 11 '15 at 07:37
  • The book says that $Ax = b$ is consistent if and only if $b$ is an element of $R(A)$. But is that not possible because $b$ doesn't contain the 0 vector? I remember doing that when checking to see if something is a subspace. – Chris Nov 11 '15 at 07:49
  • And for the part $b$, I said $A^Tb$ = $0$, meaning $(A^TA)x$ = $0$, and because the rank of $A$ is equal to the rank of $A^TA$, that means $A^TA$ has a rank of 3 and therefore has a nullity of 5, which means there are 5 free variables in $A^Tb$ = $0$ and that means $Ax=b$ has infinitely many least squares solutions. Is that right? – Chris Nov 11 '15 at 08:06
  • Re your last comment (about (b)): The nullity is actually $5-3=2$, but otherwise, yes. – Christopher Carl Heckman Nov 11 '15 at 21:38
  • Re (a): Let $W$ be a subspace. If $b$ is in $W^\perp$, then $b\cdot c=0$ for all $c$ in $W$. But if $b$ is in $W$ as well, then ... – Christopher Carl Heckman Nov 11 '15 at 21:40

1 Answers1

1

a

The system is inconsistent. Example: $$ \left[ \begin{array}{cc} 1 & 0 \\ 0 & 0 \end{array} \right] % \left[ \begin{array}{cc} x_{1} \\ x_{2} \end{array} \right] = \left[ \begin{array}{c} 0 \\ 1 \end{array} \right] $$

b

No, the system is not unique. The solution is an affine space: infinite solutions. In linear algebra the number of solution is either 0 (no existence), 1 (existence and uniqueness), or infinite (existence, no uniqueness).

A pencil and paper exercise: Unique least square solutions

Theoretical treatment: Is a least squares solution to Ax=bAx=b necessarily unique

To see a diagram Is the unique least norm solution to Ax=b the orthogonal projection of b onto R(A)?

How the SVD produces the Moore-Penrose pseudoinverse: How does the SVD solve the least squares problem?

dantopa
  • 10,768