2

It makes intuitive sense to me that the very definition of the null space - all x's that produce the zero vector when multiplied by the rows of a matrix A - would coincide with the conditions for orthogonality (dot product of two vectors = 0). But how would one show that this applies for all combinations of A? I guess i'm trying to wrap my head around a more rigorous way of understanding the notion that the row space is the orthogonal compliment to the null space of a matrix... I hope this makes sense and I am looking forward to hearing some insights!

Saleh
  • 105
  • This question has been asked several times in other threads, for example: https://math.stackexchange.com/questions/21144/intuitive-explanation-of-the-fundamental-theorem-of-linear-algebra – littleO Apr 12 '20 at 06:23

2 Answers2

3

Note that matrix multiplication can be defined via dot products. In particular, suppose that $A$ has rows $a_1$, $a_2, \dots, a_n$, then for any vector $x = (x_1,\dots,x_n)^T$, we have: $$ Ax = (a_1 \cdot x, a_2 \cdot x, \dots, a_n \cdot x) $$ Now, if $x$ is in the null-space, then $Ax = \vec 0$. So, if $x$ is in the null-space of $A$, then $x$ must be orthogonal to every row of $A$, no matter what "combination of $A$" you've chosen.

Ben Grossmann
  • 234,171
  • 12
  • 184
  • 355
  • 6
    can correct me if I'm wrong. but I don't think this answers the OPs question. He seems to understand that the vectors in the nullspace are orthogonal to those in the rowspace. But he doesn't seem to understand why the nullspace contains ALL possible vectors that are orthogonal to any vector in the rowspace. Essentially his qeustion is getting at the difference between "orthogonal subspace" and "orthogonal complement". the former is always a subset of the latter, but the latter requires that it contain ALL possible vectors that are orthogonal to that subspace. – makansij Apr 04 '18 at 06:20
  • @Hunle I think you're right; not sure what I was thinking at the time. That being said, this question is more than 2 years old, and it is unlikely that the OP cares about the question at this point. – Ben Grossmann Apr 04 '18 at 10:29
  • 1
    But I still care <3. How do we know it contains all of the vectors? – makansij Apr 04 '18 at 21:34
  • @Hunle consider any vector $x$ which is perpendicular to the column space. Note that $Ax$, which contains the dot products $a_i \cdot x$, will be $0$. Thus, $x$ is in the null space of $A$. – Ben Grossmann Apr 04 '18 at 21:43
  • 1
    I think this amounts to changing all the ifs in the answer to if-and-only-ifs, right? Anyway, I've upvoted the answer so this question shouldn't keep getting bumped now. –  Nov 10 '18 at 17:09
  • This proves that they are orthogonal, but it does not prove that they are complements ($\mathrm R(A)\cap\mathrm N(A)=\left{\vec0\right}$). – gen-ℤ ready to perish Apr 01 '20 at 20:32
  • @gen-zreadytoperish see the discussion in the comments to this question. Your question is answered by my last comment – Ben Grossmann Apr 01 '20 at 20:43
  • @Omnomnomnom There are no comments under the question. If you’re referring to “consider any vector $x$…” then I don’t see how that addresses the complement issue – gen-ℤ ready to perish Apr 01 '20 at 20:45
  • @gen-zreadytoperish yes that's what I was referring to; I meant this answer rather than this question. You're right though: my comment is insufficient. – Ben Grossmann Apr 01 '20 at 20:52
  • @gen-zreadytoperish Actually never mind, every thing is addressed. My answer shows that $N(A) \subset R(A)^\perp$. My comment shows that $R(A)^\perp \subset N(A)$. It follows that $R(A)^\perp = N(A)$, which is what we want. – Ben Grossmann Apr 01 '20 at 20:54
  • @Omnomnomnom I worked out a proof on my own – gen-ℤ ready to perish Apr 01 '20 at 21:00
  • @BenGrossmann there is still a small mistake, in your comment it should say "consider any vector $x$ which is perpendicular to the row space", not colum space – Ken Tjhia Apr 28 '25 at 23:09
  • @Ken Yes, that’s right. Unfortunately you can’t edit old comments – Ben Grossmann Apr 29 '25 at 11:23
2

Omnomnomnom has proven that the row space $\newcommand{\R}{\mathrm{R}} \R(A)$ and null space $\newcommand{\N}{\mathrm{N}} \N(A)$ are orthogonal to each other; that is, $\newcommand{\r}{\vec r} \newcommand{\n}{\vec n} \forall\r\in\R(A)\ \forall\n\in\N(A): \r\perp\n$. I will show that they are complements of each other: $\R(A)\cap\N(A)=\left\{\vec0\right\}$. Both of these criteria must be met for two subspaces to be orthogonal complements.

Proof: Suppose $\newcommand{\v}{\vec v} \v\in\N(A)$ and $\v\in\R(A)$. Recall that, as Omnomnom has shown, if $\n\in\N(A)$ then $$\n\cdot\r=0$$ where $\r\in\R(A)$. Now $$\v\cdot\v=0=\left\|\v\right\|^2$$ Therefore $\left\|\v\right\|=0$, so any vector $\v$ in both $\R(A)$ and $\N(A)$ must equal $\vec0$. Therefore $\R(A)\cap\N(A)=\left\{\vec0\right\}$, and so by definition $\R(A)$ and $\N(A)$ are complementary subspaces as well as orthogonal. $\blacksquare$

gen-ℤ ready to perish
  • 6,732