So I'm complete stuck with something. I know it the following statements are true (or at least the seem to be from the results that I got from messing around with it a bit on MATLAB), but I don't understand why they are true or how to show so. Let $A$ be and $m$X$n$ matrix. Show that:
a) if $x \in N(A^TA)$ then $Ax$ is in both $R(A)$ and $N(A^T)$.
For this one I messed around with it with my own examples and I got $Ax=0$, therefore satisfing the statement, but I don't understand what's actually going on.
b) $N(A^TA)=N(A)$
again, makes sense when I see the results in MATLAB, but don't undestand why it works.
c) $A$ and $A^TA$ have the same rank
d) If $A$ has linearly independent columns, the $A^TA$ is nonsingular.
For the last two I have no idea on how to even start showing the relationship. I feel like I'm missing some crucial relationship between $A$ and $A^TA$, but I'm just not seeing it.
I would greatly appreciate any help of sugestions on how to show that these statements are true.
Thank you very much.
$\endgroup$ 22 Answers
$\begingroup$a) By definition $Ax\in R(A)$; on the other hand, $A^TAx=0$, by assumption, so $Ax\in N(A^T)$.
b) It is clear that $N(A)\subseteq N(A^TA)$. Suppose $x\in N(A^TA)$; then $x^TA^TAx=0$ as well, so $(Ax)^T(Ax)=0$, which implies $Ax=0$.
c) The rank-nullity theorem says that, if $B$ is an $m\times n$ matrix, then $\dim R(B)+\dim N(B)=n$. Let $A$ be $m\times n$ and apply the rank nullity theorem to $A$ and $A^TA$: \begin{align} n&=\dim R(A)+\dim N(A) \\ n&=\dim R(A^TA)+\dim N(A^TA) \end{align} Since by part b we have $\dim N(A)=\dim N(A^TA)$, we conclude $\dim R(A)=\dim R(A^TA)$.
d) If $A$ has linearly independent columns, its rank is $n$; so also $A^TA$ has rank $n$, hence it is invertible.
$\endgroup$ $\begingroup$Let us show that (c) and (d) are both consecuences of (b):
Let $A\in R^{nxm}$ and $nul(A) = nul(A^TA)$
$ \dim nul(A) = \dim nul(A^TA) $
$ n-rank(A) = n-rank(A^TA)$
$rank(A)=rank(A^TA)$
and, if $A$ has linearly independent columns, $nul(A)=\{0_{R^m}\} = nul(A^TA)$; and thus is proven that $\det(A^TA) \neq 0$
As for the proof of (b), and a more rigorous proof of $rank(A) = rank(A^TA)$, I'll post it after dinner.
EDIT:
$\dim col(A) + \dim nul(A) = n$
This property is what I had left unproven in my proof that (c) is consequence of (b), so there:
Let $A = \left( \begin{array}{ccc} - & v_1 & -\\ & \ldots &\\ - & v_n & - \end{array} \right)$ then:
$Ax = 0$ iff $v_ix = 0, \forall i$
$x \perp v_i, \forall i$
$x$ is perpendicular to every column of $A^T$, so it follows that
$x \in nul(A)$ iff $x \in (col(A^T))^\perp$
And so:
$\dim nul(A) = \dim col(A^T)^\perp$
$ = n- \dim col(A^T) = n-rank(A^T) = n-rank(A)$
EDIT:
Proof of (b):
Let $A \in R^{nxm}$:
$\endgroup$$nul(A^TA) = \{ x \in R^m | A^TAx = 0_{R^m} \}$
$= \{ x \in R^m | Ax = 0_{R^n} \quad$ or $\quad Ax \in nul(A^T) \}$
but we have proven that $x \in nul(A^T) \quad$ iff $\quad x \in (col(A))^\perp$
There can be no $Ax \neq 0_{R^m}$ such that $A^TAx = 0_{R^n} \quad$ !
and thus: $\quad nul(A^TA) = \{ x \in R^m | Ax = 0_{R^n} \} = nul(A)$
