Skip to main content

Why A^{T}A is invertible? (4) Linear Algebra

Appendix A: null space and column space
We use the null space for the proof, therefore, I will explain the null space a bit. If you know about the null space, of course you can skip this entry.

The null space of a matrix A is a set of non-zero vector x that satisfies Ax = 0.
Let me show you an example square matrix A that has null space.
When x \neq 0 , following x is a solution.
Therefore, this x is a null space of A. When an a is an scalar, ax, a \neq 0 are also the solutions. It means these are also null space. In this example, the matrix is singular (and square). If the matrix is not singular, the solution must be 0 only. Because, if a square matrix is not singular, there is the inverse,
Therefore, x = 0. In this case, we say there is no null space.

Let me show you another example, but this time a rectangle matrix A that has null space.
The solution is the same as the last example.

By the way, these are all about definition of null space. I could finish the story about null space at this point, however, I would like to think about the meaning of null space a bit more. I would like to talk about the existence of the inverse matrix, so let's talk about square matrices. Because a rectangle matrix can be interpreted as a matrix that has too many equations or too less equations, there is no inverse for such matrices.

null space is related with a matrix is singular or not, that means the matrix has independent columns or not. Because, if we write down an A with column vector, and multiply with x,

Therefore, Ax is a linear combination of the column vectors a_i with coefficients x_i. If this result is 0, x is a null space. This is one aspect of null space.

When we look at the columns of Ax = b, b is the linear combination of column vector of A. A space consists of column vectors is called column space. If we have n independent columns in n-dimensional space, we could express any point in the n-dimensional space. This means we have a solution for any b and we have the inverse. If A has the inverse, the linear combination of the column vector becomes 0 iff x= 0. This also means there is no null space (= or only 0 exists).

Now we see the null space, column space, and the existence of the inverse. Do you think you have now a bit clearer view about the relationship among them?

Acknowledgements

Thanks to Marc D. who listened my explanation and gave me comments.
Labels:

Comments

Post a Comment

Popular posts from this blog

Geometric Multiplicity: eignvectors (2)

If eigenvectors of a matrix A are independent, it is a happy property. Because the matrix A can be diagonalized with a matrix S that column vectors are eigenvectors of A . For example, Why this is a happy property of A? Because I can find A's power easily. A^{10} is not a big deal. Because Λ is a diagonal matrix and power of a diagonal matrix is quite simple. A^{10} = SΛ^{10} S^{-1} Then, why if I want to compute power of A ? That is the same reason to find eigenvectors. Eigenvectors are a basis of a matrix. A matrix can be represented by a single scalar. I repeat this again. This is the happy point, a matrix becomes a scalar. What can be simpler than a scalar value. But, this is only possible when the matrix S's columns are independent. Because S^{-1} must be exist. Now I come back to my first question. Is the λ's multiplicity related with the number of eigenvectors? This time I found this has the name. Geometric multiplicity (GM): the number of in...
Post a Comment
Read more

Gauss's quote for positive, negative, and imaginary number

Recently I watched the following great videos about imaginary numbers by Welch Labs. https://youtu.be/T647CGsuOVU?list=PLiaHhY2iBX9g6KIvZ_703G3KJXapKkNaF I like this article about naming of math by Kalid Azad. https://betterexplained.com/articles/learning-tip-idea-name/ Both articles mentioned about Gauss, who suggested to use other names of positive, negative, and imaginary numbers. Gauss wrote these names are wrong and that is one of the reason people didn't get why negative times negative is positive, or, pure positive imaginary times pure positive imaginary is negative real number. I made a few videos about explaining why -1 * -1 = +1, too. Explanation: why -1 * -1 = +1 by pattern https://youtu.be/uD7JRdAzKP8 Explanation: why -1 * -1 = +1 by climbing a mountain https://youtu.be/uD7JRdAzKP8 But actually Gauss's insight is much powerful. The original is in the Gauß, Werke, Bd. 2, S. 178 . Hätte man +1, -1, √-1) nicht positiv, negative, imaginäre (oder gar um...
Post a Comment
Read more

Tezuka Osamu's Black Jack, "Shrinking"

I like several novel authors. My first favorite author is probably Teduka, Osamu. I still love him. The list grows by adding Hoshi, Shinichi, Agatha Christie, Hermann Hesse, and so forth. My first favorite article of Tezuka was Atom as most of the (boy's) Tezuka fans did. But my favorite is Black Jack. I try to summarize one story, it is still quite vivid in my memory. I first read this story when I was 13 - 15 years old. I re-read it at least several times since Black Jack is composed of many short episodes. The title should be "ちぢむ (SHRINKING)" or it might be "縮む(Shrinking)". (It is not so convenient to translate this to English, since English does not have a system to say the exact same word in several ways. So I just simulate it with capital letters.) Black Jack is a genius surgeon, but he does not have the license. In short, his medical activity is illegal. His skill is top level in the world, but, the fee is also out-of-law expensive. In the story ...
Post a Comment
Read more