Here I want to put matrix according to the order of this story, however, I will skip the matrix story for simplicity.
Vector is a sequence of scalars. The sequence order of scalars is very important. If we changed the order of scalars, they are totally different vectors. For example, we had a vector that represents position, it was [direction distance] that means 50 degree from north, distance 5 km. If we exchange the direction and distance, it becomes [distance direction] that means 5 degree from north, 50 km distance. That is the different position (at least in the Euclidean space).
If we can put 4 scalars. Figure 1's the second from the top shows the vector that has 4 scalars. A scalar is one (real) number and a vector is a sequence of scalars. If we add more scalars in a vector, what happens? If we add scalars more and more, infinite number of scalars are added, then it becomes a function as shown in the Figure. OK, I cheated here a bit. We need some more prerequisite to make a function from a vector. However, I think this is a good intuition. (We can detour a bit here about this intuition. Some mathematics book wrote about 'inner product of functions', this is strange, how can we have inner product of functions. But, if we can follow this intuition, a function is a extension of a vector, we can see the inner product of functions. A correlation coefficient is also a inner product, therefore, 1 is correlated, 0 is un-correlated. It is a inner product, 0 means perpendicular, then there is no relationship. This intuition gave me these understanding, so I think this is not so bad.) I hope now you see one of the relationships among scalar, vector, and function.
Figure 1: Scalar, vector, and function
If we see in this way, the difference of scalar, vector, and function is only the difference of number of pairs: X and Y.
- Scalar: One X (e.g., height) and its value (e.g., 170cm)
- Vector: multiple X (e.g., [angle distance]) and its values (e.g., [50 5])
- Function: continuous X and its value Y (e.g., Y = f(X))
Now, what is invariant of scalar, vector, and function? This kind of abstraction is common in mathematics. Why we care such invariant? Because it could be the substance of the object. Also if something does not change, then, we don't need to distinguish them. That's great we have not so much time, so we can skip to distinguish them. We are not so interested in masks, we want to know the real face under the masks. Masks can be changed, but, the real face can not be changed. Something does not change under many conditions, it could be the real face -- substance. Therefore, we want to know the invariant. Eigenvalue and transfer function need this abstraction, so I first talked about this.
Next, let's talk about eigenvalue.