Home
Contact
Sitemap
Bookmark us
Home
Services
Products
Biometrics
eFinance
Company
Vector spaces and subspaces - Properties and examples
A vector space is something where you can take any two things in it, add them together, and get a third. You can also multiply by scalers, and still get something in it. This is the most useful property of a vector space. A subspace is just a vector space that is inside a bigger vector space. We will look at some examples. One we have seen already is the solutions of a
homogeneous
linear system.
Here are all the properties of vector spaces: (In the following, V is a vector space)
Really important properties you need to check for:
if
u
and
v
are in V, then so is
u
+
v
.
if
u
is in V, so is k
u
for any number k.
Pretty important properites:
the zero vector is in V. if you add zero to anything, it doesn't change them.
if
u
is in V, then so is -
u
.
Other properties (These are usually so obvious you might forget about them)
u
+
v
=
v
+
u
(
u
+
v
)+
w
=
u
+(
v
+
w
)
c(
v
+
u
)=c
v
+c
u
(c+d)
v
=c
v
+d
v
1
u
=
u
c(d(
u
)=(cd)
u
A
subspace
of a vector space is a vector space that is inside another vector space.
Some Examples we'll look at:
R
2
- supspaces are lines through the origin, and also the point (0,0).
R
2
- supspaces are lines and planes through the origin
Vector space of quadratic polynomials, and some subspaces.
Magic squares
Exercise
Is the set of points (x,y) with x=3 a subspace of R
2
?
If it is, prove it. If it's not, give an example to show why not.
Answer
It is not a subspace, since (3,0) and (3,9) are both points in this set, but (3,0)+(3,9)=(6,9) is not a point in the set.
Note, this set of points also fails several other properties that a vector space should satisfy. Eg, it fails the second property, for example, 5*(3,1)=(15,5), so (3,1) is in this set, but not all it's multiples are. It also fails the third property, as (0,0) is not in this set. And it fails the fourth, since (3,1) is in the set, but (-3,-1) is not. But we don't need to check all these; even if only one fails, it still is not a subspace.
Basis of a vector space
A basis of a vector space is a set of vectors that you can use to get to any point of the vector space. There can be many different choices of basis for a vector space, eg, R
2
has a basis {(1,0), (0,1)}, but it also has a basis {(1,1), (1,-1)}. Whatever basis we choose will always have the same number of elements.
The basis is about the most important thing about a vector space; it means that even though there are infinitely many things in the vector space, you can epxress them in terms of a few "basic" vectors, that is, in terms of the basis.
Dimension of a vector space
section 5.5
what is the dimension of the space we live in? How do you know? The answer in this section is that the dimension is the number of basis vectors; it's how many vectors you need to get everywhere in the vector space.
Rank and Nullity
sections 5.2, 5.6
The idea of the rank nulity formular is that if you have n variables, then if you add one condition, you expect to get n-1 free varibales, and generally, each time you add a condition, you cut down the number of free variables by 1.
Eg: for the variables x,y,z, if we add the condition x+y+z=0, then now we have two free variables, y and z, and we write x in terms of them.
Number of free variables (x,y,z) - number of conditions (x+y+z-0) = number of free variables (y,z)
3-2=1
The idea of "number of conditions" becomes "rank", since we are only intersted in the number of independent conditions.
The idea of "free variables" becomes the "nullity" of the matrix.
Rank
The rank of a matrix is the dimension of the space spanned by it's columns.
Nullity
The nullity of a matrix A is the dimension of the space of solutions of
A
x
=
0
Problem 1
(this is a rank problem)
What is the dimension of the space spanned by the vectors:
/
1
\
/
1
\
/
0
\
|
2
|
,
|
1
|
,
|
1
|
?
\
3
/
\
1
/
\
2
/
Problem 2
(this is a nullity problem)
What is the dimension of the space of solutions of the system
x
+
y
=
0
2x
+
y
+
z
=
0
3x
+
y
+
2z
=
0
It turns out that if we know the answer to one of these problems, then we can easily find the answer to the other, by the rank nulity formula.
Answer to problem 1:
This problem is the same as asking what is the dimension of the space spanned by the column vectors of the following matrix:
/
1
1
0
\
|
2
1
1
|
\
3
1
2
/
How do we work out the answer? The column space means all the points we can get to from the vectors that are the columns of the matrix. The points you can get to are of the form:
/
1
\
/
1
\
/
0
\
a
|
2
|
+
b
|
1
|
+
c
|
1
|
\
3
/
\
1
/
\
2
/
The set of all vectors you can get to like this is a subspace of R
3
, since if you can get to a point
v
and a point
u
, then you can also get to the point
v+u
, using these three vectors. And if you multiply by a constant k, then k
v
can also be reached with these vectors.
A basis is a set of vectors that will get you everywhere (ie, they span the subspace), and they have to be linearly independent.
Well, this subspace is where you can get to with (1,2,3), (1,1,1), and (0,1,2), so they do span it.
So that means the dimension is at most 3. But it might be less than three, eg, if we took a space spanned by (1,1,1), and (2,2,2) and (3,3,3), it could be spanned with just (1,1,1), so only has dimension 1. To find the dimension, we need to find a linearly independent set of vectors that span. We need to cut down on the number of vectors we give if there are any relationships between them.
How do we find if there is any relationships between the vectors?
As usual, we solve this problem by row reduction of a matrix.
Important note
: When you make row operations, the columns get changed, but the relationships between the columns are not changed.
Row reduction works as follows:
/
1
1
0
\
/
1
1
0
\
/
1
1
0
\
/
1
1
0
\
R
1
-R
2
/
1
0
1
\
|
2
1
1
|
R
2
-2R
1
|
0
-1
1
|
|
0
-1
1
|
-R
2
|
0
1
-1
|
|
0
1
-1
|
\
3
1
2
/
R
3
-3R
1
\
0
-2
2
/
R
3
-2R
1
\
0
0
0
/
\
0
0
0
/
\
0
0
0
/
What are the relationships between the columns of the final matrix?
It's not too hard to see that the third column depends on the first two. We have:
/
1
\
/
0
\
/
1
\
|
0
|
-
|
1
|
=
|
-1
|
\
0
/
\
0
/
\
0
/
So for this final matrix, the dimension of the column space is 2, since it is spanned by the columns (1,0,0) and (0,1,0), since the last column depends on the first two, and so is not neccessary to get us to all points in the column space.
Now, we can look at each matrix in the sequence of the row reduction, and see that each time we have the same relationship:
C
1
-C
2
=C
3
That is, in each case, the third column is the first minus the second, and so the dimension is two in each case. In particular we have:
/
1
\
/
1
\
/
0
\
|
2
|
-
|
1
|
=
|
1
|
\
3
/
\
1
/
\
2
/
and the dimension of the column space of the original matrix is 2.
In general, to find the dimension of the column space of a matrix, we have to row reduce, and then the number of leading ones is equal to the dimension of the column space. This number is called the rank of the matrix.
So the rank of
/
1
1
0
\
|
2
1
1
|
\
3
1
2
/
is 2.
Answer to problem 2:
To solve this problem, we need to solve the homogeous system, by reducing the following augmented matrix:
/
1
1
0
|
0
\
|
2
1
1
|
0
|
\
3
1
2
|
0
/
This involves the same work as for the above problem:
/
1
1
0
|
0
\
/
1
1
0
|
0
\
/
1
1
0
|
0
\
/
1
1
0
|
0
\
R
1
-R
2
/
1
0
1
|
0
\
|
2
1
1
|
0
|
R
2
-2R
1
|
0
-1
1
|
0
|
|
0
-1
1
|
0
|
-R
2
|
0
1
-1
|
0
|
|
0
1
-1
|
0
|
\
3
1
2
|
0
/
R
3
-3R
1
\
0
-2
2
|
0
/
R
3
-2R
1
\
0
0
0
|
0
/
\
0
0
0
|
0
/
\
0
0
0
|
0
/
There is one row of zeros, and it's zero after the dividing line too, so the system is consistant, and it has one free variable, so the dimension of the space of solutions is 1. That means the nullity of the matrix is 1.
Explicitly, the reduced matrix tells us that
x=-z
y=z
So the general solution is
/
x
\
/
-z
\
/
-1
\
|
y
|
=
b
|
z
|
=
z
|
1
|
\
z
/
\
z
/
\
1
/
So the set of solutions is a subspace of R
3
of dimension 1.
Relation between rank and nullity
In the above problems, we have
rank
=
2
nullity
=
1
number of columns
=
3
notice that
Rank + Nullity = number of columns
The same result holds for any matrix A:
Rank of A + Nullity of A = number of columns of A
Example 1:
If there is a system of six homogeneous equations in five unknowns,
A
x
=
0
and all the solutions are a multiples of one particular solution, then will there be a solution to the system
A
x
=
b
for all possible choices of the vector
b
?
Answer:
First we need to work out what the question is telling us, and what it is asking.
First, there are 6 equations and 5 unknowns, so that is something like:
?
x
1
+
?
x
2
+
?
x
3
+
?
x
4
+
?
x
5
=
0
?
x
1
+
?
x
2
+
?
x
3
+
?
x
4
+
?
x
5
=
0
?
x
1
+
?
x
2
+
?
x
3
+
?
x
4
+
?
x
5
=
0
?
x
1
+
?
x
2
+
?
x
3
+
?
x
4
+
?
x
5
=
0
?
x
1
+
?
x
2
+
?
x
3
+
?
x
4
+
?
x
5
=
0
?
x
1
+
?
x
2
+
?
x
3
+
?
x
4
+
?
x
5
=
0
As an augemented matrix:
/
?
?
?
?
?
|
0
\
|
?
?
?
?
?
|
0
|
|
?
?
?
?
?
|
0
|
|
?
?
?
?
?
|
0
|
|
?
?
?
?
?
|
0
|
\
?
?
?
?
?
|
0
/
If all the solutions of A
x
=
0
are multiples of one solution, say the solution
v
, that means that
v
spans the space of solutions, and so
v
is a basis of the space of solutions, so the space of solutions of A
x
=
0
is one dimensional, and the nullity of A is 1.
The question is asking if we can get to any
b
in R
6
from the columns of A, can we find a solution of
/
?
?
?
?
?
|
b
1
\
|
?
?
?
?
?
|
b
2
|
|
?
?
?
?
?
|
b
3
|
|
?
?
?
?
?
|
b
4
|
|
?
?
?
?
?
|
b
5
|
\
?
?
?
?
?
|
b
6
/
for all
b
=( b
1
,b
2
, b
3
, b
4
, b
5
, b
6
). So it's asking about what space the columns span, and do they span all of R
6
?
The space spanned by the columns of a is called the rank of A, and we have the result that
rank+nullity=number of columns
so we have
rank+1=5
So the rank must be 4, so the dimension of the column space is 4, so the columns can't span all of R
6
.
note: It should be pretty clear that the columns can't span all of R
6
, since there are only 5 of them.
Example 2:
If there are 6 unknowns and 2 equations, what can the dimension of the null space be?
The unknown is a vector in R
6
. Suppose we had no equations, or all our equtions are just zeros, and don't put any condition on the vector:
0
x
1
+
0
x
2
+
0
x
0
+
0
x
0
+
0
x
0
=
0
Then in this case every vector is a solution, so the nullity is 6, and the rank of the matrix is 0.
If there is just one equation, and it's not zero, then the one equation will give a matrix that has one dimensional column space, since the columns are only one vector high. And there will be 4 free variables, and x
1
can be written in terms of them, eg, for
1
x
1
-
2
x
2
+
3
x
3
+
2
x
+
0
x
5
=
0
If there are two equations, and they are linearly independent, then we've added another dimension to the rank, since the columns now have length two, so the rank should be 2, but the extra condition means the space of solutions should go down one, and there will only be three free variable, and the nullity will be three, eg:
1
x
1
+
0
x
2
+
3
x
3
+
2
x
4
+
0
x
5
=
0
0
x
1
+
1
x
2
+
0
x
3
+
7
x
0
+
8
x
5
=
0
So for 6 unknowns and two equations we can have the following possibilities:
rank
nullity
number of columns
comments
0
6
6
no conditions, so all vectors satisfy conditions.
1
5
6
one condition and 5 free variable
2
4
6
two independent equations cuts us down to 4 free variables