Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Week 2. Vectors, linear combinations, spans,

matrix-vector product. Free variables and spans.

Math 4A – originals by Scharlemann, some changes by Labutin

in 2021, 2023

1/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

An m-vector [column vector, vector in Rm ] is an m × 1 matrix:

 
a1
 a2 
 
~a =  a3 
 
 .. 
 . 
am
Can add two m-vectors in the obvious way, or multiply a vector by
a real number:
         
a1 b1 a1 + b 1 a1 ca1
 a2   b2   a2 + b2   a2   ca2 
         
~a+~b =  a3 + b3  =  a3 + b3  ; c~a = c  a3  =  ca3 
         
 ..   ..   ..   ..   .. 
 .   .   .   .   . 
am bm am + bm am cam

Do not multiply two vectors together like this.

2/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Examples:
         
1 4 5 1 −1
2 + 5 = 7 ; −1 · 2 = −2
  
3 6 9 3 −3

     
1 0 1
3 · 2 − 7 · 2 + 2 · −1 =
    
3 4 0
   
3·1−7·0+2·1 5
3 · 2 − 7 · 2 + 2 · (−1) = −10
3·3−7·4+2·0 −19

3/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

       
1 0 1 5
7 · 2 − 3 · 2 − 2 · −1 = 10
3 4 0 9

Next, picturing vectors and geometric intuition.

4/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Recall the real number line

0 a b

with addition (or subtraction):

0 a b a+b

(but it matters which side of 0 the numbers start out)

a 0 a+b b

and multiplication (or division):

0 a 5a

You can do this without knowing the actual values of a and b!

5/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

You can combine addition and multiplication:

0 a b
-3a 0 a b-3a b

Easier to describe if you think of numbers as arrows from 0:

0 a b

0 a b

-3a 0 a b

-3a 0 a b-3a b

Addition: Put the head of one arrow at tail of the other.

Multiplication: Scale the length &, if negative, reverse direction
6/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Use the same idea for 2- and 3-vectors:

think of vector as arrow from 0
multiply number with vector via scaling.
add vectors head-to-tail;
x
2

x1

7/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Use the same idea for 2- and 3-vectors:

think of vector as arrow from 0
multiply number with vector via scaling.
add vectors head-to-tail;
x
2

x1

8/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Use the same idea for 2- and 3-vectors:

think of vector as arrow from 0
multiply number with vector via scaling.
add vectors head-to-tail;
x
2
u+v

x1

9/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Use the same idea for 2- and 3-vectors:

think of vector as arrow from 0
multiply number with vector via scaling.
add vectors head-to-tail: parallelogram rule;
x
2
u+v

x1

10/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Application: Gives more flexible way to describe a line.

~ use
For a line through a point p, in direction d,
~
~x = p~ + t · d, t∈R

Argument:
x
2

d
x1

11/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Application: Gives more flexible way to describe a line.

~ use
For a line through a point p, in direction d,
~
~x = p~ + 1 · d, t=1

Argument:
x
2
p+d
p

d
x1

12/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Application: Gives more flexible way to describe a line.

~ use
For a line through a point p, in direction d,
~
~x = p~ + −1 · d, t = −1

Argument:
x
2

p-d

d
x1

13/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Application: Gives more flexible way to describe a line.

~ use
For a line through a point p, in direction d,
~
~x = p~ + t · d, t∈R

Argument:
x
2

p + td
p

d
x1

14/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
Pictured are points u, v ∈ R2 .
Which point represents u~ − 3~v ?
x
2

x1
v

15/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
Pictured are points u, v ∈ R2 .
Which point represents u~ − 3~v ?
x
2 D
C
E

B
u
A

x1
v

16/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
Pictured are points u, v ∈ R2 .
Which point represents u~ − 3~v ?
x
2

x1
v

17/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
Pictured are points u, v ∈ R2 .
Which point represents u~ − 3~v ?
x
2

-3v

-v u

x1
v

18/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

~ , ~x , ~y ∈ Rn and s, t ∈ R, we have the following.

For all w
~x + ~y = ~y + ~x (commutative)
(~x + ~y ) + w
~ = ~x + (~y + w
~) (associative)
~z + ~0 = ~0 + ~z = ~z
~x + (−~x ) = −~x + ~x = ~0
t(~x + ~y ) = t~x + t~y (distributive law)
(s + t)~x = s~x + t~x (distributive law)
s(t~x ) = (st)~x (associative)
1~x = ~x

19/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Definition
Suppose {t1 , t2 . . . tk } are all real numbers.
The vector
~y = t1 ~v1 + · · · + tk ~vk
is called a linear combination of the vectors {~v1 , ~v2 . . . ~vk }.

Sample problem:
Given vectors {~a1 , ~a2 . . . ~an , ~b} in Rm , find real numbers
{t1 , t2 . . . tn } so that

t1 ~a1 + · · · + tn ~an = ~b.

Off hand, could have any number of {t1 , t2 . . . tn } solutions.

20/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

How to think about solving for {t1 , t2 . . . tn } in the equation

t1 ~a1 + · · · + tn ~an = ~b :

Let    
b1 a1j
 b2   a2j 
   
~b = 
 b3  ;

~aj =  a3j  for 1 ≤ j ≤ n,
 
 ..   .. 
 .   . 
bm amj
notice j is second index. Then for any 1 ≤ i ≤ m, the i th row of
the equation becomes:

t1 ai1 + t2 ai2 + · · · + tn ain = bi or

ai1 t1 + ai2 t2 + · · · + ain tn = bi

21/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

In other words, solving

t1 ~a1 + · · · + tn ~an = ~b :

is the same as solving this system of m linear equations:

a11 t1 + . . . + a1n tn = b1
a21 t1 + . . . + a2n tn = b2
..
.
am1 t1 + . . . + amn tn = bm

We just learned how to do this! (Wrote x-s instead of t-s.)

22/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Solving
t1 ~a1 + · · · + tn ~an = ~b :
is the same as solving a system of m linear equations.
The system has augmented matrix (watch the rule for writing i
and j in aij )  
a11 a12 . . . a1n b1
 a21 a22 . . . a2n b2 
 
 .. .. .. .. .. 
 . . . . . 
am1 am2 . . . amn bm
Since each ~aj is a column of i numbers, can just write
h i
~a1 ~a2 . . . ~an ~b

23/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example (previous week):

Suppose
       
0 0 6 0
2 2 −1 6
4 ~a2 = 4 ~a3 =  1  ~a4 = 10
~a1 =        

8 8 −1 26

(~a1 = ~a2 , not a problem) and want to find c1 , c2 , c3 , c4 so that

 
12
4
c1 ~a1 + c2 ~a2 + c3 ~a3 + c4 ~a4 =  
13
23

24/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

This translates to the system of linear equations whose augmented

matrix is  
0 0 6 0 12
 2 2 −1 6 4 
 
 4 4 1 10 13 
8 8 −1 26 23

which we saw last time row reduces to:

0 32
 
1 1 0
 0 0 1 0 2 
 
 0 0 0 1 12 
0 0 0 0 0

and so has general solution

25/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

3
c1 = − c2 , c2 free
2
c3 = 2
1
c4 = ,
2

or equivalently
3 1
c2 = anything , c1 = − c2 , c3 = 2, c4 =
2 2

26/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Definition
Given a collection {~v1 , ~v2 , . . . , ~vk } of vectors in Rm , the set of all
linear combinations of these vectors, that is all vectors that can be
written as
c1 ~v1 + · · · + ck ~vk
for some c1 , . . . , ck ∈ R is denoted

Span {~v1 , . . . , ~vk }

and is called the span of {~v1 , . . . , ~vk }.

Easy example: If k = 1 so there is only one vector ~v , then

Span{~v } is just all vectors that are multiples of ~v . That is,
Span{~v } = {c~v | c ∈ R}

27/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Picturing the span when m = 2, 3:

When there is only one vector ~v then Span{~v } = {c~v | c ∈ R}
is just the line that contains both ~0 (take c = 0) and ~v (take c = 1).

u , ~v } = {c1 u~ + c2 ~v } pictured via

With two vectors u~ and ~v , Span{~
the parallelogram rule (Span = entire plane; ci ≥ 0 highlighted):

(3u+5v)/2
u
v

28/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Important POV change: solution to a system of linear equations

can be written as a span.

29/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Repeat of example above: The matrix and reduced echelon form:

1 1 0 0 23
   
0 0 6 0 12
2 2 −1 6 4   0 0 1 0 2 
4 4 1 10 13 → 
   
0 0 0 1 12 
8 8 −1 26 23 0 0 0 0 0
has general solution:
3 1
c2 = anything , c1 = − c2 , c3 = 2, c4 =
2 2

We can write this as

  3  3   3  
c1 2 − c2 2 −1 2 −1
c2 
  =  c2  =
  0 1 0 1
  + c2   =  +t 
c3   2  2 0 2 0
1 1 1
c4 2 2 0 2 0

30/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

So we can think of the set of all solutions as

3  
2 
 −1 
0  
  + Span  1 

2   0 
 
1
0
 
2

So we can picture the solution as a line in the direction of the

second vector, going through the point given by the first vector
(but in R4 !)
x
2

p + td
p

d
x1
31/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Another from last time:

Example with two free variables {x2 , x4 }

3
0 23 − x2 − πx4
 
1 1 0 π x1 =
 0 2
0 1 e 0 2 
 ⇒ x3 = 2 − ex4
 0 0 0 0 1 12 
0 0 0 0 0 0 1
x5 =
2
which can be written:
  3  3    
x1 2 − x2 − πx4 2 −1 −π
x2   x2  0 1  0 
         
x3  =  2 − ex4  =  2  + x2  0  + x4  −e  =
         
x4   x4  0 0  1 
1 1
x5 2 2 0 0

32/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

3    
2 
 −1 −π  
0    0 

 1   

  
=  2  + Span  0  ,  −e  ,
     
0 

 0   1  

 
1
0 0
 
2
Intuition from physical space – span of two vectors is a plane
through ~0.  
3/2
 0 
 
Thus all solutions are ”plane shifted by fixed p~ ”, p~ = 
 2 .

 0 
1/2

33/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

We shall further investigate connections between systems of

equations, linear combinations, and spans.

34/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Definition
The linear combination

x1 ~a1 + · · · + xk ~ak

is abbreviated  
x1
x2 
 
. . . ~ak x3  .

~a1 ~a2 ~a3
 
 .. 
.
xk

Here ~a1 ~a2 ~a3 . . . ~ak is the matrix A with i th column ~ai .
 

35/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Now view backwards. The rule to memorize for computing

 
x1
x2 
 
x3 

~a1 ~a2 ~a3 . . . ~ak   = x1 ~a1 + · · · + xk ~ak
 .. 
.
xk

is column-times-coordinate: multiply j-th column ~aj of matrix A

by j-th coordinate xj of ~x and then sum over all j.

Can do that only if number of columns of A equals number of

coordinates of ~x . Both are k.

36/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Simplest example: each a~i ∈ R1 , i. e. each column in the matrix is

just a number:

 
b1
b2 

 
a1 a2 . . . ak  .  = column×coord = a1 b1 +a2 b2 +. . . ak bk .
 .. 
bk
ai ∈ R1 ) the column-times-coordinate is the same as
In this case (~
row-times-column (equals a number!).
 
7
  3
Example: 1 −2 3 −4  1 =


= 1 · 7 + (−2) · 3 + 3 · 1 + (−4) · 2 = 7 − 6 + 3 − 8 = −4 ∈ R1 .
37/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Swap 2 and 1 to get a different vector

 
3
  7 1
1 −2 3 −4  2 = row × column = −9 ∈ R .


38/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

More complicated example: For a~i ∈ R2 , i = 1, 2, 3:

 
  2      
2 3 −1   2 3 −1
1 = clmn × crdt = 2 +1 +4
4 −2 5 4 −2 5
4
so
 
  2    
2 3 −1   2·2+3·1−1·4 3
1 = = ∈ R2 .
4 −2 5 4·2−2·1+5·4 26
4

Think of doing the simplest case on each row of the matrix A by

row-times-column rule:

39/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Apply the simplest case to first row:

 
  2    
2 3 −1   2·2+3·1−1·4 3
1 = =
4 −2 5 4·2−2·1+5·4 26
4

Apply simplest case to second row:

 
  2    
2 3 −1   2·2+3·1−1·4 3
1 = = ∈ R2 .
4 −2 5 4·2−2·1+5·4 26
4

40/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Further abbreviation:
Use vector notation:  
x1
x2 
 
x3 
  = ~x
 .. 
.
xk
then
 
x1
x2 
 
. . . ~ak x3  = A~x

x1 ~a1 + · · · + xk ~ak = ~a1 ~a2 ~a3
 
 .. 
.
xk

This is the definition of A~x , the matrix-vector product.

41/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Summary: For A an m × n matrix, and a vector ~x ∈ Rn ,

multiplication A~x is defined and gives a vector in Rm .

Multiplication has two important properties:

For any vectors u~, ~v ∈ Rn , A(~
u + ~v ) = A~ u + A~v
~ n
For any vector u ∈ R and any c ∈ R, A(c u~) = c(A~ u ).
Just apply column×coordinate rule. For example:
   
u1 v1
u2  v2 
     
u3  v3 
A(~ ~ ~ ~ ~ ~
u + v ) = a1 a2 a3 . . . an (  +  ) =
 ..   .. 
. .
un vn

(u1 +v1 )~a1 +· · ·+(un +vn )~an = (u1 ~a1 +· · ·+un ~an )+(v1 ~a1 +· · ·+vn ~an ) =
A~
u + A~v
42/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
 
  3
1 −2 3 −4 
7 =?

−2 1 −4 3 2
1
The matrix is 2 × 4 and the vector is in R4 , so may multiply,
result is in R2 .
Either by column×coordinate rule (add 4 R2 -vectors)
       
1 −2 3 −4
3+ 7+ 2+ 1=
−2 1 −4 3

or by row×column rule (twice, for 2 rows)

   
1 · 3 + (−2) · 7 + 3 · 2 + (−4) · 1 −9
=
(−2) · 3 + 1 · 7 + (−4) · 2 + 3 · 1 −4

43/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Sample problem from before:

Given vectors {~a1 , ~a2 . . . ~an } and ~b in Rm , find real numbers
{x1 , x2 . . . xn } so that

x1 ~a1 + · · · + xn ~an = ~b.

Equivalently, is ~b in Span{~a1 , ~a2 . . . ~an }?

It is equivalent to solving system of m equations

ai1 x1 + · · · + ain xn = bi , i = 1, . . . , m

New: equivalent translation to matrix equation:

Given m × n matrix A and ~b ∈ Rm find a vector ~x ∈ Rn so that

A~x = ~b
 
where A = ~a1 ~a2 ~a3 . . . ~an .
44/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Note: the Matrix-vector multiplication A~x makes sense only if the

number of columns in A matches the number of entries in x.

45/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

One upshot: If A is an m × n matrix, and ~b ∈ Rm then we already

know how to find solutions (if any) to the matrix equation
A~x = ~b :
h i
Write the augmented matrix A ~b =
 
a11 a12 . . . a1n b1
 a21 a22 . . . a2n b2 
 
 .. .. .. .. .. 
 . . . . . 
am1 am2 . . . amn bm
Reduce to echelon form:
 
$ ∗ ∗ ∗ ∗ ∗
0
 $ ∗ ∗ ∗ ∗
0
 0 0 $ ∗ ∗
0 0 0 0 0 0
0 0 0 0 0 0
46/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Clear out pivot columns to get reduced echelon form:

 
1 0 ∗ 0 ∗ ∗
0
 1 ∗ 0 ∗ ∗ 
0
 0 0 1 ∗ ∗ 
0 0 0 0 0 0
0 0 0 0 0 0

Now solve.
Solutions, if any, may be described either as a formula with
free variables, if any, or in the form

~x ∈ p~ + Span{~v1 , . . . ~vk }.

47/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Another phrasing:
~b ∈ Span{~a1 , ~a2 . . . ~an }

if and only if the matrix equation A~x = ~b has a solution.

In particular the collection of vectors {~a1 , ~a2 . . . ~an } span all of Rm

if and only if for every ~b ∈ Rm there is a vector ~x ∈ Rn so that
A~x = ~b.

48/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
     
6 2 3 −1
Is in Span{ }?
−5 4 −2 5

Translation: For
 
  x1
2 3 −1
A= ; ~x = x2 
4 −2 5
x3
 
6
is there a solution ~x to the matrix equation A~x = ?
−5

49/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Answer:
   
2 3 −1 6 2 3 −1 6
→
4 −2 5 −5 0 −8 7 −17
This is echelon form. Proceed to reduced echelon form:
2 0 21−8 48−51
   
2 3 −1 6 8 8
→ =
0 −8 7 −17 0 −8 7 −17
13 −3
 
2 0 8 8
0 −8 7 −17
There is one free variable x3 so there is a line of solutions.
(Note: this will be true for any ~b, so the 3 vectors span all of R2 .)
Let’s find a solution!

50/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

13 −3
 
2 0 8 8
0 −8 7 −17
If set free variable x3 = 0 get 2x1 = −3/8, −8x2 = −17.
 
−3/16  
6
So ~x = 17/8 should be a solution to A~x =
  .
−5
0

51/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Another solution: Set free variable x3 = 1. Then

−3
2 0 13
 
8 8
0 −8 7 −17

gives 2x1 + 13/8 = −3/8 (first row) and −8x2 + 7 = −17 ⇒

2x1 = −16/8
  (first row) and −8x2 = −24 ⇒ x1 = −1, x2 = 3
−1
so ~x =  3  is another solution.
1

52/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Exercise: General solution is

   13 
−1 − 16
~x =  3  + t  78  t ∈ R.
1 1
Why?

53/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

We have learned so far that system of linear equations is a bunch

of linear equations:

a11 x1 + a12 x2 + . . . + a1n xn = b1

a21 x1 + a22 x2 + . . . + a2n xn = b2
..
.
am1 x1 + am2 x2 + . . . + amn xn = bm

There may be lots of solutions. Let us understand ”how many”

exactly.

Define S to be the set of all possible solutions to the system.

Have seen that S is either empty, or a single point, or infinitely
many points (with some extra structure)

54/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

New way of thinking about the problem:

A solution is a vector ~x ∈ Rn that satisfies the matrix equation

A~x = ~b

where  
  b1
a11 a12 . . . a1n  b2 
 a21 a22 . . . a2n   
A= .

.. .. ..  ;
 ~b = 
 b3 

 .. . . .   .. 
 . 
am1 am2 . . . amn
bm

55/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

We know that S can be found by reducing the augmented matrix

 
a11 a12 . . . a1n b1
 a21 a22 . . . a2n b2 
 
 .. .. .. .. .. 
 . . . . . 
am1 am2 . . . amn bm

to reduced echelon form.

Then back-solve for the xi .

56/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

When ~b = ~0 it is called a homogeneous system of linear equations.

That is, in a homogeneous system, we are asked to find ~x so that

A~x = ~0.

This means a system in which each bi = 0, so we have

a11 x1 + a12 x2 + . . . + a1n xn = 0

a21 x1 + a22 x2 + . . . + a2n xn = 0
..
.
am1 x1 + am2 x2 + . . . + amn xn = 0

A homogeneous system always has solution

x1 = x2 = . . . = xn = 0 This is called the trivial solution. The
interesting question: are there nontrivial solutions? That is,
solutions ~x 6= ~0.
57/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

We focus on the homogeneous system A~x = ~0 for now.

58/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Theorem
A homogeneous system has nontrivial solutions if and only if there
is at least one free variable.
Reason:
If no free variables, then in reduced echelon form the augmented
matrix is:  
1 0 ... 0 0
 0 1 ... 0 0 
 
 .. .. . . .. .. 
 .
 . . . . 
 0 0 ... 1 0 
 
 0 0 ... 0 0 
 
 .. .. . . .. .. 
 . . . . . 
0 0 ... 0 0
Only solution is x1 = x2 = . . . = xn = 0.
59/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

On the other hand, if there are free variables, can set them to
anything and back-substitute a solution:

For example, the reduced echelon form

 
1 0 0.35 0
 0 1 0 0 
0 0 0 0

has x3 as a free variable.

The matrix comes from the pair of equations:

x1 + 0x2 + 0.35x3 = 0
0x1 + 1x2 + 0x3 = 0

Forces x2 = 0 but x3 can be anything: just take x1 = −0.35x3 .

For example: x3 = 1, x1 = −0.35 satisfies both equations.
60/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example: for the system

 
1 −2 3 −4
A~x = ~0 where A = 3 2 −2 5 
4 0 1 7

is the solution set S trivial or not?

Answer: at once without computations, the unknown is ~x ∈ R4 ,

have 3 rows (3 equations), echelon form must have at least one
free variable. Hence S is non-trivial.
General fact: if A is m × n matrix with m < n then echelon form of
A has free variables and A~x = ~0 has non-trivial solutions.
If m ≥ n then cannot say anything! Both cases are possible. Need
to solve for given A.
61/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
For the system
 
1 −2 3
A~x = ~0 where A = 3 2 −2
4 0 1

we are told that the last row is the sum of the first two rows. Is
the solution set S trivial or not? Here ~x ∈ R3 .

Answer: we can replace the last row by [0 0 0] (why?). Then the

echelon form must have at least one free variable. Hence S is
non-trivial.

62/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example: for the system

 
1 3 4
A~x = ~0 where A = −2 2 0
3 −2 1
we are told that the last column is the sum of the first two
columns. Is the solution set S trivial or not?

Answer: write the matrix as A = [~a1 ~a2 ~a3 ], then we are told that
~a3 = ~a1 + ~a2 .
This means
 
1
~a1 + ~a2 + (−1)~a3 = ~0 ⇒ A  1  = ~0.
−1
Hence S is nontrivial.
If S is nontrivial, then it is infinite. Why?
63/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

General fact:
If reduced echelon form has k free variables,
then there are k vectors in Rn which exactly span the set of solutions:

So, if exactly one free variable then S is line through ~0 in Rn .

If exactly two free variables, then S is plane through ~0 in Rn .

We saw that when we wrote solution with free variables as a span!

This gives a picture of all solutions of a homogeneous system.

64/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

What about a non-homogeneous system?

That is, suppose ~b 6= ~0 in the system A~x = ~b.

65/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

First observation:
If the last column of echelon form of augmented matrix is a pivot,
then there is no solution. We say the system of equations is
inconsistent.
 
$ ∗ ∗ ∗ ∗ ∗
 0 $ ∗ ∗ ∗ ∗ 
 
 0 0 0 $ ∗ ∗ 
 
 0 0 0 0 0 $ 
0 0 0 0 0 0
$ = non-zero entry (so a pivot)
The last non-zero row says 0x1 + 0x2 + · · · + 0xn = $. Since $ 6= 0
this is impossible.

66/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

On the other hand, if the last column of echelon form of

augmented matrix is not a pivot, then can back-fill solution. We
say the system of equations is consistent.
To find a solution, move to reduced echelon form
 
1 0 ∗ 0 ∗
 0 1 ∗ 0 ∗ 
 
 0 0 0 1 ∗ 
 
 0 0 0 0 0 
0 0 0 0 0

If no free variables, then solution is unique, given by the entries in

the last (augmentation) column.
If there are free variables, how can we describe S?

67/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

In this case observe:

If all free variables are set to 0 then equations easily give one
solution s~0 :
Example: Supposed the reduced echelon form looks like
1 0 ∗ 0 ∗ π
 
 0 1 ∗ 0 ∗ e 
 √ 
 0 0 0 1 ∗ 2 
 
 0 0 0 0 0 0 
0 0 0 0 0 0
Set the free variables x3 = x5 = 0. Then the equations become

x1 + 0x2 + ∗0 + 0x4 + ∗0 = π
0x1 + x2 + ∗0 + 0x4 + ∗0 = e
√
0x1 + 0x2 + ∗0 + x4 + ∗0 = 2
√
Clearly x1 = π, x2 = e, x4 = 2 is a solution.
68/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

So far, we’ve found how to determine one solution s~0 . That is,

s0 = ~b
A~

How to find all solutions?

Suppose s~0 is another solution, so also As~0 = ~b.
Then
A(s~0 − s~0 ) = As~0 − A~
s0 = ~b − ~b = ~0
In other words, the vector (s~0 − s~0 ) = ~s is solution to A~x = ~0.
Equivalently s~0 = s~0 + ~s .
Hence the solution set S for the system A~x = b is

S = s~0 + Span{s1 , ..., sk }

where Span{s1 , ..., sk } is solution to homogeneous system A~x = ~0.

69/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

The picture in Rn of possible solutions to A~x = ~0 and A~x = ~b:

Ax = b

s0
Ax = 0

70/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example (understand it for the exam):

find all solutions of non-homogeneous system

x1 + 2x2 − 3x3 + 2x4 − 4x5 = 1

2x1 + 4x2 − 5x3 + 1x4 − 6x5 = 3
5x1 + 10x2 − 13x3 + 4x4 − 16x5 = 7

Translation A~x = ~b

homogeneous system:

x1 + 2x2 − 3x3 + 2x4 − 4x5 = 0

2x1 + 4x2 − 5x3 + 1x4 − 6x5 = 0
5x1 + 10x2 − 13x3 + 4x4 − 16x5 = 0

Translation A~x = ~0
71/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Associated augmented matrix:

 
1 2 −3 2 −4 1
 2 4 −5 1 −6 3 
5 10 −13 4 −16 7

Replace L2 by −2L1 + L2 and L3 by −5L1 + L3 :

 
1 2 −3 2 −4 1
 0 0 1 −3 2 1 
0 0 2 −6 4 2

Replace L3 by −2L2 + L3 :
 
1 2 −3 2 −4 1
 0 0 1 −3 2 1 
0 0 0 0 0 0

Consistent! so there is a solution ©.

72/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

 
1 2 −3 2 −4 1
 0 0 1 −3 2 1 
0 0 0 0 0 0
To find solution need reduced echelon. Replace L1 with L1 + 3L3
 
1 2 0 −7 2 4
 0 0 1 −3 2 1 
0 0 0 0 0 0
Free variables: x2 , x4 , x5 . Set them to 0 and get:
x1 = 4, x3 = 1. Hence one solution is
 
4
 0 
 
s~0 = 
 1 .

 0 
0
73/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

For solution of the homogeneous system

 
1 2 0 −7 2 0
 0 0 1 −3 2 0 
0 0 0 0 0 0

translates to
         
x1 −2x2 + 7x4 − 2x5 −2 7 −2
x2   x2  1 0 0
         
x3  = 
   3x4 − 2x5  = x2  0  + x4 3 + x5 −2
      
x4   x4  0 1 0
x5 x5 0 0 1

74/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Hence all solutions to the homogeneous system given by:

     

 −2 7 −2 
     0 
1 0

      


Span  0  , 3 , −2
     
0 1 0 
     
 

 
0 0 1
 

Picture:

Span{u,v}
u

75/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Answer: all solutions to the original non-homogeneous system

given by:
       
4 
 −2 7 −2 
0  1  0  0 

      

  
1 + Span  0  , 3 , −2
~x =        
0 
  0  1  0  
 
0 0 0 1
 

Picture:

s0+Span{u,v}
u
s0
v

76/76