Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Week 2. Vectors, linear combinations, spans,

matrix-vector product. Free variables and spans.

Math 4A – originals by Scharlemann, some changes by Labutin

in 2021, 2023

1/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

An m-vector [column vector, vector in Rm ] is an m ⇥ 1 matrix:

2 3
a1
6 a2 7
6 7
6 7
~a = 6 a3 7
6 .. 7
4 . 5
am
Can add two m-vectors in the obvious way, or multiply a vector by
a real number:
2 3 2 3 2 3 2 3 2 3
a1 b1 a1 + b1 a1 ca1
6 a2 7 6 b 2 7 6 a2 + b 2 7 6 a2 7 6 ca2 7
6 7 6 7 6 7 6 7 6 7
6 7 6 7 6 7 6 7 6 7
~a+~b = 6 a3 7+6 b3 7 = 6 a3 + b3 7 ; c~a = c 6 a3 7 = 6 ca3 7
6 .. 7 6 .. 7 6 .. 7 6 .. 7 6 .. 7
4 . 5 4 . 5 4 . 5 4 . 5 4 . 5
am bm am + b m am cam

Do not multiply two vectors together like this.

2/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Examples:
2 3 2 3 2 3 2 3 2 3
1 4 5 1 1
425 + 455 = 475 ; 1 · 4 25 = 4 25
3 6 9 3 3

2 3 2 3 2 3
1 0 1
3 · 425 7 · 425 + 2 · 4 15 =
3 4 0
2 3 2 3
3·1 7·0+2·1 5
43 · 2 7 · 2 + 2 · ( 1)5 = 4 105
3·3 7·4+2·0 19

3/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

2 3 2 3 2 3 2 3
1 0 1 5
7 · 4 25 3 · 4 25 2 · 4 15 = 4105
3 4 0 9

Next, picturing vectors and geometric intuition.

4/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Use the same idea for 2- and 3-vectors:

think of vector as arrow from 0
multiply number with vector via scaling.
add vectors head-to-tail;
x
2

Mt
24
u

x1

7/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Use the same idea for 2- and 3-vectors:

think of vector as arrow from 0
multiply number with vector via scaling.
add vectors head-to-tail;
x
2

v
1

x1

8/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Use the same idea for 2- and 3-vectors:

think of vector as arrow from 0
multiply number with vector via scaling.
add vectors head-to-tail;
x
2
u+v

x1

9/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Use the same idea for 2- and 3-vectors:

think of vector as arrow from 0
multiply number with vector via scaling.
add vectors head-to-tail: parallelogram rule;
x
2
u+v

x1

10/76
 V

v
Yu
t In p

U T

a
T T
T
o te

tan s
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Application: Gives more flexible way to describe a line.

~ use
For a line through a point p, in direction d,
~
~x = p~ + t · d, t2R

Argument:
x
2

d
x1

11/76
LINE alongthe
origin in the direction of I

t I t ER L

I
Lie in the direction of I passing along E

L t I p t e 112
 d THE AWE
EVAN's w
passing through
a R E

ti
o
o
t
L
É
L t E E T FER Y
th ut tu te IR

t E T to te IR

t CE E t T ter
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Application: Gives more flexible way to describe a line.

~ use
For a line through a point p, in direction d,
~
~x = p~ + 1 · d, t=1

Argument:
x
2
p+d
p

d
x1

12/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Application: Gives more flexible way to describe a line.

~ use
For a line through a point p, in direction d,

~x = p~ + ~
1 · d, t= 1

Argument:
x
2

p-d

d
x1

13/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Application: Gives more flexible way to describe a line.

~ use
For a line through a point p, in direction d,
~
~x = p~ + t · d, t2R

Argument:
x
2

p + td
p

d
x1

14/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
Pictured are points u, v 2 R2 .
Which point represents u~ 3~v ?
x
2

x1
v

15/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
Pictured are points u, v 2 R2 .
Which point represents u~ 3~v ?
x
2 D
C
E
3W
B
u
A

x1
v

16/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
Pictured are points u, v 2 R2 .
Which point represents u~ 3~v ?
x
2

x1
v

17/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
Pictured are points u, v 2 R2 .
Which point represents u~ 3~v ?
x
2

-3v

-v u

x1
v

18/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

~ , ~x , ~y 2 Rn and s, t 2 R, we have the following.

For all w
~x + ~y = ~y + ~x (commutative)
(~x + ~y ) + w
~ = ~x + (~y + w
~) (associative)
~z + ~0 = ~0 + ~z = ~z
~x + ( ~x ) = ~x + ~x = ~0
t(~x + ~y ) = t~x + t~y (distributive law)
(s + t)~x = s~x + t~x (distributive law)
s(t~x ) = (st)~x (associative)
1~x = ~x

19/76
 PLANE IN RB Along THE origin Containing IT
All WEAR comb INDIOS E J
of
112

Xz

O
X2

IT
X

at d
BE MEIR
plane containing E T T
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Definition
Suppose {t1 , t2 . . . tk } are all real numbers.
The vector
~y = t1 ~v1 + · · · + tk ~vk
is called a linear combination of the vectors {~v1 , ~v2 . . . ~vk }.

Sample problem:
Given vectors {~a1 , ~a2 . . . ~an , ~b} in Rm , find real numbers
{t1 , t2 . . . tn } so that

t1 ~a1 + · · · + tn ~an = ~b.

O↵ hand, could have any number of {t1 , t2 . . . tn } solutions.

20/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

How to think about solving for {t1 , t2 . . . tn } in the equation

t1 ~a1 + · · · + tn ~an = ~b :

Let 2 3 2 3
b1 a1j
6 b2 7 6 a2j 7
6 7 6 7
~b = 6 7
6 b3 7 ;
6 7
~aj = 6 a3j 7 for 1  j  n,
6 .. 7 6 .. 7
4 . 5 4 . 5
bm amj
notice j is second index. Then for any 1  i  m, the i th row of
the equation becomes:

t1 ai1 + t2 ai2 + · · · + tn ain = bi or

ai1 t1 + ai2 t2 + · · · + ain tn = bi

21/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

ai
In other words, solving

t1 ~a1 + · · · + tn ~an = ~b :
on
is the same as solving this system of m linear equations:

a11 t1 + . . . + a1n tn = b1
I
a21 t1 + . . . + a2n tn = b2
..
.
am1 t1 + . . . + amn tn = bm

We just learned how to do this! (Wrote x-s instead of t-s.)

22/76
Can
3 be watten as a linear combination

91 9
i www.fndtitit

t.lk tty tts1 l

t t t 3 1
L ta t t 2
ti ta T3 3
Does This system have solution
l 0 1 I

1
on
I 1
o
3 1 0 1 I 2

I 0 1 I
o l 2 IT Has sown w
O o 1 O
t I 2 2 3 0

0 0 1
I 0
Doesthe vector can be written as

a linear arbitration

Cannae t ta ta at
find
til
t 9
t.li
IIE t
z
O 1 1 2
ta

0 I 1 2 0 0 0
p
2

T
no solution prot
Ng
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Solving
t1 ~a1 + · · · + tn ~an = ~b :
is the same as solving a system of m linear equations.
The system has augmented matrix (watch the rule for writing i
and j in aij ) 2 at at ai I3
a11 a12 . . . a1n b1
6 a21 a22 . . . a2n b2 7
6 7
6 .. .. .. .
. .
. 7
4 . . . . . 5
am1 am2 . . . amn bm
Since each ~aj is a column of i numbers, can just write
h i
~a1 ~a2 . . . ~an ~b

23/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example (previous week):

Suppose
2 3 2 3 2 3 2 3
0 0 6 0
6 27 6 27 6 17 667
~a1 = 4 5 ~a2 = 4 5 ~a3 = 4 5 ~a4 = 6
6 7 6 7 6 7
4
7
4 4 1 105
8 8 1 26

(~a1 = ~a2 , not a problem) and want to find c1 , c2 , c3 , c4 so that

2 3
12
647
c1 ~a1 + c2 ~a2 + c3 ~a3 + c4 ~a4 = 6
4135
7 L
23

24/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

This translates to the system of linear equations whose augmented

matrix is 2 3
0 0 6 0 12
6 2 2 1 6 4 7
6 7
4 4 4 1 10 13 5
8 8 1 26 23

which we saw last time row reduces to:

2 3
1 1 0 0 32
6 0 0 1 0 2 7
6 7
4 0 0 0 1 21 5

0 0 0 0 0

and so has general solution

25/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

3
c1 = c2 , c2 free
2
c3 = 2
1
c4 = ,
2

or equivalently
3 1
c2 = anything , c1 = c2 , c3 = 2, c4 =
2 2

26/76
lit I it
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Definition
Given a collection {~v1 , ~v2 , . . . , ~vk } of vectors in Rm , the set of all
linear combinations of these vectors, that is all vectors that can be
written as
c1 ~v1 + · · · + ck ~vk
for some c1 , . . . , ck 2 R is denoted

Span {~v1 , . . . , ~vk }

and is called the span of {~v1 , . . . , ~vk }.

Easy example: If k = 1 so there is only one vector ~v , then

Span{~v } is just all vectors that are multiples of ~v . That is,
Span{~v } = {c~v | c 2 R}

27/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Picturing the span when m = 2, 3:

When there is only one vector ~v then Span{~v } = {c~v | c 2 R}
is just the line that contains both ~0 (take c = 0) and ~v (take c = 1).

u , ~v } = {c1 u~ + c2 ~v } pictured via

With two vectors u~ and ~v , Span{~
the parallelogram rule (Span = entire plane; ci 0 highlighted):

(3u+5v)/2
u
v

28/76
 I te IR

span I I a It a t 4 he 112

FEET we
alway a plane
only if it not a multiple of Fc
is
in different directions
They pout

in R
a
I

span 18 il
 af tb horizontal plane
x alongthe origin

o
d
span
1,181,1 o
spen 1871

i ii
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Important POV change: solution to a system of linear equations

can be written as a span.

29/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Repeat of example above: The matrix and reduced echelon form:

2 3 2 3 3
0 0 6 0 12 1 1 0 0 2
62 2 1 6 47 6 0 0 1 0 2 7
6 7!6 7
44 4 1 10 135 4 0 0 0 1 2 1 5

8 8 1 26 23 0 0 0 0 0
has general solution:
3 1
c2 = anything , c1 = c2 , c3 = 2, c4 =
2 2

We can write this as

2 3 23 3 233 3 2 233 3 2
c1 2 c 2 2 1 2 1
6 c2 7 6 7 607 6 7 607 6 7
6 7 = 6 c2 7 = 6 7 + c2 6 1 7 = 6 7+t6 1 7
4 c3 5 4 2 5 425 405 425 405
1 1 1
c4 2 2 0 2 0

30/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

So we can think of the set of all solutions as

233 82 39
>
> 1 >
2
607 <6 7> =
6 7 + Span 6 7 1
425 > 4 0 5>
>
: >
;
1
2 0
So we can picture the solution as a line in the direction of the
second vector, going through the point given by the first vector
(but in R4 !)
x
2

p + td
p

d
x1
31/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Another from last time:

Example with two free variables {x2 , x4 }

2 3 3
1 1 0 ⇡ 0 32 x1 = x2 ⇡x4
6 0 2
6 0 1 e 0 2 77
4 0 1 5) x3 = 2 ex4
0 0 0 1 2
0 0 0 0 0 0 1
x5 =
2
which can be written:
2 3 23 3 233 2 3 2 3
x1 2 x2 ⇡x4 2 1 ⇡
6 x2 7 6 x2 7 607 6 17 6 0 7
6 7 6 7 6 7 6 7 6 7
6x3 7 = 6 2 ex4 7 = 6 2 7 + x2 6 0 7 + x4 6
7 e7
6 7 6 7 6 7 6 6 7=
4 x4 5 4 x4 5 405 4 0 5 4 1 5
1 1
x5 2 2 0 0

32/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

233 82 3 2 39
2 >
> 1 ⇡ >>
607 >
> >
6 7 <66 17 6
7 6 0 7>
7 =
= 6 2 7 + Span 6
6 7
6 07 6
7,6 e7
7> ,
>
>
405 >
>
4 05 4 1 5>>
>
1 : ;
2 0 0
Intuition from physical space – span of two vectors is a plane
through ~0. 2 3
3/2
6 0 7
6 7
Thus all solutions are ”plane shifted by fixed p ”, p = 6 2 7
~ ~ 6
7.
4 0 5
1/2

33/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

We shall further investigate connections between systems of

equations, linear combinations, and spans.

34/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Definition
The linear combination

x1 ~a1 + · · · + xk ~ak

is abbreviated 2 3
x1
6 x2 7
⇥ ⇤6 7
6 x3 7
~a1 ~a2 ~a3 . . . ~ak 6 7 .
6 .. 7
4.5
xk
⇥ ⇤
Here ~a1 ~a2 ~a3 . . . ~ak is the matrix A with i th column ~ai .

35/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Now view backwards. The rule to memorize for computing

2 3
x1
6 x2 7
⇥ ⇤6 7
6 x3 7
~a1 ~a2 ~a3 . . . ~ak 6 7 = x1 ~a1 + · · · + xk ~ak
6 .. 7
4.5
xk

is column-times-coordinate: multiply j-th column ~aj of matrix A

by j-th coordinate xj of ~x and then sum over all j.

Can do that only if number of columns of A equals number of

coordinates of ~x . Both are k.

36/76
 ri
it 3 1
I 3 xp
is
1 bt

tu Je V3
LIE x

t Vi t ta ta t t V3

ti to
3 t
E

E
te t 2 tr
z te t ta t t
3 t t b Tu t It x
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Simplest example: each a~i 2 R1 , i. e. each column in the matrix is

just a number:

2 3
b1
⇥ ⇤6 7
6 b2 7
a1 a2 . . . ak 6 . 7 = column⇥coord = a1 b1 +a2 b2 +. . . ak bk .
4 .. 5
bk
ai 2 R1 ) the column-times-coordinate is the same as
In this case (~
row-times-column (equals a number!).
2 3
7
⇥ ⇤ 6 37
Example: 1 2 3 4 6 7
4 15 =
2

= 1 · 7 + ( 2) · 3 + 3 · 1 + ( 4) · 2 = 7 6+3 8= 4 2 R1 .
37/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Swap 2 and 1 to get a di↵erent vector

2 3
3
⇥ ⇤ 6 77
1 2 3 4 6 7
425 = row ⇥ column = 9 2 R1 .
1

38/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

More complicated example: For a~i 2 R2 , i = 1, 2, 3:

2 3
 2   
2 3 1 4 5 2 3 1
1 = clmn ⇥ crdt = 2 +1 +4
4 2 5 4 2 5
4
so
2
3 Bx e
2 3
 2  
2 3 1 4 5 2·2+3·1 1·4 3
1 = = 2 R2 .
4 2 5 4·2 2·1+5·4 26
4
2 1

Think of doing the simplest case on each row of the matrix A by

row-times-column rule:

39/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Apply the simplest case to first row:

2 3
 2  
2 3 1 4 5 2·2+3·1 1·4 3
1 = =
4 2 5 4·2 2·1+5·4 26
4

Apply simplest case to second row:

2 3
 2  
2 3 1 4 5 2·2+3·1 1·4 3
1 = = 2 R2 .
4 2 5 4·2 2·1+5·4 26
4

40/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Further abbreviation:
Use vector notation: 2 3
x1
6 x2 7
6 7
6 x3 7
6 7 = ~x
6 .. 7
4.5
xk
then
2 3
x1
6 x2 7
⇥ ⇤6 7
6 x3 7
x1 ~a1 + · · · + xk ~ak = ~a1 ~a2 ~a3 . . . ~ak 6 7 = A~x
6 .. 7
4.5
xk

This is the definition of A~x , the matrix-vector product.

41/76
 l
I
I 2
ix

1 4 27
1 3

3 1

2 I 4
o o 3 3

l l 21 41 61.2 2.0

1
l
exe
 yaw solve the sister
a 2 2 43 1
y y xz 0

X 2 2 13 1 PLANE NOTThoughtTHE Origin É NOT IN THE PLANE

E at Eg
I n

Pune span a if E
IEedombinations
7 1 and I

X t 43 0 PLANE Passing Along the origin 8

E at
E E a
i a1
spool g if
song 1
n n o x Xz
Xz 112
12 42
Solutions INFINITELY MANY
 I 1.1
x
4 11 1111 1
STRAIGHT UWE
X t 2 2 Xz 1
voice PARTICULAR
Y Solution
of
X Xz o

t 2 2 43 0
sowtiousy
X t Xz 0

HOMOGENEOUS SYSTEM ASSOCIATED TO hit 2 2 43 1

X Xy O

Qi Xi tr r r
t aint a 0

1
Aau X t Amal D

HOMOGENEOUS SYSTEM p
A X ya o Mes
Soutine
 CONSIDER THE LINEAR SYSTEM

X X 2 3 4 4 5
2x 2x 3 3 44 3 T 2 2 3 1 3

MATRIX
134 tsx ex za

REDUCED
1 I 0 10 INFINITELY Many
0 0 1 7 7
ECHELON Form o o o o socutious
T
Xi Ie Xz 10 4 9
Sovutious
9
FREE VAIABLES
fit B 7 4 7
4 X2 10 4
y

fI
m th txt
EI 7 4
Xu
7
g
span 1

I
PARTICULAR Sowtied
YIImogw.org PU SOLUTION
X Xz 2 3 44 0
0
2xit4z 3xst4
3 1 342 48 2 4 0
SYSTEM

ti th 10 4 0

1 00 I 7
g Xs 7 4 0

11 1
I 4
 i is
2 x 2 2 3 3 44 3

and span I i E y sowtions

7,2
3 hit by a 4 3 2 4 0

2 2 3 1
to
x t ta lo ta o

13 7 X 4 0 X3 7 Xa

IH Gi

g g

at
Review Let ai at an be n columns
fog
1 gig
VECTORS IN RM
Definition
if Fat ai trait than Term
WE SAY THAT T IS A LINEAR COMBINATION
of AT AT In
DEFINITION THE SET
ofALLPOSSIBLE WEAR COMBINATIONS Of T.az AI
is CAUED THE SPAN ATAT
of In
Span ai ai at t ait trait than ti their

EXAMPLES
1 LET G EIR ai to span at tan ter
IS THE LIVE N THE DIRECTION of a passingAlongTHE origin
I s span at

2 LET T T E IR I E T E NONCOUNEAN I
SPAN I I t It tar G te Elr
is the plane in R containing I T and E

3 Let ai ai 1 I 9 ER
SPAN ai ai as 1123

Let
a ai a as a
E e 1123

Span ai ai ai ai e123 Xz o PLANE

X

X
 LINES IN A IN THE Direction Y T passingThrough TEL
L
b titi ter
span is

WE Passing through E F EIR 7

y

L ten F tw ter E
to F ti ter

PLANES IN IR EI X 2 2 4 3 4 3

Pursue D X 12 2 4 3 4

4
L 21 taker
a y
to tts
g t ta BER

af tx
q ta taek

spell 413 1
Given at Infirm and given DEIR
Does Bespro ai.az In ie
It tz their sit t.at trait than I
at.to then at ta
t.fi tt.f
t

antitantzt aint ay
Az ti t anti t taza tu be

am t tamitat amntn bur

 a a a
92 Azz n Azn

I s iT in in

AE I
ANSWER I hiver combination of the columns
of A
A 5 has solution
Besponigg
Examples Does
3 belong to span un g veil
t
1 til S III Tz 2

I 9 E ten t.az

1 al I
For which values of a

1 belong other 1 f
th as txu t has solution
I 2
X t te t 3 3 Xa p
Xy Xy 2 0 0 1 1 2
X to 0
a 0 I I
t stay
6
p
o

i i
exists solution e andonly y a 1 In 0

Find THE GENERAL SOUND of X t Xr 3 3 1

2X Xz Xz 0

i s
o 37 a i i

if if
113 A b thx
df
1 33 3
4 X2 13 X2 43 713 3
3 33
33.4 43
É 73 3 2 a 3 t
Iz
Line

spool 1 Eg

Particular sowtion
g
SOLUTION
SPAN 433 E
ofTHE HOMOGENEOUSASSOCIATEDSYSTEM

Xz 3 3 0 t
B
2 1 X 4 3 0

2 t 7 3 3
p p B 20

212ps Tp 36 0

Spa Fit I
GENERAL FORM
of THE solution
 X 3 242 3 Xu t 2 5 2

3h 9 2 7 3 Xy t 3 5 7 3 9 7 I 3 7
z
2 Get x taxa sa

3 5 4 8 5

I
2
0 0 I 2 3 tr

p T T
1 l 2 4 3 5 t

FREE p no 5

teal o Ig

Tiedt Tespaw I f
f Brian sown

Ar
l
i 3 2 1
A 3 9 7 I 3 cheer it
2 6 7 4 5

H
A1
5

Afi
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Summary: For A an m ⇥ n matrix, and a vector ~x 2 Rn ,

multiplication A~x is defined and gives a vector in Rm .

Multiplication has two important properties:

Aman
For any vectors u~, ~v 2 Rn , A(~
u + ~v ) = A~ u + A~v
axe
For any vector u~ 2 Rn and any c 2 R, A(c u~) = c(A~ u ).
Just apply column⇥coordinate rule. For example:
2 3 2 3
u1 v1
6 u 2 7 6 v2 7
⇥ ⇤ 6 7 6 7
6 u 3 7 6 v3 7
A(~u + ~v ) = ~a1 ~a2 ~a3 . . . ~an (6 7 + 6 7) =
6 .. 7 6 .. 7
4.5 4.5
un vn

(u1 +v1 )~a1 +· · ·+(un +vn )~an = (u1 ~a1 +· · ·+un ~an )+(v1 ~a1 +· · ·+vn ~an ) =
A~
u + A~v
42/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Sample problem from before:

Given vectors {~a1 , ~a2 . . . ~an } and ~b in Rm , find real numbers
{x1 , x2 . . . xn } so that

x1 ~a1 + · · · + xn ~an = ~b.

Equivalently, is ~b in Span{~a1 , ~a2 . . . ~an }?

It is equivalent to solving system of m equations

ai1 x1 + · · · + ain xn = bi , i = 1, . . . , m

New: equivalent translation to matrix equation:

Given m ⇥ n matrix A and ~b 2 Rm find a vector ~x 2 Rn so that

A~x = ~b
⇥ ⇤
where A = ~a1 ~a2 ~a3 . . . ~an .
44/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Note: the Matrix-vector multiplication A~x makes sense only if the

number of columns in A matches the number of entries in x.

45/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

One upshot: If A is an m ⇥ n matrix, and ~b 2 Rm then we already

know how to find solutions (if any) to the matrix equation
A~x = ~b :
h i
Write the augmented matrix A ~b =
2 3
a11 a12 . . . a1n b1
6 a21 a22 . . . a2n b2 7
6 7
6 .. .. .. .. .. 7
4 . . . . . 5
am1 am2 . . . amn bm
Reduce to echelon form:
2 3
$ ⇤ ⇤ ⇤ ⇤ ⇤
60 $ ⇤ ⇤ ⇤ ⇤7
6 i 7
60 0 0 $ ⇤ n ⇤
7
6 7
40 0 0 0 0 I 05
l
0 0 0 0 0 0
46/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Clear out pivot columns to get reduced echelon form:

2 3
1 0 ⇤ 0 ⇤ ⇤
60 1 ⇤ 0 ⇤ ⇤7
6 7
60 0 0 1 ⇤ ⇤7
6 7
40 0 0 0 0 05
0 0 0 0 0 0

Now solve.
Solutions, if any, may be described either as a formula with
free variables, if any, or in the form

~x 2 p~ + Span{~v1 , . . . ~vk }.

47/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Another phrasing:
~b 2 Span{~a1 , ~a2 . . . ~an }

if and only if the matrix equation A~x = ~b has a solution.

In particular the collection of vectors {~a1 , ~a2 . . . ~an } span all of Rm

if and only if for every ~b 2 Rm there is a vector ~x 2 Rn so that
A~x = ~b.

T
48/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

E x E a 3 x
s
Example:
   
6 2 3 1
Is in Span{ }?
5 4 2 5

Translation: For
E 2 3
E
A
 x1
A=
2 3 1
; ~x = 4x2 5 T
4 2 5
x3

6
is there a solution ~x to the matrix equation A~x = ?
5

49/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Answer:
 
2 3 1 6 2 3 1 6
!
4 2 5 5 0 8 7 17
This is echelon form. Proceed to reduced echelon form:
 
2 3 1 6 2 0 218 8 48 8 51
! =
0 8 7 17 0 8 7 17
 13 3
2 0 8 8
0 8 7 17
There is one free variable x3 so there is a line of solutions.
(Note: this will be true for any ~b, so the 3 vectors span all of R2 .)
Let’s find a solution!

50/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

 13 3
2 0 8 8
0 8 7 17
If set free variable x3 = 0 get 2x1 = 3/8, 8x2 = 17.
2 3
3/16 
6
So ~x = 4 17/8 5 should be a solution to A~x = .
5
0

51/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Another solution: Set free variable x3 = 1. Then


2 0 13 8 8
3

0 8 7 17

gives 2x1 + 13/8 = 3/8 (first row) and 8x2 + 7 = 17 )

2x1 = 216/83 (first row) and 8x2 = 24 ) x1 = 1, x2 = 3

1
so ~x = 4 3 5 is another solution.
1

l o 3 346
16

0 1 17
718 8
52/76
X t
136 3 36
17
X2
I Xz 8

3 13
16 16 3

É no
3
xox
41 1
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Exercise: General solution is

2 3 2 13
3
1 16
~x = 4 3 5 + t 4 7
8
5 t 2 R.
1 1
Why?

53/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

We have learned so far that system of linear equations is a bunch

of linear equations:

a11 x1 + a12 x2 + . . . + a1n xn = b1

a21 x1 + a22 x2 + . . . + a2n xn = b2
..
.
am1 x1 + am2 x2 + . . . + amn xn = bm

There may be lots of solutions. Let us understand ”how many”

exactly.

Define S to be the set of all possible solutions to the system.

Have seen that S is either empty, or a single point, or infinitely
many points (with some extra structure)

54/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

New way of thinking about the problem:

A solution is a vector ~x 2 Rn that satisfies the matrix equation

A~x = ~b

where 2 3
2 3 b1
a11 a12 ... a1n 6 b2 7
6 a21 a22 ... a2n 7 6 7
6 7 ~b = 6 7
A=6 . .. .. .. 7 ; 6 b3 7
4 .. . . . 5 6 .. 7
4 . 5
am1 am2 . . . amn
bm

55/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

We know that S can be found by reducing the augmented matrix

2 3
a11 a12 . . . a1n b1
6 a21 a22 . . . a2n b2 7
6 7
6 .. .. .. .
.. .. 7
.
4 . . . 5
am1 am2 . . . amn bm

to reduced echelon form.

Then back-solve for the xi .

56/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

When ~b = ~0 it is called a homogeneous system of linear equations.

That is, in a homogeneous system, we are asked to find ~x so that

A~x = ~0.

This means a system in which each bi = 0, so we have

a11 x1 + a12 x2 + . . . + a1n xn = 0

a21 x1 + a22 x2 + . . . + a2n xn = 0
..
.
am1 x1 + am2 x2 + . . . + amn xn = 0

A homogeneous system always has solution

x1 = x2 = . . . = xn = 0 This is called the trivial solution. The
interesting question: are there nontrivial solutions? That is,
solutions ~x 6= ~0.
57/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

We focus on the homogeneous system A~x = ~0 for now.

58/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Theorem
A homogeneous system has nontrivial solutions if and only if there
is at least one free variable.
Reason:
If no free variables, then in reduced echelon form the augmented
matrix is: 2 3
1 0 ... 0 0
6 0 1 ... 0 0 7
6 7
6 .. .. . . .. .. 7
6 . . . . . 7
6 7
6 0 0 ... 1 0 7
6 7
6 0 0 ... 0 0 7
6 7
6 .. .. . . .. .. 7
4 . . . . . 5
0 0 ... 0 0
Only solution is x1 = x2 = . . . = xn = 0.
59/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

On the other hand, if there are free variables, can set them to
anything and back-substitute a solution:

For example, the reduced echelon form

2 3
1 0 0.35 0
4 0 1 0 0 5
0 0 0 0

has x3 as a free variable.

The matrix comes from the pair of equations:

x1 + 0x2 + 0.35x3 = 0
0x1 + 1x2 + 0x3 = 0

Forces x2 = 0 but x3 can be anything: just take x1 = 0.35x3 .

For example: x3 = 1, x1 = 0.35 satisfies both equations.
60/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example: for the system

2 3
1 2 3 4
A~x = ~0 where A = 43 2 2 55
4 0 1 7

is the solution set S trivial or not?

Answer: at once without computations, the unknown is ~x 2 R4 ,

have 3 rows (3 equations), echelon form must have at least one
free variable. Hence S is non-trivial.
General fact: if A is m ⇥ n matrix with m < n then echelon form of
A has free variables and A~x = ~0 has non-trivial solutions.
If m n then cannot say anything! Both cases are possible. Need
to solve for given A.
61/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example:
For the system
2 3
1 2 3
A~x = ~0 where A = 43 2 25 3 2 2
4 0 1

we are told that the last row is the sum of the first two rows. Is
the solution set S trivial or not? Here ~x 2 R3 .

Answer: we can replace the last row by [0 0 0] (why?). Then the

echelon form must have at least one free variable. Hence S is
non-trivial.

62/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example: for the system

2 3
1 3 4
A~x = ~0 where A=4 2 2 05
3 2 1
we are told that the last column is the sum of the first two
columns. Is the solution set S trivial or not?

Answer: write the matrix as A = [~a1 ~a2 ~a3 ], then we are told that
~a3 = ~a1 + ~a2 .
This means
3 2
1
~a1 + ~a2 + ( 1)~a3 = ~0 ) A 4 1 5 = ~0.
1
Hence S is nontrivial.
If S is nontrivial, then it is infinite. Why?
63/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

A5 6
General fact:
If reduced echelon form has k free variables,
then there are k vectors in Rn which exactly span the set of solutions:

So, if exactly one free variable then S is line through ~0 in Rn .

If exactly two free variables, then S is plane through ~0 in Rn .

We saw that when we wrote solution with free variables as a span!

This gives a picture of all solutions of a homogeneous system.

64/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

What about a non-homogeneous system?

That is, suppose ~b 6= ~0 in the system A~x = ~b.

65/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

First observation:
If the last column of echelon form of augmented matrix is a pivot,
then there is no solution. We say the system of equations is
inconsistent.
2 3
$ ⇤ ⇤ ⇤ ⇤ ⇤
6 0 $ ⇤ ⇤ ⇤ ⇤ 7
6 7
6 0 0 0 $ ⇤ ⇤ 7
6 7
4 0 0 0 0 0 $ 5
0 0 0 0 0 0
$ = non-zero entry (so a pivot)
The last non-zero row says 0x1 + 0x2 + · · · + 0xn = $. Since $ 6= 0
this is impossible.

66/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

On the other hand, if the last column of echelon form of

augmented matrix is not a pivot, then can back-fill solution. We
say the system of equations is consistent.
To find a solution, move to reduced echelon form
2 3
1 0 ⇤ 0 ⇤
6 0 1 ⇤ 0 ⇤ 7
6 7
6 0 0 0 1 ⇤ 7
6 7
4 0 0 0 0 0 5
0 0 0 0 0

If no free variables, then solution is unique, given by the entries in

the last (augmentation) column.
If there are free variables, how can we describe S?

67/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

In this case observe:

If all free variables are set to 0 then equations easily give one
solution s~0 :
Example: Supposed the reduced echelon form looks like
2 3
1 0 ⇤ 0 ⇤ ⇡
6 0 1 ⇤ 0 ⇤ e 7
6 p 7
6 0 0 0 1 ⇤ 2 7
6 7
4 0 0 0 0 0 0 5
0 0 0 0 0 0
Set the free variables x3 = x5 = 0. Then the equations become

x1 + 0x2 + ⇤0 + 0x4 + ⇤0 = ⇡
0x1 + x2 + ⇤0 + 0x4 + ⇤0 = e
p
0x1 + 0x2 + ⇤0 + x4 + ⇤0 = 2
p
Clearly x1 = ⇡, x2 = e, x4 = 2 is a solution.
68/76
 Au Anz A
As
en

far an a n

Im Anz AM

rux aufitAizXzt
Ain be t Xue
aux tan fat tan Xm be AT D
i
bin E 5 15
am.at tamaxu
E
At 5 has a satins
I
I
HI 5 In
I
Aletta Ain 5 8 5
II
EXAMPLE 0 I 2

11
111 1
 o

12 1 3
1
particular solution Yg
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

So far, we’ve found how to determine one solution s~0 . That is,

s0 = ~b
A~

How to find all solutions?

Suppose s~0 is another solution, so also As~0 = ~b.
Then
A(s~0 s~0 ) = As~0 A~ s0 = ~b ~b = ~0
In other words, the vector (s~0 s~0 ) = ~s is solution to A~x = ~0.
Equivalently s~0 = s~0 + ~s .
Hence the solution set S for the system A~x = b is

S = s~0 + Span{s1 , ..., sk }

where Span{s1 , ..., sk } is solution to homogeneous system A~x = ~0.

69/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

The picture in Rn of possible solutions to A~x = ~0 and A~x = ~b:

Ax = b

s0
Ax = 0

70/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Example (understand it for the exam):

find all solutions of non-homogeneous system

x1 + 2x2 3x3 + 2x4 4x5 = 1

2x1 + 4x2 5x3 + 1x4 6x5 = 3
5x1 + 10x2 13x3 + 4x4 16x5 = 7

Translation A~x = ~b

homogeneous system:

x1 + 2x2 3x3 + 2x4 4x5 = 0

2x1 + 4x2 5x3 + 1x4 6x5 = 0
5x1 + 10x2 13x3 + 4x4 16x5 = 0

Translation A~x = ~0
71/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Associated augmented matrix:

2 3
1 2 3 2 4 1
4 2 4 5 1 6 3 5
5 10 13 4 16 7

Replace L2 by 2L1 + L2 and L3 by 5L1 + L3 :

2 3
1 2 3 2 4 1
4 0 0 1 3 2 1 5
0 0 2 6 4 2

Replace L3 by 2L2 + L3 :
2 3
1 2 3 2 4 1
4 0 0 1 3 2 1 5
0 0 0 0 0 0

Consistent! so there is a solution ©.

72/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

2 3
1 2 3 2 4 1
4 0 0 1 3 2 1 5
0 0 0 0 0 0
To find solution need reduced echelon. Replace L1 with L1 + 3L3
2 3
1 2 0 7 2 4
4 0 0 1 3 2 1 5
0 0 0 0 0 0
Free variables: x2 , x4 , x5 . Set them to 0 and get:
x1 = 4, x3 = 1. Hence one solution is
2 3
4
6 0 7
6 7
s~0 = 6 7
6 1 7.
4 0 5
0
73/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

For solution of the homogeneous system

2 3
1 2 0 7 2 0
4 0 0 1 3 2 0 5
0 0 0 0 0 0

translates to
2 3 2 3 2 3 2 3 2 3
x1 2x2 + 7x4 2x5 2 7 2
6 x2 7 6 x2 7 617 6 07 6 07
6 7 6 7 6 7 6 7 6 7
6 x3 7 = 6 3x4 2x5 7 6 7 6 7 6 27
6 7 6 7 = x 2 6 0 7 + x 4 6 37 + x 5 6 7
4 x4 5 4 x4 5 405 4 15 4 05
x5 x5 0 0 1

74/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Hence all solutions to the homogeneous system given by:

82 3 2 3 2 39
>
> 2 7 2 >>
>
> >
< 6 1 7 6 07 6 0 7 >
6 7 6 7 6 7 =
Span 6 6 07 7 ,6
6 377 ,6
6 27
7>
>
>
>
>
4 0 5 4 1 5 4 0 5>>
>
: ;
0 0 1
Picture:

Span{u,v}
u

75/76
Vectors Picturing vectors Properties of vectors Linear combination Span Compact notation Matrix-vector multiplicatio

Answer: all solutions to the original non-homogeneous system

given by:
2 3 82 3 2 3 2 39
4 >
> 2 7 2 >>
607 >
> 6 7 6 7 6 7 >
>
6 7 < 1 0
6 7 6 7 6 7= 0
~x = 6 7 6 7 6 7 6 7
617 + Span >6 0 7 , 637 , 6 27>
405 >
> 4 0 5 415 4 0 5> >
>
: >
;
0 0 0 1
Picture:

s0+Span{u,v}
u
s0
v

76/76
 Ac Ket t AcaXu b inhomogeneous
SYSTEM
am at tamutu bur
din

1 am
t
all

It 5
A D ArgumerTED MDrix
I 21
O 1 A
1
t
Solution Easts a pivot
Xa Xo her variables 7 heevariables

FIND A PORTOLAN SOUL

DOWN THE PROBLEM
X2 Xe Yo to all free variable

5
6 Panwar sound
 o o e homogeneous
mean amatol
to

7 Reeverrebles

7dam caches
span of
GENERAL Sourav will be
5 spare ri I
T
Patiala all solutions
ofhangars