Linear Algebra ReSupply Date III.

Replacement Theorem vs Dimension.

Definition 1. zero Vector
Let V be a vector space, then we denoted by the zero vector of V by 0V .
在開始之前，我們先介紹一個 theorem(exercise)，主要去描述一個 linear dependent set 內部的
元素「長怎樣」。簡單來說，就是內部的 vector 與 vector 之間的關係。你可以去這樣想，如果拉
到平面上兩個 vector 平行若且為若兩個 vector 是 linear dependent，如果今天我們討論 n 維或是
vector space 是否也有這樣關係呢? 當然一定有 (就是 linear dependent)，但是太抽象了，在這裡我
們就是要將 linear dependent 具象化 (像是在平面上那樣)，那我在這裡也希望各位可以自己練習。
Exercises. §1.5 Exercises 15 {
u1 = 0 V
Let S = {u1 , ..., un } be a finite set, then S is linear dependent ⇔
uk ∈ span({u1 , ..., uk−1 })

Proof.
We show that necessary and sufficient conditions:

(⇒) If S is linear independent, then there exists a not all zero solution a1 , ..., an ∈ F such that
a1 u1 + · · · + an un = 0.
Let k + 1 largest number such that ak+1 ̸= 0 for some k ∈ [0, n) ∩ N.

(1) If k = 0, then we have a1 u1 = 0V . Since a1 ̸= 0, then we get u1 = 0V .

(2) If k > 0, then we consider that a1 u1 + · · · + ak+1 uk+1 = 0.

Thus, we know that uk+1 ∈ span({u1 , ..., uk }).

(⇐) (i) If u1 = 0V , then we have a1 0V + 0u2 + ... + 0un = 0 for all a1 , and hence S is linear
dependent.

(ii) If uk+1 = a1 u1 + ... + ak uk for all ai ∈ F , then we know that S = span({u1 }), and hence
S is linear dependent.

回想一下之前學 basis 時，我們會著重去證明 linear independent 和 generating set。在這裡

有個很國小的想法，basis 可以當成房子的地基梁柱，理論上來講梁柱應該要條條分明 (linear
independent)，再來透過這些梁柱慢慢往上蓋蓋出房子 (generating set)，OK fine。在簡單說，在
直角坐標平面上，任兩條不平行 (linear independent) 的 vector，可以成為平面上任一條 vector 的
linear independent(generating set)。去好好想清楚，等下的 theorem 將會圍繞 basis 展開!!

Review. Basis
A basis β for a vector space V is a linear independent subset of V and generating V .

(a) β ⊆ V is linear independent subset (b) span(β) = V

We defined the set span(ϕ) = {0V }

1 | Linear Algebra ReSupply Date III JunWei,Zhang Edit

 首先，我們要讓證明比較嚴謹，如果我們選定了一個 vector space 的 basis 我們可以用這個
basis 去「唯一表達」(uniqueness represent) 任意一個在這個 vector space 上的 vector。

Theorem 1. Uniqueness Represent of Vector about Basis (theorem 1.8)

Let V be a vector space over F and β = {x1 , ...xn } ⊆ V , then we have
β is a basis for V ⇔ for all y ∈ V can be uniquely expressed as a linear combination of element in β.
i.e. y = a1 x1 + · · · + an xn is unique for all y ∈ V .

Proof.
We show that necessary and sufficient conditions:

(⇒) Let β is a basis for V , and let y ∈ span(β) = V , then we have y = a1 x1 + · · · + an xn for some
ai ∈ F, xi ∈ β.
Suppose that there exists 2 expressions i.e. y = a1 x1 + · · · + an xn = b1 x1 · · · + bn xn , then we
have
(a1 − b1 )x1 + · · · + (an − bn )xn = 0
Since β is linear independent, then we get ai − bi = 0 for all i, and hence ai = bi for all i.
Thus, we know that it is unique.

(⇐) (i) Let y = a1 x1 + · · · + an xn be a unique expressions for all y ∈ V . Since ai ∈ F, xi ∈ β,

then we have
V ⊆ span(β) ⊆ V ⇒ span(β) = V
by definition.

(ii) If there exists a linear relation among β = {x1 , ..., xn }, then we say c1 x1 +· · ·+cn xn = 0V .
Thus, we have

y = y + 0V = (a1 x1 + · · · + an xn ) + (c1 x1 + · · · + cn xn ) = (a1 + c1 )x1 + · · · + (an + cn )xn

It is contradiction.
Hence, we know that there does not exist a linear relation among β implies that β is
linear independent.

再來，我們要開始建構 Replacement Theorem，在 Exercise 我們將 linear dependent 具象化，

而在下列的 lemma 則是將 linear independent set 和 linear dependent 作上連結，再來我們再用
簡單的方法去證明 basis，試著脫離 linear independent(因為 lemma 成立 linear independent 的條
件)。

2 | Linear Algebra ReSupply Date III JunWei,Zhang Edit

Lemma. (theorem 1.9)
Let S ⊆ V be a linear independent, then we have S∪{x} is linear dependent if and only if x ∈ span(S)

Proof.
We show that necessary and sufficient conditions:

(⇒) If S ∪ {x} is linear dependent, then there exists not all zero a1 , ..., an ∈ F such that

a1 x 1 + a2 x 2 + · · · + an x n = 0 V

and one of x1 , x2 , ..., xn must be x (if not, then S is linear independent (→←))
We say x = x1 , then we know that a1 ̸= 0 implies that a−1 1 exists.
−1
Thus, we get x = a1 (−a2 x2 + · · · − an xn ) implies that x ∈ span(S).

(⇐) If x ∈ span(S), then we have x = a1 x1 + · · · + an xn for all xi ∈ S.

Thus, we get a1 x1 + · · · + an xn + (−1)x = 0V implies that S ∪ {x} is linear independent.

Theorem 2. (theorem 1.9)

If a vector space V is generated by a finite set S0 i.e. S0 ⊆ V, |S0 | < ∞, V = span(S0 ).
Then there exists S0 ⊆ V is a basis for V .

Proof.
We show that the following:

(i) If S0 = ∅, then S0 = {0V }, then we have V = {0V } and ϕ ⊆ S0 .

Thus, we get span(ϕ) = {0V } = V .
Note that. we usually defined the empty set is linear independent.

(ii) If S0 contains a nonzera element x1 , then we know that {x1 } is linear independent.
Continute the possible, satisfy the following statements:

(1) S := {x1 , ..., xr } is linear independent.

(2) S ′ := {x1 , ..., xr , xr+1 }is linear dependent.

We claim that S = {x1 , .., xr } is a basis for V i.e. Show that span(S) = V .
Since S ⊆ S0 , then we know that if S0 ⊆ span(S), then we get

V = span(S0 ) ⊆ span(S) ⇒ span(S0 ) = span(S) = V

We show that S0 ⊆ span(S).

Let x ∈ S0 , then we have

(1) If x ∈ S, then we know that x ∈ span(S).

(2) If x ∈/ S and x ∈/ span(S), then S ∪ {x} is linear independent by Lemma 1.
It is contradiction with the max assumption of S.
Thus, we know that x ∈ span(S), and hence S0 ⊆ span(S).

3 | Linear Algebra ReSupply Date III JunWei,Zhang Edit

 Review. Theorem 1.8 vs Theorem 1.9

(1.8) Let V be a vector space over F and β = {x1 , ...xn } ⊆ V , then we have

β is a basis for V ⇔ y = a1 x1 + · · · + an xn is unique expression for all y ∈ V .

(1.9) If a vector space V is generated by a finite set S0 i.e. S0 ⊆ V, |S0 | < ∞, V = span(S0 ).
Then S0 ⊆ V is a basis for V .

上述的 theorem 是相當重要的，在作一些 basis 上的操作與證明，你一定要熟練他們，如果想

知道更多可以去參考 Reference Material(2)，作者在這方面的證明我覺得寫得相當詳細。那接著我
們要開始去談 Replacement Theorem，簡單來說就是將一個比較小 linear independent set 擴展到
(extend to) 一個 basis。

Theorem 3. Replacement Theorem

Let V be a vectoe space over F with a basis β and |β| = n. If S = {y1 , ..., ym } ⊆ V be a linear
independent with m ≤ n. then there exists S1 ⊆ β with |S1 | = n − m such that span(S ∪ S1 ) = V
(and S ∪ S1 is linear independent).

Proof.
By induction to mathematic on m where n is being held fixed:

(i) If m = 0, then we know that S = ϕ, and hence S1 = β we done!!

(ii) Assume that the theorem holds for some m < n. (Indection Assumption)
We will show that the theorem is true for m + 1.
Let S = {y1 , ..., ym , ym+1 } ⊆ V be a linear independent, then we know that

S ′ = {y1 , ..., ym } ⊆ V is linear independent

By assumption, there exists a S1′ = {x1 , ..., xn−m } ⊆ β such that span(S ′ ∪ S1′ ) = V .
Thus, there exists ai , bj ∈ F such that ym+1 = a1 x1 + · · · + am ym + b1 x1 + · · · + bn−m xn−m .

(iii) If bi is all zero, then we know that S1′ is linear dependent, and hence bi is not all zero.
Now, we say b1 ̸= 0, then we have

x1 = −(a1 y1 + · · · + am ym + (−ym+1 ) + b2 x2 + · · · + bn−m xn−m )(∗)

Thus, we get x1 ∈ span({y1 , ..., ym , ym+1 , x2 , ..., xn−m }),and hence we know that

{y1 , ..., ym , ym+1 , x1 , x2 , ..., xn−m } ⊆ span({y1 , ..., ym , ym+1 , x2 , ..., xn−m })

4 | Linear Algebra ReSupply Date III JunWei,Zhang Edit

 Proof.
By induction to mathematic on m where n is being held fixed:

(iv) We clearly to see that

span({y1 , ..., ym , ym+1 , x1 , x2 , ..., xn−m }) ⊆ span({y1 , ..., ym , ym+1 , x2 , ..., xn−m })

Since we have
V = span(S ′ ∪ S1′ ) = span({y1 , ..., ym , x1 , x2 , ..., xn−m })
and
ym+1 = a1 x1 + · · · + am ym + b1 x1 + · · · + bn−m xn−m
Then we have
V = span({y1 , ..., ym , ym+1 , x1 , x2 , ..., xn−m })
By (∗), we heve

span({y1 , ..., ym , ym+1 , x1 , x2 , ..., xn−m }) ⊇ span({y1 , ..., ym , ym+1 , x2 , ..., xn−m })

Thus, we get V = span({y1 , ..., ym , ym+1 , x2 , ..., xn−m }).

這個 theorem 有很多的 corollary 在初學的時候，而未來當你討論 Matrix Representation,

Change of Coordinates 這個 theorem 是不可或缺的。而在後面我覺得最有趣的就是和 cannical
form 作連結 (Diagonalization, Jordan form, rational form) 到那時候這個 theorem 根本是常識了。
所以如果你對 linear algebra 有興趣並且要想繼續讀下去，一定要熟悉這個 theorem 的內容。

Theorem 4.
Let V be a vectoe space over F with a basis β and |β| = n, then we have

Any linear independent S ⊆ V , containing exactly n elements, is a basis for V .

Proof.
By replacement theorem , we have span(S) = span(S ∪ ϕ) = V , and hence S is a basis for V .

Theorem 5.
Let V be a vectoe space over F with a basis β and |β| = n, then we have

Any S ⊆ V , containing more than n elements, is linear dependent, is a basis for V .

Proof.
Let S ⊆ V with |S| ≥ n + 1, and assume that S is linear independent, and let S1 ⊆ S and |S1 | = n.
Since S1 ⊆ S, then we know that S1 is linear independent.
Thus, we know that S1 is a basis by Corollary 1.
Since S − S1 ̸= ϕ, then we say x ∈ S − S1 , and hence x is a linear combination of S1 .
Thus, it is contradiction, and hence we know that S is linear dependent.

5 | Linear Algebra ReSupply Date III JunWei,Zhang Edit

Theorem 6.
Let V be a vectoe space over F with a basis β and |β| = n, then every basis β for V contain n
elements.

Proof.
Let β be a basis for V with |β| = m, then we know that |m| ≤ n by Corollary 2.
Likewise, we get n ≤ m, and hence n = m.

Review. Dimension
A vector space V is called finite dimensional if it has a basis β, consisting of a finite nimber of
elements. We defined the deimesion of V by dim(V ) = |β|.
If a vector space is not finite deimensional, then it is called infinite deimensional.
如果我們要強調係數是用 F 中元素，我們通常會寫 dim(S) = dimF (S).

Theorem 7.
Let V be a vectoe space over F with dim(V ) = n. If S ⊆ V with |S| ≤ n and span(S) = V .
Then S is a basis for V .

Proof.
By theorem 1.9, there exists S1 ⊆ S such that S1 is a basis for V .
Thus, we know that |S1 | = n by corollary 3 to theorem 1.10, and hence we have

n = |S1 | ≤ |S| = n

Hence, we get S is a basis for V .

Theorem 8. Let V be a vectoe space over F with dim(V ) < ∞ and β is a basis with |β| < ∞.
If S ⊆ V is linear independent, then there exists S1 ⊂ β such that S ∪ S1 is a basis for V .

Proof.
Since S ⊆ V is linear independent, then we have m = |S| ≤ |β| = n.

(i) If |S| = n, then we know that S is a basis for V by Corollary 2.

(ii) If |S| < n, then there exists S1 ⊆ β with |S1 | = n − m such that span(S ∪ S1 ) = V by
theorem 1.10. Thus, we get S ∪ S1 is a basi for V .

Remark.
Every linear independent subset of V can be extended to basis for V .

6 | Linear Algebra ReSupply Date III JunWei,Zhang Edit

Theorem 9.
Let V be a vectoe space over F with dim(V ) < ∞. If W ⊂ V is a subspace of V .
Then dim(W ) < ∞ and dim(W ) ≤ dim(V ); If dim(W ) = dim(V ), then W = V .

Proof.
Let dim(V ) = n, then we consider the following:

(i) If W = {0V }, then we have W = span(∅) and |ϕ| = 0.

Thus, we know that dim(W ) < ∞ and dim(W ) = 0 ≤ dim(V ).

(ii) Otherwise, W contains a nonzero { element x1 , then we have {x1 } is a linear independent set.
{x1 , ..., xk } ⊆ W is linear independent.
Continuing the way, we have
{x1 , ..., xk+1 } ⊆ W is linear dependent.
(This process must end, linear independent subset in V has at most in elements)
Since {x1 , ..., xk } ⊆ W , then we have span({x1 , ..., xk }) ⊆ W .

(iii) Now, we show that W ⊆ span({x1 , ..., xk }).

By Lemma of theorem 1.9, we have

{x1 , ..., xk } ∪ {y} is linear dependent for all y ∈ W ⇔ y ∈ span({x1 , ..., xk }).

Thus, we get W ⊆ span({x1 , ..., xk }) implies that W = span({x1 , ..., xk }).

Hence, we know that {x1 , ..., xk } is a basi for V and we heve dim(W ) = k ≤ n.

(iv) If k = n, then {x1 , ..., xk } is a basis for V , and hence V = W .

Notation. Reference Material

(1) Edition. NTNU Math113 Jun-Wei,Zhang - Date. 2021-12-10

(2) Linear Algebra (4th Edition 2003) - Stephen H.Friedberg, Arnold J. Insel, Lawrence E. Spence.

(3) Linear Algebra (2nd Edition 1971) - Kenneth M Hoffman, Ray Kunze.

7 | Linear Algebra ReSupply Date III JunWei,Zhang Edit