Scribd
Upload
110 views

Slide 12 - Dimentionality Reduction - PCA

This document discusses dimensionality reduction using principal component analysis (PCA). It explains that PCA finds a lower dimensional surface to project the data onto in order to minimize projection error. The algorithm involves computing the covariance matrix of the data, then performing SVD on the covariance matrix to obtain the eigenvectors, which are the principal components. The top k eigenvectors corresponding to the largest eigenvalues are selected to form the projection matrix.

Uploaded by

Lôny Nêz
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
110 views

Slide 12 - Dimentionality Reduction - PCA

This document discusses dimensionality reduction using principal component analysis (PCA). It explains that PCA finds a lower dimensional surface to project the data onto in order to minimize projection error. The algorithm involves computing the covariance matrix of the data, then performing SVD on the covariance matrix to obtain the eigenvectors, which are the principal components. The top k eigenvectors corresponding to the largest eigenvalues are selected to form the projection matrix.

Uploaded by

Lôny Nêz
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Dimensionality,

Reduc1on,
Mo1va1on,I:,,
Data,Compression,

Machine,Learning,
Data$Compression$
Reduce,data,from,,
(inches),

2D,to,1D,

(cm),

Andrew,Ng,
Data$Compression$
Reduce,data,from,,
(inches),

2D,to,1D,

(cm),

Andrew,Ng,

data compression makes our algorithm rum more faster


Data$Compression$
Reduce,data,from,3D,to,2D,

Andrew,Ng,
Dimensionality,
Reduc1on,
Mo1va1on,II:,,
Data,Visualiza1on,

Machine,Learning,
Data$Visualiza2on$
Mean,
Per,capita, Poverty, household,
GDP, GDP,, Human, Index, income,
(trillions,of, (thousands, DevelopQ Life, ,(Gini,as, (thousands,
Country, US$), of,intl.,$), ment,Index,expectancy,percentage), of,US$), …,
Canada, 1.577, 39.17, 0.908, 80.7, 32.6, 67.293, …,
China, 5.878, 7.54, 0.687, 73, 46.9, 10.22, …,
India, 1.632, 3.41, 0.547, 64.7, 36.8, 0.735, …,
Russia, 1.48, 19.84, 0.755, 65.5, 39.9, 0.72, …,
Singapore, 0.223, 56.69, 0.866, 80, 42.5, 67.1, …,
USA, 14.527, 46.86, 0.91, 78.3, 40.8, 84.3, …,
…, …, …, …, …, …, …,
[resources,from,en.wikipedia.org], Andrew,Ng,
Data$Visualiza2on$
Country,
Canada, 1.6, 1.2, plot these countries in
R2 and use that to try
China, 1.7, 0.3, to understand the space
of features of different
India, 1.6, 0.2, countries the better.
So, what you can do is

Russia, 1.4, 0.5,


Singapore, 0.5, 1.7, Reduce Data
from 50D to 2D
USA, 2, 1.5,
…, …, …,
Andrew,Ng,
Data Visualization
Data$Visualiza2on$

Per. Person
GDP
(economic
activity)

Country size GDP


(Gross Domestic Product)

The axes Z1 and Z2 can help you to capture really what are the two main dimensions of
the variations among different countries. Andrew,Ng,
Dimensionality,
Reduc1on,
Principal,Component,
Analysis,problem,
formula1on,
Machine,Learning,
Let's say we have a data set like
this, and we want to reduce the
dimension of the data from two-
dimensional to one-dimensional.

In other words, find a line onto


which to project the data.
what PCA does formally is, it tries to find a lower
dimensional surface, onto which to project the data so that
the sum of squares of these little blue line segments is
minimized. The length of those blue line segments
sometimes also called the projection error.
if we project the data onto this second line, the projection
errors will be huge. So these points have to move a huge
distance in order to get projected. And so that's why PCA will
choose the red line.
In this example, hope that PCA will find the red
vector “u1”, so when projecting the data onto this
line we end up with pretty small reconstruction
errors. Whether PCA gives “u1” or “-u1”, it doesn't
matter.
PCA$is$not$linear$regression$

X —> Y

Andrew,Ng,
PCA$is$not$linear$regression$

Andrew,Ng,
Dimensionality,
Reduc1on,
Principal,Component,
Analysis,algorithm,

Machine,Learning,
Data$preprocessing$
Training,set:,
Preprocessing,(feature,scaling/mean,normaliza1on):,

Replace,each,,,,,,,,with,,,,,,,,,,,,,,,.,
If,different,features,on,different,scales,(e.g.,,,,,,,,,,,,size,of,house,,,
,,,,,,,,,,number,of,bedrooms),,scale,features,to,have,comparable,
range,of,values.,

Andrew,Ng,
Principal$Component$Analysis$(PCA)$algorithm$
Reduce,data,from,,,Qdimensions,to,,,Qdimensions,
Compute,“covariance,matrix”:,
,
,
Compute,“eigenvectors”,of,matrix,,,,,:,
,[U,S,V] = svd(Sigma);,

Andrew,Ng,
Principal$Component$Analysis$(PCA)$algorithm$
From [U,S,V]
, , = svd(Sigma)
, ,
,,,,,,we,get:,,

Ureduce
Andrew,Ng,
Principal$Component$Analysis$(PCA)$algorithm$summary$
Ader,mean,normaliza1on,(ensure,every,feature,has,
zero,mean),and,op1onally,feature,scaling:,,
Sigma =

[U,S,V] = svd(Sigma);
Ureduce = U(:,1:k);
z = Ureduce’*x;,

Andrew,Ng,

You might also like