Scribd
Upload
6 views7 pages

2 SupervisedLearning LinearRegression

Uploaded by

João Paulo Dellasta do Nascimento
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
6 views7 pages

2 SupervisedLearning LinearRegression

Uploaded by

João Paulo Dellasta do Nascimento
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

ITCS 6156/8156 – Machine Learning

Spring 2024 – Lecture 2

Supervised Learning & Linear


Regression

Christian Kuemmerle
Types of Machine Learning:
• Supervised Learning:
Constructing computer programs that learn from experience to perform
well on a given task.
E.g.: Predict spam/no-spam based on dataset correctly labeled as spam/no-spam.
• Unsupervised Learning:
Given data, try to discover common patterns/ structure or sub-spaces.
E.g.: Cluster news articles by topic
• Reinforcement Learning:

Try to learn from delayed feedback


E.g.: Robots learn to walk; learning to play chess well from playing chess
Mathematical formulation of the pipeline
Dataset:

𝒟 = {(x 1 , y1), …, (xn, y n)}, xi ∈ ℝd, yi ∈ 𝒞 (xi, yi) ∼ 𝒫 ( X , Y ) i . i . d .

Hypothesis:
e.g., a neural network-based
h : ℝd ↦ 𝒞 classifier that maps image to label
of cat or dog

Hypothesis class E.g.:


- a large family of NNs with
ℋ = {h} different parameters
- all linear models
- all quadratic models
MathematicalFormulation
8:d-dimensional feature space

xi:inpat/feature
vector of
in the sample/data point
-

y::
ith Label/response variable output variable
· 2: set oflabels setofoutputs
·

P(x,y):data distribution

Examples: Binary classification e=


40,1) span (no span
·

Multiclass Classification classification


C (1. k) image
·

=
.

.,
.

Regression e=R E.g.predict


of
temperatore
patient
·

heightofperson
·house
price
Examples of hypothesis
Inductive bias (i.e., assumptions) encoded in the hypothesis class

Ex: is a linear function h(x) = sign(w⊤x); Ex: is nonlinear h(x) =


ℋ contains all possible linear functions sign(w⊤(ReLU(Ax)));
ℋ contains all possible one-layer NN
Do we need to make assumptions on the data?
No free lunch theorem says that we must make such assumptions

Informal theorem: for any machine learning algorithm 𝒜 , there must


exist a task 𝒫 on which it will fail

We use prior knowledge (i.e., we believe


linear function is enough) to design an ML
algorithm here
House Prices in Portland

You might also like