Scribd
Upload
6 views

2 SupervisedLearning LinearRegression

Uploaded by

João Paulo Dellasta do Nascimento
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
6 views

2 SupervisedLearning LinearRegression

Uploaded by

João Paulo Dellasta do Nascimento
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

ITCS 6156/8156 – Machine Learning

Spring 2024 – Lecture 2

Supervised Learning & Linear


Regression

Christian Kuemmerle
Types of Machine Learning:
• Supervised Learning:
Constructing computer programs that learn from experience to perform
well on a given task.
E.g.: Predict spam/no-spam based on dataset correctly labeled as spam/no-spam.
• Unsupervised Learning:
Given data, try to discover common patterns/ structure or sub-spaces.
E.g.: Cluster news articles by topic
• Reinforcement Learning:

Try to learn from delayed feedback


E.g.: Robots learn to walk; learning to play chess well from playing chess
Mathematical formulation of the pipeline
Dataset:

𝒟 = {(x 1 , y1), …, (xn, y n)}, xi ∈ ℝd, yi ∈ 𝒞 (xi, yi) ∼ 𝒫 ( X , Y ) i . i . d .

Hypothesis:
e.g., a neural network-based
h : ℝd ↦ 𝒞 classifier that maps image to label
of cat or dog

Hypothesis class E.g.:


- a large family of NNs with
ℋ = {h} different parameters
- all linear models
- all quadratic models
MathematicalFormulation
8:d-dimensional feature space

xi:inpat/feature
vector of
in the sample/data point
-

y::
ith Label/response variable output variable
· 2: set oflabels setofoutputs
·

P(x,y):data distribution

Examples: Binary classification e=


40,1) span (no span
·

Multiclass Classification classification


C (1. k) image
·

=
.

.,
.

Regression e=R E.g.predict


of
temperatore
patient
·

heightofperson
·house
price
Examples of hypothesis
Inductive bias (i.e., assumptions) encoded in the hypothesis class

Ex: is a linear function h(x) = sign(w⊤x); Ex: is nonlinear h(x) =


ℋ contains all possible linear functions sign(w⊤(ReLU(Ax)));
ℋ contains all possible one-layer NN
Do we need to make assumptions on the data?
No free lunch theorem says that we must make such assumptions

Informal theorem: for any machine learning algorithm 𝒜 , there must


exist a task 𝒫 on which it will fail

We use prior knowledge (i.e., we believe


linear function is enough) to design an ML
algorithm here
House Prices in Portland

You might also like