Scribd
Upload
46 views

Linear Classi Ers: Prediction Equations: Michael (Mike) Gelbart

This document discusses linear classifiers in Python. It explains that linear classifiers make predictions by computing the dot product of a set of coefficients and the features, plus an intercept term. If the result is positive, it predicts one class, and if negative, the other class. It provides an example using LogisticRegression to demonstrate how it makes predictions. Additionally, it discusses loss functions, explaining that least squares loss is used for regression while 0-1 loss or logistic loss are more suitable for classification problems. It includes diagrams to illustrate different loss functions geometrically.

Uploaded by

NourheneMbarek
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
46 views

Linear Classi Ers: Prediction Equations: Michael (Mike) Gelbart

This document discusses linear classifiers in Python. It explains that linear classifiers make predictions by computing the dot product of a set of coefficients and the features, plus an intercept term. If the result is positive, it predicts one class, and if negative, the other class. It provides an example using LogisticRegression to demonstrate how it makes predictions. Additionally, it discusses loss functions, explaining that least squares loss is used for regression while 0-1 loss or logistic loss are more suitable for classification problems. It includes diagrams to illustrate different loss functions geometrically.

Uploaded by

NourheneMbarek
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 22

Linear classi ers:

prediction equations
LIN EAR CLAS S IF IERS IN P YTH ON

Michael (Mike) Gelbart


Instructor, The University of British
Columbia
Dot Products
x = np.arange(3) np.sum(x*y)
x

14
array([0, 1, 2])

x@y
y = np.arange(3,6)
y
14

array([3, 4, 5]) x@y is called the dot


product of x and y , and
x*y
is written x ⋅ y.

array([0, 4, 10])

LINEAR CLASSIFIERS IN PYTHON


Linear classi er prediction
raw model output = coefficients ⋅ features + intercept
Linear classi er prediction: compute raw model output, check the
sign
if positive, predict one class

if negative, predict the other class

This is the same for logistic regression and linear SVM


fit is different but predict is the same

LINEAR CLASSIFIERS IN PYTHON


How LogisticRegression makes predictions
raw model output = coefficients ⋅ features + intercept

lr = LogisticRegression()

lr.fit(X,y)

lr.predict(X)[10]

lr.predict(X)[20]

LINEAR CLASSIFIERS IN PYTHON


How LogisticRegression makes predictions (cont.)
lr.coef_ @ X[10] + lr.intercept_ # raw model output

array([-33.78572166])

lr.coef_ @ X[20] + lr.intercept_ # raw model output

array([ 0.08050621])

LINEAR CLASSIFIERS IN PYTHON


The raw model output

LINEAR CLASSIFIERS IN PYTHON


The raw model output

LINEAR CLASSIFIERS IN PYTHON


The raw model output

LINEAR CLASSIFIERS IN PYTHON


Let's practice!
LIN EAR CLAS S IF IERS IN P YTH ON
What is a loss
function?
LIN EAR CLAS S IF IERS IN P YTH ON

Michael Gelbart
Instructor, The University of British
Columbia
Least squares: the squared loss
scikit-learn's LinearRegression minimizes a loss:
n
∑(true ith target value − predicted ith target value)2
i=1

Minimization is with respect to coef cients or parameters of the


model.

Note that in scikit-learn model.score() isn't necessarily the loss


function.

LINEAR CLASSIFIERS IN PYTHON


Classi cation errors: the 0-1 loss
Squared loss not appropriate for classi cation problems (more on
this later).

A natural loss for classi cation problem is the number of errors.

This is the 0-1 loss: it's 0 for a correct prediction and 1 for an
incorrect prediction.

But this loss is hard to minimize!

LINEAR CLASSIFIERS IN PYTHON


Minimizing a loss
from scipy.optimize import minimize

minimize(np.square, 0).x

array([0.])

minimize(np.square, 2).x

array([-1.88846401e-08])

LINEAR CLASSIFIERS IN PYTHON


Let's practice!
LIN EAR CLAS S IF IERS IN P YTH ON
Loss function
diagrams
LIN EAR CLAS S IF IERS IN P YTH ON

Michael (Mike) Gelbart


Instructor, The University of British
Columbia
The raw model output

LINEAR CLASSIFIERS IN PYTHON


0-1 loss diagram

LINEAR CLASSIFIERS IN PYTHON


Linear regression loss diagram

LINEAR CLASSIFIERS IN PYTHON


Logistic loss diagram

LINEAR CLASSIFIERS IN PYTHON


Hinge loss diagram

LINEAR CLASSIFIERS IN PYTHON


Hinge loss diagram

LINEAR CLASSIFIERS IN PYTHON


Let's practice!
LIN EAR CLAS S IF IERS IN P YTH ON

You might also like