Scribd
Upload
22 views

NPTEL

Uploaded by

poojaajithan160701
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
22 views

NPTEL

Uploaded by

poojaajithan160701
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 39

Fairness in Machine Learning - Overview

Arun Rajkumar
Dept of DSAI, IITM

Disclaimer: Several images used in this presentation are sourced from google images. No copyright violation intended
Machine Learning Pipeline

Data Algorithm Model


Principal Component Analysis - Recap

Input Dataset

Principal Components

Test Another Test Image


Image
Logistic Regression - Recap

1
0.8
0.7
0.6

0.5 0.5

0.4
0.3
0.2
0

Linearly Separable Dataset Probabilisitc Model Logistic Function


P(y=1/x) = 1/(1+exp(w.x))
Logistic Regression - Recap
Logistic Regression - Recap
Logistic Regression - Recap

Gradient
Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2208240
Biases in Word Embeddings

https://blog.acolyer.org/2020/12/08/bias-in-word-embeddings/
How do we fix it?

But first, what exactly is fairness?


Fair unsupervised learning
UnFair PCA

https://sites.google.com/site/ssamadi/home/fair-pca-homepage
UnFair PCA

https://sites.google.com/site/ssamadi/home/fair-pca-homepage
UnFair PCA

https://sites.google.com/site/ssamadi/home/fair-pca-homepage
https://sites.google.com/site/ssamadi/home/fair-pca-homepage
Fair Supervised learning
No single answer
Statistical Parity

X – set of all data points (people)


C – Set of all data points (people) belonging to a certain protected group
M: X -> {0,1} - a classifier, say logistic regression

Bias of classifier for the protected group:


Parity(M,C) = Pr(M(x) = 1/x in C) - Pr(M(x) = 1)

Ideal classifier => Parity = 0


In practice, min |parity(M,C)}
Equality of Opportunity

X – set of all data points (people)


C – Set of all data points (people) belonging to a certain
protected group
M: X -> {0,1} - a classifier, say logistic regression.

opportunity_inequality(M,C) = Pr(M(x) = 1| y=1 and C) - Pr(M(x) = 1|y=1)

Ideal classifier: opportunity_inequality = 0


In practice: min |opp_ineq(M,C)|
Individual Fairness

X – set of all data points (people)


M X -> Δ (simplex) - a classifier, say logistic regression.
d: X * X -> R - distance function
D: Δ * Δ -> R – distance between probability vectors

D(M(x),M(x’)) <= d(x,x;’) for any x.x’


Outcome Test (Predictive Parity)

X – set of all data points (people)


C – Set of all data points (people) belonging to a certain
protected group
M: X -> {0,1} - a classifier, say logistic regression.

Pred_parity(M,C) = Pr(y=1| M(x) = 1 and C) - Pr(y=1| M(x) = 1)

Ideal classifier: Pred_parity = 0


In practice: min |Pred_parity(M,C)|
Impossiblity Result

Opportunity_inequality(M,C) = Pr(M(x) = 1| y=1 and C) - Pr(M(x) = 1|y=1)

Opportunity_inequality(M,C) = Pr(M(x) = 1| y=1 and C) - Pr(M(x) = 1|y=1)

Calibration within groups – Pr(M(x) = 1| x in C) ~= Pr(y=1| x in C)


Fair Logistic Regression
https://dl.acm.org/doi/fullHtml/10.1145/3308560.3317584

Parity = |Pr(M(x) = 1|C = 1 ) - Pr(M(x) = 1| C= 0 )|

Regularized logistic loss on dataset

Measures how correlated


are the prediction probabilities
to the Protected attribute.

Protected attribute for


data point i
Fair Multi Armed Bandits

0.8 0.4 0.5 0.9

Case in point: Swiggy/Zomato wants to assign partners to orders.


Goal: Maximize expected reward over time.

A simple strategy: Pick current best arm with (0.9) probability and uniformly at random with 0.1 prob.
Fair Multi Armed Bandits

Notion of Fairness:

If partner-a is truly better than partner b,


then
Probability that algorithm assigns partners a >=
Probability that algorithm assigns partners b

A simple strategy: Pick current best arm with (0.9) probability and uniformly at random with 0.1 prob.
0.6

0.5

0.4
0.5 0.35 0.6 0.38
0.3

IDEA: Sample uniformly from those whose confidence interval overlaps with the winner
● Fair PCA
● Fair LogReg
● Fair MAB
Original Loss function

Perturbed Loss function


Fairness
constraint `
space
Model
Thank you!

https://cerai.iitm.ac.in

[email protected]

You might also like