AI, Ethics,

and Society

Introduction
China's social credit system keeps a critical eye on everyday behavior:
https://www.youtube.com/watch?v=Onm6Sb3Pb2Y

China’s Social Credit System

What is Big
Data?
 “Big data is the term
increasingly used to describe
the process of applying “The definition
serious computing power – of big data?
the latest in machine learning Who cares?
and artificial intelligence – to It’s what you’re
seriously massive and often doing with it”
highly complex sets of
information.”
(From Bill Franks,
Chief Analytics Officer)

(From Microsoft Blog)

Data Landscape
As of 2019, there are 2.5 quintillion
bytes of data created each day at our
current pace.
Every Minute of the Day:
⬣ Snapchat users share 527,760 photos
⬣ More than 120 professionals join
LinkedIn
⬣ Users watch 4,146,600 YouTube
videos
⬣ 456,000 tweets are sent on Twitter
⬣ Instagram users post 46,740 photos
Google now processes more than
40,000 searches EVERY second (3.5 For more
billion searches per day) information
check out
conversationprism.com
AI/ML algorithms are being deployed by organizations to make decisions
that impact us in direct ways:

⬣ filtering applicants for jobs, admissions, …

⬣ approving applicants for loans, bank credit,
credit cards, …
⬣ predicting risk levels for insurance, health
benefits, …
⬣ user profiling for policing purposes and to
predict risk of criminal recidivism, …
⬣ identifying risk of developing mental health
problems, …

Why does this matter?

 Goal
⬣ How do we design algorithms that
effectively deal with the large
amounts of data that are used to train
them, while ensuring their outcomes
aren't misused?

⬣ We will examine various AI/ML

techniques that can be used to
counterbalance the potential abuse
and misuse of learning from big data,
with a focus on the impacts of these
technologies on society.
 Artificial Intelligence

Machine Learning

Data Science Deep

Learning

Big Data

Note on Terminology
https://mc.ai/no-machine-learning-is-not-just-glorified-statistics/
 AI, Ethics,
and Society

Overview
Google Analytics
In-Class Exercise

How unique are you?

https://aboutmyinfo.org/

⬣Birthdate

⬣Gender

⬣Zipcode
⬣Big data allows organizations
to now target specific
demographics
⬣This was not possible 20
years ago
⬣Big data also allows
organizations to do a lot more
than targeted advertising

Why is this in my news feed?

 So What’s the Problem?

⬣Humans are biased

⬣Garbage in = Garbage out

Wu, Xiaolin and Xi Zhang. “Automated
Inference on Criminality using Face Images.”
We’ve seen this before:
⬣ Physiognomy: The practice of using people’s
outer appearance to infer inner character. In
physiognomy, there is the belief that there are
inferior “types” of people, identifiable by their
facial features and body measurements.
⬣ 1876: Using precise measurements that could
be measured with instruments like calipers
and craniographs, Lombroso recorded
physical traits he found indicative of a
criminal: sloping forehead, ears of unusual
size, asymmetry of the face, receding
forehead, strongly developed jaw, strongly
developed cheeks, ...
ORIGINS OF CRIMINOLOGY;
CESARE LOMBROSO AND THE FACE OF CRIME
 AI &
Unintended
Consequences
Face Recognition Applications

⬣ A young female, of Taiwanese decent,

blogged about her camera labelling her as
“blinking” (Rose 2010).
⬣ An automated passport system was unable to
detect that an Asian person had his eyes open
(Griffiths 2016).
⬣ Google labeled black people as “gorillas” in its
new photos application for iOS and Android
(Pulliam-Moore 2015), (Simonite 2018)

Review: AI & Unintended Consequences

Natural Language Processing (NLP) Applications

⬣ In testing Alexa and Google’s Home, researchers

found notable disparities in how people from
different parts of the U.S. are understood (Harwell
2018)
⬣ Microsoft’s neo-Nazi sexbot was a great lesson for
makers of AI assistants (Metz 2018) (Hornigold
2019)
⬣ Gender bias identified in Google's speech
recognition software (Tatman 2016) (Vincent 2018)

Review: AI & Unintended Consequences

Predictive Algorithms and Search Applications

⬣ Entering “three white teenagers” on Google image search resulted mostly in

images of happy people whereas the search phrase “three black teenagers”
offered an array of mug shots (Guarino 2016).
⬣ Search results for “doctors” retrieved significantly more images of men
whereas a search for “nurses” provided significantly more images of women
(Kay 2015).
⬣ Amazon scraps secret AI recruiting tool that showed bias against women
(Dastin 2018)
⬣ There’s software used across the country to predict future criminals. And it’s
biased against blacks (Angwin 2016).

Review: AI & Unintended Consequences

 Next Class:

Ethics vs Law
 AI, Ethics,
and Society

Ethics vs Law
 Right Wrong
Ethics versus Law
Law:
⬣ Is the system of rules of conduct established by the government of a society to
maintain stability and justice
⬣ Defines the legal rights and duties of the people and provides the means of
enforcing these rights and duties

Ethics:
⬣ defined as the set of moral principles that distinguish what is right from what is
wrong.
⬣ Moral Standards: rules about the kinds of actions that are morally right or
wrong, as well as the values placed on what is morally good or bad

Laws carry sanctions of a governing authority; ethics do not

Ethics and Law

 The Relationship Between
Ethics and Law
⬣ A relationship exists between law and
ethics. In some instances, law and
ethics overlap (related) and what is
supposed as unethical is also illegal.
⬣ In other situations, they do not
Law Ethics
overlap. In some cases, what is
supposed as unethical is still legal,
and in others, what is illegal is
perceived as ethical.
⬣ As a result, ethics and law will
sometimes conflict.
Overview of
US Laws
⬣ Civil: governs nation or state; manages relationships/conflicts
between organizational entities and people; recorded in volumes of
legal “code” available for review by the average citizen
⬣ Criminal: addresses violations harmful to society; actively enforced
by the state
⬣ Private: regulates relationships between individuals and
organizations; encompasses family law, commercial law, and labor
law
⬣ Public: regulates structure/administration of government agencies
and relationships with citizens, employees, and other governments;
encompasses criminal, administrative, and constitutional law

Types of Law (US-Centric)

⬣ There are a number of domains that are regulated by government
(which also extends to business practices involving marketing and
advertising)
⬣ Purpose of regulations is to achieve certain publicly desired goals
that the market may fail to realize (i.e. protect the public)
⬣ Credit (Equal Credit Opportunity Act)
⬣ Education (Education Amendments of 1972; Civil Rights Act of 1964)
⬣ Employment (Civil Rights Act of 1964)
⬣ Housing and ‘Public Accommodation’ (Fair Housing Act; Civil
Rights Act of 1964)

Regulated Domains in Law

 ⬣ Law prohibits unfair treatment/decisions made on the basis of
human characteristics; which results in harm suffered by persons
⬣ Discrimination is prohibited on the basis of membership in a
protected class-group which has historically been discriminated
against.
Forms of Discrimination (or Bias):

⬣ Intentional: Deliberate and conscious, based on negative

stereotypes about group
⬣ Unconscious: Not deliberate, but unconsciously based on
negative stereotypes

Discrimination Law
⬣ Race (Civil Rights Act of 1964, 1991)
⬣ Color (Civil Rights Act of 1964, 1991)
⬣ Sex (Equal Pay Act of 1963; Civil Rights Act of 1964, 1991)
⬣ Religion (Civil Rights Act of 1964, 1991)
⬣ National origin (Civil Rights Act of 1964, 1991)
⬣ Citizenship (Immigration Reform and Control Act)
⬣ Age (Age Discrimination in Employment Act of 1967) (over 40)
⬣ Pregnancy (Pregnancy Discrimination Act)
⬣ Familial status (Civil Rights Act of 1968)
⬣ Disability status (Rehabilitation Act of 1973; Americans with Disabilities Act
of 1990)
⬣ Veteran status (Vietnam Era Veterans' Readjustment Assistance Act of
1974; Uniformed Services Employment and Reemployment Rights Act)
⬣ Genetic information (Genetic Information Nondiscrimination Act)

Legally recognized ‘protected classes’

Statistical
⬣ Numerical analysis that suggests that groups are treated or
affected differently
Problem - one must rule out other explanations; which might be
difficult to do (Stats 101 – next Module)

Direct evidence showing deliberate intention in a particular case

⬣ Various documents, conversations, compensation patterns etc.

Problem - this type of evidence is often not available to

individuals (Not addressing in this Course)

Evidence of Discrimination/Bias
⬣ Equality of opportunity is typically concerned with ensuring that
decision-making processes treat similar people similarly on the basis
of relevant features, given their current degree of similarity
⬣ Equality of outcome is a notion of equality of opportunity that forces
decision-making to treat seemingly dissimilar people similarly, on the
belief that their current dissimilarity is the result of past injustices

This is also where ethics comes in…

https://www.encyclopedia.com/social-sciences/encyclopedias-
almanacs-transcripts-and-maps/equality-opportunity

Equality of opportunity vs Equality of outcome

 Ethical
Decision
Making
⬣ Dilemma occurs when a decision, that could benefit a person,
organization, or both, might also cause harm to others
⬣ One way of thinking about benefits and harms is to understand our
vital life interests (food, water, air, shelter, and bodily integrity) as well
as our quality-of-life interests (health, happiness, family, friendship,
social reputation, liberty, autonomy, knowledge, privacy, economic
security, respectful and fair treatment by others, education, meaningful
work, and opportunities for leisure, play, entertainment, and creative
and political expression, …)
⬣ Cultural differences also create difficulty in determining what is and is
not ethical. Difficulties arise when one community’s ethical construct
conflicts with ethics of another community

Ethical Decision Making

 compared with
Behavior or act
Prevailing norms
that has been
of acceptability
committed

Fellow Fellow Regions of

Workers Workers Country

Family Profession
Value judgments
and perceptions The
Individual
of the observer
Friends Employer

The Law Religious Society at

Beliefs Large
Ethical Decision Making
Codes of
Ethics
Codes of Ethics and Professional Organizations

Professional boards and organizations have

written codes which hold members to a higher
standard than the law imposes - ethics codes
⬣ E.g. Association of Computing Machinery
(ACM) established in 1947 as “the world's first
educational and scientific computing society”
⬣ ACM code of ethics contains references to
protecting information confidentiality, causing
no harm, protecting others’ privacy, and
respecting others’ intellectual property
Violation can lead to revocation of license,
suspension of rights to practice, censure, or
penalties
Computer Ethics Institute: http://computerethicsinstitute.org/
1. Thou Shalt Not Use A Computer To Harm Other People.
2. Thou Shalt Not Interfere With Other People’s Computer Work.
3. Thou Shalt Not Snoop Around In Other People’s Computer Files.
4. Thou Shalt Not Use A Computer To Steal.
5. Thou Shalt Not Use A Computer To Bear False Witness.
6. Thou Shalt Not Copy Or Use Proprietary Software For Which You have Not Paid.
7. Thou Shalt Not Use Other People’s Computer Resources Without Authorization Or
Proper Compensation.
8. Thou Shalt Not Appropriate Other People’s Intellectual Output.
9. Thou Shalt Think About The Social Consequences Of The Program You Are Writing Or
The System You Are Designing.
10. Thou Shalt Always Use A Computer In Ways That Insure Consideration And Respect
For Your Fellow Humans.

Example Code of Ethics

 Ethical
Issues for
Big Data
 Accuracy: who is
Privacy: right of responsible for the
individual to authenticity, fidelity,
control personal and accuracy of
information information?

Accessibility: what information

Property: Who
does an organization have the
owns the
right to collect? Under what
information? Who
safeguards? What can they do
controls access?
with it after?
 What motivates us to view Are we consistent in the
issues a certain way? way we approach
ethical issues?

What approach do we use to

resolve conflicts?

Examples of two basic camps:

Consequence-based Rule-based

Framework for Ethics

 Consequence-Based Ethics

⬣ Priority is given to choices that

lead to a “good” outcome
(consequence)
⬣ The outcome outweighs the
method
⬣ Utilitarian View – the “right
choice” delivers the greatest good
to the most people.
⬣ Individualism View – the “right
choice” is best for long-term self-
interest.
 Rule-Based Ethics
⬣ Priority is given to following the rules
without undue regard to the outcome
⬣ Rules are often thought to codify
principles like truthfulness, right to
freedom, justice, etc.
⬣ Moral-Rights View – the “right choice” is
that which respects fundamental rights
shared by all human beings (i.e. “never
tell a lie”)
⬣ Rule-Based/Justice View – the “right
choice” is that which is impartial, fair, and
equitable in treating people; exists for the
benefit of society and should be followed
 Example
Ethical Issue
 Cancer Detection, Inc.
Competitor, Inc. Moral-Rights Viewpoint
Utilitarian Viewpoint
The right thing to
We can save millions do is to tell the truth
of lives now; time is of
regardless of the
the essence. consequences.

Example Scenario
 https://www.nytimes.com/2018/12/31/technology/human-
resources-artificial-intelligence-humu.html

It digs through employee surveys using artificial

intelligence to identify one or two behavioral changes
that are likely to make the biggest impact on elevating
a work force’s happiness. Then it uses emails and text
messages to “nudge” individual employees into small
actions that advance the larger goal.

In-Class Exercise (Group of 5-6) – Evaluate: https://www.scu.edu/ethics-app/

At end, record your value and report back. Your option is ??

 Next Class:

Data Collection
AI, Ethics, and
Society

Data Collection
https://www.youtube.com/watch?v=VDR8qGmyEQg
Information
Privacy
⬣ Information privacy (or data privacy) is
the relationship between the collection and
dissemination of data, technology, the
public expectation of privacy, legal and
political issues surrounding them
(Wikipedia)
⬣ The ability to collect information on
individuals, combine facts from separate
data sources, and merge this data with
other non-personal types of information
has resulted in archives of information
that were previously unheard of
⬣ This aggregation of data also permits
organizations to build databases of facts
(and non-facts) with frightening, perhaps
unethical, outcomes
Financial Services Modernization Act, or Gramm-Leach-Bliley
Act of 1999
⬣ Requires due notice by financial organizations to customers so that they
can request that their information not be shared with third parties
⬣ Ensures that the organization’s privacy policies are fully disclosed (and
distributed annually) when a customer initiates a business relationship

Federal Privacy Act of 1974

⬣ Regulates the government in the protection of individual privacy

Electronic Communications Privacy Act of 1986

⬣ Regulates the interception of wire, electronic, and oral communications

Privacy of Customer Information

Privacy of Customer Information Section of the common
carrier regulation

⬣ Specifies that any proprietary information shall be used explicitly for providing
services, and not for any marketing purposes
⬣ Stipulates that carriers cannot disclose this information except when
necessary to provide their services

Health Insurance Portability and Accountability Act of 1996 (HIPAA)

⬣ Severely restrict the dissemination and distribution of private health

information by organizations without documented consent.
⬣ Provides patients the right to know who has access to their information, and
who has accessed it

Privacy of Customer Information

 Data
Management
Practices
2 billion active users on Facebook
More than a The world internet population has grown
quarter of the 7.5% from 2016 and now represents 3.7
3.7
billion people. 3.4
world’s 7 billion
3.0
humans are 2.5

active on 2012 … 2014 … 2016 2017

Facebook https://www.domo.com/learn/data-never-sleeps-5

Instagram (now owned by Facebook)

⬣ There are 600 million Instagrammers
⬣ Each day 95 million photos and videos are shared on Instagram
⬣ 100 million people use the Instagram “stories” feature daily

We’ve been using Cloud for a while…

 ROOM
FOR RENT
ATM
Intellectual
Property
Rights and
Protections
 Intellectual Property Rights

⬣ Intellectual property (IP) rights are

the rights given to persons over the
creations of their minds. They usually
give the creator an exclusive right over
the use of his/her creation for a certain
period of time. When you create
information, you typically acquire the
rights to that information.
⬣ Thus, unauthorized replication and/or
re-use of your information would be a
breach of your IP rights.
https://policies.google.com/terms?hl=en
https://www.facebook.com/terms.php
Small Hiccup: GDPR imposes new rules on organizations that offer goods and
services to people in the European Union (EU), or that collect and analyze data
tied to EU residents, no matter where they are located.

⬣ Valid consent must be explicit (opt-in based) for data collected and the purposes
data is used
⬣ The data subject has the right to request erasure of personal data related to them
⬣ A person shall be able to transfer their personal data from one electronic
processing system to and into another, without being prevented from doing so
⬣ Has force of law, effective May 2018. This is a EU Regulation, rather than a EU
Directive (and has steep penalties).
⬣ The law is extraterritorial, i.e. it applies to any organization which stores, transfers
or otherwise processes data from EU citizens, regardless of whether that
organization is based in the EU or not.

General Data Protection Regulation (GDPR)

Next Lecture:

Fairness and
Bias
 AI, Ethics,
and Society

Fairness and
Bias
What is
Bias?
 Algorithmic Bias Concepts

⬣ Motivation and examples of bias

⬣ Sources of algorithmic bias

⬣ Types of Bias

⬣ Measures of Bias
 A predisposition, partiality, prejudice, preference, or predilection.

Biases have to do with:

⬣ Stereotype: an exaggerated belief, image
or distorted truth about a person or group
⬣ Prejudice: an opinion, prejudgment or
attitude about a group or its individual
members
⬣ Discrimination: a behavior that treats
people unequally because of their group
memberships

What is Bias?
 Economics
Age
Appearance
Geography
Occupation

Family Status Personality

Scientific research has
demonstrated that Sexual
Physical/
Mental Ability
Orientation
everyone has biases,
Military Status
hidden or not. Education
Race
Politics Gender
Lifestyle
Religion

Adapted from http://www.safeschoolscoalition.org/

Forms of Biases
An Incorrect
Assumption
about AI
⬣ Discrimination is prohibited on the
basis of membership in a protected
class-group
⬣ Law prohibits unfair treatment or
decisions made on the basis of
human characteristics; which
results in harm suffered by persons
⬣ People's decisions include objective
and subjective elements, thus, they
can be biased
⬣ If algorithmic inputs include only
objective elements, they thus should
not be biased
 COMPAS (Correctional Offender Management Profiling
for Alternative Sanctions): 137-questions questionnaire
and predictive model for "risk of recidivism"

Prediction accuracy of recidivism for black and white defendants was

~62%, but ...
⬣ Black defendants who did not reoffend were nearly twice as likely
to be misclassified as higher risk compared to their white
counterparts (45 percent vs. 23 percent)
⬣ White defendants who re-offended were mistakenly labeled low
risk almost twice as often as black re-offenders (48 percent vs. 28
percent).

Judiciary use of COMPAS scores

 Reinforcement of Stereotypes
Researchers looked at the top 100 image
search results for 45 different jobs. They
then compared the gender breakdown
of those results to actual gender data
from the Bureau of Labor Statistics for
each job.
⬣ Results for "CEO" in Google Images:
11% female, US 27% female CEOs
⬣ Results for telemarketers in Google Images:
36% male, US 50% male
⬣ Results for bartender in Google Images:
23% female, US 60% female
AdFisher: a tool for running
Automated Experiments on
Personalized Ad Settings
⬣ https://github.com/tadatitam/
info-flow-experiments
⬣ Used to demonstrate that setting
gender = female results in less
ads for high-paying jobs on job-
related websites.
https://adssettings.google.com/
⬣ Gender: Is it Correct? Added or
Inferred?

Google Ads
Algorithmic
Bias
Amazon Scraps Secret AI-Recruiting tool

⬣ AI was used to give job candidates

scores ranging from 1 to 5 stars
⬣ By 2015, the company realized its new
system was not rating candidates for
technical posts in a gender-neutral
way
⬣ Amazon’s computer models were
trained to vet applicants by observing
patterns in resumes submitted over
past 10-year period.

Self-perpetuating algorithmic biases

⬣ Ranking algorithm for finding job
candidates puts Sam at bottom of list.
⬣ Hence, Sam gets less jobs offers.
⬣ Hence, Sam gets less job
experience.
⬣ Hence, Sam becomes less
employable.

The same spiral happens in other applications such as

credit scoring and college admissions.

Example
 Confounding matters...

⬣ Algorithms are black boxes; i.e.

there’s an inability to see the inside of
an algorithm and understand how it
arrives at a decision
⬣ Algorithms are considered:
⬣ Proprietary information
⬣ Outside regulators’ jurisdiction
⬣ Bias therefore becomes obscured
⬣ Mitigation becomes difficult
http://moralmachine.mit.edu/
Biases in
Data
 Data inputs:
⬣ Incomplete, incorrect, or outdated
⬣ Poorly Selected/Selected with bias (e.g., US-centric)
⬣ Perpetuating and promoting historical biases

Algorithmic processing:

⬣ Decision making systems that assume correlation implies causation

⬣ Algorithms that do not compensate for datasets that
disproportionately represent populations
⬣ Output models that are hard to understand or explain

Sources of Bias in Data

 Collection of data

⬣ Demographic, geographic, behavioral, temporal biases

Measurement of data

⬣ What do we choose to measure? How do we measure

(e.g., happiness)?

Pre-existing biases in data

⬣ Gender roles in text and images, racial stereotypes in language

Biases in Data
⬣Limited and coarse features
⬣Sample size disparity
⬣Less data (by definition) about minority populations
⬣Skewed sample
⬣Feedback loops
⬣Tainted examples
⬣Features that act as proxies Unintentional
⬣Conscious prejudice Intentional

Biases in Data
 Problem

Its difficult to identify

which specific group
the algorithm might
be biased against.
 A Connection between
Privacy and Bias
Finding if bias exists with respect to
individuals having attribute X is like
inferring attribute X from a
database in which:
⬣ the attribute X was removed
⬣ a new attribute (the decision),
which is based on X, was
added
This is similar to trying to reconstruct
a column from a privacy-scrubbed
dataset
Fairness
Algorithmic fairness: how can we ensure that our algorithms act in
ways that are fair ?
⬣ This definition is vague and somewhat
circular
⬣ Related to Accountability: How do we
supervise/audit machines which have large
impact?
⬣ Related to Transparency: Why does an
algorithm behave in a certain way? Can we
understand its decisions? Can it explain itself?
⬣ Related to AI safety: How can we make AI
without unintended negative consequences?
Aligns with our values?

Fairness
 Tension

⬣Between different
notions of fairness
⬣Between fairness
and accuracy
⬣Between different
methods for
achieving fairness

Fairness
⬣ Suppose we are a bank trying to fairly decide who should get a loan
⬣ i.e. Who is most likely to pay us back?
⬣ Suppose we have two groups, A and B (the sensitive attribute)
⬣ The simplest approach is to remove the sensitive attribute from
the data, so that our classifier doesn’t know the sensitive attribute

Age Gender Postal Code Req Amt A or B? Pay

46 F GT5E $300 A 1
24 M UGA4C $1000 B 0
33 M GT3H $250 A 1
34 F GT3C $2000 A 1
71 F GT5B $200 A 1
28 M UGA5W $1500 B 0

Why Fairness is Hard

⬣ However, if the sensitive attribute is correlated with the other
attributes, this isn’t good enough
⬣ It is easy to predict class if you have lots of other information (e.g.
home address, spending patterns)
⬣ More advanced approaches are necessary

Age Gender Postal Code Req Amt A or B? Pay

46 F GT5E $300 ? 1
24 M UGA4C $1000 ? 0
33 M GT3H $250 ? 1
34 F GT3C $2000 ? 1
71 F GT5B $200 ? 1
28 M UGA5W $1500 ? 0

Why Fairness is Hard

 Principles
for
Quantifying
Fairness
 Requiring that the same percentage of A
and B receive loans

⬣ When we make an error, the direction of that error is equally

likely for both groups
⬣ P(loan|no repay , A) = P(loan|no repay , B)
⬣ P(no loan|would repay , A) = P(no loan|would repay , B)
⬣ Issue: What if 80% of A is likely to repay, but only 60% of B is? Then
is statistical parity the correct thing to do?

Group Fairness (Statistical Parity)

 “Treat similar examples similarly”: A consistent or
individually fair algorithm is one in which similar people
experience similar outcomes

These measures compare the protected group against the unprotected group:

⬣ Risk difference = RD = p1 - p2
(Mentioned in UK Law)

⬣ Risk ratio or relative risk = RR = p1 / p2

(Mentioned by EU Court of Justice)

⬣ Issue: Consistency might also result in everyone being equally treated badly.
Then is an individually fair algorithm the correct thing to do?

Individual Fairness (Consistency)

Next Module:

The BS of
Stats