Scribd
Upload
186 views

The Future of Ai: Yann Lecun

The document discusses the current state and future of artificial intelligence. It covers: 1) Recent advances in deep learning for tasks like speech recognition, image understanding, and language translation. 2) Obstacles that remain for achieving human-level general AI, including integrating representation learning with reasoning, attention, planning and memory. 3) The importance of unsupervised learning for building models of the world and common sense, which is seen as the "dark matter" of AI. 4) Predictions that machine intelligence will be different than human intelligence since AI systems will not inherently have drives like humans do unless explicitly built in. Autonomous systems will need to be designed with safeguards to avoid harming humans

Uploaded by

Efrain Tito
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
186 views

The Future of Ai: Yann Lecun

The document discusses the current state and future of artificial intelligence. It covers: 1) Recent advances in deep learning for tasks like speech recognition, image understanding, and language translation. 2) Obstacles that remain for achieving human-level general AI, including integrating representation learning with reasoning, attention, planning and memory. 3) The importance of unsupervised learning for building models of the world and common sense, which is seen as the "dark matter" of AI. 4) Predictions that machine intelligence will be different than human intelligence since AI systems will not inherently have drives like humans do unless explicitly built in. Autonomous systems will need to be designed with safeguards to avoid harming humans

Uploaded by

Efrain Tito
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

The Future of AI

Yann LeCun
Facebook AI Research,
Center for Data Science & Courant Institute,
New York University
http://yann.lecun.com
Y LeCun

Artificial Intelligence:
Where are we Now?
The Deep Learning Revolution
Y LeCun

Speech Recognition
Image & Video Understanding
Natural Language Understanding
Language Translation
All using combinations of Convolutional
Nets and Recurrent Nets.
Memory/Stack-Augmented Recurrent Nets
Y LeCun

[Sukhbataar, Szlam, Weston, Fergus NIPS 2015, ArXiv:1503.08895]


Weakly-supervised MemNN: no need to tell which memory location to use.
[Joulin & Mikolov, ArXiv:1503.01007]
Near Future
Y LeCun

Question Answering
Dialog Systems
Driving Assistance
Specialized and largely non-autonomous systems.
You car will drive you better than you.
We are (very) far from human-level general AI.
Y LeCun

Obstacles to Progress
in
Artificial Intelligence:
Three missing pieces for AI (besides computation) Y LeCun

Integrating Representation/Deep Learning with Reasoning, Attention,


Planning and Memory
A lot of recent work on reasoning/planning, attention, memory,
learning “algorithms”.
Memory-augmented neural nets
“Differentiable” algorithms

Integrating supervised, unsupervised and reinforcement learning into a


single “algorithm”.
Boltzmann Machines would be nice if they worked.
Stacked What-Where Auto-Encoders, Ladder Networks….

Effective ways to do unsupervised Learning


Discovering the structure and regularities of the world by
observing it and living in it like animals and humans do.
Three Types of Learning
Y LeCun

Reinforcement Learning
The machine predicts a scalar
reward given once in a while.
A few bits for some samples

Supervised Learning PLANE


The machine predicts a category
or a few numbers for each input
10→10,000 bits per sample
CAR
Unsupervised Learning
The machine predicts any part of
its input for any observed part.
Predicts future frames in videos
Millions of bits per sample
How Much Information Does the Machine Need to Predict?
Y LeCun

Reinforcement Learning (cherry)


The machine predicts a scalar
reward given once in a while.
A few bits for some samples

Supervised Learning (icing)


The machine predicts a category
or a few numbers for each input
10→10,000 bits per sample

Unsupervised Learning (cake)


The machine predicts any part of
its input for any observed part.
Predicts future frames in videos
Millions of bits per sample
Unsupervised Learning is the “Dark Matter” of AI
Y LeCun

Most of the learning performed by animals and


humans is unsupervised
We learn how the world works by observing it
We learn that the world is 3-dimensional
We learn that objects can move
independently of each other
We learn that object permanence
We learn to predict what the world will
look like one second or one hour from
now.

We build a model of the world through


predictive unsupervised learning
This predictive model gives us “common sense”

Unsupervised learning discovers regularities


in the world.
Common Sense through Unsupervised Learning
Y LeCun

Learning a predictive model of the world gives us common sense.

If I say: “Bernhard picks up his bag and leaves the room”


You can infer:
Bernhard stood up, extended his arm, walked towards the door,
opened the door, walked out.
He and his bag are not in the room anymore.
He probably didn't dematerialize of fly out.
Unsupervised Learning is the “Dark Matter” of AI
Y LeCun

Unsupervised learning is the only form of learning that can provide enough
information to train large neural nets with billions of parameters.
Supervised learning would take too much labeling effort
Reinforcement learning would take too many trials
But we don't know how to do unsupervised learning (or even formulate it)
We have lots of ideas and methods
They just don't work that well yet.

Why is it so hard? The world is unpredictable!


Predictors produce an average of all
possible futures → Blurry image.

Predictor (multiscale ConvNet Encoder-Decoder)


Predictive Unsupervised Learning
Y LeCun

Some success with “adversarial training”


[Mathieu, Couprie, LeCun arXiv:1511:05440]
But we are far from a complete solution.
Discovering Regularities
Y LeCun

LAPGAN & DCGAN:


adversarial training
to generate images.
[Metz, Chintala 2015]
[Denton et al 2015] Random Generator Fake
Vector Network Image Discriminator Real/
Network Fake
Random Training Real
Index Set Image
Y LeCun

Machine Intelligence
Will be very different from
Human Intelligence
What Will AI Be Like?
Y LeCun

Human and animal behavior has basic “drives” hardwired by evolution


Fight/flight, hunger, self-preservation, pain avoidance,
desire for social interaction, etc…

Humans do bad things to each other because of these drives (mostly)


Violence under threat, desire for material resource and social power…

But an AI system will not have these


drives unless we build them into it.

It's difficult to imagine an intelligent


entity without these drives

Although we have plenty of


examples in the animal world
He will we get autonomous AI systems to like us?
Y LeCun

We will build a few basic, immutable, hardwired drives:


To not hurt human and to interact with humans
To crave positive feedback from trusted human trainers
Human trainers will associate rewards with behaviors that make surrounding
humans happy and comfortable.

This is how children


learn how to behave in
society.

Can we prevent unsafe


AI?

Yes, the same way we


prevent unsafe airplanes
and cars.
[from Yael Niv]
How Will Human-Level AI Emerge
Y LeCun

The emergence of human-level AI will not be an “event”.


It will be progressive

It will not happen in isolation


No single entity has a monopoly on good ideas

Advancing AI is a scientific question right now, not a technological challenge


Formulating unsupervised learning is our biggest challenge
Individual breakthroughs will be quickly reproduced
AI research is a world-wide community

The majority of good ideas will come from Academia


Even if the most impressive applications come from industry

It is important to distinguish intelligence from autonomy


Most intelligent systems will not be autonomous.
Y LeCun

Thank you
PREDICTIVE
LEARNING

You might also like