Scribd
Upload
7 views

Lecture 02

Uploaded by

r.duzinho10
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
7 views

Lecture 02

Uploaded by

r.duzinho10
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Machine Learning Tribes

Master in Data Science and Advanced


Analytics
BA and DS majors

Roberto Henriques

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
How do computers get new knowledge?

▪ Fill in gaps in existing knowledge

▪ Emulate the brain

▪ Simulate evolution

▪ Reduce uncertainty

▪ Check for similarities between old and new

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Tribes of Machine Learning

▪ Associated to each of these ways, we have Machine Learning areas


▪ Symbolism
▪ Connectionism
▪ Evolutionary Computation
▪ Statistical Learning
▪ Analogy Modelling

The Master Algorithm: How the


Quest for the Ultimate Learning
Machine Will Remake Our World
• Pedro Domingos, 2015
• ISBN:978-0465065707

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Tribes of Machine Learning

Tribe Origins Master Algorithm


Symbolists Logic, philosophy Inverse deduction
Connectioninsts Neuroscience Backpropagation
Evolutionaries Evolutionary biology Genetic programming
Bayesians Statistics Probabilistic inference
Analogizers Psycology Kernel Machines (SVM)

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Symbolists

▪ As in mathematical operations

Addition 2+2=?

Subtraction 2+ ? = 4

▪ Inverse deduction (induction)

Deduction Induction

Socrates is human Socrates is human


+ Humans are mortal + ??
=? = Socrates is mortal

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Symbolists

▪ Logic programming
▪ Prolog
▪ Inductive and Abductive logic programming

▪ Expert systems
▪ Knowledge bases, Inference Engines,

▪ Decision Trees
▪ C4.5 the most popular, but has many variants, including regression trees and
random forests

▪ Functional programming – Lisp …

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Connectionists

▪ World is not logic!


▪ Draws on Biology, more specificially Biological Neurons
▪ Model neural cells, McCullor and Pitts (1943)
▪ Milestones
▪ 1957: Perceptron invented by Frank Rosenblatt.
▪ 1968: Minsky and Papert publish the book Perceptrons, criticizing single-layer
perceptrons.
▪ 1986: Backpropagation (re)invented, and connectionist research restarts
▪ 2006: Hinton et al. publish A fast learning algorithm for deep belief nets, which
rejuvinates interest in Deep Learning. Important points of Deep Learning success
and transfer learning
▪ 2017: AlphaGo defeats reigning Go world champion, using DL

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Connectionists

▪ Main algorithms
▪ Perceptron (simple perceptron)
▪ Multilayer Perceptron (MLP)
▪ Varying structures, including feedback and time-delay
▪ DeepLearning Networks
▪ Convolutional Neural Networks
▪ Hopfield Networks
▪ Boltzman Networks

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Connectionists: MLP

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Evolutionary

▪ Draws on Biology, the Theory of Evolution and Natural Selection


▪ Models natural selection, crossover, mutations

▪ Milestones:
▪ 1964: Lawrence Fogel published on evolutionary programming
▪ 1975: John Holland invents genetic algorithms.
▪ 1981: Richard Forsyth coins genetic programming and demonstrates it using
trees of actions.
▪ 1991: Dorigo writes a thesis on Ant Colony Optimization
▪ 1995: Kennedy & Eberhart invent Particle Swarm Optimization

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Evolutionaries: GA and GP

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Bayesians

▪ Draws on Statistics
▪ Bayes Theorem, and evidence

▪ Everything is uncertainty, so compute probabilities


▪ Milestones:
▪ 1953: Monte Carlo Markov Chain (MCMC) invented. Bayesian inference
finally becomes tractable on real problems.
▪ 1968: Hidden Markov Model (HMM) invented.
▪ 1988: Judea Pearl authors Probabilistic Reasoning in Intelligent Systems, and
creates the discipline of probabilistic graphical models (PGMs).
▪ 2000: Pearl authors Causality: Models, Reasoning, and Inference, and creates
the discipline of causal inference on PGMs
Instituto Superior de Estatística e Gestão de Informação
Universidade Nova de Lisboa
Bayesians: algorithms

▪ Bayesian Classifiers
▪ NB, MAP, ML

▪ Bayesian Belief Networks

▪ Variants of Markov Models

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa https://luminousmen.com/media/data-science-bayes-theorem-2.jpg
Analogizers

▪ Everything we learn is reasoning by analogy

▪ Draws on Human behaviour and Statistics

▪ Milestones:
▪ 1968: k-nearest neighbor algorithm increases in popularity.
▪ 1979: Douglas Hofstadter publishes Godel, Escher, Bach.
▪ 1982. Kohonen’s book on Self-Organization and Associative Memory
▪ 1992: Support Vector Machines (SVMs) invented

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Analogizers: kNN and SVM

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Can we unify all tribes?

Tribe Origins Master Algorithm


Symbolists Logic, philosophy Inverse deduction
Connectioninsts Neuroscience Backpropagation
Evolutionaries Evolutionary biology Genetic programming
Bayesians Statistics Probabilistic inference
Analogizers Psycology Kernel Machines (SVM)

Instituto Superior de Estatística e Gestão de Informação


Universidade Nova de Lisboa
Components of ML

▪ Representation
▪ Probabilistic logic (Markov logic Networks)
▪ Each rule has a weight

▪ Evaluation
▪ Posterior probability
▪ User-defined objective function

▪ Optimization
▪ Formula discovery: Genetic programming
▪ Weight learning: Backprop

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World Pedro Domingos, 2015
Instituto Superior de Estatística e Gestão de Informação
Universidade Nova de Lisboa
Instituto Superior de Estatística e Gestão de Informação
Universidade Nova de Lisboa

You might also like