Introduction to Machine Learning
- 1. HANDS-ON ONLINE TRAINING ON DATA SCIENCE KNOWLEDGE AND SKILLS FORUM INTRODUCTION TO MACHINE LEARNING FAITHFUL ONWUEGBUCHE OLUWASEUN ODEYEMI AUGUSTINE OKOLIE
- 2. WHAT IS MACHINE LEARNING • Machine learning is a type of artificial intelligence (AI) that enables computers to learn and make decisions without being explicitly programmed. • Using algorithms that iteratively learn from data, machine learning allows computers to find hidden insights without being explicitly programmed where to look.
- 3. THINK AND LEARN LIKE A BABY
- 4. MACHINE LEARNING VS. STATISTICS VS. COMPUTER SCIENCE Machine Learning Statistics Computer Science Objective Focuses on learning from data to make predictions or decisions without being explicitly programmed. It prioritizes prediction accuracy and generalizability. Aims to infer properties of an underlying distribution from a data sample. It emphasizes understanding and interpreting data and probabilistic models. Focuses on the creation and application of algorithms to manipulate, store, and communicate digital information. Methodologie s Typically uses complex models (like neural networks) and large amounts of data to train models for prediction. Utilizes both supervised and unsupervised learning methods. Often employs simpler, more interpretable models. Focuses on hypothesis testing, experimental design, estimation, and mathematical analysis. Involves algorithm design, data structures, computation theory, computer architecture, software development, and more. Validation Measures model performance through methods like cross- validation and seeks to improve generalization to unseen data. Validates models using methods such as confidence intervals, p- values, and hypothesis tests to quantify uncertainty. Uses formal methods for verifying correctness, analyzing computational complexity, and proving algorithmic bounds. Primary Concern Creating models that can learn from and make decisions or predictions based on data. Drawing valid conclusions and quantifying uncertainty about observed data and underlying distributions. Creating efficient algorithms and data structures to solve computational problems.
- 5. TYPES OF MACHINE LEARNING Three Types of Problems • Supervised • Unsupervised • Reinforcement
- 6. SUPERVISE D • Trained using labeled examples • Desired output is known • Methods include classification, regression, etc. • Uses patterns to predict the values of the label on additional unlabeled data • Algorithms: • Linear regression • Logistic regression • K-Nearest Neighbors (KNN) • Decision Trees and Random Forests • Support Vector Machines • Naive Bayes • Neural Networks
- 7. UNSUPERVIS ED • Used against data that has no historical labels • Desired output is unknown • Goal is to explore the data and find some structure within the data • Algorithms • Anomaly detection • K-means clustering • Hierarchical clustering • DBSCAN • Principal Component Analysis (PCA) • Neural Networks
- 8. REINFORCEMEN T • Algorithm discovers through trial and error which actions yield the greatest rewards. • Three primary components: • the agent (the learner or decision maker), • the environment (everything the agent interacts with) • actions (what the agent can do). • Objective: the agent chooses actions that maximize the expected reward over a given amount of time. • Algorithms • Markov Decision Process • Q-Learning • Deep Q Network (DQN)
- 9. WHY USE IT? • Machine learning based models can extract patterns from massive amounts of data which humans cannot do because • We cannot retain everything in memory or we cannot perform obvious/redundant computations for hours and days to come up with interesting patterns. • “Humans can typically create one or two good models in a week; machine learning can create thousands of models in a week” (Thomas H. Davenport) • Solve problems we simply could not before
- 10. USE CASES • Email spam filter • Recommendation systems • Self driving car • Finance • Image Recognition • Competitive machines
- 11. TYPICAL MACHINE LEARNING PROCESS Source: INTRODUCING AZURE MACHINE LEARNING, pg. 5
- 12. TO GIVE CREDIT, OR NOT TO GIVE CREDIT • You are asked by your boss while working at Big Bank Inc. to develop an automated decision maker on whether to give a potential client credit or not.
- 13. WHAT IS THE QUESTION? • What question are we trying to answer here? What problem are we looking to solve?
- 14. SELECTING DATA Feature • An individual measurable property of a phenomenon being observed • Best found through industry experts
- 15. SELECTING DATA Feature Extraction • Feature extraction is a general term for methods of constructing combinations of the variables to get around certain problems while still describing the data with sufficient accuracy. • Analysis with a large number of variables generally requires a large amount of memory and computation. • Reducing the amount of resources required to describe a large set of data
- 16. SELECTING DATA PCA (Principal ComponentAnalysis) • We have a huge list of different features • Many of them will measure related properties and so will be redundant • Summarize with less features
- 17. PREPARING DATA • Cleaning • Units • Missing Values • Metadata
- 18. DEVELOPING MODEL • What is the problem being solved? • What is the goal of the model? • Minimize error on the “training” data • Training data is the data used to train the model (all of it but the part we removed)
- 19. DEVELOPING MODEL Linear Model • Relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data • Complex way of saying the model draws a line between two categories (classification) or to estimate a value (regression) • Linear regression the most common form of linear model
- 20. DEVELOPING MODEL Non-linear Model • A nonlinear model describes nonlinear relationships in experimental data • The parameters can take the form of an exponential, trigonometric, power, or any other nonlinear function
- 21. DEVELOPING MODEL Overfitting vs. Bias in Machine Learning Overfitting Bias Definition Overfitting occurs when a model fits the data more than is warranted. It captures the noise along with the underlying pattern in the data. Bias is error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss relevant relations between features and target outputs. Consequence Overfitting leads to a smaller error on the training data set but a larger one on unseen data, reducing the model's ability to generalize. High bias often leads to underfitting, where the model oversimplifies the data and doesn't capture its complexity. Example Creating a very complex decision tree that classifies each training instance perfectly, but performs poorly on unseen data. Fitting a quadratic dataset using a linear model – the model will consistently fail to capture the true relationship and make errors.
- 22. DEVELOPING MODEL Overfitting vs. Bias in Machine Learning
- 24. DEVELOPING MODEL Rule Addition • Minimize error on the “training” data • AND make sure that error in the “unseen” data is close to error in the “training” data
- 25. DEVELOPING A MODEL • Keep it Simple • Go for simpler models over more complicated models • Generally, the fewer parameters that you have to tune the better • Cross-Validation • K-fold cross validation is a great way to estimate error on training data • Regularization • Can sometimes help penalize certain sources of overfitting. • LASSO • Forces the sum of the absolute value of coefficients to be less than a fixed value • Effectively choosing a simpler model
- 26. DEVELOPIN G A MODEL Data Snooping or Data Dredging • It's a form of bias that arises when you make decisions based on the same data you've used to train and test your model. • “If a data set has affected any step in the learning process, its ability to access the outcome has been compromised” • Experimenting • Reuse of the same data set to determine quality of model • Once a data set has been used to test the performance of a data set, it should be considered contaminated Source: Learning From Data, pg. 1
- 27. INTERPRETING RESULTS • Validation • Cross validation • Test set • Once the test set has been used, you must find new data!
Editor's Notes
- #6: Coin: that’s a quarter, that’s a dime Unsupervised: That’s a cluster, that’s another cluster Renforcement:
- #8: Unsupervised learning works well on transactional data. For example, it can identify segments of customers with similar attributes who can then be treated similarly in marketing campaigns. Or it can find the main attributes that separate customer segments from each other. Popular techniques include self-organizing maps, nearest-neighbor mapping, k-means clustering and singular value decomposition. These algorithms are also used to segment text topics, recommend items and identify data outliers.
- #9: The agent will reach the goal much faster by following a good policy. So the goal in reinforcement learning is to learn the best policy.
- #14: In WWII, Allied bombers were key to strategic attacks, yet these lumbering giants were constantly shot down over enemy territory. The planes needed more armor, but armor is heavy. So extra plating could only go where the planes were being shot the most. A man named Abraham Wald, a Jewish mathematician who’d been locked out of university positions and ultimately fled the persecution in his own home country of Hungary, was brought in to oversee the operation. He started with a simple diagram—the outline of a plane—and he marked bullet holes corresponding to where each returning bomber had been shot. The result was the anatomy of common plane damage. The wings, nose, and tail were blackened with bullet holes, so these were the spots that needed more armor.
- #19: Draw on board
- #21: Draw example on board, why ever use linear then?
- #25: Draw example on board
- #26: More data