Logistic Regression | Logistic Regression In Python | Machine Learning Algorithms | Simplilearn
22 likes5,237 views
The document serves as an introduction to machine learning basics, particularly focusing on supervised learning and the classification algorithm logistic regression. It explains the differences between linear and logistic regression, their applications, and includes practical examples like predicting the survival of Titanic passengers and identifying numbers in images. Additionally, it covers concepts like decision trees, k-nearest neighbors, and provides a mathematical understanding of logistic regression through its equations.
Logistic Regression | Logistic Regression In Python | Machine Learning Algorithms | Simplilearn
- 2. Surviving the Titanic Suppose, you have to build a model to predict how many people survived the Titanic shipwreck
- 3. Surviving the Titanic Suppose, you have to build a model to predict how many people survived the Titanic shipwreck
- 4. Surviving the Titanic • ID • Survived • Class • Name • Sex • Age • Siblings • Parents/children abroad • Ticket • Fare • Cabin • Place of Embarkment Teaching the model with the passenger dataset Dropping the non-essential components of the dataset Determining the survival of passengers and evaluating the model
- 5. Agenda What is Supervised Learning? What is Classification? What are some of its solutions? What is Logistic Regression? Comparing Linear and Logistic Regression Logistic Regression applications Use Case – Predicting the number in an image
- 6. What is Supervised Learning?
- 7. What is Supervised Learning? That’s an apple! apple Teacher teaches child Child recognizes an apple when she sees it again A model is able to make predictions based on past data
- 8. Where does Logistic Regression fit it? Machine Learning Supervised Learning Unsupervised Learning AssociationClusteringClassification Regression The systems predicts future outcomes based on training from past input
- 10. A few Classification Solutions A B We take decisions using a tree structure. Each branch node represents a choice, and leaf node represents a decision Decision Trees
- 11. A few Classification Solutions A B Decision Trees We take decisions using a tree structure. Each branch node represents a choice, and leaf node represents a decision It helps determine what the given object is, based on its similarity to the objects it is compared toK=3 K=7 K-Nearest Neighbor
- 12. It helps determine what the given object is, based on its similarity to the objects it is compared toK=3 K=7 K-Nearest Neighbor Decision Trees A few Classification Solutions A B We determine the probability of an event occurring with the help of a tree structure A dataset with one or more independent variables is used to determine binary output of the dependent variable Logistic Regression
- 13. What is Logistic Regression?
- 14. What is Logistic Regression? Imagine it’s been a few years since you serviced your car. One day you wonder…
- 15. What is Logistic Regression? It is a classification algorithm, used to predict binary outcomes for a given set of independent variables. The dependent variable’s outcome is discrete. Regression model created based on other users’ experience 0.60 0.20 0.40 0.80 1.00 1 2 3 4 5 6 Years since service Probabilityofbreakdown How long until the car breaks down? You provide years since last service
- 16. What is Logistic Regression? Probability>0.50 Value rounded off to 1 : The car will breakdown Probability<0.50 Value rounded off to 0: The car will not breakdown Here, the threshold value 0.50 indicates that the car is more likely to breakdown after 3.5 years of usage Model makes predictions 0.60 0.20 0.40 0.80 1.00 1 2 3 4 5 6 Years since last service Probabilityofbreakdown 0.50 0.29 0.90 Threshold Value
- 17. What is Linear Regression?
- 18. Linear Regression It is a statistical method that helps find the relationship between an independent and dependent variable, both of which are continuous He performed well in the last quarter. How much raise should he get?
- 19. Linear Regression 41 2 3 5 5 10 15 20 25 Employee rating Salaryhike Collection of ratings and corresponding hikes Linear Regression is performed on data The management provides the corresponding salary hike Employee rating
- 20. Linear and Logistic Regression Here’s the graph of how linear regression would be, for a given scenario x y
- 21. Linear and Logistic Regression What if you wanted to know whether the employee would get a promotion or not based on their rating 41 2 3 5 0 =No 1=Yes Employee rating Probabilityofgettinga promotion
- 22. Linear and Logistic Regression This graph would not be able to make such a prediction. So we clip the line at 0 and 1. 41 2 3 5 Employee rating 0 =No 1=Yes Probabilityofgettinga promotion
- 23. Linear and Logistic Regression So, how did this… …become this? 41 2 3 5 Employee rating 41 2 3 5 Employee rating 0 =No 1=Yes Probabilityofgettinga promotion 0 =No 1=Yes Probabilityofgettinga promotion
- 24. The Math behind Logistic Regression To understand Logistic Regression, let’s talk about the odds of success Odds (𝜃) = Probability of an event happening Probability of an event not happening or 𝜃 = 𝑝 1 − 𝑝 The values of odds range from 0 to ∞ The values of probability change from 0 to 1
- 25. The Math behind Logistic Regression Type equation here. Take the equation of the straight line 𝛽0 x y Here, 𝛽0 is the y-intercept 𝛽1 is the slope of the line x is the value of the x co-ordinate y is the value of the prediction The equation would be: 𝑦 = 𝛽0 + 𝛽1 𝑥
- 26. The Math behind Logistic Regression Type equation here. Now, we predict the odds of success e𝑙𝑛 𝑝 𝑥 1 − 𝑝 𝑥 = e 𝛽0+𝛽1 𝑥 log 𝑝 𝑥 1−𝑃 𝑥 = 𝛽0 + 𝛽1 𝑥 Exponentiating both sides: 𝑝 𝑥 1 − 𝑝 𝑥 = e 𝛽0+𝛽1 𝑥 Let Y = e 𝛽0+𝛽1 𝑥 Then 𝑝 𝑥 1−𝑝 𝑥 = Y 𝑝 𝑥 = 𝑌 1 − 𝑝 𝑥 𝑝 𝑥 = 𝑌 − 𝑌 𝑝 𝑥 𝑝 𝑥 + 𝑌 𝑝 𝑥 = 𝑌 𝑝 𝑥 1 + 𝑌 = 𝑌 𝑝 𝑥 = 𝑌 1 + 𝑌 𝑝 𝑥 = e 𝛽0+𝛽1 𝑥 1 + e 𝛽0+𝛽1 𝑥 The equation of a sigmoid function: 𝑝 𝑥 = e 𝛽0+𝛽1 𝑥 1 + e 𝛽0+𝛽1 𝑥 𝑝 𝑥 = 1 1 + e−(𝛽0+𝛽1 𝑥)
- 27. The Math behind Logistic Regression Type equation here. Now, we predict the odds of success e𝑙𝑛 𝑝 𝑥 1 − 𝑝 𝑥 = e 𝛽0+𝛽1 𝑥 log 𝑝 𝑥 1−𝑃 𝑥 = 𝛽0 + 𝛽1 𝑥 Exponentiating both sides: 𝑝 𝑥 1 − 𝑝 𝑥 = e 𝛽0+𝛽1 𝑥 Let Y = e 𝛽0+𝛽1 𝑥 Then 𝑝 𝑥 1−𝑝 𝑥 = Y 𝑝 𝑥 = 𝑌 1 − 𝑝 𝑥 𝑝 𝑥 = 𝑌 − 𝑌 𝑝 𝑥 𝑝 𝑥 + 𝑌 𝑝 𝑥 = 𝑌 𝑝 𝑥 1 + 𝑌 = 𝑌 𝑝 𝑥 = 𝑌 1 + 𝑌 𝑝 𝑥 = e 𝛽0+𝛽1 𝑥 1 + e 𝛽0+𝛽1 𝑥 The equation of a sigmoid function: 𝑝 𝑥 = e 𝛽0+𝛽1 𝑥 1 + e 𝛽0+𝛽1 𝑥 𝑝 𝑥 = 1 1 + e−(𝛽0+𝛽1 𝑥) 41 2 3 5 0 0.25 0.50 0.75 1 Employee rating Probabilityofgettinga promotion A sigmoid curve is obtained!
- 28. Comparing Linear and Logistic Regression
- 29. How is Linear and Logistic Regression different? Linear Regression Logistic Regression • Used to solve Regression Problems
- 30. How is Linear and Logistic Regression different? Linear Regression Logistic Regression • Used to solve Classification Problems • Used to solve Regression Problems
- 31. How is Linear and Logistic Regression different? Linear Regression Logistic Regression • Used to solve Classification Problems • Used to solve Regression Problems • The response variables are continuous in nature
- 32. How is Linear and Logistic Regression different? Linear Regression Logistic Regression • Used to solve Classification Problems • The response variable is categorical in nature • Used to solve Regression Problems • The response variables are continuous in nature
- 33. How is Linear and Logistic Regression different? Linear Regression Logistic Regression • Used to solve Classification Problems • The response variable is categorical in nature • Used to solve Regression Problems • The response variables are continuous in nature • It helps estimate the dependent variable when there is a change in the independent variable.
- 34. How is Linear and Logistic Regression different? Linear Regression Logistic Regression • Used to solve Classification Problems • The response variable is categorical in nature • It helps calculate the possibility of a particular event taking place. • Used to solve Regression Problems • The response variables are continuous in nature • It helps estimate the dependent variable when there is a change in the independent variable.
- 35. How is Linear and Logistic Regression different? Linear Regression Logistic Regression • Used to solve Classification Problems • The response variable is categorical in nature • It helps calculate the possibility of a particular event taking place. • Used to solve Regression Problems • The response variables are continuous in nature • It helps estimate the dependent variable when there is a change in the independent variable. • Is a straight line.
- 36. How is Linear and Logistic Regression different? Linear Regression Logistic Regression • Used to solve Classification Problems • The response variable is categorical in nature • It helps calculate the possibility of a particular event taking place. • An S-curve. (S = Sigmoid) • Used to solve Regression Problems • The response variables are continuous in nature • It helps estimate the dependent variable when there is a change in the independent variable. • Is a straight line.
- 37. Logistic Regression Applications Weather Prediction Helps determine the kind of weather that can be expected
- 38. Identifies the different components that are present in the image, and helps categorize them Logistic Regression Applications Humans Animals Vehicles Image Categorization
- 39. Determines the possibility of patient survival, taking age, ISS and RTS into consideration Logistic Regression Applications Healthcare (TRISS) Patient survival % Revised Trauma Score Injury Severity Score Age
- 40. Use Case – Predicting numbers in images
- 41. Use Case – Predicting numbers Can you guess what number I am? Are you a 3? 4? I don’t know! 8x8 image
- 42. Use Case – Predicting numbers Dividing the data set Training set Test set The model being trained Model identifies number in images Test set applied A number 4 A number 1
- 43. Use Case – Implementation Importing libraries and their associated methods Determining the total number of images and labels
- 44. Use Case – Implementation Displaying some of the images and labels
- 45. Use Case – Implementation Dividing dataset into Training and Test set
- 46. Use Case – Implementation Import the Logistic Regression model Making an instance of the model and training it Predicting the output of the first element of the test set Predicting the output of the first 10 elements of the test set
- 47. Use Case – Implementation Predicting for the entire dataset Determining the accuracy of the model Representing the confusion matrix in a heat map
- 48. Use Case – Implementation Accurately predicting the image to contain a zero Inaccurately predicting the image to contain a seven
- 49. Use Case – Implementation Presenting predictions and actual output
- 50. Use Case – Predicting numbers Dividing the data set Training set Test set The model being trained Model identifies number in images Test set applied A number 4 A number 2
- 51. Here’s What You’ve Learnt so Far
- 52. So what’s your next step?