Scribd
Upload
3 views5 pages

Regression Models

A regression model estimates the relationship between a dependent variable and one or more independent variables, helping to answer questions about their interactions. Common types of regression algorithms include Linear, Polynomial, Ridge, Lasso, and others, each suited for different data characteristics and analysis needs. The document also provides example Python code for implementing various regression techniques using libraries like sklearn.

Uploaded by

Bhavya Kolluru
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views5 pages

Regression Models

A regression model estimates the relationship between a dependent variable and one or more independent variables, helping to answer questions about their interactions. Common types of regression algorithms include Linear, Polynomial, Ridge, Lasso, and others, each suited for different data characteristics and analysis needs. The document also provides example Python code for implementing various regression techniques using libraries like sklearn.

Uploaded by

Bhavya Kolluru
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

What is a Regression Model?

A regression model estimates the relationship between one dependent variable (target) and one
or more independent variables (features). It helps answer questions like:

 How does the number of hours studied affect exam scores?


 How does advertising budget influence sales?

Common Types of Regression Algorithms

Algorithm Description When to Use


Models a straight-line relationship
Simple relationships; fast and
1. Linear Regression between dependent and independent
interpretable.
variables.
2. Polynomial Fits a polynomial equation (e.g., When data shows a curved
Regression quadratic, cubic) to the data. trend.
Linear regression with L2
3. Ridge Regression When multicollinearity exists.
regularization to reduce overfitting.
Linear regression with L1
For feature selection and
4. Lasso Regression regularization; can shrink coefficients
sparse models.
to zero.
Combines L1 and L2 regularization When you want benefits of
5. Elastic Net
(Ridge + Lasso). both Ridge and Lasso.
Used for classification, not actual Binary classification problems
6. Logistic Regression
regression. (e.g., spam or not spam).
7. Support Vector Uses support vector machines to Complex relationships with
Regression (SVR) predict continuous values. high-dimensional data.
8. Decision Tree Predicts outcomes by learning decision Non-linear relationships, easy
Regression rules from data. to visualize.
9. Random Forest An ensemble of decision trees; reduces High accuracy, non-linear
Regression variance. problems, handles outliers.
10. Gradient Boosting Builds models sequentially to correct When accuracy is top priority
Regression previous errors. (but slower).
11. K-Nearest
Predicts based on average of k nearest No assumptions on data
Neighbors (KNN)
neighbors. distribution.
Regression
12. XGBoost / Advanced boosting algorithms with Kaggle-level solutions; very
LightGBM Regression high performance. fast and accurate.
from sklearn.linear_model import LinearRegression

from sklearn.model_selection import train_test_split

from sklearn.metrics import mean_squared_error

# Example data

X = [[5], [10], [15], [20]] # Hours studied

y = [50, 60, 70, 80] # Exam scores

# Split data

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

# Model

model = LinearRegression()

model.fit(X_train, y_train)

# Prediction

y_pred = model.predict(X_test)

# Evaluation

print("MSE:", mean_squared_error(y_test, y_pred))


Polynomial Regression – Python Code

import numpy as np

import matplotlib.pyplot as plt

from sklearn.linear_model import LinearRegression

from sklearn.preprocessing import PolynomialFeatures

from sklearn.metrics import mean_squared_error, r2_score

# Example data: hours studied vs exam score (non-linear relationship)

X = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).reshape(-1, 1)

y = np.array([20, 25, 30, 38, 48, 60, 75, 90, 105, 120])

# Create polynomial features (degree = 2)

poly = PolynomialFeatures(degree=2)

X_poly = poly.fit_transform(X)

# Fit model

model = LinearRegression()

model.fit(X_poly, y)

# Predict

y_pred = model.predict(X_poly)

plt.scatter(X, y, color='blue', label='Actual')

plt.plot(X, y_pred, color='red', label='Polynomial Fit (Degree 2)')

plt.title('Polynomial Regression Example')

plt.xlabel('Hours Studied')

plt.ylabel('Exam Score')

plt.legend()
plt.grid(True)

plt.show()

mse = mean_squared_error(y, y_pred)

r2 = r2_score(y, y_pred)

print("Mean Squared Error:", mse)

print("R² Score:", r2)

Ridge

import numpy as np

import matplotlib.pyplot as plt

from sklearn.linear_model import Ridge

from sklearn.model_selection import train_test_split

from sklearn.metrics import mean_squared_error, r2_score

# Split the dataset

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Ridge Regression with alpha = 1.0

ridge_model = Ridge(alpha=1.0)

ridge_model.fit(X_train, y_train)

# Predict

y_pred = ridge_model.predict(X_test)

mse = mean_squared_error(y_test, y_pred)

r2 = r2_score(y_test, y_pred)
print("Coefficients:", ridge_model.coef_)

print("Intercept:", ridge_model.intercept_)

print("Mean Squared Error:", mse)

print("R² Score:", r2)

import numpy as np

from sklearn.linear_model import Lasso

from sklearn.model_selection import train_test_split

from sklearn.metrics import mean_squared_error, r2_score

# Split the dataset

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Train Lasso model (alpha = regularization strength)

lasso_model = Lasso(alpha=0.1)

lasso_model.fit(X_train, y_train)

# Predict

y_pred = lasso_model.predict(X_test)

mse = mean_squared_error(y_test, y_pred)

r2 = r2_score(y_test, y_pred)

print("Coefficients:", lasso_model.coef_)

print("Intercept:", lasso_model.intercept_)

print("Mean Squared Error:", mse)

print("R² Score:", r2)

You might also like