Linear Regression with PyTorch
Your first step towards deep learning
In my previous posts we have gone through
Here we will try to solve the classic linear regression problem using pytorch tensors.
1 What is Linear regression ?
y = Ax + B.
A = slope of curve
B = bias (point that intersect y-axis)
y=target variable
x=feature variable
1.1 Linear regression example 1:
We’ll create a model that predicts crop yields for apples and oranges (target variables) by looking at the average temperature, rainfall and humidity (input variables or features) in a region. Here’s the training data:
yield_apple = w11 * temp + w12 * rainfall + w13 * humidity + b1
yield_orange = w21 * temp + w22 * rainfall + w23 * humidity + b2
Learning rate of linear regression is to find out these w,b values to make accurate predictions on unseen data. This optimization technique for linear regression is gradient descent which slightly adjusts weights many times to make better predictions.Below is the matrix representation
1.1.1 import Libraries & load data set:
First we are importing necessary libraries , we are passing inputs and targets as numpy arrays first and then converting them into tensor.
#import Libraries
import torch
import torch.nn as nn
import pandas as pd
import numpy as np# Input (temp, rainfall, humidity)
inputs = np.array([[73, 67, 43], [91, 88, 64], [87, 134, 58],
[102, 43, 37], [69, 96, 70], [73, 67, 43],
[91, 88, 64], [87, 134, 58], [102, 43, 37],
[69, 96, 70], [73, 67, 43], [91, 88, 64],
[87, 134, 58], [102, 43, 37], [69, 96, 70]],
dtype='float32')# Targets (apples, oranges)
targets = np.array([[56, 70], [81, 101], [119, 133],
[22, 37], [103, 119], [56, 70],
[81, 101], [119, 133], [22, 37],
[103, 119], [56, 70], [81, 101],
[119, 133], [22, 37], [103, 119]],
dtype='float32')inputs = torch.from_numpy(inputs)
targets = torch.from_numpy(targets)inputs
targets
1.1.2 Data set & Data Loader :
Here we are creating Tensor data set which allows us to access as tuples through indexing operation. Here our Tensor data set contains first element as input variables and second elements as target variables.
Data loader helps us to split data into batches and we can split and shuffle them , which helps in randomizing the input to optimization where faster reduction of loss takes place.
# Tensor Data set
from torch.utils.data import TensorDataset
# Define dataset
train_ds = TensorDataset(inputs, targets)
train_ds[0:3]#Data loader
from torch.utils.data import DataLoader
# Define data loader
batch_size = 5
train_dl = DataLoader(train_ds, batch_size, shuffle=True)
1.1.3 Define model , loss and optimizer:
- We are defining model using nn.linear class. In nn.Linear(3,2) input dimension is 3,output dimension 2 ie number of input variables (temp,rainfall,humidity), output variables(apples,oranges)in our example.
- list(model.parameters) will return list of weights and bias matrices.
- torch.nn.functional module contains builtin loss function mean squared error loss which is mse_loss
- torch.optim.SGD stands for Stochastic Gradient Descent optimizer, module.parameters is passed so that SGD optimizer knows which matrices to be modified during each step. lr=le-5 stands for learning rate which controls the amount by which parameters are modified.
# Define linear model
model = nn.Linear(3, 2)
print(model.weight)
print(model.bias)# Parameters
list(model.parameters())# Define Loss
import torch.nn.functional as F
loss_fn = F.mse_loss# Define optimizer
opt = torch.optim.SGD(model.parameters(), lr=1e-5)
1.1.4 Train & fit model , generate predictions:
The process flow is as follows
1.Generate predictions
2.Calculate the loss
3.Compute gradients w.r.t the weights and biases
4.Adjust the weights by subtracting a small quantity proportional to the gradient
5.Reset the gradients to zero
Let us understand code line by line.
- Define a utility function fit which takes num_epochs,model,loss_fn,train_dl as arguments.
- To repeat for a given number of epochs we are creating for loop.
- To get batches of data for every iteration we use data loader.
- generate predictions by passing xb into model.
- calculate loss defined in loss_fn which takes pred,yb as arguments.
- loss.backward() computes dloss/dx for every parameter x which has requires_grad=True. These are accumulated into x.grad for every parameter x.
- opt.step() used to update the parameters weights & biases
- opt.zero_grad() resets the gradients to zero
- The log statement prints the progress on every 10 th epoch
- fit(100, model, criterion , optimizer ,train_dl) Training the model for 100 epochs using the utility function we defined
- preds=model(inputs) generate predictions
# Utility function to train the model
def fit(num_epochs, model, loss_fn, opt, train_dl):
# Repeat for given number of epochs
for epoch in range(num_epochs):
# Train with batches of data
for xb,yb in train_dl:
# 1. Generate predictions
pred = model(xb)
# 2. Calculate loss
loss = loss_fn(pred, yb)
# 3. Compute gradients
loss.backward()
# 4. Update parameters using gradients
opt.step()
# 5. Reset the gradients to zero
opt.zero_grad()
# Print the progress
if (epoch+1) % 10 == 0:
print('Epoch [{}/{}], Loss: {:.4f}'.format(epoch+1, num_epochs, loss.item()))#fit model for 100 epochs
fit(100, model, criterion , optimizer ,train_dl)# Generate predictions
preds = model(inputs)
preds
1.2 Linear regression example 2:
We will create a linear regression model and train it on given set of observations of experiences and respective salary and will try to predict salaries for new set of experiences. A simple linear regression problem.
1.2.1 Import libraries & load data:
Importing libraries and reading the data from csv file as pandas data frame.Visualizing our data set as a scatter plot using matplotlib.
#Import libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
import torch
import torch.nn as nn#Loading the data set
data=pd.read_csv('/content/exp_vs_salary_Data.csv')#let us try to visualize our data
data.head()# lets visualize our data
import matplotlib.pyplot as plt
plt.scatter(data.YearsExperience,data.Salary)
plt.xlabel("YearsExperience")
plt.ylabel("Salary")
plt.title("Years Experience VS Salary")
plt.show()
1.2.2 Train and test split and tensor creation:
Here we are splitting the data set into train and test data set with 80:20.Converting these train and test data sets onto pytorch tensors x_train,y_train,x_test,y_test.
#Splitting the dataset into training and testing dataset
train, test = train_test_split(data, test_size = 0.2)#Converting training data into tensors for Pytorch
X_train = torch.Tensor([[x] for x in list(train.YearsExperience)])
y_train = torch.torch.FloatTensor([[x] for x in list(train.Salary)])#Converting test data into tensors for Pytorch
X_test = torch.Tensor([[x] for x in list(test.YearsExperience)])
y_test = torch.torch.FloatTensor([[x] for x in list(test.Salary)])
1.2.3 Tensor data set & Data set loader:
As we seen in previous example we are using tensor data set and data loader to pass the data set
#Data set & Data set Loader
from torch.utils.data import TensorDataset
train_data=TensorDataset(X_train, y_train)
train_data[0:5]# Define data loader
from torch.utils.data import DataLoader
batch_size = 5
train_dl = DataLoader(train_data, batch_size, shuffle=True)
1.2.3 Define model,loss,optimizer
Define linear model using nn.Linear where input dimension,output dimension is passed as parameters. Mean squared error is the loss function. SGD optimizer with a learning rate of 0.01 is set.
# Define model
model = nn.Linear(1, 1) # nn.Linear(in_features,out_features)
print(model.weight)
print(model.bias)# printing the model Parameters
print(list(model.parameters()))#Define the loss function
loss_fun = nn.MSELoss()# Define SGD optimizer with learning rate 0.01
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
alternatively we can define a linear regressor as below.Let us understand this class in detail.
- Creating a LinearRegression class that inherits nn.Module.
- __init__ method for constructor is invoked when an object is created for the class.
- super().__init__() initializes the constructor of parent class.
- Creating object for PyTorch’s Linear class with parameters in_features and out_features.
- Defining the forward function for passing the inputs to the regressor object initialized by the constructor returns predicted values
class LinearRegression(nn.Module):
def __init__(self, in_size, out_size):
super().__init__()
self.lin = nn.Linear(in_features = in_size, out_features = out_size)
def forward(self, X):
pred = self.lin(X)
return(pred)1.2.4 Train & fit model , generate predictions:
Training the model for 100 epochs and creating predictions
# Utility function to train the model
def lrmodel(num_epochs, model, loss_fun, optimizer, train_dl):
# Repeat for given number of epochs
for epoch in range(num_epochs):
# Train with batches of data
for xb,yb in train_dl:
# 1. Generate predictions
pred = model(xb)
# 2. Calculate loss
loss = loss_fun(pred, yb)
# 3. Compute gradients
loss.backward()
# 4. Update parameters using gradients
optimizer.step()
# 5. Reset the gradients to zero
optimizer.zero_grad()
# Print the progress
if (epoch+1) % 10 == 0:
print('Epoch [{}/{}], Loss: {:.4f}'.format(epoch+1, num_epochs, loss.item()))#Training for 100 epochs
num_epochs=100
lrmodel(num_epochs, model, loss_fun, optimizer, train_dl)# Generate predictions
preds = model(X_train)
preds# Compare with targets
y_train
1.2.5 Predicting for unseen data
x_test is passed on to the model and predictions is compared with actual values which will give us the accuracy of our model on unseen data.
#Predicting for X_test
y_pred_test = model(X_test)#Converting predictions from tensor objects into a listy_pred_test = [y_pred_test[x].item() for x in range(len(y_pred_test))]
# Comparing Actual and predicted values
df = {}
df['Actual Observation'] = y_test
df['Predicted Salary'] = y_pred_test
df = pd.DataFrame(df)
print(df)
In my next post I will be discussing on logistic regression using pytorch tensors.
Please find the github link for the code — https://github.com/Arun-purakkatt/Deep_Learning_Pytorch
Stay connected : https://www.linkedin.com/in/arun-purakkatt-mba-m-tech-31429367/
Credits & References :
