Homebreadcumb forward arrow iconBlogbreadcumb forward arrow iconArtificial Intelligencebreadcumb forward arrow iconLinear Regression Using Gradient Descent Method [Explained With Coding Example]

Linear Regression Using Gradient Descent Method [Explained With Coding Example]

Last updated:
21st Dec, 2020
Read Time
5 Mins
share image icon
In this article
Chevron in toc
View All
Linear Regression Using Gradient Descent Method [Explained With Coding Example]

Machine learning is still making rounds no matter whether you are aspiring to be a software developer, data scientist, or data analyst. To make serious efforts in linear regression, you must be well versed with Python. Getting started on an initial phase might be a tedious task, this article will help you understand regression more thoroughly.

Top Machine Learning and AI Courses Online

The gradient descent method is opted in various iterations because of the optimization techniques it has to offer. With the algorithm, it is feasible to reduce the size, for example, Logistic regression and neural network. Before starting off with gradient let’s just have a look over Linear regression.

Read: Machine Learning Algorithms for Data Science

Ads of upGrad blog

What is Linear Regression?

It is the linear approach that is taken towards the modelling of the relationship between a dependent variable and one or more independent variables. The linear relationship between such variables would be expressed in a y= mx+b equation format. 

It is a monitored machine learning algorithm that will enhance its learning curve from a given x dependent variable and y as the other responsible for causing an effect. This nature helps predict the values and certainty factors for x and y.

Trending Machine Learning Skills

Enrol for the Machine Learning Course from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

What is Gradient Descent?

The use of gradient regression involves optimizing the algorithm used to find the values of required parameters of a function which enables to directly minimize the cost of a function.

 Let us understand the concept with a scenario, imagine you want to descend a fort in a pitch dark surrounding. During this let’s assume that you are completely handicapped and need to come up with the shortest and easiest distance to come back down. Gradient descent will be the resource used to find out the optimized way to reach your destination. With a basic directional input, the algorithm would be possible to chart and suggest the best route.

The gradient is one the most used and widely accepted algorithms in machine learning it is also considered to lay the foundation to mastering machine learning in the earlier stages.

For a better approximation of gradient let’s try to implement it with a code in a sample, working on python with the help of NumPy.

 from NumPy import *

 # y = mx + b

 # m is slope, b is the y-intercept

 def compute_error_for_line_given_points(b, m, points):

   totalError = 0

   for i in range(0, len(points)):

   x = points[i, 0]

   y = points[i, 1]

   totalError += (y – (m * x + b)) ** 2

   return totalError / float(len(points))

   def step_gradient(b_current, m_current, points, learningRate):

   b_gradient = 0

   m_gradient = 0

   N = float(len(points))

   for i in range(0, len(points)):

   x = points[i, 0]

   y = points[i, 1]

   b_gradient += -(2/N) * (y – ((m_current * x) + b_current))

   m_gradient += -(2/N) * x * (y – ((m_current * x) + b_current))

   new_b = b_current – (learningRate * b_gradient)

   new_m = m_current – (learningRate * m_gradient)

   return [new_b, new_m]

def gradient_descent_runner(points, starting_b, starting_m, learning_rate, num_iterations):

   b = starting_b

   m = starting_m

   for i in range(num_iterations):

   b, m = step_gradient(b, m, array(points), learning_rate)

   return [b, m]

   def run():

   points = genfromtxt(“data.csv”, delimiter=”,”)

   learning_rate = 0.0001

   initial_b = 0 # initial y-intercept guess

   initial_m = 0 # initial slope guess

   num_iterations = 1000

   print “Starting gradient descent at b = {0}, m = {1}, error = {2}”.format(initial_b, initial_m, compute_error_for_line_given_points(initial_b, initial_m, points))

   print “Running…”

   [b, m] = gradient_descent_runner(points, initial_b, initial_m, learning_rate, num_iterations)

   print “After {0} iterations b = {1}, m = {2}, error = {3}”.format(num_iterations, b, m, compute_error_for_line_given_points(b, m, points))

   if __name__ == ‘__main__’:


Code reference

This is a visual representation of the gradient search program where the problems are solved in the linear regression by plotting the points in a single line. The code is a demonstration of how it works and helps to set several points along a line. Gradient descent attempts to find the best values for these parameters concerning an error function.

The code contains a peculiar function labeled ‘run’. It helps define a set of parameters used in the algorithm to make an initial set of predictions based on the behavior of the variables and the line slope. The other factors involve the number of iterations required to achieve the gradient descent in the format shown below:

initial_b = 0 # initial y-intercept guess

initial_m = 0 # initial slope guess

num_iterations = 1000

You can easily come to an understanding that the Gradient method is quite simple and straightforward. Once you understand its functioning abilities then the only part you need to focus on is the cost of function that you are interested in optimizing. 

The goal is to make continuous efforts to make different iterations for each of the values of the variables, to evaluate their costs, and to create new variables that would initiate a better and low cost in the program.

Must Read: Machine Learning Interview Questions

Tips for Gradient Descent

1. Learning Rate

The optimization protocol helps to reduce the learning rate value even at smaller decimals, try to shuffle different values suitable for the platform, and then opt for the best working value. The learning can be much faster and fruitful, to make that happen make sure to limit the number of passes through each dataset. The ideal number would be between 1 to 10.

2. Plot Mean Cost

Ads of upGrad blog

The training time for each dataset instance can cause a delay due to the extra time taken during running the algorithm.  For better results choose the average over 100 or 1000 for better odds of finding a better learning trend for the algorithm.

Popular AI and ML Blogs & Free Courses


In this article you learned about gradient and how to create such an algorithm, this helps to make precise and more effective predictions with a learned regression model. To understand on a much comprehensive and deeper level with real case scenarios, enroll with upGrad. We offer curated courses specially structured for aspiring Data Scientists and Machine Learning applicants.


Pavan Vadapalli

Blog Author
Director of Engineering @ upGrad. Motivated to leverage technology to solve problems. Seasoned leader for startups and fast moving orgs. Working on solving problems of scale and long term technology strategy.
Get Free Consultation

Select Coursecaret down icon
Selectcaret down icon
By clicking 'Submit' you Agree to  
UpGrad's Terms & Conditions

Our Popular Machine Learning Course

Explore Free Courses

Suggested Blogs

Data Preprocessing in Machine Learning: 7 Easy Steps To Follow
Summary: In this article, you will learn about data preprocessing in Machine Learning: 7 easy steps to follow. Acquire the dataset Import all the cr
Read More

by Kechit Goyal

29 Oct 2023

Natural Language Processing (NLP) Projects & Topics For Beginners [2023]
What are Natural Language Processing Projects? NLP project ideas advanced encompass various applications and research areas that leverage computation
Read More

by Pavan Vadapalli

04 Oct 2023

15 Interesting MATLAB Project Ideas & Topics For Beginners [2023]
Learning about MATLAB can be tedious. It’s capable of performing many tasks and solving highly complex problems of different domains. If youR
Read More

by Pavan Vadapalli

03 Oct 2023

Top 16 Artificial Intelligence Project Ideas & Topics for Beginners [2023]
Summary: In this article, you will learn the 16 AI project ideas & Topics. Take a glimpse below. Predict Housing Price Enron Investigation Stock
Read More

by Pavan Vadapalli

27 Sep 2023

Top 15 Deep Learning Interview Questions & Answers
Although still evolving, Deep Learning has emerged as a breakthrough technology in the field of Data Science. From Google’s DeepMind to self-dri
Read More

by Prashant Kathuria

21 Sep 2023

Top 8 Exciting AWS Projects & Ideas For Beginners [2023]
AWS Projects & Topics Looking for AWS project ideas? Then you’ve come to the right place because, in this article, we’ve shared multiple AWS proj
Read More

by Pavan Vadapalli

19 Sep 2023

Top 15 IoT Interview Questions & Answers 2023 – For Beginners & Experienced
These days, the minute you indulge in any technology-oriented discussion, interview questions on cloud computing come up in some form or the other. Th
Read More

by Kechit Goyal

15 Sep 2023

45+ Interesting Machine Learning Project Ideas For Beginners [2023]
Summary: In this Article, you will learn Stock Prices Predictor Sports Predictor Develop A Sentiment Analyzer Enhance Healthcare Prepare ML Algorith
Read More

by Jaideep Khare

14 Sep 2023

Why GPUs for Machine Learning? Ultimate Guide
In the realm of modern technology, the convergence of data and algorithms has paved the way for groundbreaking advancements in artificial intelligence
Read More

by Pavan Vadapalli

14 Sep 2023

Schedule 1:1 free counsellingTalk to Career Expert
footer sticky close icon