Building a lane detection system
with Python 3 & OpenCV
In this tutorial, we will learn how to build a software pipeline for tracking road lanes using computer vision techniques.
challenges in road lane detection ?
The lines on the road that show us where the lanes are act as our constant reference. We are using canny detector-Hough transform based lane detection.
The following diagram is an overview of our pipe line.
The Approach:
Canny Edge detector needs grey scale images, hence we need to convert our image into grey scale. We are collapsing 3 channels of pixel value (Red, Green, and Blue) into a single channel with a pixel value range of [0,255].
Creating a Gaussian blur over our grey scale image , this is not mandatory as in canny detector will do this step for us.
Let us understand canny edge detector, You can find the detailed documentation here. Here is the code snippet.
By reading several articles I found that , each of these preprocessing steps are data set dependent.
Lane lines are always yellow and white. Yellow can be a tricky color to isolate in RGB space, so lets convert instead to Hue Value Saturation or HSV color space. You can find a target range for yellow values by a Google search. The ones I used are below. Next, we will apply a mask to the original RGB image to return the pixels we’re interested in.
Setting up Environment
Make sure that you have opencv installed. Install numpy & matplotlib libraries as we needs them on processing.
pip install opencv-pythonimport libraries
import cv2
import matplotlib.pyplot as plt
import numpy as npPreprocessing of image
Greyscale image: complexity of gray level images is lower than that of color image. We can talk about lot of features of images brightness, contrast, edges, shape, contours, texture, perspective, shadows, and so on, without addressing color. After presenting a gray-level image model , it can be extended to color images.
Gaussian Filter: The purpose of the gaussian filter is to reduce noise in the image. We do this because the gradients in Canny are really sensitive to noise, so we want to eliminate the most noise possible.
cv2.GaussianBlur parameters: img,ksize,sigma
img:Image which we are going to take
ksize:dimension of the kerenel which we convolute over the image.
sigma: defines the standard deviation along x axis.
canny-edge detection: Fundamental idea is to detect sharp changes in luminosity such as shift from black to white, white to black & will define it as edges. Noise reduction, Intensity Gradient, Non-Maximum suppression, Hysteresis thresholding. It have 3 parameters.
- The img parameter defines the image that we’re going to detect edges on.
- The threshold-1 parameter filters all gradients lower than this number (they aren’t considered as edges).
- The threshold-2 parameter determines the value for which an edge should be considered valid.
- Any gradient in between the two thresholds will be considered if it is attached to another gradient who is above threshold-2.
#convert into grey scale image
def grey(image):
image=np.asarray(image)
return cv2.cvtColor(image,cv2.COLOR_RGB2GRAY)#Gaussian blur to reduce noise and smoothen the image
def gauss(image):
return cv2.GaussianBlur(image,(5,5),0)#Canny edge detection
def canny(image):
edges = cv2.Canny(image,50,150)
return edges
This is how our image looks after canny edge detection.
So we could see that it contains all edges in our image & we should isolate edges of lane lines.
Now that we’ve defined all the edges in the image, we need to isolate the edges that correspond with the lane lines. Here’s how we’re going to do that.
def region(image):
height, width = image.shape
triangle = np.array([
[(100, height), (475, 325), (width, height)]
])
mask = np.zeros_like(image)
mask = cv2.fillPoly(mask, triangle, 255)
mask = cv2.bitwise_and(image, mask)
return maskThis function will isolate a certain hard-coded region in the image where the lane lines are. It takes one parameter, the Canny image and outputs the isolated region.
In line 1, we’re going to extract the image dimensions using the numpy.shape function.
In line 2–4, we’re going to define the dimensions of a triangle, which is the region we want to isolate.
In line 5 and 6, we’re going to create a black plane and then we’re going to define a white triangle with the dimensions that we defined in line 2.
In line 7, we’re going to perform the bitwise and operation which allows us to isolate the edges that correspond with the lane lines.
Hough line transform:
This one line of code is the heart of the whole algorithm. It is called Hough Transform, the part that turns those clusters of white pixels from our isolated region into actual lines.
lines = cv2.HoughLinesP(isolated, rho=2, theta=np.pi/180, threshold=100, np.array([]), minLineLength=40, maxLineGap=5)Parameter 1: The isolated gradients
Parameter 2 and 3: Defining the bin size, 2 is the value for rho and np.pi/180 is the value for theta
Parameter 4: Minimum intersections needed per bin to be considered a line (in our case, its 100 intersections)
Parameter 5: Placeholder array
Parameter 6: Minimum Line length
Parameter 7: Maximum Line gap
Optimizing and Displaying lines
To average the lines, we’re going to define a function called “average”.
def average(image, lines):
left = []
right = []
for line in lines:
print(line)
x1, y1, x2, y2 = line.reshape(4)
parameters = np.polyfit((x1, x2), (y1, y2), 1)
slope = parameters[0]
y_int = parameters[1]
if slope < 0:
left.append((slope, y_int))
else:
right.append((slope, y_int))This function averages out the lines made in the cv2.HoughLinesP function. It will find the average slope and y-intercept of the line segments on the left and the right and output two solid lines instead (one on the left and other on the right).
In the output of the cv2.HoughLinesP function, each line segment has 2 coordinates: one denotes the start of the line and the other marks the end of the line. Using these coordinates, we’re going to calculate the slopes and y-intercepts of each line segment.
Then, we’re going to collect all the slopes of the line segments and classify each line segment into either the list corresponding with the left line or the right line (negative slope = left line, positive slope = right line).
- Line 4: Loop through the array of lines
- Line 5: Extract the (x, y) values of the 2 points from each line segment
- Line 6–9: Determine the slope and y-intercept of each line segment.
- Line 10–13: Add the negative slopes to the list for the left lines and the positive slope to the list with the right lines.
NOTE: Normally, a positive slope=left line and a negative slope=right line but in our case, the image’s y-axis is inversed, hence the reason why the slopes are inversed (all images in OpenCV have inversed y-axes).
Next, we have to take the average of the slopes and y-intercepts from both lists.
right_avg = np.average(right, axis=0)
left_avg = np.average(left, axis=0)
left_line = make_points(image, left_avg)
right_line = make_points(image, right_avg)
return np.array([left_line, right_line])- Lines 1–2: Takes the average of all the line segments on for both lists (the left side and the right side).
- Lines 3–4: Calculates the start point and endpoint for each line. (We’ll define make_points function in the next section)
- Line 5: Output the 2 coordinates for each line
Now that we have the average slope and y-intercept for both lists, let’s define the start and endpoints for both lists.
def make_points(image, average):
slope, y_int = average
y1 = image.shape[0]
y2 = int(y1 * (3/5))
x1 = int((y1 — y_int) // slope)
x2 = int((y2 — y_int) // slope)
return np.array([x1, y1, x2, y2])This function takes 2 parameters, the image with the lane lines and the list with the average slope and y_int of the line, and outputs the starting and ending points for each line.
- Line 1: Define the function
- Line 2: Get the average slope and y-intercept
- Line 3–4: Define the height of the lines (the same for both left and right)
- Lines 5–6: Calculate x coordinates by rearranging the equation of a line, from y=mx+b to x = (y-b) / m
- Line 7: Output the sets of coordinates
Just to elaborate a bit further, in line 1, we use the y1 value as the height of the image. This is because in OpenCV, the y-axis is inverted, so the 0 is at the top and the height of the image is at the origin (refer to the image below).
Also, in Line 2, we multiplied y1 by 3/5. This is because we want the line to start at the origin (y1) and end 2/5 up the image (it’s 2/5 since the y-axis is invested, instead of 3/5 up from 0, we see 2/5 down from the max height).
However, this function does not display the lines, it only calculates the points necessary to display these lines. Next, we’re going to create a function which takes these points and makes lines out of them.
def display_lines(image, lines):
lines_image = np.zeros_like(image)
if lines is not None:
for line in lines:
x1, y1, x2, y2 = line
cv2.line(lines_image, (x1, y1), (x2, y2), (255, 0, 0), 10)
return lines_imageThis function takes in two parameters: the image which we want to display the lines on and the lane lines which were outputted from the average function.
- Line 2: create a blacked-out image with the same dimensions of our original image
- Line 3: Make sure that the lists with the line points aren’t empty
- Line 4–5: Loop through the lists, and extract the two pairs of (x, y) coordinates
- Line 6: Create the line and paste it onto the blacked-out image
- Line 7: Output the black image with the lines
You may be wondering, why don’t we append the lines onto the real image instead of a black image. Well, the raw image is a little too bright, so it would be nice if we’d darken it a bit to see the lane lines a little more clearly (yes, I know, it’s not that big of a deal, but it’s always nice to find ways to make the algorithm better)
Left: Appending Lines to Image Directly. Right: Using the cv2.addWeighted function
lanes = cv2.addWeighted(copy, 0.8, black_lines, 1, 1)This function gives a weight of 0.8 to each pixel in the actual image, making them slightly darker (each pixel is multiplied by 0.8). Likewise, we give a weight of 1 to the blacked-out image with all the lane lines, so all the pixels in that keep the same intensity, making it stand out.
copy = np.copy(image1)
grey = grey(copy)
gaus = gauss(grey)
edges = canny(gaus,50,150)
isolated = region(edges)lines = cv2.HoughLinesP(isolated, 2, np.pi/180, 100, np.array([]), minLineLength=40, maxLineGap=5)
averaged_lines = average(copy, lines)
black_lines = display_lines(copy, averaged_lines)
lanes = cv2.addWeighted(copy, 0.8, black_lines, 1, 1)
cv2.imshow("lanes", lanes)
cv2.waitKey(0)Here, we simply call all the functions that we previously defined, then we output the result on lines 12. The cv2.waitKey function is used to tell the program how long to display the image for. We passed “0” into the function, meaning it will wait until a key is pressed to close the output window.
Here’s what the output looks like
Here, we simply call all the functions that we previously defined, then we output the result on lines 12. The cv2.waitKey function is used to tell the program how long to display the image for. We passed “0” into the function, meaning it will wait until a key is pressed to close the output window.
Here’s what the output looks like
