Yet Another Line Detection Project with OpenCV

In my previous story, I told about PiTanq — a robot-tank I built. Then a big goal is to make it autonomous. I was inspired by Udacity course for self-driving cars and the first task of this course is to recognize lanes on roads.

As my robot is pretty far from hitting a road I put a white tape on the floor.

The idea was to implement tracking a line based on lanes detection approach from the Udacity course.

A general solution for this task would look like:

• Filter an image by color
• Find edges with Canny edge detector
• Crop irrelevant parts of the image
• Detect lines with Hough transform

Filter image by color

Udacity course task requires to detect both yellow and white lines. For that purpose, they use HSV or HLS conversion. I need to detect only a white line so I decided to use only a grayscale filter.

Next step I picked a threshold to have a binary image:

Detect the line

Then, applying Canny edge detector, I got that picture (inside a region of interest):

Using Hough line detector I found some odd lines:

Making detection rules more restrictive:

Not good in terms of relevance (and reliability as well).

Spent some time with these experiments I was not able to get satisfying results. Then I decided to try another approach. Instead of lines detection, I used contours detection. Assuming, the biggest contour is the line and taking a centerline of the bounding box, I got a plausible direction.

Light problem

The next problem I encountered with, was different light conditions on the line. One side of the line turned out in the shadow of a couch and it was impossible to find a grayscale threshold working along the whole line loop.

The solution was to adjust the threshold individually for each image depending on a ratio of white pixels to the whole area.

def balance_pic(image):
   global T
   ret = None
   direction = 0
   for i in range(0, tconf.th_iterations):
       rc, gray = cv.threshold(image, T, 255, 0)
       crop = Roi.crop_roi(gray)
       nwh = cv.countNonZero(crop)
       perc = int(100 * nwh / Roi.get_area())
       if perc > tconf.white_max:
           if T > tconf.threshold_max:
               break
           if direction == -1:
               ret = crop
               break
           T += 10
           direction = 1
       elif perc < tconf.white_min:
           if T < tconf.threshold_min:
               break
           if  direction == 1:
               ret = crop
               break
           T -= 10
           direction = -1
       else:
           ret = crop
           break  
   return ret      

Make driving decisions

Based on computer vision technics we got a direction to move. The real decisions were made depending on the angle of this vector and its shift from the image middle point.

Determine turn actions (if required):

def check_shift_turn(angle, shift):
   turn_state = 0
   if angle < tconf.turn_angle or angle > 180 - tconf.turn_angle:
       turn_state = np.sign(90 - angle)
   shift_state = 0
   if abs(shift) > tconf.shift_max:
       shift_state = np.sign(shift)
   return turn_state, shift_state
def get_turn(turn_state, shift_state):
   turn_dir = 0
   turn_val = 0
   if shift_state != 0:
       turn_dir = shift_state
       turn_val = tconf.shift_step if shift_state != turn_state else tconf.turn_step
   elif turn_state != 0:
       turn_dir = turn_state
       turn_val = tconf.turn_step
   return turn_dir, turn_val                

Do self-driving:

       while(True):
           a, shift = get_vector()
           if a is None:
                # there is some code to recover the line if lost
                break
           turn_state, shift_state = check_shift_turn(a, shift)
           turn_dir, turn_val = get_turn(turn_state, shift_state)
           if turn_dir != 0:
               turn(turn_dir, turn_val)
               last_turn = turn_dir
           else:
               time.sleep(tconf.straight_run)
               last_turn = 0
           last_angle = a

Results

There is a debug visual info:

Algorithm settings

## Picture settings
# initial grayscale threshold
threshold = 120
# max grayscale threshold
threshold_max = 180
#min grayscale threshold
threshold_min = 40
# iterations to find balanced threshold
th_iterations = 10
# min % of white in roi
white_min=3
# max % of white in roi
white_max=12
## Driving settings
# line angle to make a turn
turn_angle = 45
# line shift to make an adjustment
shift_max = 20
# turning time of shift adjustment
shift_step = 0.125
# turning time of turn
turn_step = 0.25
# time of straight run
straight_run = 0.5
# attempts to find the line if lost
find_turn_attempts = 5
# turn step to find the line if lost
find_turn_step = 0.2
# max N of iterations of the whole tracking
max_steps = 100