In my previous story, I told about PiTanq — a robot-tank I built. Then a big goal is to make it autonomous. I was inspired by Udacity course for self-driving cars and the first task of this course is to recognize lanes on roads.
As my robot is pretty far from hitting a road I put a white tape on the floor.
The idea was to implement tracking a line based on lanes detection approach from the Udacity course.
A general solution for this task would look like:
- Filter an image by color
- Find edges with Canny edge detector
- Crop irrelevant parts of the image
- Detect lines with Hough transform
Udacity course task requires to detect both yellow and white lines. For that purpose, they use HSV or HLS conversion. I need to detect only a white line so I decided to use only a grayscale filter.
Next step I picked a threshold to have a binary image:
Then, applying Canny edge detector, I got that picture (inside a region of interest):
Using Hough line detector I found some odd lines:
Making detection rules more restrictive:
Not good in terms of relevance (and reliability as well).
Spent some time with these experiments I was not able to get satisfying results. Then I decided to try another approach. Instead of lines detection, I used contours detection. Assuming, the biggest contour is the line and taking a centerline of the bounding box, I got a plausible direction.
The next problem I encountered with, was different light conditions on the line. One side of the line turned out in the shadow of a couch and it was impossible to find a grayscale threshold working along the whole line loop.
The solution was to adjust the threshold individually for each image depending on a ratio of white pixels to the whole area.
def balance_pic(image): global T ret = None direction = 0 for i in range(0, tconf.th_iterations): rc, gray = cv.threshold(image, T, 255, 0) crop = Roi.crop_roi(gray) nwh = cv.countNonZero(crop) perc = int(100 * nwh / Roi.get_area()) if perc > tconf.white_max: if T > tconf.threshold_max: break if direction == -1: ret = crop break T += 10 direction = 1 elif perc < tconf.white_min: if T < tconf.threshold_min: break if direction == 1: ret = crop break T -= 10 direction = -1 else: ret = crop break return ret
Based on computer vision technics we got a direction to move. The real decisions were made depending on the angle of this vector and its shift from the image middle point.
Determine turn actions (if required):
def check_shift_turn(angle, shift): turn_state = 0 if angle < tconf.turn_angle or angle > 180 - tconf.turn_angle: turn_state = np.sign(90 - angle) shift_state = 0 if abs(shift) > tconf.shift_max: shift_state = np.sign(shift) return turn_state, shift_state def get_turn(turn_state, shift_state): turn_dir = 0 turn_val = 0 if shift_state != 0: turn_dir = shift_state turn_val = tconf.shift_step if shift_state != turn_state else tconf.turn_step elif turn_state != 0: turn_dir = turn_state turn_val = tconf.turn_step return turn_dir, turn_val
while(True): a, shift = get_vector() if a is None: # there is some code to recover the line if lost break turn_state, shift_state = check_shift_turn(a, shift) turn_dir, turn_val = get_turn(turn_state, shift_state) if turn_dir != 0: turn(turn_dir, turn_val) last_turn = turn_dir else: time.sleep(tconf.straight_run) last_turn = 0 last_angle = a
There is a debug visual info:
## Picture settings # initial grayscale threshold threshold = 120 # max grayscale threshold threshold_max = 180 #min grayscale threshold threshold_min = 40 # iterations to find balanced threshold th_iterations = 10 # min % of white in roi white_min=3 # max % of white in roi white_max=12 ## Driving settings # line angle to make a turn turn_angle = 45 # line shift to make an adjustment shift_max = 20 # turning time of shift adjustment shift_step = 0.125 # turning time of turn turn_step = 0.25 # time of straight run straight_run = 0.5 # attempts to find the line if lost find_turn_attempts = 5 # turn step to find the line if lost find_turn_step = 0.2 # max N of iterations of the whole tracking max_steps = 100