Arrow detection using Erle-Brain 2
The idea of this project came to my mind one day I was wondering why we could not control Erle-Rover using just some arrows placed in the office. As Erle-Brain 2 has an 5MP camera, image processing can be used to command the robot!
In order to process the images taken, I thought OpenCV could be great, because is really popular and you can use a simple python script to make this little project :D
Take a picture
First of all we need to take a picture to process it. To do so, just create a python script and paste the next lines:
import time
import picamera
# import the necessary packages
import numpy as np
import argparse
import cv2
# Explicitly open a new file called my_image.jpg
my_file = open('stream.jpg', 'wb')
with picamera.PiCamera() as camera:
camera.start_preview()
time.sleep(2)
camera.capture(my_file)
# At this point my_file.flush() has been called, but the file has
# not yet been closed
my_file.close()
This small script first imports all the needed libraries: picamera library is used to handle the camera and to obtain a picture from it. cv2 is the python library of OpenCV.
Once you launch this script, you should have an picture called "stream.jpg" store in the location where you're launching it. This is the image camera has taken:
As you can see, I have drawn an orange arrow, my purpose now is just to detect it. In order to do so, first a color filter is applied, taken a range of oranges as inputs.
This filter taken from OpenCV library:
# load the image
image = cv2.imread('stream2.jpg')
# find all the 'orange' shapes in the image
upper = np.array([90, 110, 200])
lower = np.array([30, 50, 130])
shapeMask = cv2.inRange(image, lower, upper)
new_file=open('filtered.jpg','wb')
cv2.imwrite('filtered.jpg',shapeMask)
new_file.close()
The function used is inRange and it needs an image and two color ranges to process the image. The output is in gray scale, where it will color in white the part of the image that contains the color ranges defined.
So is important to select properly the "upper" and "lower" colors that orange color of the picture has. If you want to know which BGR (OpenCV uses RGB colors in inverted order) has the arrow, just open the taken picture with an image processing software and check which RGB color code has:
Take the minimum and maximum RGB values to define "upper" and "lower" BGR values. Tune the values for a proper detection.
Once the filter values are set up, check the output of the filter:
The next step is to detect the triangle and rectangle within the arrow.
One of the most popular methods is to know how many edges the shape has in order to know which one is. For example, a triangle has three, a square four, an octagon eight, ...
Before deciding which shape contains the image Canny filter is applied. This is a popular filter detects the edges and reduces the noise of the image:
gray=cv2.bilateralFilter(shapeMask,11,17,17)
edged=cv2.Canny(gray,30,200)
gray_file=open('edge.jpg','wb')
cv2.imwrite('edge.jpg',edged)
gray_file.close()
This is the result:
Now we're ready to check shapes. To do so, findContours OpenCV function is used:
(cnts, _) = cv2.findContours(edged.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts= sorted (cnts,key = cv2.contourArea, reverse=True)[:10]
screenCnt=None
flag_t=False
flag_s=False
for c in cnts:
peri = cv2.arcLength(c,True)
approx = cv2.approxPolyDP(c, 0.02 * peri, True)
if len (approx)==3:
triangle=approx
print "triangle"
print triangle
flag_t=True
if len (approx)==4:
square=approx
print "square"
print square
flag_s=True
break
print "no square found"
#Draw square
if flag_t==True:
cv2.drawContours(shapeMask, [triangle], -1, (0,255,0), 3)
if flag_s==True:
cv2.drawContours(shapeMask, [square], -1, (0,255,0), 3)
fin=open('processed.jpg','wb')
cv2.imwrite('processed.jpg',shapeMask)
fin.close()
This last piece if code tries to detect edges and compare if finds arrows or squares. If so, draws green shapes around them and prints in the command line what has found.
This is the output I have in the command line:
square
[[[485 293]]
[[485 298]]
[[487 298]]
[[487 293]]]
Square has been detected. But not the triangle...
And this is what .drawContours fills:
In conclusion, this script needs to be improved but the potential of OpenCV and Erle Brain 2 is shown in this project. You could use this library to make a lot of cool stuff and implement decision taking algorithms based on images taken from the camera of Erle Brain 2.
Here you have a short explanation and a demo:
Comments