After having lived with both dogs and a cat in my life, I have found out an interesting difference beween them: a dog understands something's off-limits when you tell them, but a cat overfits and learns that something's off-limits while you're present. Indeed, when there's no one around it can do whatever it wants without being scolded, which in my case results in my cat walking around on the kitchen countertop and in one ocasion unlocking the cooktop with its tail, which can obviously lead to a dangerous situation.
To redress this I decided to build a system which would discourage him from walking around the countertop even when I'm not looking. Enter the Nerf gun. After sticking two servos to the base of the gun to create a pan and tilt, I attached a third mini servo with a zip tie around the trigger to shoot and mounted a webcam over the barrel. The darts needn't make contact, just being shot in the cat's vicinity is enough for him to jump down instantly, so I didn't bother accounting for parallax or camera calibration.
The program builds on qqwwee's keras-yolo3 project, itself based on the YOLO localization algorithm. It first applies a canny edge detector and a Hough lines transform to detect the longest lines, from which it filters out the non-horizontal lines. Since both the camera and the countertop are in fixed places and only the pan axis is really needed, only the long, horizontal line found in the appropriate region of the image is a valid candidate for the countertop edge. The mean y-value of this line is then calculated to set the threshold above which the cat would be in an illegal position.
After this the image is run through the YOLO detector, which returns the position, if found, of the center of gravity of the class being searched for. This point will be where the cat is assumed to be. Both the center of gravity and the vertical threshold line are drawn on the image, as well as the vertical lines setting the limits for the panning movement. Once the cat is found, if its center is above the counter line, the tracking system will move until the point is centered in the image. Then a 3-second countdown starts to make sure the cat has stopped moving before shooting a dart in its general direction.
To install the software, first create a conda environment using the provided package list and after activating it execute the installation script. This will download qqwwee's project, as well as the official Tiny YOLOv3 trained weights, modify some of the files and finally convert the whole into a model supported by Keras. A configuration file contains other important parameters, like the class to detect or the confidence threshold.
The video below shows the system in motion. Since I couldn't really test it (ie. deliberately put my cat on the counter, the exact behavior I'm trying to get rid of) I simulated its behavior by placing it just in front (and not putting any darts in the gun) and then again by removing the counter detection and enabling a full 2D tracking and shooting upon myself.
The code is available on GitHub, except for the embedded part, as a weak safeguard to stop people who don't already know how to do it from creating automated guns.
Post imported from http://www.alvaroferran.com/blog/feline-learning
Comments