Webcam Deep Learning Filters

Inspiration

I'm a software geek who solves his problems with more software. Often times creating more problems than were originally at hand...

For this project, my problem was that I was finding myself constantly in video calls, and wanted to give my significant other more privacy by accurately segmenting myself in the video.

Features
The reason this project is marked as "Advanced" isn't because I managed to do simple image segmentation to do background removal. It's because it showcases the OpenVisionCapsules runtime, which I am a contributor for.

With this project, you can easily download and drag in any capsule file (for example, from Aotu's capsule library) that is capable of Detection, and the VisionCapsules runtime will load it up onto the appropriate hardware, run the capsules in parallel, and produce a video stream with only detections (or segmentation) within it. You can even plug in a Myriad X, and it will intelligently choose that hardware over CPU! No configuration required.

For example, below I had loaded the capsule "segmentation_person_openvino"

And here I have loaded "detector_face_openvino"

Final Product
Using Zoom's built-in background remover (terrible in my complex bedroom environment):

Using this project with the "segmenter_person_openvino" capsule loaded:

Next Steps
I want more programmers people to view computer vision as a goal within reach. If I can convince some programmers that with this capsule system they can drag-and-drop a couple 'features' into this zoom background remover, that would be a win in my books.

Project Ideas going forward:
1) Use "recognizer_face_openvino" to ensure that only I am ever shown in the video- for ultimate privacy for my significant other.

2) Use "landmarks_face_openvino" to detect facial landmarks, and then 'align' the face. That way my boss will have a harder time knowing that I'm actually looking down at my phone 😁

3) Add gesture recognition. What if I could make it so that it only shows me when I'm doing a hand gesture?

Have fun hacking!