Anand Uthaman
Published © GPL3+

Touch-less Display Interfaces on Edge

Enabling HCI on Edge: Multi-threaded Gesture & Sound Control of kiosks with Intel OpenVINO. Eye Wink & Mouth Aspect with numerical models

IntermediateFull instructions providedOver 2 days5,230

Things used in this project

Hardware components

Raspberry Pi 3 Model B
Raspberry Pi 3 Model B
This is an optional component. You can use Intel PC or Laptop for kiosk implementation.
Movidius Neural Compute Stick
Intel Movidius Neural Compute Stick
This is an optional component used along with RPi. You can use Intel PC or Laptop for kiosk implementation.
Camera Module
Raspberry Pi Camera Module
This is an optional component used along with RPi and Movidius NCS stick. You can use lappy webcam itself.
Laptop with Intel CPU and Webcam
Developed the project in this setup.
Headphones, 20 Hz
Headphones, 20 Hz
Mic is needed only if you want to use Control Mode 3: Sound Control

Software apps and online services

OpenVINO toolkit
Intel OpenVINO toolkit
Python 3.6
Python 3.6 and its libraries, espcially PyAutoGUI for navigation
OpenVINO Models
These are the OpenVINO Models used: i) Detailed Facial Landmark Detection: "facial-landmarks-35-adas-0002", ii) Head Pose Estimation: "head-pose-estimation-adas-0001", iii) Gaze Estimation: "gaze-estimation-adas-0002", iv) Face Detection: "face-detection-adas-0001", v) Speech Recognition: OpenVINO Inference Engine plugin , OpenVINO Feature Extraction Library, OpenVINO Decoder Library.
Mathematical Models
These are the custom mathematical models you can select to use: i) Inverse Simoid Curve Fitting using Non-Linear Least Squares, ii) Mouth Aspect Ratio derived from EAR concept from a Research Paper [3], iii) Peak Finding Algorithm, iv) Statistical Analysis.
Shell Script

Hand tools and fabrication machines

Intel VTune
Used to find hotspots and optimize the project code.
Math Visualization Tool
Mobile as Light Source


Read more

Custom parts and enclosures

Evaluation Criteria Writeup for Judge Panel

Made this document to address each point as given in Evaluation Criteria.


Architecture Diagram

Architecture Diagram to visualize data flow across OpenVINO models, numerical models and the parallel thread for all the 4 Control Modes.

OpenVINO Download Proof

Requirement for Challenge.


Project Touch-Me-Not: Touch-less Display Interfaces on Edge

You can watch the Video Description and Project Demo here: Detailed Project Blog here: How to use the project is written in the GitHub Readme.


Anand Uthaman

Anand Uthaman

2 projects • 32 followers