This Device Lets You Type Using Hand Gestures

While some people are said to “talk with their hands,” until now this has been meant figuratively, or as a means of conveying extra information or emphasis. On the other hand (no pun intended), there’s no reason that these gestures can’t be translated into actual text using machine learning.

Federico Terzi’s latest project demonstrates this very idea, as seen in the video below where he spells out, “this is a test youtube,” by waving his Wiimote-esque controller around.

His device uses an Arduino Pro Micro along with an accelerometer to send hand motion data to a computer via Bluetooth or USB. The computer then uses a custom Python library taking advantage of the SVM (Support Vector Machine) algorithm to convert the readings into a sequence of letters and sentences.