RFTattoo Soft RFID Tattoo System Processes Facial Movements to Regenerate Speech
Applied to the face, this RFID tattoo system allows those who cannot speak to simply mouth words and have them recognized with 86% accuracy.
Researchers from Carnegie Mellon University and the Indian Institute of Technology (IIT) have developed radio-frequency identification (RFID) tattoos with a difference: They're using them to provide an assistive technology for those with voice impairments like dysphonia.
"This paper presents an RF-based assistive technology for voice impairments (i.e., dysphonia), which occurs in an estimated 1% of the global population. We specifically focus on acquired voice disorders where users continue to be able to make facial and lip gestures associated with speech," the researchers write of their work. "Despite the rich literature on assistive technologies in this space, there remains a gap for a solution that neither requires external infrastructure in the environment, battery-powered sensors on skin or body-worn manual input devices."
"We present RFTattoo, which to our knowledge is the first wireless speech recognition system for voice impairments using battery-less and flexible RFID tattoos. We design specialized wafer-thin tattoos attached around the user’s face and easily hidden by makeup. We build models that process signal variations from these tattoos to a portable RFID reader to recognize various facial gestures corresponding to distinct classes of sounds. We then develop natural language processing models that infer meaningful words and sentences based on the observed series of gestures."
The RFTattoo system is built using customised soft antennas made from stretchable conductors, connected to three RFID chips. Each is tuned for the 900MHz Industrial, Scientific, and Medical (ISM) radio band, licence-free in many countries, while off-the-shelf RFID readers are placed on the user's waist.
Each antenna and RFID is attached to the wearer using stickers, then covered with makeup, and each is asked to mouth a series of words. Radio-frequency backscatter is measured at the readers as the antennas stretch and move, and a natural language processing framework converts the readings into words.
"A detailed user study with 10 users," the researchers write, two of whom were suffering from a temporary loss of voice, "reveals 86 percent accuracy in reconstructing the top-100 words in the English language, even without the users making any sounds." The accuracy increases to 90 percent for the eleven visemes — facial movements associated with speech sounds — of the English language.
The team's paper has been publish under open access terms in the journal Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies.