Researchers at Stanford University have unveiled a new use for those ever-present glowing status LEDs on modern electronics: communicating data and positioning information for augmented reality applications.
"Augmented Reality (AR) has the potential to expand our capability for interacting with and comprehending our surrounding environment. However, current AR devices treat electronic appliances no different than common non-interactive objects, which substantially limits the functionality of AR," researchers Jackie Yang and James A. Landay explain of the core issue. "We present InfoLED, a positioning and communication system based on indicator lights that enables appliances to transmit their location, device IDs, and status information to the AR client without changing their visual design."
There's a trick involved, of course, in providing useful information to a computer vision system without humans in the vicinity being bothered. "By leveraging human insensitivity to high-frequency brightness flickering, InfoLED transmits all of that information without disturbing the original function as an indicator light."
The researchers detail three key areas in which InfoLED can be used to augment existing devices. Perhaps the most interesting of these is failure diagnosis, where a smartphone application can be pointed towards an InfoLED-equipped device to read out its current status and even suggest a means of repair. The same technique can be used to allow for AR control of remote devices, and even to visually link multiple devices to virtual controls by dragging-and-dropping links.
"We conducted three user studies, measuring the performance of the InfoLED system, the human readability of the patterns and colours displayed on the InfoLED, and users’ overall preference for InfoLED," the researchers claim in the InfoLED paper. "The study results showed that InfoLED can work properly from a distance of up to 7 metres [around 23 feet] in indoor conditions and it did not interfere with our participants’ ability to comprehend the high-level patterns and colours of the indicator light. Overall, study subjects prefer InfoLED to an ArUco 2D barcode-based baseline system and reported less cognitive load when using our system."
There is one drawback to the team's approach, however: bandwidth. Flickering a visible LED and monitoring through an off-the-shelf smartphone camera means that there is only around 60 bits per second (bps) of bandwidth available for data communication, which the team explains is enough to transmit the ID and status of the device. Another issue is that when the LED is switched off or occluded, the AR system is no longer able to track the position of the device.
Yang and Landay presented their work at the 32nd Annual ACM Symposium on User Interface Software and Technology (UIST '19), with a copy of the paper available through the ACM Digital Library.