Shedding Light on Privacy

The many devices, like smartphones and tablets, that most of us keep nearby at all times are something of a double-edged sword. They undoubtedly offer us tremendous convenience, connectivity, and access to virtually unlimited amounts of information, but the advanced sensors also present a huge attack surface to unscrupulous hackers seeking to invade our privacy. The constant connectivity that allows us to stay in touch with friends, family, and the world at large also exposes us to potential security breaches. As such, balancing the convenience and security of these devices is a pressing challenge in our increasingly interconnected world.

Much focus has been given to securing the cameras and microphones that are present in most smart devices. Manufacturers and software developers invest significant resources in developing robust security protocols to protect against unauthorized access to these crucial components. Privacy concerns have prompted the implementation of features such as app permissions, physical camera covers, and microphone muting options to provide users with more control over their devices.

These sorts of features have gone a long way towards protecting user privacy, but enterprising hackers can be exceedingly creative, capturing vital information in the most unexpected of ways. One such exploit was recently described by a team at MIT’s CSAIL. They demonstrated that the seemingly innocuous ambient light sensor found on so many devices can be turned into a sort of low-resolution camera that exposes what is in front of the device’s display screen.

Ambient light sensors are nearly ubiquitous, typically being used to automatically adjust display brightness to an appropriate level given background light levels. They only provide a very simple, single metric that roughly reveals nearby lighting levels, so they have not been considered much of a threat historically. Accordingly, their data can generally be accessed by any app without being granted special permissions.

As it turns out, these sensors have been underestimated in their ability to reveal sensitive information. The researchers showed that they can gather data about the user’s interactions with a touchscreen — swiping, sliding, scrolling, and other gestures — via an app with minimal permissions. It is only required that the app has access to the ambient light sensor (which generally requires no permissions) and the contents of the screen (which would be expected for an app that plays videos, for example). This information can be used to infer what a user is doing in an app, and can even be used to capture low-resolution images of the area in front of the device’s screen.

The exploit works by capturing low-resolution light intensity measurements while the hand is partially obstructing the display during an interaction. This information is paired with a knowledge of the screen contents, and is fed into a deep learning algorithm that reconstructs the scene in front of the device’s display. After optimization and denoising of the image, it is still heavily pixelated, but the outline of a hand is clearly visible. In this way, the technique provides sufficient data to recognize specific hand gestures.

A number of experiments were conducted with mannequin hands that demonstrated the potential of this system to effectively reconstruct the shape of objects in front of a device’s display. However, at present the method requires over three minutes of processing time to reconstruct a single frame. Accordingly, revealing complex interactions is not yet very practical. Of course this limitation will disappear over time as computing systems increase in power. As such, the researchers recommended that the precision and speed of ambient light sensors be decreased to avoid the successful deployment of this attack. They also suggest that access to this sensor be restricted, much like it is for the camera and microphone.