You’ll Want to Get Your Hands on This

Computing devices have become smaller, cheaper, and more powerful over the years, and as a result, they have become ubiquitous in our daily lives. However, interacting with these devices is still limited to two dominant types of interfaces, which are flat touch surfaces and mechanical inputs such as keyboards, buttons, and knobs.

These interfaces have risen in popularity because they have many desirable features, but they are not without their drawbacks. While touchscreens offer a flexible platform for interface designers to test and tweak, they lack tangibility and rely solely on visual feedback to operate. On the other hand, physical mechanisms such as buttons and knobs offer great tactile feedback, however, designing, implementing, and adjusting these mechanisms at scale can be difficult and costly.

In search of a middle ground that combines the best features of both touch and physical interfaces, a team of researchers at the Future Interfaces Group of Carnegie Mellon University have designed a novel input method that they call Surface I/O. This system relies on passive objects, with patterns and other structures on their exterior surfaces. These surface features are designed to give tactile feedback to a user of the device, and also to generate distinctive vibratory patterns that can be detected and interpreted. In this way, it is possible to, for example, recognize that a finger has slid across the surface of an object between two points. And that information can, in turn, be leveraged to control electronic devices.

In this study, a number of objects were created with various surface textures using 3D printers and laser cutters. It was noted, however, that the same results could be achieved using fabrication methods that are better suited to mass production. The features were designed at three levels of scale: the macro-scale (5 cm - 1 mm), meso-scale (1 mm - 200 μm), and micro-scale (<200 μm). The larger features can be felt by the finger, providing tactile feedback to the user. At the smaller end of the scale, the features are imperceptible, but provide distinctive vibratory signatures when they are interacted with.

Sensing of these vibrations was handled by a high-bandwidth, 7 mm diameter piezo element that was acoustically coupled to each Surface I/O interface with superglue. A grounded layer of copper tape served to shield the piezo element from external sources of interference. The signal is read by a custom pre-amplifier, then digitized by a USB audio interface. To make sense of this audio signal, an Extremely Randomized Trees Classifier was implemented in Python, and trained to recognize certain patterns of activity on the surface of an object, like taps and swipes.

The Surface I/O system was put to the test in a series of trials to determine how intuitive users find the interface, and also how accurate it is at sensing inputs. The results showed that users generally found the objects to be intuitive and easy to use, with simpler designs being the most effective. Certain features, like spiral and concentric dials were found to be not readily distinguished by users, and so are probably better left out of surface feature designs. Concerning accuracy, a mean texture classification accuracy of 90.1% was observed, which is reasonably good, but not quite ready for use in a commercial device. It should also be noted that only four textures were assessed for classification accuracy. It is not clear how performance might be impacted if a larger set of interactions needed to be recognized.

Aside from the issues with accuracy, there are also unanswered questions around the durability of the surface features, especially at the micro-scale. Further studies would be needed to determine if the surface features would stand up to long-term use. The researchers are presently exploring ways to optimize their data processing pipeline to improve the accuracy of Surface I/O.