The Pen Is Mightier Than the Finger

Touchscreens have become the most popular interface for quick, casual interactions with computing devices like smartphones, tablets, and kiosks. And there is no wonder why this is the case, considering how intuitive and simple these interfaces can be. But this simplicity comes at the expense of accuracy — very fine details are difficult to capture with a finger on a capacitive touchscreen. Additionally, the size of the input area is limited by the size of the display. These limitations are fine for many applications, but if one wants to create detailed artwork, for example, a touchscreen just is not the optimal input method.

Various digital pens have been introduced to address the issues of inaccuracy seen with finger-based inputs, but the input area of these devices is still limited to the size of the screen. Yet more pens have been developed that rely on either off-screen active input pads or additional hardware like cameras that track movements across passive surfaces. This latter group of devices is not very practical for most applications, especially mobile applications, because of the extra hardware that needs to be lugged around, set up, and calibrated before each use. A new prototype device called DeltaPen that was recently described by engineers at ETH Zürich may be able to address problems of inaccuracy, input area size, and practicality all in one fell swoop. DeltaPen can be used on any surface, is highly accurate, and requires no external hardware.

DeltaPen is able to sense movement along an uninstrumented surface, rotation perpendicular to that surface, as well as tilt. A pair of optical flow sensors, of the sort commonly found in a computer mouse, are leveraged to not only track movement, but also to recognize rotation. A three-axis accelerometer inside the pen provides the raw data needed to detect tilting of the device. A Teensy microcontroller board processes all of the sensor data and handles communications with a host computer via USB. A force sensing resistor in the tip of DeltaPen provides information about pressures applied during use, and haptic feedback can be given with the help of a linear resonant actuator. The components are all housed in a 3D printed shell.

The pen was tested with some common tasks like sketching and writing to see how it compares with ground truth observations. To capture the ground truth measurements, tracking markers were attached to the DeltaPen and tracked with OptiTrack infrared cameras. A total of ten participants put this system through its paces during the course of this small study. All of the metrics recorded showed that the error was very small, confirming that DeltaPen is a highly accurate input device.

Validation studies are one thing, but seeing the device in action is another entirely, so the team implemented some applications to show how DeltaPen performs under real world scenarios. The expected note taking and sketching tools were demonstrated, along with a number of generic widgets that have broad applicability. But the pen really shined when paired with a painting application. Users of this application created some truly amazing artwork that shows off how incredibly detailed work produced with the DeltaPen can be. Take a look at the stripes and musculature on the tiger, for example, and consider how difficult this result would be to achieve with a standard touchscreen.

The researchers are presently working on designing an untethered version of the DeltaPen, which will require adding a battery and wireless communications. Optimization of data transmission will also be required to minimize latency during use. They hope that their work will inspire further research into pen-like input devices that can operate without external hardware.