The Writing Is on the Wall (and the Table)

Digital pen input has applications ranging from taking handwritten notes to creating precision drawings. These pens typically rely on capacitive or resistive sensors on the drawing surface itself, however, which makes them unsuitable for virtual reality (VR) applications. In VR, a user may need to use any number of surfaces to provide input.

A team at the Sensing, Interaction and Perception Lab at ETH Zurich have devised a solution to this shortcoming of traditional digital pens with their device named Flashpen. Flashpen is a high-precision digital pen, designed for multi-surface use in VR.

Sometimes a new gadget impresses because of the idea behind it, rather than the technology itself. Flashpen certainly fits into this category. It is something of a Frankenstein’s monster of scavenged parts — an optical flow sensor (Pixart PMW3360) was taken from a high-end gaming mouse to allow for high-precision measurements. A LEGO Mindstorms hinge was used to attach the pen shaft to the sensor to allow it to be held at a natural angle. An InvenSense MPU6500 accelerometer detects rotations and accelerations, and an ESP32 microcontroller handles communication with a host computer. All of this is packaged in a plastic tube wrapped with electrical tape.

Flashpen is able to detect optical reflections at a rate of 8,000 per second. This far surpasses the rate of professional digital pens currently on the market, such as the Wacom Cintiq that can capture about 200 measurements per second. This gives Flashpen a significant advantage in capturing high-fidelity, high-precision input.

The team ran a series of trials in which participants performed a number of tasks, such as writing and tracing, with both Flashpen and a Wacom tablet. The users generally reported that using the Flashpen device felt more like writing on paper, and that they were able to produce smoother results. Unsurprisingly, they were not impressed with the ergonomics of the Flashpen, but that is to be expected with a rough prototype device.

One drawback to the Flashpen is that the sensor does not perform well when lifted off of the drawing surface. This limitation is especially apparent when writing, as repeatedly releasing and establishing surface contact is a frequent feature of writing. The researchers are investigating complementary tracking methods that may be able to improve hover-precision in the future.

The pen was tested out in a number of VR applications. The team demonstrated viewing and annotating a document, creating a “magic lens” on a two-layer map, drawing, and path animation. With a little refinement, Flashpen has the potential to outperform high-end consumer products. It is always encouraging to a hobbyist to see an inexpensive, hacked-together device like this outperform professional tools.