Can a Robot Create Convincing Handwriting Forgeries?

When trying to make a robot replicate a human creation, the difficulty comes from the imperfections. Robots are very good at moving with both precision and accuracy, while humans are bad at both of those. Humans make small errors in just about everything we do and those errors are unpredictable, which makes them hard to duplicate. Handwriting is a perfect example of this and a notorious challenge to make robots produce in a way that fools people. But Shane of Stuff Made Here had a lot of cards to write, so he decided to find out if a robot could create convincing handwriting forgeries.

You can go buy a pen plotter, like the iDraw 2.0 I reviewed last year, and get a machine that can write sentences with a pen. Just copy and paste the words into Inkscape, choose a font you like (I suggest a script style), then use the plugin to generate g-code for the plotter to follow. That's exactly how Shane started this journey and he quickly came to the same realization that we all do: the writing looks like it was done by a robot. The characters are too consistent, the kerning (the space between letters) is too noticeable, the lines are too neat, and there is no variation in how the characters lead into each other.

Anyone can identify handwriting created by a robot in the manner described above, so Shane tried numerous techniques to make it look more natural. But first he needed to streamline the process of setting up cards. He programmed an industrial Tormach robot arm to grab cards from a spring-loaded stack using a vacuum end effector, which drops the card on a vacuum table that holds the card in place as the plotter writes. Afterwards, the Tormach picks up the card and places it in an output stack.

With that sorted, Shane began experimenting with ways to program natural handwriting. He started with simple approaches like using real handwriting samples, but his test subject (his wife) was always able to identify the robot-written cards. He then attempted to develop a machine learning model that would create new paths based on the shapes of the lines in real handwriting, but he soon discovered that machine learning is really hard to implement.

Shane finally found a project by Sean Vasquez that uses recurrent neural networks to synthesize handwriting. Its output looks very real and Vasquez's work let Shane avoid the programming headaches.

Because the handwriting paths have natural-looking variation and the robot wrote on the cards with a pen, the results were impressive. Shane even sent those to handwriting expert that admitted that he wouldn't have been able to tell that they were written by a robot if he didn't already know. I think that is good enough for some "thank you" cards — though the technique may be identifiable as a forgery if actual money is on the line.