To Hell with TinyML

There are many well-established benchmarking suites that compare the performance of CPUs and rank them on a set of standardized tests. But around the water cooler and on Internet message boards, there is only one metric that anyone really cares about — will it run Doom? Demonstrating that this iconic early 1990s first-person shooter video game can run on a tiny, obscure, or unusual hardware platform has long served as a stamp of approval for the equipment (and the engineer that pulled off the job) among the hardware hacker community.

While many of these efforts are highly impressive technically, we are now living in 2023, which is the 30th anniversary of Doom’s original release. And computing technology has advanced such that for a few dollars one can buy a more powerful processing unit than the one in the big beige box that ran Doom, Netscape, and America Online (no use trying to deny it, we know you had an account!) in the mid-90s. So while the hacks can still be very interesting, some of the challenges are fading away.

After some recent work done by a former employee of Syntiant, it looks like the challenges may not actually be fading so much as they are shifting. Rather than asking if it can run Doom, David Garrett asked if it can play Doom. He chose to replace the game player sitting in front of the old beige box rather than the old beige box itself. Now that we are talking about hardware that is orders of magnitude more complex, things are getting a lot more interesting!

Garrett worked with the Syntiant NDP200 special-purpose processor that was designed with deep learning in mind. This tiny chip natively runs multiple neural network architectures. Since it operates at one milliwatt of energy consumption and is constrained by 640 kilobytes of onboard memory — a small amount even in the heyday of Doom — it looks like fair game for the challenge.

The main use case for the NDP200 is running relatively lightweight algorithms that monitor for key events, then wake more powerful systems to take it from there. But Garrett believed that the half million or so neural network parameters that can be stored on the chip should be enough to train a computer vision-based model to learn to blast some demons, perhaps even better than a person sitting at the keyboard could do.

The model was trained using reinforcement learning, in which it is shown examples of gameplay, and is either rewarded or penalized based on whether or not the action it takes in response is a good one. So, for example, shooting an enemy is rewarded, and emptying your weapon on an empty screen is punished. After running through more and more examples, the network starts to learn good strategies. Garrett noted that the NDP200 was playing Doom better than he ever could by the end of the project.

When it is not occupied battling the spawn of Hell, the NDP200 might also be seen performing some more ordinary tasks, like running a person detection algorithm for a security system. It is especially well-suited to situations where extremely low power consumption is demanded — the NDP200 has been shown to use 1/100th the energy of an Arm Cortex A53-based processor when running visual wake words algorithms. But before getting into that, one must first finish Doom. The only way out is through!