Eight "Illusion" Chips, Built with Resistive RAM, Prove Near-Ideal for Deep Neural Networks

An international team of researchers have developed what a neural network acceleration engine dubbed Illusion, designed to improve efficiency by operating in a network and with limited local on-chip memory — and initial results have proven impressive.

"Hardware for deep neural network (DNN) inference often suffers from insufficient on-chip memory, thus requiring accesses to separate memory-only chips," the researchers explain of a major bottleneck in machine learning systems. "Such off-chip memory accesses incur considerable costs in terms of energy and execution time. Fitting entire DNNs in on-chip memory is challenging due, in particular, to the physical size of the technology."

"Here, we report a DNN inference system — termed Illusion — that consists of networked computing chips, each of which contains a certain minimal amount of local on-chip memory and mechanisms for quick wakeup and shutdown."

To prove the concept the team built a series of Illusion processors each with a small quantity of on-chip 3D resistive RAM (RRAM) on-board, located in a stack above the processor logic itself. Eight of these chips were networked into a single system, which showed an energy efficiency drop and performance loss of just 3.5 percent and 2.5 percent respectively compared to a single chip using exclusively on-chip memory of the same overall performance — a major improvement on current state-of-the-art multi-chip systems.

"Illusion is flexible and configurable, achieving near-ideal energy and execution times for a wide variety of DNN types and sizes," the team claims. "Our approach is tailored for on-chip non-volatile memory with resilience to permanent write failures, but is applicable to several memory technologies. Detailed simulations also show that our hardware results could be scaled to 64-chip Illusion systems."

According to an article on the project published on IEEE Spectrum the team is already working on a second, more capable prototype — offering orders of magnitude more memory compared to the 4kB available to the neural network on each of the current test chips.

The team's work has been published under closed-access terms in the journal Nature Electronics.