Made for Each Other

The computer architecture first described by John von Neumann in 1945 is still the basis for nearly all digital computing devices we use today. The von Neumann architecture consists of a discrete central processing unit and memory unit. Because instructions and data are stored in a memory unit that is distinct from the processing unit, they must be moved from memory to the processor before they can be operated on. Since the processor can only store a small amount of data at any given time, this creates a bottleneck when running data-intensive algorithms, such as those used in machine learning applications.

If at times it seems like we are trying to shoehorn machine learning algorithms into platforms that they were not designed for, it is because we are. Modern general purpose computers were not designed specifically for this type of algorithm. However, because of the power of modern computers, and some brilliant optimization techniques, great strides have been made in churning through even massive neural network calculations. But recognizing the mismatch between platform and algorithm, it makes sense to take a step back and consider if there may be a better way to achieve our goals.

A team led by researchers at Stanford University has been giving this problem some thought, and has recently published the results of their work to develop a better, more natural, platform for performing machine learning operations. They have developed what they call NeuRRAM, a neuromorphic chip that can run a variety of neural network model architectures on-device, and with a high degree of energy efficiency. They accomplished this by eschewing traditional computational architectures in favor of a compute-in-memory approach.

NeuRRAM was developed with resistive random-access memory (RRAM), which is a type of memory that allows computations to take place directly in memory. RRAM is not new, however, previous implementations have resulted in models that have a reduced level of accuracy, and have given little flexibility in the type of models that the chip can support. These problems were addressed in NeuRRAM by introducing multiple levels of optimizations across the abstraction layers of hardware and software. The result is a single compute-in-memory chip that can run tasks as diverse as image and voice recognition.

It can perform these tasks with a high level of accuracy, as well. In a series of validation tests, the team found NeuRRAM capable of achieving 99% accuracy in recognizing handwritten digits. 85.7% accuracy was observed in an image classification task, and 84.7% accuracy was achieved when running a speech command recognition task. These results are comparable to what can be achieved with traditional digital compute chips, but with a drastic reduction in energy requirements.

The researchers measured the chip’s energy utilization using a metric called the energy-delay product (EDP). The EDP factors in both energy consumed, and the time needed to perform operations, to summarize the energy efficiency of the chip. It was found that NeuRRAM achieved up to 2.3 times lower EDP as compared with traditional chips.

At present, the team is working to improve the architecture of NeuRRAM and to adapt it to more algorithm types, like spiking neural networks. One member of the team, Rajkumar Kubendran, said “we can do better at the device level, improve circuit design to implement additional features and address diverse applications with our dynamic NeuRRAM platform.” Perhaps we will see this chip incorporated into the devices we use on a daily basis after some of these enhancements materialize.