Boost Your Memory

As artificial intelligence (AI) becomes more pervasive in our lives, hardware has become a major bottleneck in running these algorithms. According to experts, traditional computing technologies are not keeping up with the requirements for running AI models. Over the past three decades, the size of the neural networks needed to run cutting edge applications has doubled every 3.5 months, while hardware capabilities have doubled every 3.5 years. This has significant consequences for the development and deployment of AI applications.

Larger AI algorithms require vast amounts of data to be processed in real-time, and this is putting a huge strain on existing hardware. Traditional central processing units and graphics processing units are not optimized to handle the complex calculations required for AI. This has led to the development of specialized hardware such as tensor processing units and field programmable gate arrays that are specifically designed for AI applications.

However, even with these developments in specialized hardware, the demand for computing power is outpacing the rate of hardware advancements. A different type of computer memory has recently been proposed by a team of researchers at the University of Massachusetts Amherst that has the potential to supercharge AI computations, especially in edge AI devices. By combining traditional silicon-based microchip technology with metal oxide memristors, they have demonstrated that it is possible to create powerful chips that consume very little energy.

Conventional modern memory technologies represent information via the number of electrons present. Part of the team’s innovation involves instead using the positions of whole atoms to store data. Whereas electrons are highly volatile, atoms offer a stable way to store data, even when power is removed from the chip. Moreover, because the position of the atoms is used to represent information, multiple states can be represented in each unit of memory. Instead of the usual zero or one state of a bit, this new type of memory has been shown to be able to represent 2,048 distinct states (or if you prefer, 11 bits of data).

These novel properties mean that memory can be densely packed onto small devices, so more total memory can be made available to AI algorithms. And since power is not required to maintain the state of memory, less energy consumption is required for operation, and state can even be retained when the device is fully powered off. This same technology can also be programmed with the weights of a neural network, which allows it to perform computations. This breaks the von Neumann bottleneck that plagues AI applications by allowing computations and memory fetches to coexist, which dramatically reduces processing delays.

The researchers were able to achieve this feat by first identifying the underlying physics behind memristors to overcome the issues that have prevented previous efforts from representing as many states in a single memory unit. They put this basic research into practice by fabricating fully integrated chips with 256 x 256 memristor arrays that leverage their new technology. This first chip is a small step forward, but in the future, larger chip arrays may bring state of the art AI, like GPT-4, to small edge computing devices. Perhaps we will even carry these algorithms around with us in our pockets one day.