AI Is Now Totally Tubular
Engineers have created a new type of TPU composed of carbon nanotube transistors that could slash the energy consumption of AI applications.
It has become almost impossible to keep up with all of the recent advances in artificial intelligence (AI) because the field is moving forward at such a blistering pace. In fact, the field is moving forward so fast that developments in hardware have not been able to keep up with the needs of the algorithms. The traditional von Neumann architecture definitely cannot handle the latest algorithms efficiently, which anyone who has attempted to train a large AI model on CPUs can readily attest to.
You will most definitely fare better with specialized hardware like a graphics processing unit (GPU) or tensor processing unit (TPU). But while these options are much faster, they still draw far too much energy. This is manageable enough for relatively small projects, but as we continue to strive for bigger and better things it quickly becomes a limiting factor. Training a state-of-the-art large language model on internet-scale datasets, for example, may require a multi-million dollar energy budget alone, not to mention the cost of the hardware, labor, and so on.
In an effort to address these challenges, a group led by researchers at Peking University has developed a new type of TPU. Rather than relying on traditional silicon-based semiconductor technologies, they built their TPU using a more energy-efficient material — carbon nanotubes. While the proof of concept device built by the team will not be running massive algorithms like GPT-4, it is their hope that the principles they have laid out will ultimately lead to the development of more powerful chips.
The chip contains arrays of field effect transistors that are composed of carbon nanotubes. In total, 3,000 of these transistors are arranged into nine different processing units. Data flows through these processing units, from one to the next, to perform two-bit integer convolution and matrix multiplication operations in parallel. These operations are especially useful when running convolutional neural networks.
To test their hardware, the researchers designed a five-layer convolutional neural network for image recognition tasks. The algorithm was noted to have an average accuracy level of 88 percent, but the exciting part was the power competition of the chip. It consumed only 295 microwatts, which is superior to any other convolutional acceleration hardware technologies. After running some simulations, it was determined that the TPU is capable of performing over one trillion operations per watt of energy.
Of course the chip will need to be scaled up significantly to have any relevance in real-world applications — 3,000 transistors is not going to get you very far in 2024 and beyond. But should the researchers be successful in doing so, this technology could serve to democratize AI and supercharge open-source research.