Chris Greening Gets an Espressif ESP32-Powered TV Running Faster Than Ever with the Power of SIMD

Maker Chris Greening has been investigating how to get an Espressif ESP32 microcontroller to play back JPEG files faster — being pointed to a clever approach which makes use of single-instruction multiple-data (SIMD) extensions in the processor to dramatically boost decoding performance.

"A slightly cryptic [GitHub] issue appeared on the esp32-tv project," Greening explains in a post brought to our attention by Adafruit. "JPEG decoding with SIMD," GitHub user "modi12jin" wrote in the issue in question, "currently the whole frame, cannot be partial [frame], there will be more in the future."

Attached to the issue was a code snippet which used a different approach to decoding JPEG-format images, used in the esp32-tv project to stream video from microSD card and display it on a color screen. "This is pretty interesting," Greening notes of the code supplied. "SIMD (Single Instruction Multiple Data) can potentially really speed up decoding of JPEG files."

The idea is simple, though it took a few decades for computer processors to gain the ability: if you have a whole bunch of data, like the blocks which make up a JPEG image, and you need to run the same instruction across it all, SIMD offers a real acceleration — in the case of the sample code provided dropping the time required to decode a 272×233 JPEG image on an Espressif ESP32-S3 rom 109ms with the JPEGDecoder library and a best-case 32ms with the JPEGDEC library to just 20ms.

Things weren't quite that easy, sadly: Greening's initial tests revolved around simply decoding the image, but adding the step of displaying it to the screen saw it beating JPEGDEC by a mere 1ms — 37ms to 38ms. The reason: that warning from the code's creator about it working on whole frames only, stopping the microcontroller from running the direct memory access (DMA) display operations in parallel with the CPU decoding operations. Instead, the CPU has to decode the entire image then push the whole thing over DMA.

"Initially this might seem a little bit disappointing, but it’s actually not the end of the world — particularly for out TV project where we are streaming frames," Greening explains. "We can decode a frame using the CPU and kick off the DMA transfer to display the pixels. And then we can immediately kick off decoding the next frame. This overlapped decode/display process can give us a really fast frame rate."

In testing, Greening found that switching to the SIMD-based JPEG decoder provided a boost to 40 frames per second on the same hardware as before — making the esp32-tv project considerably smoother with no change to the microcontroller.

Greening's full write-up is available on his website.