Bennett Bytes Develops a Custom TikTok Smartwatch Using an ESP32

Dave Bennett recently developed a smartwatch that brings TikTok videos to your wrist. He used an API to download those clips and play them on the device, which relies on an ESP32 microcontroller with 32KB of RAM. The whole idea revolves around wirelessly transmitting videos and audio to the ESP32 and showing them on an ST7789WV 1.14" 135 x 240 LCD and 0.5W speaker. To play the sound on the ESP32, Bennett uses uncompressed audio (WAV), a digital to analog converter (DAC), and an audio amplifier.

However, the audio needs to be transmitted first. So he imported the pi dub library in his Python script to split up a WAV file into chunks before transferring them to the ESP32 via a web socket. Next, he employs a DAC so that the speaker can output the audio. This was achieved with the MAX98357A chip, which features a built-in DAC and audio amplifier, using the i2S protocol to communicate. He sticks with using the SPI interface to keep things simple for this project. The SPI driver can handle speeds of up to 60 megahertz while maintaining a speed of 115FPS for the video.

Since the device deals with raw data, it'll be working with 64 kilobytes of storage, which presents a problem due to insufficient RAM. So he compressed the 64-kilobyte image down to 8 kilobytes. But then he needed to decode that JPEG into raw data on the ESP32. Thus, he created a Python script to grab the images from the video, compress them into a JPEG, and send it to the ESP32 through a web socket. Afterward, the ESP32 essentially decodes it from a JPEG to raw data and sends it to the display.

However, the audio and video didn't sync up at first. Since the image and audio last 0.33 seconds, he packaged one image of 0.33 seconds of audio and sent that to the ESP32 via the web sockets. After receiving the package, the ESP32 decoded and separated the audio and video. Lastly, the audio is fed to the I2S, and then the video is sent to the screen simultaneously.