Eric N. Shows How OpenCV Can Squeeze Onto an Espressif ESP32, Using Joachim Burket's Shrunken Fork
Walking through the downloading, compilation, and use of a miniaturized OpenCV fork, the latest That Project video focuses on edge CV.
Semi-pseudonymous YouTuber Eric N., also known as "That Project," has put together a guide for running a specially-shrunk version of the OpenCV computer vision library, developed by Joachim Burket, directly on an Espressif ESP32 microcontroller — with no need for a more powerful external host device.
"This is running OpenCV on an ESP32," N. explains in the introduction to his latest video. "It's working as a stand-alone [device] and handling everything on the ESP32. With the image obtained from the camera, it's doing Canny edge detection with the OpenCV library in real-time. OpenCV is built to work on many platforms, but working stand-alone on ESP32 is really cool."
Cool, but perhaps not entirely straightforward. What N. has put together, in fact, is not a full port of the entire OpenCV library bundle — an impossibility, given the relatively meagre resources available on an ESP32 compared to the devices that would normally host OpenCV — but a cut-down version, forked from the original OpenCV library by Joachim Burket, which carries enough functionality to be useful without trying to cram too much into the microcontroller.
There's another twist in the tale, too: "It needs more memory than normal work[loads]," N. explains — meaning that a bare ESP32 alone lacks the resources. For the demonstration, N. has turned to a LILYGO TTGO Camera Plus, which pairs an ESP32 module with 8MB of pseudo-static RAM (PSRAM) and 4MB of flash storage plus an on-board camera and display.
Burket's OpenCV fork is compiled and flashed to the device using ESP-IDF — and a known issue with compiling the demo application worked around with the installation of a Docker-based build environment. Once flashed, the ESP32 begins running the demo: Taking the image from the camera and running it through a series of transformations, all applied in real-time at an admittedly slow frame rate.
"It's calculated to be about six frames per second on average," N. admits. "The performance isn't great, but considering the computing power of [the] ESP32 I think it's pretty good."
N.'s full video is available on the That Project YouTube channel now, while Burket's shrunken OpenCV fork is available on GitHub under the permissive three-clause BSD license — though users should note that it was forked from OpenCV two years ago, and has not been refreshed from upstream since.