Making Practical Augmented Reality a Reality

Not so very long ago, standalone augmented reality (AR) glasses took a lot of technical wizardry to build. Fitting all of the processing power, displays, supporting electronics, and batteries into a package that could reasonably fit onto the frames of a pair of glasses was no easy task. My, how the times have changed! These days, finding components that are small and energy-efficient enough for the job is as easy as browsing your favorite electronic components distributor’s website. And not only are they readily available and cheap, but there are plenty of hobbyist-grade options that are easy to work with.

These factors have made it possible for even high school students, like Nguyễn Minh HIỂN, that are still learning the ropes to get in on the action. Nguyễn has built what he calls TPGmini, a highly-customizable, clip-on solution that adds interactive AR experiences to any standard pair of glasses. TPGmini may be a bit crude in its present state, but it gets the job done and looks like a very nice platform for experimentation with the technology.

The build is quite simple, and could be assembled in an afternoon. A Seeed Studio XIAO ESP32 S3 Sense development board serves as the central processing unit. This board also comes equipped with a camera and microphone to enable some more advanced applications. A 128x64 OLED display is driven by the XIAO to produce the graphics or text one wants to present to the user. This display shines on an angled piece of clear plastic, coated with a reflective automotive heads up display sticker, that the user looks through. This causes the image to appear to float in front of them in open space.

TPGmini is designed to be removable, by simply loosening the clip that holds it on a pair of glasses. That way it can be removed after experimentation, which you are definitely going to want to do because the 3D-printed case is big and ugly. Nguyễn is working on an updated version that will be smaller, so perhaps in the future users won’t mind leaving it in place more of the time.

So far, Nguyễn has demonstrated applications that can do live captioning and sound detection (for those with hearing loss). Integration with the Gemini large language model has also been shown. However, at present, these applications all rely on an external computer to do the real work, so you can’t stray too far from it. Future updates could bring the processing onboard — or at least offload it to a smartphone — to improve portability.

The XIAO’s camera and microphone can also be put to use for capturing images or videos, which could make for some interesting applications. Furthermore, this particular development board was built with edge AI in mind, so there could be some much more interesting possibilities yet for the creative hardware hacker. If you are interested in building your own AR experimentation platform, take a look at the project write-up for more details.