Libby Miller's Music Camera Turns Printed Blocks and LEGO Bricks Into a Dynamic Tune
OpenCV running in the Chromium browser on a Raspberry Pi 4 delivers a streamlined music-making experience.
Creative technologist Libby Miller has turned a printed sheet of paper and a handful of building blocks into a customizable music generation — powered by a Raspberry Pi and OpenCV.
"I've had a visual tool in mind for a while and Makevember (you make a thing a day in November and share it) was an opportunity to take some steps towards it," Miller explains of the project. "The idea is to take some suitable parameters and add and remove them to change the musical character of a fixed set of notes. The fixed set is a printout of some data as little squares spread out on an A4 sheet. This is calibrated to fit roughly in a square in the center of the paper and be printable at A4 at 72dpi."
The resulting blocks in the printout may not look like music to the human eye, but Miller's code — initially written using the OpenCV library in Python then moved to JavaScript in a headless browser, owing to a lack of easy-to-use audio libraries — recognizes their shapes and positions and translates them to MIDI notes for playback.
Changing the song, then, would normally involve printing a new sheet of "notes" — but Miller's system is designed to be dynamic, responding to the presence or absence of colored LEGO blocks placed on the sheet to modify the music.
"From a technical perspective I learned a lot, and I like the final noise — but I don't think there's enough capacity in the variables to make enough of an interesting piece of music — and maybe this is connected with the lack of randomness or serendipity in the process," Miller notes. "But perhaps it's a first step."
"You may well ask — why not use a camera from a laptop instead of a headless browser and Chrome's dev tools? I had in mind an appliance," Miller adds, "thinking maybe I can scale it up and down, but still be able to test it on a laptop. Apart from a few teething issues, headless Chromium on a [Raspberry] Pi 4 works very well."
The full write-up is available on Miller's website.