Looking In From the Outside

The regular Hackster News reader may be surprised to find a story about a new art installation on these pages. But when that art installation involves neural networks and Raspberry Pi computers, it all starts to make sense. We can certainly appreciate art that has a power cord.

The interactive installation named Inside Inside was built by Douglas Edric Stanley. It allows viewers to play the game Inside on a PlayStation 4, in which they control a small boy trying to stay alive in a dystopian world. Images from the gameplay are fed into a neural network, which stitches together a set of related images from film and television history. In effect, the game player is also a film editor. The created film is played on a second screen.

Inside Inside is powered by a custom-designed pipeline, which uses OpenCV to preprocess imagery from gameplay for input into the neural network. A hand-curated set of relevant images were selected from the history of cinema, and the neural network was trained to make associations between gameplay imagery and training data. By carefully curating data from 128 film sequences to keep the training dataset relatively small, it was possible to generate an algorithm that can perform well on a Raspberry Pi 4 computer.

Due to the success of this art piece, the author is currently exploring additional game/media pairings, including Super Mario Bros. with Svankmajer’s Alice and Id Software’s classic Doom with Gus Van Sant’s Elephant.

As computing continues to increase in performance and decrease in cost, we expect to see more non-traditional uses of machine learning and embedded systems, and that is a trend that we can all look forward to.