See and Sense Someone Else’s Reality

Seeing someone else’s life through a headset of some sort can be interesting, though normally what you get is “simply” audio and video data. If, however, you’d like to be even more immersed in what a broadcaster is sensing, Swedish telecom company Tele2 has a novel solution: “Emotional Reality.”

This device, or rather system of devices, is meant to highlight their high-speed data plan and involves video integrated with other sensory inputs interfaced together using a Raspberry Pi. In order to ensure a smooth live streaming experience, three different SIM cards were used for data transmission.

A custom backpack was made to house a 3D-printed rig for the Raspberry Pi collecting sensor data, and a color-grading box for receiving camera footage. The footage then got sent, scaled down and compressed to a streaming service together with the audio and sensor data.

The final result was an immersive online experience, allowing viewers the chance to ‘step inside the minds’ of nine Swedish celebrities, including actor Joel Kinnaman and the Queen of Shitty Robots Simone Giertz!

Spectators of the live “Emotional Reality” broadcast got a border that changed based on the subject’s perceived emotion. They could also get other information on the subject, like heart rate or breathing, by holding the spacebar or tapping their device.

Based on the EEG (brain sensor) data, measured via a Muse brain sensing headband, the footage was treated in a way that it was presented warmer or colder, representing the emotions of the person. We built a custom iOS app to record the brain activity via the headband and sent it to our admin API.

If this sounds interesting, that’s because it is. You can read more about Tele2’s project here and see a demo of it below!

[h/t Raspberry Pi]