Experiential Diary

We live inside overlapping pressures. The small, daily frictions of life. The larger uncertainties of work and creative survival. And beyond that, the constant hum of instability — environmental collapse, political tension, systems that feel both overwhelming and distant. In all of this, the way we are expected to "record" our inner lives hasn't evolved. We are still asked to reduce complexity into neat entries, structured reflections, into static language.

But lived experience isn't static. Thought isn't linear. Struggle isn't in sentences.

This project draws from a DIY way of living — where you learn by doing, by improvising, by building with what you have. In that space, thinking is not separate from movement, from environment, from the body. You don't just "write" a feeling; you circle it, speak it, contradict it, return to it. You exist inside it. That process is messy, spatial, and deeply personal — yet most tools flatten it into something legible rather than something felt.

The intention here is to reclaim that lost dimension.

Instead of treating a diary as a document to write, it becomes something you inhabit. A dark room that responds to your presence, your voice, your hesitation — projecting back not what you said, but how it moved through you.

The room is listening on several frequencies at once. Your voice drives the visual field directly: its volume swells and contracts the forms on the wall, its pitch and frequency shift the colour — warm tones rising when speech is loud and agitated, cooler, slower palettes settling in when the voice drops or falters. Your words are transcribed in real time and passed through an AI layer that interprets not just content but emotional register — urgency, withdrawal, grief, circling — and those interpretations alter the texture and density of what is projected. Fragments of your speech surface and dissolve on the wall as text: not a transcript, but the language the room found most charged.

Your body speaks alongside it. A depth sensor reads your movement — the reach of your arms, the stillness between gestures, the speed of your hands — and maps that energy into the behaviour of the graphics. Slow, close movement draws the forms inward, coiling. Expansive gesture opens them outward. Stillness lets the image hold.

It responds to a very human need: to make sense of things when language alone isn't enough. Whether it's trying to explain an idea, process a personal difficulty, or grapple with something larger and more abstract, we often rely on fragments — gestures, tones, pauses. This project acknowledges that expression is not just verbal or visual, but embodied.

Each session is saved. Not as text, but as a visual state — the form your presence took, held at the moment it was most itself. Returned to, the diary accumulates not words but weather. Other people's emotional residue, their colours and textures, without their names or their content.

And so the act of keeping a record of the world becomes an experience rather than a task.

THE ARTIFACT

We wanted to create a connection between the user and the diary experience, so we came up with the idea of designing an artifact that would contain the microphone, allowing the user to interact directly with this central element. We also wanted it to relate in some way to the TouchDesigner projection, so we decided to design a transparent tesseract that could reflect and play with the shapes and light generated by the projector.

THE EXPERIENCE

This was the core of the project, and the part where we improvised and experimented the most with TouchDesigner and our sensors.

The plan was to create a map based on the user‘s actions that forms in real time. The map would be projected around the user to create a spatial experience that changes and flows along with their intonation, tone, volume, body language, and other parameters. These parameters would control the shapes, their sizes, colours and more.

With some help and tutorials, we created motion patterns connected to the volume of our voices through the computer’s microphone, vibration changes linked to mouse movement that made the patterns become more frenetic, and colour changes depending on the mouse’s vertical position.

The first step of these was the audio. We worked on creating a basic visual that expands with audio input and played around with effects for the same.

We then created basic interactions for the Kinect using the mouse coordinates as the input for testing, later to be replaced. We created a circle that changed a variety of attributes based on mouse x and y movement, while mapping the location to the coordinates directly.

We replaced the mouse coordinates with parameters from the Kinect and started playing around. The final visuals would not all be controlled by hand but would use a bunch of available parameters from the Kinect sensor. We then took this one step further and created a particle system that would float and follow the hand with some delay for smoothness.

We also wanted to somehow map the amount of motion, including body movement as well as movement in the room if the user walks around or has gestures. One of the trickiest parts of the project was to put all of these together in a single seamless experience. Since we wanted this to be a map one could see after the experience, we also needed the inputs, visuals and interactions to leave a trail that would linger and layer onto each other. For this we created a feedback loop with a very simple mouse trail test and messed around with opacity and other values.

This was then applied to most of the other visuals, while balancing parameters to work for those specifically. The next step was to connect all the mouse inputs to a Kinect sensor in order to create a more immersive and interactive experience, and with that and a lot of other tweaking, we had a basic experience set up.

The final step was to actually create the experience. We placed a projector in a dark room spreading across as much of the room as possible. The mic and artifact were placed in the centre of the room in order to allow the user to feel that they are speaking to the piece.