•

Published June 20, 2026

Memoirs of Disappearing Earth

Memoirs of Disappearing Earth translates the live electrical current between human skin and changing landscapes into real-time soundscapes.

IntermediateWork in progress59

Things used in this project

Software apps and online services

Arduino IDE

Arduino NANO ESP32

VCV Rack

Hand tools and fabrication machines

Drill / Driver, Cordless

Story

We often treat the Earth like a passive backdrop, forgetting that we are physically made of its materials. ‘Memoirs of Disappearing Earth’ is an immersive art project that bridges this gap through the intimacy of touch. Using a custom-built instrument, we translate the electrical oscillations between human bodies and shifting, damaged landscapes into live, real-time soundscapes. It turns ecological loss from an abstract concept into a direct, physical, acoustic experience. It’s not a metaphor, it is the literal voice of a changing planet, creating an undeniable memory that demands environmental accountability.

Having witnessed the rapid environmental transformation of the Llobregat river firsthand, we felt compelled to create a project that documents and bears witness to this changing landscape.

The project is intended to strip away the protective distance of cold environmental statistics, transforming what is usually just detached scientific metrics into intimate and emotional audio testimonies. It enables a raw human-earth skin interface. The aim is to produce a geolocated timestamped digital sound atlas of environmental scars, of landscapes in transition, documenting the real-time degradation of specific areas like the Llobregat river. The output is a shared public ritual that moves audiences from passive observers to accountable caretakers and fosters collective accountability.

left image: March 2026 - right image: June 2026

Team & Roles

Beste Cebeci - https://beste-cebeci.github.io/mdef/
audio synthesis, software development, electronics calibration & troubleshooting (Beste's Personal Blogpost)
Heba Elidrisi - https://hebaelidrisi.github.io/mdef/
physical artifact design, digital fabrication, component integration (Heba's Personal Blogpost)

Together, we co-designed the narrative, the interactive interface, and navigated multiple iterations based on the inputs received. Site visits and material collections were conducted collectively.

Technical Process

1. Hardware & Wireless Connectivity

The project utilizes a Barduino (an ESP32-based development board) configured as a wireless MIDI controller using the BLE-MIDI (Bluetooth Low Energy MIDI) protocol. To establish the connection, the Barduino pairs wirelessly with a Mac using its native Audio MIDI Setup utility (via the Bluetooth Configuration panel). Once connected, the board is recognized by the operating system as a standard MIDI input device.

2. Sensor Data Processing & MIDI Mapping

Raw sensor values—which can reach upwards of 60, 000—are calibrated and mapped down to the standard MIDI data range of 0–127.

The code categorizes these readings into three distinct interaction states

Baseline / Idle (Value: 0): No human or material contact (silence).
Material Baseline: The sensor is in contact with the transitioning material alone
Human-Material Interaction: Human touch alters the electrical capacitance/resistance, driving the MIDI values toward 127.

3. Sound Design & Modular Synthesis (VCV Rack)

For the audio engine, the mapped MIDI data is routed into VCV Rack, an open-source modular synthesis software. Within VCV Rack, each sensor is assigned to a specific MIDI channel or Continuous Controller (CC) number.

To anchor the abstract synthesis in reality, the system utilizes actual field recordings of the materials. These recordings are loaded into sampler nodes within VCV Rack and tuned to specific, researched resonant frequencies (in Hz) inherent to those materials.

Rather than just changing pitch or volume directly, the incoming sensor data acts as a VCO (Voltage-controlled oscillator) manipulating pitch and wave shapes. Finally, reverb filters are added to each channel to deepen the atmospheric and emotional quality of the collective composition.

VCV Rack

Project website: https://beste-cebeci.github.io/mode/index.html

Design Process & Challenges

Throughout the design process, we explored different options through quick prototypes with readily available materials or using recycled materials from other projects in order to quickly visualize. There were multiple conceptual pivots on both the physical artifact and the system set up, and we specifically struggled to balance systemic constraints with the desired interaction outcomes. Over the multi-week sprint, the project design evolved through both technical and physical iterations.

System Architecture Evolution

The initial prototype was brainstormed to be an interactive audio-visual system setup through Touch Designer, with the possibility of being run on a Raspberry Pi to create an autonomous computer-less experience. After technical evaluation, it was decided that the visual output was not as significant and that it was better to focus on mastering the core tactile interaction to sound output. Therefore, the system architecture was rethought to focus solely on multi-channel audio synthesis. The modular audio synthesis was switched over to VCV rack due to its low-latency and modular patching capabilities, better for real-time sensor driven control.

Sensor Integration & Capacitative Signal Routing

The sensor setup was experimented with through multiple iterations. Configurations of one, four and finally three active capacitance sensors were tested for efficiency and effect. The final physical artifact utilized three active sensors since it was connected to ECG cables that acted as conductive links, which only contained three cables.

Earlier tests had revealed that exposing the raw earth materials directly to the capacitive surface made the system read the base material or human body's baseline capacitance rather than the material's physical properties. In order to isolate the material data, the interaction topology was redesigned to use ECG cables as conductive links to shield any interferences. The material is wired in series with the sensor link. In this way, the user initiates contact with the link before touching the material, closing the capacitance loop. This allows the microcontroller to accurately read variations in data caused by the specific density, moisture content and electrical resistance of the materials themselves.

Hardware Debugging & Remapping

While integrating the final three-channel sensors on the development board, one of the pins experienced temporary lockup and the raw readings intermittently spiked to a maximum boundary value of 60,000, which halted the MIDI transmission.

After some diagnostic isolation testing, a hardware failure was identified on GPIO Pin 8. When working simultaneously with the other pins, it caused electrical instability across the board and froze the data stream. The physical circuit was rewired to bypass the defective pin and remap the third data channel to a different pin. After successfully updating the configuration, the multi-channel data stream was restored.

Environmental Calibration Constraints

Since capacitance touch sensing is highly sensitive to electric variations, ambient factors such as humidity, material volume and moisture saturation shift the sensor baselines. The testing done in the studio environment can not accurately replicate the environmental factors of other locations, therefore requiring a specific field calibration. After the raw materials (clay, soil, water) were collected directly from the focus site, the data mapping ranges were adjusted to a different range to ensure the sensors easily pick up the reading variations.

Since the hardware depended on a laptop to host the VCV Rack audio generation, the activation of the artifact was limited to actively managed live activation environments. Otherwise, users engaged with the previously recorded material audios available on the website interface, serving as the digital data archive.

References & Inspirations

The Materiality of Materials (Tim Ingold, 2007) - Ingold, T. (2007). Materials against materiality
The Mangle of Practice & Intra-Action - Andrew Pickering's The Mangle of Practice and Karen Barad's theories of posthumanist performativity
Acoustic Ecology & Locative Sound: Diegetic Materiality (James Bulley, 2018) - Bulley, J. (2018). Living Sound: The Studio, The Lab and the Field. Goldsmiths, University of London
Biodata Co-Creation - Sincresis by Lara Campos
Grounded in Time and Place & What Remains - Murray Art Museum Albury (MAMA) Exhibitions
Touch-Responsive Conductivity & Capacitance - Bela Trill Craft Sensor Ecosystem / Vicky Browne's Into the Interconnectedness

Cognitive Contribution License (CCL)

CCL Badge

Code

Capacitive Sensor Midi Controller

#include <BLEMIDI_Transport.h>
#include <hardware/BLEMIDI_ESP32.h>
BLEMIDI_CREATE_DEFAULT_INSTANCE()

struct TouchSensor {
  uint8_t pin;
  long touchMin;
  long touchMax;
  int lastCC;
};

TouchSensor sensors[] = {
  {4, 90000,  270000,  -1},
  {10, 72000,  110000, -1},
  {14, 86000,  260000, -1}
};

const int NUM_SENSORS = sizeof(sensors) / sizeof(sensors[0]);
bool isConnected = false;

void setup() {
  Serial.begin(115200);  // ADD THIS
  MIDI.begin();
  BLEMIDI.setHandleConnected([]() {
    isConnected = true;
  });
  BLEMIDI.setHandleDisconnected([]() {
    isConnected = false;
  });
}

void loop() {

  // PRINT RAW READINGS WHETHER CONNECTED OR NOT
  for (int i = 0; i < NUM_SENSORS; i++) {
    long raw = touchRead(sensors[i].pin);
    Serial.print("S");
    Serial.print(i + 1);
    Serial.print(":");
    Serial.print(raw);
    Serial.print("  ");
  }
  Serial.println();

  if (!isConnected) {
    delay(10);
    return;
  }

  for (int i = 0; i < NUM_SENSORS; i++) {
    long raw = touchRead(sensors[i].pin);
    long clamped = constrain(
      raw,
      sensors[i].touchMin,
      sensors[i].touchMax
    );
    int cc = map(
      clamped,
      sensors[i].touchMin,
      sensors[i].touchMax,
      0,
      127
    );

    if (abs(cc - sensors[i].lastCC) > 1) {
      MIDI.sendControlChange(
        i + 1,
        cc,
        1
      );
      sensors[i].lastCC = cc;
    }
  }
  delay(10);
}

Credits

Beste Cebeci

4 projects • 1 follower

Heba Elidrisi

4 projects • 1 follower

Memoirs of Disappearing Earth