Memoirs of Disappearing Earth

Memoirs of Disappearing Earth translates the live electrical current between human skin and changing landscapes into real-time soundscapes.

IntermediateWork in progress28

Things used in this project

Hardware components

Arduino Nano ESP32

Software apps and online services

Arduino IDE

VCV Rack

Hand tools and fabrication machines

Drill / Driver, Cordless

Story

We often treat the Earth like a passive backdrop, forgetting that we are physically made of its materials. ‘Memoirs of Disappearing Earth’ is an immersive art project that bridges this gap through the intimacy of touch. Using a custom-built instrument, we translate the electrical oscillations between human bodies and shifting, damaged landscapes into live, real-time soundscapes. It turns ecological loss from an abstract concept into a direct, physical, acoustic experience. It’s not a metaphor, it is the literal voice of a changing planet, creating an undeniable memory that demands environmental accountability.

Having witnessed the rapid environmental transformation of the Llobregat river firsthand, we felt compelled to create a project that documents and bears witness to this changing landscape.

left image: March 2026 - right image: June 2026

Technical Process

1. Hardware & Wireless Connectivity

The project utilizes a Barduino (an ESP32-based development board) configured as a wireless MIDI controller using the BLE-MIDI (Bluetooth Low Energy MIDI) protocol. To establish the connection, the Barduino pairs wirelessly with a Mac using its native Audio MIDI Setup utility (via the Bluetooth Configuration panel). Once connected, the board is recognized by the operating system as a standard MIDI input device.

2. Sensor Data Processing & MIDI Mapping

Raw sensor values—which can reach upwards of 60, 000—are calibrated and mapped down to the standard MIDI data range of 0–127.

The code categorizes these readings into three distinct interaction states

Baseline / Idle (Value: 0): No human or material contact (silence).
Material Baseline: The sensor is in contact with the transitioning material alone
Human-Material Interaction: Human touch alters the electrical capacitance/resistance, driving the MIDI values toward 127.

3. Sound Design & Modular Synthesis (VCV Rack)

For the audio engine, the mapped MIDI data is routed into VCV Rack, an open-source modular synthesis software. Within VCV Rack, each sensor is assigned to a specific MIDI channel or Continuous Controller (CC) number.

To anchor the abstract synthesis in reality, the system utilizes actual field recordings of the materials. These recordings are loaded into sampler nodes within VCV Rack and tuned to specific, researched resonant frequencies (in Hz) inherent to those materials.

Rather than just changing pitch or volume directly, the incoming sensor data acts as a VCO (Voltage-controlled oscillator) manipulating pitch and wave shapes. Finally, reverb filters are added to each channel to deepen the atmospheric and emotional quality of the collective composition.

VCV Rack

Project website: https://beste-cebeci.github.io/mode/index.html

Capacitive Sensor Midi Controller

#include <BLEMIDI_Transport.h>
#include <hardware/BLEMIDI_ESP32.h>
BLEMIDI_CREATE_DEFAULT_INSTANCE()

struct TouchSensor {
  uint8_t pin;
  long touchMin;
  long touchMax;
  int lastCC;
};

TouchSensor sensors[] = {
  {4, 90000,  270000,  -1},
  {10, 72000,  110000, -1},
  {14, 86000,  260000, -1}
};

const int NUM_SENSORS = sizeof(sensors) / sizeof(sensors[0]);
bool isConnected = false;

void setup() {
  Serial.begin(115200);  // ADD THIS
  MIDI.begin();
  BLEMIDI.setHandleConnected([]() {
    isConnected = true;
  });
  BLEMIDI.setHandleDisconnected([]() {
    isConnected = false;
  });
}

void loop() {

  // PRINT RAW READINGS WHETHER CONNECTED OR NOT
  for (int i = 0; i < NUM_SENSORS; i++) {
    long raw = touchRead(sensors[i].pin);
    Serial.print("S");
    Serial.print(i + 1);
    Serial.print(":");
    Serial.print(raw);
    Serial.print("  ");
  }
  Serial.println();

  if (!isConnected) {
    delay(10);
    return;
  }

  for (int i = 0; i < NUM_SENSORS; i++) {
    long raw = touchRead(sensors[i].pin);
    long clamped = constrain(
      raw,
      sensors[i].touchMin,
      sensors[i].touchMax
    );
    int cc = map(
      clamped,
      sensors[i].touchMin,
      sensors[i].touchMax,
      0,
      127
    );

    if (abs(cc - sensors[i].lastCC) > 1) {
      MIDI.sendControlChange(
        i + 1,
        cc,
        1
      );
      sensors[i].lastCC = cc;
    }
  }
  delay(10);
}

Credits

Beste Cebeci

4 projects • 1 follower

Heba Elidrisi

4 projects • 1 follower

Memoirs of Disappearing Earth