Published October 1, 2017 © GPL3+

Copy of Paper Piano

Playing music with paper instruments.

BeginnerFull instructions provided2 hours6,923

Things used in this project

Hardware components

Arduino UNO

Jumper wires (generic)

SparkFun Multiplexer Breakout - 8 Channel (74HC4051)

Software apps and online services

Processing

Hand tools and fabrication machines

aluminium foil

Story

Components

1x Arduino UNO

1x 16 channel multiplexer

jumper wires

aluminium foil (for paper instruments development)

What are the paper instruments made of?

Using aluminium foil, we were able to create a paper piano & guitar that plays notes and changes graphics on a screen.

Step by step - Paper Instrument Development

Software needed

Processing: https://processing.org/

Capacitive Arduino Library: http://playground.arduino.cc/Main/CapacitiveSensor?from=Main.CapSense

1. Draw paper piano

Use conductive ink to draw the keyboard. Make sure to not have any of the lines intersect (as the lines act as wires to the Arduino). Also, ensure that there is no gaps in your ink lines as a gap in the paint would be equivalent to a cut wire (it won't work!).

2. Wire up hardware

Follow the Fritzing Schematic on the left. The Arduino Capacitive Sensor library (see useful resources) has a great description of how the conductive painted keys act as capacitive sensors.

Before hooking up all 12 sensors (or however many you want), test your setup on one sensor first and running the Arduino sample code (http://playground.arduino.cc/Main/CapacitiveSensor?from=Main.CapSense). To wire up everything, connect a resistor (~1Mega Ohm) from Digital Pin 4 to Digital Pin 2. Connect Digital Pin 2 with a wire to the painted key. A quick way to do this is to simply tape the wire so it is physically touching the paint. (For those who are interested in a more robust ways, here is a great site for examples of doing hard/soft connections: http://www.kobakant.at/DIY/?p=1272.) Now that everything is wired up, upload the Arduino sample code and open the Serial port. You should see the values change as your finger touches the painted key. To play around with the sensitivity (how large of a change in value you read) play around with different resistors.

Now that you've got one fully working, let's connect the other 12 (or however many others you want!). In order to save the number of pins the Arduino uses, we'll use multiplexing. Multiplexing is a great cheap way to maximize the amount of analog input/output pins. For this project we will use a 16 analog multiplexer which will use a total of 5 pins on the Arduino (4 control Digital Pins and 1 Analog read pin). From this 1 Analog read pin on the Arduino we will cycle between all 12 notes (you can expand to 16). As we are only using 5 pins on the Arduino you can see that you can add even more painted keys with more multiplexing.

Follow the Fritzing schematic exactly (and reference the Useful Resources multiplexing link) and connect your painted keys to the multiplexor in the order you want to read them. Now run the code attached. Your serial monitor should now read all the keys and spit out the values, which are delimited (separated) by a semicolon (we use this in processing in the next step to separate all the signals). If this is working so far you're in great shape!

3. Connect Arduino to Processing

The final step is to connect Arduino to the Processing sketch. Essentially the Arduino is sending all the piano key values to processing through the Serial port. As such, you have to define which serial port your computer is using. Change the following line in the processing sketch:

String portName = Serial.list()[2];

(On my computer it is 2, but it might be different on yours.) And as well, the baud rate (i.e., in the Arduino sketch the line: Serial.begin(115200)) must match the processing sketch line myPort = new Serial(this, portName, 115200);.

The cool graphics is open sourced from Open Processing (see Useful Resources Processing sketch) and the midi controller to make the piano sounds is a Java library.

This should get you set to run 12 painted keys through the Arduino and create cool graphics and piano sounds through your laptop! You can definitely tweak the components (such as the resistors) to make the instrument as sensitive to touch as you want, as well as add more painted keys (since the multiplexer saved some pins on your Arduino!).

Code

#include <CapacitiveSensor.h>


float matrixValues[12];

byte controlPins[] = {
  B00000000, 
  B10000000,
  B01000000,
  B11000000,
  B00100000,
  B10100000,
  B01100000,
  B11100000,
  B00010000,
  B10010000,
  B01010000,
  B11010000,
  B00110000,
  B10110000,
  B01110000,
  B11110000 }; 



CapacitiveSensor   cs_8_0 = CapacitiveSensor(A0,8); 

void setup()
{
  Serial.begin(9600);
  cs_8_0.set_CS_AutocaL_Millis(0xFFFFFFFF);  
  DDRD = B11111111; // set PORTD (digital 7~0) to outputs
}

void setPin(int outputPin)
// function to select pin on 74HC4067
{
  PORTD = controlPins[outputPin];
}


void loop()
{
  for (int i = 0; i < 12; i++)
  {
    setPin(i); // choose an input pin on the 74HC4067
    float total1 =  cs_8_0.capacitiveSensor(30);
    matrixValues[i]=total1;
    Serial.print(matrixValues[i]);
    Serial.print(",");
  }
  Serial.println();
  delay(100);


}

import themidibus.*;

import processing.serial.*;
import java.util.ArrayList;


class particle {

  float x;
  float y;
  float px;
  float py;
  float magnitude;
  float angle;
  float mass;

  particle( float dx, float dy, float V, float A, float M ) {
    x = dx;
    y = dy;
    px = dx;
    py = dy;
    magnitude = V;
    angle = A;
    mass = M;
  }

  void reset( float dx, float dy, float V, float A, float M ) {
   x = dx;
   y = dy;
   px = dx;
   py = dy;
   magnitude = V;
   angle = A;
   mass = M;
   }
   
   void gravitate( particle Z ) {
   float F, mX, mY, A;
   if( sq( x - Z.x ) + sq( y - Z.y ) != 0 ) {
   F = mass * Z.mass;
   mX = ( mass * x + Z.mass * Z.x ) / ( mass + Z.mass );
   mY = ( mass * y + Z.mass * Z.y ) / ( mass + Z.mass );
   A = findAngle( mX - x, mY - y );
   
   mX = F * cos(A);
   mY = F * sin(A);
   
   mX += magnitude * cos(angle);
   mY += magnitude * sin(angle);
   
   magnitude = sqrt( sq(mX) + sq(mY) );
   angle = findAngle( mX, mY );
   }
   }
   
   void repel( particle Z ) {
   float F, mX, mY, A;
   if( sq( x - Z.x ) + sq( y - Z.y ) != 0 ) {
   F = mass * Z.mass;
   mX = ( mass * x + Z.mass * Z.x ) / ( mass + Z.mass );
   mY = ( mass * y + Z.mass * Z.y ) / ( mass + Z.mass );
   A = findAngle( x - mX, y - mY );
   
   mX = F * cos(A);
   mY = F * sin(A);
   
   mX += magnitude * cos(angle);
   mY += magnitude * sin(angle);
   
   magnitude = sqrt( sq(mX) + sq(mY) );
   angle = findAngle( mX, mY );
   }
   }
   
   void deteriorate() {
   magnitude *= 0.925;
   }
   
   void update() {
   
   x += magnitude * cos(angle);
   y += magnitude * sin(angle);
   
   }
   
   void display() {
   line(px,py,x,y);
   px = x;
   py = y;
   }
   
   
   }
   
   float findAngle( float x, float y ) {
   float theta;
   if(x == 0) {
   if(y > 0) {
   theta = HALF_PI;
   }
   else if(y < 0) {
   theta = 3*HALF_PI;
   }
   else {
   theta = 0;
   }
   }
   else {
   theta = atan( y / x );
   if(( x < 0 ) && ( y >= 0 )) { theta += PI; }
   if(( x < 0 ) && ( y < 0 )) { theta -= PI; }
   }
   return theta;
   }
   
   
  
   
  String val;

  int maxNumberOfSensors = 12;       // Arduino has 6 analog inputs, so I chose 6

  boolean fontInitialized = false;  // whether the font's been initialized
  PFont myFont;                     // font for writing text to the window
  int STARTED =0;

  Serial myPort; 
  particle[] Z = new particle[200];
  float colour = random(1);

  boolean PRESSED=false;
  int screenCounter=0;
  PVector v1;
  long counter=0;
  MidiBus myMidi;
  ArrayList<Note> keyboard;
  boolean[] keyIsPressed;
  int[] lastPressed;
  final float KEYDOWN_THRESHOLD = 1000f;

  void setup() {
    smooth();
    size(500, 500, P2D);  
    background(255);
    v1=new PVector(width/2, height/2);
    for (int i = 0; i < Z.length; i++) {
      Z[i] = new particle( random(width), random(height), 0, 0, 1 );
    }

    /* Setup all sound */
    myMidi = new MidiBus(this, -1, 0);
    myMidi.sendTimestamps(false);
    keyboard = new ArrayList<Note>();
    int channel = 0;
    int velocity = 127;
    for (int pitch = 71; pitch > 59; pitch--) {
      keyboard.add(new Note(channel, pitch, velocity));
    }

    keyIsPressed = new boolean[keyboard.size()];
    lastPressed = new int[keyboard.size()];

    frameRate(60);
    colorMode(RGB, 255);
    
    printArray(Serial.list());
    //String portName = Serial.list()[5];
    String portName = "COM5";
    myPort = new Serial(this, portName, 9600);
    myPort.clear();
    myPort.bufferUntil('\n');  // don't generate a serialEvent() until you get a newline (\n) byte
  }

  void draw() {
    filter(INVERT);
    float r;
    stroke(0);
    fill(255, 10);

    rect(0, 0, width, height);

    colorMode(HSB, 1);
    for (int i = 0; i < Z.length; i++) {
      if (PRESSED==true) {
        Z[i].gravitate (new particle(v1.x, v1.y, 0, 0, 5));
        fill(colour, 10);
        ellipse(v1.x, v1.y, 20, 20);
      } else {
        Z[i].gravitate (new particle(width/2, height/2, 0, 0, 1));
        screenCounter++;
        if (screenCounter<5000) {
          Z[i].repel (new particle(v1.x, v1.y, 0, 0, 1));
          screenCounter=0;
        }
        Z[i].deteriorate();
      }
      Z[i].update();
      r = float(i)/Z.length;
      stroke( colour, pow(r, 0.1), 1-r, 0.15 );
      Z[i].display();
    }
    colorMode(RGB, 255);

    colour+=random(0.01);
    if ( colour > 1 ) { 
      colour = colour%1;
    }

    filter(INVERT);
  }

  void startNote(int keyIndex) {
    if (!keyIsPressed[keyIndex]) {
      Note note = keyboard.get(keyIndex);
      myMidi.sendNoteOn(note);
      keyIsPressed[keyIndex] = true;
    }
  }

  void delay(int time) {
    int current = millis();
    while (millis() < current + time) {
      Thread.yield();
    }
  }

  void endNote(int keyIndex) {
    if (keyIsPressed[keyIndex]) {
      Note note = keyboard.get(keyIndex);
      myMidi.sendNoteOff(note);
      keyIsPressed[keyIndex] = false;
    }
  }

  void serialEvent (Serial myPort) {
    String inString = myPort.readStringUntil('\n');  // get the ASCII string
    if (inString != null) {  // if it's not empty
      inString = trim(inString);  // trim off any whitespace

      /*System.out.format("Length: %d\n", length);
      double[] incomingValues = new double[length];
      for (int i = 0; i < length; i++) {
        incomingValues[i] = Double.valueOf(tmp[i]);
      }
      System.out.format("Double: %f\n", incomingValues[0]);
      */

      String[] tmp = split(inString, ",");
      float[] incomingValues = float(tmp);
      int length = incomingValues.length;
      if (length == maxNumberOfSensors+1 && length > 0) {
        for (int i = 0; i < length-1; i++) {
          counter+=incomingValues[i];
          /* Key is Pressed */
          if (incomingValues[i] > KEYDOWN_THRESHOLD) {
            PRESSED=true;
            v1.x=width/2-(i-6)*(width/2)/8 ;
            v1.y=height/2;
            colour+=random(0.01);
            if (i==1|i==3|i==5|i==8|i==10) {
              v1.y=height/2+random(-height/2, height/2);
            }
            startNote(i);
          }

          /* Key is not pressed */
          if (counter<1000) {
            PRESSED=false;
            endNote(i);
          }
          if (i==11) {
            counter=0;
          }

          if (i>6) {
            v1.x=random(width/4, width/2);
          } else {
            v1.x=(width/2);
          }
        }
      }
    }
  }

Credits

user4573

0 projects • 2 followers

Copy of Paper Piano

Things used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Components

What are the paper instruments made of?

Step by step - Paper Instrument Development

Schematics

Circuit Diagram

Code

Arduino Code

Processing code

Credits

user4573

Comments

Embed the widget on your own site

Copy of Paper Piano

Copy of Paper Piano

Things used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Components

What are the paper instruments made of?

Step by step - Paper Instrument Development

Schematics

Circuit Diagram

Code

Arduino Code

Processing code

Credits

user4573

Comments

Related channels and tags