Digital Synth Modulin (Wintergatan-Inspired)

Story

I discovered Martin Molin (member of the band Wintergatan) about a year ago and was amazed at the instruments he built and the magic he could make with music. I absolutely loved his modulin, which is what he called his modular synthesizer that he played like a violin. He has inspired many people to create their own instruments and covers of his music. I am one of those! I decided I wanted to build my own modulin, but found out that modular synthesizers easily go over US$1000, and realized I wouldn't be able to afford it, especially since import taxes in Brazil often double the price of electronics and the currency is a bit weak relative to the US dollar there.

For the past 5 years I studied control theory and automation, and we got some of the basic experience necessary to build, program and test circuits. I decided I would build my own modulin from scratch. I wanted to have something I could enjoy for a long time, so I took the time to find the parts I wanted to use and how I would build the modulin, even though there was very little online specifically oriented to what I wanted to create: a self-contained, portable instrument that I could take with me and play music on the go.

I was able to get the ribbon controllers (composed of a position sensor and a pressure sensor) as samples due to my status as a student and mentor to others, thanks to the kind folks at Spectra Symbol and Interlink Electronics. The other components were often relatively cheap to get from China through AliExpress, but also took some time to receive, due to customs. One of the most important components, the microcontroller, would turn out to be a difficult one to choose.

I was thinking about using the Teensy 3.6 or the ESP32, mostly because of their high clock speeds and floating point capabilities, when I found the Make it Better contest. When I saw Sony's name, I was sure their board would be ideal for a music-focused project and immediately submitted my idea. This would be my first time participating in an electronics contest, but I was confident I could make a great project!

Modulin in Action

Main Components

The modulin is composed of 3 main parts, or 4 if you include the structure, which I built as well.

• There are the sensors, which are the linear potentiometers that the user can press and vary both position and pressure to control the sounds that get played, as well as the vibrato and other programmable characteristics of the sound.
• There is the microcontroller, which captures sensor signals and interprets them to generate a signal that will produce sound through the output. It (and other electronic components) is powered by a battery, which in this case is just a basic powerbank supplying 5 V.
• There is the output stage, composed of an amplifier that receives a signal from the microcontroller, and speakers, which receive a reinforced sound signal from the amplifier and turn that electronic signal into beautiful music.
• The wooden structure is the last component, which I built myself to house and protect all of the electronics, battery and support the fragile, flexible sensors.

Steps

Circuit

The first step in building the modulin is to put together the circuit as described in the Fritzing diagram below. In most cases the wires should be soldered in place, but you can also use jumper wires to allow for testing and adjustments. Note that jumper wires will result in a noisier signal and hence a noisier sound.

Softpot

The softpot potentiometer is used like a standard potentiometer with three pins. You can use 1 pin to acquire a signal that varies linearly with the position you press, but I decided to test out a more advanced circuit, in which I add 2 pull-up resistors (1kOhm + 2kOhm trimpot at 5 V level) to each outer pin of the softpot and connect the middle pin to ground. This allows me to measure a voltage at each outer pin and press the potentiometer at two positions, allowing for two notes to be played simultaneously from one ribbon. These two outer pins should also be connected to the A4 and A5 analog input pins of the Spresense extension board, which allows for fast measurements of the inputs.

ForceSensingResistor

The Force Sensing Resistor has two pins and should also be connected to 5V and a 10kOhm potentiometer. The side connected to the potentiometer is also connected to the Spresense board on the A3 analog input. The potentiometer should have its middle pin connected to ground.

I opted for potentiometers in both cases here so that I can later adjust the resistance as needed to "tune" the range of the ribbon controller.

Structure

At this point you can remove the adhesive backing from the Force Sensing Resistor (FSR) and attach it to the structure of your choice (in my case I built a box from screws and pieces of wood that I bought precut at my local contruction supply store). It is important to attach the FSR first as it is longer and should remain smooth on the surface of the structure you used. You can then remove the adhesive from the softpot and attach this directly on top of the FSR. This will allow your modulin to measure both position and pressure from the same finger press. I chose to align the ends with pins, so that I could have the FSR trailing end exposed. This allows you to program the modulin to respond differently if you press the bare FSR alone (without touching the softpot).

Power

The breadboard should have its power rails connected to 5V (red) and ground (blue), which is what I designed the circuit for. I connected a powerbank to these rails using an old USB cable that I cut to access the power lines.

Amplifier

Unfortunately the sound amplifier, PAM8403 did not seem to have a Fritzing part, so I adapted a photo of it to show which connections go where. Each of the speakers should be connected separately to the appropriate locations on the amplifier. The pins in the middle show the power connections, which are also 5 V and ground. The pins on the left are connected to an audio jack adapter board, with the pin labeled left connecting to the tip pin on the adapter board (T). The pin labeled ground is connected to the sleeve pin on the adapter board (S). Lastly the pin labeled left is connected to the ring pin on the adapter board (R). The adapter board is also known as a TRRS breakout, for the pins that it uses. The amplifier can now be connected through the TRRS breakout using a standard 3.5mm audio cable.

UploadingCodeandTesting

Once all of these connections are in place, you should double check them and then connect the Spresense board to your computer with a USB cable. Ensure that the jumper pin on the board are set for 5V on the extension board and turn on your powerbank. You can then upload the code below to the Spresense (assuming you have already uploaded the appropriate bootloader to the board according to the starting instructions included with the board). Once the code has been uploaded, you can press the softpot and listen for sounds. You can adjust the potentiometers (ideally they should have the same values) so that you can use the softpots over their entire range (this is necessary due to the use of the technique allowing for 2 notes to be played simultaneously).

You can adjust the synthesizer parameters in the code (such as the ADSR values). The ADSR values represent an envelope that forces the volume of the sound to stay within fixed ranges as the note is being played. The code below the bullet list shows the default values set in the code for these. Note that the values are in microseconds, so 500000 represents 0.5 seconds for the attack time, for example. The amplitude values (attackAmp and sustainAmp) are represented in dB for use by the Spresense board, so a more negative value is lower in volume.

• The A (for attack) value represents how quickly the volume rises from zero as soon as the note starts being played. You can also adjust the attack amplitude, which is how high the volume rises at the end of this step.
• The D (for decay) value represents how quickly the volume drops after hitting the maximum value, until it reaches the sustain level.
• The S (for sustain) value represents the volume the note remains at after the decay step, while the note is being pressed.
• The R (for release) value represents how quickly the note drops off after the note is released.

 envelopeADSR(){
   attackTime = 500000.0;
   decayTime = 200000.0;
   attackAmp = -25;
   sustainAmp = -29;
   releaseTime = 500000.0;
 }

In addition, other things can be implemented and adjusted, including the type of oscillator used for creating the sound wave, including sine waves, sawtooth waves, square waves, and triangle waves. These allow the user to generate diverse sounds, including adjusting to create sounds that mimic real instruments.

That's it! You've got a fully functional digital synthesizer. Over time I will be adding new features to the project (especially the Github repository), including support for multiple ribbon controllers, modulation wheels for controlling pitch bending and volume, RGB LED strips on the sides to follow the sounds being played, and additional parameters to adjust the sounds produced.

Below are a couple of the first tests that I performed as I built the modulin. Note that these were at testing stages in which not all of the software/hardware components were complete, unlike what is included above.

End Notes

I was somewhat disappointed in the lack of documentation of the Spresense board and its focus more on the NuttX development environment. This impeded developing many functions for my project, as I spent lots of time trying to figure out why a function I was accustomed to using for Arduino projects did not work the same way for the Spresense. I think a lot of headaches/frustration can be saved by having more complete Arduino IDE documentation for the Spresense board. I imagine some users may have been discouraged by not finding what they needed or having difficulties they could not explain.

A couple of personal frustrations for me were that the mainboard and extension boards do not fit snuggly together and will often come loose on a table from simply connecting/disconnecting the USB cable. The plastic pins that should hold the two boards together do not hold the two boards tightly together. The second frustration is that the DAC used in creating the sound signal that is used for the sound jack is not a proper DAC in that it does not generate an analog waveform, but rather a Pulse Density Modulation (PDM) signal, which is quite difficult to analyze and observe on an oscilloscope, if one is expecting a true analog waveform.

It seems the board has some improvements to make, but considering it is the first iteration that has been made available to the public, it is understandable that there will be some issues. Hopefully Sony will take user feedback into account and roll out updates to improve user experience and capture the hobbyist/enthusiast crowd as the Arduino boards have done.