The Shape of the World
Just before leaving for my tour in Japan, a new piece appeared, and I called it “The Shape of the World”. I use only four strings of the guitar to play it, but somehow it sounds like a small orchestra.
On the fourth string, I use a harmonizer with delay that creates the bass part, the rhythmic foundation and what is effectively a tenor harmony line. The third string produces a different kind of chord, again created with the Eventide harmonizer, then compressed, filtered and passed through a glitch generator.
The melody on the second string is equalised to bring it closer to the tone of an electric archtop, while the first string feeds several harmonised delay lines that open up when the chorus arrives.
Making the guitar control the visuals
One of the distinctive aspects of my music is that I use a guitar with a polyphonic pickup, made by Cycfi Research, which allows me to process each string independently and apply completely different effects to each one.
This makes it possible for me to perform pieces alone that would otherwise be almost impossible to play live in their original form, and of course the resulting sound would also be completely different if I approached the same music in another way. So I started wondering whether I could treat the video signal in exactly the same way as I treat the audio signal.
Some time ago I bought EboSuite, a set of video tools that integrates with Ableton Live. I should probably point out that this experiment actually began in 2018, and it was only around 2024 that I began to understand how I might implement it seriously.
Instead of manipulating video clips with effects using a DJ controller, which is more or less what some of the demonstrations suggest and is not really the aesthetic I am interested in, I decided to use its ISF generator and create six different visual paths, one for each guitar string, with various parameters controlled by envelope followers.
ISFs are essentially programmable visual effects. Instead of simply playing back a prerecorded video, the computer generates or transforms the image in real time, which means that the visuals can change and react while the performance is happening.
Technically, ISFs are GLSL shaders accompanied by information describing their available controls and inputs, which allows the same visual effect to be used in different compatible applications.
Designing the visuals around the music
I have no idea where the title “The Shape of the World” came from, but since it already had such a strong visual quality, using planet Earth as the fixed centre of the image seemed like a fairly obvious place to begin.
I wanted the whole thing to feel hand drawn… in fact, considering the amount of work I eventually did manually in Photoshop, I suspect I might have saved time by drawing the whole thing frame by frame myself.
I then created the ISFs using a kind of chalkboard aesthetic.
The fourth string controls a series of small lines around the planet that appear only when the string is played. Since I play that part in constant eighth notes, the result is a very natural pulsation that mirrors something already present in the music.
For each note of the melody, I made the stars move. This is controlled by an ISF made of small dots with an expansion parameter, so every note causes the visual field to shift slightly. Then there is the Earth’s “sign of life”, a signal that seems to spread outward into space, which is controlled by the chord played on the third string.
When the chorus arrives, I have the fifth and sixth strings muted musically, so I used them instead as controllers to trigger the change in the colour of the background.
The guitar is still playing the video, even when those strings are not producing any sound.
When a guitar note becomes a visual event
Of course, none of this was quite as straightforward as it sounds. For example, I spent almost an entire day trying to make the Earth Signal, triggered by the third string, behave exactly as I had imagined it.
The first version used the string’s Envelope Follower to control the animation directly. It worked, except for one rather important detail: as the note decayed, the envelope fell, and the signal travelled back towards Earth, which was technically correct but visually completely wrong.
Eventually I abandoned that approach and rendered the animation as a .mov file with an alpha channel, so the background remained transparent. More importantly, I changed the question I was trying to solve. Instead of asking the guitar to control the animation continuously, I needed the pluck to become an event: “play this animation now”.
Finding a way to make that apparently simple thing happen turned into another small adventure. My solution: the Envelope Follower now feeds a MIDI Gate Trigger, which generates a B-1 note when the threshold is crossed. Note-Off Killer removes the corresponding Note Off, and Tom Cosm’s MIDI Note to Clip Slot Trigger uses that single note to launch slot 24 on the neighbouring eSession track.
But now a single pluck on one string becomes one discrete visual event. The animation starts at Earth, travels into space and completes on its own, regardless of how long the guitar note sustains.
It took a surprisingly long route to get there, but I am extremely happy with the result. NOTE: Read the full story https://fabiomittino.substack.com/p/the-shape-of-the-world
