Warning: Potentially Loud!

I didn't really know how to go about this, however. I started with the idea that I'd sample the whole song and turn that into an instrument somehow, but that didn't quite sound how I wanted it to. I then had the idea to turn the song into a Midi file and create a waveform based on the midi notes.

It took some experimentation, but ultimately I got something interesting by taking the notes at each time (averaging any chords to get a single value), mirroring the resulting "graph" (to make sure the waveform was continuous) and smoothing it all a little.

Here's a little visual -- grey points are the original notes, white are the averaged note values, and then we linearly interpolate between the points!

Then we flip and apply some smoothing and we have our waveform! You might notice that the smoothing is fairly aggressive -- I tried a bunch of different values for the smoothing coefficient and this consistently resulted in more interesting sounds than leaving things more jagged.

Waveform in hand the next step was to make it do something. My synth knowledge is shamefully weak, and I didn't have much experience programming synths. I ended up grabbing Tone.js and trying to feed my custom waveform in as the main oscillator frequency. It sounded horrible.

I spent a very long time grappling with Tone.js and trying to jam my waveform just about anywhere I could make it fit. I ultimately ended up using the waveform to shape a pair of LFOs and used those to modulate parameters of an FM synth (both the detune of the synth and the modulation). I feel like I may have done it wrong based on how much difficulty I had getting Tone.js to do what I wanted to, but I'll chalk that up to not knowing what the hell I was doing.

I'm still not entirely satisfied with the sound -- I really wanted a super distinct difference between songs. The difference is there, but I feel like with a little more knowledge I could have really milked the waveform and made it do much cooler things. Some feedback loops or something. As it stands I slowed down each song to 1/4 speed so the LFO can be heard (and to make things more mysterious).

For the visualization I knew I wanted to try writing a webGL shader. I don't have ways to express how long I've wanted to learn how to write shaders! I haven't exercised my math muscles in centuries, but I managed to get the visualisation I wanted: A void to stare into while the music played. It doesn't stare back, per say, but if you stare long enough it will mess up your vision.

Overall I'm pretty happy how this one turned out! Ultimately this is a very basic shader, a very simple Tone.js project, and a very weak understanding of sythesizers -- but I have always been a breadth-over-depth kind of person, and I thoroughly enjoyed dabbling in each of the different aspects of this project.

Turns out I kinda miss math.

If you want to find more midi files to try out with this project, I recommend the old Kingdom Hearts Insider midi collection.