Magenta, Google's research project that looks at how AI can help people become more creative, has just launched an experimental open-source instrument called NSynth Super. NSynth Super uses NSynth from Magenta, Google's neural network that generates sounds, and instructions to build your own Super NSynth are available from Github.
The NSynth Super is a piece of hardware that brings NSynth technology to life. As Douglas Eck, a research scientist in the Google Brain team, says in the video above, NSynth does not generate notes, but the real sound of an instrument. The NSynth algorithm learns the basic qualities of what constitutes an individual sound and can then combine sounds to create something completely new.
The hardware that Google has designed to play with the NSynth algorithm has a central X / Y pad in which each quadrant can be assigned an instrument (something like using effects in a Kaoss Pad). These instruments can be mixed by sliding your finger across the pad. What is particularly unique is that NSynth Super is not just superimposing sounds on top of each other. Rather, it is synthesizing a completely new sound based on the acoustic qualities of the individual instruments. This gives some unexpected results. In the previous demo video, mixing a flute and a box makes the sound glassy and almost clear, without any open "drum" quality.
NSynth Super synthesizes completely new sounds based on the acoustic qualities of individual instruments
The NSynth Super has some parallels to the German-made Hartmann Neuron, a polyphonic synthesizer produced between 2002 and 2005 that analyzes and creates sound models that can be controlled in various ways. Finally, although the idea was novel and the synthesizer created otherworldly noises, it was quite expensive and not at all easy to use.
NSynth Super can be played through any MIDI source, such as DAW, sequencer or keyboard, and has built-in parameter settings, such as basic ADSR (attack, decay, sustain, release). Although the NSynth Super is not available for purchase, Google has offered instructions to create one from scratch using Raspberry Pi. It even includes details on how to create the PCB (printed circuit board) and the schematics of the housing. The instructions will not give you the nice rainbow screen that you see in the Google prototype, but you will get a fantastic neural instrument that generates tremendously ingenious sounds.