Neural synth

A variational autoencoder (VAE) is a type of neural network that learns how to represent high-dimensional data with only a few numbers. In the case of music, we can train an autoencoder to represent all the notes and chords you might play on a piano using only two variables. You can think of this as the network learning a relationship between (x,y) values and a different collection of notes. To generate music, click on the pad below to choose an (x,y) value, and the autoencoder will use that value to generate a collection of notes.

The generating process is probabilistic, which means the same location on the pad won't generate the same exact notes every time. Still, nearby locations on the pad correspond to similar sounding chords, which makes it easy to learn to control after you play around with it for a while. For example, if you click in the top-left of the pad below, you will likely play a C major chord, while clicking in the top-right of the pad will usually play an A minor chord.

The notes the network generates depends a lot on the data you train it with. In this case, the autoencoder was trained on 382 of Bach's four-part chorales (source), with the songs transposed to either C major or A minor.

Listen below to the results when you use a VAE (and two other similar models) to generate an entire song.

Code and more information is available here.