We use AI Listener as a blanket term to refer to research topics that are related to music and sound analysis, such as source separation, music transcription, structure analysis, sound event recognition, instrument recognition, and emotion recognition. The goal is to learn computer models that can understand and appreciate music and audio signals in the same way as human beings. Such models can find applications in music retrieval (e.g. similarity search, content based recommendation), music education, and also music generation.

The goal of AI Composer is to build machines that can compose new music. In particular, an AI Composer creates music in the so-called "symbolic" domain, in formats such as melody lines, lead sheets, or multitrack pianorolls (similar to MIDIs). The generation of such musical scores could be either unconditioned (from random seeds) or conditioned (e.g. given a prime melody, given a chord sequence, given the lyrics, given some tags such as genre or emotion tags, or given an image or a video sequence). We hope that our AI Composer models can help musicians or music lovers create music in an interactive way.

Music creation is typically composed of two parts: composing the musical score, and then performing the score (with certain instruments) to make sounds. The musical score mainly specify what notes to be played, but usually not how to play them. Musicians leverages this freedom to interpret the score and add expressiveness in their own ways, to “bring the music to life”. Accordingly, in addition to AI Composer models that deal with symbolic-domain music generation, we need AI Composer models to deal with audio-domain music generation and create musical audio from scores.

Disc Jockeys (DJs) are professional audio engineers whose role is to generate music artwork such as electronic dance music (EDM) and to manipulate musical elements to create music medley, mashup and remix. It is a profession that adds values to music. In addition to generating new music from scratch using AI Composer models, we can also build AI DJ models that generate new music by reusing, recombining and manipulating existing musical pieces, perhaps directly in the audio domain. In addition to considering AI DJ as another way to generate music, we believe AI DJ can also be deployed on smart speakers to deliver personalized music recommendation in a DJ way.