Open DJ Project: AI for automatic and personalized DJing
Since Feb 1, 2017
Project member: Yu-Siang HuangVibert Thio
Keywords: DJMashupSequencingThumbnailingMedley

Disc Jockeys (DJs) are professional audio engineers whose role is to generate music artwork such as electronic dance music (EDM) and to manipulate musical elements to create music medley, mashup and remix. It is a profession that adds values to music. For end users, DJ music has become a major way to enjoy and engage with music. It has been very popular in Western countries to go to a live concert featuring the performance of a human DJ, and we can see similar trends in Asian countries including Taiwan, Korea, and Japan. DJs are reshaping the music industry and the way people interact with music.

There are three possible major applications if we have an AI model that masters the DJ skills. Firstly, the AI DJ model can work in tandem with an AI music composing model (let’s call it “AI composer”) to make more interesting music. While a human composer typically focuses on the creation of each individual musical piece one by one, a human DJ knows how to combine existing music pieces in an innovative and pleasant ways. The joint effort of AI DJ and an AI composer would therefore likely generate more interesting music. Such AI-generated music can find applications in advertisement (i.e. generating copyright free music to be used in an Add), gaming, and video accompaniment, etc.

Secondly, smart speakers such as Amazon Echo are getting popular. An AI DJ can be deployed on smart speakers to select and playback music in a personalized way. For such applications, the AI DJ has to learn the techniques and skills of a professional human DJ. Therefore, it is addressing the challenge whether we can build a fully- or semi-automatic DJ.

Thirdly, AI can also help human beings who want to become a professional DJ to learn DJ skills. Typically, it takes years of training for a novice to become a DJ professional. An online platform that analyzes existing tracklists created by professional DJs can help make this learning curve less steep and encourage more people to appreciate and to learn to become a DJ. This is fairly different from the aforementioned two applications in that in this one the AI plays a role of assisting people in acquiring knowledge and skills.

We are interested in all the three aforementioned applications. In parituclar, we have built the following models thus far:

  • Music thumbnailing/highlight extraction (TIMISR'18) (demo)
  • Music sequencing (AAAI'18) (demo)

We are currently working on the following topics:

  • Music mashup generation
  • Music medley generation
  • Drum fill generation