Making “Cognitive Music” with IBM Watson

Share this post:

It’s no surprise to find music at the center of many important moments of our lives – think of a wedding march, a “Happy Birthday,” or even a funeral procession. Music is deeply tied to the human experience; each song and each moment is personal to the listener. Over the last year, Watson has been studying up on music themes, theory, moods, and emotions – and how they correlate and connect with each other.

Now, Watson is inspiring musical creativity with Grammy-winning music producer Alex Da Kid, who used Watson’s technology to inspire his new song about heartbreak, “Not Easy.” This is the first song in Alex’s 4-track EP in collaboration with Watson.

For this partnership, Watson analyzed the last five years of culture and music data to uncover new emotional insights to augment Alex’s creative process. To identify the most pervasive themes, the team used the Watson Alchemy Language API to read and understand Nobel Peace Prize speeches, New York Times articles, Billboard song lyrics, movie synopses and more.

The Watson Tone Analyzer API then ingested more than 2 million lines of related social content to understand the emotional sentiment surrounding these themes. Combined, these APIs helped us to build the “emotional fingerprint” of each of the last five years.

Alex then tapped into Watson BEAT to examine and better understand popular musical trends. And Watson BEAT is more than your average music-suggestion engine. Today, music-recommending apps use genre terms and ratings to suggest similar songs we might like. For example – if you like Wiz Khalifa’s 2015 hit “See You Again”, you might also like Maroon 5’s 2012, “Payphone” (featuring Wiz Khalifa). They recommend based on other songs, not how a person feels in the moment.

Using machine learning algorithms, Watson BEAT is capable of learning from songs by deconstructing a song’s pitch, time and key signatures, note sequence, and note velocity (how hard a note was struck).

Combined with theories about emotional responses to music, Watson BEAT can generate completely new musical scores based on a variety of preferred moods (joyful, devastated, anxious etc.) or feelings (spooky, mysterious, cheerful, etc.).

Using this technology, Alex could create new songs – even snippets of songs, like a bass line – until he found a sound that inspired him. And while “Not Easy” is an original work, Watson BEAT gave Alex many new ways to experience musical themes and human emotions.

To dive even deeper into the insights, the IBM team used the Cognitive Color Design Tool to create an interactive visualization of the data based on the images, album artwork and colors that inspire Alex. The tool blends together understanding of psychological effects of colors, the interrelationships between emotions, and image aesthetics to create a custom color palette, tailored to the individual.

At the heart of this collaboration is the power that artificial intelligence brings to human expertise. ‘Cognitive Creativity’ – augmenting human creativity with the power of AI, allows us to create ideas, stories and concepts previously unimagined. Watson’s ability to understand, reason and learn at scale using natural language is emerging as a source of creativity in a number of industries – from the culinary world (Chef Watson), to fashion (Cognitive Dress) to entertainment (Cognitive Movie Trailer).

Today, Watson is working in music – the team at IBM is already imagining where Watson’s ability to inspire will go next.