Improvasher: a real-time mashup system for live musical input

Matthew E. P. Davies1, Adam M. Stark2, Fabien Gouyon1 and Masataka Goto3
1INESC TEC, Porto, Portugal
2Independent Researcher, London, United Kingdom
3National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan

This webpage contains video examples to accompany the paper accepted at the NIME 2014 conference.

Demo 1 – Live Guitar / Michael Jackson mashup


In this demo, beat slices from two Michael Jackson songs are used to make the real-time mashup accompaniment.


Demo 2 – Average White Band / Slap Bass mashup


In this demo, a recording of slap bass playing is used to make the real-time mashup to the Average White Band.


Demo 3 – Live acoustic guitar / Bass mashup


In this demo, we use the bass to accompany the live acoustic guitar input.


Demo 4 – Steely Dan DJ style live mashup


In this demo the user controls when the mashup is active adding it over the top of the input in real-time.

 

This research was partially funded by the FCT postdoctoral grant (SFRH/BPD/88722/2012), the Media Arts and Technologies project (MAT), NORTE-07-0124-FEDER-000061, financed by the North Portugal Regional
Operational Programme (ON.2 – O Novo Norte), under the National Strategic Reference Framework (NSRF), through the European Regional Development Fund (ERDF), and by national funds, through the Portuguese funding agency, Fundação para a Cîencia e a Tecnologia (FCT), and by OngaCrest, CREST, JST.

WEVE

This is a CouchApp for conducting automated listening tests online. CouchApps are web applications which can be served directly from CouchDB. This gives them the nice property of replicating just like any other data stored in CouchDB. They are also simple to write as they can use the built-in jQuery libraries and plugins that ship with CouchDB.

 

Wikifacets

Given a music tag as query, this system permits to classify the tag by means of semantic facets (e.g. genre, instruments, locale, etc.), and reversely, given a facet as query to obtain the most representative tags for that facet. The semantic facets are anchored upon the structure of Wikipedia and last.fm.

The system is described in our paper: Sordo M., Gouyon F., Sarmento L., Celma O., Serra X., “Inferring Semantic Facets of a Music Folksonomy with Wikipedia”, Journal of New Music Research, in print

The code and documentation is available here.

 

BeatStation

An open-source annotation tool for recording tapping data from real-time audio using the keyboard or a midi interface. The software was used during the ISMIR 2012 conference for a beat tapping experiments.

  • Binaries for Windows, Mac OS X and Linux can be found here.
  • The source code and build instructions can be found on GitHub

The system is described in a paper presented at SMC 2013

GimmeDaBlues

GimmedaBlues is a application for iPhone/iPod Touch/iPad devices that allows anyone to play jazz keyboard and solo instruments along a predefined harmonic progression, using the multi-touch properties of the iOS devices.

While the user plays keyboard and/or solo instruments, the application automatically generates the bass and drums parts, responding to the user’s activity.

Dynamic mapping of the notes and chords available in the graphical interface provides an intuitive and natural way to play otherwise complex chords and scales, while maintaining a physical playability that will be familiar to experienced keyboard players, and provides an entertaining, yet challenging experience for non-musicians.

gimmedablues

Recombinator

Recombinator is a software application for recombining in real time MIDI drum loops that makes use of a novel analysis of rhythmic patterns that sorts them in order of their complexity.

We measure rhythmic complexity by comparing each rhythmic pattern found in the loops to a metrical template characteristic of its time signature.

The complexity measure is used to sort the MIDI loops prior to utilizing them in the recombination algorithm. This way, the user can effectively control the complexity and variation in the generated rhythm during performance.

This software is part of the Kinetic.Toolbox.

 

Rhythmicator

Rhythmicator is a novel Max/MSP application for automatically generating rhythms in real time in a certain meter. The generated rhythms are “generic” in the sense that they are characteristic of each time signature without belonging to a specific musical style.

The application is based on a stochastic model in which various aspects and qualities of the generated rhythm can be controlled intuitively and in real time.

During a performance with the Rhythmicator, the user can control the density of the generated events per bar, the amount of variation in generation, the amount of syncopation, the metrical strength, and of course the meter itself, through descriptive and intuitive graphic controls.

 

This software is part of the Kinetic.Toolbox.
Download Here

IBT

IBT (INESC Porto Beat Tracking software)

A real-time system for tempo induction and beat tracking of music audio signals written in C++ and part of the free MARSYAS framework (see IBT documentation).

Here is a scientific paper explaining the software.

Binaries (command line)

Max/MSP audio beat tracking external

Pure Data audio beat tracking external

Sonic Visualiser audio beat tracking Vamp plugin

**Note, this plugin does not currently work with Sonic Visualiser. However it will run with version 0.7 of Sonic Annotator. Many thanks to Chris Cannam and Matthias Mauch for pointing this out.

RoboNova dancing to Blur. Each sequence of moves is synchronized to the musical beats on-the-fly, retrieved from the real-time INESC Porto Beat Tracker (IBT). [paper1]; [paper2]