Musical Gestures Toolbox

The Musical Gestures Toolbox (MGT) is a collection of tools for creating visualizations of video files. It was primarily developed for music researchers, but is also used for sports, dance, healthcare, architecture, and interaction design.

Standalones

VideoAnalysis

If you just want a simple solution for creating video visualizations, check out VideoAnalysis (non-realtime) and AudioVideoAnalysis (realtime). These are standalone apps for OSX and Windows. The toolboxes below all require that you know some programming (Max, Matlab, Python, Bash).

Musical Gestures Toolbox for Matlab

The Matlab port of MGT contains a large collection of video visualization functions, batch processing, and server processing. It also integrates with the MIR Toolbox and MoCap Toolbox.

Musical Gestures Toolbox for Python

The Python port of MGT is the latest edition, and the one in most active development. Most of the functionality of the Matlab version has been ported. It can be run from Jupyter Notebook. 

Musical Gestures Toolbox for Max

The original MGT was a collection of realtime modules for Max/MSP/Jitter. The toolbox is currently distributed as part of the Jamoma package.

Musical Gestures Toolbox for Terminal

Musical Gestures Toolbox for doing various video-related tricks in the terminal. This is mainly based on the power of FFmpeg and ImageMagick.

History

The Musical Gestures Toolbox has been developed by Alexander Refsum Jensenius since 2004. It started out as a patcher for the graphical multimedia programming environment Max/MSP/Jitter, and was quickly merged into modules and components in the Jamoma project. The modules have been used in a number of music and dance performances over the years.

When the course MUS2006 Music and Body Movements start up at University of Oslo in 2009, it was necessary to provide the non-programming students with an easy solution for creating video visualizations. The result was the standalone app VideoAnalysis, which saw a big overhaul in 2020.

For scientific usage, functions from the original toolbox were ported to MGT for Matlab in 2015 and MGT for Python in 2019. With the need to run some code on servers, there is now also a more limited version of MGT for Terminal. All of these scripting-based toolboxes are slightly different in implementation, but they share the vision of creating powerful, yet fairly simple to use, operations for video visualization.

The software is maintained by the fourMs lab at RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion at the University of Oslo.

Publications

License

All the different versions of the Musical Gestures Toolbox are open source, and are shared with The GNU General Public License v3.0.