Motion capture of pianists

By Lynda Joy Gerry (Aalborg Univ. Copenhagen), visiting researcher at RITMO during March-April 2019

Lynda Joy Gerry
Lynda Joy Gerry

During my monthlong collaboration with the Interdisciplinary Centre for Studies in Rhythm, Time, and Motion (RITMO) at University of Oslo under the co-supervision of Rolf Inge Rodøy and Alexander Refsum Jensenius, I conducted a short motion capture study exploring expert piano movements.

The Qualysis system was used to track the movements of expert-level piano players using markers on the head, shoulders, elbows, top of hands, and fingers at the primary bending joints. Fifteen 5 mm reflective tracking markers on the joints of the fingers (Figure 1), in addition to two 20 mm trackers for right and left top of hands, elbows, and shoulders (Figure 2). Four 20 mm trackers were used for the head and one tracker was used for the back of the neck (see Figure 3). The room-scale Qualysis tracking system contained 10 cameras, and we added two additional mo-cap cameras for close-up finger and hand tracking.

Figure 1. Top Left shows the two additional Qualysis cameras that were added to the 10-camera system for up-close motion capture, as well as the trackers on the head and back of the performer’s neck. The top right mage shows the trackers on the head, shoulders, elbows, and top of hands. The bottom picture shows the 5 mm trackers on the finger joints.

The goal for this data collection was to establish early-stage target parameters in the performer’s movements as identifiers of more developed motor skill. Moreover, I worked with the Qualysis software to learn how to label the trackers and import this labeled motion data frame-by-frame into Matlab. In Matlab, I used the Musical Gestures Toolbox to analyze movements and movement trajectories.

So far, this analysis has taught me what types of movement features I can extract with the Musical Gestures Toolbox and a video analysis. For instance, I can get an estimate of the overall smoothness of a movement and determine which part of the body leads the movement. So far in my analysis I can see that the shoulder and elbow joints seem to be leading most of the movements in expert piano performance, and that the movement from the fingers are quite fast and light in comparison to those of the shoulders, wrists, and elbows. This data will be compared to that of novice-level piano players to explore the key differences in playing technique vis a vis the movements used while playing a pre-selected musical piece that exhibits key challenges to piano playing developed in more expert-level players.

By Lynda Joy Gerry
Published Dec. 27, 2019 5:04 PM - Last modified Mar. 23, 2020 4:28 PM