Embodied Intelligence in Human Performance

Expert music performance combines elaborate motor sequences with underlying cognitive states, such as fluctuating attention, arousal, and cognitive load. We investigated how similar physiological signals thought to be associated with cognitive states are to one another when playing the same musical phrases within / across concerts, hypothesizing that similar internal state trajectories evoked by the same phrases may lead to similar physiological dynamics. Four professional players performed the same program at two concerts, with differing audience presence. Respiration, pupil diameter, and eye movements were recorded with nasal temperature sensors and Pupil Labs Neon eye trackers, then aligned to musical time using audio-based dynamic time warping. For each performer and phrase, we compared correlations between repeated sections within a concert with those between corresponding sections across concerts. Results revealed that while eye-related metrics are consistently high both within- and across-concert, respiration dynamics are only high within-concert across players. These findings suggest that eye-related dynamics are relatively constrained by score-specific structure, whereas respiration may reflect a more flexible, context-sensitive physiological state that can follow different trajectories across performances.

When and how do bodies work together in concert? Some music requires unison action from performers while others involve cooperative but independent contributions. Some genres and events invite active audience participation, others expect discreet attention. With a new analysis strategy called subset coherence, this talk compares the prominence of respiratory, cardiac, and body motion synchrony across genres of music and categories of participants. Besides structural differences in genres and pieces, the evaluated ratios of subset coherence distinguish audience familiarity and musical roles across performing ensembles. The opportunities to physically "come together" in music are contingent on what that music is and how we can take part.

Successful badminton returns require players to extract predictive visual information and prepare a response within a few hundred milliseconds. Most previous research has used video-based prediction tasks, leaving unclear how gaze and movement are coordinated during live play. We recorded gaze and waist acceleration in 17 male university badminton players as they returned shots from an opponent. Skilled players looked at the opponent’s upper body more frequently and maintained their gaze there longer, shifting to the contact point later. They also made fewer return errors, although the timing of movement initiation was similar between groups. These findings suggest that skilled performance depends less on responding earlier and more on maintaining opponent-related visual information during movement preparation to support accurate action selection.

Pupil diameter is widely used as an index of arousal and brain state, yet it remains unclear whether slow pupil-linked state fluctuations systematically modulate the effective capacity of temporal attention. Here, we tested this question using an auditory attentional blink paradigm, in which participants were required to detect the first (T1) and second (T2) targets. In Experiment 1, trial-by-trial analyses revealed that successful T2 detection, given correct T1 detection (T2|T1), was associated with smaller baseline pupil size. Furthermore, analyses focusing on slow pupil fluctuations (< 0.2 Hz) revealed that the T2|T1 detection accuracy increased during the pupil dilation phase occurring 0–2 seconds after pupil constriction. In Experiment 2, we used real-time pupillometry to trigger stimulus presentation during predefined phases of ongoing slow pupil dynamics. This closed-loop manipulation produced reliable phase-dependent differences in T2|T1 detection accuracy. Critically, the effect of pupil phase remained significant in a linear mixed-effects model that included baseline pupil size as a covariate, indicating that it cannot be explained by baseline pupil size alone. Together, these findings demonstrate that temporal attentional capacity is shaped not only by arousal level but also by the phase of slow pupil-linked brain-state fluctuations. Our results suggest that the attentional blink reflects a dynamically regulated, state-dependent limitation rather than a fixed processing bottleneck.

Flamenco offers a rich setting in which to study improvisation and flow, where dancer and guitarist co-improvise inside a shared form, with a predominantly leading dancer using the body as a percussion instrument to produce rhythm and mark the tempo of the performance. This work examines how improvisatory freedom relates to flow in professional dancer–guitarist dyads, first through post-performance ratings and then through a whole-body kinematic model. Pose trajectories extracted from 2D video are used to estimate kinetic energy, permutation entropy of that energy series, and the variability of permutation entropy. This talk discusses how movement kinematics and temporal organisation characterize flamenco palos, and challenges the assumption that greater improvisatory freedom, or more intense flow, is marked simply by more unpredictability. In joint improvisation, stability and instability are both necessary: local predictability can coexist with longer-timescale fluctuation.

Flow is a state of intense engagement whose physiological signatures remain poorly understood, particularly during social interaction. We investigated whether pupil dynamics track individual and shared flow using a custom visuomotor game. Participants played individually (Solo) or cooperatively with a partner (Team) under Easy, Adaptive, and Overload conditions. In the Adaptive condition, game difficulty was continuously adjusted to participants’ performance to maintain an optimal challenge level and facilitate flow. Pupil diameter was recorded while task-irrelevant background luminance changes were introduced periodically to evoke pupil light responses (PLRs). Subjective ratings confirmed that the Adaptive condition elicited the strongest flow experience. PLRs were selectively attenuated during Adaptive play after controlling for baseline pupil size. During Team play, interpersonal pupil synchronization increased beyond synchronization attributable to common luminance input and tracked subjective flow and team experience. These findings reveal complementary pupillary signatures of individual and shared flow.

Cognitive facets of trait empathy may facilitate interpersonal synchrony, but the contributions of affective trait empathy and state empathy remain underexplored. This study investigated how both dispositional and experimentally induced empathy influence rhythmic entrainment and interpersonal synchrony. Thirty-three dyads completed two synchrony tasks (tapping with a metronome, and playing a melody with a partner) and underwent an empathy induction, in which participants narrated personal events while partners either empathised or memorised factual details. Trait empathy was measured with the Interpersonal Reactivity Index. Induced empathy showed no main effects, but both affective and cognitive trait empathy contributed to synchrony: higher Empathic Concern predicted more consistent tapping in Task 1, and higher Fantasy predicted greater synchrony in Task 2. Trait Fantasy also interacted with the empathy manipulation in predicting listeners’ mirroring of narrators' head movements. These findings suggest that successful coordination draws on both cognitive and emotional aspects of empathy.

There are many ways for people to participate in music-making: as performers, listeners, dancers, composers, and so on. Increasingly, machines can also participate as interfaces for sound production, digital scores providing a visual source of ideas, or interactive musical partners, among other possibilities. This study investigated musical interaction between people and a “dancing” AI-driven robot arm called RAMI. RAMI was designed as a digital score for improvisation but also integrates some aspects of a musical partner through its interactivity and embodied, movement-based cues. Our aim was to investigate how people integrate RAMI into their music-making. Specifically, we questioned whether people would relate to RAMI as a musical partner and what effect this would have on their musical engagement. We carried out two mixed-methods experiments to address this question. In Experiment 1, we manipulated whether RAMI used its interactive AI or, instead, ran a non-interactive script. Musicians improvised with RAMI while we tracked their body motion and electrodermal activity as indices of physiological engagement; afterwards, they provided ratings of their experience. In Experiment 2, the musicians improvised with RAMI, then watched playback of their recorded improvisations for stimulated recall and provided continuous ratings of their sense of flow. In both experiments, musicians also completed semi-structured interviews. Our findings showed that musicians were engaged by RAMI’s interactive AI, as evidenced by their ratings and EDA responses. They were sensitive to (mis)matches between RAMI’s motion energy and their own sound energy, and a positive correlation between motion and sound energy was linked to stronger flow ratings in Exp. 2. A thematic analysis of the interview data builds on these findings to illustrate how musicians attempted to relate to RAMI as a musical partner, then renegotiated a new role for it when it did not meet their expectations of how a partner should behave.