When
Thematic Session 5: Mapping and Control (Tuesday, 15:30)
Abstract
This artistic research project by Federico Visi, Nicola L. Hein, Simon Rose and Ingo Reulecke is focussed on the improvised interaction of human musicians, dancers and artificial musical agents. The research consists of three steps:
1. Nicola L. Hein (guitar, electronics) / Federico Visi (sensor armbands, electronics) / Simon Rose (baritone saxophone) / Ingo Reulecke (dance) formed a group concerned with the interaction of artificial and human musical and movement agents. Hein-Visi-Rose-Reulecke aimed to perform together with two machine learning based artificial agents trained on audio recordings of Hein and Rose, performatively investigating the emerging relationships in the human-machine system.
2. Visi and Hein chose two different methods of engaging with the artificial agents, one based on gestural interaction and reinforcement learning, deploying electromyographic (EMG) sensing of muscular activity, the other based on MFCC and the k-NN analysis of an audio corpus. The quartet focussed on the exploration of the resulting space and relationships between human and artificial agents.
3. The modes of interaction were explored in a rehearsal and the public performance took place at the international conference “Improvisation, Ecology and Digital Technology” in the NRW-Forum Düsseldorf, Germany (24.06.2022).
In the interaction between artificial and human agents, the embodied nature of human interaction was foregrounded by the different modes of exchange between the human and the artificial agents, as well as the interaction of music and dance.
For EmAI we will reflect on the modes of interaction and improvisation that emerged between human performers and artificial agents and how these provided a framework for the performance. They result from the exploration of the different modes of co-agency between artificial and human agents. The modalities of exchange that characterise the agents as well as modalities of performance adopted by the human performers all played a role in how the improvisation unfolded.
Bio
Federico Visi (he/they) is a researcher, composer and performer based in Berlin, Germany. He carried out his doctoral research on instrumental music and body movement at the Interdisciplinary Centre for Computer Music Research (ICCMR), University of Plymouth, UK. He currently teaches and carries out research on embodiment in network performance at Universität der Künste Berlin and at Luleå University of Technology, where is part of the “GEMM))) Gesture Embodiment and Machines in Music” research cluster.
Nicola L. Hein (he/him) is a sound artist, guitarist, creative technologist, composer and researcher in music and aesthetics. He is a professor of digital creation at the University of Music Lübeck, Germany. In his artistic work he uses physical and electronic extension of the electric guitar, sound installations, cybernetic human-machine interaction with A.I. interactive music systems, Augmented Reality, telematic real-time art, ambisonic sound projection, instrument building, conceptual compositions.