Embodied Perspectives on Musical AI (EmAI)

In this two-day workshop consisting of keynote speeches, performances, and thematic sessions, we explore musical artificial intelligence's past, present, and future through the lenses of embodied cognition.

Image may contain: Sleeve, Gesture, Performing arts, Font, Entertainment.

The image is generated by OpenAI's DALL·E 2 from the text prompt, "Embodied Perspectives on Musical AI."

The workshop is free and open for anyone interested in the topic.

Sign up for the workshop

About

Embodiment, or, more concretely, musical embodiment, denotes how the body shapes our musical experiences. For example, you may exert more or less effort depending on the uncertainty of some musical situations or while playing technically challenging tasks. Such varying levels of effort during a live performance can lead to particular affective states resulting in bodily arousals. Or, you can also use your body functionally, such as full-body swaying to facilitate keeping the groove or nodding your head to signal your bandmate to return to the tune’s main melody. From an enactive perspective, our perception is shaped by our actions. As such, cognition emerges not just through information processing but mainly from the dynamic interaction between the agent and the environment. All in all, we experience the music with our body, using more modalities than hearing, regardless if we perform or listen to it. 

Most musical artificial intelligence (AI) and multi-agent systems (MAS) focus on the music information found in the auditory domain. Modeling instrumental acoustics, synthesizing raw audio, or generating symbolic music data are highly complex tasks that AI can already accomplish to some extent. Still, it is unclear how these technologies can collaborate with musicking humans. How will machines perceive humans as diverse embodied entities, and how will humans communicate with machines exploiting multiple modalities? How can embodiment theories contribute to creating intelligent musical agents? At large, how will we make music with AI in the future? The interaction and artistic contexts, perceptual-motor constraints, affective states, environmental features, sensing devices, and computational systems are just a few that can come into play in an attempt to answer such questions.

An interdisciplinary research model encompassing natural sciences, humanities, cognition, and performing arts is often necessary to understand conventional forms of musical collaboration and create novel music technologies. Thus, in this two-day workshop, we will convene a group of scholars, artists, and engineers from diverse disciplines to emphasize embodied perspectives on musical human-AI interactions. Together we will explore theories and practices in this intriguing domain and discuss musical AI's past, present, and future through the lenses of embodied cognition.

 

Submissions

The two-day workshop will consist of thematic sessions in addition to two keynote lectures and one evening performance. We invite submissions for the thematic sessions. The aim is to encourage dialogue so that presenters will give short 10-minute presentations followed by a panel discussion. The proposals can include demos, completed projects, works-in-progress, and provocations. We solicit anything of relevance to the topic of Embodied Perspectives on Musical AI. Short proposal texts (200-300 words) can be submitted at: https://nettskjema.no/a/emai.

Important dates

  • Abstract submission deadline: 1 September 2022 (AoE, UTC-12)
  • Workshop notifications sent to authors: 21 September 2022
  • Registration open: 28 September 2022
  • Announcement of the final program: 21 October 2022
  • Workshop: 21-22 November 2022

Where

This workshop will be in a hybrid format, meaning you can choose to join us via Zoom or in person. The first day of the workshop will take place in Salen in the ZEB building, while the second day will be in Forsamlingssalen in Harald Schjelderups Hus.

Live Stream

Day 1

Day 2

Performances

Workshop program

All times are in Central European Time (CET), UTC +1 

Monday 21 November 2022 @Salen

09:15

Pre-workshop hands-on:

Pre-registration is required

Bilge Serdar Göksülük University of Oslo
11:30 Coffee and registration
12:00 Welcome Alexander R. Jensenius University of Oslo
12:10 Introducing the program Çağrı Erdem University of Oslo
12:15 Keynote Embodied Interaction in Dance: methods, creations, and critical reflections (Sarah Fdili Alaoui) LISN-Université Paris Saclay & Ex)Situ INRIA
14:00 Break 
14:15

Thematic Session 1: Design and Interaction

Chair: Carsten Griwodz

Memristor based Spiking Neural Network Interfaces (SNNIs) for Embodied Musical AI (Davin Browner)

Holistic approaches to musical AI interfaces (Chris Kiefer)

The iOSCahedron - developing a hybrid spatialization instrument (Florian Goeschke)

AI for traditional instrument making (Sebastian Gonzalez)

Embodied Learning in Music Theory through Augmented Reality (Christopher Tralie)

Intuition as a Parameter of Interaction Experience (Greg Corness)

Panel discussion

15:25 Break & Demos
15:40

Thematic Session 2: Software and Synthesis

Chair: Dan Overholt

Identity and Inspiration: perspectives on automatic generation of dance (Andre Holzapfel)

Embodiment of audio evolution (Björn Thór Jónsson)

Neural synthesis and embodied practice on Musical AI (Darragh Kelly)

Performing Voices Without Bodies (Jonathan Chaim Reus)

The Role of the Vocal Persona in Natural and Synthesized Speech (Camille Noufi)

Conditioning Neural Network for Embodied Nonlinear Interaction (Riccardo Simionato)

Panel discussion

17:00 Break & Demos
17:30 Bubbles and tapas
Tuesday 22 November 2022 @Forsamlingssalen
09:15 Coffee and registration
09:40

Thematic Session 3: Modeling and Analysis

Chair: Kyrre Glette 

Embodied AI in North Indian Classical Music (Tejaswinee Kelkar)

Embodied Music Emotion Representation Learning with Deep Multimodal Networks (Oluremi Falowo)

Embodied Beat Tracking with a Virtual Quadruped (Alex Szorkovzsky)

Enacting a Musical Habitus: here deviance is possible (Shane Byrne)

Transgressing the System: AI and corporeal knowledge (Marco Donnarumma)

Robotic drummer with embodied cognition (Mojtaba Karbasi)

Panel discussion

10:50 Break & Demos
11:05

Thematic Session 4: Creativity and Expressivity

Chair: Nanette Nielsen

Anomaly detection as a means of sensing subtlety and nuance in musical gesture: toolchain and preliminary results (Teresa Pelinski)

Instruments that push back: Manipulating feedback networks with embodied musical AI (Charles Martin)

Drawing Stick-figures with Music: Reflections on using Lerdahl's Cognitive Constraints for Composition (Kenrick Ho)

Improvising with an Algorithm: A Performer's Perspective knowledge (Mike McCormick)

Working with AI to create the world of The Hum (Bálint Laczkó)

Investigating the Lived Experience of Statistical AI with Microphenomenology and Notochord (Victor Shepardson)

Panel discussion

12:15 Break & Demos
12:30 Lunch 
13:15 Keynote Keynote: Aesthetics of Machine Learning Art and Music in Embodied Agents (Sofian Audry) University of Quebec in Montreal
15:00 Break & RITMO tour!
15:30

Thematic Session 5: Mapping and Control

Chair: Tejaswinee Kelkar

Artificial Agents and Moving Bodies (Federico Visi & Nicola Leonard Hein)

Interactive Sonification of Expressive Gesture: the DanzArTe - Emotion Wellbeing Technology Project (Andrea Cera)

ImprovAIze: A new bridging project for Embodied Perspectives on Musical AI (Dan Overholt)

Faceplayer: A modular embodiment system for ML-enabled performance technologies (Jason Palamara)

Using Biosensors to Amplify Music Embodiment (Jason Snell)

Continuous gestural interactions in live coding (Georgios Diapoulis)

Panel discussion

16:45 Wrapping-up
17:00 Break & Demos
Tuesday 22 November 2022 (cont'd) @Realfagsbiblioteket
19:00 Performance The [Optic]Fiber Ensemble University of Oslo
19:20 Break
19:30 Keynote Performace Communal; AI as a Material, Instrument and the Other (Koray Tahiroğlu) Aalto University
20:30

Panel discussion: How can we make music with AI in the future?

Moderator: Stefano Fasciani

Panelists: Benedikte Wallace, Alexander JenseniusSarah Fdili Alaoui, Koray Tahiroglu, Sofian Audry

21:15 Closing

Workshop convenors

Çağrı Erdem, Riccardo Simionato, Sayed Mojtaba Karbasi, Alexander Refsum Jensenius

Tech crew: Aleksander Tidemann, Pedro Pablo Lucas Bravo, Maham RiazBjörn Thor Jónsson

Administration: Eirik Slinning Karlsen, Marit Johanne Furunes, Julie Øderud Danielsen

Tags: artificial intelligence, embodied cognition, Music Technology
Published July 20, 2022 10:24 PM - Last modified Feb. 1, 2023 12:09 AM