When
Thematic Session 5: Mapping and Control (Tuesday, 15:30)
Abstract
Hanson Robotics (the developers of the Sophia robot) have attributed Sophia's popularity not due to her strong chatbot abilities but to her embodiment in a human-like form. Researchers at the Machine Musician Lab know that collaborating with a machine that improvises novel music can be engaging, but watching a human performer collaborate with a musical machine is much less engaging when it does not have a face. To solve this problem, the team presents an easy-to-use system (Faceplayer) that allows quick prototyping of sound-visual connections using videos of human dancers and musicians as stand-ins for robot bodies. While a computer-bound ML-enabled system actively listens and responds by playing a Disklavier, the Faceplayer provides a visual cue for the audience that provides the impression that the movement and the sound are linked. The Faceplayer image may then be projected onto any surface for many different effects. The team has devised several tests and use cases, and early results look promising. Later this Fall, the system will be utilized in a live musical Turing test at IUPUI. Attendees of this presentation will see video clips of the Faceplayer in action and will hear a detailed description of how they achieve these effects; lastly, the team will discuss their plans for developing the system.
Bio
Jason Palamara is a technologist and performer on acoustic and electronic instruments. As an Assistant Professor of Music Technology at IUPUI, he specializes in the development of machine learning-enabled performance technologies for music. He is the founder/director of IUPUI’s 30+ member DISEnsemble (Destructive/Inventive Systems Ensemble), which builds or hacks musical and non-musical stuff and plays live concerts. He regularly performs and composes music for modern dance as a solo artist and maintains a long-term creative partnership percussionist-composer Scott Deal, with whom he developed AVATAR. AVATAR is an autonomous music system uses machine learning to play along with live improvisation.