Norwegian version of this page

EPEC - Engineering Predictability with Embodied Cognition

How can multimodal systems sense, learn, and predict future events?

EPEC: engineering predictability with embodied cognition

Humans are superior to computers and robots when it comes to perceiving with eyes, ears and other senses as well as combining perception with learned knowledge to choose the best actions. This project aims to develop human-inspired models of behaviour and perception and to show that these models can predict future actions accurately.

Our inspiration comes from embodied cognition, a concept from psychology proposing that our bodies, perceptions, abilities, and form, influences how we think. Our goal is to exploit the form of various systems to develop predictive reasoning models as alternatives to traditional reactive systems. These models will be applied in interdisciplinary fields of music technology and robotics. In music, we aim to provide everyday people new ways to move within musical spaces. Our models learn about their interactions with smartphones to proactively assist with their future actions. In robotics, we are developing robots with dynamic forms that can change their thinking in response to new body shapes.

EPEC explores applications in musical interaction on smartphones and robotics
Musical interaction on smartphones and robotic systems, are EPEC's application areas for new predictive models.

EPEC is directed by Professor Jim Tørresen, who also leads the ROBIN research group in the Department of Informatics. The project employs two post doctoral fellows, Kai Olav Ellefsen and Charles Martin, and PhD researcher Tønnes Nygaard. The project also includes Associate Professor Kyrre Glette, PhD researcher Jørgen Nordmoen, and a number of masters students in machine learning, robotics and music technology.

Objectives

Design, implement and evaluate multimodal systems that are able to sense, learn and predict future events.

Sub-projects

  • Internal Models: Predicting real-world effects through internal simulations
  • DyRET: Dynamic Robot for Embodied Testing
  • Interactive music systems: Computer systems for extending and enhancing musical listening, performance and collaboration.

Master Projects

Researchers from the EPEC group supervise master projects in robotics, music technology, and machine learning. Come work with us on predictive models, embodied interactive systems and new robotic interactions! More information available here.

Funding

Supported by The Research Council of Norway under FRINATEK grant agreement 240862 from 2015 to 2019. The grant funds 1 PhD and 2 post-doc positions (10% of prop. funded).

Publications

  • Charles Patrick Martin & Henry Gardner (2019). Free-Improvised Rehearsal-as-Research for Musical HCI, In Marcelo M. Wanderley; Tom Mudd; Andrew McPherson; Simon Holland & Katie Wilkie-McKenna (ed.),  New Directions in Music and Human-Computer Interaction.  Springer.  ISBN 978-3-319-92068-9.  Chapter 17.  s 269 - 284
  • Torgrim Rudland Næss & Charles Patrick Martin (2019). A Physical Intelligent Instrument using Recurrent Neural Networks, In Marcelo Queiroz & Anna Xambo Sedo (ed.),  Proceedings of the International Conference on New Interfaces for Musical Expression.  Universidade Federal do Rio Grande do Sul.  Conference Paper.  s 79 - 82
  • Charles Patrick Martin & Jim Tørresen (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks, In Anna Xambo Sedo & Marcelo Queiroz (ed.),  Proceedings of the International Conference on New Interfaces for Musical Expression.  Universidade Federal do Rio Grande do Sul.  Conference Paper.  s 260 - 265
  • Andrei Faitas; Synne Engdahl Baumann; Torgrim Rudland Næss; Jim Tørresen & Charles Patrick Martin (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks, In Marcelo Queiroz & Anna Xambo Sedo (ed.),  Proceedings of the International Conference on New Interfaces for Musical Expression.  Universidade Federal do Rio Grande do Sul.  Conference Paper.
  • Jørgen Halvorsen Nordmoen; Tønnes Frostad Nygaard; Kai Olav Ellefsen & Kyrre Glette (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion, In Manuel López-Ibáñez (ed.),  GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference.  Association for Computing Machinery (ACM).  ISBN 978-1-4503-6111-8.  Kapittel.
  • Aline Weber; Charles Patrick Martin; Jim Tørresen & Bruno Castro da Silva (2019). Identifying Reusable Early-Life Options, In  Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics.  2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).  ISBN 0000000000000.  Conference Paper.
  • Bjørn Ivar Teigen; Kai Olav Ellefsen & Jim Tørresen (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods, In  Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics.  2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).  ISBN 0000000000000.  Kapittel.
  • Rafael Garcia; Alexandre Xavier Falcao; Alexandru C. Telea; Bruno Castro da Silva; Jim Tørresen & Joao Luiz Dihl Comba (2019). A Methodology for Neural Network Architectural Tuning Using Activation Occurrence Maps, In Chrisina Jayne & Zoltán Somogyvári (ed.),  2019 International Joint Conference on Neural Networks (IJCNN).  IEEE.  ISBN 978-1-7281-1985-4.  Artikkel.
  • Aline Weber; Lucas N. Alegre; Jim Tørresen & Bruno Castro da Silva (2019). Parameterized Melody Generation with Autoencoders and Temporally-Consistent Noise, In Federico Visi (ed.),  Music Proceedings of the International Conference on New Interfaces for Musical Expression.  Universidade Federal do Rio Grande do Sul.  Artikkel.  s 174 - 179
  • Charles Patrick Martin; Alexander Refsum Jensenius & Jim Tørresen (2018). Composing an ensemble standstill work for Myo and Bela, In Luke Dahl; Tom Martin & Doug Bowman (ed.),  Proceedings of the International Conference On New Interfaces For Musical Expression.  Virginia Tech.  KAPITTEL.  s 196 - 197
  • Tønnes Frostad Nygaard; Charles Patrick Martin; Eivind Samuelsen; Jim Tørresen & Kyrre Glette (2018). Real-world evolution adapts robot morphology and control to hardware limitations, In hernan aguirre (ed.),  GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference.  Association for Computing Machinery (ACM).  ISBN 978-1-4503-5618-3.  1.  s 125 - 132
  • Jørgen Halvorsen Nordmoen; Eivind Samuelsen; Kai Olav Ellefsen & Kyrre Glette (2018). Dynamic mutation in MAP-Elites for robotic repertoire generation, In  The 2018 Conference on Artificial Life.  MIT Press.  ISBN 9780262355766.  Konferanseartikkel.  s 598 - 605
  • Justinas Miseikis; Inka Brijacak; Saeed Yahyanejad; Kyrre Glette; Ole Jacob Elle & Jim Tørresen (2018). Transfer Learning for Unseen Robot Detection and Joint Estimation on a Multi-Objective Convolutional Neural Network, In  2018 IEEE International Conference on Intelligence and Safety for Robotics (ISR 2018).  IEEE.  ISBN 9781538655481.  1.  s 337 - 342
  • Justinas Miseikis; Inka Brijacak; Saeed Yahyanejad; Kyrre Glette; Ole Jacob Elle & Jim Tørresen (2018). Multi-Objective Convolutional Neural Networks for Robot Localisation and 3D Position Estimation in 2D Camera Images, In  2018 15th International Conference on Ubiquitous Robots (UR 2018).  IEEE.  ISBN 9781538663356.  1.  s 597 - 603
  • Justinas Miseikis; Patrick Knobelreiter; Inka Brijacak; Saeed Yahyanejad; Kyrre Glette; Ole Jacob Elle & Jim Tørresen (2018). Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks, In  2018 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2018).  IEEE.  ISBN 9781538618554.  1.  s 181 - 187
  • Charles Patrick Martin & Jim Tørresen (2017). MicroJam: An App for Sharing Tiny Touch-Screen Performances, In Cumhur Erkut (ed.),  Proceedings of the International Conference on New Interfaces for Musical Expression.  Aalborg University Copenhagen.  Chapter.  s 495 - 496
  • Charles Patrick Martin & Jim Tørresen (2017). Exploring Social Mobile Music with Tiny Touch-Screen Performances, In Vesa Välimäki; Jukka Pätynen & Tapio Lokki (ed.),  Proceedings of the 14th Sound and Music Computing Conference 2017.  Aalto University.  ISBN 978-952-60-3729-5.  Conference Paper.  s 175 - 180
  • Kai Olav Ellefsen & Jim Tørresen (2017). Evolving neural networks with multiple internal models, In David Parsons; Carole Knibbe; Guillaume Beslon; Dusan Misevic; Jonathan Rouzaud-Cornabas; Nicolas Bredèche; Salima Hassas; Olivier Simonin & Hedi Soula (ed.),  Proceedings of the European Conference on Artificial Life 2017.  MIT Press.  ISBN 978-0-262-34633-7.  Artikkel.  s 138 - 145
  • Torkil Aamodt & Jim Tørresen (2017). Comparing Neural Networks for Predicting Stock Markets, In Chrisina Jayne; Giacomo Boracchi; Lazaros Iliadis & Aristidis Likas (ed.),  Proceedings of 18th International Conference, EANN 2017, Athens, Greece, August 25–27, 2017.  Springer.  ISBN 978-3-319-651712.  Artikkel.  s 363 - 375
  • Charles Patrick Martin; Kai Olav Ellefsen & Jim Tørresen (2017). Deep Models for Ensemble Touch-Screen Improvisation, In Mathieu Barthet; Tony Stockman & George Fazekas (ed.),  Proceedings of the 12th International Audio Mostly Conference: Augmented and Participatory Sound and Music Experiences.  Association for Computing Machinery (ACM).  ISBN 978-1-4503-5373-1.  Conference paper.
  • Kristian Nymoen; Arjun Chandra & Jim Tørresen (2016). Self-awareness in Active Music Systems, In Xin Yao; Jim Tørresen; Bernhard Rinner; Peter R. Lewis & Marco Platzner (ed.),  Self-aware Computing Systems.  Springer.  ISBN 978-3-319-39674-3.  Kapittel 14.  s 279 - 296
  • Justinas Miseikis; Kyrre Glette; Ole Jakob Elle & Jim Tørresen (2016). Automatic Calibration of a Robot Manipulator and Multi 3D Camera System, In Yasuhisa Hirata (ed.),  Proc. of 2016 IEEE/SICE International Symposium on System Integration.  IEEE conference proceedings.  ISBN 978-1-5090-3329-4.  Artikkel.  s 735 - 741
  • Tønnes Frostad Nygaard; Jim Tørresen & Kyrre Glette (2016). Multi-objective Evolution of Fast and Stable Gaits on a Physical Quadruped Robotic Platform, In Yaochu Jin & Stefanos Kollias (ed.),  Proc. of 2016 IEEE Symposium Series on Computational Intelligence (SSCI).  IEEE conference proceedings.  ISBN 978-1-5090-4240-1.  Artikkel.
  • Jim Tørresen; Andreas Høyer Iversen & Ralf Greisiger (2016). Data from Past Patients used to Streamline Adjustment of Levels for Cochlear Implant for New Patients, In Stefanos Kollias & Yaochu Jin (ed.),  Proc. of 2016 IEEE Symposium Series on Computational Intelligence (SSCI).  IEEE conference proceedings.  ISBN 978-1-5090-4240-1.  Artikkel.
  • Andreas Færøvig Olsen & Jim Tørresen (2016). Smartphone Accelerometer Data used for Detecting Human Emotions, In Lipo Wang & Xiang Fei (ed.),  Proceedings of 2016 3rd International Conference on Systems and Informatics.  IEEE conference proceedings.  ISBN 978-1-5090-5520-3.  Artikkel.  s 410 - 415
  • Justinas Miseikis; Kyrre Glette; Ole Jakob Elle & Jim Tørresen (2016). Multi 3D Camera Mapping for Predictive and Reflexive Robot Manipulator Trajectory Estimation, In Yaochu Jin & Stefanos Kollias (ed.),  Proc. of 2016 IEEE Symposium Series on Computational Intelligence (SSCI).  IEEE conference proceedings.  ISBN 978-1-5090-4240-1.  1.

View all works in Cristin

  • Jim Tørresen (2019). Future and Ethical Perspectives of Robotics and AI.
  • Kai Olav Ellefsen & Jim Tørresen (2019). Evolutionary Robotics: Automatic design of robot bodies and control.
  • Jim Tørresen (2019). Hva er kunstig intelligens?.
  • Jun Miura & Jim Tørresen (2019). Intelligent Robot Technologies for Care and Lifestyle Support.
  • Joao Luiz Dihl Comba & Jim Tørresen (2019). Visual Data Analysis of Unstructured and Big Data.
  • Katharina J. Rohlfing & Jim Tørresen (2019). Explainability: an interactive view.
  • Kyrre Glette (2019). Kunstig intelligens for tilpasningsdyktige roboter.
  • Jim Tørresen (2019). Kunstig intelligens – hvem, hva og hvor. (Eng. Artificial Intelligence – who, what and where).
  • Jim Tørresen (2019). Kunstig intelligens – hvem, hva og hvor.
  • Kyrre Glette (2019). Kunstig intelligens for tilpasningsdyktige roboter.
  • Charles Patrick Martin; Torgrim Rudland Næss; Andrei Faitas & Synne Engdahl Baumann (2019). Session on Musical Prediction and Generation with Deep Learning.
  • Jim Tørresen (2019). Artificial Intelligence and Applications in Health and Care.
  • Bjørn Ivar Teigen; Kai Olav Ellefsen & Jim Tørresen (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods..
  • Tønnes Frostad Nygaard; Charles Patrick Martin; Jim Tørresen & Kyrre Glette (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing.
  • Tønnes Frostad Nygaard; Jørgen Halvorsen Nordmoen; Charles Patrick Martin; Jim Tørresen & Kyrre Glette (2019). Lessons Learned from Real-World Experiments with DyRET: the Dynamic Robot for Embodied Testing.
  • Torgrim Rudland Næss; Jim Tørresen & Charles Patrick Martin (2019). A Physical Intelligent Instrument using Recurrent Neural Networks.
  • Justinas Miseikis; Inka Brijacak; Saeed Yahyanejad; Kyrre Glette; Ole Jacob Elle & Jim Tørresen (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN.
  • Jim Tørresen (2019). Making Robots Adaptive and Preferable to Humans.
  • Kai Olav Ellefsen; Joost Huizinga & Jim Tørresen (2019). Guiding Neuroevolution with Structural Objectives.
  • Kai Olav Ellefsen & Jim Tørresen (2019). Self-Adapting Goals Allow Transfer of Predictive Models to New Tasks.
  • Jørgen Halvorsen Nordmoen; Tønnes Frostad Nygaard; Kai Olav Ellefsen & Kyrre Glette (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion.
  • Tønnes Frostad Nygaard; Jørgen Halvorsen Nordmoen; Kai Olav Ellefsen; Charles Patrick Martin; Jim Tørresen & Kyrre Glette (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing.
  • Charles Patrick Martin & Jim Tørresen (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks.
  • Andrei Faitas; Synne Engdahl Baumann; Jim Torresen & Charles Patrick Martin (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks.
  • Charles Patrick Martin & Jim Torresen (2019). An Interactive Music Prediction System with Mixture Density Recurrent Neural Networks.
  • Jim Tørresen (2019). Intelligent and Adaptive Robots in Real-World Environment.
  • Jim Tørresen (2019). Design and Control of Robots for Real-World Environment.
  • Jim Tørresen (2019). Sensing Human State with Application in Older People Care and Mental Health Treatment.
  • Jim Tørresen (2019). Supporting Older People with Robots for Independent Living.
  • Tønnes Frostad Nygaard; Vegard Dønnem Søyseth; Jørgen Halvorsen Nordmoen & Kyrre Glette (2018). Stand with the DyRET robot.
  • Charles Patrick Martin (2018). Deep Predictive Models in Interactive Music.
  • Vegard Dønnem Søyseth; Tønnes Frostad Nygaard; Charles Patrick Martin; Md Zia Uddin & Kai Olav Ellefsen (2018). ROBIN-Stand ved Cutting Edge 2018.
  • Kai Olav Ellefsen (2018). Evolusjonær Robotikk: Automatisk design og kontroll av roboter.
  • Kai Olav Ellefsen & Jim Tørresen (2018). Evolutionary Robotics: Automatic design of robot controllers and bodies.
  • Jim Tørresen (2018). Artificial Intelligence Applied for Real-World Systems.
  • Adrian Stoica & Jim Tørresen (2018). Robots on the Moon, and their Role in a Future Lunar Economy.
  • Jim Tørresen (2018). Artificial Intelligence – State-of-the-art.
  • Jim Tørresen (2018). Roboter kommer nærmere – skal vi glede eller grue oss?.
  • Jim Tørresen (2018). Kunstig Intelligens – Lærende og tilpasningsdyktig teknologi.
  • Charles Patrick Martin (2018). Creative Prediction with Neural Networks.
  • Jim Tørresen (2018). Remote Lab and Applications for High Performance and Embedded Architectures.
  • Charles Patrick Martin; Alexander Refsum Jensenius & Jim Tørresen (2018). Composing an ensemble standstill work for Myo and Bela.
  • Charles Patrick Martin; Kyrre Glette; Tønnes Frostad Nygaard & Jim Tørresen (2018). Self-Awareness in a Cyber-Physical Predictive Musical Interface.
  • Charles Patrick Martin & Jim Tørresen (2018). Predictive Musical Interaction with MDRNNs.
  • Charles Patrick Martin (2018). Predictive Music Systems for Interactive Performance.
  • Charles Patrick Martin; Kyrre Glette & Jim Tørresen (2018). Creative Prediction with Neural Networks.
  • Enrique Alejandro Garcia Ceja; Kai Olav Ellefsen; Charles Patrick Martin & Jim Tørresen (2018). Prediction, Interaction, and User Behaviour.
  • Tønnes Frostad Nygaard (2018). Real-World Evolution Adapts Robot Morphology and Control to Hardware Limitations.
  • Kyrre Glette (2018). Automatic design of bodies and behaviors for real-world robots.
  • Tønnes Frostad Nygaard; Charles Patrick Martin; Jim Tørresen & Kyrre Glette (2018). Exploring Mechanically Self-Reconfiguring Robots for Autonomous Design.
  • Jim Tørresen; Enrique Alejandro Garcia Ceja; Kai Olav Ellefsen & Charles Patrick Martin (2018). Equipping Systems with Forecasting Capabilities.
  • Jim Tørresen (2018). Intelligent Systems for Medical and Healthcare Applications.
  • Jim Tørresen (2018). Ethical Robots and Autonomous Systems.
  • Kyrre Glette (2018). Automatic design of shapes and behaviors for robots.
  • Kyrre Glette (2018). Robotics, Intelligent Systems and Evolutionary Computation.
  • Jim Tørresen (2018). UiO Visit to UFRJ – An overview of research.
  • Kai Olav Ellefsen (2017). Evolutionary Robotics: Automatic design of robot bodies and control.
  • Kai Olav Ellefsen (2017). Automating Robot Design with Evolutionary Algorithms.
  • Jim Tørresen (2017). Artificial Intelligence by Nature Inspired Computing.
  • Jim Tørresen (2017). Unleashing Artificial Intelligence in the Real World.
  • Jim Tørresen (2017). Sensing and Reasoning Technology Applied within Mental Health Treatment and Elderly Care Robot Companions.
  • Jim Tørresen (2017). Robotics activity at Robotics and Intelligent Systems Group University of Oslo.
  • Jim Tørresen (2017). Making Systems Intelligent by Biologically Inspired Computing.
  • Jim Tørresen (2017). Opportunities and Challenges with Artificial Intelligence.
  • Jim Tørresen (2017). Making Technology Adaptive by Biologically Inspired Computing.
  • Tønnes Frostad Nygaard (2017). Presentation on Robotics: Today and the future.
  • Jim Tørresen (2017). Brazilian Student Exchange Opportunities to University of Oslo.
  • Charles Patrick Martin (2017). MicroJam: A Social App for Making Music.
  • Jim Tørresen (2017). Artificial Intelligence – What is it and what can it be applied to?.
  • Charles Patrick Martin (2017). Making Social Music with MicroJam.
  • Charles Patrick Martin (2017). Virtuosic Interactions / Performing with a Neural iPad Band.
  • Jim Tørresen (2016). Artificial intelligence in autonomous systems.
  • Jim Tørresen (2016). Teknologi som kan tilpasse seg gjennom læring.
  • Kyrre Glette (2016). Self-teaching robots.

View all works in Cristin

Tags: machine learning, robotics, interactive music
Published May 24, 2016 3:38 PM - Last modified Nov. 13, 2018 6:10 AM