Norwegian version of this page

EPEC - Engineering Predictability with Embodied Cognition (completed)

How can multimodal systems sense, learn, and predict future events?

EPEC.  Engineering predictability with embodied cognition. It says with letters.. Logo.

Humans are superior to computers and robots when it comes to perceiving with eyes, ears and other senses as well as combining perception with learned knowledge to choose the best actions. This project aims to develop human-inspired models of behaviour and perception and to show that these models can predict future actions accurately.

Our inspiration comes from embodied cognition, a concept from psychology proposing that our bodies, perceptions, abilities, and form, influences how we think. Our goal is to exploit the form of various systems to develop predictive reasoning models as alternatives to traditional reactive systems. These models will be applied in interdisciplinary fields of music technology and robotics. In music, we aim to provide everyday people new ways to move within musical spaces. Our models learn about their interactions with smartphones to proactively assist with their future actions. In robotics, we are developing robots with dynamic forms that can change their thinking in response to new body shapes.

A hand holding a smartphone and feet on some robotics. Photo.
Musical interaction on smartphones and robotic systems, are EPEC's application areas for new predictive models.

EPEC is directed by Professor Jim Tørresen, who also leads the ROBIN research group in the Department of Informatics. The project employs two post doctoral fellows, Kai Olav Ellefsen and Charles Martin, and PhD researcher Tønnes Nygaard. The project also includes Associate Professor Kyrre Glette, PhD researcher Jørgen Nordmoen, and a number of masters students in machine learning, robotics and music technology.

Objectives

Design, implement and evaluate multimodal systems that are able to sense, learn and predict future events.

Sub-projects

  • Internal Models: Predicting real-world effects through internal simulations
  • DyRET: Dynamic Robot for Embodied Testing
  • Interactive music systems: Computer systems for extending and enhancing musical listening, performance and collaboration.

Master Projects

Researchers from the EPEC group supervise master projects in robotics, music technology, and machine learning. Come work with us on predictive models, embodied interactive systems and new robotic interactions!

Funding

Supported by The Research Council of Norway under FRINATEK grant agreement 240862 from 2015 to 2019. The grant funds 1 PhD and 2 post-doc positions (10% of prop. funded).

Publications

  • Nygaard, Tønnes; Martin, Charles Patrick; Tørresen, Jim; Glette, Kyrre & Howard, David (2021). Real-world embodied AI through a morphologically adaptive quadruped robot. Nature Machine Intelligence. doi: 10.1038/s42256-021-00320-3. Full text in Research Archive
  • Nygaard, Tønnes; Martin, Charles Patrick; Howard, David; Tørresen, Jim & Glette, Kyrre (2021). Environmental Adaptation of Robot Morphology and Control Through Real-world Evolution. Evolutionary Computation. ISSN 1063-6560. doi: 10.1162/evco_a_00291. Full text in Research Archive
  • Martin, Charles Patrick; Glette, Kyrre; Nygaard, Tønnes & Tørresen, Jim (2020). Understanding Musical Predictions With an Embodied Interface for Musical Machine Learning. Frontiers in Artificial Intelligence. ISSN 2624-8212. 3(6). doi: 10.3389/frai.2020.00006.
  • Nygaard, Tønnes; Howard, David & Glette, Kyrre (2020). Real world morphological evolution is feasible. In Coello Coello, Carlos A. (Eds.), GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery (ACM). ISSN 978-1-4503-7127-8. p. 1392–1394. doi: 10.1145/3377929.3398095.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing. IEEE International Conference on Robotics and Automation (ICRA). ISSN 1050-4729. 2019-May, p. 9446–9452. doi: 10.1109/ICRA.2019.8793663. Full text in Research Archive
  • Martin, Charles Patrick & Gardner, Henry (2019). Free-Improvised Rehearsal-as-Research for Musical HCI. In Holland, Simon; Mudd, Tom; Wilkie-McKenna, Katie; McPherson, Andrew & Wanderley, Marcelo M. (Ed.), New Directions in Music and Human-Computer Interaction. Springer. ISSN 978-3-319-92068-9. p. 269–284. doi: https%3A/doi.org/10.1007/978-3-319-92069-6_17. Full text in Research Archive
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-adapting Goals Allow Transfer of Predictive Models to New Tasks, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 978-3-030-35664-4. p. 28–39. doi: https%3A/doi.org/10.1007/978-3-030-35664-4_3.
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 978-3-030-35664-4. p. 58–68. doi: https%3A/doi.org/10.1007/978-3-030-35664-4_6.
  • Becker, Artur; Herrebrøden, Henrik; Gonzalez Sanchez, Victor Evaristo; Nymoen, Kristian; Dal Sasso Freitas, Carla Maria & Tørresen, Jim [Show all 7 contributors for this article] (2019). Functional Data Analysis of Rowing Technique Using Motion Capture Data. In Coleman, Grisha (Eds.), Proceedings of the 6th International Conference on Movement and Computing. ACM Publications. ISSN 978-1-4503-7654-9. doi: 10.1145/3347122.3347135.
  • Weber, Aline; Alegre, Lucas N.; Tørresen, Jim & Castro da Silva, Bruno (2019). Parameterized Melody Generation with Autoencoders and Temporally-Consistent Noise. In Visi, Federico (Eds.), Music Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 174–179. Full text in Research Archive
  • Garcia, Rafael; Falcao, Alexandre Xavier; Telea, Alexandru C.; Castro da Silva, Bruno; Tørresen, Jim & Comba, Joao Luiz Dihl (2019). A Methodology for Neural Network Architectural Tuning Using Activation Occurrence Maps. In Jayne, Chrisina & Somogyvári, Zoltán (Ed.), 2019 International Joint Conference on Neural Networks (IJCNN) . IEEE. ISSN 978-1-7281-1985-4. doi: 10.1109/IJCNN.2019.8852223.
  • Martin, Charles Patrick & Tørresen, Jim (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 260–265. Full text in Research Archive
  • Faitas, Andrei; Baumann, Synne Engdahl; Næss, Torgrim Rudland; Tørresen, Jim & Martin, Charles Patrick (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. Full text in Research Archive
  • Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion. In López-Ibáñez, Manuel (Eds.), GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 978-1-4503-6111-8. p. 133–141. doi: 10.1145/3321707.3321762. Full text in Research Archive
  • Næss, Torgrim Rudland & Martin, Charles Patrick (2019). A Physical Intelligent Instrument using Recurrent Neural Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 79–82. Full text in Research Archive
  • Martin, Charles Patrick & Tørresen, Jim (2019). Data Driven Analysis of Tiny Touchscreen Performance with MicroJam. Computer Music Journal. ISSN 0148-9267. 43(4). Full text in Research Archive
  • Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE. ISSN 978-1-5386-8129-9. p. 189–194. doi: 10.1109/DEVLRN.2019.8850685.
  • Weber, Aline; Martin, Charles Patrick; Tørresen, Jim & da Silva, Bruno Castro (2019). Identifying Reusable Early-Life Options, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE. ISSN 978-1-5386-8129-9. doi: 10.1109/DEVLRN.2019.8850725.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Evolving Robots on Easy Mode: Towards a Variable Complexity Controller for Quadrupeds. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11454 LNCS, p. 616–632. doi: 10.1007/978-3-030-16692-2_41. Full text in Research Archive
  • Wallace, Benedikte & Martin, Charles Patrick (2019). Comparing models for harmony prediction in an interactive audio looper. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11453 LNCS, p. 173–187. doi: 10.1007/978-3-030-16667-0_12.
  • Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives. Evolutionary Computation. ISSN 1063-6560. 28(1), p. 115–140. doi: 10.1162/evco_a_00250. Full text in Research Archive
  • Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN. IEEE International Conference on Robotics and Automation (ICRA). ISSN 1050-4729. 2019-May, p. 8883–8889. doi: 10.1109/ICRA.2019.8794077. Full text in Research Archive
  • Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav & Glette, Kyrre (2018). Combining MAP-Elites and Incremental Evolution to Generate Gaits for a Mammalian Quadruped Robot. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10784 LNCS, p. 719–733. doi: 10.1007/978-3-319-77538-8_48. Full text in Research Archive
  • Garcia, Rafael; Telea, Alexandru C; Castro da Silva, Bruno; Tørresen, Jim & Dihl Comba, Joao Luiz (2018). A task-and-technique centered survey on visual analytics for deep learning model engineering. Computers & graphics. ISSN 0097-8493. 77, p. 30–49. doi: 10.1016/j.cag.2018.09.018.
  • Martin, Charles Patrick & Tørresen, Jim (2018). RoboJam: A musical mixture density network for collaborative touchscreen interaction. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10783 LNCS, p. 161–176. doi: 10.1007/978-3-319-77583-8_11. Full text in Research Archive
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Samuelsen, Eivind; Tørresen, Jim & Glette, Kyrre (2018). Real-world evolution adapts robot morphology and control to hardware limitations. In aguirre, hernan (Eds.), GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 978-1-4503-5618-3. p. 125–132. doi: 10.1145/3205455.3205567. Full text in Research Archive
  • Nordmoen, Jørgen Halvorsen; Samuelsen, Eivind; Ellefsen, Kai Olav & Glette, Kyrre (2018). Dynamic mutation in MAP-Elites for robotic repertoire generation, The 2018 Conference on Artificial Life. MIT Press. ISSN 9780262355766. p. 598–605. doi: https%3A/doi.org/10.1162/isal_a_00110. Full text in Research Archive
  • Martin, Charles Patrick; Jensenius, Alexander Refsum & Tørresen, Jim (2018). Composing an ensemble standstill work for Myo and Bela. In Dahl, Luke; Bowman, Doug & Martin, Tom (Ed.), Proceedings of the International Conference On New Interfaces For Musical Expression. Virginia Tech. ISSN 2220-4792. p. 196–197. Full text in Research Archive

View all works in Cristin

  • Ellefsen, Kai Olav & Rohlfing, Katharina J. (2019). Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE. ISBN 978-1-5386-8129-9. 340 p.

View all works in Cristin

  • (2020). Evolutionary algorithms for intelligent robots.
  • Tørresen, Jim (2019). Making Robots Adaptive and Preferable to Humans.
  • Tørresen, Jim (2019). Kunstig intelligens – hvem, hva og hvor. (Eng. Artificial Intelligence – who, what and where).
  • Becker, Artur; Herrebrøden, Henrik; Gonzalez Sanchez, Victor Evaristo; Nymoen, Kristian; Dal Sasso Freitas, Carla Maria & Tørresen, Jim [Show all 7 contributors for this article] (2019). Functional Data Analysis of Rowing Technique Using Motion Capture Data.
  • (2019). Her er universitetets mest avanserte, selvlærende robot. [Business/trade/industry journal]. Apollon.
  • Tørresen, Jim (2019). Intelligent Robots and Systems in Real-World Environment.
  • (2019). Kunstig intelligens for tilpasningsdyktige roboter.
  • Ellefsen, Kai Olav (2019). Hva Kan Roboter Lære av Biologisk Liv?
  • Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Intelligent, Adaptive Robots in Real-World Scenarios.
  • Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Adaptive Robot Body and Control for Real-World Environments.
  • Rohlfing, Katharina J. & Tørresen, Jim (2019). Explainability: an interactive view.
  • Miura, Jun & Tørresen, Jim (2019). Intelligent Robot Technologies for Care and Lifestyle Support .
  • Comba, Joao Luiz Dihl & Tørresen, Jim (2019). Visual Data Analysis of Unstructured and Big Data.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing.
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-Adapting Goals Allow Transfer of Predictive Models to New Tasks.
  • Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods.
  • Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives.
  • Nordmoen, Jørgen Halvorsen & Fadelli, Ingrid (2019). A new method to enable robust locomotion in a quadruped robot. [Internet]. TechXplore.
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing.
  • Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion .
  • Martin, Charles Patrick; Næss, Torgrim Rudland; Faitas, Andrei & Baumann, Synne Engdahl (2019). Session on Musical Prediction and Generation with Deep Learning.
  • Martin, Charles Patrick & Torresen, Jim (2019). An Interactive Music Prediction System with Mixture Density Recurrent Neural Networks.
  • Næss, Torgrim Rudland; Tørresen, Jim & Martin, Charles Patrick (2019). A Physical Intelligent Instrument using Recurrent Neural Networks.
  • Martin, Charles Patrick & Tørresen, Jim (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks.
  • Faitas, Andrei; Baumann, Synne Engdahl; Torresen, Jim & Martin, Charles Patrick (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks.
  • Glette, Kyrre; Nygaard, Tønnes Frostad & Vogt, Yngve (2019). Her er universitetets nest selvlærende robot. [Business/trade/industry journal]. Teknisk ukeblad.
  • Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN.
  • Tørresen, Jim (2019). Design and Control of Robots for Real-World Environment.
  • Tørresen, Jim (2019). Supporting Older People with Robots for Independent Living.
  • Tørresen, Jim (2019). Intelligent and Adaptive Robots in Real-World Environment.
  • (2019). Kunstig intelligens for tilpasningsdyktige roboter .
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Lessons Learned from Real-World Experiments with DyRET: the Dynamic Robot for Embodied Testing.
  • Tørresen, Jim (2019). Artificial Intelligence and Applications in Health and Care .
  • Tørresen, Jim (2019). Hva er kunstig intelligens?
  • Tørresen, Jim (2019). Sensing Human State with Application in Older People Care and Mental Health Treatment.
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Evolutionary Robotics: Automatic design of robot bodies and control.
  • Tørresen, Jim (2019). Future and Ethical Perspectives of Robotics and AI.
  • Tørresen, Jim (2018). Roboter kommer nærmere – skal vi glede eller grue oss?
  • Tørresen, Jim (2018). Remote Lab and Applications for High Performance and Embedded Architectures.
  • Martin, Charles Patrick; Xambó, Anna; Visi, Federico; Morreale, Fabio & Jensenius, Alexander Refsum (2018). Stillness under Tension.
  • Jensenius, Alexander Refsum; Martin, Charles Patrick; Bjerkestrand, Kari Anne Vadstensvik & Johnson, Victoria (2018). Stillness under Tension.
  • Martin, Charles Patrick; Jensenius, Alexander Refsum & Tørresen, Jim (2018). Composing an ensemble standstill work for Myo and Bela.
  • Tørresen, Jim; Garcia Ceja, Enrique Alejandro; Ellefsen, Kai Olav & Martin, Charles Patrick (2018). Equipping Systems with Forecasting Capabilities .
  • Martin, Charles Patrick (2018). Creative Prediction with Neural Networks.
  • Garcia Ceja, Enrique Alejandro; Ellefsen, Kai Olav; Martin, Charles Patrick & Tørresen, Jim (2018). Prediction, Interaction, and User Behaviour.
  • Martin, Charles Patrick; Glette, Kyrre & Tørresen, Jim (2018). Creative Prediction with Neural Networks.
  • Martin, Charles Patrick (2018). Predictive Music Systems for Interactive Performance.
  • Nygaard, Tønnes Frostad & Khattar, Shivani (2018). How a shape-shifting robot is learning from its mistakes. [TV]. NBC news.
  • Martin, Charles Patrick (2018). MicroJam.
  • Nygaard, Tønnes Frostad & Gonzales, Robbie (2018). How a Flock of Drones Developed Collective Intelligence. [Business/trade/industry journal]. Wired Science.
  • Nygaard, Tønnes Frostad & Papadopoulos, Loukia (2018). New Evolving Robot Teaches Itself to Walk Through Trial and Error. [Business/trade/industry journal]. Interesting Engineering.
  • Nygaard, Tønnes Frostad & Simon, Matt (2018). The shape-shifting robot that evolves by falling down. [Business/trade/industry journal]. Wired Science.
  • Nygaard, Tønnes Frostad & Dormehl, Luke (2018). This robot taught itself how to walk and it’s as clumsy as a newborn deer. [Business/trade/industry journal]. Digital Trends.
  • Nygaard, Tønnes Frostad & Simon, Matt (2018). How Roboticists Are Copying Nature To Make Fantastical Machines. [Internet]. Wired Science.
  • Nygaard, Tønnes Frostad & Nordmoen, Jørgen Halvorsen (2018). Dynamic Robot for Embodied Testing, Open Source Material.
  • (2018). Automatic design of bodies and behaviors for real-world robots.
  • Moen, Hans Jonas Fossum; Glette, Kyrre; Nygaard, Tønnes Frostad & Johnsrud, Mette (2018). Fem felt der vi får en førerløs fremtid. [Internet]. Titan.uio.no.
  • (2018). Real-World Evolution Adapts Robot Morphology and Control to Hardware Limitations.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2018). Exploring Mechanically Self-Reconfiguring Robots for Autonomous Design.
  • Nygaard, Tønnes Frostad; Søyseth, Vegard Dønnem; Nordmoen, Jørgen Halvorsen & Glette, Kyrre (2018). Stand with the DyRET robot.
  • Tørresen, Jim (2018). Kunstig Intelligens – Lærende og tilpasningsdyktig teknologi.
  • Tørresen, Jim (2018). Artificial Intelligence – State-of-the-art.
  • Tørresen, Jim (2018). Ethical Robots and Autonomous Systems.
  • Tørresen, Jim (2018). Intelligent Systems for Medical and Healthcare Applications.
  • Tørresen, Jim (2018). Når etikk betyr alt. Dagens næringsliv. ISSN 0803-9372.
  • Martin, Charles Patrick (2018). Deep Predictive Models in Interactive Music.
  • Næss, Torgrim Rudland; Martin, Charles Patrick & Tørresen, Jim (2019). A Physical Intelligent Instrument using Recurrent Neural Networks. Universitetet i Oslo.
  • Tørresen, Jim; Teigen, Bjørn Ivar & Ellefsen, Kai Olav (2018). An Active Learning Perspective on Exploration in Reinforcement Learning. Universitetet i Oslo.
  • Wallace, Benedikte & Martin, Charles Patrick (2018). Predictive songwriting with concatenative accompaniment. Universitetet i Oslo.
  • Fjeld, Matias Hermanrud & Tørresen, Jim (2018). 3D Spatial Navigation in Octrees with Reinforcement Learning. Universitetet i Oslo.
  • Brustad, Henrik & Martin, Charles Patrick (2018). Digital Audio Generation with Neural Networks. Universitetet i Oslo.

View all works in Cristin

Tags: machine learning, robotics, interactive music
Published May 24, 2016 3:38 PM - Last modified Aug. 27, 2020 9:59 PM