Norwegian version of this page

EPEC - Engineering Predictability with Embodied Cognition (completed)

How can multimodal systems sense, learn, and predict future events?

EPEC.  Engineering predictability with embodied cognition. It says with letters.. Logo.

Humans are superior to computers and robots when it comes to perceiving with eyes, ears and other senses as well as combining perception with learned knowledge to choose the best actions. This project aims to develop human-inspired models of behaviour and perception and to show that these models can predict future actions accurately.

Our inspiration comes from embodied cognition, a concept from psychology proposing that our bodies, perceptions, abilities, and form, influences how we think. Our goal is to exploit the form of various systems to develop predictive reasoning models as alternatives to traditional reactive systems. These models will be applied in interdisciplinary fields of music technology and robotics. In music, we aim to provide everyday people new ways to move within musical spaces. Our models learn about their interactions with smartphones to proactively assist with their future actions. In robotics, we are developing robots with dynamic forms that can change their thinking in response to new body shapes.

A hand holding a smartphone and feet on some robotics. Photo.
Musical interaction on smartphones and robotic systems, are EPEC's application areas for new predictive models.

EPEC is directed by Professor Jim Tørresen, who also leads the ROBIN research group in the Department of Informatics. The project employs two post doctoral fellows, Kai Olav Ellefsen and Charles Martin, and PhD researcher Tønnes Nygaard. The project also includes Associate Professor Kyrre Glette, PhD researcher Jørgen Nordmoen, and a number of masters students in machine learning, robotics and music technology.

Objectives

Design, implement and evaluate multimodal systems that are able to sense, learn and predict future events.

Sub-projects

  • Internal Models: Predicting real-world effects through internal simulations
  • DyRET: Dynamic Robot for Embodied Testing
  • Interactive music systems: Computer systems for extending and enhancing musical listening, performance and collaboration.

Master Projects

Researchers from the EPEC group supervise master projects in robotics, music technology, and machine learning. Come work with us on predictive models, embodied interactive systems and new robotic interactions!

Funding

Supported by The Research Council of Norway under FRINATEK grant agreement 240862 from 2015 to 2019. The grant funds 1 PhD and 2 post-doc positions (10% of prop. funded).

Publications

  • Nygaard, Tønnes; Martin, Charles Patrick; Howard, David; Tørresen, Jim & Glette, Kyrre (2021). Environmental Adaptation of Robot Morphology and Control Through Real-world Evolution. Evolutionary Computation. ISSN 1063-6560. doi: 10.1162/evco_a_00291. Full text in Research Archive
  • Nordmoen, Jørgen; Nygaard, Tønnes Frostad; Samuelsen, Eivind & Glette, Kyrre (2021). On Restricting Real-Valued Genotypes in Evolutionary Algorithms. In Castillo, Pedro A. & Jiménez Laredo, Juan Luiz (Ed.), Applications of Evolutionary Computation. Springer. ISSN 978-3-030-72699-7. p. 3–16. doi: 10.1007/978-3-030-72699-7_1. Full text in Research Archive
  • Nygaard, Tønnes; Martin, Charles Patrick; Tørresen, Jim; Glette, Kyrre & Howard, David (2021). Real-world embodied AI through a morphologically adaptive quadruped robot. Nature Machine Intelligence. doi: 10.1038/s42256-021-00320-3. Full text in Research Archive
  • Martin, Charles Patrick; Glette, Kyrre; Nygaard, Tønnes & Tørresen, Jim (2020). Understanding Musical Predictions With an Embodied Interface for Musical Machine Learning. Frontiers in Artificial Intelligence. ISSN 2624-8212. 3(6). doi: 10.3389/frai.2020.00006.
  • Nygaard, Tønnes; Howard, David & Glette, Kyrre (2020). Real world morphological evolution is feasible. In Coello Coello, Carlos A. (Eds.), GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery (ACM). ISSN 978-1-4503-7127-8. p. 1392–1394. doi: 10.1145/3377929.3398095.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing. IEEE International Conference on Robotics and Automation (ICRA). ISSN 1050-4729. 2019-May, p. 9446–9452. doi: 10.1109/ICRA.2019.8793663. Full text in Research Archive
  • Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-5386-8129-9. p. 189–194. doi: 10.1109/DEVLRN.2019.8850685.
  • Garcia, Rafael; Falcao, Alexandre Xavier; Telea, Alexandru C.; Castro da Silva, Bruno; Tørresen, Jim & Comba, Joao Luiz Dihl (2019). A Methodology for Neural Network Architectural Tuning Using Activation Occurrence Maps. In Jayne, Chrisina & Somogyvári, Zoltán (Ed.), 2019 International Joint Conference on Neural Networks (IJCNN) . IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-7281-1985-4. doi: 10.1109/IJCNN.2019.8852223.
  • Weber, Aline; Alegre, Lucas N.; Tørresen, Jim & Castro da Silva, Bruno (2019). Parameterized Melody Generation with Autoencoders and Temporally-Consistent Noise. In Visi, Federico (Eds.), Music Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 174–179. Full text in Research Archive
  • Wallace, Benedikte & Martin, Charles Patrick (2019). Comparing models for harmony prediction in an interactive audio looper. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11453 LNCS, p. 173–187. doi: 10.1007/978-3-030-16667-0_12.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Evolving Robots on Easy Mode: Towards a Variable Complexity Controller for Quadrupeds. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11454 LNCS, p. 616–632. doi: 10.1007/978-3-030-16692-2_41. Full text in Research Archive
  • Martin, Charles Patrick & Tørresen, Jim (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 260–265. Full text in Research Archive
  • Næss, Torgrim Rudland & Martin, Charles Patrick (2019). A Physical Intelligent Instrument using Recurrent Neural Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 79–82. Full text in Research Archive
  • Faitas, Andrei; Baumann, Synne Engdahl; Næss, Torgrim Rudland; Tørresen, Jim & Martin, Charles Patrick (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. Full text in Research Archive
  • Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion. In López-Ibáñez, Manuel (Eds.), GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 978-1-4503-6111-8. p. 133–141. doi: 10.1145/3321707.3321762. Full text in Research Archive
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 978-3-030-35664-4. p. 58–68. doi: https%3A/doi.org/10.1007/978-3-030-35664-4_6.
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-adapting Goals Allow Transfer of Predictive Models to New Tasks, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 978-3-030-35664-4. p. 28–39. doi: https%3A/doi.org/10.1007/978-3-030-35664-4_3.
  • Becker, Artur; Herrebrøden, Henrik; Gonzalez Sanchez, Victor Evaristo; Nymoen, Kristian; Dal Sasso Freitas, Carla Maria & Tørresen, Jim [Show all 7 contributors for this article] (2019). Functional Data Analysis of Rowing Technique Using Motion Capture Data. In Coleman, Grisha (Eds.), Proceedings of the 6th International Conference on Movement and Computing. ACM Publications. ISSN 978-1-4503-7654-9. doi: 10.1145/3347122.3347135.
  • Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN. IEEE International Conference on Robotics and Automation (ICRA). ISSN 1050-4729. 2019-May, p. 8883–8889. doi: 10.1109/ICRA.2019.8794077. Full text in Research Archive
  • Weber, Aline; Martin, Charles Patrick; Tørresen, Jim & da Silva, Bruno Castro (2019). Identifying Reusable Early-Life Options, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-5386-8129-9. doi: 10.1109/DEVLRN.2019.8850725.
  • Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives. Evolutionary Computation. ISSN 1063-6560. 28(1), p. 115–140. doi: 10.1162/evco_a_00250. Full text in Research Archive
  • Martin, Charles Patrick & Gardner, Henry (2019). Free-Improvised Rehearsal-as-Research for Musical HCI. In Holland, Simon; Mudd, Tom; Wilkie-McKenna, Katie; McPherson, Andrew & Wanderley, Marcelo M. (Ed.), New Directions in Music and Human-Computer Interaction. Springer. ISSN 978-3-319-92068-9. p. 269–284. doi: https%3A/doi.org/10.1007/978-3-319-92069-6_17. Full text in Research Archive
  • Martin, Charles Patrick & Tørresen, Jim (2019). Data Driven Analysis of Tiny Touchscreen Performance with MicroJam. Computer Music Journal. ISSN 0148-9267. 43(4). Full text in Research Archive
  • Garcia, Rafael; Telea, Alexandru C; Castro da Silva, Bruno; Tørresen, Jim & Dihl Comba, Joao Luiz (2018). A task-and-technique centered survey on visual analytics for deep learning model engineering. Computers & graphics. ISSN 0097-8493. 77, p. 30–49. doi: 10.1016/j.cag.2018.09.018.
  • Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2018). Transfer Learning for Unseen Robot Detection and Joint Estimation on a Multi-Objective Convolutional Neural Network, 2018 IEEE International Conference on Intelligence and Safety for Robotics (ISR 2018). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781538655481. p. 337–342. doi: 10.1109/IISR.2018.8535937. Full text in Research Archive
  • Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2018). Multi-Objective Convolutional Neural Networks for Robot Localisation and 3D Position Estimation in 2D Camera Images, 2018 15th International Conference on Ubiquitous Robots (UR 2018). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781538663356. p. 597–603. doi: 10.1109/URAI.2018.8441813. Full text in Research Archive
  • Miseikis, Justinas; Knobelreiter, Patrick; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre & Elle, Ole Jacob (2018). Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks, 2018 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2018). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781538618554. p. 181–187. doi: 10.1109/AIM.2018.8452236. Full text in Research Archive
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Samuelsen, Eivind; Tørresen, Jim & Glette, Kyrre (2018). Real-world evolution adapts robot morphology and control to hardware limitations. In aguirre, hernan (Eds.), GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 978-1-4503-5618-3. p. 125–132. doi: 10.1145/3205455.3205567. Full text in Research Archive
  • Martin, Charles Patrick & Tørresen, Jim (2018). RoboJam: A musical mixture density network for collaborative touchscreen interaction. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10783 LNCS, p. 161–176. doi: 10.1007/978-3-319-77583-8_11. Full text in Research Archive
  • Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav & Glette, Kyrre (2018). Combining MAP-Elites and Incremental Evolution to Generate Gaits for a Mammalian Quadruped Robot. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10784 LNCS, p. 719–733. doi: 10.1007/978-3-319-77538-8_48. Full text in Research Archive
  • Tørresen, Jim (2018). A Review of Future and Ethical Perspectives of Robotics and AI. Frontiers in Robotics and AI. ISSN 2296-9144. 4. doi: 10.3389/frobt.2017.00075. Full text in Research Archive

View all works in Cristin

View all works in Cristin

  • (2020). Evolutionary algorithms for intelligent robots.
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Lessons Learned from Real-World Experiments with DyRET: the Dynamic Robot for Embodied Testing.
  • Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN.
  • Tørresen, Jim (2019). Intelligent and Adaptive Robots in Real-World Environment.
  • Rohlfing, Katharina J. & Tørresen, Jim (2019). Explainability: an interactive view.
  • Miura, Jun & Tørresen, Jim (2019). Intelligent Robot Technologies for Care and Lifestyle Support .
  • Tørresen, Jim (2019). Intelligent Robots and Systems in Real-World Environment.
  • (2019). Her er universitetets mest avanserte, selvlærende robot. [Business/trade/industry journal]. Apollon.
  • (2019). Kunstig intelligens for tilpasningsdyktige roboter.
  • Tørresen, Jim (2019). Making Robots Adaptive and Preferable to Humans.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing.
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-Adapting Goals Allow Transfer of Predictive Models to New Tasks.
  • Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion .
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing.
  • Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods.
  • Nordmoen, Jørgen Halvorsen & Fadelli, Ingrid (2019). A new method to enable robust locomotion in a quadruped robot. [Internet]. TechXplore.
  • Ellefsen, Kai Olav (2019). Hva Kan Roboter Lære av Biologisk Liv?
  • Becker, Artur; Herrebrøden, Henrik; Gonzalez Sanchez, Victor Evaristo; Nymoen, Kristian; Dal Sasso Freitas, Carla Maria & Tørresen, Jim [Show all 7 contributors for this article] (2019). Functional Data Analysis of Rowing Technique Using Motion Capture Data.
  • Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Intelligent, Adaptive Robots in Real-World Scenarios.
  • Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Adaptive Robot Body and Control for Real-World Environments.
  • (2019). Kunstig intelligens for tilpasningsdyktige roboter .
  • Martin, Charles Patrick & Torresen, Jim (2019). An Interactive Music Prediction System with Mixture Density Recurrent Neural Networks.
  • Næss, Torgrim Rudland; Tørresen, Jim & Martin, Charles Patrick (2019). A Physical Intelligent Instrument using Recurrent Neural Networks.
  • Martin, Charles Patrick; Næss, Torgrim Rudland; Faitas, Andrei & Baumann, Synne Engdahl (2019). Session on Musical Prediction and Generation with Deep Learning.
  • Faitas, Andrei; Baumann, Synne Engdahl; Torresen, Jim & Martin, Charles Patrick (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks.
  • Tørresen, Jim (2019). Design and Control of Robots for Real-World Environment.
  • Tørresen, Jim (2019). Future and Ethical Perspectives of Robotics and AI.
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Evolutionary Robotics: Automatic design of robot bodies and control.
  • Tørresen, Jim (2019). Artificial Intelligence and Applications in Health and Care .
  • Tørresen, Jim (2019). Sensing Human State with Application in Older People Care and Mental Health Treatment.
  • Tørresen, Jim (2019). Hva er kunstig intelligens?
  • Tørresen, Jim (2019). Supporting Older People with Robots for Independent Living.
  • Martin, Charles Patrick & Tørresen, Jim (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks.
  • Comba, Joao Luiz Dihl & Tørresen, Jim (2019). Visual Data Analysis of Unstructured and Big Data.
  • Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives.
  • Tørresen, Jim (2019). Kunstig intelligens – hvem, hva og hvor. (Eng. Artificial Intelligence – who, what and where).
  • Glette, Kyrre; Nygaard, Tønnes Frostad & Vogt, Yngve (2019). Her er universitetets nest selvlærende robot. [Business/trade/industry journal]. Teknisk ukeblad.
  • Tørresen, Jim (2018). Artificial Intelligence Applied for Real-World Systems.
  • Tørresen, Jim (2018). Kunstig intelligens – hvem, hva og hvor.
  • Stoica, Adrian & Tørresen, Jim (2018). Robots on the Moon, and their Role in a Future Lunar Economy .
  • Tørresen, Jim (2018). Frelsende eller fatalt? [Business/trade/industry journal]. Forskningsetikk.
  • (2018). Robotics, Intelligent Systems and Evolutionary Computation .
  • (2018). Automatic design of shapes and behaviors for robots.
  • Martin, Charles Patrick & Tørresen, Jim (2018). Predictive Musical Interaction with MDRNNs.
  • Tørresen, Jim (2018). UiO Visit to UFRJ – An overview of research .
  • Ellefsen, Kai Olav (2018). Evolusjonær Robotikk: Automatisk design og kontroll av roboter.
  • Søyseth, Vegard Dønnem; Nygaard, Tønnes Frostad; Martin, Charles Patrick; Uddin, Md Zia & Ellefsen, Kai Olav (2018). ROBIN-Stand ved Cutting Edge 2018.
  • Ellefsen, Kai Olav & Tørresen, Jim (2018). Evolutionary Robotics: Automatic design of robot controllers and bodies.
  • Tørresen, Jim (2018). Roboter kommer nærmere – skal vi glede eller grue oss?
  • Tørresen, Jim (2018). Artificial Intelligence – State-of-the-art.
  • Tørresen, Jim (2018). Remote Lab and Applications for High Performance and Embedded Architectures.
  • Tørresen, Jim (2018). Kunstig Intelligens – Lærende og tilpasningsdyktig teknologi.
  • (2018). Real-World Evolution Adapts Robot Morphology and Control to Hardware Limitations.
  • Nygaard, Tønnes Frostad; Søyseth, Vegard Dønnem; Nordmoen, Jørgen Halvorsen & Glette, Kyrre (2018). Stand with the DyRET robot.
  • Martin, Charles Patrick (2018). MicroJam.
  • Nygaard, Tønnes Frostad & Papadopoulos, Loukia (2018). New Evolving Robot Teaches Itself to Walk Through Trial and Error. [Business/trade/industry journal]. Interesting Engineering.
  • Nygaard, Tønnes Frostad & Gonzales, Robbie (2018). How a Flock of Drones Developed Collective Intelligence. [Business/trade/industry journal]. Wired Science.
  • Nygaard, Tønnes Frostad & Dormehl, Luke (2018). This robot taught itself how to walk and it’s as clumsy as a newborn deer. [Business/trade/industry journal]. Digital Trends.
  • Nygaard, Tønnes Frostad & Khattar, Shivani (2018). How a shape-shifting robot is learning from its mistakes. [TV]. NBC news.
  • Nygaard, Tønnes Frostad & Simon, Matt (2018). The shape-shifting robot that evolves by falling down. [Business/trade/industry journal]. Wired Science.
  • Nygaard, Tønnes Frostad & Simon, Matt (2018). How Roboticists Are Copying Nature To Make Fantastical Machines. [Internet]. Wired Science.
  • Nygaard, Tønnes Frostad & Nordmoen, Jørgen Halvorsen (2018). Dynamic Robot for Embodied Testing, Open Source Material.
  • Martin, Charles Patrick; Glette, Kyrre & Tørresen, Jim (2018). Creative Prediction with Neural Networks.
  • Martin, Charles Patrick (2018). Predictive Music Systems for Interactive Performance.
  • Garcia Ceja, Enrique Alejandro; Ellefsen, Kai Olav; Martin, Charles Patrick & Tørresen, Jim (2018). Prediction, Interaction, and User Behaviour.
  • Tørresen, Jim; Garcia Ceja, Enrique Alejandro; Ellefsen, Kai Olav & Martin, Charles Patrick (2018). Equipping Systems with Forecasting Capabilities .
  • (2018). Automatic design of bodies and behaviors for real-world robots.
  • Næss, Torgrim Rudland; Martin, Charles Patrick & Tørresen, Jim (2019). A Physical Intelligent Instrument using Recurrent Neural Networks. Universitetet i Oslo.

View all works in Cristin

Tags: machine learning, robotics, interactive music
Published May 24, 2016 3:38 PM - Last modified Aug. 27, 2020 9:59 PM