Norwegian version of this page

EPEC - Engineering Predictability with Embodied Cognition (completed)

How can multimodal systems sense, learn, and predict future events?

EPEC.  Engineering predictability with embodied cognition. It says with letters.. Logo.

Humans are superior to computers and robots when it comes to perceiving with eyes, ears and other senses as well as combining perception with learned knowledge to choose the best actions. This project aims to develop human-inspired models of behaviour and perception and to show that these models can predict future actions accurately.

Our inspiration comes from embodied cognition, a concept from psychology proposing that our bodies, perceptions, abilities, and form, influences how we think. Our goal is to exploit the form of various systems to develop predictive reasoning models as alternatives to traditional reactive systems. These models will be applied in interdisciplinary fields of music technology and robotics. In music, we aim to provide everyday people new ways to move within musical spaces. Our models learn about their interactions with smartphones to proactively assist with their future actions. In robotics, we are developing robots with dynamic forms that can change their thinking in response to new body shapes.

A hand holding a smartphone and feet on some robotics. Photo.
Musical interaction on smartphones and robotic systems, are EPEC's application areas for new predictive models.

EPEC is directed by Professor Jim Tørresen, who also leads the ROBIN research group in the Department of Informatics. The project employs two post doctoral fellows, Kai Olav Ellefsen and Charles Martin, and PhD researcher Tønnes Nygaard. The project also includes Associate Professor Kyrre Glette, PhD researcher Jørgen Nordmoen, and a number of masters students in machine learning, robotics and music technology.

Objectives

Design, implement and evaluate multimodal systems that are able to sense, learn and predict future events.

Sub-projects

  • Internal Models: Predicting real-world effects through internal simulations
  • DyRET: Dynamic Robot for Embodied Testing
  • Interactive music systems: Computer systems for extending and enhancing musical listening, performance and collaboration.

Master Projects

Researchers from the EPEC group supervise master projects in robotics, music technology, and machine learning. Come work with us on predictive models, embodied interactive systems and new robotic interactions!

Funding

Supported by The Research Council of Norway under FRINATEK grant agreement 240862 from 2015 to 2019. The grant funds 1 PhD and 2 post-doc positions (10% of prop. funded).

Publications

  • Nygaard, Tønnes; Martin, Charles Patrick; Howard, David; Tørresen, Jim & Glette, Kyrre (2021). Environmental Adaptation of Robot Morphology and Control Through Real-world Evolution. Evolutionary Computation. ISSN 1063-6560. doi: 10.1162/evco_a_00291. Full text in Research Archive
  • Nordmoen, Jørgen; Nygaard, Tønnes Frostad; Samuelsen, Eivind & Glette, Kyrre (2021). On Restricting Real-Valued Genotypes in Evolutionary Algorithms. In Castillo, Pedro A. & Jiménez Laredo, Juan Luiz (Ed.), Applications of Evolutionary Computation. Springer. ISSN 978-3-030-72699-7. p. 3–16. doi: 10.1007/978-3-030-72699-7_1. Full text in Research Archive
  • Nygaard, Tønnes; Martin, Charles Patrick; Tørresen, Jim; Glette, Kyrre & Howard, David (2021). Real-world embodied AI through a morphologically adaptive quadruped robot. Nature Machine Intelligence. doi: 10.1038/s42256-021-00320-3. Full text in Research Archive
  • Martin, Charles Patrick; Glette, Kyrre; Nygaard, Tønnes & Tørresen, Jim (2020). Understanding Musical Predictions With an Embodied Interface for Musical Machine Learning. Frontiers in Artificial Intelligence. ISSN 2624-8212. 3(6). doi: 10.3389/frai.2020.00006.
  • Nygaard, Tønnes; Howard, David & Glette, Kyrre (2020). Real world morphological evolution is feasible. In Coello Coello, Carlos A. (Eds.), GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery (ACM). ISSN 978-1-4503-7127-8. p. 1392–1394. doi: 10.1145/3377929.3398095.
  • Martin, Charles Patrick & Tørresen, Jim (2019). Data Driven Analysis of Tiny Touchscreen Performance with MicroJam. Computer Music Journal. ISSN 0148-9267. 43(4). Full text in Research Archive
  • Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives. Evolutionary Computation. ISSN 1063-6560. 28(1), p. 115–140. doi: 10.1162/evco_a_00250. Full text in Research Archive
  • Garcia, Rafael; Falcao, Alexandre Xavier; Telea, Alexandru C.; Castro da Silva, Bruno; Tørresen, Jim & Comba, Joao Luiz Dihl (2019). A Methodology for Neural Network Architectural Tuning Using Activation Occurrence Maps. In Jayne, Chrisina & Somogyvári, Zoltán (Ed.), 2019 International Joint Conference on Neural Networks (IJCNN) . IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-7281-1985-4. doi: 10.1109/IJCNN.2019.8852223.
  • Martin, Charles Patrick & Tørresen, Jim (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 260–265. Full text in Research Archive
  • Næss, Torgrim Rudland & Martin, Charles Patrick (2019). A Physical Intelligent Instrument using Recurrent Neural Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 79–82. Full text in Research Archive
  • Becker, Artur; Herrebrøden, Henrik; Gonzalez Sanchez, Victor Evaristo; Nymoen, Kristian; Dal Sasso Freitas, Carla Maria & Tørresen, Jim [Show all 7 contributors for this article] (2019). Functional Data Analysis of Rowing Technique Using Motion Capture Data. In Coleman, Grisha (Eds.), Proceedings of the 6th International Conference on Movement and Computing. ACM Publications. ISSN 978-1-4503-7654-9. doi: 10.1145/3347122.3347135.
  • Weber, Aline; Alegre, Lucas N.; Tørresen, Jim & Castro da Silva, Bruno (2019). Parameterized Melody Generation with Autoencoders and Temporally-Consistent Noise. In Visi, Federico (Eds.), Music Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 174–179. Full text in Research Archive
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing. IEEE International Conference on Robotics and Automation (ICRA). ISSN 1050-4729. 2019-May, p. 9446–9452. doi: 10.1109/ICRA.2019.8793663. Full text in Research Archive
  • Weber, Aline; Martin, Charles Patrick; Tørresen, Jim & da Silva, Bruno Castro (2019). Identifying Reusable Early-Life Options, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-5386-8129-9. doi: 10.1109/DEVLRN.2019.8850725.
  • Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-5386-8129-9. p. 189–194. doi: 10.1109/DEVLRN.2019.8850685.
  • Faitas, Andrei; Baumann, Synne Engdahl; Næss, Torgrim Rudland; Tørresen, Jim & Martin, Charles Patrick (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. Full text in Research Archive
  • Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion. In López-Ibáñez, Manuel (Eds.), GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 978-1-4503-6111-8. p. 133–141. doi: 10.1145/3321707.3321762. Full text in Research Archive
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 978-3-030-35664-4. p. 58–68. doi: https%3A/doi.org/10.1007/978-3-030-35664-4_6.
  • Wallace, Benedikte & Martin, Charles Patrick (2019). Comparing models for harmony prediction in an interactive audio looper. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11453 LNCS, p. 173–187. doi: 10.1007/978-3-030-16667-0_12.
  • Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN. IEEE International Conference on Robotics and Automation (ICRA). ISSN 1050-4729. 2019-May, p. 8883–8889. doi: 10.1109/ICRA.2019.8794077. Full text in Research Archive
  • Martin, Charles Patrick & Gardner, Henry (2019). Free-Improvised Rehearsal-as-Research for Musical HCI. In Holland, Simon; Mudd, Tom; Wilkie-McKenna, Katie; McPherson, Andrew & Wanderley, Marcelo M. (Ed.), New Directions in Music and Human-Computer Interaction. Springer. ISSN 978-3-319-92068-9. p. 269–284. doi: https%3A/doi.org/10.1007/978-3-319-92069-6_17. Full text in Research Archive
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Evolving Robots on Easy Mode: Towards a Variable Complexity Controller for Quadrupeds. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11454 LNCS, p. 616–632. doi: 10.1007/978-3-030-16692-2_41. Full text in Research Archive
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-adapting Goals Allow Transfer of Predictive Models to New Tasks, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 978-3-030-35664-4. p. 28–39. doi: https%3A/doi.org/10.1007/978-3-030-35664-4_3.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Samuelsen, Eivind; Tørresen, Jim & Glette, Kyrre (2018). Real-world evolution adapts robot morphology and control to hardware limitations. In aguirre, hernan (Eds.), GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 978-1-4503-5618-3. p. 125–132. doi: 10.1145/3205455.3205567. Full text in Research Archive
  • Martin, Charles Patrick; Jensenius, Alexander Refsum & Tørresen, Jim (2018). Composing an ensemble standstill work for Myo and Bela. In Dahl, Luke; Bowman, Doug & Martin, Tom (Ed.), Proceedings of the International Conference On New Interfaces For Musical Expression. Virginia Tech. ISSN 2220-4792. p. 196–197. Full text in Research Archive
  • Garcia, Rafael; Telea, Alexandru C; Castro da Silva, Bruno; Tørresen, Jim & Dihl Comba, Joao Luiz (2018). A task-and-technique centered survey on visual analytics for deep learning model engineering. Computers & graphics. ISSN 0097-8493. 77, p. 30–49. doi: 10.1016/j.cag.2018.09.018.
  • Tørresen, Jim (2018). A Review of Future and Ethical Perspectives of Robotics and AI. Frontiers in Robotics and AI. ISSN 2296-9144. 4. doi: 10.3389/frobt.2017.00075. Full text in Research Archive
  • Nordmoen, Jørgen Halvorsen; Samuelsen, Eivind; Ellefsen, Kai Olav & Glette, Kyrre (2018). Dynamic mutation in MAP-Elites for robotic repertoire generation, The 2018 Conference on Artificial Life. MIT Press. ISSN 9780262355766. p. 598–605. doi: https%3A/doi.org/10.1162/isal_a_00110. Full text in Research Archive
  • Martin, Charles Patrick & Tørresen, Jim (2018). RoboJam: A musical mixture density network for collaborative touchscreen interaction. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10783 LNCS, p. 161–176. doi: 10.1007/978-3-319-77583-8_11. Full text in Research Archive

View all works in Cristin

View all works in Cristin

  • (2020). Evolutionary algorithms for intelligent robots.
  • Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives.
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing.
  • Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods.
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-Adapting Goals Allow Transfer of Predictive Models to New Tasks.
  • Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion .
  • Tørresen, Jim (2019). Design and Control of Robots for Real-World Environment.
  • Tørresen, Jim (2019). Artificial Intelligence and Applications in Health and Care .
  • Tørresen, Jim (2019). Sensing Human State with Application in Older People Care and Mental Health Treatment.
  • Tørresen, Jim (2019). Hva er kunstig intelligens?
  • Tørresen, Jim (2019). Supporting Older People with Robots for Independent Living.
  • Tørresen, Jim (2019). Future and Ethical Perspectives of Robotics and AI.
  • Ellefsen, Kai Olav & Tørresen, Jim (2019). Evolutionary Robotics: Automatic design of robot bodies and control.
  • Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Adaptive Robot Body and Control for Real-World Environments.
  • Miura, Jun & Tørresen, Jim (2019). Intelligent Robot Technologies for Care and Lifestyle Support .
  • Tørresen, Jim (2019). Intelligent and Adaptive Robots in Real-World Environment.
  • Nordmoen, Jørgen Halvorsen & Fadelli, Ingrid (2019). A new method to enable robust locomotion in a quadruped robot. [Internet]. TechXplore.
  • Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Lessons Learned from Real-World Experiments with DyRET: the Dynamic Robot for Embodied Testing.
  • Martin, Charles Patrick & Tørresen, Jim (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing.
  • Tørresen, Jim (2019). Intelligent Robots and Systems in Real-World Environment.
  • (2019). Her er universitetets mest avanserte, selvlærende robot. [Business/trade/industry journal]. Apollon.
  • Rohlfing, Katharina J. & Tørresen, Jim (2019). Explainability: an interactive view.
  • Comba, Joao Luiz Dihl & Tørresen, Jim (2019). Visual Data Analysis of Unstructured and Big Data.
  • Glette, Kyrre; Nygaard, Tønnes Frostad & Vogt, Yngve (2019). Her er universitetets nest selvlærende robot. [Business/trade/industry journal]. Teknisk ukeblad.
  • (2019). Kunstig intelligens for tilpasningsdyktige roboter.
  • (2019). Kunstig intelligens for tilpasningsdyktige roboter .
  • Martin, Charles Patrick & Torresen, Jim (2019). An Interactive Music Prediction System with Mixture Density Recurrent Neural Networks.
  • Næss, Torgrim Rudland; Tørresen, Jim & Martin, Charles Patrick (2019). A Physical Intelligent Instrument using Recurrent Neural Networks.
  • Martin, Charles Patrick; Næss, Torgrim Rudland; Faitas, Andrei & Baumann, Synne Engdahl (2019). Session on Musical Prediction and Generation with Deep Learning.
  • Faitas, Andrei; Baumann, Synne Engdahl; Torresen, Jim & Martin, Charles Patrick (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks.
  • Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN.
  • Tørresen, Jim (2019). Making Robots Adaptive and Preferable to Humans.
  • Tørresen, Jim (2019). Kunstig intelligens – hvem, hva og hvor. (Eng. Artificial Intelligence – who, what and where).
  • Ellefsen, Kai Olav (2019). Hva Kan Roboter Lære av Biologisk Liv?
  • Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Intelligent, Adaptive Robots in Real-World Scenarios.
  • Becker, Artur; Herrebrøden, Henrik; Gonzalez Sanchez, Victor Evaristo; Nymoen, Kristian; Dal Sasso Freitas, Carla Maria & Tørresen, Jim [Show all 7 contributors for this article] (2019). Functional Data Analysis of Rowing Technique Using Motion Capture Data.
  • Martin, Charles Patrick; Glette, Kyrre; Nygaard, Tønnes Frostad & Tørresen, Jim (2018). Self-Awareness in a Cyber-Physical Predictive Musical Interface.
  • Tørresen, Jim; Garcia Ceja, Enrique Alejandro; Ellefsen, Kai Olav & Martin, Charles Patrick (2018). Equipping Systems with Forecasting Capabilities .
  • Martin, Charles Patrick (2018). Creative Prediction with Neural Networks.
  • Garcia Ceja, Enrique Alejandro; Ellefsen, Kai Olav; Martin, Charles Patrick & Tørresen, Jim (2018). Prediction, Interaction, and User Behaviour.
  • Nygaard, Tønnes Frostad & Papadopoulos, Loukia (2018). New Evolving Robot Teaches Itself to Walk Through Trial and Error. [Business/trade/industry journal]. Interesting Engineering.
  • Nygaard, Tønnes Frostad & Gonzales, Robbie (2018). How a Flock of Drones Developed Collective Intelligence. [Business/trade/industry journal]. Wired Science.
  • Nygaard, Tønnes Frostad & Dormehl, Luke (2018). This robot taught itself how to walk and it’s as clumsy as a newborn deer. [Business/trade/industry journal]. Digital Trends.
  • Nygaard, Tønnes Frostad & Khattar, Shivani (2018). How a shape-shifting robot is learning from its mistakes. [TV]. NBC news.
  • Nygaard, Tønnes Frostad & Simon, Matt (2018). How Roboticists Are Copying Nature To Make Fantastical Machines. [Internet]. Wired Science.
  • Martin, Charles Patrick (2018). MicroJam.
  • Tørresen, Jim (2018). Ethical Robots and Autonomous Systems.
  • Tørresen, Jim (2018). Kunstig intelligens – hvem, hva og hvor.
  • Tørresen, Jim (2018). Frelsende eller fatalt? [Business/trade/industry journal]. Forskningsetikk.
  • Stoica, Adrian & Tørresen, Jim (2018). Robots on the Moon, and their Role in a Future Lunar Economy .
  • Tørresen, Jim (2018). Roboter kommer nærmere – skal vi glede eller grue oss?
  • Tørresen, Jim (2018). Artificial Intelligence – State-of-the-art.
  • Martin, Charles Patrick; Jensenius, Alexander Refsum & Tørresen, Jim (2018). Composing an ensemble standstill work for Myo and Bela.
  • Jensenius, Alexander Refsum; Martin, Charles Patrick; Bjerkestrand, Kari Anne Vadstensvik & Johnson, Victoria (2018). Stillness under Tension.
  • Martin, Charles Patrick; Xambó, Anna; Visi, Federico; Morreale, Fabio & Jensenius, Alexander Refsum (2018). Stillness under Tension.
  • Tørresen, Jim (2018). Remote Lab and Applications for High Performance and Embedded Architectures.
  • Tørresen, Jim (2018). Kunstig Intelligens – Lærende og tilpasningsdyktig teknologi.
  • Ellefsen, Kai Olav (2018). Evolusjonær Robotikk: Automatisk design og kontroll av roboter.
  • Søyseth, Vegard Dønnem; Nygaard, Tønnes Frostad; Martin, Charles Patrick; Uddin, Md Zia & Ellefsen, Kai Olav (2018). ROBIN-Stand ved Cutting Edge 2018.
  • Martin, Charles Patrick (2018). Deep Predictive Models in Interactive Music.
  • (2018). Automatic design of bodies and behaviors for real-world robots.
  • Moen, Hans Jonas Fossum; Glette, Kyrre; Nygaard, Tønnes Frostad & Johnsrud, Mette (2018). Fem felt der vi får en førerløs fremtid. [Internet]. Titan.uio.no.
  • Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2018). Exploring Mechanically Self-Reconfiguring Robots for Autonomous Design.
  • Nygaard, Tønnes Frostad & Nordmoen, Jørgen Halvorsen (2018). Dynamic Robot for Embodied Testing, Open Source Material.
  • Næss, Torgrim Rudland; Martin, Charles Patrick & Tørresen, Jim (2019). A Physical Intelligent Instrument using Recurrent Neural Networks. Universitetet i Oslo.
  • Wallace, Benedikte & Martin, Charles Patrick (2018). Predictive songwriting with concatenative accompaniment. Universitetet i Oslo.
  • Fjeld, Matias Hermanrud & Tørresen, Jim (2018). 3D Spatial Navigation in Octrees with Reinforcement Learning. Universitetet i Oslo.
  • Brustad, Henrik & Martin, Charles Patrick (2018). Digital Audio Generation with Neural Networks. Universitetet i Oslo.
  • Tørresen, Jim; Teigen, Bjørn Ivar & Ellefsen, Kai Olav (2018). An Active Learning Perspective on Exploration in Reinforcement Learning. Universitetet i Oslo.

View all works in Cristin

Tags: machine learning, robotics, interactive music
Published May 24, 2016 3:38 PM - Last modified Sep. 28, 2022 12:53 PM