RITPART International Partnerships

• Anais, Llorens; Bellier, Ludovic; Blenkmann, Alejandro Omar; Ivanovic, Jugoslav; Larsson, Pål Gunnar & Lin, Jack J. [Show all 9 contributors for this article] (2023). Decision and response monitoring during working memory are sequentially represented in the human insula. iScience. ISSN 2589-0042. 26(10). doi: 10.1016/j.isci.2023.107653. Full text in Research Archive
• Weber, Jan; Iwama, Gabriela; Solbakk, Anne-Kristin; Blenkmann, Alejandro Omar; Larsson, Pål Gunnar & Ivanovic, Jugoslav [Show all 9 contributors for this article] (2023). Subspace partitioning in the human prefrontal cortex resolves cognitive interference. Proceedings of the National Academy of Sciences of the United States of America. ISSN 0027-8424. 120(28). doi: 10.1073/pnas.2220523120. Full text in Research Archive
• Johnson, Elizabeth L.; Chang, William K.; Dewar, Callum D.; Sorensen, Donna; Lin, Jack J. & Solbakk, Anne-Kristin [Show all 12 contributors for this article] (2022). Orbitofrontal cortex governs working memory for temporal order. Current Biology. ISSN 0960-9822. 32(9), p. R410–R411. doi: 10.1016/j.cub.2022.03.074.
• Jensenius, Alexander Refsum & Erdem, Cagri (2022). Gestures in ensemble performance. In Timmers, Renee; Bailes, Freya & Daffern, Helena (Ed.), Together in Music: Coordination, expression, participation. Oxford University Press. ISSN 9780198860761. Full text in Research Archive Show summary

  The topic of gesture has received growing attention among music researchers over recent decades. Some of this research has been summarized in anthologies on "musical gestures", such as those by Gritten and King (2006), Godøy and Leman (2010), and Gritten and King (2011). There have also been a couple of articles reviewing how the term gesture has been used in various music-related disciplines (and beyond), including those by Cadoz and Wanderley (2000) and Jensenius et al. (2010). Much empirical work has been performed since these reviews were written, aided by better motion capture technologies, new machine learning techniques, and a heightened awareness of the topic. Still there are a number of open questions as to the role of gestures in music performance in general, and in ensemble performance in particular. This chapter aims to clarify some of the basic terminology of music-related body motion, and draw up some perspectives of how one can think about gestures in ensemble performance. This is, obviously, only one way of looking at the very multifaceted concept of gesture, but it may lead to further interest in this exciting and complex research domain.

• Lesteberg, Mari & Jensenius, Alexander Refsum (2022). MICRO and MACRO - Developing New Accessible Musicking Technologies. In Iber, Michael & Enge, Kajetan (Ed.), Audio Mostly 2022: What you hear is what you see? Perspectives on modalities in sound and music interaction. ACM Publications. ISSN 978-1-4503-9701-8. p. 147–150. doi: 10.1145/3561212.3561231. Full text in Research Archive Show summary

  This paper describes the development of two musical instrument prototypes developed to explore how non-haptic music technologies can be accessed from a web browser and how they can offer accessibility for people with low fine motor skills. Two approaches to browser-based motion capture were developed and tested during an iterative design process. This was followed by observational studies of two user groups: one with low fine motor skills and one with normal motor skills. Contrary to our expectations, we found that avoiding the use of buttons and mice did not make the apps more accessible for the participants with low fine motor skills. Furthermore, motion speed was considered more important for people with low motor skills than the size of the control action. The most important finding is that browser-based musical instruments using sensor-based and video-based motion tracking are not only feasible but allow for reaching much larger groups of people than previously possible. This may ultimately lead to both more personalized and accessible musical experiences.

• Blenkmann, Alejandro Omar; Solbakk, Anne-Kristin; Ivanovic, Jugoslav; Larsson, Pål Gunnar; Knight, Robert Thomas & Endestad, Tor (2022). Modeling intracranial electrodes. A simulation platform for the evaluation of localization algorithms. Frontiers in Neuroinformatics. ISSN 1662-5196. 16, p. 1–20. doi: 10.3389/fninf.2022.788685. Full text in Research Archive
• Katarina Slama, Slama; Jimenez, Richard; Saha, Sujayam; King-Stephens, David; Laxer, Kenneth D. & Weber, Peter B. [Show all 12 contributors for this article] (2021). Intracranial recordings demonstrate both cortical and medial temporal lobe engagement in visual search in humans. Journal of Cognitive Neuroscience. ISSN 0898-929X. 33(9), p. 1833–1861. doi: 10.1162/jocn_a_01739.
• Masu, Raul; Melbye, Adam Pultz; Sullivan, John & Jensenius, Alexander Refsum (2021). NIME and the Environment: Toward a More Sustainable NIME Practice. In Dannenberg, Roger & Xiao, Xiao (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. The International Conference on New Interfaces for Musical Expression. ISSN 2220-4792. Full text in Research Archive Show summary

  This paper addresses environmental issues around NIME research and practice. We discuss the formulation of an environmental statement for the conference as well as the initiation of a NIME Eco Wiki containing information on environmental concerns related to the creation of new musical instruments. We outline a number of these concerns and, by systematically reviewing the proceedings of all previous NIME conferences, identify a general lack of reflection on the environmental impact of the research undertaken. Finally, we propose a framework for addressing the making, testing, using, and disposal of NIMEs in the hope that sustainability may become a central concern to researchers.

• Solbakk, Anne-Kristin; Lubell, James; Leske, Sabine; Funderud, Ingrid; Anais, Llorens & Blenkmann, Alejandro Omar [Show all 10 contributors for this article] (2021). Monitoring of self-paced action timing and sensory outcomes after lesions to the orbitofrontal cortex. Journal of Cognitive Neuroscience. ISSN 0898-929X. 33(9), p. 1956–1975. doi: 10.1162/jocn_a_01733.
• Liebrand, Matthias; Solbakk, Anne-Kristin; Funderud, Ingrid; Buades-Rotger, Macià; Knight, Robert Thomas & Krämer, Ulrike M (2021). Intact Proactive Motor Inhibition after Unilateral Prefrontal Cortex or Basal Ganglia Lesions. Journal of Cognitive Neuroscience. ISSN 0898-929X. 33(9), p. 1862–1879. doi: 10.1162/jocn_a_01691.
• Llorens, Anaïs; Tzovara, Athina; Bellier, Ludovic; Bhaya-Grossman, lIina; Bidet-Caulet, Aurelie & Chang, William K [Show all 45 contributors for this article] (2021). Gender bias in academia: A lifetime problem that needs solutions. Neuron. ISSN 0896-6273. 109(13), p. 2047–2074. doi: 10.1016/j.neuron.2021.06.002.
• Jensenius, Alexander Refsum (2021). Best versus Good Enough Practices for Open Music Research. Empirical Musicology Review. ISSN 1559-5749. 16(1). Full text in Research Archive Show summary

  Music researchers work with increasingly large and complex data sets. There are few established data handling practices in the field and several conceptual, technological, and practical challenges. Furthermore, many music researchers are not equipped for (or interested in) the craft of data storage, curation, and archiving. This paper discusses some of the particular challenges that empirical music researchers face when working towards Open Research practices: handling (1) (multi)media files, (2) privacy, and (3) copyright issues. These are exemplified through MusicLab, an event series focused on fostering openness in music research. It is argued that the "best practice" suggested by the FAIR principles is too demanding in many cases, but "good enough practice" may be within reach for many. A four-layer data handling "recipe" is suggested as concrete advice for achieving "good enough practice" in empirical music research.

• Laczko, Balint & Jensenius, Alexander Refsum (2021). Reflections on the Development of the Musical Gestures Toolbox for Python. In Kantan, Prithvi Ravi; Paisa, Razvan & Willemsen, Silvin (Ed.), Proceedings of the Nordic Sound and Music Computing Conference. Aalborg Universitetsforlag. Full text in Research Archive Show summary

  The paper presents the Musical Gestures Toolbox (MGT) for Python, a collection of modules targeted at researchers working with video recordings. The toolbox includes video visualization techniques such as creating motion videos, motion history images, and motiongrams. These visualizations allow for studying video recordings from different temporal and spatial perspectives. The toolbox also includes basic computer vision methods, and it is designed to integrate well with audio analysis toolboxes. The MGT was initially developed to analyze music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.

• Côté-Allard, Ulysse; Gagnon-Turcotte, Gabriel; Phinyomark, Angkoon; Glette, Kyrre Harald; Scheme, Erik & Laviolette, François (2021). A Transferable Adaptive Domain Adversarial Neural Network for Virtual Reality Augmented EMG-Based Gesture Recognition. IEEE transactions on neural systems and rehabilitation engineering. ISSN 1534-4320. doi: 10.1109/TNSRE.2021.3059741. Show summary

  Within the field of electromyography-based (EMG) gesture recognition, disparities exist between the offline accuracy reported in the literature and the real-time usability of a classifier. This gap mainly stems from two factors: 1) The absence of a controller, making the data collected dissimilar to actual control. 2) The difficulty of including the four main dynamic factors (gesture intensity, limb position, electrode shift, and transient changes in the signal), as including their permutations drastically increases the amount of data to be recorded. Contrarily, online datasets are limited to the exact EMG-based controller used to record them, necessitating the recording of a new dataset for each control method or variant to be tested. Consequently, this paper proposes a new type of dataset to serve as an intermediate between offline and online datasets, by recording the data using a real-time experimental protocol. The protocol, performed in virtual reality, includes the four main dynamic factors and uses an EMG-independent controller to guide movements. This EMG-independent feedback ensures that the user is in-the-loop during recording, while enabling the resulting dynamic dataset to be used as an EMG-based benchmark. The dataset is comprised of 20 able-bodied participants completing three to four sessions over a period of 14 to 21 days. The ability of the dynamic dataset to serve as a benchmark is leveraged to evaluate the impact of different recalibration techniques for long-term (across-day) gesture recognition, including a novel algorithm, named TADANN. TADANN consistently and significantly ( p<0.05 ) outperforms using fine-tuning as the recalibration technique.

• Sato, Fumiaki; Laeng, Bruno; Nakauchi, Shigeki & Minami, Tetsuto (2020). Cueing the Necker cube: Pupil dilation reflects the viewing-from-above constraint in bistable perception. Journal of Vision. ISSN 1534-7362. 20(4). doi: 10.1167/jov.20.4.7. Full text in Research Archive
• Côté-Allard, Ulysse; Gagnon-Turcotte, Gabriel; Phinyomark, Angkoon; Glette, Kyrre; Scheme, Erik & Laviolette, Francois (2020). Unsupervised Domain Adversarial Self-Calibration for Electromyography-Based Gesture Recognition. IEEE Access. ISSN 2169-3536. 8, p. 177941–177955. doi: 10.1109/ACCESS.2020.3027497. Full text in Research Archive Show summary

  Surface electromyography (sEMG) provides an intuitive and non-invasive interface from which to control machines. However, preserving the myoelectric control system’s performance over multiple days is challenging, due to the transient nature of the signals obtained with this recording technique. In practice, if the system is to remain usable, a time-consuming and periodic recalibration is necessary. In the case where the sEMG interface is employed every few days, the user might need to do this recalibration before every use. Thus, severely limiting the practicality of such a control method. Consequently, this paper proposes tackling the especially challenging task of unsupervised adaptation of sEMG signals, when multiple days have elapsed between each recording, by introducing Self-Calibrating Asynchronous Domain Adversarial Neural Network (SCADANN). SCADANN is compared with two state-of-the-art self-calibrating algorithms developed specifically for deep learning within the context of EMG-based gesture recognition and three state-of-the-art domain adversarial algorithms. The comparison is made both on an offline and a dynamic dataset (20 participants per dataset), using two different deep network architectures with two different input modalities (temporal-spatial descriptors and spectrograms). Overall, SCADANN is shown to substantially and systematically improves classification performances over no recalibration and obtains the highest average accuracy for all tested cases across all methods.

• Llorens, Anaïs; Funderud, Ingrid; Blenkmann, Alejandro Omar; Lubell, James; Foldal, Maja Dyhre & Leske, Sabine Liliana [Show all 11 contributors for this article] (2020). Preservation of Interference Effects in Working Memory After Orbitofrontal Damage. Frontiers in Human Neuroscience. ISSN 1662-5161. 13, p. 1–15. doi: 10.3389/fnhum.2019.00445. Full text in Research Archive Show summary

  Orbitofrontal cortex (OFC) is implicated in multiple cognitive processes, including inhibitory control, context memory, recency judgment, and choice behavior. Despite an emerging understanding of the role of OFC in memory and executive control, its necessity for core working memory (WM) operations remains undefined. Here, we assessed the impact of OFC damage on interference effects in WM using a Recent Probes task based on the Sternberg item-recognition task (1966). Subjects were asked to memorize a set of letters and then indicate whether a probe letter was presented in a particular set. Four conditions were created according to the forthcoming response ("yes"/"no") and the recency of the probe (presented in the previous trial set or not). We compared behavioral and electroencephalography (EEG) responses between healthy subjects (n = 14) and patients with bilateral OFC damage (n = 14). Both groups had the same recency pattern of slower reaction time (RT) when the probe was presented in the previous trial but not in the current one, reflecting the proactive interference (PI). The within-group electrophysiological results showed no condition difference during letter encoding and maintenance. In contrast, event-related potentials (ERPs) to probes showed distinct within-group condition effects, and condition by group effects. The response and recency effects for controls occurred within the same time window (300-500 ms after probe onset) and were observed in two distinct spatial groups including right centro-posterior and left frontal electrodes. Both clusters showed ERP differences elicited by the response effect, and one cluster was also sensitive to the recency manipulation. Condition differences for the OFC group involved two different clusters, encompassing only left hemisphere electrodes and occurring during two consecutive time windows (345-463 ms and 565-710 ms). Both clusters were sensitive to the response effect, but no recency effect was found despite the behavioral recency effect. Although the groups had different electrophysiological responses, the maintenance of letters in WM, the evaluation of the context of the probe, and the decision to accept or reject a probed letter were preserved in OFC patients. The results suggest that neural reorganization may contribute to intact recency judgment and response after OFC damage.

• Foldal, Maja Dyhre; Blenkmann, Alejandro Omar; Llorens, Anaïs; Knight, Robert T.; Solbakk, Anne-Kristin & Endestad, Tor (2020). The brain tracks auditory rhythm predictability independent of selective attention. Scientific Reports. ISSN 2045-2322. 10, p. 1–13. doi: 10.1038/s41598-020-64758-y. Full text in Research Archive Show summary

  The brain responds to violations of expected rhythms, due to extraction- and prediction of the temporal structure in auditory input. Yet, it is unknown how probability of rhythm violations affects the overall rhythm predictability. Another unresolved question is whether predictive processes are independent of attention processes. In this study, EEG was recorded while subjects listened to rhythmic sequences. Predictability was manipulated by changing the stimulus-onset-asynchrony (SOA deviants) for given tones in the rhythm. When SOA deviants were inserted rarely, predictability remained high, whereas predictability was lower with more frequent SOA deviants. Dichotic tone-presentation allowed for independent manipulation of attention, as specific tones of the rhythm were presented to separate ears. Attention was manipulated by instructing subjects to attend to tones in one ear only, while keeping the rhythmic structure of tones constant. The analyses of event-related potentials revealed an attenuated N1 for tones when rhythm predictability was high, while the N1 was enhanced by attention to tones. Bayesian statistics revealed no interaction between predictability and attention. A right-lateralization of attention effects, but not predictability effects, suggested potentially different cortical processes. This is the first study to show that probability of rhythm violation influences rhythm predictability, independent of attention.

• Kam, Julia W.Y.; Helfrich, Randolph H.; Solbakk, Anne-Kristin; Endestad, Tor; Larsson, Pål Gunnar & Lin, Jack J. [Show all 7 contributors for this article] (2020). Top-Down Attentional Modulation in Human Frontal Cortex: Differential Engagement during External and Internal Attention. Cerebral Cortex. ISSN 1047-3211. 31(2), p. 873–883. doi: 10.1093/cercor/bhaa262. Show summary

  Decades of electrophysiological research on top-down control converge on the role of the lateral frontal cortex in facilitating attention to behaviorally relevant external inputs. However, the involvement of frontal cortex in the top-down control of attention directed to the external versus internal environment remains poorly understood. To address this, we recorded intracranial electrocorticography while subjects directed their attention externally to tones and responded to infrequent target tones, or internally to their own thoughts while ignoring the tones. Our analyses focused on frontal and temporal cortices. We first computed the target effect, as indexed by the difference in high frequency activity (70-150 Hz) between target and standard tones. Importantly, we then compared the target effect between external and internal attention, reflecting a top-down attentional effect elicited by task demands, in each region of interest. Both frontal and temporal cortices showed target effects during external and internal attention, suggesting this effect is present irrespective of attention states. However, only the frontal cortex showed an enhanced target effect during external relative to internal attention. These findings provide electrophysiological evidence for top-down attentional modulation in the lateral frontal cortex, revealing preferential engagement with external attention.

• Sadaghiani, Sepideh; Dombert, Pascasie L.; Løvstad, Marianne; Funderud, Ingrid; Meling, Torstein Ragnar & Endestad, Tor [Show all 9 contributors for this article] (2019). Lesions to the fronto-parietal network impact alpha-band phase synchrony and cognitive control. Cerebral Cortex. ISSN 1047-3211. 29(10), p. 4143–4153. doi: 10.1093/cercor/bhy296.
• Suzuki, Yuta; Minami, Tetsuto; Laeng, Bruno & Nakauchi, Shigeki (2019). Colorful glares: Effects of colors on brightness illusions measured with pupillometry. Acta Psychologica. ISSN 0001-6918. 198, p. 1–10. doi: 10.1016/j.actpsy.2019.102882. Full text in Research Archive
• Blenkmann, Alejandro Omar; Collavini, Santiago; Lubell, James; Anais, Llorens; Funderud, Ingrid & Ivanovic, Jugoslav [Show all 13 contributors for this article] (2019). Auditory deviance detection in the human insula: An intracranial EEG study. Cortex. ISSN 0010-9452. 121, p. 189–200. doi: 10.1016/j.cortex.2019.09.002. Full text in Research Archive
• Kam, Julia W.Y.; Solbakk, Anne-Kristin; Endestad, Tor; Meling, Torstein Ragnar & Knight, Robert Thomas (2018). Lateral prefrontal cortex lesion impairs regulation of internally and externally directed attention. NeuroImage. ISSN 1053-8119. 175(15), p. 91–99. doi: 10.1016/j.neuroimage.2018.03.063.
• Johnson, Elizabeth L.; Adams, Jenna N.; Solbakk, Anne-Kristin; Endestad, Tor; Larsson, Pål Gunnar & Ivanovic, Jugoslav [Show all 9 contributors for this article] (2018). Dynamic frontotemporal systems process space and time in working memory. PLoS Biology. ISSN 1544-9173. 16(3). doi: 10.1371/journal.pbio.2004274. Full text in Research Archive
• Perry, Anat; Saunders, Samantha N.; Stiso, Jennifer; Dewar, Callum; Lubell, James & Meling, Torstein Ragnar [Show all 9 contributors for this article] (2017). Effects of prefrontal cortex damage on emotion understanding: EEG and behavioural evidence. Brain. ISSN 0006-8950. 140(4), p. 1086–1099. doi: 10.1093/brain/awx031.
• Johnson, Elizabeth L.; Dewar, Callum; Solbakk, Anne-Kristin; Endestad, Tor; Meling, Torstein Ragnar & Knight, Robert T (2017). Bidirectional Frontoparietal Oscillatory Systems Support Working Memory. Current Biology. ISSN 0960-9822. 27(12), p. 1829–1835. doi: 10.1016/j.cub.2017.05.046. Full text in Research Archive

View all works in Cristin

• Jensenius, Alexander Refsum (2022). Sound Actions: Conceptualizing Musical Instruments. MIT Press. ISBN 9780262544634. 304 p. Full text in Research Archive Show summary

  A techno-cognitive look at how new technologies are shaping the future of musicking. “Musicking” encapsulates both the making of and perception of music, so it includes both active and passive forms of musical engagement. But at its core, it is a relationship between actions and sounds, between human bodies and musical instruments. Viewing musicking through this lens and drawing on music cognition and music technology, Sound Actions proposes a model for understanding differences between traditional acoustic “sound makers” and new electro-acoustic “music makers.” What is a musical instrument? How do new technologies change how we perform and perceive music? What happens when composers build instruments, performers write code, perceivers become producers, and instruments play themselves? The answers to these pivotal questions entail a meeting point between interactive music technology and embodied music cognition, what author Alexander Refsum Jensenius calls “embodied music technology.” Moving between objective description and subjective narrative of his own musical experiences, Jensenius explores why music makes people move, how the human body can be used in musical interaction, and how new technologies allow for active musical experiences. The development of new music technologies, he demonstrates, has fundamentally changed how music is performed and perceived.

View all works in Cristin

• Jensenius, Alexander Refsum & Poutaraud, Joachim (2023). Video Visualization. Show summary

  This workshop is targeted at students and researchers working with video recordings. Even though the workshop will be based on quantitative tools, the aim is to provide solutions for qualitative research. This includes visualization techniques such as motion videos, motion history images, and motiongrams, which, in different ways, allow for looking at video recordings from different temporal and spatial perspectives. It also includes basic computer vision analysis modules, such as extracting quantity and centroid of motion, and using such features in analysis. The participants will learn to use the Musical Gestures Toolbox for Python, a collection of high-level modules for easily generating all of the above-mentioned visualizations and analyses. This toolbox was initially developed for analyzing music-related body motion but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, psychology, medicine, and educational sciences.

• Lesteberg, Mari & Jensenius, Alexander Refsum (2022). MICRO and MACRO - Developing New Accessible Musicking Technologies. Show summary

  This paper describes the development of two musical instrument prototypes developed to explore how non-haptic music technologies can be accessed from a web browser and how they can offer accessibility for people with low fine motor skills. Two approaches to browser-based motion capture were developed and tested during an iterative design process. This was followed by observational studies of two user groups: one with low fine motor skills and one with normal motor skills. Contrary to our expectations, we found that avoiding the use of buttons and mice did not make the apps more accessible for the participants with low fine motor skills. Furthermore, motion speed was considered more important for people with low motor skills than the size of the control action. The most important finding is that browser-based musical instruments using sensor-based and video-based motion tracking are not only feasible but allow for reaching much larger groups of people than previously possible. This may ultimately lead to both more personalized and accessible musical experiences.

• Blenkmann, Alejandro Omar; Solbakk, Anne-Kristin; Leske, Sabine Liliana; Llorens, Anaïs; Funderud, Ingrid & Collavini, Santiago [Show all 13 contributors for this article] (2022). Human brain network involved in auditory deviance detection. An intracranial EEG study.
• Asko, Olgerta; Blenkmann, Alejandro Omar; Leske, Sabine Liliana; Foldal, Maja Dyhre; Llorens, Anaïs & Funderud, Ingrid [Show all 10 contributors for this article] (2022). Altered hierarchical auditory predictive processing after lesions to the orbitofrontal cortex. Show summary

  In this study, we tested the causal involvement of the OFC in noticing breaches of predictions (i.e., PEs) at different hierarchical levels of task structural complexity. With this aim, we examined the event-related potentials (ERPs) of patients with focal OFC lesions and healthy adults while performing an auditory local-global oddball paradigm. Altogether, we found that after OFC damage, low-level PEs (i.e., processing of stimuli that are unpredicted at the local level) and combined low- and high-level PEs (i.e., processing of stimuli that are unpredicted at both the local and global level) were impacted. However, the processing of standard tones was not affected. We conclude that the OFC may contribute to a top-down process that modulates the deviance detection system in the primary auditory cortices, and may be involved in connecting PEs at lower hierarchical areas with predictions at higher areas. The study sheds new light on the poorly explored deficits of hierarchical auditory prediction in patients with damaged OFC.

• Fuhrer, Julian; Glette, Kyrre; Ivanovic, Jugoslav; Larsson, Pål Gunnar; Bekinschtein, Tristan & Kochen, Silvia [Show all 10 contributors for this article] (2022). Direct brain recordings reveal continuous encoding of structure in random stimuli.
• Solbakk, Anne-Kristin; Endestad, Tor & Knight, Robert Thomas (2022). Oslo - Berkeley collaboration in cognitive neuroscience and neuropsychology.
• Jensenius, Alexander Refsum (2022). Erfaringer med å lage 3xMOOC. Show summary

  I denne presentasjonen vil jeg presentere hvordan vi gjennom årene har utviklet tre komplette nettkurs ved Universitetet i Oslo: Music Moves (2016), Motion Capture (2022) og Pupillometry (2023). Fokuset vil ligge på muligheter og utfordringer i video i utdanningssammenheng.

• Jensenius, Alexander Refsum; Furmyr, Frida; Poutaraud, Joachim; Widmer, Marcus & Laczko, Balint (2022). The Musical Gestures Toolbox for Python. Show summary

  The Musical Gestures Toolbox for Python is a collection of tools for video visualization and video analysis.

• Upham, Finn; Zelechowska, Agata; Gonzalez, Victor & Jensenius, Alexander Refsum (2021). Quiet Breathing to Heard Music.
• Masu, Raul; Melbye, Adam Pultz; Sullivan, John & Jensenius, Alexander Refsum (2021). NIME Eco Wiki: A Crash Course. Show summary

  In this workshop, hosted by the three NIME environmental officers, participants will be introduced to the NIME Eco Wiki, a repository for addressing environmental and sustainability issues within the NIME community. During the workshop, the participants will discuss how practices on the communal as well as the individual level may become more sustainable and they will create new additions and ideas for the Wiki.

• Masu, Raul; Melbye, Adam Pultz; Sullivan, John & Jensenius, Alexander Refsum (2021). NIME and the Environment: Toward a More Sustainable NIME Practice. Show summary

  This paper addresses environmental issues around NIME research and practice. We discuss the formulation of an environmental statement for the conference as well as the initiation of a NIME Eco Wiki containing information on environmental concerns related to the creation of new musical instruments. We outline a number of these concerns and, by systematically reviewing the proceedings of all previous NIME conferences, identify a general lack of reflection on the environmental impact of the research undertaken. Finally, we propose a framework for addressing the making, testing, using, and disposal of NIMEs in the hope that sustainability may become a central concern to researchers.

• Jensenius, Alexander Refsum (2021). Visibility for researchers on university web pages. Show summary

  Academics need to be visible online. If you don’t publish and disseminate your research, it won’t have an impact. So it is in our own interest to have up-to-date personal pages with information about what we do. I would argue that it is also in the interest of universities that their employee’s personal pages are up-to-date and look good.

• Solbakk, Anne-Kristin; Leske, Sabine Liliana; Lubell, James Isaac; Blenkmann, Alejandro Omar; Llorens, Anais & Funderud, Ingrid [Show all 13 contributors for this article] (2021). Auditory prediction and prediction error in self-generated tones.
• Fasciani, Stefano & Jensenius, Alexander Refsum (2021). Sound and Music Computing at the University of Oslo. Show summary

  A presentation of labs and studios at the Department of Musicology and RITMO Centre for Interdisciplinary Studies in Rhythm, Time, and Motion at the University of Oslo.

• Laczko, Balint & Jensenius, Alexander Refsum (2021). Reflections on the Development of the Musical Gestures Toolbox for Python. Show summary

  The paper presents the Musical Gestures Toolbox (MGT) for Python, a collection of modules targeted at researchers working with video recordings. The toolbox includes video visualization techniques such as creating motion videos, motion history images, and motiongrams. These visualizations allow for studying video recordings from different temporal and spatial perspectives. The toolbox also includes basic computer vision methods, and it is designed to integrate well with audio analysis toolboxes. The MGT was initially developed to analyze music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.

• Tørresen, Jim (2020). Interdisciplinary Research Collaboration in Rhythm, Time and Motion.
• Jensenius, Alexander Refsum (2020). Video Visualization Strategies at RITMO.
• Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN.
• Jensenius, Alexander Refsum (2019). The RITMO Centre at University of Oslo .
• Jensenius, Alexander Refsum (2019). Tutorial: Musical Gestures Toolbox. Show summary

  An intensive PhD-level training course on sound and motion analysis with experts in sound and music computing from the Nordic countries.

• Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Intelligent, Adaptive Robots in Real-World Scenarios.
• Jensenius, Alexander Refsum (2019). Sound actions: An embodied approach to a digital organology. Show summary

  What is an instrument in our increasingly electrified world? In this talk I will present a set of theoretical building blocks from my forthcoming book on "musicking in an electronic world". At the core of the argument is the observation that the introduction of new music technologies has led to an increased separation between action and sound in musical performance. This has happened gradually, with pianos and organs being important early examples of instruments that introduced mechanical components between the performer and resonating objects. Today's network-based instruments represent an extreme case of a spatiotemporal dislocation between action and sound. They challenge our ideas of what an instrument can be, who can perform on them, and how they should be analyzed. In the lecture I will explain how we can use the concepts of action-sound couplings and mappings to structure our thinking about such instruments. This will be used at the heart of a new organology that embraces the qualities of both acoustic and electroacoustic instruments.

• Leske, Sabine Liliana; Lubell, James Isaac; Blenkmann, Alejandro Omar; Llorens, Anaïs; Funderud, Ingrid & Foldal, Maja Dyhre [Show all 13 contributors for this article] (2019). Auditory prediction and prediction error in self-generated tones.
• Blenkmann, Alejandro Omar; Solbakk, Anne-Kristin; Lubell, James; Leske, Sabine Liliana; Llorens, Anaïs & Funderud, Ingrid [Show all 14 contributors for this article] (2019). Human brain network involved in auditory deviance detection: Evidence from intracranial EEG recordings.
• Leske, Sabine Liliana; Lubell, James; Blenkmann, Alejandro Omar; Llorens, Anaïs; Funderud, Ingrid & Foldal, Maja Dyhre [Show all 13 contributors for this article] (2019). Action-based auditory predictions. Show summary

  Sensory consequences of actions are predicted by the brain via an internal forward model to prepare sensory cortical areas, referred to as motor prediction. In a similar vein, the predictive coding framework suggests that perception is based on internal models making predictions about sensory events, based on statistical probabilities of the stimuli. In the current study we investigated action-based sensory predictions. We used a self-paced, two-choice random generation task, infrequently inducing deviant outcomes of voluntary action. Participants repeatedly pressed a right and a left button normatively associated with a 70 ms long 1 kHz and 2 kHz tone, respectively. Occasional deviants occurred, inverting the learned button-tone association. Participants were instructed that their button presses should be random, at a regular but self-paced tempo of one press per 1-2 s, and that they should press both buttons with equal probability. They were informed that the tones are task-irrelevant. We used intracranial EEG (iEEG) data recorded from 10 adult patients with electrodes localized in frontal and temporal lobes. The patients had drug resistant epilepsy and were undergoing presurgical monitoring via implanted stereotactic electrodes. Electrode coordinates and anatomical labels were obtained from coregistered MRI and CT images using iElectrodes toolbox. Initial results indicate that violations of action intentions modulated high frequency band activity (HFA, 75-145 Hz) in distributed brain regions including temporal and prefrontal cortices.

• Blenkmann, Alejandro Omar (2019). Auditory deviance detection network in the human brain.
• Gonzalez Sanchez, Victor Evaristo (2019). Toward a Generic Measure of Fluency. Show summary

  Virtuosity in music performance is often associated with fast, precise, and efficient sound-producing movements. The generation of such highly skilled movements involves complex joint and muscle control by the central nervous system, and depends on the ability to anticipate, segment, and coarticulate motor elements, all within the biomechanical constraints of the human body. When successful, such motor skill should lead to what we characterize as fluency in musical performance. Detecting typical features of fluency could be very useful for technology-enhanced learning systems, assisting and supporting students during their individual practice sessions by giving feedback and helping them to adopt sustainable movement patterns. In this study, we propose to assess fluency in musical performance as the ability to smoothly and efficiently coordinate while accurately performing slow, transitionary, and rapid movements.

• Gonzalez Sanchez, Victor Evaristo (2019). MICRO: Human Bodily Micromotion in Music Perception and Interaction. Show summary

  This talk will highlight links between music and human movement, aiming at providing insight into crucial aspects of human perception, cognition, and sensorimotor systems. It will analyze responses to a wide range of music and sound features, exploiting concepts such as the groove, embodied music cognition, and entrainment. Victor will be glad to discuss potential implications of movement-analysis research for embodiment perspectives on technologically enabled conceptual learning.

• Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Adaptive Robot Body and Control for Real-World Environments.
• Gonzalez Sanchez, Victor Evaristo (2019). Characterizing Movement Fluency in Musical Performance. Show summary

  Virtuosity in music performance is often associated with fast, precise, and efficient sound-producing movements. The generation of such highly skilled movements involves complex joint and muscle control by the central nervous system, and depends on the ability to anticipate, segment, and coarticulate motor elements, all within the biomechanical constraints of the human body. When successful, such motor skill should lead to what we characterize as fluency in musical performance. Detecting typical features of fluency could be very useful for technology-enhanced learning systems, assisting and supporting students during their individual practice sessions by giving feedback and helping them to adopt sustainable movement patterns. In this study, we propose to assess fluency in musical performance as the ability to smoothly and efficiently coordinate while accurately performing slow, transitionary, and rapid movements.

• Westner, Britta; Andersen, Lau Møller; Blenkmann, Alejandro Omar & Leske, Sabine Liliana (2019). RITMO International EEG FieldTrip workshop, Lectures and Hands-on sessions .
• Blenkmann, Alejandro Omar (2019). Origins of the EEG signal and Technical setup of EEG.
• Funderud, Ingrid & Solbakk, Anne-Kristin (2019). Event Related Potentials and their functional relevance and Experimental design issues in ERP studies.
• Leske, Sabine Liliana (2019). Oscillations and their functional relevance and Experimental design issues in EEG studies investigating oscillations.
• Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Lessons Learned from Real-World Experiments with DyRET: the Dynamic Robot for Embodied Testing.
• Jensenius, Alexander Refsum (2019). Sound actions: An embodied approach to a digital organology. Show summary

  What is an instrument in our increasingly electrified world? In this talk I will present a set of theoretical building blocks from my forthcoming book on "musicking in an electronic world". At the core of the argument is the observation that the introduction of new music technologies has led to an increased separation between action and sound in musical performance. This has happened gradually, with pianos and organs being important early examples of instruments that introduced mechanical components between the performer and resonating objects. Today's network-based instruments represent an extreme case of a spatiotemporal dislocation between action and sound. They challenge our ideas of what an instrument can be, who can perform on them, and how they should be analyzed. In the lecture I will explain how we can use the concepts of action-sound couplings and mappings to structure our thinking about such instruments. This will be used at the heart of a new organology that embraces the qualities of both acoustic and electroacoustic instruments.

• Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing.
• Spiech, Connor (2019). Predicting the Groove: A Combined EEG-Pupillometry Study.
• Foldal, Maja Dyhre; Blenkmann, Alejandro Omar; Llorens, Anaïs; Knight, Robert Thomas; Solbakk, Anne-Kristin & Endestad, Tor (2019). The brain tracks global temporal regularity in auditory patterns.
• Spiech, Connor (2019). Predicting the Groove: A Combined EEG-Pupillometry Study.
• Blenkmann, Alejandro Omar; Solbakk, Anne-Kristin; Lubell, James; Leske, Sabine Liliana; Llorens, Anaïs & Funderud, Ingrid [Show all 14 contributors for this article] (2019). Human brain network involved in auditory deviance detection: Evidence from intracranial EEG recordings. Show summary

  The neural network underlying human auditory deviance detection is not fully understood. To address this, we recorded intracranial EEG from 22 adult patients with drug resistant epilepsy undergoing presurgical monitoring who had depth electrodes implanted in all brain lobes (1193 channels in total). Patients passively heard a stream of bilaterally presented tones while reading. We used the Optimum-1 paradigm, that consisted of 300 standard tones interleaved with 300 randomly presented deviant tones per block. Patients completed between 3 to 10 blocks. Deviant tones differed from standards in: 1) intensity (louder or softer), 2) frequency (higher or lower), 3) sound source location (right or left), 4) a shorter duration, or 5) a silent gap in the middle of the tone (Näätänen et al., 2004). Electrode coordinates were obtained from coregistered MRI and CT images using iElectrodes toolbox (Blenkmann et al., 2017). Channels were bipolar referenced and high frequency band activity (HFA) analytic amplitude signal was obtained using the Hilbert transform (75-145 Hz). Compared to the baseline period, significant HFA responses to tones in general were observed in 31% of the channels. We used an ANOVA to quantify the HFA response variance across trials that could be explained by the different factors of the stimuli: Intensity, Laterality, Frequency, Duration and Gap. We estimated the amount of explained variance by using ?2 (Siegel et al., 2015). Eighteen % of the channels showed a significant increase of the condition-specific explained variance. Some channels showed condition-specific activations to one particular deviant, while others showed activations to a combination of two or more deviants. The channels showing responses to tones in general and condition-specific effects were mostly observed bilaterally in temporal cortex. Frontal, anterior cingulate, and parietal cortices were also involved to a lesser extent. Our results, in line with the predictive coding framework, reveal that a distributed brain network is involved in auditory processing and deviance detection.

• Jensenius, Alexander Refsum (2019). Lecture-demo: Music-Related Micromotion. Show summary

  This presentation will summarize findings from my research into music-related micromotion. This includes the smallest human motion that we can perform and perceive, typically measured at at a scale of millimeters. We have carried out a series of studies of such micromotion, in which people have been asked to try to stand still on the floor, both in silence and with (musical) sound. By measuring their bodily responses with different types of motion tracking and physiological devices we find a number of similarities between people's quantity and quality of motion. This has been the starting point for exploring the use of micromotion in musical practice, what I call 'sonic microinteraction'. This includes standstill performances with interactive sound and light. It also includes several installations with our ensemble of self-playing guitars. These are hybrid instruments, using digital sound-production through acoustically resonating guitars. They are controlled through inverse microinteraction, meaning that you need to focus on standing still to produce any sound. This challenges our traditional understanding of the affordance of musical instruments, and opens for both artistically and scientifically interesting perspectives.

• Endestad, Tor; Solbakk, Anne-Kristin; Ivanovic, Jugoslav; Larsson, Pål Gunnar; Knight, Robert T. & Blenkmann, Alejandro Omar (2019). A robust intracranial electrode localization algorithm. Show summary

  The accurate localization of electrodes in relationship to the brain’s anatomy is the foundation of the spatial resolution of intracranial EEG recordings. However, in “difficult cases“ the localization needs to be done manually since automatic methods fails, e.g high density arrays up to 3 mm inter-electrode distance, overlapping electrodes, low resolution CT images, or connection cables ovelaying grids. Here, we present a new automatic method that models a flexible array of electrodes and fits it to the artifacts observed in post implantation CT images. We evaluated data from 18 adult patients with drug resistant epilepsy implanted with depth electrodes and/or subdural grids (18 patients, 3261 electrodes). The automatic method was contrasted against manual localization. The main processing steps (Fig. 1 A) were: Thresholding and selection of a cloud of CT voxels containing the electrode artifacts Assembling a model of the grid (depth) array of electrodes Fitting the model to a smooth surface (line) approximation of CT artifacts Fitting the model to the cloud of voxels by minimizing the energy function E = -Ec + Et + Ed Ec was the gaussian weighted spatial correlation between the electrodes and the cloud of voxels. Et penalized the translation of electrodes, and Ed the deformation of a spring grid connecting the electrodes. Automatic localization resulted to be more precise than manual selection, observed as a significant reduction of the inter-electrode distance variance (Fig. 1 B). We provide a robust method for intracranial electrode localization that is applicable to “difficult cases” were previous automatic methods fail (Fig. 1 C). The method was implemented in the open-source iElectrodes toolbox and is available to the research community (Blenkmann et al., 2017).

• Kam, Julia W.Y.; Solbakk, Anne-Kristin; Funderud, Ingrid; Endestad, Tor; Meling, Torstein Ragnar & Knight, Robert Thomas (2018). Orbitofrontal damage reduces auditory sensory response in humans. Cortex. ISSN 0010-9452. 101, p. 309–312. doi: 10.1016/j.cortex.2017.12.023.
• Tørresen, Jim & Laeng, Bruno (2018). UiO/ROBIN – Toyohashi University of Technology/Japan seminar . Show summary

  Seminar with collaborators from University of Technology, Japan: Shigeki Nakauchi <https://www.tut.ac.jp/english/schools/faculty/cs/169.html> og Tetsuto Minami <https://www.tut.ac.jp/english/schools/faculty/eiiris/573.html> and their students.

• Blenkmann, Alejandro Omar; Lubell, James; Llorens, Anais; Funderud, Ingrid; Collavini, Santiago & Larsson, PG [Show all 13 contributors for this article] (2018). Human insula response to auditory deviants: Evidence from intracranial EEG recordings.
• Jensenius, Alexander Refsum & Zhou, Bo (2018). Musical Gestures Toolbox for Matlab. Show summary

  The Musical Gestures Toolbox (MGT) is a Matlab toolbox for analysing music-related body motion, using sets of audio, video and motion capture data as source material.

• Kam, Julia W.Y.; Lin, Jack J.; Solbakk, Anne-Kristin; Endestad, Tor; Larsson, Pål Gunnar & Knight, Robert T. (2018). Functional coupling between default network and fronto-parietal control network supports internally directed attention. .
• Solbakk, Anne-Kristin; Endestad, Tor & Knight, Robert T. (2018). Neural oscillations and human behavior.
• Vestre, Eskil Olaf; Danielsen, Anne; Jensenius, Alexander Refsum; London, Justin; Schia, Katja Henriksen & Abramczyk, Filip (2018). Rytmen er en danser. [Business/trade/industry journal]. Ballade.
• Jensenius, Alexander Refsum; Danielsen, Anne; London, Justin; Schia, Katja Henriksen & Abramczyk, Filip (2018). Intellectual warm-up: "Rhythm".
• Suzuki, Yuta; Minami, Tetsuto; Laeng, Bruno & Nakauchi, Shigeki (2018). The differential effect of glowing appearance in the glare illusion: evidence from pupillometry.
• Sato, Fumiyaki; Laeng, Bruno; Nakauchi, Shigeki & Minami, Tetsuto (2018). Pupil dilation reflects "Viewing from above bias" in the effort to control perception.
• Kam, J.W.Y.; Lin, J.J.; Endestad, Tor; Solbakk, Anne-Kristin; Larsson, P.G. & Griffin, S. [Show all 7 contributors for this article] (2017). Contributions of medial prefrontal cortex to internally directed attention.
• Johnson, Elizabeth L.; Adams, J.N.; Griffin, S.M.; Solbakk, Anne-Kristin; Endestad, Tor & Larsson, P.G. [Show all 9 contributors for this article] (2017). Dynamic frontotemporal systems for episodic working memory.
• Johnson, Elizabeth L.; Dewar, C.; Solbakk, Anne-Kristin; Endestad, Tor; Meling, Torstein Ragnar & Knight, Robert T. (2017). Causal evidence that bidirectional frontoparietal rhythms support working memory.
• Slama, S.J.K.; Solbakk, Anne-Kristin; Endestad, Tor; Larsson, P.G.; Lin, Jack J. & King-Stephens, D. [Show all 9 contributors for this article] (2017). Intracranial recordings define a cortical-mesial temporal network in top-down and bottom-up attention.
• Sato, Fumiyaki; Laeng, Bruno; Nakauchi, Shigeki & Minami, Tetsuto (2017). Pupil dilation during perception of the Necker cube reflects the viewing-from-above bias.

View all works in Cristin