Balint Laczko

• Balint Laczko, Marie Elisabeth Rognes & Alexander Refsum Jensenius (2025). Image Sonification as Unsupervised Domain Transfer. I Angela McArthur, Emma-Kate Matthews & Tom Holberton (Red.), Proceedings of the 17th International Symposium on Computer Music Multidisciplinary Research. The Laboratory PRISM “Perception, Representations, Image, Sound, Music”. ISSN 9791097498061. s. 596–607. doi: 10.5281/zenodo.17497987. Fulltekst i vitenarkiv Vis sammendrag

  The process of image sonification maps visual features into perceived auditory features. Most established sonification methods rely on identifying salient visual features in the input data and then mapping their distribution to a proportional distribution of auditory features. However, this approach requires both domain expertise and manual feature engineering. Here, we propose a new method of image sonification, leveraging recent advances in representation learning and domain transfer. Our approach introduces a pair of variational auto-encoder models that learn disentangled latent representations of the images and sounds, respectively, and a separate network that maps between these representations. The resulting sonification system encodes images into the latent space and then decodes them as sounds. Both representations and their mapping are learned in an entirely unsupervised manner. When evaluating the system in an interactive real-time setting, we observed that the model successfully learned disentangled representations of image and sound factors in our synthetic datasets.

• Balint Laczko & Alexander Refsum Jensenius (2025). Pixasonics: An Image Sonification Toolbox for Python. I F. Amílcar Cardoso, Paul Vickers, Pedro Martins & Stephen Roddy (Red.), Proceedings of the 30th International Conference on Auditory Display (ICAD 2025). Department of Informatics Engineering, University of Coimbra, Portugal. ISSN 9798991456210. s. 28–35. doi: https:/hdl.handle.net/1853/79958. Fulltekst i vitenarkiv Vis sammendrag

  Pixasonics is a new Python library for interactive image analysis and exploration through image sonification. It uses real-time audio and visualization to help uncover patterns in image data. With Pixasonics, users can launch one or more small web applications (running in a Jupyter Notebook), probe image data using various feature extraction methods, and map those feature vectors to synthesis parameters. The target users are researchers interested in exploring image and volumetric data and creative users who want an intuitive tool for experimental sound design. Pixasonics’ design aims to strike a balance between an easy-to-use web application with minimal boilerplate code necessary and a library that can be integrated into more advanced workflows. Real-time exploration is at the heart, but it can also be used to script non-real-time sonifications of large datasets. This paper presents Pixasonics, its structure, interface, and advanced features, and discusses preliminary feedback from biology researchers and music technologists.

• Jinyue Guo, Anna-Maria Christodoulou, Balint Laczko & Kyrre Glette (2024). LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search. I Xiaodong Li & Julia Handl (Red.), GECCO '24 Companion: Proceedings of the Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery (ACM). ISSN 9798400704956. s. 667–670. doi: 10.1145/3638530.3654432. Fulltekst i vitenarkiv Vis sammendrag

  Evolutionary Algorithms and Generative Deep Learning have been two of the most powerful tools for sound generation tasks. However, they have limitations: Evolutionary Algorithms require complicated designs, posing challenges in control and achieving realistic sound generation. Generative Deep Learning models often copy from the dataset and lack creativity. In this paper, we propose LVNS-RAVE, a method to combine Evolutionary Algorithms and Generative Deep Learning to produce realistic and novel sounds. We use the RAVE model as the sound generator and the VGGish model as a novelty evaluator in the Latent Vector Novelty Search (LVNS) algorithm. The reported experiments show that the method can successfully generate diversified, novel audio samples under different mutation setups using different pre-trained RAVE models. The characteristics of the generation process can be easily controlled with the mutation parameters. The proposed algorithm can be a creative tool for sound artists and musicians.

• Balint Laczko & Alexander Refsum Jensenius (2024). Synth Maps: Mapping The Non-Proportional Relationships Between Synthesizer Parameters and Synthesized Sound. I Tim Ziemer, Prithvi Ravi Kantan, Samuel Chabot & Jonas Braasch (Red.), Proceedings of the 29th International Conference on Auditory Display (ICAD 2024). The International Community for Auditory Display. ISSN 9798991456203. s. 181–184. doi: 10.5281/zenodo.11237788. Fulltekst i vitenarkiv Vis sammendrag

  Parameter Mapping (PM) is probably the most used design approach in sonification. However, the relationship between a synthesizer’s input parameters and the perceptual distribution of its output sounds might not be proportional, limiting its ability to convey relationships within the source data in the sound. This study evaluates a basic Frequency Modulation (FM) synthesis module with perceptually motivated descriptors, measures of spectral energy distribution, and latent embeddings of pre-trained audio representation models. We demonstrate how these metrics do not indicate straightforward relationships between synthesis parameters and perceived sound. This is done using interactive audiovisual scatter plots—Synth Maps—that can be used to explore the sound distribution of the synthesizer and qualitatively evaluate how well the different representations align with human perception. Link to the code and the interactive Synth Maps are available.

• Balint Laczko (2024). The Hum: 3D Audiovisual Live Performance for Projection and Spatial Audio. I Tim Ziemer, Prithvi Ravi Kantan, Samuel Chabot & Jonas Braasch (Red.), Proceedings of the 29th International Conference on Auditory Display (ICAD 2024). The International Community for Auditory Display. ISSN 9798991456203. s. 201–203. Fulltekst i vitenarkiv Vis sammendrag

  The Hum is a scalable audiovisual live performance instrument. It is a complete world full of sounding objects that interact with each other. It is based on a fictional story. The Hum currently contains around five thousand musical phrases. The sound dataset was then analysed with a bespoke set of descriptors, and the resulting representation was projected into 3D space using UMAP. The live performance involves exploring this 3D space both in terms of real-time visuals, and in a spatial audio setup. At its core the performance is the sonification of the various timbral and musical relationships of the sound dataset, expressed by various interactions triggered between neighboring sounds or using the whole dataset. The interactions may evoke associations of weather (blowing wind, rain or chain lightning). The structure of the performance follows a 3-stage plan, that is realized via improvisation. Thus the duration of the performance is flexible and can last between 15 to 30 minutes. The Hum is realized entirely in Max/MSP.

• Balint Laczko & Alexander Refsum Jensenius (2021). Reflections on the Development of the Musical Gestures Toolbox for Python. I Prithvi Ravi Kantan, Razvan Paisa & Silvin Willemsen (Red.), Proceedings of the Nordic Sound and Music Computing Conference. Aalborg Universitetsforlag. Fulltekst i vitenarkiv Vis sammendrag

  The paper presents the Musical Gestures Toolbox (MGT) for Python, a collection of modules targeted at researchers working with video recordings. The toolbox includes video visualization techniques such as creating motion videos, motion history images, and motiongrams. These visualizations allow for studying video recordings from different temporal and spatial perspectives. The toolbox also includes basic computer vision methods, and it is designed to integrate well with audio analysis toolboxes. The MGT was initially developed to analyze music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.

Se alle arbeider i NVA

• Balint Laczko (2026). ICAD2026 Workshop on Pixasonics, an Image Sonification Toolbox for Python. Fulltekst i vitenarkiv Vis sammendrag

  This workshop provides a hands-on introduction to Pixasonics, a Python library for interactive image analysis and exploration through image sonification. Pixasonics strikes a balance between an easy-to-use graphical application and a flexible programming library. Its design centers on real-time interaction and auditory–visual feedback in a Jupyter Notebook environment. Participants will learn to design simple Parameter Mapping Sonification applications and advanced systems by implementing their own custom feature extraction, mapping, and audio synthesis algorithms. The workshop will also cover new functionality that has been added to Pixasonics since its initial publication, such as the use of instance masks, and new visualization and interaction features.This session is designed for researchers exploring volumetric data and creative practitioners interested in experimental sound design.

• Balint Laczko, Marie Elisabeth Rognes & Alexander Refsum Jensenius (2025). Poster for "Image Sonification as Unsupervised Domain Transfer". doi: 10.5281/zenodo.17513165. Fulltekst i vitenarkiv Vis sammendrag

  The process of image sonification maps visual features into perceived auditory features. Most established sonification methods rely on identifying salient visual features in the input data and then mapping their distribution to a proportional distribution of auditory features. However, this approach requires both domain expertise and manual feature engineering. Here, we propose a new method of image sonification, leveraging recent advances in representation learning and domain transfer. Our approach introduces a pair of variational auto-encoder models that learn disentangled latent representations of the images and sounds, respectively, and a separate network that maps between these representations. The resulting sonification system encodes images into the latent space and then decodes them as sounds. Both representations and their mapping are learned in an entirely unsupervised manner. When evaluating the system in an interactive real-time setting, we observed that the model successfully learned disentangled representations of image and sound factors in our synthetic datasets.

• Balint Laczko (2025). Presentation of PhD project: Perceptually Aligned Deep Image Sonification. doi: https:/comma.eecs.qmul.ac.uk/creative-audio-synthesis-and-interfaces-workshop/. Fulltekst i vitenarkiv Vis sammendrag

  Imaging technology has dramatically expanded our understanding of biological systems. This overabundance of images has come with unique problems, such as visual overload, which can potentially obscure data relationships and induce eye fatigue or divert vision from important tasks. Image sonification offers potential solutions to these problems by channeling data into the auditory domain, leveraging our natural pattern recognition skills through hearing. In my PhD project I have been exploring the potential of Machine Learning in solving the two fundamental challenges of image sonification: the perceptually aligned representations of images and sounds, and the cross-modal mapping between them. In this talk I will present my journey through timbre spaces, Differentiable DSP, and cross-modal domain transfer in search of new methods for image sonification.

• Balint Laczko (2025). C2HO Workshop on Image Sonification with Pixasonics. doi: https:/www.hf.uio.no/imv/english/research/networks/creative-computing-hub-oslo/pages/c2ho-workshops/image-sonification-workshop.html. Fulltekst i vitenarkiv Vis sammendrag

  How can we turn images into sound? And what can we learn about those images by listening? Why listen instead of looking? These are some questions Bálint Laczkó's research on image sonification aims to find answers to. Bálint has been researching applications of image sonification in biology and medical research and started working on a toolbox for Python, which he will be presenting during this workshop. The toolbox originates from bio-medical research, but it can also be used creatively for sound design. Bring your laptop with Python (or Anaconda) installed, and some images you'd like to squeeze some sounds out of!

• Balint Laczko (2025). Insight Through Sound: Image Sonification in Biology Research. doi: https:/www.uio.no/forskning/satsinger/livsvitenskap/livsvitenskapskonferansen/2025/sidearrangementer/workshop/insight-through-sound-image-sonification-in-biolog.html. Fulltekst i vitenarkiv
• Balint Laczko (2025). Oral presentation on ICAD 2025 about Pixasonics: An Image Sonification Toolbox for Python. Fulltekst i vitenarkiv Vis sammendrag

  Pixasonics is a new Python library for interactive image analysis and exploration through image sonification. It uses real-time audio and visualization to help uncover patterns in image data. With Pixasonics, users can launch one or more small web applications (running in a Jupyter Notebook), probe image data using various feature extraction methods, and map those feature vectors to synthesis parameters. The target users are researchers interested in exploring image and volumetric data and creative users who want an intuitive tool for experimental sound design. Pixasonics’ design aims to strike a balance between an easy-to-use web application with minimal boilerplate code necessary and a library that can be integrated into more advanced workflows. Real-time exploration is at the heart, but it can also be used to script non-real-time sonifications of large datasets. This paper presents Pixasonics, its structure, interface, and advanced features, and discusses preliminary feedback from biology researchers and music technologists.

• Alexander Refsum Jensenius & Balint Laczko (2024). Workshop notebook: Video Visualization. doi: 10.5281/zenodo.10480526. Fulltekst i vitenarkiv Vis sammendrag

  This is the notebook used in the Video Visualization workshop at the University of Oslo Library on 10 January 2024 during the Digital Scholarship Days. For a general introduction to the toolbox, check out the Jupyter Notebook tutorial and the Wiki . This workshop is targeted at students and researchers working with video recordings. You will learn to use MG Toolbox, a Python package with numerous tools for visualizing and analyzing video recordings. This includes visualization techniques such as motion videos, motion history images, and motiongrams; techniques that, in different ways, allow for looking at video recordings from different temporal and spatial perspectives. It also includes some basic computer vision analysis, such as extracting quantity and centroid of motion, and using such features in analysis.MG Toolbox for Python is a collection of high-level modules for generating all of the above-mentioned visualizations and analyses. This toolbox was initially developed to analyze music-related body motion but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, psychology, medicine, and educational sciences.

• Alexander Refsum Jensenius & Balint Laczko (2024). Video Visualization. Fulltekst i vitenarkiv Vis sammendrag

  This workshop is targeted at students and researchers working with video recordings. You will learn to use MG Toolbox, a Python package with numerous tools for visualizing and analyzing video recordings. This includes visualization techniques such as motion videos, motion history images, and motiongrams; techniques that, in different ways, allow for looking at video recordings from different temporal and spatial perspectives. It also includes some basic computer vision analysis, such as extracting quantity and centroid of motion, and using such features in analysis.MG Toolbox for Python is a collection of high-level modules for generating all of the above-mentioned visualizations and analyses. This toolbox was initially developed to analyze music-related body motion but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, psychology, medicine, and educational sciences.

• Balint Laczko & Alexander Refsum Jensenius (2024). Poster for "Synth Maps: Mapping The Non-Proportional Relationships Between Synthesizer Parameters and Synthesized Sound". Fulltekst i vitenarkiv Vis sammendrag

  Parameter Mapping (PM) is probably the most used design approach in sonification. However, the relationship between a synthesizer’s input parameters and the perceptual distribution of its output sounds might not be proportional, limiting its ability to convey relationships within the source data in the sound. This study evaluates a basic Frequency Modulation (FM) synthesis module with perceptually motivated descriptors, measures of spectral energy distribution, and latent embeddings of pre-trained audio representation models. We demonstrate how these metrics do not indicate straightforward relationships between synthesis parameters and perceived sound. This is done using interactive audiovisual scatter plots—Synth Maps—that can be used to explore the sound distribution of the synthesizer and qualitatively evaluate how well the different representations align with human perception. Link to the code and the interactive Synth Maps are available.

• Balint Laczko (2024). Two-part guest lecture about spatial audio and Ambisonics for MCT students. Fulltekst i vitenarkiv
• Balint Laczko (2023). Workshop on Spatial Audio and Ambisonics. Fulltekst i vitenarkiv
• Balint Laczko (2023). Online guest lecture about The Hum - a real-time 3D audiovisual performance in Max. Fulltekst i vitenarkiv
• Balint Laczko (2023). Two-part guest lecture about spatial audio and Ambisonics for MCT students. Fulltekst i vitenarkiv
• Balint Laczko (2023). Guest lecture about granular synthesis with onset detection in Max. Fulltekst i vitenarkiv
• Amani al Outa, Helene Knævelsrud, Balint Laczko & Alexander Refsum Jensenius (2022). Winner of RRI-inspired transdisciplinary side quest call. [Internet]. Centre for Digital Life Norway. Fulltekst i vitenarkiv Vis sammendrag

  Centre for Digital Life Norway (DLN) is excited to congratulate the team behind the project “The autophagic symphony – Unveiling the final rhythm” as winner of DLN’s RRI-inspired transdisciplinary side quest call.

• Balint Laczko & Alexander Refsum Jensenius (2021). Reflections on the Development of the Musical Gestures Toolbox for Python. Fulltekst i vitenarkiv Vis sammendrag

  The paper presents the Musical Gestures Toolbox (MGT) for Python, a collection of modules targeted at researchers working with video recordings. The toolbox includes video visualization techniques such as creating motion videos, motion history images, and motiongrams. These visualizations allow for studying video recordings from different temporal and spatial perspectives. The toolbox also includes basic computer vision methods, and it is designed to integrate well with audio analysis toolboxes. The MGT was initially developed to analyze music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.

• Balint Laczko, Alexander Refsum Jensenius & Marie Elisabeth Rognes (2026). Instrumental Image Sonification: Towards an Accessible and Reliable Method for Life Science and Sonic Design. Universitetet i Oslo. Fulltekst i vitenarkiv Vis sammendrag

  Biological image sonification—the transformation of data relationships in biological images into perceived auditory relationships—offers potential for improved medical diagnosis, immersive multimodal data exploration, new interfaces for musical expression, and educational innovation. However, trust in auditory displays is undermined by the non-proportional relationship between control parameters and perceived sound, limitations in reproducibility, and the inherent bias in human-handcrafted feature extraction. The first paper in this dissertation is a systematic review that establishes the state of the art, identifies Parameter Mapping Sonification as the dominant strategy, catalogs software tools, and traces the integration of machine learning. The second paper proposes Pixasonics as a standardized Python toolbox for image sonification, intended for both real-time and non-real-time applications. Furthermore, in the third paper, an interactive “Synth Maps” method demonstrates the non-proportionality between synthesis parameters and the auditory result, highlighting a critical limitation of Parameter Mapping Sonification and the inconsistent perceptual alignment of current audio representations. Finally, the fourth paper introduces the method “image sonification as unsupervised domain transfer” (ISUDoT), which leverages self-supervised representation learning to ensure proportionality in cross-modal domain transfer, thereby removing human bias in feature engineering. Collectively, these contributions advance image sonification as a reliable scientific instrument for life science researchers and an inspiring sound design tool for everyone else, accessible to both experts and non-experts.

Se alle arbeider i NVA