Guest Talks

Speaker: Frederick Gallun
Title of the Talk: “Does Psychoacoustics Have to be Boring? Exploring Gamification of Auditory Testing”
Affiliation: Oregon Health and Science University, Portland, USA
Abstract: The study of the perceptual ability to hear and interpret sound (“psychoacoustics”) has traditionally involved the presentation of large numbers of very similar stimuli to which the listener makes only one or two types of simple responses, usually a button press. When the participant is attentive and properly motivated, these methods can result in reliable and repeatable data from which strong conclusions can often be drawn. However, the wandering of attention and the flagging of attention is as potentially detrimental to the experiment as damage to the equipment or errors in programming or design of the experiment. To address these issues, which are especially common in patient populations, we have been exploring introducing game-like elements into the experiments. This “gamification” has the potential to lead to more sustained attention and higher motivation, potentially improving the quality of the data, or at least the willingness of participants to perform the task. One potential downside of gamification is that it might distract from the actual listening task, leading to worse performance or increased variability. These effects may potentially be different for different participants, depending on factors such as age, cognitive ability, and health status. Two experiments will be described in which gamification has been implemented in a test of spatial release from masking (SRM) and results will be compared to a more traditional testing approach. Findings suggest that gamification can produce similar results to traditional tests, and that some participants can actually enjoy gamified psychoacoustical testing.

Speaker: Z. Ellen Peng
Co-authors: Kristen Janky, Mackenzie Reagan, Jessie Patterson, Chris Stecker
Title of the Talk: " The role of postural stability on functional spatial hearing in children with typical and atypical development of hearing and vestibular functions"
Affiliation: Boys Town National Research Hospital, Omaha, Nebraska, USA
Abstract: Children spend the first decade in life learning to process sensory inputs to make sense of the world around them. Until early adolescence, physical maturation alters auditory spatial cues which likely require children to update their auditory spatial mapping using cross-modal sensory inputs, such as from the visual and/or vestibular systems. This may present additional challenges for children with early onset of deafness and vestibular dysfunction to gain access to functional spatial hearing. In this work, we examine the role of postural stability on spatial hearing development between 10 adolescents (12-14 years old) and 21 young adults with normal hearing (NH), and between two groups of bilateral cochlear implant (CI) users with perilingual onset of deafness with (n = 7) or without (n = 11) additional vestibular dysfunction. Vestibular function was assessed using a standard clinical battery including rotary chair. Functional spatial hearing was measured using localization, minimum audible angle, and spatial unmasking under three balance conditions: seated, standing on hard wood floor, and standing on foam for increasing demand of postural control. Preliminary results suggest some resilience in changing postural demand across spatial hearing tasks in NH children and even among bilateral CI users.

Speaker: Eleni Vlahou 
Co-authors:
Title of the Talk: “Exploring Spatial Auditory Attention in Gamified Cocktail Party Settings”
Affiliation: University of Thessaly, Greece
Abstract:

Speaker: Virginia Best
Co-authors:
Title of the Talk: “When and why do hearing aids disrupt sound externalization?”
Affiliation: Boston University, Boston, USA
Abstract: When listening through hearing aids, the natural sound path to the ear canals is disrupted, with consequences for the spatial perception of sounds in the environment. We showed recently that certain styles of hearing aid disrupt sound externalization, leading to sound images that are too close or even inside the head (Best and Roverud, 2024). In this talk I will briefly overview a number of ongoing studies that are aiming to investigate this phenomenon further, to better understand both the causes and the consequences. Given the theme of this workshop, I hope to discuss implications of this work for virtual and augmented reality in listeners with normal hearing and with hearing loss.   

Speaker: Bernhard Laback
Co-authors: Jonas Schültke
Title of the Talk: "Dilation of Auditory Space by Short-Term Context"
Affiliation: Acoustics Research Institute, Austrian Academy of Sciences,Vienna, Austria
Abstract: The perceived azimuth of a sound source is biased by a preceding source (precursor), typically, towards midline (medial) by a lateral and towards the side (lateral) by a central precursor. Little is known about effects of intermediate precursor azimuths. By testing a variety of precursor and target azimuths, we studied three hypotheses regarding the precursor effect: a) binaural cue adaptation, predicting medial bias for any combination of lateral precursor and target and lateral bias for medial precursor, b) repulsion, predicting bias generally away from the precursor, regardless of spatial configuration, and c) local spatial contrast enhancement, predicting dilation of auditory space around precursor azimuth. Ten normal-hearing listeners localized 300-ms targets following 600-ms precursors using a head-pointing task in a virtual audio-visual environment. Both target and precursor azimuths were systematically varied across the azimuth range from left (-90°) to right (+90°). Stimuli were white noises, filtered with listener-specific head-related transfer functions. The data showed little evidence for hypothesis a, partial support for hypothesis b, and clear support for hypothesis c. The results are, thus, consistent with the idea that the auditory system enhances spatial contrast around the azimuth of a preceding sound and compresses contrast at remote azimuths.

Speaker: Anja Pahor
Title of the Talk: “Development and validation of mobile measures of executive function”
Affiliation: University of Maribor, Slovenia
Northeastern University, College of Science, Department of Psychology, Boston, USA
Abstract: Executive functions are a set of mental skills that help us plan, monitor, and successfully execute our goals. These mental skills encompass various abilities such as working memory, attention, cognitive flexibility, and inhibitory control, all of which are essential for navigating daily tasks and achieving long-term objectives. Recent studies have underscored the significance of executive functions in facilitating effective communication, particularly in challenging auditory environments characterized by background noise. Individuals with robust executive functioning demonstrate enhanced capacity to filter out irrelevant information, maintain focus on relevant cues, and adaptively allocate cognitive resources to comprehend speech amidst noise interference. Such findings not only emphasize the centrality of executive functions in speech processing but also underscore their broader implications for real-world communication scenarios. In response to the growing recognition of the critical role played by executive functions in daily life, the Brain Game Center has developed a comprehensive suite of configurable mobile tools aimed at assessing various facets of cognitive control. Through a combination of standardized tasks and adaptive algorithms, these tools offer a nuanced understanding of an individual's cognitive profile, enabling tailored measurement and targeted cognitive training programs. I will provide an overview of the diverse range of cognitive assessments developed by the Brain Game Center, and I will present empirical evidence regarding the reliability and validity of these assessments, where available.

Speaker: Jyrki Ahveninen
Co-authors:
Title of the Talk: “Multimodal studies on attention and crossmodal influences in auditory networks”
Affiliation: Mass General Research Institute / Harvard Medical School, Boston, USA
Abstract:

Speaker: G. Christopher Stecker
Co-authors:
Title of the Talk: “The roles and bases of auditory spatial awareness"
Affiliation: Boys Town National Research Hospital, Omaha, Nebraska, USA
Abstract:

Speaker: Mathieu Lavandier 1
Co-authors: Jim Kates 2
Title of the Talk: “Modelling speech intelligibility in noise: from differences in SRTs to full psychometric functions?”
Affiliation: 1 ENTPE – University of Lyon, France
2 University of Colorado, Boulder, USA
Abstract: This presentation will deal with predicting speech intelligibility in noise, and in particular the influence of having two ears to understand speech in noise and the prediction of spatial release from masking. After an introduction presenting the issue of predicting full psychometric functions rather than only speech reception thresholds (a particular point of these functions), a modelling study will be detailed, which aimed at making binaural a monaural model predicting psychometric functions: HASPI, for hearing aid speech perception index. This model, along with a binaural front-end developed to be combined with monaural HASPI, has been tested on three datasets from Andersen et al. (2018), which involved spatial release from masking measured in anechoic for normal-hearing listeners in the presence of different types of maskers (stationary, non-stationary, isotropic), with also non-linear processing sometimes involved (ideal binary mask processing simulating noise suppression in hearing aids).

Speaker: Kaisu Lankinen
Co-authors:
Title of the Talk: “Using 7T fMRI to examine functional properties of the human auditory pathways”
Affiliation: Mass General Research Institute / Harvard Medical School, Boston, USA
Abstract: High-resolution 7 Tesla functional and anatomical magnetic resonance imaging (MRI) have enabled new ways to investigate the human brain. Here, we demonstrate through our recent studies some opportunities that 7 T fMRI can bring to auditory cognitive research. We illustrate how the superior spatial resolution facilitates precise localization of articulatory motor areas linked to speech perception, and enables individual-level cortical parcellation of human auditory cortex. Importantly, beyond enhanced spatial resolution, 7 T fMRI enables the capture of layer-specific functional signals within the gray matter. This capability allows for revealing the information flow between brain regions through feedforward (FF) and feedback (FB) connections, and thus can inform us of the hierarchical organization of the brain. We illustrate the FF and FB type activation patterns in auditory cortex in context of cross-sensory activity. Furthermore, we present methodological advancements in layer-specific connectivity analysis, which promise to yield neuroscientifically relevant insights. These innovations hold the potential to unveil intricate neural networks and their functional implications.

Speaker: Lauren Calandruccio
Co-authors:
Title of the Talk: “Listening in noise in multiple languages: considerations for enhancing the efficiency of speech recognition testing”
Affiliation: Case Western Reserve University, USA
Abstract: In the United States, almost all audiology research exploring speech recognition is conducted on native-speakers of English. Historically, this has been done in part to ensure a more homogeneous listener sample to more accurately understand and interpret the data. However, there are hundreds of languages spoken in the US and there are tens of millions of people who speak a language or languages other than English. This has left US audiologists without strong evidence nor appropriate tools to assess and care for their multilingual patients. In this presentation, data exploring masked English-speech recognition for people that speak English as their second language, as well as people that are simultaneous bilinguals will be presented. How people that speak English as their second language are able to recognize spectrally sparse English speech, as well as how these same listeners can integrate spectral information across frequency bands will be presented. The effect of masker modulation on the recognition of English speech for people that speak English as a second language will also be discussed. Lastly, a new mentoring program designed specifically to enhance the diversity of the US’s National Institute on Deafness and Other Communication Disorders’ extramural workforce will be highlighted.

Speaker: Piotr Majdak 1
Co-authors: Roberto Barumerli 1,2
Title of the Talk: "Introduction to the framework for auditory modeling based on Bayesian inference”
Affiliation: 1 Acoustics Research Institute, Austrian Academy of Sciences, Vienna, Austria
2 Department of Neuroscience, Biomedicine, and Movement, University of Verona, Italy
Abstract: When listeners interact with the environment, their hearing system plays a significant role in delivering spatial information to higher circuits in the brain. Several computational models targeting specific components of that spatial-hearing process have been proposed, however, their implementation often relies on specific computational solutions. This impairs the understanding of model details and complicates the reproducibility of the results, thus, limiting the code reusability and scientific progress. In our talk, we will describe a software framework for auditory models based on Bayesian inference (FrAMBI). FrAMBI uses Bayesian inference as a theoretical tool grounded in probability theory and applies it to model of auditory perception, considering the extraction of perceptually relevant auditory features, the integration of prior beliefs, and mechanisms for the belief update. Integrated in the Auditory Modeling Toolbox, FrAMBI provides flexible model templates and functions estimating free parameters, aiming to help hearing scientists to develop new models and run model-based analyses.

Speaker: Robert Baumgartner 
Co-authors:
Title of the Talk: “Short-term adaptation of spatial hearing”
Affiliation: Austrian Academy of Sciences, Vienna, Austria
Abstract: Our perception of auditory space relies on beliefs about the hidden states of the acoustic environment, which undergo continuous updates to mitigate uncertainty and adapt to potential environmental changes. In this presentation, I will discuss two studies exploring this perceptual adaptability across different spatial dimensions. In the horizontal plane, participants were tasked with localizing stochastic sound sequences subject to change points. We demonstrate that participants' localization responses, as well as ongoing electroencephalographic and pupillometric measures, reflect continuous belief updating using Bayesian inference—incorporating new evidence to revise or adjust existing beliefs in a probabilistically optimal manner. In the vertical plane, we investigated whether listeners establish beliefs about the source spectrum to resolve interference with monaural spectral-shape cues for sound localization. We found that listeners enhance their localization performance by adapting to consistent source spectra over the course of multiple sound bursts. Together, these studies highlight the adaptability of both spatial and non-spatial beliefs in sound localization.

Speaker: Aaron Seitz
Co-authors:
Title of the Talk: “New games to train speech in competition; from psychoacoustics to music”
Affiliation: Northeastern University, Boston, USA
Abstract:

Speaker: Antje Ihlefeld
Co-authors:
Title of the Talk: “Spatial Audio and fNIRS”
Affiliation: Meta, USA
Abstract:

Speaker: Norbert Kopčo 1
Co-authors: René Šebeňa 1,  Jyrki Ahveninen 2, Virginia Best 3
Title of the Talk: “The reference frame of the electrophysiological correlates of attentional cueing in auditory spatial discrimination”
Affiliation: 1 Institute of Computer Science, P. J. Šafárik University in Košice, Slovakia
2 Mass General Research Institute / Harvard Medical School, Boston, MA, USA
3 Boston University, Boston, MA, USA
Abstract: A behavioral experiment and EEG recordings were combined to examine how directing automatic auditory spatial attention affects performance in a location discrimination task while the eyes fixate a neutral location, and whether the effect is cue modality dependent. Analyses on behavioral data found 1) better performance with auditory valid vs. invalid cues, 2) no difference in performance with visual valid vs. invalid cues; and 3) bias toward perceiving the sounds as moving away from the fixation direction, the strength of which depended on cue modality and validity. EEG analyses revealed that the late components of the target-elicited ERPs covaried with the behavioral performance. The analysis of cue-elicited ERPs examined the auditory-evoked occipital response contralateral to an auditory cue (ACOP), previously reported as a correlate of attentional processing. When cue location was described in head-centered reference frame, a complex interaction of correctness and cue-position was found in the ACOP interval of 300-400 ms. When eye-centered reference frame was used, contralateral positivity was observed, significantly higher trials with incorrect than correct responses. These results demonstrate that both eye fixation and auditory cue location affect auditory spatial discrimination, while the visual cue has only a small effect. However, the auditory cue effect is more a distracting one when the cue is invalid than an enhancing one when the cue is valid, and the cue-related ERP predicting accuracy of response appears to be encoded in eye-centered reference frame.

[Work supported by Horizon Europe HORIZON-MSCA-2022-SE-01 grant N°101129903 and VEGA 1/0350/22]

Speaker:  Ľuboš Hládek 1,2
Co-authors: Aaron R. Seitz 3, Norbert Kopčo 1
Title of the Talk: “Learning auditory distance perception over multiple days: Role of sound level and reverberation-related cues”
Affiliation: 1 Institute of Computer Science, P. J. Šafárik University in Košice, Slovakia
2 Acoustic Research Institute, Austrian Academy of Sciences, Vienna, Austria
3 Northeastern University, Boston, MA, USA
Abstract: The human auditory system must often adapt to acoustics of places we visit or inhabit. Here, we hypothesize that spontaneous learning of room acoustics takes place over multiple days and leads to improvements in auditory distance perception when listeners are engaged in a distance localization task. We used two training regimes to probe the role of individual distance cues when room learning exhibits itself in distance perception. The level of the targets was either roved by ±6 dB on a trial-by-trial basis (R) or was fixed (F) within eighty-trial blocks. We expected improvements in R condition and transfer into the F condition after R training but not after F training as, in F condition, the level cue provides sufficient information and people can ignore the room-related cues. We measured room learning in testing blocks at the beginning and the end of training. Participants localized broadband 300-ms-long noise samples in training and testing sessions spread across multiple days in a small reverberant room with real acoustic targets at 0.69 m to 2.03 m in front the listener. After R training, rank correlation coefficients (CC) increased in R but not in F testing. After F training, the CC increased unexpectedly in both F and R testing. The improvements happened mainly between the days. Thus, the consistency of perceptual cues and memory consolidation are important factors for learning the mapping from distance cues to percepts in new environments.

[Work supported by Horizon Europe HORIZON-MSCA-2022-SE-01 grant N°101129903 and VEGA 1/0350/22; Recipient of a Seal of Excellence Postdoctoral Fellowship of the Austrian Academy of Sciences]

Contributed Talks

Speaker: Keerthi Doreswamy 1,2
Co-authors: Jyrki Ahveninen 2 , Samantha Huang 2 , Stephanie Rossi 2 , Norbert Kopco 1
Title of the Talk: "Auditory distance perceptions and cues encoding in the human auditory cortex"
Affiliation: 1 Institute of Computer Science, P. J. Safarik University, Kosice, 04001, Slovakia
2 Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Harvard Medical School/Massachusetts General Hospital, Charlestown, MA, 02129, USA
Abstract: Perceiving the sound source distance is important in many everyday activities. For sources near the listener, two dominant intensity-independent cues are available, the direct-to-reverberant energy ratio (DRR) and the interaural level difference (ILD). Previous studies identified the planum temporale (PT) and superior temporal gyrus (STG) as auditory cortical areas important for distance processing. However, it is not clear whether the identified areas represent the distance percept per se or the cues. To address this, we conducted behavioral and neuroimaging experiments in a virtual reverberant environment using broadband noise stimuli. The availability and congruency of the DRR and ILD cues were manipulated to identify cortical areas encoding distance cues vs. distance percepts. Behavioral results showed that distance percepts were stronger when both cues were available and congruent, confirming that both cues are used when listeners judge distance. A univariate fMRI analysis identified areas in the PT+STG as encoding the DRR cue. An ROI-based multi-voxel pattern analysis (MVPA) over whole PT+STG region found a significant difference between congruent and incongruent stimuli, likely representing the distance percept. Thus, the PT+STG region encodes both the distance cues and percepts. However, while the cue encoding is non-distributed, the percepts are encoded in a distributed network.

[Work supported by Horizon Europe HORIZON-MSCA-2022-SE-01 grant N°101129903; VEGA 1/0350/22 and APVV DS-FR-19-0025 to NK & NIDCD grants R01DC017991, R01DC016765, R01DC016915 to JA]

Author:  Nancy Sotero Silva 1  
Co-authors: Felix Bröhl 2, Christoph Kayser 1 
Title: "Comparing Eye movement-related eardrum oscillations induced by different sensory targets"
Affiliations: 1 Bielefeld University, Faculty of Biology, Bielefeld, Germany
2 Max Planck Institute for Human Development, Berlin, Germany
Abstract: Spatial information is provided by different sensory inputs, and these may interact already in peripheral structures of the nervous system. One example of such interactions between vision and hearing are low-frequency displacements of the tympanic membrane induced by saccadic eye movements towards visual targets - named Eye movement-related eardrum oscillations (EMREOs). EMREOs were shown to scale in a parametric manner with the amplitude and direction of visually-guided saccades. We asked whether EMREOs are induced and related to saccades in the same manner regardless of which sensory modality guides these saccades. We presented both visual and auditory targets in positions raging from -9° to 9° horizontally and compared the characteristics of EMREOs induced by those stimuli. Eardrums oscillations were recorded via in-ear microphones and eye movements were recorded using infrared eye tracking. Our results suggest that the amplitude and direction of saccades determine the EMREOs contour and timing independent of the modality of the saccade-inducing stimuli, suggesting that EMREOs are more linked to motor planning or execution than sensory signals.

Speaker: Sreeram Kaithali Narayanan 1,2
Co-authors: Axel Ahrens 1, Filip Rønne 3, Virginia Best 4, Torsten Dau 1, Tobias Neher 2
Title of the Talk: "Effects of Hearing Loss and Hearing Aid Directionality on Search Behavior in Complex Audio-Visual Environments"
Affiliation: 1 Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Kgs. Lyngby, DK
2 Department of Clinical Research, University of Southern Denmark, Odense, DK
3 WS Audiology, Lynge, DK
4 Department of Speech, Language and Hearing Sciences, Boston University, Boston, MA, USA   
Abstract: Listeners with hearing loss often struggle to understand speech in the presence of multiple competing speakers. These difficulties become more pronounced with an increase in the number of speakers and the presence of reverberation. Here, we investigated how both normal-hearing and hearing-impaired listeners localize and identify a specific speaker in reverberant audio-visual scenes. The audio-visual scenes were presented using a 64-channel loudspeaker array and virtual-reality glasses. These scenes varied in terms of the target speaker’s position, the number of competing speakers, and the spatial separation between them. The participants were tested in different conditions: without assistance, with omnidirectional hearing-aid settings, and with directional settings. Amplification was based on NAL-NL2 target gains to ensure audibility compensation. The outcome measures included accuracy in speaker localization, response time, and the trajectories of head movements and eye-gazes during task completion. Currently, data collection and analysis are ongoing. We anticipate that this study will shed light on strategies used by listeners to identify target speech in complex listening environments and on how hearing loss and directional processing impact their behavior.

Speaker: Michaela Svoboda 1 2
Co-authors: Kateřina Chládková 1 2 3
Title of the Talk: "Crossmodal activation in preschool children with hearing prostheses: an fNIRS study"
Affiliation: 1 Institute of Psychology, Czech Academy of Sciences, Czech Republic 
2 Faculty of Arts, Charles University, Czech Republic
3 Faculty of Medicine in Hradec Králové, Charles University, Czech Republic 
Abstract: In our contribution, we introduce a neuroimaging experiment that assesses the degree of language-related cross-modal activation in hearing-impaired (HI) children fitted with hearing prosthesis. The term "cross-modal activation" refers to a state in which brain structures primarily associated with the processing of stimuli in one modality are involved in the processing of stimuli in different modalities, often as a result of sensory deprivation. Cross-modal activation and neural reorganisation have been studied for decades in people with HI, but its influence on the acquisition of spoken language is not yet understood (Giraud & Lee, 2007, Zhou et al. 2015). In the present experiment, cross-modal reorganization is measured in preschool children during exposure to speech with restricted visual cues, silent speech, and sign language. Functional near-infrared spectroscopy (fNIRS) is used to measure neural activation bilaterally in the superior temporal gyrus and in the visual association areas in the temporal lobe. We hypothesize that the degree of crossmodal activation in response to visually-cued language will be positively correlated with the children's language outcomes. The findings will be related to the ongoing discussion on the benefits of early sign language input in young HI children. In April, the experiment will be at the piloting stage and we are open for discussion regarding the experiment design and data analyses.

Speaker: Martina Dvořáková 1,2,3
Title of the Talk: "Newborns' neural tracking of infant-directed and adult-directed speech in native and foreign language."
Affiliation: 1 Institute of Psychology, Czech Academy of Sciences, Czech Republic
2 Institute of Czech Language and Theory of Communication, Charles University, Czech Republic 
3 Faculty of Medicine in Hradec Kralove, Charles University, Czech Republic
Abstract: At birth, the human brain is tuned to spoken language in general and to some extent also to native language in particular. In behavioral studies, infants also prefer to listen to infant-directed speech (IDS) to adults-directed speech (ADS), apparently most robustly in their native language. Recent studies demonstrated that this preference has correlates at the neural level as well. We test whether newborns show differential neural tracking of native over foreign, rhythmically different language. We assess neural tracking of native and non-native speech in Czech-exposed newborns. Newborns' were played a children's story in two rhythmically different languages, Czech (lacking acoustic cues to word-level stress) and Russian (acoustically salient word-level stress), in IDS or ADS, while their EEG was recorded. We predicted stronger neural tracking of the native Czech, evident in larger inter-trial phase coherence (ITC), and total power. Preliminary data (n = 27 out of planned 60) suggest this language-specific effect is most prominent in the theta band corresponding to the syllable rate. We will further test whether this native-language effect would be more prominent in ADS or IDS. Data collection is underway and preliminary results will be presented & discussed at the conference.

Speaker: Rudolf Kirchner
Co-authors: Carrick Davis, Winnie Yeh, Jeremy I. Skipper
Title of the Talk: "The Brain’s Silent Bet: Naturalistic fMRI Study of Non-Verbal Emotional Cues in Word Prediction"
Affiliation: University College London, Faculty of Brain Sciences, Language, Action and Brain Lab. London, United Kingdom
Abstract: Non-verbal cues, including facial expressions and gestures, are essential for signalling predictions about forthcoming verbal information, yet the neural mechanism underlying this process is not well understood. Here, we recruited seven participants (four male, three female, Mage = 21.00, SDage = 0.82) from University College London to undergo a functional magnetic resonance imaging scan while watching naturalistic video stimuli of two actresses having a conversation. At the onset of each video, a non-verbal emotional cue (e.g. a smile, a frown) or a neutral facial expression was presented, followed by an emotionally-valenced or neutral verbal content. Watching videos where an emotional cue preceded emotional verbal content elicited significantly lower levels of activity in the frontal, temporal, parietal, and motor regions than videos without the preceding cue. In addition, processing emotional language, compared to neutral language, showed increases in whole-brain activity. These findings indicate that non-verbal cues free up neural resources during language processing and highlight the widespread organisation of multi-modal language in the brain. These insights should be integrated into models of communication disorders such as developmental language disorder, aphasia, and autism spectrum disorder.

Speaker: Natálie Kikoťová 1 2
Co-authors: Kateřina Chládková 1 2
Title of the Talk: "Neural oscillatory and ERP indices of prediction in emotional speech"
Affiliation: 1 Institute of Psychology, Czech Academy of Sciences, Czech Republic 
2 Faculty of Arts, Charles University, Czech Republic
Abstract: The present study investigated the neural underpinnings of predictive processing of angry and neutral speech. We presented participants (N = 26) with recordings of angry and neutral conversation segments, as well as speech-shaped noise, while their EEG was recorded. Oscillatory power in the gamma frequency band (30–80 Hz) and the N400 component of event-related potentials (ERP) to sentence-final words were analyzed. The amplitude of the N400 was significantly reduced by negative emotional valence in contrast to neutral valence. Furthermore, we observed larger gamma power when participants listened to angry speech in comparison to neutral speech. The results indicate increased prediction and facilitated semantic analysis in negative as compared to neutral speech. To date, studies have only reported the predictability effects of gamma power in association with the semantic-lexical content of words. The present results provide evidence that gamma band power is also influenced by the emotional content of speech.

Contributed Posters

Author: Péter Nagy
Co-authors: Luca Béres, Ádám Boncz, István Winkler
Title: "Movement Synchrony and Gaze Coordination during Face-to-Face Communication"
Affiliation: HUN-REN, Research Centre for Natural Sciences, Budapest, Hungary   
Abstract: Face-to-face communicative actions by themselves are always ambiguous, to the extent that their meaning depends on the shared knowledge of the agents. This shared understanding is developed step-by-step during the interaction. Since shared concepts are not observable directly, previous studies focused on verbal, behavioral, and neural synchrony, or task performance to characterize the extent of the shared conceptual space. The current study explores this concept by analyzing head movement synchrony and gaze coordination of two interlocutors measured during a communication task. The task is the so-called “Bargaining Game” involving verbal negotiations over prices of items. Communication success was measured through task performance while task difficulty, the modality of communication (audio-only vs. audio and video), and pre-task familiarity of the interlocutors was manipulated. We found that participants adapted to the constraints of the conditions: head movement synchrony decreased but gaze coordination increased in the audio-only condition. As a general trend, gaze coordination affected task performance while head movement synchrony showed positive correlation with the reported likability of the partner. These preliminary results serve as valuable starting points to analyze the dynamics of the development of shared understanding.

Author: Petra Kovács
Co-authors: István Winkler, Brigitta Tóth
Title: "Using virtual reality training to improve spatial hearing skills"
Affiliation:HUN-REN, Research Centre for Natural Sciences, Budapest, Hungary   
Abstract:Hearing impaired people struggle with spatial hearing, a fundamental ability in everyday life. In this research project, we test whether spatial hearing skills can be improved by virtual reality (VR) training. Our normal hearing participants took part in two sessions of 30 minutes VR training on two consecutive days, using the BEARS package. They completed pre- and posttests both in VR and in the free field (using loudspeakers) to assess their sound localization skills before and after training. We also modeled unilateral hearing impairment by using earplugs in the right ear during testing and training. According to our preliminary results, errors of localizing Gaussian white noise bursts decreased during the course of training in the VR test, and localization accuracy in the free field test improved for both white noise bursts and pseudowords. This suggests that training in VR can improve sound localization skills, and this improvement generalizes to free field listening. Our results yet need to be confirmed by testing a control group who engage in a VR training program targeting some skill other than sound localization. These results will be discussed and possible future steps will be suggested in the poster.

Author: Luca Béres
Co-authors: Ádám Boncz, Péter Nagy, István Winkler
Title: "Synchrony of pupil size and prosodic features during face-to-face communication"
Affiliation:HUN-REN, Research Centre for Natural Sciences, Budapest, Hungary   
Abstract: Face-to-face communicative actions (e.g. speech, facial displays, gestures, etc.) by themselves are almost always ambiguous, yet, interlocutors readily resolve the challenges of coordinating meaning and building a shared understanding during everyday interactions. How is this coordination achieved? Previous research has shown that people tend to mimic one another in interactions. Specifically, they reuse each other’s phrases, syntactic structures, synchronize their movements, prosody and pupil size (although evidence is scarce for the latter). Approaches focusing on shared conceptual space have assumed that behavioral synchrony indicates the extent of conceptual alignment and, thus, predicts communication success. The current study explores this concept. We asked pairs of participants to solve a few rounds of a computer-mediated communicative task involving negotiations, while pupil size, audio and video are collected from both participants, along with objective measures of communication success (through task performance). Our preliminary results indicate that pairs of participants synchronize their behavior in terms of pupil size and prosodic features and that communication success is affected by both. We also tested the effects of the modality of communication (audio-only vs. audio and video) and found that participants adapted to the constraints of the conditions: speech-related synchrony increased in the audio-only condition.

Author: Natálie Brožová 1 
Co-authors: Lukas Vollmer 1, Björn Kampa 2, Christoph Kayser 3, Janina Fels 1
Title: "Investigating the Neural Basis of Audiovisual Cross-modal Correspondences and Their Influence on Perceptual Decision-Making"
Affiliations:1 Institute for Hearing Technology and Acoustics (IHTA), RWTH Aachen University, Aachen, Germany
2 Systems Neurophysiology Department, Institute of Zoology, RWTH Aachen University, Aachen, Germany
3 Department of Cognitive Neuroscience, Universität Bielefeld, Bielefeld, Germany  
Abstract: Audiovisual cross-modal correspondences refer to the brain's inherent ability to subconsciously and consistently connect auditory and visual information. Such correspondences reveal the mechanisms through which our brain integrates and makes sense of multisensory information. Earlier studies have shown that congruent cross-modally associated stimuli enhance behavioral performance, improving reaction times and accuracy. However, the impact on information processing and decision-making, as well as the neural origins, may differ among the types of cross-modal correspondences. In this study, two types are examined: statistical and semantically mediated. Building on the Implicit Association Test and incorporating concurrent EEG measurements, we explore neural activity elicited by different instances of cross-modal correspondences. Our research extends previous findings by examining both auditory and visual trials, highlighting early and late EEG components that differentiate between congruent and incongruent trials. This approach provides insights into the processing stages at which congruency between auditory and visual stimuli affect perceptual decision-making, contributing to a deeper understanding of cross-modal perception.

Author: Carolin Breuer 
Co-authors: Lukas Vollmer, Janina Fels 
Title: "Investigating visual priming on auditory selective attention in Virtual Reality"
Affiliation: Institute for Hearing Technology and Acoustics (IHTA), RWTH Aachen University, Aachen, Germany    
Abstract: With the rising interest in virtual reality applications for research, it is increasingly harder to distinguish between auditory and visual influences on perception. In a study on auditory selective attention which transfers a purely auditory task into a virtual classroom scenario, we found indications for enhanced attention in the virtual environment. Possible explanations for this finding could be gamification or immersion, which might motivate participants. To get further insight into possible cross-modal effects on this paradigm as well as to create a more realistic scenario, the mentioned paradigm was extended by visual stimuli. The previous auditory task was to focus auditory attention on a cued direction and classify spoken animal names coming from this direction into flying and non-flying animals. To investigate the cross-modal effects, images of the animals were added which could be either congruent or incongruent with the auditory target stimulus. Preliminary results suggest higher reaction times when an image of the animal was presented. However, only tendencies for an audio-visual congruence effect were found.

Author: Khaled Abdellatif  
Co-authors: Isa Samira Winter; Moritz Wächtler, Hartmut Meister 
Title: "Investigating audiovisual cues of linguistic prosody in cochlear implant recipients"
Affiliations: Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital of Cologne and Jean-Uhrmacher-Institute for Clinical ENT-Research, University of Cologne, Geibelstraße 29-31, D-50931 Cologne, Germany   
Abstract: Linguistic prosody plays a vital role in verbal communication. Amongst others, it carries information on sentence stress or the distinction between questions and statements. Cochlear Implant (CI) recipients are restricted in the use of acoustic prosody cues, especially in terms of the voice fundamental frequency. However, prosody is also perceived visually, as head and facial movements accompany the vocal expression. To date, few studies have addressed multimodal prosody perception. Controlled manipulations of acoustic cues are a valuable method to examine and quantify prosody perception. It could be shown that CI recipients had about four times higher F0-detection thresholds and shallower psychometric functions than normal-hearing listeners. For visual prosody, however, such a technique is more demanding. Here, a novel approach based on animations via virtual humans is used. Such a method has the advantage that – in parallel to acoustic manipulations – head and facial movements can be parametrized. Based on this method a study is shown that examines audiovisual prosody perception in CI recipients. Besides behavioral measures, eye-tracking and pupillometry data are collected to assess selective attention and cognitive load of the participants. Preliminary results from postlingually deafened CI recipients are compared with a control group of normal-hearing listeners. 

Supported by Deutsche Forschungsgemeinschaft, grant ME 2751-4.1 to HM 

Author: Udbhav Singhal 1
Co-authors: Norbert Kopčo 1, Maike Klingel 2
Title: "A Signal Detection Theory Model of Binaural Cue Reweighting using Discrimination Task"
Affiliations: 1 Institute of Computer Science, P. J. Šafárik University in Kosice, Slovakia
2 Acoustics Research Institute, Austrian Academy of Sciences, Vienna, Austria
Abstract: Background: Normal-hearing listeners weight binaural localization cues depending on the sound’s frequency content. Interaural time differences (ITDs) dominate at low frequencies, and interaural level differences (ILDs) dominate at high frequencies. However, the relative weighting of the cues used by the listeners can be sub-optimal, resulting in poorer localization in many scenarios, e. g., in reverberation. Previous studies showed that various training protocols can be used to induce a binaural cue reweighting both to increase the ILD weight and to increase the ITD weight. These studies used various measures to determine the ILD/ITD weight, making it difficult to compare the effectiveness of the different training protocols, as well as to compare the weights obtained in these studies to the standard “trading ratio”. E.g., in our previous study using discrimination training, the proportion of discrimination responses following the ILD (PILD) was used as the relative weight measure. The current study proposes and develops a signal-detection-theory (SDT) model, based on which a measure of the ILD/ITD weight can be estimated that is independent of the task and conditions used in each study. Further, we analyzed the training-session from to understand how training worked overtime within and between sessions and its dependence on various training parameters. 
Results: The derived relative weight estimate wLT, when applied to our previous results, shows that the effectiveness of the training was approximately equal for both ITD training and ILD training. Also, the measure is more robust to noise and less affected by outliers, compared to the PILD measure used originally. Our findings indicate that performance improvement occurred more significantly between training sessions rather than within them, applying to both the ILD and ITD groups. Additionally, the results showed that discrimination training led to a gradual enhancement in performance over a span of three days.  

[Work supported by Horizon Europe HORIZON-MSCA-2022-SE01 grant N° 101129903 and Science Grant Agency of the Slovak Rep. VEGA 1/0350/22] 

Author: Jakab Pilaszanovich  
Co-authors: Marc Schönwiesner 
Title: "Cortical encoding and perceptual adaptation to auditory distance cues"
Affiliations: Department of Biology, University of Leipzig, Germany
Abstract: The cortical encoding of sound distance is poorly understood. Several monaural cues contribute to the perception of sound source distance, including intensity, direct-to-reverberant sound energy, and vocal effort. Binaural cues (interaural time and intensity differences) play little role in this percept. Several of these distance cues are thought to be extracted in the cortex. We measured the cortical encoding of auditory distance through a series of EEG experiments employing different acoustic features as distance cues. Two sets of intensity equalised stimuli were generated: 1) unidentifiable sound objects compiled from anechoic recordings of impact sounds of everyday objects which were rendered at different distances in a simulated rectangular room (10x30x3m), 2) anechoically recorded vocalisations with different levels of vocal effort. The two sets of stimuli were designed and selected such that acoustically non-overlapping features evoke perceptions of distance: in set 1) direct-to-reverb sound energy ratio, in set 2) exerted vocal effort of a given vocalisation. These sets of stimuli were then presented in a within-subject EEG experimental design. We report systematic covariations of EEG source components with sound source distance. The comparison across stimulus conditions allows us to differentiate genuine representations of distance from representations of the underlying acoustic cues. As a follow up study we observed the effect of extended exposure to reverberant sounds on distance judgement. These results show a systematic “compression” of auditory space for which the reasons are yet unknown.

Author: Paul Friedrich
Co-authors: Marc Schönwiesner 
Title: "Adaptation rate and persistence across multiple sets of spectral cues for sound localisation"
Affiliations: Department of Biology, University of Leipzig, Germany
Abstract: The adult human auditory system can adapt to changes in spectral cues for sound localization. This plasticity was demonstrated by changing the shape of the pinna with earmolds. Previous studies investigating this adaptation process have focused on the effects of learning one additional set of spectral cues. However, adaptation to multiple pinna shapes could reveal limitations in the auditory systems’ ability to simultaneously encode discrete spectral-to- spatial mappings without interference and thus help to determine the mechanism underlying spectral cue relearning. In the present study, adult listeners learned to localize sounds with two different sets of earmolds within consecutive five-day adaptation periods. To establish both representations in quick succession, participants underwent daily sessions of sensory- motor training. Throughout the experiment, the acoustical and behavioral impact of the molds and the trajectories of individuals’ adaptation to the modified pinnae were measured. To test whether learning a new set of spectral cues interferes with a previously learned mapping, the persistence of adaptation to the initial earmolds was recorded after participants adapted to the second pair of molds. As a control, this measurement was repeated for the second set after five days without molds. Individuals’ ability to localize sounds was repeatedly disrupted by the successive pinna modifications and significantly recovered within each adaptation period. After the second adaptation, participants were able to access three different sets of spectral cues for sound localization. Results show that learning the second set of modified spectral cues did not interfere with the recently learned representation, as adaptation persistence did not differ between earmolds.

Author: Kateřina Kynčlová 1, 2
Co-authors: Kateřina Chládková 1, 2
Title: "Neural Indices of Online Statistical Learning in Visual Speech"
Affiliations: 1 Institute of Psychology, Czech Academy of Sciences, Czech Republic
2 Faculty of Arts, Charles University, Czech Republic
Abstract: The unintentional adaptation to regularities in the environment, referred to as statistical learning (SL), is exploited by both infants and adults during word detection in continuous speech. Previous studies investigating SL in speech were limited to the auditory domain, i.e. the primary modality of spoken communication. Additionally, recent findings suggest that the typical exposure-posttest design might fail to record the entire learning process, and argue for online assessment. The present study examined online neural indices of SL in speech devoid of auditory and contextual cues to investigate whether SL in speech depends on auditory anchors, or whether it is a key process during speech perception regardless of its modality. Adult participants were exposed to naturally mouthed silent speech streams in two within-subject conditions, namely, to twelve randomly occurring syllables, and to four repeating trisyllabic statistical words. The phase synchronisation of the neural activity, measured as the intertrial phase coherence in the EEG, was assessed at the syllabic and word rate. Largest phase synchronisation was detected at word rate during the exposure to the structured speech, indicating a successful neural tracking of words. Statistical learning in speech can thus be effectively measured online even in the absence of auditory cues.

Author: Gabriela Andrejková
Co-authors: Stanislava Linková, Norbert Kopčo
Title: "Mechanisms of Contextual Plasticity Explain Shifts in Human Sound Localization"
Affiliations: Perception and Cognition Laboratory, Faculty of Science, P. J. Šafárik University in Košice, Slovakia
Abstract: Contextual plasticity is a form of plasticity in sound localization induced by preceding stimulations. It is observed as shifts in responses and in standard deviations to a target click stimulus when, on interleaved trials, the target is preceded by an identical adaptor coming from a fixed location. Here we present the results of two experiments, one performed in real and one in virtual environment, evaluated in the context of two models of the neural mechanisms underlying spatial hearing in humans. The first model (Carlile et al., 2001) encodes spatial location by activity of a large population of neurons aimed at accurately encoding the stimulus location. The second model (Lingner et al., 2018) assumes that spatial location is encoded in activity of 4 opponent-processing channels optimized for sound source separation, not localization. The modeling found that performance in the real environment is more aligned with the first model, while performance in the virtual environment is more aligned with the second model, suggesting that listeners use different strategies and/or neural mechanisms when localizing sounds in real vs. virtual environments.

[Work supported by Horizon Europe HORIZON-MSCA-2022-SE-01 grant N° 101129903 and VEGA 1/0350/22]

Author: Peter Lokša
Co-authors: Norbert Kopčo
Title: "A Model of the Reference Frame of the Ventriloquism Aftereffect Considering Saccade Adaptation"
Affiliations: Perception and Cognition Laboratory, Faculty of Science, P. J. Šafárik University in Košice, Slovakia
Abstract: 
Background: The ventriloquism aftereffect (VAE), observed as a shift in the perceived locations of sounds after audio-visual stimulation, requires reference frame (RF) alignment since hearing and vision encode space in different frames (head-centered vs. eye-centered). Previous experimental studies observed inconsistent results: a mixture of head-centered and eye-centered RFs for the VAE induced in the central region vs. a predominantly head-centered RF for the VAE induced in the periphery on one side, and for the aligning audiovisual calibration on the other side. A previous model proposed to describe these data required different parameter fits to predict the central vs. peripheral aftereffect data as well as the AV-aligned data. Here, a new version of the model is introduced to provide a unified prediction of all data sets considering that saccade responses, used to measure VAE, are also adapted.
Methods: VAE was measured using eye-tracked saccades to the perceived locations of sound (i.e., “auditory saccades”) presented via loudspeakers. The model has two components: an auditory space representation component and a saccade-representation component. The former is adapted by ventriloquism signals in the head-centered RF. The later one characterizes adaptation in auditory saccade responses in eye-centered RF.
Results: The updated version of the model provides a unified prediction of all the data, even if only head-centered RF is considered in the auditory space representation, while proposing that there are a priori biases in the auditory saccades and that the auditory saccades also undergo ventriloquism adaptation.
Conclusion: The results suggest that purely head-centered RF is used for adaptation of auditory spatial representation in the ventriloquism aftereffect, and that the apparently mixed eye-and-head centered RF observed experimentally is most probably due to saccade-related biases that are eye-centered. 

[Work supported by Horizon Europe HORIZON-MSCA-2022-SE-01 grant N°101129903 and VEGA 1/0350/22]

Author: Myroslav Fedorenko 1,2
Co-authors: Norbert Kopčo 1
Title: "Effects of preceding exposure on distance perception and distance learning in varying and fixed virtual environments"
Affiliations: 1 Perception and Cognition Laboratory, Faculty of Science, P. J. Šafárik University in Košice, Slovakia
2 M. Dragomanov Ukrainian State University, Kyiv, Ukraine
Abstract: 
Background: Previous research showed that listeners calibrate to the acoustic environment when judging distance in real reverberant rooms, resulting in gradually improving performance even without explicit training. In contrast, similar enhancements are not observed in anechoic environments, suggesting that the improvements are due to tuning to specific reverberation-related cues in a given room. The mechanisms underlying this "room learning" are not well understood. A previous study [Schoolmaster, Kopčo, & Shinn-Cunningham, J Acoust Soc Am 113, 2285, 2003], examined 1) how consistency in simulated room presentation affects performance, and 2) whether previous exposure to consistent vs. inconsistent simulation influences the ability to interpret the distance cues in different rooms. It showed that distance perception is more accurate when the simulated environment is consistent than randomly chosen from trial to trial. Here, we further analyze the data to examine how they depend on type of simulated room (anechoic vs. reverberant) and the direction of stimulus presentation (frontal vs. lateral). And, we develop a model that proposes how subjects combine available distance cues in different contexts.
Methods: Three environments were simulated in the experiment, anechoic, center of a classroom and corner of a classroom. Participants were divided into two groups. The first group started with 6 sessions in a fixed environment (each session maintaining a consistent room condition in a random order) and then proceeded to 6 sessions in a mixed environment (room conditions randomly selected from trial to trial). The second group proceeded in the reverse order. Each session comprised eight runs, each consisting of 45 trials. Nine distances ranging from 15 to 170 cm were presented randomly within each run. Each run kept the direction of simulation (frontal vs. lateral) fixed. Results: Overall performance, evaluated using the correlation coefficient, showed a complex pattern of results. The group that started with inconsistent room generally performed poorly, and even when later exposed to consistent simulation only improved performance slightly. On the other hand, the group starting with consistent simulation performed very accurately in each room and, when switched to inconsistent simulation, its performance deteriorated differentially depending on simulated room and direction. Analysis of response biases showed that these results are consistent with a model that assumes that 1) the listeners use room-specific weighting parameter values to combine the distance cues in consistent simulation, but that 2) they use one non-specific weighting parameter set when simulation is inconsistent.
Conclusions: These results show that listeners use past experience when calibrating to specific environments and that they might not be able to use an optimal room-specific tuning even for dramatically different environments (like anechoic vs. reverberant) when the environments switch rapidly.

[Work supported by Horizon Europe HORIZON-MSCA-2022-SE-01 grant N° 101129903 and VEGA 1/0350/22]

Author: Ľuboš Hládek
Co-authors: Piotr Majdak, Robert Baumgartner
Title: "RELCOM: Recovery from Effortful Listening and Communication"
Affiliations: Acoustic Research Institute, Austrian Academy of Sciences, Vienna, Austria
Abstract:
When people talk at a cocktail party, selective auditory attention helps them to segregate acoustic targets from interferers. Listening among competing interferers is, however, effortful, especially for people with hearing disability. Currently, deficits in attentional processing due to effortful listening are not implemented in standard audiological methods. The current methods are thus insufficient at predicting the effectiveness of communication in real-life settings. RELCOM will test the hypothesis that effortful listening causes a short-term depletion of auditory attention due to reduction in available cognitive resources. A new protocol will be developed enabling the capture of the attention depletion magnitude and the temporal profile of the recovery from a task-induced listening effort. In that new protocol, auditory attention will be probed using an auditory streaming paradigm shortly before and after a real-life task of effortful listening. The probing will consider objective measures such as pupillometry and electroencephalography, both combined with behavioral performance. Listening effort will be induced in a conversation that takes place in a virtual acoustic reality controlled by real-time signal processing creating realistic acoustic conditions. This research is essential for future audiological applications because the obtained measures have potential to guide individualization of hearing technologies and acoustic environments with respect to listener’s own needs or disabilities.

[Recipient of a Seal of Excellence Postdoctoral Fellowship of the Austrian Academy of Sciences.]