Brain oscillations and synchronization during language processing and action instruction

Acronym: 
BRAINSYNC
Term: 
2008-03 till 2012-10
Research Areas: 
C
D
Abstract: 

The main goal of this project is the investigation of language processing by means of behavioral data analysis (reaction time, rating) and the recording of electrical brain activity (EEG). Behavioral and neuronal data are obtained while participants process concrete and abstract sentences dealing with motor actions and various sensory modalities. According to the embodied cognition hypotheses, the modalities that have to be integrated during sentence comprehension should influence the participants processing time, error rate, learning performance and neuronal synchronization pattern.

Methods and Research Questions: 

The main aim of this project is the investigation of common characteristics and differences of concrete and abstract language on the basis of behavioral experiments (reaction time, rating) and EEG recordings (ERP, EEG coherence). Understanding object names or simple action instructions (concrete language) is often successfully implemented in technical systems, however, the parsing of complex abstract language is still a major problem. Especially problematic are cases in which the abstract meaning constitution is based on concrete lexical entities (e.g., verbs in „Sie ergreift die Gartenschere” versus "Sie ergreift die Gelegenheit”). From our point of view the combination of the „Embodied-Language-Comprehension-Theory (ELC) on the cognitive linguistic side and the Binding-by-Synchrony-Theory (BBS) on the neuronal side may allow a new view on the processing of abstract language. According to ELC-Theory neuronal substrates underlying senso-motoric processes are not only engaged during perception and action but also during language comprehension. More specifically, it has been proposed that not only the processing of concrete but also abstract language partly base on the perception of physical entities and action but also on the mental simulation of affective states and contextual situations. As a consequence, the processing of concrete and abstract language may share common neuronal activity patterns related to perceptual and motoric processing but also different ones. An unsolved issue is the way the brain integrates the cognitive and linguistic processes ocurring more or less simultaneously during language comprehension. One hypothesis raised in the neurosciences that provides a possible solution to this “binding-problem” is the Binding-by-Synchrony-Theory (BBS). EEG data were recorded whilst participants processed various concrete and abstract sentences and were analyzed in the light of the BBS-Theory. A central issue was the relation of linguistic processing to increased and decreased neuronal cooperation of brain oscillations in various frequency bands. Spectral analysis of the EEG signals was performed with both classical Fourier transform and more sophisticated adaptive, bivariate auto-regressive moving average models (ARMA) in order to evaluate the dynamics of functional relationships between signals associated with language-specific or language-relevant processes. In particular, the importance of EEG coherence analysis dealing with the functional temporal and frequency-specific coupling of brain activity was emphasized for the investigation of the neurobiological basis of language comprehension.

Outcomes: 

Experiments investigated the processing of concrete or abstract sentences either dealing with one or two sensory modalities (seeing, feeling, hearing, etc.). Behavioral experiments revealed a longer reaction time for visually presented sentences dealing with visual modalities during semantic judgement. Similar results were obtained for auditorily presented sentences. Furthermore, sentences dealing with only one modality were processed more quickly than sentences dealing with two modalities. These results hint at increased processing costs due to modality switching during sentence processing for both concrete and abstract language. EEG coherence data demonstrated that concrete and abstract language share neural networks in the theta- but not in the beta1- frequency band and that the degree of concreteness of a sentence correlates with the height of EEG coherence during its processing.

The present results give evidence for a model on the relation between brain oscillations, EEG frequencies, neuronal synchronization and mental simulation during language processing. Knowledge about neurocognitive processes during verbal comprehension provides a module for building theories and consequently the technical conversion in human-computer interaction.

Publications: