Speaker: Neil Cohn
Affiliation: Department of Communication and Cognition, Tilburg University
Title: The grammar of visual narratives: The structure and (neuro)cognition of sequential images
Abstract: Sequences of images are all around us—from historical scrolls and cave paintings to instruction manuals and contemporary comics. Just how do we comprehend these sequences of images? Recent research has shown that the comprehension of visual narratives extends beyond the meaningful relations between images and uses a “narrative grammar” that organizes this semantic information. I will show that this structure, based on contemporary construction grammars from linguistics, packages meaning into categorical roles into hierarchic constituents to account for phenomena like long distance dependencies and structural ambiguities. In addition, using measurements of brainwaves (EEG/ERPs), I will show that this grammar is independent of meaning (e.g., N400), and engages similar neurocognitive processing as syntax in language (e.g., anterior negativities, P600). Finally, I will show that sequential image processing is modulated by a person’s fluency in the specific narrative grammars found in different “visual languages” of the world. Altogether, this work introduces emerging research from the linguistic and cognitive sciences that challenges conventional wisdom with a new paradigm of thinking about the connections between language and graphic communication.