Speaker: Mesian Tilmatine
Affiliation: Freie Universität Berlin & Radboud University
Title: Text-based Prediction of the Neural Response to Narrative Poetry Reading
Abstract (long version below): Analyses of subjective rating behaviour alone are not data-rich enough to understand the entire complexity of the cognitive and affective processes underlying literary reading. Specific neural systems have been theorized to be related to processes like narrative immersion, appreciation of the beauty of language use, textual comprehensibility, as well as cognitive enrichment and its resulting pleasure. By measuring fMRI during the fixed-pace reading of long naturalistic works of narrative poetry, we locate the effects of these processes in the brain and predict their effects with text properties relevant to each process.
Long abstract
Literary texts do not merely communicate information, but ideally also affect the reader on an emotional level, typically through subjective reader experiences like aesthetic appreciation and narrative immersion (Willems & Jacobs, 2016). To investigate this process, we can identify emotion-evoking literary text properties that predict the reader response as measured by subjective ratings (Jacobs, 2019). In a series of previous studies (partially presented at the last few IGEL conferences), we were able to apply this paradigm of textual predictions to a broader set of text properties and rating variables as well as also to eye-tracking data. To better account for the various processes theorized to emerge during literary reading, we started using stimuli that have the potential to elicit both aesthetic and immersive processes, i.e., narrative poetry.
However, analyses of eye-tracking and subjective rating behaviour cannot differentiate which underlying cognitive processes are recruited during literary reading. In this study, we measure neural activity to test the predictions of different models of literary processing like PIA (Graf & Landwehr, 2015) and NCPM (Jacobs, 2015).
PIA and NCPM assume neurocognitive systems that are largely separated from each other but each contribute to the emotional effect of literary texts on readers. These systems would be related to processes like narrative immersion, appreciation of the beauty of language use, and textual comprehensibility. To be more precise, NCPM predicts separate trajectories for aesthetic versus immersive processing, and PIA predicts a moderating role of readability for any sort of literary processing. By measuring fMRI and eye-tracking during the fixed-pace reading of narrative poetry by Goethe (around 220 000 words in 1088 verse lines, presented as multiple-verse pages), we locate the effects of these processes in the brain and predict their effects with text property sets relevant to each process.
To localize the brain regions related to different processes, we look at how the variability in text properties across pages is associated with activity in different brain regions. To study the processes involved in reading, we link each property to 3 distinct text features. Narrative immersion is measured by arousal potential, meaning coherence (semantic similarity) between pages, and narrative event probability. Aesthetic appreciation is measured by aesthetic-affective potential, semantic page (non-)uniqueness (“eigensimilarity”), and sonority. Finally, textual comprehensibility is measured by semantic concreteness, the position of the page within the text, and word frequency in general language use.
To locate a collective effect of each of the three theorized processes, we use whole-brain GLMs (F-test) for each corresponding set of three text properties. Additional artificial neural network analyses identify moderation effects and the feature importance of each of the individual text properties for the prediction in the regions of interest identified with the location analyses.
Taken together, this study will help us to identify the neural underpinnings of the cognitive processes that emerge during literary reading as a product of the interaction between reader and text. It is important to notice that these analyses do not take into account individual differences between readers, but rather help us understand the average reader response to a given literary text and its measurable properties.
References
Graf, L. K., & Landwehr, J. R. (2015). A dual-process perspective on fluency-based aesthetics. Personality and Social Psychology Review, 19(4), 395-410. https://doi.org/10.1177/1088868315574978
Jacobs, A. M. (2015). Neurocognitive poetics: Methods and models for investigating the neuronal and cognitive-affective bases of literature reception. Frontiers in Human Neuroscience, 9. Frontiers | Neurocognitive poetics: methods and models for investigating the neuronal and cognitive-affective bases of literature reception
Jacobs, A. M. (2019). Sentiment analysis for words and fiction characters from the perspective of computational (neuro-)poetics. Frontiers in Robotics and AI, 6. Frontiers | Sentiment Analysis for Words and Fiction Characters From the Perspective of Computational (Neuro-)Poetics
Willems, R. M., & Jacobs, A. M. (2016). Caring About Dostoyevsky: The Untapped Potential of Studying Literature. Trends in Cognitive Sciences, 20(4), 243–245. Redirecting