Speaker: Irmak Hacımusaoğlu @hacimusa
Affiliation: Tilburg University, Department of Communication and Cognition
Title: Navigating meaning in the spatial layouts of comics: A cross-cultural corpus analysis
Abstract (long version below): In visual narratives, comprehenders need to track situational shifts in time, characters, and space across a spatial layout. We examined whether situational changes interacted with spatial organization of panels in a corpus of 134 comics from North America, Europe, and Asia. Overall, situational dimensions changed the most while moving across rows or columns, and across pages. The boundary of a page carried the most situational discontinuity (i.e., no shift in time while characters and location change) and aligned with especially changes in location. Our results indicate a partial correspondence between layout, and meaning, but with different cross-cultural constraints.
Long abstract
In visual narratives like comics, comprehenders need to track situational shifts in time, characters, and locations across sequential images. These depictions of events are suggested to feed into mental representations of a discourse in readers’ minds (Cohn, 2020b; Loschky et al., 2020), integrated with their prior knowledge (Kintsch & van Dijk, 1983; Zwaan & Radvansky, 1998). In visual narratives, these depictions are embedded within a spatial layout, like in a page. Thus, physical properties of the page layout (i.e., form) itself might interact with situational changes (i.e., meaning) as a way to segment and group meaning spatially. Both scholars and comic artists have maintained an assumption of an interaction between the form and meaning (Caldwell, 2012; Groensteen, 2007), but few works have examined this within the structures of actual comics. Therefore, we investigated the potential interaction between situational changes (time, characters, and space) and the affordances of the visuospatial layout in comics. To do so, we analyzed 134 annotated comics from North America, Europe, and Asia within Visual Language Research Corpus (VLRC; Cohn et al., In Prep).
Our corpus was gathered through donations of comic companies and convenience sampling and annotated over 5 years as a part of different subprojects. We only analyzed the stories with both situational changes and layout properties annotated within VLRC. Changes in time were annotated as either time passes across panels (rated as 1) or not (rated as 0). For characters and spatial location, partial changes (rated as 0.5) across panels were also annotated in addition to full changes when whole characters/location change (1) and no change in characters/location (0). Then, we recorded the instances of no change in time with shifts in characters and location across panels as ‘situational discontinuity’ (Klomberg et al., 2022). Properties of page layout were annotated based on the directionality between panels relative to their precedent panel (left, right, up-angle, down-angle etc.), and the first panel in each page was recorded as a new page. Lateral (right-to-left or left-to-right) and down directions correspond to panels within rows and columns, respectively. Thus, for this study, we collapsed those directions as within-constituent transitions since they maintain in a row or a column. Then, because up- and down-angle directions signal e.g., a row break, we considered them as across-constituent transitions. Finally, new pages were used as an indicator of a page boundary.
For each transition type (within constituents, across constituents, and across pages) we calculated the proportion of instances of each situational change (in time, characters, and space) out of the total number of each layout constituent – e.g., average spatial change while crossing a page out of the total number of new pages – per book. We did the same calculation for situational discontinuity as well. These yielded cross-cultural analyses of situational changes and situational discontinuity in transition types, respectively.
We found that situational dimensions changed the least in panels shifting within constituents (e.g., moving within a row), while moving across constituents (like in a row break) and pages carried more changes. The boundary of a page especially aligned with changes in location of the scene, indicating a correspondence between spatial meaning and its manifestation in the spatial layout. Yet, this was open to cultural variation since Asian comics did not maintain such a segmentation strictly. In addition, situational discontinuity primarily occurred while crossing a page, again suggesting the page as a segmental boundary.
These findings have psychological implications such as that placing situational boundaries at physical page boundaries might make it easier to segment the situation in mental representations (Cohn, 2020b; Loschky et al., 2020). In other words, shifting to a new physical space in the given layout could reinforce the accompanying change in the narrative. This might then help updating the location in readers’ mental representation. However, cross-cultural analyses indicated North American and European comics use the page as a unit to group and segment spatial information, whereas this was less of a pattern in Asian comics. Prior works have already shown distinct patterns in Asian comics’ situational changes (Klomberg et al., 2022; McCloud, 1993) and layout properties individually (Cohn, 2020a), which is also in line with this result. Thus, besides psychological implications, our results also indicate insights about cross-cultural storytelling. Altogether, the spatial organization of panels through layout appears to provide segmental boundaries for meaning, as suggested by comics creators and scholars, but with different cultural constraints.