Speaker: Gunther Martens @gnmarten
Affiliation: Ghent University
Title: Comparing Professional and Platform-driven Critics: A Digital and Sociological Analysis of Literary Gatekeeping and Gatewatching
Abstract (long version below): This paper examines the role of peer-to-peer recommendation systems and layperson critics as new literary gatekeepers and compares their evaluations to those of professional critics. The research project analyzes the evaluative âtalk of literatureâ surrounding prominent literary prizes by both groups, using fine-grained aspect-based sentiment analysis of an annotated corpus consisting of the official jury discussions and social media posts. The paper discusses the annotation system, the results of the annotation process, and future pathways for research aided by the arrival of the language bots. The study sheds light on the evolving role of traditional gatekeepers and the emergence of new gatekeepers in the literary field and beyond, providing insights into the evaluative criteria used by different groups to evaluate cultural production.
Long abstract
Recent research (Pianzola et al. 2020, Rebora 2021, Salgaro 2022) has explored the possibility of utilizing cultural analytics to ingest and chart online data related to literary taste. However, it remains unclear how the opinions of platform-based critics compare to those of professional literary experts. While review-aggregating platforms for films and music (like âRotten Tomatoesâ) are commonly consulted for aggregated review scores (Seaver 2022), the field of literature still prefers to rely on select gatekeepers rather than the collective wisdom of the cloud or recommendation systems. This paper aims to examine the role of peer-to-peer recommendation systems and layperson critics as new literary gatekeepers and cultural transmitters, and to compare their evaluations to those of professional critics.
The paper presents the results of the FWO-funded research project âEvaluation of literature by professional and layperson critics. A digital and literary sociological analysis of evaluative talk of literature through the prism of literary prizes (2007-2017)â. The project has mined the evaluative âtalk of literatureâ of both professional and layperson critics surrounding prominent literary prizes. The evaluative criteria used by both groups were analyzed to determine the extent of their overlap in determining âgoodâ and âbadâ literature. We present the results obtained through fine-grained aspect-based sentiment analysis of an annotated corpus consisting of automatically diarized transcriptions of the official jury discussions, posts mentioning the literary prizes on social media platforms. This analysis allowed for the detection of sentiments expressed about specific aspects (e.g. contender, nominated book, jury) on different media and by different groups. While the demography of those engaging in the online evaluation cannot be fully grasped due to anonymity, we will present an attempt to sharpen the profile of the audience by contrasting it with a parallel corpus of online comments on popular casting shows, revealing more striking differences in habitus, expressivity and choice of criteria.
The paper discusses the results of the annotation process, the annotation system, and the technical challenges faced by the project. With the arrival of LLM (Large Language Models), the slow process of annotation has received a boost from the ability to generate fully annotated ââmock dataââ. It will be explored to what extent both real and artificial training data have been helpful in the process of shedding light on the evolving role of traditional gatekeepers and the emergence of new literary gatekeepers, providing insights into the evaluative criteria used by different groups to evaluate literature.
Cited References
Pianzola, Federico, Simone Rebora, and Gerhard Lauer, âWattpad as a Resource for Literary Studies. Quantitative and Qualitative Examples of the Importance of Digital Social Reading and Readersâ Comments in the Marginsâ, PLOS ONE, 15.1 (2020), e0226708 https://doi.org/10.1371/journal.pone.0226708
Rebora S, Boot P, Pianzola F, et al. Digital humanities and digital social reading. Digital Scholarship in the Humanities. 2021;36(Supplement_2):ii230-ii250.
Salgaro, Massimo, âLiterary Value in the Era of Big Data. Operationalizing Critical Distance in Professional and Non-Professional Reviewsâ, Journal of Cultural Analytics, 7.2 (2022), 36446
Seaver, Nick, Computing Taste: Algorithms and the Makers of Music Recommendation (Chicago, IL: University of Chicago Press, 2022) https://press.uchicago.edu/ucp/books/book/chicago/C/bo183892298.html [accessed 9 September 2022]