Intention and Attention in Image-Text Presentations: A Coherence Approach

Ilana Torres, Kathryn Slusarczyk, Malihe Alikhani, Matthew Stone

Abstract


In image-text presentations from online discourse, pronouns can refer to entities depicted in images, even if these entities are not otherwise referred to in a text caption. While visual salience may be enough to allow a writer to use a pronoun to refer to a prominent entity in the image, coherence theory suggests that pronoun use is more restricted. Specifically, language users may need an appropriate coherence relation between text and imagery to license and resolve pronouns. To explore this hypothesis and better understand the relationship between image context and text interpretation, we annotated an image-text data set with coherence relations and pronoun information. We find that pronoun use reflects a complex interaction between the content of the pronoun, the grammar of the text, and the relation of text and image.


Keywords


ELM; NLP; discourse; coherence; pronoun resolution; computational linguistics; semantics; pragmatics

Full Text:

PDF


DOI: https://doi.org/10.3765/elm.1.4873

Copyright (c) 2021 Ilana Torres, Kathryn Slusarczyk, Malihe Alikhani, Matthew Stone

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.