A corpus of science journalism for analyzing writing quality

Annie Louis, Ani Nenkova

Abstract


We introduce a corpus of science journalism articles, categorized in three levels of writing quality. The corpus fulfills a glaring need for realistic data on which applications concerned with predicting text quality can be developed and evaluated. In this article we describe how we identified, guided by the judgements of renowned writers, samples of extraordinarily well-written pieces and how these were expanded to a larger set of typical journalistic writing. We provide details about the corpus and the text quality evaluations it can support. Our intention is to further extend the corpus with annotations of phenomena that reveal quantifiable differences between levels of writing quality. Here we introduce two of the many types of annotation on the sentence level that distinguish amazing from typical writing: text generality/specificity and communicative goal. We explore the feasibility of acquiring annotations automatically, and verify that such features are indeed predictive of writing quality. We find that the annotation of general/specific on sentence level can be performed reasonably accurately fully automatically, while automatic annotations of
communicative goal reveals salient characteristics of journalistic writing but does not align with categories we wish to annotate in future work.

Full Text: PDF


www.dialogue-and-discourse.org ISSN: 2152-9620   Journal doi: 10.5087/dad