Are we biased against AI-made haiku poems?

Abstract

Haiku is a traditional Japanese unrhymed short poem of 17 syllables with ‘a seasonal word’ and ‘a cut letter’ to divide 17 syllables into smaller units of [5][75], [57][5], or [5][7][5]. Due to its brevity, haiku often lack arguments and predicates, and a cut letter separates the text into smaller parts, making it difficult to establish local coherence. However, Gilbert (2024) claims that the play of disjunction and coherence is a taproot of haiku. Haiku, or a version of it, is now written by poets worldwide and even by AI. This paper examines how humans evaluate AI-generated haiku poems through experiments.

Recently, AI-generated text has made much progress in art and literary works. Kawamura and his team (2019, 2021, among others) have generated 100 million AI- generated haiku via three steps: (i) deep learning of 400,000 haiku by Long Short-Term Memory (LSTM) and making word sequences, (ii) filtering out the word sequences for the traditional 17-syllable rule and a seasonal word, and (iii) scoring the generated haiku (cf. Hirata et al. 2022). They report that the scoring process is the most challenging.

Chamberlain et al. (2017) claim that humans have a bias against computer-generated art, and we will investigate whether this bias also applies to AI-generated haiku in this paper. Three in-person experiments were conducted. Experiment 1 aimed to establish a standard for evaluating human-generated haiku. Experiments 2 and 3 investigated how humans evaluate AI-generated and human-generated haiku based on the evaluation standard gained by Experiment 1. We recruited 140 university students and asked them to evaluate 30 human-generated and AI-generated haiku on a 5-point Likert scale. In Experiment 3, but not in Experiment 2, the subjects were informed which haiku was AI-generated. After Chamberlain et al., we predicted human-generated haiku would be more highly evaluated, but the results show that human cognition is more complex.