Reviewed by Francisco Yus, University of Alicante

Nowadays, ‘multimodality’—‘interdependence of semiotic resources in a text’, as defined by the editors—is attracting a great deal of scholarly attention. This book is an example of this growing interest. It comprises twelve papers by different authors in two parts plus the editors’ introduction. Despite the ubiquity of audio-verbal-visual media in today’s communication, it is relatively recent that the developments of the various possibilities of combining communication modes have forced scholars to think about the particular characteristics of these modes and the way they semiotically function and combine (1).

Part 1, ‘Multimodal issues’, starts with ‘In between modes: Language and image in printed media’ (Ch. 1, by Hartmut Stöckl, 9–30), in which he develops a system network for language and image relationships in printed media. These are sustained in combinations of core modes, medial variants, peripheral modes, sub-modes, and features. In ‘Modelling multiple semiotic systems: The case of gesture and speech’ (Ch. 2, by Peter Muntigl, 31–50) it is argued that linguistic and nonlinguistic semiotic systems (for instance, gestures) share properties and are even similar in grammatical structure (i.e. content and expression planes).

‘Problematising ‘semiotic resource’’ (Ch. 3, by Victor Linn Fei, 51–63) is an analysis of signs within multimodality. As in Ch. 2, it is also claimed that visual signs operate on content and expression planes, but with an emphasis on the degree of arbitrariness between these planes. In ‘Multimodality and empiricism: Preparing for a corpus-based approach to the study of multimodal meaning-making’ (Ch. 4, 65–87), John Bateman, Judy Delin, and Renate Henschel argue that current claims about multimodality are merely informal or interpretative because they are not sustained in empirical support. Therefore, the authors introduce the first steps given in an empirical direction, specifically a multimodal annotated corpus called the GeM (genre and multimodality) model.

Part 2, ‘Analyses and applications’, is devoted to innovative multimodal analyses and starts with ‘On the effectiveness of mathematics’ (Ch. 5, by Kay L. O’Halloran, 91–117), in which the evolution of the language of mathematics (from multimodal discourse to symbolic language) has had an effect on the development of descriptions of the physical world. ‘Multimodality in language teaching CD-ROMs’ (Ch. 6, by Martin Kaltenbacher, 119–36) describes the necessary verbal-visual qualities of teaching CD-ROMs in order to be effective as teaching tools.

‘The multiple modes of Dirty dancing: A cultural studies approach to multimodal discourse analysis’ (Ch. 7, by Markus Rheindorf, 137–52) deals with how different modes combine in the film Dirty dancing, concluding that dancing lies at the heart of how the protagonists build up their semiotic roles. In ‘Multimodal text analysis and subtitling’ (Ch. 8, 153–72), Christopher Taylor analyzes the role that multimodal text analysis plays in subtitling. For instance, the direct reference of text to visual parts of the film is seen as challenging for subtitling.

‘Multimodality in the translation of humour in comics’ (Ch. 9, by Klaus Kaindl, 173–92) deals with comics such as Asterix. In this study the author stresses the fact that most of the humor in comics arises from visual input, rather than from the text. Then he moves on to suggest eight different strategies for the translation of multimodal humor. ‘Multimodality in operation: Language and picture in a museum’ (Ch. 10, by Andrea Hofinger and Eija Ventola, 193–209) addresses multimodal issues in designing a museum on the life of composer Amadeus Mozart.

In ‘Drawing on theories of inter-semiotic layering to analyse multimodality in medical self-counselling texts and hypertexts’ (Ch. 11, 211–26), Eva Martha Eckkrammer addresses what she calls ‘inter-semiotic layering’ of multimodal texts. In short, verbal and visual input do not provide meaning separately, but through a combination or interaction between the semiotic modes involved. Finally, ‘On the multimodality of interpreting in medical briefings for informed consent: Using diagrams to impart knowledge’ (Ch. 12, by Kristin Bührig, 227–41) describes visual materials (for instance, diagrams and charts) and their interpretation at hospitals, where there are linguistic barriers and different levels of professional expertise.