Annotated Bibliography - Multimodal Discourse Analysis

The aim of the paper is discovering multimodal discourse analysis itself and show different people’s take on the concept because there are a lot of theories in the field. With this I could explain the close relationship between language and technology and also how they depend on each other when analysing. The different methods shown in the papers will be a base for a short analysis of some sort of print media at the end of the paper as an example for multimodal discourse analysis to make the final paper a bit less theoretical.

LeVine, P., & Scollon, R. (2004). Discourse and technology. Washington, D.C.: Georgetown University Press.

In a multimodal world a linguist cannot turn a blind eye to the significance of visual communication. Most of the time language and meaning is signalled by visual aids. Moreover, it is not only a preferred way but most of the time people can decode the message much faster when using visual aids. For example in drafts and diagrams we often use arrows to show the relation between two items. Decoding the sentence “There was a car crash " Two people are in the hospital” is easier than this sentence: “There was a car crash, as a result, now two people are in the hospital.” Visual aids are not always so explicit. Typography’s importance, for example, grew remarkably in the past couple of years. It used to only exist as the “transmitter of the written word” but today it is a communicative mode and thus multimodal as well. It was stuck on a level where typography was just a tool for us the connect letters and being able to read them. Fortunately this has changed and typography itself became more significant and capable of expressing complex meanings.

This source would be perfect to start the actual research paper with because it has a phycology side and how our brain works. It could be used to explain why people need visual aids (as it said: it makes understanding easier) and by explaining this phenomenon I would justify the importance of multimodal discourse analysis and also the aim of the paper itself. Also is has a small historical background concerning typography which is always a good ways to start a research paper: with a little genealogy.

O'Halloran, K. (2008, November 1). Systemic functional-multimodal discourse analysis (SF-MDA): constructing ideational meaning using language and visual imagery. Visual Communication, 443-475.

The systemic functional approach to multimodal discourse analysis is meant to analyse the meaning of a multimodal text. Such texts combine multiple semiotic resources in discourse analysis ranging from TV shows and commercials to the static electronic or written text. The aim of the method is the meaning making of language and visual imagery in printed texts. SF-MDA approach also examines intersemiosis across language and visual imagery based on Royce’s (1998) framework called Intersemiotic Complementarity. The framework has six major points which explain how a multimodal text can be cohesive and thus how imagery and typography can be of help to transmit the meaning and aid the language in multimodal text. These six points are the following: Semiotic Cohesion means “system choices function to make the text cohesive” meaning that, for example, an image can symbolise a particular text. Semiotic Adoption describes an event when a semiotic resource is used in more system choices (e.g.: ‘x’ is used in both linguistics and mathematics). Semiotic Mixing is quite the opposite: when an item contains two system choices, for example a sign that says: “Call 815-234”. Juxtaposition and Spatiality means “the multimodal text is compositionally arranged to facilitate intersemiosis.” e.g.: a sign with a huge “warning” text printed on it. Semiotic Transition occurs when a system choice changes the intersemiotic discourse (e.g.: the instruction “Use your graph…” shifts from linguistic/symbolic description to the visual imagery. Lastly, Semiotic Metaphor or SM is when the “functional status of elements is not preserved and new elements are introduced”, for example when a picture is used to symbolise something and thus gets another connotation which is not necessarily the original one.

The six points can be helpful when writing the practical part of the essay and also a better way to explain the whole multimodal analysis field because the examples shown in the research would be also used in the final paper to illustrate to points and thus making the understanding of the topic deeper and easier. With these points raised in the research paper it made this source the most helpful to my paper and I would base the content of the essay mostly on this academic source.

Ventola, E., Charles, C., & Kaltenbacher, M. (2004). Perspectives on multimodality. Amsterdam: J. Benjamins Pub. Co.

Multimodality refers to processes which combine various sign systems and modes. It contains a lot of core modes and sub modes and they consist of image, language, vectors, colours, type size, fonts, colours and shadings any many more. These make up a whole network in which the parts are heavily dependent on each other and they can not necessarily be put into categories because they overstep their boundaries. The two major modes are language and visual. The linguistic and the pictorial mode can come together in two ways. Firstly: a verbal text can acquire image qualities with typography and layout. Or the more common option is when the verbal text is paired with and image. Since there are this many features in multimodality, the language-image-links are complicated and pose number of challenges. They can only be encoded through a multi-faceted and integrated analysis. To make the message clear and understanding it well enough not only the producers have to deploy the right meaning but also the recipients have to interpret the message in an appropriate way with these multi-faceted analyses.

The research paper had a table explaining how different aspects of multimodality effect our sensors when reading ranging from typeface to colours. Based on those aspects and also by using them the practical analysis would be an easier task. Also I would use the message of the ending of this research paper as my closing argument as well. (As in: not only the reader or the audience has to show the effort in analysing but the construction of the content should also be conscious.