* ABSTRACT: Multimodality, which is characteristic of texts that gather two or more semiotic codes (KRESS; van LEEUWEN, 2006), requires an approach that emphasizes the verbalvisuality (BRAIT, 2013) of these compositions. The aim of this study is to analyze categories of the compositional metafunction (KRESS; van LEEUWEN, 2006) from the perspective of multimodality in infographics of the advertising sphere, with the purpose of understanding how the selected messages, configured in infographics, are organized. In order to do so, it is necessary to: (i) characterize the selected infographics based on categories of the compositional metafunction; (ii) understand how the multimodal and functional aspects relate to each other in these texts; (iii) investigate how the multimodal aspects act in the production of meaning. The sample consisted of five infographics from the advertising area selected from the Pinterest website. The multimodal analysis of the texts is carried out from categories of the compositional metafunction, from the Grammar of Visual Design (KRESS; van LEEUWEN, 2006). The analysis shows that there is not only a visual focus but also a variable number of multimodal elements that, with their different sizes, colors, and communicative functions, contribute to the creation of various degrees of salience. Reading is done from the signifiers that stand out the most in the ad pieces, thus producing meanings.
* KEYWORDS: Multimodality. Compositional metafunction. Infographic. Advertising.
* RESUMO: A multimodalidade, característica dos textos que reúnem dois ou mais códigos sígnicos (KRESS; van LEEUWEN, 2006) requer um enfoque que enfatize a verbo-visualidade (BRAIT, 2013) dessas composiçoes. O objetivo deste estudo é analisar categorias da metafunçâo composicional (KRESS; van LEEUWEN, 2006) pela perspectiva da multimodalidade em infográficos da esfera publicitária, com a finalidade de compreender como se organizam as mensagens selecionadas que se configuram em infográficos. Para tanto, pretende-se: caracterizar os infográficos selecionados com base em categorias da metafunçâo composicional; compreender como os aspectos multimodais se relacionam aos aspectos funcionais nesses textos; averiguar como atuam os aspectos multimodais na produçâo dos sentidos. A amostra foi composta por cinco infográficos da área da publicidade selecionados a partir do site Pinterest. A análise multimodal dos textos será realizada a partir de categorias da metafunçâo composicional, da Gramática do Design Visual (KRESS; van LEEUWEN, 2006). A análise demonstra que nâo há somente um foco visual, mas um número variável de elementos multimodais que, com seus diferentes tamanhos, cores e funçoes comunicativas, contribuem para a criaçâo de diferentes graus de saliencia. A partir dos significantes que mais se destacam nas peças, a leitura é efetuada, produzindo-se significados.
* PALAVRAS-CHA VE: Multimodalidade. Metafunçâo composicional. Infográfico. Publicidade.
Introduction
In the media, the extensive use of images emphasizes the importance of understanding the meaning these resources in a text. Multimodality, characteristic of texts that gather two or more semiotic codes (KRESS; van LEEUWEN, 2006), such as words and images, requires an approach focusing attention on the verbal-visuality (BRAIT, 2013) of these compositions. Due to the complex process of production and interpre· tation of multimodal texts, since it covers diverse modes of meaning and a multiplicity of aspects that are part of a sociocultural universe, studies such as those developed within the perspectives of multiliteracies (LEMKE, 2010; ROJO; BARBOSA, 2015), and digital literacy, (BARTON; LEE, 2015) are needed.
Multiliteracy studies have proven to be valuable for the resignification of the teaching role in the contemporary context of new information and communication technologies, since teaching is a social process, and learning takes place as the subject interacts with different genres in various socio-cultural practices. Thus, analyzing genres from all different social context spheres1 is in a particular manner to address them from a multiliteracies perspective, conceiving text as a constitutive artifact of social relations.
The objective of this study is to analyze categories of the compositional2 metafunction (KRESS; van LEEUWEN, 2006) from the perspective of multimodality, namely, information value, framing, and salience, in infographics of the advertising sphere, in order to understand how the selected messages configured in infographics are organized. In order to do that, the infographics were characterized using the compositional metafunction categories; then, the relation between the multimodal and functional aspects in these texts and the role of multimodal aspects in the construction of meaning in the texts are discussed.
The sample of this qualitative study is composed of five infographics of the advertising sphere, selected from a search on the Pinterest website. As a theoreticalmethodological contribution, the focus is on multimodality from the Grammar ofVisual Design by Kress and van Leeuwen (2006), which offers useful categories for the study of languages in verbal-visual texts. Thus, the visual should be understood not only as consisting of the presence of images in texts but also formal and functional aspects as compositional elements, i.e., an essential feature to the meaning of the text.
The topicality of the theme is one of the justifying factors for this study, since it is perceived that infographics have been increasingly used in a variety of supports and genres, both in the advertising and journalistic sphere. Among the studies on infographics in journalism, it is worth mentioning: Teixeira (2007) who analyzed infographics in Superinteressante and Saúde, popular scientific magazines; Teixeira and Rinaldi (2008), whose study focused on the infographics section of an Internet portal; Modolo (2007) studied infographics in Brazilian magazines, providing a historical retrospect on the production of infographics in the journalistic field. Multimodality studies have focused on micro-chronicles published in the Zero Hora newspaper (GEHRKE, 2015), infographics of magazines through the Grammar of Visual Design (PAIVA, 2009), and the analysis of an editorial cartoon (KNOLL, 2015).
Furthermore, the relevance of this theoretical-methodological approach is given by the increasingly diversified repertoire of genres that integrate everyday social practices, considering the virtual contexts of interaction. . Mobile media, such as notebooks, tablets and smartphones, provide the contact with multimodal and hypermedia texts that group several languages. Thus, even though the reading of images is generally undertaken intuitively by people, difficulties remain in the composition and interpretation of messages conveyed in this modality of meaning making.
The study of language, in this case, is performed as a formal and functional analysis of the modalities which compose the selected examples. It is important to note that the Grammar of Visual Design, by Kress and van Leeuwen (2006), originates from Halliday's Systemic-Functional Grammar (1985), a theory from which the authors developed categories for visual analysis, i.e., there is a direct relation between multimodality and linguistic studies. The infographic is a fundamentally graphic and visual ad piece whose analysis will be crucial for the understanding of the functioning of language in this highly diffused genre in the digital media.
Thus this study is justified by the wide circulation of infographics, in diverse discursive spheres (TEIXEIRA, 2007), from the journalistic to the educational field, both in distance and conventional education, in teaching materials, as an educational resource in the classroom. Furthermore, in the view of the lack of specific guidance on the production and reading of this genre, regarding its characteristics and functions, it is necessary to address language as a semiotic system constitutive of interaction. Therefore, the contribution of this work if to provide examples of application of the multimodality approach to the analysis of these texts, grounded on the Grammar of Visual Design principles, by Kress and van Leeuwen (2006). It is also noteworthy the currentness of studies on specific social network texts, which is due to the diffusion of mobile technologies (smartphone and tablets) and by the launch of several websites and applications of social networks in the last ten years, such as the Pinterest, created in 2010. The messages selected from this social network were probably produced specifically for the medium,3 with all the linguistic, imagery and functioning particularities that a textual materiality has.
This study is composed of the following sections: first, the concepts of text and genre are discussed; then the infographic is characterized as a genre; in the following, the compositional metafunction and multimodality are described; next, the methodological framework is presented; and, finally, the multimodality in infographics is detailed, followed by the final considerations.
Text and genre
In any other activity, but especially in teaching, texts should be seen as a deposit of messages or as a set of grammatical elements (KLEIMAN, 2004), because there is a doubly constitutive relationship between text (discursive tissue) and social context.
According to Bakhtin (2010, p. 307, our translation), text is "the immediate reality (the reality of thought and experiences), the only one from which these disciplines and this thought can come." Thus understood, the text is a unit of manifestation, a product and, at the same time, process of human creation (FIORIN, 2006). For materializing discourses, a text is composed of language units (or other semiotic code) and a conventionally shared language, maintaining dialogical relations with texts that preceded it.
The set of categories proposed by Kress and van Leeuwen (2006) for the analysis of other modalities in association with verbal language derives from Systemic-Functional Linguistics, which is based on Halliday's (1985) grammar, a perspective also known as Social Semiotics, for covering signs and meaning intrinsically bound to the social context in which the messages take place. For this reason, the concept of text should be revised from this perspective.
Hodge and Kress (1988) differentiated message, text, and discourse in order to emphasize the social action, the context, and the use involved when analyzing language. Thus, the message was defined as "the smallest semiotic form that has concrete existence", because it has a source and an address (a goal), "a social context and a purpose" (HODGE; KRESS, 1988, p. 5).
In this sense, as Hodge and Kress (1988, p. 6) put it, from Halliday's SystemicFunctional Linguistics (1985), semiosis "is the social process by which meanings are constructed and exchanged in messages". The text is a larger semiotic unit, whose Latin origin of the term (textus) means "something woven together", that is, it is the materiality formed by a set of messages that coexist together, because they operate in the same context and the same situation. For example, the page of a newspaper is a text composed of several messages that, together, are semi-semantically processed by the reader. The discourse is, according to Hodge and Kress (1988), broader than the text for it is the social process from which the text emerges as produced materiality. For social semiotics, there is no text without discourse, nor discourse without text; there is a constitutive relationship between them.
Added to this are the hypermediality and hypertextuality4 of texts produced and shared using technological devices in mediated interactions, and the consequent increase in the networks of meaning and communicative flow complexity. Some texts are produced and distributed directly in the cyberspace, which is also called network, arising "from the worldwide interconnection of computers"5, involving not only machines or infrastructure, "but an ocean of information"6 (LÉVY, 1999, p. 17, our translation). Along with the development of cyberspace, there is a set of techniques, social practices, interactions, values, and ways of thinking that develop, composing the cyberculture (LEVY, 1999). Hence, the various genres in cyberspace are part of the cyberculture.
Because texts are not only the material products, but also semiotic units entirely dependent on the reality of social interactions, the plethora of languages not only represent meanings but also build them. Therefore, genres are culturally and historically constituted; that is, it is not possible to consider them separately from the context of communication. A genre is recognizable to participants in a given socio-historical instant.
Besides, it is necessary to highlight the heterogeneity of genres, resulting from the infinite potential for interaction: "genres include all sorts of everyday dialogues, as well as enunciations of public, institutional, artistic, scientific and philosophical life"7 (MACHADO, 2005, p. 155, our translation). For Kress (2008), there is no communication apart from those engendered in some genre, because a message or a text respond to a social fact, which is the genre. In this sense, "genre is essential in all attempts to understand text, whatever its modal constitution"8 (KRESS, 2008, p. 107, our translation).
For this work, the socio-rhetorical perspective of genres was prioritized. In this view, genres are social actions that: "(a) materialize a class of events; (b) share communicative purposes; (c) have specific prototypical features; (d) present innate logic; and (e) determine specific linguistic uses according to the discursive community"9, as explained by Pereira and Rodrigues (2009, p. 6, our translation).
Bazerman (2005) defined genres as recognizable social facts (events) since they are based on recurrences as to form and, especially, function. For this linguist, genres "arise in social processes of people trying to understand each other well enough to coordinate activities and share meanings for their practical purposes"10 (BAZERMAN, 2005, p. 31, our translation). Genres typify more than the textual form they encompass the configuration of social practices and interactions organizing activities and people. Each sphere of language use has its more or less numerous, but always dynamic, repertoire of genres, which are essential typifications in language use for the establishment of communication. In this sense, the use of infographics in advertising is related to specific conditions and purposes of this field of activity, which is reflected not only in its content but also in the semiotic resources used in its composition.
Infographic as a genre
One of the first infographics on record was published by the London newspaper The Times in 1806 to illustrate or reconstitute a murder (PELTZER, 1993). The infographic is sometimes seen as an aesthetic resource, a substitute for photographs or written texts, a text that arises forjournalistic purposes, but spreads through various areas "whenever it is intended to explain something clearly, mainly when the text alone is not enough to do it in an objective way"11 (TEIXEIRA, 2007, p. 112, our translation).
As Modolo (2007, p. 5, our translation) pointed out, the term infographic originated from informational graphics. The union between text and image results in a "visually attractive [message] to the reader, but with information forcefulness"12. For the author, however, the verbal text subsidizes the image, and not the contrary. For Paiva (2009), in this mode of composition, the diverse semiotic modalities are integrated in a proportional way, i.e., they act concomitantly in the production of meanings, which means that images or verbal texts are not complementary to each other, but essential resources to meaning. Among the graphic-textual resources, there are photographs, illustrations, icons, and maps, using elements that are part of graphic language and a sociocultural context (TEIXEIRA; RINALDI, 2008).
In the dictionary definition, an infographic is a "graphic creation that uses visual resources (drawings, photographs, tables, among others), combined with short texts to present information"13 (RABAÇA; BARBOSA, 2002, p. 238, our translation). Dionisio (2006, p. 139, our translation) identified that this is one of the fastest-growing graphic creations in journalism and its reading "can be done in several ways"14, starting from the verbal text, the imagery text, or the text as a whole. Because it is a fundamentally informative text, the infographic must be autonomous, that is, providing information even without the support of any other text, especially in electronic or hypertextual media, because they can act separately (TEIXEIRA; RINALDI, 2008), being even recontextualized.15
Drawing on Bazerman (2005), infographics are a genre because they are recognizable social facts. This means that the reader recognizes these texts as infographics due to both recurrences in their composition, which defines similarities between different exemplars and its mode of operation and purpose: the infographic necessarily unites verbal text and image to inform, explain or demonstrate something. As Kress (2008, p. 106) stated, genre is not only a linguistic concept. Although it was created by linguistics theorists, it applies to "all forms of representation and communication", which includes texts consisting only of images or the combination of linguistic signs and images.
Considering that genres typify not only texts but social and interactive practices, the situations or practices in which infographics can be constituted are also familiar to readers: infographics appear in newspapers and magazines (as support to other genres, such as news and reports), textbooks, airport signs, instruction manuals and, more recently, on social networking websites. In the latter case, its use has been expanded by the dissemination of information and communication technologies (ICT), since "this language, which privileges images, seems to fit more appropriately to the lifestyle of the population"16 (MODOLO, 2007, p. 4, our translation).
In addition, a genre is recognizable by participants as a particular genre at a point in time, in the reality of interactions. This happens with infographics, recognized due to their specificities, which differentiate them from other texts. This recognition historically processed, as infographics were first produced and published in the journalistic sphere and, soon after, in the most diverse contexts of use, becoming part of the repertoire of genres of every society. Besides the communicative purpose, one of the main specificities of this genre is the combination between verbal text and image, which can be explained based on the concept of multimodality.
Multimodality
Since childhood, people are encouraged to draw, long before they write. Texts used in school are usually illustrated in support to verbal texts or even as a predominant resource in the early years. According to Kress and van Leeuwen (2006), however, in these situations, images are seen as a form of expression, rather than communication. An image should be considered for what it really is: a semiotic system conveying meanings and working autonomously as an image or in relation to a verbal text, in consonance, complement, support, contrast, repudiation or reaffirmation, for example, of what is verbally expressed.
There was a time when, due to the means of communication and production processes limitations, language was understood as verbal language. This contrasts with a contemporaneous view, in which a multiplicity of semiotic resources is not only available but also in focus (KRESS; van LEEUWEN, 2001). Multimodality was defined by Kress and van Leeuwen (2006) as the feature of texts using more than one semiotic code, such as words and images (static or moving). Added to this, nowadays, "writing is in itself a form of visual communication", given the typography and colors used (KRESS; van LEEUWEN, 2006, p. 17). It is about understanding the relevance that forms, icons, drawings, photographs, frames, saturations, and colors, among other visual elements, have for the meaning of a text.
Given the information and communication technologies and the scope these new media have, there is no escape to the analysis and reading of written texts, which has motivated multimodal literacy (or multiliteracies) studies (KRESS; van LEEUWEN, 2001, 2006; KRESS, 2008; ROJO; BARBOSA, 2015). According to Lemke (2010, our translation), "a literacy is always a literacy in some genre and it must be defined with respect to the sign systems deployed, the material technologies involved and the social contexts of production, circulation and use of the particular genre"17. Thus, each genre requires a range of skills, both for its understanding and production.
The theory of multimodality has been developed in the field of social semiotics, which studies the exchange of messages inextricably bound to the context. Hodge and Kress (1988, p. 261) explained that "social semiotics focuses, in the first place, on human semiotics as an inherently social phenomenon in its origins, functions, contexts and effects" and emphasized that all possible semiotic systems are of interest to this field. The choice of signs and the construction of texts are driven by specific, communicative purposes.
With the Design Grammar of Visual Design, Kress and van Leeuwen (2006) provided language studies with a set of categories and options for multimodal text analysis. This grammar is based on Systemic-Functional Grammar, that is, the three metafunctions (meanings) that functionally organize language underlie it: the ideational, interpersonal and textual metafunctions (HALLIDAY, 1985). It is thus considered that "different systemic networks codify different species of meaning linked to different language functions. Grammar, therefore, organizes the options"18, as Figueiredo-Gomes and Souza (2013, p. 288, our translation) observed, regarding Halliday's theory (1985).
The ideational metafunction refers to how experience of the world is organized in language; the interpersonal metafunction encompasses the way the participants interact socially through language (the forms of social distance from the reader, for example); the textual metafunction corresponds to the modes of organization of a text (HALLIDAY, 1985). Kress and van Leeuwen (2006) called these metafunctions representative, interactive and compositional, respectively.
Meanings in texts are constructed simultaneously by these three functions. However, depending on particularities of the analyzed text and genre, one function may predominate over the others. In this study, due to research focus and previous reading of the samples, the analysis will be centered on the compositional metafunction of language.
The Compositional Metafunction of Language
In the compositional metafunction of language, Kress and van Leeuwen (2006) identified categories such as information value, framing, and salience.
The areas of an image determine the value of information, i.e., the way the elements are positioned and distributed in the layout19 is a form of meaning making. Information value can be distributed from left and right, top and bottom, center, and margin (Fig. 1).
According to this diagram, the left side of a multimodal text generally contains already known information, the Given, while the right side presents the New, therefore, it is the zone of prominence in relation to the first. The element located in the center of a multimodal text is the central one, whereas when located at the margins, it is marginal. In addition, there are the lower and upper axes: at the top of a text, the element is the Ideal (essentially an idealized or generalized information and also the most salient part), while, at the bottom, it is the Real, which usually has more specific or practical informative value than the Ideal (KRESS; van LEEUWEN, 2006).
Framing refers to the use of frames or lines that connect or disconnect certain elements of the image, creating meanings of belonging or exclusion between information. The frame contributes to attribute value to information, i.e., elements organized by frames differ from others, and a unit can be established between grouped elements, highlighting to a greater or lesser degree other elements, according to the salience (KRESS; van LEEUWEN, 2006).
Salience is related to the visual weight of elements in a layout, because it is derived from a complex interaction between factors, such as relative size of elements, focus sharpness, contrast of tones, contrast of colors, and location of elements in the piece, perspective, and cultural factors - such as the use of a culturally valuable symbol (van LEEUWEN, 2006).
Research method
After delimiting the topic, the universe of analysis was defined. Pinterest, created in 2010, in the United States, is a social network in which people from various parts of the world collect, organize, publish, and share content related to varied themes and interests. The published content, called pin is necessarily visual: images, photographs, diagrams, graphics, infographics, maps, and illustrations. Image formats supported by the website include still image (PNG, JPG or JPEG) and moving image (GIF). Participants or users are any people who register for free on the social network, through a computer or mobile application (smartphone or tablet). This is how people publish, republish, save, comment on public pins and follow other users. All available texts on the website are found by searching for a specific topic. On the website, there are the profiles of content producers, who generally are ordinary people identified by name or surname, companies, design offices, social networks influencers (for example, bloggers), in short, any individual or company who has registered on the website. However, it is not possible to identify the primary source of content production.
The functioning of this social network is similar to a mural or panel of ideas (for this reason, every user has a board) on varied subjects such as professions, exercises, decoration, design, do it yourself (DIY), recipes, fashion, education, among others. On its home screen, Pinterest is defined as "the worldwide catalogue of ideas" (Fig. 2).
Social networking websites enable diverse response mechanisms, such as like, share, react, comment. As Rojo and Barbosa (2015, p. 123, our translation) state, "these publications/answers are also multimodal: they mix different languages (in addition to verbal, video, audio, static or moving images, etc.)20". In this way, it is a fruitful contemporary universe for the analysis of multimodal and hypertextual texts.
In this research, the words "infographic" and "advertising" were simultaneously inserted in the search field on the Pinterest website in May 2017. Hence, countless21 publications of social network users have appeared. However, three selection criteria were established. The text should: i) contain, in its title or subtitle, the terms "advertising", "design", "brand" or "consumer",22 ii) be written in Brazilian Portuguese; iii) present legible texts (good image resolution for reading). The first five infographics that fitted these criteria were included in the selection (see the five texts in the Appendix), following the order shown in the search results return. Infographics that did not fit any of the criteria were discarded (Chart 1).
The qualitative research started with the exploratory method, with the theoretical deepening of the concept of infographic and the categories of multimodality. Then, the descriptive and interpretative method was used, and the multimodal analysis of the texts was performed; finally, the results were interpreted. In the analysis, formal and functional specificities of infographics and hypertexts were considered, since the collected infographics are virtually propagated, that is, in a social media whose content is published and shared in diverse genres.
The multimodality was analyzed based on the Grammar of Visual Design, by Kress and van Leeuwen (2006), which offers relevant analytical tools for this study. In view of the proposed objectives, the research focused on the compositional metafunction of language, that is, the use of language in the organization of a text. For Kress and van Leeuwen (2006), the analysis of integrated multimodal texts, i.e., texts in which meaning occurs through two semiotic modes simultaneously, as occurs in infographics, shall be performed through three systems: a) information value; b) framing; c) salience, as shown in Chart 2.
In addition, the the analytical categories of linearity and nonlinearity, according to Kress and van Leeuwen (2006), were considered. The reading path possibilities are established by the cited characteristics and, above all, by information hierarchy. This category was also interpreted in light of the concept of hypertext, based on Lévy (1999) and Xavier (2010).
The corpus analysis procedures are systematized as follows: a) previous reading of the infographics; b) observations regarding the purpose of each infographic; c) identification and description of the compositional metafunction multimodal categories; d) considerations about the linearity and nonlinearity of the text; e) interpretation of results.
Multimodality in infographics
This section presents and discusses the results of the multimodal analysis of the infographics, which began by gathering information about the context of publication of the texts. In regards to the context of publication, besides what has already been stated concerning the Pinterest website, infographics are available for access by any person registered in this social network. However, as the verbal texts are written in Brazilian Portuguese, the scope is possibly restricted to individuals who understand or share, at least at some level, this language. Furthermore, as pins can be virtually saved on the social network or on the user's device (notebook, tablet or smartphone), they could, at any time, be recontextualized, that is, taken to another context of language use, including the classroom, as educational objects or multimodal resources, for example. The social network, in this sense, functions as a space for interaction between users through texts, comments and saved pins and, at the same time, as an object repository.
The function of the selected infographics of the advertising sphere corresponds to the synthesis, through verbal and visual resources, of content related to this sphere: C1 summarizes, in a timeline, the history of advertising; C2 clarifies the concept of branding; C3 makes distinctions between printed and digital design; C4 lists 10 types of visual content in which brands could invest; and C5 presents six definitions of neuromarketing that synthesize the consumer mind (Fig 3).
Among the multimodal characteristics, it was observed that all ad pieces present images and verbal texts, with some variation in the amount of visual or verbal information. The shape of the infographics is also variable, the exception being that four of them (C1, C2, C3, and C4) present a much higher height than width. This affects the way these texts are read and is related to the media and social network in which they are inserted: the Internet. The access from mobile devices (smartphone and tablet) presents vertical navigation (displacement made by the user when moving the page or the text). When it comes to the Pinterest application for mobile devices, the verticalization of texts is frequent, since the possible text displacement is from top to bottom, and vice versa. This demonstrates how the characteristics of the context and media used in the production, publication, and reading of infographics have effects on multimodality.
Concerning the compositional metafunction categories, information value in terms of Given and New was analyzed. In C1, for example, just below the title, the year 2000 BC was presented on the left side as a Given, while the year 2011,23 on the right, is the New, which matches the value of information according to Kress and van Leeuwen (2006): previous or already known information is Given, and the most recent or updated information is New. Furthermore, the question "What is advertising?" is Given, and the new information, that is, the answer to the question is New. In the events that form the timeline, the organization of Given and New changes as the position of the visual icon and the year changes. One line presents the icon and the year on the right, while the informed event is on the left; on the next line, the organization reverses, with icon and year on the left and event on the right. Thus, the reader's expectation is broken. At the same time, the visual balance of the multimodal text was favored; otherwise there would be poles with different weights if visual icons were concentrated on one side, whereas verbal texts were on the other side (Fig. 4).
The C3 infographic also presents a division between Given and New, that is, between the left and right sides of the image linked to the meanings of the text:, the characteristics of the printed design are presented as Given, and those of the digital design as New, which is more recent (the novelty) than the printed one. In C2 and C4, few relevant occurrences of the relation between the left and right sides of the ad piece were found, because the messages used a more vertically oriented structure (Ideal and Real).
The C5 infographic (Fig 5) differs from the other infographics in regards to the organization of the information value, because it uses, mostly, the center-margin system, which enables the orbital reading mode. In the center of the text is the illustration of a human brain and, from this image, with the support of a framing structure, verbal information is placed on the margins. The visual representation of the brain in the highest information value position contributes to draw the reader's attention to the textual topic. The center of the text is the brain, and secondary information is displayed from this representation, that is, its six characteristics are provided.
By analyzing the lower and upper axes, it was observed that, in the five pieces, the title and, when available, the subtitle are located at the top of each infographic. The value of this information in the Ideal position is higher than the value of the information at the bottom, the Real (KRESS; van LEEUWEN, 2006). The Real, in C1, presents the "Stay tuned!" imperative, the signature of the person who made the ad piece and the address of the blog that translated it into Portuguese; in C2, sources of references used, accompanied by a set of icons that form a working table; in C3, there is the question "Do you want to interact with the graphic version of this material? Print the infographic on the link below", followed by the logo of the person who made the ad piece; in C4, the name and email address of the person who created it, and the logo ; in C5, the question "Do you want to know more about [...]?" and the email addresses of the person who authored the text.
In intermediate position, that is, between the Ideal and the Real, there is the information that composes the multimodal content of the infographics (verbal texts and images that relate to the topic). From the point of view of Real and Ideal, information located at the bottom of infographics is, as advocated by Kress and van Leeuwen (2006), linked to a practical function: in this case, the references consulted, the authorship or the invitation and address to access more information.
As for framing, in C1, there are textbox frames framing subtitles and blocks of verbal text. The boxes are differentiated by colors and present equivalent format for information of the same category (for the pre-industrial period, it was used purple, whereas, for the nineteenth century, red was used, for example), i.e., the color choice and frames were made in order to organize or categorize the message content. Framing frames the milestones in each season, reinforcing the idea of a timeline. In C2, the framing occurred through boxes filled with different colors for each information category (backgrounds). The use of indicative lines to categorize information and connect small blocks of verbal text to images with symbolic meaning were identified. For instance, mission points to the head of the represented man, vision points to one of his eyes, positioning points to the mouth, and essence points to his chest or heart (Fig 6). In C3, the same strategy is used, i.e., boxes of different colors for each information, except that there are other organizing frames and connecting lines within the colored box, as shown in Fig. 6 (keys, rectangular and circular frames, and dotted lines). The function of this resource in the infographic is to organize and segment information, especially those placed side by side, to confront them, reinforcing the comparisons and distinctions between printed design and digital design.
The C4 infographic uses the same framing feature as C3 and C2: distinct color backgrounds (color alternation) for particular categories of information and lines for separation and connection of elements. In this case, the lines connect visual icons and verbal texts (Fig. 7), ratifying the informative and demonstrative objective of the text, to indicate types of visual content for the brand.
C5, in its turn, has the framing in a vertical line separating the visual icon from the title and subtitle, as well as uses a black rectangle to organize the background for an ad at the bottom. However, the frame that separates each of the six characteristics of the consumer's brain has a particular format in relation to the other analyzed samples, because of its undefined shape: it is a brushstroke or a stain that works as a background for the verbal text blocks. In addition, the brush strokes are in distinct colors for each information, which contributes to the differentiation of the reported characteristics. This fluid format of framing is linked to the textual topic: neuromarketing and the consumer mind.
In salience, the visual weight of elements in a layout, the occurrences indicated in Table 3 were found.
The choice of highlighting these elements in relation to others within the text can be explained by the need to establish a visual hierarchy that indicates points of greatest attraction in the multimodal message. It is these points that contribute to establishing, as far as possible, a reading path.
The reading paths of the C1, C2, C3, and C4 infographics are initially vertical, i.e., the infographic is read from top to bottom. In a second moment, considering each framing in the pieces that present it, as a unity of meaning, there are alternative paths of reading that are established by the position of the verbal and imagery elements, either in Given-New or Ideal-Real terms, or even, from the center to the margins. In the frames, the reading paths were indicated, as shown in Fig. 8.
The case of the C5 infographic differs from the previous ones, a fact that is already observed in the image, less vertical (or less high) than the other samples - the exception being C4, in which a reading path is observed from the center to the margins, similar to C5. Besides having a more harmonic format regarding the relation between width and height, C5 is organized on the center-margin system, described by Kress and van Leeuwen (2006). For this reason, its reading takes place on a particular path: from the center to the margins. In addition, it was found that, once the title and subtitle were read, and the central figure was visualized, it does not matter which marginal element will be read first, because there is no required sequence for reading the message (Fig 9).
Additionally, the visual weight of the elements contributes to establishing whether the function of the images is to support a verbal text or centralize the main information and, in the reverse sense, having the verbal text as their support. In other words, based on the most salient elements, the role of images in these infographics can be perceived. Among what was found with the analysis of the texts, it was pointed out that the verbal titles are always salient. When there is an image closely associated with the title, this image also presents salience. The images (predominantly illustrations, icons, and some symbols) have salience, but are not self-explanatory, because they require verbal text to build the message and add meaning. In other words, the infographics of the analyzed sample do not work without one of its modalities, since it is a multimodal resource in which both verbal and imagetic components are mutually supportive. In the same way, without icons, illustrations, symbols, and, mainly, without the use of framing, there would be no infographic, but rather a linear verbal text.
Final considerations
The multimodal analysis combines useful categories to describe images and their functioning when conjoined with verbal texts. Of the categories proposed in the Grammar of Visual Design by Kress and van Leeuwen, those related to the compositional metafunction of language were investigated: information value; framing and salience. Regarding the information areas, it was found that there were recurrences regarding the division between Given and New and Ideal and Real in almost all infographics. In this sense, an infographic that showed center-margin system predominance was highlighted. In relation to framing, its fundamental importance to categorize, classify, and even rank information in the messages was detected. Salience, in its turn, occurs by the combination of several multimodal elements, such as size, color, texture, position, cultural factors associated with images, among others.
The process of nonlinear reading of the advertising infographic, a genre that becomes a hypertext in the cyberspace, integrating cyberculture, makes it a "self-service" reading, as well defined by Xavier (2010). In the analyzed infographics, there is not only one focus but also a variable number of multimodal elements that acquire diverse sizes, colors, and communicative functions in a visual configuration, which contributes to the creation of different degrees of salience. Reading is done from the signifiers that are emphasized in the ad pieces, thus, producing meanings.
Text reading paths are predominant in the vertical direction, from top to bottom, explained by the media or technological devices in which they are generally consumed, i.e., the applications accessed through a smartphone. In this sense, the infographic whose reading path is orbital and whose format presents lower height than the other texts is differentiated; after all, this text can be accessed and read without the displacement of the page or screen by the reader, both on the smartphone and the computer.
Finally, for a future study, the development of an analysis of the other language metafunctions, as well as the expansion of the corpus of study are intended.
KNOLL, G.; FUZER, C. Análise de infográficos da esfera publicitaria: multimodalidade e metafunçâo composicional. Alfa, Säo Paulo, v. 63, n.3, p.589-612, 2019.
1 The concept of spheres of human activity, fields or spheres of language use, is based on Bakhtin (2010) who mentions these spheres repeatedly when dealing with genres. From the Bakhtinian dialogical theory, Sobral (2009, p. 121, our translation) explains that the spheres are "a socio-historical, ideological cutout of the world, [...] endowed with a greater or lesser degree of stabilization, depending on their degree of formalization or institutionalization, within society and history, according to specific conjunctures". Each sphere has its particularities regarding the production, reception, and circulation of discourses; however, they are not polarized, static or watertight, since one sphere can dialogue with others, exchanging characteristics, contents and forms of functioning, as occurs in the advertising and journalistic spheres.
2 The compositional metafunction, by Kress and van Leeuwen (2006), consists of the transposition or adaptation of the textual metafunction of Systemic-Functional Linguistics, proposed by Halliday (1985). While the textual metafunction encompasses the modes of organization and composition of the message, with a focus on the linguistic text, the compositional metafunction emphasizes the modes of organization and composition of multimodal texts.
3Inference based on the specificity of the texts published and shared there, because the social network has restrictions to certain formats and image sizes.
4As Xavier (2010, p. 207, our translation) put it, reading is a "process of co-production of meaning of texts and hypertexts." Hypertext is defined as the unit "hybrid, dynamic and flexible language [unit] that dialogues with other semiotic interfaces, adds and conditions to its surface other forms of textuality" (XAVIER, 2010, p. 208, our translation). In this language technology, the presence of sounds, graphics, diagrams, hyperlinks occurs, all gathered in the same message. Thus, according to the author, the reading of hypertexts require more than simply decoding, because it involves a nonlinear, hyperconnected, and multidimensional path.
5 In the original: "surge da interconexao mundial dos computadores" (LÉVY, 1999, p. 17)
6 In the original: "mas um oceano de informaçoes" (LEVY, 1999, p. 17)
7 In the original: "os generos discursivos incluem toda sorte de diálogos cotidianos, bem como enunciaçoes da vida pública, institucional, artística, científica efilosófica" (MACHADO, 2005, p. 155)
8 Mode refers to the semiotic mode or representation mode used in the composition of a text.
9 In the original: "(a) materializam uma classe de eventos; (b) compartilham propósitos comunicativos; (c) possuem traços específicos prototípicos; (d) apresentam lógica inata; e (e) determinam usos lingüísticos específicos de acordo com a comunidade discursiva" (PEREIRA; RODRIGUES, 2009, p. 6)
10 In the original: "emergem nos processos sociais em que pessoas tentam compreender umas as outras suficientemente bem para coordenar atividades e compartilhar significados com vistas a seus propósitos práticos" (BAZERMAN, 2005, p. 31)
11 In the original: "sempre que se pretende explicar algo, de uma forma clara e, sobretudo, quando só o texto nao é suficiente para faze-lo de maneira objetiva" (TEIXEIRA, 2007, p. 112)
12 In the original: "visualmente atraente para o leitor, mas com contundencia de informaçao" (MODOLO, 2007, p. 5)
13 In the original: "criaçao gráfica que utiliza recursos visuais (desenhos, fotograbas, tabelas etc.), conjugados a textos curtos para apresentar informaçoes" (RABAÇA; BARBOSA, 2002, p. 238)
14 In the original: "a leitura de um genero textual que contém infográfico pode ser realizada de várias formas" (DIONISIO, 2006, p. 139)
15 Recontextualization means taking a text out of its context of primary production and reinserting it in a secondary context, through an articulation between social practices (MOTTA-ROTH; LOVATO, 2011). For example, it is a practice of recontextualizing to remove an image from the Pinterest social network and take it to a classroom, to an Internet blog or to a textbook.
16In the original: "essa linguagem, que dá preferencia á imagem, parece encaixar-se mais adequadamente ao estilo de vida dapopulaçao" (MÓDOLO, 2007, p. 4)
17 In the original: "um letramento é sempre um letramento em algum genero e deve ser definido com respeito aos sistemas sígnicos empregados, as tecnologías materiais usadas e aos contextos sočiais de produçao, circulaçao e uso de um genero particular" (LEMKE, 2010).
18 In the original: "as diferentes redes sistemicas codificam diferentes espécies de significado que se ligam ás diferentes funçoes da linguagem. A gramática, portanto, organiza as opçoes [de escolha]" (FIGUEIREDO-GOMES; SOUZA, 2013, p. 288)
19Layout is the visual arrangement of a graphic text (RABAÇA; BARBOSA, 2002).
20 In the original: "essas publicaçoes/respostas também podem ser multimodais: podem misturar diferentes linguagens (para além da verbal, vídeos, áudios, imagens de diferentes tipos, estáticas ou em movimento etc.)" (ROJO; BARBOSA, 2015, p. 123)
21 Countless because the site does not display the number of texts obtained through the search.
22 Thus, images that possibly mistakenly categorized advertising content were avoided in the corpus of analysis.
23Since the timeline runs until 2011, it is understood that this was the year in which the C1 infographic was produced.
24 The infographics were displayed in gray to highlight the route.
25 The infographics were displayed in gray to highlight the route.
REFERENCES
BAKHTIN, M. M. Estética da criaçâo verbal. Trad. do russo: Paulo Bezerra. 5. ed. Säo Paulo: WMF Martins Fontes, 2010.
BARTON, D.; LEE, C. Linguagem online: textos e práticas digitais. Säo Paulo: Parábola, 2015.
BAZERMAN, C. Generos textuais, tipificaçâo e interaçâo. Organizado e traduzido por: Angela Paiva Dionisio e Judith Chambliss Hoffnagel. Sāo Paulo: Cortez, 2005.
BRAIT, B. Olhar e ver: verbo-visualidade em perspectiva dialógica. Bakhtiniana, Säo Paulo, v. 8, n. 2, p. 43-66, 2013.
DIONÍSIO, A. P. Generos multimodais e letramento. In: KARWOSKI, A. M. et al. (org.). Generos textuais: reflexőes e ensino. 2. ed. Rio de Janeiro: Lucerna, 2006. p. 119-132.
FIGUEIREDO-GOMES, J. B.; SOUZA, M. M. de. Gramaticalizaçâo e metafunçöes da linguagem: uma análise da expressäo é que. D.E.L.T.A., Säo Paulo, v. 29, n. 2, p. 281-310, 2013. Disponível em: http://www.scielo.br/pdf/delta/v29n2/05.pdf. Acesso em: 10 out. 2019.
FIORIN, J. L. Interdiscursividade e Intertextualidade. In: BRAIT, B. (org.). Bakhtin: outros conceitos-chave. Säo Paulo: Contexto, 2006. p. 161-193.
GEHRKE, N. A. Foto do dia ou microcrônica verbo-visual: um genero na perspectiva da Escola de Sidney. 2015. 212 f. Tese (Doutorado em Letras) - Universidade Federal de Santa Maria, Santa Maria, 2015.
HALLIDAY, M. A. K. An Introduction to Functional Grammar. Baltimore: Edward Arnold, 1985.
HODGE, R.; KRESS, G. Social Semiotics. New York: Cornell University Press, 1988.
KLEIMAN, Â. B. Leitura: Ensino e pesquisa. Säo Paulo: Pontes, 2004.
KNOLL, G. F. Multimodalidade, texto e contexto: categorias úteis a análise da charge. (Con)textos Linguísticos, Vitória, v. 9, n. 12, p. 57-74, 2015.
KRESS, G. Literacy in the New Media Age. London; New York: Routledge, 2008.
KRESS, G.; van LEEUWEN, T. Reading Imagens: the grammar of visual design. London: Routledge, 2006.
KRESS, G.; van LEEUWEN, T. Multimodal discourse: the modes and media of contemporary communication. London: Arnold, 2001.
LEMKE, J. L. Letramento metamidiático: transformando significados e mídias. Trabalhos em Linguística Aplicada, Campinas, v. 49, n. 2, s.p., jul./dez. 2010. Disponível em: http://www.scielo.br/scielo.php?pid=S0103-18132010000200009&script=sci_ abstract&tlng=pt. Acesso em: 10 out. 2019.
LÉVY, P. Cibercultura. Säo Paulo: Ed. 34, 1999.
MACHADO, I. Generos discursivos. In: BRAIT, B. (org.). Bakhtin: conceitos-chave. Säo Paulo: Contexto, 2005. p.151-167.
MÓDOLO, C. M. Infográficos: características, conceitos e principios básicos. In: CONGRESSO BRASILEIRO DE CIENCIAS DA COMUNICAÇÂO DA REGIÄO SUDESTE, 12., 2007, Juiz de Fora. Anais [...] Juiz de Fora: Intercom, 2007. Disponível em: http://www.intercom.org.br/papers/regionais/sudeste2007/resumos/R0586-1.pdf. Acesso em: 10 abr. 2019.
MOTTA-ROTH, D.; LOVATO, C. O poder hegemônico da ciencia no discurso de popularizaçâo científica. Calidospópio, Säo Leopoldo, v. 9, n. 3, p. 251-268, set./dez. 2011.
PAIVA, F. A. A leitura de infográficos da Revista Superinteressante: procedimentos de leitura e compreensäo. 2009. 204 f. Dissertaçâo (Mestrado em Estudos Lingüísticos) - Universidade Federal de Minas Gerais, Belo Horizonte, 2009.
PELTZER, G. Jornalismo iconográfico. Lisboa: Planeta, 1993.
PEREIRA, R. A.; RODRIGUES, R. H. Perspectivas atuais sobre generos do discurso no campo da Lingüística. Letra Magna, Säo Paulo, ano 5, n. 11, p. 1-18, 2009.
RABAÇA, C. A.; BARBOSA, G. Dicionário de comunicaçâo. Säo Paulo: Ática, 2002.
ROJO, R.; BARBOSA, J. P. Hipermodernidade, multiletramentos e generos discursivos. Säo Paulo: Parábola, 2015.
SOBRAL, A. Do dialogismo ao genero: as bases do pensamento do círculo de Bakhtin. Campinas: Mercado de Letras, 2009.
TEIXEIRA, T. A presença da infografia no jornalismo brasileiro: proposta de tipologia e classificaçâo como genero jornalístico a partir de um estudo de caso. Fronteiras: estudos midiáticos, Säo Leopoldo, v. 9, n. 2, p. 111-120, maio/ago 2007. Disponível em: http:// revistas.unisinos.br/index.php/fronteiras/article/view/5847. Acesso em: 10 abr. 2019.
TEIXEIRA, T; RINALDI, M. Promessas para o futuro: as características do infográfico no ciberjornalismo a partir de um estudo exploratorio. In: ENCONTRO NACIONAL DE PESQUISADORES EM JORNALISMO, 6., 2008. Säo Paulo. Anais [...] Säo Paulo: UMESP/SBPJor, 2008. Disponível em: http://ddiprojeto2.xpg.uol.com.br/ as_caracteristicas_do_infografico_no_ciberjomalismo.pdf. Acesso em: 24 mar. 2019.
van LEEUWEN, T. Introducing social semiotics. London; New York: Routledge, 2006.
XAVIER, A. C. Leitura, texto e hipertexto. In: MARCUSCHI, L. A.; XAVIER, A. C. (org.). Hipertexto e generos digitais: novas formas de constr^äo de sentido. 3. ed. Säo Paulo: Cortez, 2010. p. 207-220.
Received on December 27, 2017
Approved on February 3, 2019
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
© 2019. This work is published under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Abstract
Multimodality, which is characteristic of texts that gather two or more semiotic codes (KRESS; van LEEUWEN, 2006), requires an approach that emphasizes the verbalvisuality (BRAIT, 2013) of these compositions. The aim of this study is to analyze categories of the compositional metafunction (KRESS; van LEEUWEN, 2006) from the perspective of multimodality in infographics of the advertising sphere, with the purpose of understanding how the selected messages, configured in infographics, are organized. In order to do so, it is necessary to: (i) characterize the selected infographics based on categories of the compositional metafunction; (ii) understand how the multimodal and functional aspects relate to each other in these texts; (iii) investigate how the multimodal aspects act in the production of meaning. The sample consisted of five infographics from the advertising area selected from the Pinterest website. The multimodal analysis of the texts is carried out from categories of the compositional metafunction, from the Grammar of Visual Design (KRESS; van LEEUWEN, 2006). The analysis shows that there is not only a visual focus but also a variable number of multimodal elements that, with their different sizes, colors, and communicative functions, contribute to the creation of various degrees of salience. Reading is done from the signifiers that stand out the most in the ad pieces, thus producing meanings.