MUSIC AND MOVEMENT IN PIXAR: THE TSU's AS AN ANALYTICAL RESOURCE

Abstract

The music for the animation cinema is closely linked with the characters' movement and the narrative action. This paper presents the Temporary Semiotic Units (TSU's) proposed by Delalande, as a multimodal tool for the music analysis of the actions in cartoons, following the tradition of the Mickey Mousing. For this, a profile with the applicability of the nineteen TSU's was applied to the fourteen Pixar movies produced between 1995-2013. The results allow us to state the convenience of the use of the TSU's for the music comprehension in these films, especially in regard to the subject matter and the characterization of the characters and as a support to the visual narrative of this genre.

Headnote

ABSTRACT

The music for the animation cinema is closely linked with the characters' movement and the narrative action. This paper presents the Temporary Semiotic Units (TSU's) proposed by Delalande, as a multimodal tool for the music analysis of the actions in cartoons, following the tradition of the Mickey Mousing. For this, a profile with the applicability of the nineteen TSU's was applied to the fourteen Pixar movies produced between 1995-2013. The results allow us to state the convenience of the use of the TSU's for the music comprehension in these films, especially in regard to the subject matter and the characterization of the characters and as a support to the visual narrative of this genre.

KEY WORDS

Animation cinema, Music - Pixar - Temporary Semiotic Units - Mickey Mousing - Audiovisual Narrative - Multimodality

MÚSICA Y MOVIMIENTO EN PIXAR: LAS UST COMO RECURSO ANALÍTICO

RESUMEN

La música para el cine de animación está estrechamente ligada con el movimiento de los personajes y la acción narrativa. Este artículo presenta las Unidades Semióticas Temporales (UST) propuestas por Delalande, como una herramienta multimodal para el análisis musical de las acciones que suceden en las películas de dibujos animados, siguiendo la tradición del Mickey Mousing. Para ello se ha aplicado una ficha con las diecinueve UST a los catorce largometrajes de Pixar producidos entre 1995-2013. Los resultados nos permiten plantear la conveniencia del uso de las UST para la comprensión de la música en estos productos, especialmente en lo que respecta a la temática y caracterización de los personajes, y como apoyo a la narrativa audiovisual de este género.

PALABRAS CLAVE

Cine de animación - Música - Pixar - Unidades Semióticas Temporales - Mickey Mousing - Narrativa Audiovisual - Multimodalidad

1.INTRODUCTION

Animated cinema, specifically the cartoons are a little studied genre, despite its broad social acceptance at all ages. If we also refer to its music, research in this area have been almost nil compared to the influence that it has, in the film itself, as in the collective memory (Arostegui, 2009). In this context, the films produced by Pixar Animation Studios figure prominently in the film market of this kind. Since 1995, this company, -which initially belonged since 1986 to Lucasfilm Ltd. and in 2006 became part of The Walt Disney Company-, has released fourteen high-quality films, both for their screenplays, as for its visual and musical performance being worthy of twelve Academy Awards, four of which reward their musical quality and sound.

This article presents an analysis of the music present in the fourteen Pixar films between 1995 and 2013, by means of the application of the so-called Temporary Semiotics Units (TSU), a multimodal resource inspired in the Mickey Mousing that allows us to identify and describe the this musical content in these films, where the movement of the characters is fundamental in the audiovisual narrative.

Thus, through this research we considered the feasibility of TSU as a tool to analyze the music in the cartoons, performing a detailed account and comparison of their presence of them at Pixar, and their possible involvement with the subject matter, atmosphere or characters of each film. In the same way, and when knowing the expressive differences between each one of the TSU, we consider their suitability to reflect the music of each scene and its congruence with the audiovisual narrative.

1.1 Multimodality and animation cinema

Animation cinema and cartoons in particular, are an audiovisual production aimed at children par excellence, where we find a strong relationship with the traditional story (Valvassori, 2006). As in literary narrative, animated cinema is usually structured through the three classic acts of the plot: presentation, exposition and denouement (Duran, 2009). Both sound and picture are subject to this tradition, supporting the emotions and actions of the characters in a substantive and diegetic way (Torello & Duran, 2014), especially in products aimed preferably at the child public, although at present, that public it is no longer only child, so that the message has been adapted to reach and captivate viewers of all ages (Aróstegui).

In the case of cartoons, multimodality, understood as the simultaneity of more than one form of semiotic expression (Kress & Van Leeuwen, 2001), takes on a special relevance, as it aims to stimulate the different perceptual channels of the public through the combination of music, dialogues, frenzied movements, sudden sound effects, etc. (López-Cozar, Callejas & Gea, 2005). So, to understand the audiovisual product, we must analyze it in a multimodal key, both from the understanding of each of its elements, and their interactions. For this we must be aware that the co-articulation has the potential to generate a greater synergy than the sum of each of its individual meanings (Lemke, 1998). In that multiple articulation, the intersemiotic complementarity proposes us to analyze sound and visual modes together (Royce, 1998).

This leads us to recognize some commonly used analogies in all audiovisual and especially in animated film, such as tempo changes associated with the movement of the characters, frequency changes with the ascent or descent in space, the intensity changes with distance changes, etc. (Gustems & Calderon, 2014). In this sense, it is recalled that the music of cartoons, beyond emotionality, has a function of kinesthetic vehicle, accompanying and even emphasizing the action (Whalen, 2004), so, as proposed by Chion (1994), the sublime incorporation of sound in the space of animation is produced.

1.2.: The mickey mousing movement and the TSU

In general, the cartoons are strongly characterized by the movement that is given to each of the characters involved (Jones, 1946). In this sense, and as described Radigales (2001), the soundtrack becomes a crucial element in giving credibility through the strengthening of gestures and movements mentioned. Such reinforcement, produced by the congruency and synchrony between sound action and reaction, has one of its greatest exponents in the resource named Mickey Mousing (Cohen, 2013).

Beyond the early days of animation in which some pianists accompanied silent stories seeking synchrony through sound imitation of what was happening on the screen (Neumeyer & Buhler, 2001), the Mickey Mousing transformed the way we understand the cartoons, allowing the physical space to revolve around the whims of the protagonist in each scene, relegating to the background the ambient sounds and focusing on how the fictional universe created accurding to each character would sound (Whalen, 2004). Thus, the interaction of diegetic and non-diegetic sounds will be loaded with intentionality and subjectivity (Bruner, 1986).

The Mickey Mousing concept has evolved since that Steamboat Willie (1928) starring the character that gave name to the association between sound and movement (Rosenberg, 1983), making more flexible the total synchrony between both. For this reason, since, the close relationship developed between The Walt Disney Company film director and the composer of the music still remains. So now they devote many hours to planning the animation along with the writing of the music before doing any actual production (Farrar, 2003). From this relationship described by Farrar, an alternative form that fused the musical notation and animation emerged: the bar sheet or dope sheet. Through this conception the processing of visual and sound elements has reached in a holistic manner, and can affirm the existence of "a whole" narrative-perceptive.

Also, to talk about Mickey Mousing takes us to the field of dance, understanding it from the metaphor of gestures produced by a character depending on the ambient sound (Duerden, 2007), which allows us to establish a direct association between this character and its characterization through its movements (Smith, 2000) (that is, the protagonist "is" as "how it moves"). Thus, understanding the multiple connections between music and dance, music and Mickey Mousing, gesture and dance, etc. we only establish a relationship between each character and the sound that accompanies it, or what is the same, each character and the movement it makes on the screen.

Thus, to understand the movement and sound as transcendental parts of the lifestyle of a cartoon, a study and a combined analysis are essential in film animation. In this regard we find the proposal of the Temporary Semiotics Units (TSU) as an indispensable tool to show the analogies so far described. Such TSU were developed by the teacher and composer François Delalande and his research group at the Laboratoire Musique de Informatique de Marseille (MIM, 1996, 2002) and are presented as a useful system for understanding and expressing figurative correspondences between music and movement. Although originally allowed to describe music devoid of score, as Delalande (1996) itself argued, these units are based on the need to introduce a significant kinetic aspect in the description of the sound elements. Therefore, they are presented to us as an alternative to the traditional writing and analysis of sound fragments used in the audiovisuals, especially in cartoons, which give more importance to elements such as attacks, phrasing, instrumental density, transparency, etc. that to the issues addressed in the traditional analysis of music: harmony, melody and rhythm.

The TSU are defined as a construct with which it is intended to describe the temporal significance of a short musical fragment, according to their morphological and kinetic organization (Hautbois, 2010); these units or segments must maintain their meaning and significance once isolated from their context and produce a similar effect in various contexts or, in our case, in different situational contexts of cartoons. The properties that are considered necessary to describe the TSU are of two types: morphological and semantic (MIM, 2002). The morphological characteristics include duration, repetition, phases, stability of matter, acceleration and temporal development; while semantic include the direction, the movement and the energy. The lexicon used to define it turns out to the metaphorical resource.

So far nineteen TSU have been described divided into two sections (Alcazar, Gustems & Calderon, 2014), on the one hand the TSU not limited in time, which would be:

* In flotation: slow onset of events on a smooth continuum; without sense of waiting or "suspense".

* In suspension: balance of forces which produces a feeling of immobility; feeling of indecisive waiting: something will come but it is not known when or what.

* Heaviness: slowness, difficulty to move forward.

* Obsession: insistent character, mechanical process of constant repetition.

* In waves: impression of being propelled forward, cycles that are restarted.

* That progresses: it takes us forward on a regular basis, advances forward decisively.

* That rotates: animated by a rotational movement, lack of progression.

* That wants to start: it tries to get going, intend to do something.

* Without direction by divergence of information: multiplicity of directions without apparent links causing indecision.

* Undirected by information excess: multiple elements, impression of confusion, apparent independence of the elements.

* Stationary: impression of being immobile without feeling of waiting.

* Inexorable trajectory: the end is not expected, the advance, the descent does not end -.

On the other hand, the TSU limited in time:

* Fall: unstable equilibrium that is broken, lost potential energy that is converted into kinetic energy.

* Contraction-extension: feeling of compression that yields afterward suppressing the resistance and relaxing.

* Impulse: application of a force from a state of equilibrium that causes an acceleration; projection starting from a support.

* Stretching: go to the maximum of a process; elongation subject to tension.

* Braking: Forced slowdowns, sudden retention.

* Suspension-interrogation: movement that is interrupted in a waiting position.

* Inertia: progress thanks to the speed acquired; predictability of the development up to its extinction.

Since its inception, the TSU were shown as a result of cooperative work between researchers, musicians, artists and composers, particularly highlighting their interactions with the visual field. Dance has also explored the TSU as a description of the choreographic gesture, so its use in the study and analysis of the characteristic gesture of cartoon characters is shown entirely relevant.

2.METHODOLOGY

Our study is based on interpretive paradigm starting from the collection of quantitative data. To collect these data fourteen Pixar films released between 1995 and 2013 were analyzed (Toy Story, A Bugs Life, Toy Story 2, Monsters Inc., Finding Nemo, The Incredibles, Cars, Ratatouille, Wall-E, Up, Toy Story 3, Cars 2, Brave, Monsters University) representing a total length of approximately 1405 minutes. Each of these films has been applied an ad hoc analysis sheet of double entry, in which nineteen TSU are placed on the vertical axis and a cut made by minutes on the horizontal axis. To complete the record, the first ten seconds of each minute are displayed (ie, from 46:00 to 46:10 minutes, from 47:00 to 47:10 minute minute, etc.), analyzing the music that started or ended in that temporal cut just to determine the TSU present, and they may even occur more than one at a time. In the event that there were no music it has been pointed out shading all the boxes corresponding to the temporary cut. In the case of the songs, they have also been analyzed as ambient music, choosing only the ten seconds discussed above.

The application of the analysis records of TSU to the fourteen feature films took place throughout the months of October and November 2014, and for it was counted by two experts who completed the records separately. Once finished, they got together and in the places where there was no inter-judges agreement, the occasional participation of a third judge who settled in favor of one or another proposal of TSU. Thus it came to the results presented in this paper.

Then, as an example, we show the first twelve minutes of the records made, corresponding to the film Toy Story (1993) (see Figure 1).

3.AS A CONCLUSION

The results of our analysis shows a total of 2021 TSU in fourteen films, representing an average of 1.4 TSU by each of the 1405 cuts analyzed. The following table shows the distribution of the different TSU depending on each of the films. To do this, we have identified each of the films with the first letter of the title on the horizontal axis, so that, for example, Finding Nemo (2003) will be identified as "FN" and all movies are listed in order according to the year in which they were released. For reasons of space each of the TSU has been replaced by a serial number, as shown in Table 1, so for example the TSU "heaviness" corresponds to the number "3". On the same axis we add a row dedicated to the total number of TSU in each feature film (T), and another row dedicated to spaces without music in each film (EWM). On the vertical axis a column with total times each TSU (TO) have been included (see Figure 2).

As we can see, all films are capable of being analyzed by the TSU since they appear in all of them and substantially (1'4 TSU on average, by cutting), so we can say that the TSU can be a means of analysis of the sound in the animated film sample analyzed. Specifically, some movies like Wall-E or Toy Story 3 the use extensively (215 and 190 times, respectively), while others, like Monsters Inc. or Up do so in a less generalized way, probably because of much greater use of dialogues against the need to encourage sound characters of those films. Despite these differences, we do not appreciate a statistical significance in the chronological evolution of data over the fourteen films analyzed, confirming the thesis that the business situation of Pixar at the beginning of the studied period and in the end would not have affected the construction of musical discourse regarding the movement, as they have been analyzed with TSU.

However, not all Semiotics Units are equally present in all the films, there are major differences in this regard. So, if we make a global calculation, we see that the TSU that appears most is the one called "advancing", with a total of 265 times; followed by "suspended", which appears in 195 times. The TSU "floating" and "waves" can be found 158 and 153 times respectively. The fifth most used TSU, and in turn, the first in the category of "bounded in time" is the "momentum" with 150 appearances. It's no wonder the importance given to these TSU that match the basic forward motion of the action in cartoons, especially escalations shares representing themselves as waves. It is also noteworthy interest also acoustically illustrate fluctuating movements such as those made in the water and in the air; in this sense "float" or "suspend" sound analogies can be of great interest and novelty. By contrast, the TSU that appear least correspond to "unaddressed for too much information" and "undirected by divergence of information" with only 31 and 25 times. The scarcity of such actions could be related to the usual directionality of movement in children's animation films where the characters move with clear intentions and without hesitation. In this sense, the sound would favor the smaller to apprehend the intrinsic characteristics of the characters and their relationship with their action in an integrated way.

Regarding the absence of music, 187 spaces without TSU were detected, which would represent 13.3% of total 1405 minutes analyzed. The films in which more silences appear are Cars (28) and The Incredibles (26), and the ones with least are Toy Story 3 (4) and A Bug's Life (6). The overall percentage found is substantial and suggests that their use is intentional and is due to the creation of expectations, to some perceptive ecology and narrative articulation in sections, being an essential part of the resources of audiovisuals (Marimon, 2014).

Another objective of this work was to find the relationship between the subject matter of each film and the preferential use of the TSU. In that regard it is noteworthy some coincidences that seem really significant: the TSU most frequently punctuated in Finding Nemo is "floating" (21 times), probably due to water scenes; In The Incredibles it is " in suspension" (18 times), by the air jumps that are made by its characters; while in Cars it is "advancing" (24 times) for its car racing subject matter. These figures show us how the visual effects of movements in each scene, are directly related to the use of certain sound formulas collected in the TSU.

As to whether or not there are significant differences between the TSU limited in time and not limited in time, the data show that there are no such differences, and its use is indistinct, leading us to believe that in the film editing of these films, music has both a function of continuity between scenes as delimitation (Llinares, 2012).

Regarding the use of the TSU to underscore the narrative aspects of the films included the increased presence of TSU in the second acts primarily intended for plot development, and therefore with much more action than in the first acts, which presented the characters and situations which contain many more dialogues. These would be, for example cases of Toy Story 2 and Toy Story, respectively.

As a prospective study we must point out the need to compare the data obtained with those of any other company of the audiovisual sector, as well as other audiovisual products for all ages such as video games. We would also note the appropriateness of comparing the results obtained by TSU with the strictly visual content, a link in which multimodality is based and ultimately, is the essence of audiovisual art.

References

6. REFERENCES

Complete books:

Alcazar, A. J.; Gustems, J. & Calderón, D. (2014). Los modos de escucha como generadores de pensamiento musical: a propósito de François Delalande, en Observar, n ° 8. Págs. 86-108.

Aróstegui, J. L. (2009). La música en los dibujos animados, en Olarte, M. (Ed.). Reflexiones en torno a la música y la imagen desde la Musicología española (553-563). Salamanca: Plaza Universitaria.

Bruner, J. (1986). Actual Minds, Possible Worlds. Cambridge: Harvard University Press.

Callejas, Z.; López-Cózar, R. & Gea, F. M. (2005). Análisis de Metodologías de evaluación de Sistemas de Diálogo Multimodal, en Procesamiento del lenguaje natural, n° 34. Págs. 9-16.

Chion, M. (1993). La audiovisión. Introducción a un análisis conjunto de imagen y sonido. Barcelona: Paidós.

Cohen, A. J. (2013). Film Music and the Unfolding Narrative, en Strüngmann, M. A. (Ed.). Language, Music, and the Brain (173-200). Cambridge: MIT Press, 173-200.

Delalande, F. (1996). Les Unités Sémiotiques Temporelles: problématique et essai de défnition, en MIM. Les Unités Sémiotiques Temporelles (16-25). Marsella: MIM.

Duerden, R. (2007). Dancing in the Imagined Space of Music, en Dance Research, n° 25, vol. 1. Págs. 73-83.

Duran, J. (2009). Narrativa audiovisual i cinema d'animació per ordenador. (Tesis doctoral). Universitat de Barcelona, Barcelona.

Gustems, J. & Calderón, C. (2014). El análisis multimodal en la escucha de los audiovisuales, en Gustems, J. (Ed.). Música y Audición en los Géneros Audiovisuales (15-28). Barcelona: Publicacions i Edicions de la Universitat de Barcelona.

Kress, G. & Van Leeuwen, T. (2001). Multimodal discourse. The modes and media of contemporary Communication. Londres: Edward Arnold.

Lemke, J. (1998). Multiplying Meaning: Visual and Verbal Semiotics in Scientific Text, en Martin, M. A. & Veel, R. (Eds.). Reading Science (87-113). Londres: Routledge.

Llinares, F. (2012). El sonido como recurso expresivo en los audiovisuales, en Gustems, J. (ed.). Música y Sonido en los audiovisuales (135-145). Barcelona: Universitat de Barcelona.

Marimon, J. (2014). El montaje cinematográfico. Del guión a la pantalla. Barcelona: Publicacions i Edicions de la Universitat de Barcelona.

MIM (1996). Les Unités Sémiotiques Temporelles. Marsella: MIM .

MIM (2002). Les Unités Sémiotiques Temporelles: nouvelles clés pour l'écoute. Ouil d'analyse musicale. Marsella: MIM.

Neumeyer, D. & Buhler, J. (2001). Analytical and Interpretive Approaches to Film Music (I): Analysing the Music, en Donnelly, K. J. (Ed.). Film Music: Critical Approaches (16-38). Nueva York: The Continuum International Publishing Group.

Radigales, J. (2001). Creació i muntatge musical en el dibuix animat clàssic, en Trípodos, n° 11. Págs. 107-122.

Smith, M. (2000). Ballet and Opera in the age of Giselle. Princeton: Princeton University Press.

Torelló, J. & Duran, J. (2014). Michel Chion en La audiovisión y una propuesta práctica sobre un fragmento de Nostalgia de Andrei Tarkovski, en L'Atalante, n° 18. Págs. 111-117.

Valvassori, M. (2006). El personaje trickster o "burlador" en el cuento tradicional y en el cine de dibujos animados, en Culturas Populares, Revista electrónica, n° 1. Págs. 1-27.

Whalen, Z. (2004). Play along. An Approach to Videogame Music, en Game Studies, n° 4, vol. 1. Págs. 1-31.

Newspaper publication:

Articles in paper and electronic:

Farrar, E. (2003). A Method for Mapping Expressive Qualities of Music to Expressive Qualities of Animation. Recuperado de http:/ / accad.osu.edu/~efarrar/thesis/proposal120602.pdf. Consultado el 13/11/2015.

Hautbois, X. (2010). Les Unités Sémiotiques Temporelles: de la semiotique musicale vers una sémiotique générale du temps dans les arts, en Les UST: enjeux pour l'analyse et la recherche. Musimédiane. Revue audiovisuelle et multimédia d'analyse musical. Recuperado de http://www.musimediane.com/ Consultado el 14/10/2015

Jones, C. (1946). Music and the Animated Cartoon, en Hollywood Quarterly, n° 1, vol. 4.Págs. 364-370.

Royce, T. (1998). Synergie on the page: exploring intersemiotic complementarity in page-based multimodal text, en Japan Association Systemic Functional Linguistic Occasional Papers, n° 1, vol. 1. Págs 25-50.

Rosenberg, N. (1983). Image and Stereotype: Bluegrass Sound Tracks, en American Music, n°1, vol. 3. Págs. 1-22.