Mnemovie: Visual Mnemonics for Creative Interactive Video

Abstract

There is a problem with storing and retrieving audio-visual digital media files using information and communication technologies employing text-based indexing systems. Fundamentally, the complexities of language as a semantic system do not serve well the complexities of the motion picture document.

The objective is to propose effective and affecting means by which creators and audiences can store and retrieve the video files with which we work, communicate and entertain ourselves, increasingly each day. The research has employed practice-based research to extend our understanding of the precept of a taxonomy based on the visual mnemonics of the motion picture document.

The research approach draws on the work of Schön: “…our knowing is in our action…” (Schön, 1983) 49, together with Norman’s description of two modes of cognitive behaviour, the experiential and the reflective (Norman 1993) 16. This is echoed by recent work on ‘the configuration of indexicals’ (indexicality) where communities of expertise can collaboratively establish ‘..shared meaningful objects…’ within a referential network (Sarmiento and Stahl, 2007). It joins many others, who have identified the activity of searching a database or collection as “…part of a creative process.”(Kules 2006). These researchers have informed the production of evidence in my research, that takes the form of experimental models from which data has been gathered, both in the making of the artefacts and their evaluation.

A series of seven experimental Models have been built using movie files encountered as full screen motion-picture images, navigated with four-way gestural interactivity. Mnemonics – aids to memory – are deployed taking two broad approaches: a schema, (from the Greek skhema, meaning shape), imparted with a word description at the outset of the interactive encounter of a primitive to describe navigational principles for each Model; and the images and sounds within the movies, associatively and semantically related mnemonically to the knowledge domain of the collection.

Conclusions emerge from two areas of practice-based research, the artist/designer and the potential user group. Initially, evaluation of the objective of each experiment with the creativity support tool - the Mnemovie engine – revealed the need to design interactive movie Models specifically for each collection of movies. Subsequently, observational data from the test subjects both confirmed and contradicted the precept, leading to the description by participants of their own navigational designs using the Mnemovie system for personal movie collections.

Further research objectives are reported emerging from the conclusions, proposing specifications for a system, or series of systems, incorporating further development of the Mnemovie engine support tool, live performance collaborative projects, generative systems, and opportunities for interactive sensing systems technolgy.