Film and Video Analysis in the Digital Humanities – An Interdisciplinary Dialog

Abstract

This editorial introduces the special issue on “Digital Humanities & Film Studies: Analyzing the Modalities of Moving Images,” which contains a total of eight exciting contributions. Moreover, this editorial aims to highlight the complementary disciplinary perspectives on the computer-aided analysis of moving images, which are important prerequisites to better understand and situate a common DH perspective

1. The Infrastructural Perspective: GLAM Institutions and Film Archives

The computer-aided analysis of moving images is becoming an increasingly important topic within the digital humanities and it has been an interdisciplinary field since its beginnings: Computer science and media informatics, film and media studies as well as film archives and documentation centers have developed their own, independent approaches to the digital analysis of film. At the same time, however, several bridges have already been built between the individual disciplines: In computer science, for example, there has been research on automatic film and video analysis for more than 20 years, but it is still carried out without any visible connection to the field of film studies. Since the digital humanities have always been a place of interdisciplinary encounter and exchange, it is not surprising that the discovery of digital film studies (see Grant [2012]) as a combination and continuation of the above-mentioned disciplinary traditions, has become a major topic in recent years. This special issue on "Digital Humanities & Film Studies: Analyzing the Modalities of Moving Images" aims to promote an interdisciplinary dialog by presenting a variety of different articles from different fields. In order to get this dialog started, this introduction will try to work out the different disciplinary strands of computer-aided film and video analysis and to interconnect them in terms of a common digital humanities perspective.

A first important strand can be found in the so-called GLAM (galleries, libraries, archives, museums) institutions, with their close connection to information science. Cultural heritage institutions traditionally consider it their mission to facilitate the creation and dissemination of knowledge and works of art by providing access to their collections. This not only means engaging with academics, but also connecting with a wider community and addressing their diverse needs. In libraries, archives and museums, systems had already been established over centuries to support information retrieval in their collections, for instance bibliographies, finding aids, citation systems and concordances. This information was also made available in publications, thus linking the publishers, librarians, archivists and curators to the researchers. Film archives lag considerably behind when it comes to actively facilitate sharing their collections, but recent developments within the international film archiving community show a growing interest in exploring alternative ways of providing access to filmographic metadata, inventory information and contextual information likewise.

The beginnings of the fundamental restructuring of knowledge and the academic world can arguably be dated to 2003, when the National Science Foundation (NSF), the American government organization tasked with promoting basic research across disciplinary borders, commissioned a report, later colloquially named after then chairman Dan Atkins (2003). In two subsequent reports from 2006 and 2007, published by the NSF and the American Council of Learned Societies (ACLS), the vision of a new "cyberinfrastructure ... that supports peer-to-peer collaboration and new modes of education based upon broad and open access to leadership computing; data and information resources; online instruments and observatories; and visualization and collaboration services" was conclusively defined and disseminated [Arden 2007]. Already in 2009, in the pivotal "Digital Humanities Manifesto 2.0", Schnapp and Presner asked for a reevaluation of the relation between scholars and GLAM professionals and "affirm that modern universities still tend to separate scholarship from curation, a fact that is hardly deniable. The latter is normally reduced to a secondary and supportive role, thus sending curators within the museums, archives and libraries into exile" [Heftberger 2014, 142]. If we understand curation as the overarching process of taking care of the collection in various ways, it is probably closer to how the tasks in GLAM institutions have changed over the last ten years.

GLAM institutions are typically interested in two types of digital humanities: On the one hand, they themselves have a need for new technologies in order to be able to analyze, enrich, index and make their holdings accessible. On the other hand, they often act as consulting institutions for connected researchers to support them with knowledge, infrastructure and technology (for a comprehensive discussion of this topic, see Schaffner and Erway [2014]). It is, however, still not clear where the personnel for the mentioned tasks could come from. Are new digital humanities graduates a way to fill these positions? Is it better to form tandems of humanities scholars and computer scientists, or even change archival educational programs (see Heftberger [2018])? Questions like these were openly discussed at a recent workshop (February 2020), held at the German National Library in Frankfurt, which dealt with the broader question of how cultural heritage institutions position themselves when it comes to digital humanities.

Nowadays, data management, data wrangling, data mapping and data sharing have become core elements of the daily work in archives, libraries or museums. As Julia Flanders describes, "most digital humanities work – as performed by library staff, IT staff, and other para-academic staff who are not faculty – is conceptualized according to one of the other models: hourly, by full-time equivalent, or as an agenda of projects that granularizes and regulates the work in quantifiable ways" [Flanders 2012, 303]. According to Flanders, there is a tendency to define assignments in the framework of digital humanities (e.g. databases, online presentations, also perhaps long-term archiving) as projects and to outsource them to third parties. Although such a procedure is understandable, a chance is being missed: the opportunity to build in-house resources which would also enable engagement with collection content alongside the digital infrastructure and information management (see Heftberger [2018]: 20).

One core task in the film archive is film identification and data enrichment via subject indexing, which is usually still done manually. In the meantime, it has become possible to annotate video files, to provide them with keywords to the precision of a frame, to annotate them with geographical data and to comment on them in free text. Potentially digital collections can be mined for patterns and objects, and research can take place at the level of content but also at the levels of metadata generation and meaningful web presentations. In recent years, in addition to an increasing tendency to aggregate metadata and inventory data (e.g. filmportal.de), it can also be observed that the topic of Linked Open Data is becoming more important. In international working groups (e.g. in the Cataloguing Commission of the FIAF), ontologies for film as well as workflows for the use of identifiers from, for example, Wikidata are being collaboratively developed, and their usage will hopefully gain momentum (see Heftberger and Duchesne [2020]). If we look at how processes of knowledge transformation in the twenty-first century can take place and how they can be understood by means of computer technology, interpretation must remain an important component of the discipline:

What the community can do with the results of a digital humanities project is, like art, often outside what a creator or project team might have envisioned for it — and this is where the interpretation becomes important for multivalent digital humanities projects. What does it mean that a database has been structured in a certain way? What are the larger consequences for one design over another? How does a certain project push the boundaries of what we consider acceptable digital humanities work? How can new analytical processes or methodologies be applied in different contexts? These are subjective and interpretative questions that we must openly discuss. [Gibbs 2011]

Institutions can usually identify relatively clearly which tasks are suitable to be supported by software. Typically, subject indexing and visualizing collections online are named by professionals. In addition to these continuous duties, there are a large number of exciting sub-projects that could be worked on in collaboration with scholars and/or artists, and which would offer tremendous added value for both groups. Film collecting institutions bring their knowledge about analog material to the table as well as a comprehensive knowledge of film titles in their holdings, which are not digitally available yet. Their staff can assist with expertise in digitization as well as opening up film collections by viewing and cataloging titles which have not been widely disseminated yet. Thus, they can contribute to broaden a so far narrow view on film history which too many times only focuses on iconic titles and male directors. Even just looking into cast and credit roles and how they changed over the course of time in different languages may change our perspective, for example on the involvement of female editors (see Pearlman and Heftberger 2018). Making more film titles available and connecting resources digitally via sharing metadata and contextual information as well as visualizing their collections is a key task for film archives nowadays. One good example would be the BFI Filmography, which so far is a rare attempt to consider filmographic metadata research data for film historians and film scholars [Pearlman and Heftberger 2018].

2.The Computational Perspective: Multimedia Information Retrieval

Film scholars have a long tradition in the qualitative and quantitative analysis of film and more recently video. The automatic and automatically assisted analysis of filmic content has – compared to that – a rather short history. Due to the large amounts of data produced by digitized film and video and the associated high computing costs as well as the initially rare availability of digitized footage, research on automatic film analysis could only slowly establish when first digital media retrieval methods started to develop in the mid 1990s. Since then, we have observed a tremendous development of automatic film and video analysis which was fueled by breakthroughs in machine learning methodology (especially deep learning) as well as in the availability of annotated training data. Today, automatic film and video analysis has achieved a certain level of maturity and — as a result — is applied more and more in the digital humanities. We describe the origins of automatic film analysis in computer science and its rather young history in the following and identify important future challenges for automatic film analysis in the context of the digital humanities.

The research field associated with automatic film analysis is called "Multimedia Information Retrieval" (MMIR). MMIR has originally evolved as a branch of Information Retrieval (IR) (for some early examples see Mooers [1950], Salton and McGill [1983], van Rijsbergen [1997]) and has grown to a broad and independent field of research in recent years. In the last two decades, the amount of available digital media data has increased tremendously. In addition to textual information, image, audio and video data have become omnipresent due to the availability of cheap capturing hardware, the development of efficient compression and transmission techniques and the establishment of large, publicly accessible databases and media platforms on the Internet. Still, the access to these large amounts of multimedia data is rather restricted, as media data are often lacking textual descriptions. Also, it is difficult to derive semantically meaningful information from the content itself, for instance the individual pixels of an image or the samples of an audio signal.

This is exactly where MMIR comes into play. MMIR aims at making sense of the raw media to make the media itself searchable by content [Lew et al. 2006]. Thereby, MMIR makes large media databases instantly searchable. This enables a variety of useful applications, such as finding people and objects in images, retrieving videos that show and explain certain activities, finding unusual events in long-term video sequences, but also retrieving syntactic and semantic concepts from films [Zaharieva et al. 2010] to support qualitative and quantitative studies by film scholars and to open up large-scale film and video corpora for systematic analyses. Research on MMIR goes back to the early 1990s [Grosky 1994] and has grown so broad today that it has become difficult to oversee the entire field [Pouyanfar et al. 2018]. The great diversity of media data has led to the development of several MMIR research branches, such as Content-Based Image Retrieval (CBIR), Content-Based Audio Retrieval (CBAR), and Content-Based Video Retrieval (CBVR), which over time have become research areas that can stand for themselves.

Additionally, several cohesive modalities (e.g. video and its corresponding audio track) can be combined in multimodal retrieval approaches to leverage complementary information for more robust retrieval. Multimodal retrieval is often used synonymous with multimedia information retrieval and is also sometimes referred to as "MMIR" in the literature. Multimodal retrieval can be defined as follows: "A multimedia information retrieval (MMIR) problem is multimodal if it includes multiple input modalities, such as images, audio, 3D data, textual and contextual information" [Baltrusaitis et al. 2018]. Multimodal retrieval can be considered an overarching branch of MMIR that combines and integrates methodology from different modality-specific branches. In multimodal retrieval, basically arbitrary data sources can be combined as long as they have a certain semantic relationship. Today's approaches toward automatic film and video analysis integrate methodology of all of the above-mentioned branches of MMIR. For this reason, they are briefly introduced in the following.

2.1 Content-based image retrieval (CBIR)

CBIR focuses on the retrieval of images in an image archive or database from a given query, which can be either textual or provided as an example image (query-by-example). The first content-based retrieval systems came up in the 1990s and included systems like QBIC [Flickner et al. 1995] and VIRAGE [Gupta and Jain 1997], which were able to query a multimedia database by example images. Aside from finding similar images to a query, typical CBIR problems are the retrieval of objects, concepts and scene properties [Zhao et al. 2017] as well as image classification [Druzhkov and Kustikova 2016]. CBIR is thus strongly interlinked with the fields of computer vision and machine learning. Research in CBIR focuses on the retrieval of patterns from various types of images, including medical images, remote sensing imagery and user-generated content (e.g. in personal photo collections and in social media). Broad surveys on the field of CBIR are provided by Datta et al. (2008) and more recently by Zhou et al. (2017). Automatic film analysis makes intensive use of content-based image analysis. Even though the study object are films and videos, many visual analysis problems in film analysis (e.g. object detection in movies [Flückiger 2019], visual composition analysis [Mitrovic et al. 2001] and the recognition of shot size and shot type [Zaharieva et al. 2010]) can be performed on individual images without taking the temporal dimension into account.

2.2 Content-based video retrieval (CBVR)

Closely related to CBIR is Content-Based Video Retrieval (CBVR). Typical problems in CBVR are the retrieval of similar videos to a query as well as temporal segmentation of a video in shots and scenes [Cotsaces et al. 2006], action and activity recognition [Cheng et al. 2015] event recognition [Tzelepis et al. 2016], motion retrieval [Zeppelzauer et al. 2011a], video summarization [Del Molino et al. 2016] and video browsing [Schöffmann et al. 2010]. Content-based video retrieval integrates many techniques from CBIR and extends them with temporal analysis (e.g. motion analysis or object tracking). As a first step in CBVR, the video stream is usually partitioned into smaller units, such as key frames, shots and scenes [Rui et al. 1998]. Next, information at the frame as well as the shot level are extracted and aggregated for retrieval and classification. Typical types of video data include sports videos, news videos and surveillance videos. Beyond these types of motion images, there is a large corpus of literature on the automatic analysis of movies, documentaries and archive film [Ewerth et al. 2009] [Zaharieva et al. 2010], which has developed in parallel to visual research in the digital humanities. A frequent pattern that can be observed from the literature is that a broad range of filmic concepts has been investigated already in the computer science, such as film rhythm, tempo, style and narrative structures [Adams et al. 2001] [Adams et al. 2002] [Choudhary et al. 2019] but with rather simplified definitions of these concepts.

2.3 Content-based audio retrieval (CBAR)

CBAR has developed similarly to CBIR in the early 1990s. Pioneering work is presented by Wold et al. (1996), with their development of an audio retrieval system called "Muscle Fish" that was able to automatically distinguish between different types of sounds (see Pfeiffer et al. [1996]). Since then, there has been rapid development in different directions, such as audio segmentation, music information retrieval and environmental sound retrieval. Furthermore, the analysis and recognition of speech, which has developed separately, starting already in the 1950s, can be considered one sub-branch of CBAR [Juang and Rabiner 2005]. Compared to the visual modality, the acoustic modality is under-represented in automatic film analysis. One reason is that the acoustic modality alone is often not sufficient for film analysis. Acoustic information is, however, highly useful when combined with visual information in a multimodal approach [Snoek and Worring 2005]. Information from soundtracks can significantly help to classify film genre, scene types and film style [Choudhary et al. 2019]. Furthermore, the analysis of the affective dimension of film requires acoustic and visual clues [Soleymani et al. 2009]. Finally, film montage may exhibit correlated patterns between the acoustic and visual track [Zeppelzauer et al. 2011b].

2.4 Concluding remarks on the field of multimedia information retrieval (MMIR)

From the brief review of the field of MMIR, we can observe that a large body of automatic analysis approaches exists that would (at least in theory) be applicable to filmic content investigated by film scholars. Especially in the early years of MMIR, however, these methods were not powerful enough to model the complex semantic concepts investigated by film scholars. Today, automatic visual (and audio-visual) analysis is well advanced. Developments in machine learning and especially in deep learning in the last decade have made significant progress in the extraction of information from audio-visual media and in modeling semantic relationships. Certain problems which appeared unsolvable 15 years ago can now be solved automatically with almost human-level performance [He et al. 2015]. This has therefore changed the situation for the digital humanities fundamentally. MMIR has reached a level of maturity that can provide a real added value for film scholars and the digital humanities in general. Many previously unanswered research questions from film scholars are waiting to be (re)investigated by the support of automatic film and video analysis methods.

3. The Media Perspective: Digital Encounters in Film and Media Studies

Recapitulating the history of digital tools, methods and practices within film and media studies is a particularly challenging task. Wherever the field has dealt with this topic, reflections on the status of digital tools and technologies have often superseded their use as a facilitator of research. The reason for this situation is as simple as it is convincing: it is the very insight that "the medium is the message" [McLuhan 1964]. Hence, we should not just consider technology a tool, a means to an end that lies at the heart, at least of media studies.[1] The decision, however, to make technology, and thus digital media, a research object of epistemological, cultural and sociological enquiry itself has also another consequence. It creates awareness about the fact that common narratives about the purpose of a certain piece of technology and actual usages of the same technology can be two very different things. It therefore sensitizes to alleged misusages of technologies. These misusages often exist in the margins of those approaches that are propagated by certain tool ideologies and instigate their own techno-social configurations. The histories of digital tool usage in film and media studies are thus entangled with different perspectives on the contingent nature of digital technologies and different understandings of scholarly engagement.

3.1 Notions of the digital and digital methodologies

Using the plural form in histories is more than just acknowledging the democratic attitude that writing history enforces. It is a reaction to the fact that it even lacks an agreement about the same temporal dimension when researchers of the field try to systematize the history of the use of digital technologies in film and media studies. In Germany, for instance, Julian Sittel (2017) states that film studies lags behind when it comes to the adoption of digital humanities methods, while Patrick Vonderau (2017) suggests that digital humanities is the continuity of a certain way to use digital technologies that has long been superseded by other usage types. This claim is illustrated by an analysis of the use of digital technologies in film and media studies that distinguishes three consecutive phases between 1985 and 2005: The first phase produced tools to manually annotate and segment movies. The second phase introduced database-driven quantification of annotated and classified units in moving images. The third phase gives testimony of a repurposing of the goal of databases themselves. They now shift from facilitating statistical analysis to becoming a metaphor and facilitator for the situational and explorative organization of content of any kind in research in the logic of the collage. Databases are more conceived of as an expression of an abstract notion of media practice instead of just being a tool to query and count [Böhme et al. 2012].

In Vonderau's perspective, digital humanities is just an attempt to perfect narrow use of digital technology that was predominant in the second phase. It lacks a broad theoretical framing for digital transformation and technology as such and thus misses new emerging paradigms of technology use. In Sittel's line of argument, however, digital humanities are the clear sign of a more fundamental shift; a shift that reconfigures the entanglement between theory, technology and research – with respect to the first paragraph the media/message relationship – itself and that, accordingly, cannot be integrated into film and media studies appropriately. Claims such as those about "the end of theory" [Anderson 2008] or the "fourth paradigm" [Hey et al. 2009] come to mind here. In summary, different notions of technology instigate different historical narratives as well as different methodological portfolios and needs. Each of these configurations are, furthermore, linked to struggles for disciplinary autonomy but also to debates about the internal structure of the field itself. This is most obvious when Vonderau declares that actually the main object of film studies – the analysis of movies – mostly disappeared in film studies with the arrival of digital technologies, while Chavéz-Heras (2012) announces that now, with digital tools, film analysis is reborn.

Such media-reflexive preoccupations have caused a lot of alienation between film and media studies on the one hand and digital humanities on the other.[2] It would, however, be misleading if digital humanities just considered it an internal issue of film and media studies. In fact, the digital humanities have cultivated their own version of the above-mentioned tension, a tension that recently created a schism within in the field itself. Since 2014, concepts such as the post-digital humanities [Berry 2014] or the critical digital humanities [Grimshaw 2018] [Dobson 2019] emerged within debates on digital humanities. The former relates to the methodological inconsistencies behind the term digital methods, while the latter stresses cultural conditions and impacts of algorithmic meaning making processes. In opposition to these branches, in 2019, the computational humanities research community spun off the field of digital humanities, preceeded by debates that digital humanities had lost its core – humanities computing – and that rather "techie" topics are not sufficiently represented [Elwert 2019].

3.2 Entanglements with the Digital and Post-Digital Humanities

This example is just the entertaining expression of an open space of entanglements between digital technologies and doing research. In film and media studies these include approaches such as artistic research [Ruszev 2018], speculative computing [Drucker 2009], statistics and stylometry [Casey and Williams 2014] [Baxter et al. 2017], digital surrealism [Ferguson 2015] [Ferguson 2017], or multimodal storytelling and videographic criticism [Erlend 2012] [Keathley et al. 2019]. The case of videographic criticism or the video-essay, as it is sometimes called, is especially interesting because it demonstrates how an object of research becomes the means of analysis itself. It investigates movies by producing short movies and follows the media reflexive idea that it might be best to represent facts of a certain modality within that same modality. Although not computational in a strict digital humanities sense, the video essay clearly is interpreted as a product of digital technologies and an established method of film studies research [McWhirter 2015]. Together with the above-mentioned dispute about the obsolescence or the revival of the analysis of movies, the whole episode demonstrates very well that questions raised by digital technologies are not just questions of new methods, and even less questions of computational or statistical methods. They are, with similar importance, questions about the changing status of research objects within a changing, partially digital, environment. Accordingly, the challenge is not just to define better or new methods, but also to develop a new sense of research objects and research ecologies – all at the same time.

With this in mind, it is necessary to critically re-approach the notion that film and media studies lag behind. The holistic angle on digital research that not only considers new methods for known objects but also new cultural embeddings for known as well as unknown objects, suggests another kind of endeavor. In this respect it is a meaningful observation that in the history of digital humanities, fields with clearly definable and established research entities – the edition, the textual source and the artifact among others – were the most influential. It goes without saying that the analysis of our partially digital environment and its objects enforces to also apply digital humanities methods in a stricter sense of the term and cannot be carried out comprehensively without them. In consequence, film and media studies researchers that might not have welcomed these methods, due to methodological reasons, in the first place are, nevertheless, beginning to open up as has been mentioned before. The challenge, both for the digital humanities and for film and media studies, remains to integrate perspectives of method, object and ecology without creating a hierarchy among them; be it methods for the digital humanities or objects and ecology for film and media studies.

It was a striking feature of Vonderau’s periodization that it ended around 2005, which is more or less the same time in which the digital humanities began to aspire. Both histories, the one that ended in the article's narrative and the one that started with a companion, were motivated by the intent to separate one research field from the other. The first history tried to show that the newly emerging DH was in some sense always already part of the established field of film and media studies. The second history presupposes that only by creating a new field, the old will actually become the new. Comparing the history of digital humanities methods in film and media studies with the history of film and media studies topics in the digital humanities clearly shows that both lines of arguments get it wrong, because in the growing number of encounters between film and media studies and digital humanities, a new reality emerges that is both post-humanistic and post-digital [Cramer 2014].

Certainly, film and media studies have to engage more profoundly with the methodological part of statistical and algorithmic approaches. This has slowly begun to happen throughout the last years, as projects such as the large-scale video analytics project [Kuhn et al. 2015], activities of the scientific networks Digital Cinema Studies[3] or New Directions in Film Historiography[4] or projects like FilmColors [Flückiger 2017] and Kinepoetics[5] suggest. At the same time, it appears to be equally important for the digital humanities to reflect on their rather broad conception of technology, in order to continue to make substantial contributions to the reconfiguration of research by digital technologies. A fuzzy reference to the "big tent" [Terras 2013] or the "big humanities" [Svensson 2016] will not suffice in this respect.

4. Toward an Integrated Perspective: Digital Film Studies and Digital Humanities

In this section we bring together the individual disciplinary perspectives on digital film studies in an integrated perspective within the digital humanities. The three disciplinary perspectives of (1) GLAMs and archives, (2) computer science, and (3) film and media studies align very well with the three sub-areas of Digital Humanities as identified by Roth (2019): the digitized humanities, the numerical humanities and humanities of the digital. In the following, we illustrate the different variants of digital film studies with respect to the three sub-areas of digital humanities by means of concrete examples. We also use this basic classification to introduce the eight research projects and studies that are part of this special issue.

4.1 Digitized humanities

This branch of DH is mostly concerned with the digitization of cultural artifacts. This also includes the modeling of analog artifacts, i.e. the formal representation of a cultural object. This formalization is an important prerequisite for digital preservation and dissemination strategies, but also for computational analyses of the material. For the formal representation of film, the fundamental question is: how can we model a time-based and multimodal medium such as film to be represented in discrete categories? As this is obviously a demanding and complex task, it is important for scholars to be clear in advance – i.e. before the actual modeling – about what they want to investigate: Are setting and scene boundaries relevant for later analyses? Are they interested in constellations of characters or the use of leitmotifs? Or do they want to take a closer look at the dialog level? Due to the modeling effort involved, it is hardly possible in practice to model everything pro forma, so a prior research question is indispensable for the digitization and modeling of film and video.

As a consequence, it is rather challenging to create film and video resources that can be reused by a variety of scholars, as the research questions (and modeling implications, cf. Unsworth [2002]) vary widely and as there are hardly any established annotation standards as known for other types of media, for instance the Text Encoding Initiative (TEI) or the Music Encoding Initiative (MEI).[6] Further challenges are raised by born digital videos, for example YouTube videos or Twitch.tv streams, as specific concepts for offline storage and potential long-term preservation are needed that lead to open copyright issues and new requirements for digital infrastructure. In this special issue, Eva Hielscher contributes to the topic of modeling and representing films by presenting an extensive case study on video annotation for a corpus of "City Symphonies" in her article titled "The Phenomenon of Interwar City Symphonies: A Combined Methodology of Digital Tools and Traditional Film Analysis Methods to Study Visual Motifs and Structural Patterns of Experimental-Documentary City Films".[7] Another article in this special issue is on "Matching Computational Analysis and Human Experience. Performative Arts and the Digital Humanities".[8] Here, Jan-Hendrik Bakels et al. investigate the combination and mediation of experience-based and data-driven analytical procedures in film studies. They transfer common film analytical vocabulary into a machine-readable ontology to set up a systematic annotation routine for films that can be visualized and queried.

Digitization of films and videos is also important for film heritage institutions, as it helps to collect filmographic information from diverse sources by means of specialized search functions, OCR or image recognition among others. This can also support digital archives with "mining" their objects, for example finding certain subjects/topics/people etc. Therefore, archives are excellent agents for advancing the standardization of filmographic metadata, for instance by benefitting from experiences with metadata standards such as EN 15907 (see Heftberger [2014]: 143). Another key task here is the presentation of collections (or parts thereof) online in an appealing, sensible and effective way. Institutions are fully aware that building good interfaces is critical, but there seem to be many obstacles still in their way. Insufficient financial resources might be one of those obstacles, being stuck in building traditional entry points to collections may be another one. Masson et al. provide a deep and comprehensive discussion of traditional and new paradigms in this area in their article titled "Exploring Digitised Moving Image Collections: The SEMIA Project, Visual Analysis and the Turn to Abstraction".[9] Additionally, they are able to substantiate this discussion by introducing the SEMIA platform, a convincing example for how adequate interfaces might be designed.

In the past film archives, for the most part, differentiated between technical expertise and work with content. However, a greater permeability between the areas of responsibility is required in the future in order to comply with the abovementioned challenges. It is already the case that many tasks are no longer addressable by traditional roles. Digital data management, for instance, cannot be restricted to just one area of material (e.g. films, photos or documents) and metadata must be managed across the boundaries of collections in order to use technical infrastructures and personnel resources in the best possible way among other reasons. The strategy must therefore be twofold in the future: Teaming up with researchers and artists as well as staffing their places with technically savvy people who at the same time understand archival objects and are interested in exhibition. Digital humanities as digitized humanities have organized around such entanglements for years and may, therefor, provide valuable points of reference for the field of digital film studies.

4.2 Numerical humanities

This second category is oftentimes also referred to as computational humanities, highlighting the aspect of empirical analyses by means of statistical procedures and machine learning techniques. In recent years, we have seen a gradual extension of the established formal-stylistic analyses [Salt 1974] [Tsivian 2009], which mostly relied on the lengths and frequencies of shots within a movie. Going well "beyond shot lengths" [Burghardt et al. 2016], a whole series of studies have been carried out recently, for example to examine the color dimension [Burghardt et al. 2017] [Flückiger 2017] [Pause and Walkowski 2018] [Pause and Walkowski (forthcoming)] or the dialogs of films [Bednarek 2018] [Hołobut et al. 2016]. Another branch of work in this category can be found in the area of visual analytics [Burghardt et al. 2018] [Hoyt et al. 2014] [Kurzhals et al. 2016], as visualizations can greatly facilitate the analysis of visual time-based media such as film and video [Manovich 2013].

This special issue features two numerical humanities studies that investigate the textual dimension of series and movies. In their article on "The Stylometry of Film Dialogue: Pros and Pitfalls",[10] Agata Hołobut and Jan Rybicki analyze 300 film dialogs by means of stylometry and sentiment analysis to detect patterns of similarity and difference between screenwriters and/or a priori IMDB-defined genres. Joanna Byszuk goes in a similar direction with her article called "The Voices of Doctor Who – How Stylometry Can be Useful in Revealing New Information About TV Series".[11] More concretely, she uses stylometry to examine the changes driving the development of the "Doctor Who" series from a character-oriented to a showrunner-oriented series.

As part of the numerical humanities, deep learning approaches are also increasingly being used in the sense of "deep watching" [Howanitz et al. 2019] or "distant viewing" [Arnold and Tilton 2019], in order to automatically segment films and identify objects or people. In this special issue, Barbara Flückiger and Gaudenz Halter contribute an article on "Methods and Advanced Tools for the Analysis of Film Colors in Digital Humanities",[12] which provides an excellent example for numerical humanities, showcasing a sophisticated deep learning approach to investigate color usage in film. Another deep learning example is provided in the article "Automated Visual Content Analysis for Film Studies: Current Status and Challenges",[13] by Ralph Ewerth et al. The authors provide a thorough survey of approaches and tools for automated video analysis to support quantitative and qualitative analysis in film studies.

For digital film studies, the digital humanities as numerical humanities mostly represent methods that try to automate the identification of established entities and units within film studies research. By doing so, they mimic analytical methods of the field, but on a much larger scale than would be possible by means of qualitative analyses of – typically – just a few films. In recent time, however, there are also new approaches that aim at a semi-automatic and, thus, more interactive analysis which seem to prioritize exploration instead of extraction. This re-evaluation of priorities may well open-up the perspective for contributions to a more diverse understanding of numerical and computational approaches to the study of films.

4.3 Humanities of the digital

This category describes humanities scholars’ broader interest for the inclusion of phenomena that emerge from the "digital culture." In the field of film studies, this might, for instance, entail the abovementioned investigation of born digital videos, which very often also brings with it a dimension of mass communication and thus big data: either, because a multitude of videos are available, or because there is additional mass communication channel available, e.g. in the form of comments about a video (reception aesthetics). It also includes an examination of digital aesthetics as well as of new forms of production and reception in an increasingly digitized, post-cinema film culture. Such changes inevitably raise theoretical questions that, at the same time, are necessarily linked to new research practices. Due to the sheer volume of digitally available material, a serious opening of the humanities towards digital culture obviously requires an increased integration of digital methods into scholarly practices as well.

From the perspective of film studies, the central challenge humanities of the digital imposes, is the need for a holistic instead of a pragmatic integration of digital methods, not the least because these methods have become methods of production and analysis (film essay) at the same time. A central task in this context is to reflect on the changing relationship between research instrument and research subject, to bring together quantitative and qualitative methods, to find ways to mediate between metrization and interpretation. It is important to demonstrate that approaches of data-driven research can be particularly productive in the field of film and media studies when they are conceived of as tools within the framework of a diverse practice (cf. Antonijevic [2015]) or mixed methods [Pereira 2019]. In this setup, digital methods are part of a practice, but not its overarching theme. One example for such a more praxeologically oriented approach to the integration of digital methods in film studies has been recently systematized under the term scalable viewing [Burghardt et al. 2018] [Pause and Walkowski 2020].

In this context, the "Humanities of the Digital" provide theoretical foundations for new notions of the digital that have the potential to make digital methods more accessible for film studies research. Such notions root deeper in the tradition of the humanities, while at the same time highlighting the necessity and added value of algorithmic strategies. In this special issue, Sarah-Mai Dang contributes to this third sub-field of digital humanities by presenting an article on "Unknowable Facts and Digital Databases: Reflections on the Women Film Pioneers Project and Women in Film History".[14] More concretely, she discusses the epistemological function and the scholarly implications of databases in film and media studies.

5. Conclusions

The challenges that have been outlined throughout the entire editorial call for a joint interdisciplinary effort of computer scientists, film studies scholars and archivists and represent an exciting research agenda for the coming years. While it seems to be the case that each of the digital humanities sub-areas gravitates around one particular discipline – namely digitization with GLAM, numerical with computer science and digital with film and media studies – it would be dangerous to just equate them with each other. Tools that are used for video criticism often build upon the same statistical and machine learning methods that are used in the numerical humanities, and without doubt, both GLAM and computer science are evaluating the broad field of new research objects in digital media. The horizon, however, that is particularly addressed by the digital humanities, is one in which one discipline is not just informed by the others but in which each discipline contributes on all levels to a shared space of insights, developments and practices.

The FilmColors [Flückiger 2017] project, for example, started from the insight, that approaches for the extraction of dominant colors, common in computer science have significant drawbacks. Based on this insight, the project defined a new approach that makes use of cultural concepts about how spectators process the kinematic image. It, however, did not just provide these insights to computer scientists, it also implemented a new machine learning pipeline for colorimetric analysis that is immediately available to the field of computer science for further optimization. The project, moreover, introduced innovations to the presentation of colorimetric information on the user interface level that are certainly inspiring for the aforementioned issues in the GLAM sector. It is precisely these overlaps out of which film and moving image analysis inside the umbrella of digital humanities might emerge as a field of its own.

Notes

[1] Despite the controversy about McLuhan’s concept of "media" and the bold claim he derives from it no one, today, denies the idea that a medium significantly and often unnoticed shapes what it mediates.

[2] It is important to mention that in Europe this tension is stronger due to the fact that in the US there has been branch of digital humanities that for long came out of media studies and related fields [Svensson 2010].

[3] https://www.digitalcinemastudies.com

[4] https://www.uni-marburg.de/en/fb09/institutes/media-studies/research/research-projects/digitalfilmhistoriography

[5] http://www.cinepoetics.fu-berlin.de/en/index.html

[6] Note that there have been first steps toward establishing a Film Encoding Initiative (FEI) that is based on the MEI: https://github.com/cemfi/FEI. However, this format does not seem to be as established as TEI and MEI.

[7] http://www.digitalhumanities.org/dhq/editorial/000495/000495.html

[8] http://www.digitalhumanities.org/dhq/editorial/000496/000496.html

[9] http://www.digitalhumanities.org/dhq/editorial/000497/000497.html

[10] http://www.digitalhumanities.org/dhq/editorial/000498/000498.html

[11] http://www.digitalhumanities.org/dhq/editorial/000499/000499.html

[12] http://www.digitalhumanities.org/dhq/editorial/000500/000500.html

[13] http://www.digitalhumanities.org/dhq/editorial/000518/000518.html

[14] http://www.digitalhumanities.org/dhq/editorial/000528/000528.html

References

Adams et al. 2001 Adams, B., Dorai, C., and Venkatesh, S. "Automated film rhythm extraction for scene analysis." ICME 2001: Proceedings of the IEEE International Conference on Multimedia and Expo (2001): 1056-1059.

Adams et al. 2002 Adams, B., Dorai, C., and Venkatesh, S. "Formulating Film Tempo. The Computational Media Aesthetics Methodology in Practice." In C Dorai and S Venkatesh (eds.),Media Computing: Computational Media Aesthetics. Kluwer Academic Publishers, Norwell, MA (2002): 57-84.

Anderson 2008 Anderson, C. "The End of Theory: The Data Deluge Makes the Scientific Method Obsolete." Wired 16.7 (2008).

Antonijevic 2015 Antonijevic, S. Amongst Digital Humanists. An ethnographic study of digital knowledge production. Palgrave Macmillan, London (2015).

Arden 2007 Arden, L. "Cyberinfrastructure Vision for 21st Century Discovery." National Science Foundation. US National Science Foundation (2007). [https://www.nsf.gov/pubs/2007/nsf0728/nsf0728.pdf]

Arnold and Tilton 2019 Arnold, T. and Tilton, L. "Distant Viewing: Analyzing Large Visual Corpora." Digital Scholarship in the Humanities (2019).

Atkins 2003 Atkins, D. "Revolutionizing Science and Engineering through Cyberinfrastructure. Report of the National Science Foundation Blue-Ribbon Advisory Panel on Cyberinfrastructure," US National Science Foundation (2003). [ http://www.nsf.gov/cise/sci/reports/atkins.pdf]

Baltrusaitis et al. 2018 Baltrusaitis, T., Ahuja, C., and Morency, L.-P. "Multimodal machine learning: A survey and taxonomy." IEEE Transactions on Pattern Analysis & Machine Intelligence 41.2 (2018): 423-443.

Baxter et al. 2017 Baxter, M., Khitrova, D., and Tsivian, Y. "Exploring Cutting Structure in Film, with Applications to the Films of D. W. Griffith, Mack Sennett, and Charlie Chaplin." Digital Scholarship in the Humanities 32.1 (2017).

Bednarek 2018 Bednarek, M. Language and Television Series: A Linguistic Approach to TV Dialogue. Cambridge University Press, Cambridge (2018).

Berry 2014 Berry, D. M. "Post-Digital Humanities: Computation and Cultural Critique in the Arts and Humanities." Educause 49.3 (2014): 22-26. [ http://www.educause.edu].

Burghardt et al. 2016 Burghardt, M., Kao, M., and Wolff, C. "Beyond Shot Lengths. Using Language Data and Color Information as Additional Parameters for Quantitative Movie Analysis." Book of Abstracts of the International Digital Humanities Conference (DH) (2016).

Burghardt et al. 2017 Burghardt, M., Hafner, K., Edel, L., Kenaan, S., and Wolff, C. "An Information System for the Analysis of Color Distributions in MovieBarcodes." Proceedings of the 15th International Symposium of Information Science (ISI) (2017).

Burghardt et al. 2018 Burghardt, M., Kao, M., and Walkowski, N.-O. "Scalable MovieBarcodes. An Exploratory Interface for the Analysis of Movies." IEEE Vis4DH Workshop (2018).

Böhme et al. 2012 Böhme, S., Nohr, R. F., and Wiemer, S. Sortieren, Sammeln, Suchen, Spielen. Die Datenbank als mediale Praxis. LIT Verlag, Münster (2012).

Casey and Williams 2014 Casey, M., and Williams, M. "ACTION (Audio-Visual Cinematic Toolbox for Interaction, Organization, and Navigation): An Open-Source Python Platform." White Paper HD-51394-11 (2014). [https://securegrants.neh.gov/publicquery/main.aspx?f=1&gn=HD-51394-11].

Chavéz-Heras 2012 Chávez-Heras, D. "The Malleable Computer: Software and the Study of the Moving Image." Frames Cinema Journal 1.1 (2012). [https://framescinemajournal.com/article/the-malleable-computer/].

Cheng et al. 2015 Cheng, G., Wan, Y., Saudagar, A. N., Namuduri, K., and Buckles, B. P. "Advances in human action recognition: A survey." arXiv preprint, arXiv:1501.05964 (2015).

Choudhary et al. 2019 Choudhary, P., Goel, N., and Saini, M. "A Multimedia Based Movie Style Model." IEEE International Conference on Multimedia & Expo Workshops (ICMEW) (2019): 72-77.

Cotsaces et al. 2006 Cotsaces, C., Nikolaidis, N., and Pitas, I. "Video shot detection and condensed representation. A review." IEEE Signal Processing Magazine 23.2 (2006): 28-37.

Cramer 2014 Cramer, F. "What Is ‘Post-Digital’?." A Peer-Reviewed Journal About Post Digital Research (2014). [https://aprja.net/article/view/116068]

Datta et al. 2008 Datta, R., Joshi, D., Li, J., and Wang, J. Z. "Image retrieval: Ideas, influences, and trends of the new age." ACM Computing Surveys (CSUR) 40.2 (2008): 5.

Del Molino et al. 2016 Del Molino, A. G., Tan, C., Lim, J.-H., and Tan, A.-H. "Summarization of egocentric videos: A comprehensive survey." IEEE Transactions on Human-Machine Systems 47.1 (2016): 65-76.

Dobson 2019 Dobson, J. Critical Digital Humanities: The Search for a Methodology. University of Illinois Press, Urbana (2019). [https://www.jstor.org/stable/10.5406/j.ctvfjd0mf].

Drucker 2009 Drucker, J. SpecLab: Digital Aesthetics and Projects in Speculative Computing. University of Chicago Press, Chicago (2009).

Druzhkov and Kustikova 2016 Druzhkov, P. and Kustikova, V. "A survey of deep learning methods and software tools for image classification and object detection." Pattern Recognition and Image Analysis 26.1 (2016): 9-15.

Elwert 2019 Elwert, F. "The Sudden but Consequential Rise of the Computational Humanities. Billet." A Belter's Life (Blog) (2019). [https://belter.hypotheses.org/64].

Erlend 2012 Erlend, L. "The Video Essay: The Future of Academic Film and Television Criticism?" Frames Cinema Journal 1 (2012). [https://framescinemajournal.com/article/the-video-essay-the-future/]

Ewerth et al. 2009 Ewerth, R., Mühling, M., Stadelmann, T., Gllavata, J., Grauer, M., and Freisleben, B. "Videana: a software toolkit for scientific film studies." In M. Ross, M. Grauer, B. Freisleben (eds.),Digital Tools in Media Studies. Analysis and Research. An Overview. Transcript, Bielefeld (2009): 101–116.

Ferguson 2015 Ferguson, K. L. "Volumetric Cinema." Transition: Journal of Videographic Film and Moving Image Studies 2 (2015).

Ferguson 2017 Ferguson, K. L. "Digital Surrealism: Visualizing Walt Disney Animation Studios." Digital Humanities Quarterly 11.1 (2017). [http://www.digitalhumanities.org/dhq/vol/11/1/000276/000276.html].

Flanders 2012 Flanders, J. "Time, Labor, and ‘Alternate Careers.’" In M. K. Gold (ed.), Digital Humanities Knowledge Work. Debates in the Digital Humanities. University of Minnesota Press, Minneapolis (2012): 292-308.

Flickner et al. 1995 Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Qian Huang, Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. "Query by image and video content: The QBIC system." Computer 28.9 (1995): 23-32.

Flückiger 2017 Flückiger, B. "A Digital Humanities Approach to Film Colors." The Moving Image: The Journal of the Association of Moving Image Archivists 17.2 (2017): 71-94.

Flückiger 2019 Flückiger, B. "Cinematic worlds in color. Technology, aesthetics, analysis." In BITForum. Film techniques and technology: illusion, reality, virtuality, innovation. Busan, South Korea (2019): 11-39.

Gibbs 2011 Gibbs, F. "Critical Discourse in the Digital Humanities." Journal of Digital Humanities 1.1 (2011)

Grant 2012 Grant, C. "Film and moving image studies: Re-born digital? Some participant observations."Frames Cinema Journal 1.1 (2012).

Grimshaw 2018 Grimshaw, M. "Towards a Manifesto for a Critical Digital Humanities: Critiquing the Extractive Capitalism of Digital Society." Palgrave Communications 4.1 (2018): 1-8.

Grosky 1994 Grosky, W. I. "Multimedia information systems." IEEE Multimedia 1.1 (1994): 12-24.

Gupta and Jain 1997 Gupta, A. and Jain, R. "Visual information retrieval." Communications of the ACM 40.5 (1997): 70-79.

He et al. 2015 He, K., Zhang, X., Ren, S., and Sun, J. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE International Conference on Computer Vision (2015): 1026-1034.

Heftberger 2014 Heftberger, A. "Film archives and digital humanities - an impossible match? New job descriptions and the challenges of the digital era." MedieKultur: Journal of media and communication research 30.57 (2014): 135-153 [http://ojs.statsbiblioteket.dk/index.php/mediekultur/issue/view/2200 https://tidsskrift.dk/mediekultur/article/view/16487/17534].

Heftberger 2018a Heftberger, A. Digital Humanities and Film Studies: Visualising Dziga Vertov's Work. Springer International Publishing, Cham (2018).

Heftberger 2018b Heftberger, A. "The Current Landscape of Film Archiving and How Study Programs can Contribute." SYNOPTIQUE - An Online Journal of Film and Moving Image Studies, Special Issue. The Current Landscape of Film Archiving and How Study Programs can Contribute 6.1 (2018): 58-69. [http://synoptique.ca/wp-content/uploads/2018/07/9-adelheid-heftberger-the-current-landscape-of-film-archiving-and-how-study-programs-can-contribute.pdf]

Heftberger and Duchesne 2020 Heftberger, A. and Duchesne, P. "Cataloguing Practices in the Age of Linked Open Data: Wikidata and Wikibase for Film Archives." International Federation of Film Archives, Brussels, Belgium (2020). [https://www.fiafnet.org/pages/E-Resources/Cataloguing-Practices-Linked-Open-Data.html]

Hey et al. 2009 Hey, T., Tansley, S., and Tolle, K. The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, Redmond (2009).

Howanitz et al. 2019 Howanitz, G., Bermeitinger, B., Radisch, E., Sebastian G., Rehbein, M., and Handschuh, S. "Deep Watching - Towards New Methods of Analyzing Visual Media in Cultural Studies." Book of Abstracts of the International Digital Humanities Conference (DH) (2019).

Hoyt et al. 2014 Hoyt, E., Ponot, K., and Roy, C. "Visualizing and Analyzing the Hollywood Screenplay with ScripThreads." Digital Humanities Quarterly 8.4 (2014). [http://www.digitalhumanities.org/dhq/vol/8/4/000190/000190.html]

Hołobut et al. 2016 Hołobut, A., Rybicki, J., and Woźniak, M. "Stylometry on the Silver Screen: Authorial and Translatorial Signals in Film Dialogue." Book of Abstracts of the International Digital Humanities Conference (DH) (2016).

Juang and Rabiner 2005 Juang, B.-H. and Rabiner, L. R. "Automatic speech recognition - a brief history of the technology development." Technical report. Georgia Institute of Technology, Atlanta Rutgers University and the University of California, Santa Barbara (2005).

Keathley et al. 2019 Keathley, C., Mittell, J., Grant, C., Faden, E. and Lee, K.B. The Videographic Essay: Criticism in Sound & Image. Caboose Books, Montréal (2019).

Kuhn et al. 2015 Kuhn, V., Craig, A., Simeone, M., Satheesan, S. P., and Marini, L. "The VAT: Enhanced Video Analysis." Proceedings of the 2015 XSEDE Conference on Scientific Advancements Enabled by Enhanced Cyberinfrastructure - XSEDE '15. Louis, Missouri (2015): 1-4.

Kurzhals et al. 2016 Kurzhals, K., John, M., Heimerl, F., Kuznecov, P., and Weiskopf, D. "Visual Movie Analytics." IEEE Transactions on Multimedia 18.11 (2016).

Lew et al. 2006 Lew, M. S., Sebe, N., Djeraba, C., and Jain, R. "Content-based multimedia information retrieval: State of the art and challenges." ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 2.1 (2006): 1-19.

Manovich 2013 Manovich, L. "Visualizing Vertov." Russian Journal of Communication 5.1 (2013): 44-55.

McLuhan 1964 McLuhan, M. Understanding Media. The Extensions of Man. 1st ed. McGraw-Hill, New York (1964).

McWhirter 2015 McWhirter, A. "Film Criticism, Film Scholarship and the Video Essay." Screen 56.3 (2015): 369-377.

Mitrovic et al. 2001 Mitrovic, D., Zeppelzauer, M., Zaharieva, M., and Breiteneder, C. "Retrieval of Visual Composition in Film." Proceedings of the 12th International Workshop on Image Analysis for Multimedia Interactive Services. Delft (2001).

Mooers 2007 Mooers, C. N. "The theory of digital handling of non-numerical information and its implications to machine economics." Proceedings of the Association for Computing Machinery Conference. National Science Foundation (2007). [http://www.nsf.gov/pubs/2007/nsf0728/index.jsp?org=EEC]

Pause and Walkowski (forthcoming) Pause, J. and Walkowski, N.O. "Zombies, Conspiracies, and Esthetics in Crisis. A computational engagement with film genre." In I. Ritzer (eds.), Media and Genre. Dialogues in Aesthetics and Cultural Analysis. Cham (forthcoming)

Pause and Walkowski 2018 Pause, J. and Walkowski, N.O. "Everything is Illuminated. Zur Numerischen Analyse von Farbigkeit in Filmen." Zeitschrift für digitale Geisteswissenschaften (2018).

Pause and Walkowski 2020 Pause, J. and Walkowski, N.O. "Welten abnehmenden Lichts. Ein Multi-Scale Viewing des Politthrillers im 21. Jahrhundert." montage AV. Zeitschrift für Theorie und Geschichte audiovisueller Kommunikation 29 (2020): 59.

Pearlman and Heftberger 2018 Pearlman, K. and Heftberger, A. "Editorial: Recognising Women’s Work as Creative Work." In A. Heftberger and K. Pearlman (eds.), Women at the Editing Table: Revising Soviet Film History of the 1920s and 1930s. Special Issue of Apparatus. Film, Media and Digital Cultures in Central and Eastern Europe 6 (2018).

Pereira 2019 de sá Pereira, M. "Mixed Methodological Digital Humanities." In M. K. Gold and L. F. Klein (eds.), Debates in the Digital Humanities 2019. University of Minnesota Press, Minneapolis (2019): 405-412.

Pfeiffer et al. 1996 Pfeiffer, S., Fischer, S., and Effelsberg, W. "Automatic audio content analysis." Proceedings of the Forth ACM International Conference on Multimedia '96. Boston, MA (1996).

Pouyanfar et al. 2018 Pouyanfar, S., Yang, Y., Chen, S.-C., Shyu, M.-L., and Iyengar, S. "Multimedia big data analytics: A survey." ACM Computing Surveys (CSUR) 51.1 (2018): 10.

Roth 2019 Roth, C. "Digital, digitized, and numerical humanities." Digital Scholarship in the Humanities 34.3 (2019): 616-632.

Rui et al. 1998 Rui, Y., Huang, T. S., and Mehrotra, S. "Exploring video structure beyond the shots." Proceedings of the IEEE International Conference on Multimedia Computing and Systems (1998): 237-240.

Ruszev 2018 Ruszev, S. "Rhythmic Trajectories. Visualizing Cinematic Rhythm in Film Sequences." In Heftberger, A. and Grgic, A. (eds.), Women Cutting Movies: Editors from East and Central Europe. Special Issue of Apparatus. Film Media and Digital Cultures of Central and Eastern Europe 7 (2018).

Salt 1974 Salt, B. "Statistical Style Analysis of Motion Pictures." Film Quarterly 28.1 (1974): 13-22.

Salt 2006 Salt, B. Moving into Pictures. More on Film History, Style, and Analysis. Starword Publishing, London (2006).

Salton and McGill 1983 Salton, G. and McGill, M. Introduction to modern information retrieval. McGraw-Hill, New York (1983).

Schaffner and Erway 2014 Schaffner, J. and Erway, R. "Does Every Research Library Need a Digital Humanities Center?" OCLC Research, Dublin, Ohio (2014). [http://www.oclc.org/content/dam/research/publications/library/2014/oclcresearch-digital-humanities-center-2014.pdf]

Schöffmann et al. 2010 Schöffmann, K., Hopfgartner, F., Marques, O., Boeszoermenyi, L., and Jose, J. M. "Video browsing interfaces and applications: a review." SPIE Reviews 1.1 (2010).

Sittel 2017 Sittel, J. "Digital Humanities in der Filmwissenschaft." MEDIENwissenschaft: Rezensionen | Reviews 4 (2017): 472.

Snoek and Worring 2005 Snoek, C. G. and Worring, M. "Multimodal video indexing: A review of the state-of-the-art." Multimedia Tools and Applications 25.1 (2005): 5-35.

Soleymani et al. 2009 Soleymani, M., Chanel, G., Kierkels, J. J., and Pun, T. "Affective characterization of movie scenes based on content analysis and physiological changes." International Journal of Semantic Computing 3.2 (2009): 235-254.

Svensson 2010 Svensson, P. "The Landscape of Digital Humanities." Digital Humanities Quarterly 4.1 (2010). [http://digitalhumanities.org/dhq/vol/4/1/000080/000080.html]

Svensson 2016 Svensson, P. Big Digital Humanities: Imagining a Meeting Place for the Humanities and the Digital. University of Michigan Press, Ann Arbor (2016).

Terras 2013 Terras, M. "Peering Inside the Big Tent." In M. Terras, J. Nyhan and E. Vanhoutte (eds.), Defining Digital Humanities, London (2013): 263-270.

Tsivian 2009 Tsivian, Y. "Cinemetrics, Part of the Humanities' Cyberinfrastructure." In M. Ross, M. Grauer and B. Freisleben (eds.), Digital Tools in Media Studies. Bielefeld (2009): 93-100.

Tzelepis et al. 2016 Tzelepis, C., Ma, Z., Mezaris, V., Ionescu, B., Kompatsiaris, I., Boato, G., Sebe, N., and Yan, S. "Event-based media processing and analysis: A survey of the literature." Image and Vision Computing 53 (2016): 3-19.

Vonderau 2017 Vonderau, P. "Quantitative Werkzeuge." In M. Hagener and V. Pantenburg (eds.), Handbuch Filmanalyse. Wiesbaden (2017): 1-15.

Wold et al. 1996 Wold, T., Blum, D., and Wheaton, J. "Content-based classification, search, and retrieval of audio." IEEE Multimedia 3 (1996): 27-36.

Zaharieva et al. 2010 Zaharieva, M., Zeppelzauer, M., Mitrovic, D., and Breiteneder, C. "Archive film comparison." International Journal of Multimedia Data Engineering and Management 1.3 (2010): 41-56.

Zeppelzauer et al. 2011a Zeppelzauer, M., Zaharieva, M., Mitrovi¢, D., and Breiteneder, C. "Retrieval of motion composition in Film." Digital Creativity 22.4 (2011): 219-234.

Zeppelzauer et al. 2011b Zeppelzauer, M., Mitrovic, D., and Breiteneder, C. "Cross-modal analysis of audio-visual film montage." Proceedings of 20th International Conference on Computer Communications and Networks (ICCCN) (2011): 1-6.

Zhao et al. 2017 Zhao, B., Feng, J., Wu, X., and Yan, S. "A survey on deep learning-based fine-grained object classification and semantic segmentation." International Journal of Automation and Computing 14.2 (2017): 119-135.

Zhou et al. 2017 Zhou, W., Li, H., and Tian, Q. "Recent advance in content-based image retrieval: A literature survey." arXiv preprint, arXiv:1706.06064 (2017).

van Rijsbergen 1979 van Rijsbergen, C. Information Retrieval. Butterworth-Heinemann, London (1979).