From still to moving images and vice versa: Analysing technological cycles and the use of AI to study cinema history

by Beatriz Tadeo Fuica and Arthur Lezer

Introduction

Since at least 1996, when Toy Story (John Lasseter), the first exclusively digitally-made film was released, the relationship between cinema and this new technology has radically changed. As Thomas Elseasser and Malte Hagener have argued, this ‘marked a crucial moment in film history’.^[1] In addition to production, digital technologies have been employed by film archivists to digitise, preserve, and provide access to their collections.^[2] For a few decades already, the digital turn has shaken the definition of cinema, announcing its death, exploring changes in the experience of film viewing, promoting discussion on film preservation and access, and enabling alternative means of circulation and distribution.^[3] Thanks to an increasing availability of tools for the computational treatment of images, digital technologies are also promoting different approaches to cinema analysis. These new technologies invite us to revise established theories, disciplinary needs, and methods.

In cinema studies, the use of digital tools for quantitative and qualitative analyses has been growing steadily, promoting and exploring crossovers with digital humanities.^[4] Some cinema and media historians began using computer-aided tools to explore specialised publications and geo-localise their research findings.^[5] The incorporation of data such as the quantification of shot length, specific camera movements and other similar techniques, using more or less sophisticated tools to analyse aspects of cinematic visual and time-based components, has been in discussion for a few decades.^[6] Film scholars have also adopted diverse annotation systems to keep track of specific information, either manually, automatically, or using a combination of both; and their advantages and disadvantages have been largely evaluated.^[7]Data-visualisation tools and methods have also been used and discussed.^[8] Moreover, experiments have been conducted that explore the possibilities of carrying out automatic studies of visual compositions and different aspects of sound in moving images.^[9] In this context, this article uses an object retrieval AI as a tool to bring cinema historians’ attention to similar images contained in a large film collection. By identifying similar objects in photogrammes of films that belong to a corpus, the AI is used to make large data ‘scalable’,^[10] enabling the analysis of the images through qualitative studies of contextual and historical factors.

The AI that we have used for our theoretical and methodological reflections searches for objects in a film corpus, extracting the photogrammes which contain them. Although cinema is an audio-visual medium – and a crucial difference between photography and cinema is that the latter concerns moving images – as we demonstrate below, there are fruitful ways of analysing cinema history by establishing both aesthetic and historical connections between photogrammes from a corpus. Studying cinema through an object retrieval AI system invites us to explore a technological cycle that started when cinema was born and instantaneous photographs were set into motion. Today, an object retrieval AI allows us to focus on details in photogrammes in ways that are both similar and different from the possibilities we had before: similar, because whether a photogramme is retrieved by an AI or chosen manually by a researcher, in both cases, it is a still from a film; different, because an object retrieval AI allows us to work with large collections of films at once and it can identify similarities between photogrammes that would escape the naked eye. This means that we can move beyond the analysis of a specific film narrative and study connections between images coming from a film corpus constructed under specific circumstances, for example, with films that shared a distribution channel, as our case study shows.

After providing some historical information on the technological cycle that we have followed and reflecting theoretically on how objects repeated in photogrammes within a corpus absorb, reflect, and shape history, we propose a practical analysis. Through the search of wheels in a corpus of films conceived by members of the pioneer generation of film archivists at the beginning of the 1950s, we demonstrate that aesthetic connections trigger historical analyses that bring to the fore the specificities of the context in which this corpus was created. We also reflect on the possibilities and limitations of using an object retrieval AI to launch the search of such a ubiquitous object. Ultimately, this article shows one way in which a cinema history research project can incorporate this tool.

Technological cycles: Visible or invisible to the ‘naked eye’?

The invention of cinema led to the emergence of the photogramme, a particular kind of photograph which was set into motion.^[11] At the beginning, studying photogrammes was not easy. As Laura Mulvey emphasised, ‘celluloid consists of a series of still frames that have been, by and large, inaccessible to the film spectator throughout its history’.^[12] Indeed, writing at the end of the 1970s, Raymond Bellour mentions the need for an editing or a re-winding table to see ‘the perpetual oscillation, from one frame to another’.^[13] In spite of the difficulties to access photogrammes, in the 1960s and 1970s, these minimal units were still perceived as necessary for film analysis and fruitful theoretical debates emerged.^[14]Specific attention to this unit has been growing as photogrammes became more accessible thanks to the possibility of freezing the flow of the film. As Mulvey argues ‘[n]ow, cinema’s stillness, a projected film’s best-kept secret, can be easily revealed at the simple touch of a button, carrying with it not only the suggestion of the still frame, but also of the stillness of photography’.^[15]

Writing at the beginning of the 21^st century, Mulvey points at the action of a ‘touch of a button’ and focuses on a way of accessing still frames manually. First, magnetic, and then, digital video made this possible. The implications of the ‘pause’ button is also at the core of Nicholas Rombes’ study.^[16] Today, almost 20 years after the publication of Mulvey’s book and barely five years after Rombes’ revised 2009 edition, newer technologies such as AI systems capable of retrieving similar objects in large corpora of films open innovative ways of watching and analysing films. By extracting relevant photogrammes, an AI system that retrieves objects allows researchers to approach cinema history through deep analysis of the secrets and details contained in film frames.

Since AI is a very fluid concept, which includes different training methods and purposes, it is important to briefly describe the technology on which our reflections are based. For this article we used Snoop, an Image Retrieval (IR) software developed by the INA (French National Audiovisual Institute) and the INRIA (French National Institute for Research in Digital Science and Technology).^[17] Snoop is a visual search engine aimed at retrieving similar images or excerpts of video within large collections. Its distinctive feature is its ‘human-in-the-loop’ design, which relies on collecting feedback from the user to improve the relevance of search results.^[18] The search process in Snoop is iterative and relies on Active Learning.^[19] This means that the classification model incorporates the user feedback to yield new and better search results each time.^[20]

There are two phases in Snoop’s training. Before Snoop is accessed by any user, there is an ‘offline phase’. Later, the ‘online phase’ defines the interaction between Snoop and the user, during which Active Learning takes place. The concepts of offline/online used here are unrelated to accessing data from the internet, which Snoop does not. Instead, they refer to parsing all the available dataset of images once (offline) vs learning iteratively from a few examples (online). At the beginning of the offline phase, still images are extracted from digital copies of the films, coded at 25 frames per second. One out of every four frames is kept. Then, each image is represented by a vector: a mathematical object which encodes its most salient features. This is done using a Convolutional Neural Network following the InceptionV3 architecture,^[21] which was pre-trained on a subset of the ImageNet corpus.^[22] The pre-training data consists of 1.3 million images, mostly photographs from Flickr. Each image represents one of 1,000 categories of objects, including people. These categories were chosen to introduce maximum diversity of elements such as scale, clutter, texture, and colour distinctiveness in the pre-training data. From these diverse images, the network learns features which are highly generalisable to other datasets, such as background/foreground delimiters, shapes, and objects. It is worth emphasising that the choice of the pre-trained model during this stage determines which data is kept in order to compute similarity at the search time, and it has an impact on the results regardless of the user feedback. For instance, it has been demonstrated that features learned from ImageNet emphasise textures more than shapes.^[23] Finally, descriptor vectors are indexed using a hashing function designed to allow efficient similarity search during the online phase.^[24]

The online phase begins with the user providing one or several image examples as a query. Similar images are identified and returned using the approximate K-nearest neighbours (ANN) algorithm. The user is presented with a set of 0-100 search results, which she may label as relevant or irrelevant to the query. For instance, the user may want to retrieve images of dogs. False positive results, such as images containing cats, horses or any other kind of irrelevant content retrieved by Snoop, would be marked as negative, and only dogs will be confirmed as positive. Then the user may begin another iteration of that same query. At each iteration, a classification function is automatically learned to update the set of images to be presented to the user for relevance evaluation; that is active selection.

At each iteration, the set of results that is prioritised to be shown to the user for annotation reflects, by default, the best compromise between relevancy (images likely to be positive) and novelty (previously unseen images). The user can choose other functions that, unlike the default selection function, yield more novel, negative, or ambiguous images, whose annotation may be useful to improve the classifier. The search time elapsed between two iterations is between 2-3 seconds. No information is used or displayed as to which original film a given image belongs to. The researcher can rely on her previous knowledge to recognise the original film, but in the context of Snoop, the search space is defined as a corpus of unrelated still images. It is up to the researcher to decide when a cycle of iterations is over; it can be when the classification model exhibits satisfactory accuracy and robustness, when novel images no longer appear within the search results, or simply when enough relevant results have been retrieved. When the search is over, the researcher can save the results and export all images from a class into a PDF document, along with metadata such as the film of origin and timestamp. In this way, the original moving images that composed the corpus become a collection of selected still photogrammes.

If at the end of the 20^th century cinema put instantaneous photographs into motion, today, almost at the end of the first quarter of the 21^st, an object retrieval AI offers the possibility to stop that motion and extract photogrammes for research with a level of precision unattainable to humans. This means that we can think of research questions to be answered by analysing film corpora rather than only a few films, and that our analyses can focus on aesthetic and historical connections rather than on interpretations of the films’ narratives. This method allows us to assess novel ways in which elements of the mise-en-scène can transform the historical understanding of cinema, focusing, for example, on how specific images interact with the context in which a film corpus was created, as we shall demonstrate below.

Theoretical implications: Third meaning and beyond

In 1970, Roland Barthes published an influential article in Les Cahiers du cinema that was translated into English as ‘The Third Meaning. Research notes on some Eisenstein stills’.^[25] According to Barthes, photogrammes have three layers of meaning: first, an information level or level of communication; second, a symbolic level or level of signification; and third, what he calls ‘third or obtuse meaning’, which triggers different sets of associations and knowledge. In order to present the characteristics of this third (or obtuse) meaning, Barthes points at details from stills of Eisentein’s Battleship Potemkin (1925) and Ivan the Terrible (1943) that attracted his attention, such as: ‘finely traced eyebrows’,^[26] ‘the headscarf holding in the hair’,^[27] ‘Ivan’s beard’,^[28] ‘a single bun of hair’.^[29] In order to analyse these details, the photogrammes which illustrate his article have been observed as units of meaning. Indeed, as Barthes pointed out, ‘obtuse meaning is discontinuous, indifferent to the story and to the obvious meaning (as signification of the story)’.^[30] Thinking of photogrammes as independent units that could trigger associations not necessarily linked to the film narrative they belong to is an important theoretical step for studying cinema history through the stills rendered by an object retrieval AI system. Nowadays, details can be looked for in large collections of films to explore new meanings and associations.

When Jean Luc Godard gave the lectures at the University of Montreal that preceded his eight-episode series Histoire(s) du cinéma, he referred to the importance of establishing connections when approaching cinema history. The methodology he applied on those occasions was based on screenings of one of his films and either another film or fragments of films that he thought were somehow connected to his. A follow-up discussion with students would reflect on these not-always-obvious connections that Godard had in mind when choosing the films and were the result of his affective relationship with cinema. The eight-episode series that came out of this project, made between 1988 and 1998, is the result of Godard’s belief that ‘cinema history is inside it: images following one another’.^[31] His film depicts and reflects on the mutual influence of cinema and the 20^th century based on associations of images and ideas that Godard established between cinema itself, cinema and other arts, and his own cultural baggage.^[32]

In Histoire(s), Godard points at the importance of objects, for example when referring to Alfred Hitchcock’s films, and emphasises that although we forget plot narratives we remember the objects contained in the British filmmaker’s oeuvre, such as a handbag, a bus, a glass of milk, sails of a windmill, a hairbrush, bottles in a line, a pair of glasses, etc. According to Godard, these objects trigger memories that we associate with the films.^[33] In relation to these same objects Mulvey refers to the exhibition Hitchcock and Art, in which, ‘[i]n a darkened room, each on a Perspex pedestal, carefully lit and placed on a red velvet cushion, were the objects that each encapsulate a certain Hitchcock movie’.^[34]Although not every film from cinema history would necessarily provide us with such an ‘encapsulating’ object, ordinary film objects have frequently been the source of cinema and media scholars’ analysis.^[35]

Analysing specific objects contained in photogrammes is a well-established practice in film studies that allows different interpretations based on their symbolic, affective, haptic, and narrative functions. Some scholars provide analysis contained within the narrative of the films;^[36] others also reach out to the production context to interpret the presence of specific objects within film narratives and approach their studies on historical grounds;^[37] they all share the fact of analysing only a few films at a time, which has been the most accepted methodology.^[38] However, when analysing a corpus, an object retrieval AI offers the possibility of establishing historical analyses that go beyond film narratives, focusing mainly on objects, on how they are framed, on how they relate to production and distribution contexts, to gain a further understanding of the corpus to which they belong. Rather than establishing emotional connections, the methodology herein presented aims to propose academic analysis. As Mulvey suggests,

As it is repeated, the image insists and persists so that the repetition and return enabled by the machine are echoed in the image’s own repetition of meaning. This process is not only useful for an academic or critical practice, it has its own visual pleasures and rewards that do not replace, but complement, those of watching a film in its traditional temporality and context.^[39]

Mulvey’s reflections, which originate when analysing Imitation of Life (Douglas Sirk, 1959), are relevant to our proposition albeit our different applications. Rather than focusing on insistence and repetition within a single film, as Mulvey does, we suggest focusing on repetition within a corpus, under the belief that there are further meanings to be explored when establishing connections within a large film corpus created under specific circumstances. For this, the relationship between an object and the narrative is not the core of the analysis any longer. Rather, the images found in several films and the connections they can trigger become the focus of the study. As Mulvey has acknowledged, ‘extracting it [the image] from its narrative surroundings, also allows it to return to its context and to contribute something extra and unexpected, to a deferred meaning, to the story’s narration’.^[40] Rather than contributing to the story’s narration, given the large number of films we can handle with AI, we would suggest establishing connections with the extra-diegetic context. This is a possibility that Mulvey has also explored when linking images from Imitation of Life with the racial issues raised by the Civil Right movements in the United States at the time of the film’s release.^[41]

The last theoretical aspect to deal with before putting our ideas into practice is the definition of corpus or collection that we consider relevant to this kind of analysis. Erika Balsom, in her study of how mechanisms of legal and illegal film copying have affected cinema’s mobility, emphasises that ‘distribution’ and ‘circulation’ are key factors that intervene in the possibility of a film reaching its audience. Although these concepts can be defined differently, we adhere to the definitions that Balsom proposes, highlighting that ‘distribution designates the infrastructures (whether formal or informal) that make work available to be seen, and circulation designates the trajectories particular works can take through one or more distribution models’.^[42]

In a study using AI the way we propose here, possible corpora to be analysed include the group of films screened at any city at a specific time, the catalogue of an online platform, as well as a selection for a film festival or a set of films exhibited in an art gallery. As Balsom argues, ‘[d]istribution participates in the generation of value and canon formation, as particular works may be made widely available to be seen and written about, while others remain inaccessible’.^[43]Similarly, films that belong to the collection of a cinematheque are more likely to enter cinema history than those which are not. However, the intention here is not to focus on particular works or unveiling how certain distribution channels contributed to the canonisation of specific titles or filmmakers, as it has been done elsewhere.^[44] Rather, turning to asearch of the shared presence of specific objects within the films of a corpus distributed and circulated through specific channels allows us to reflect on how these images construct further meaning. This approach sees each corpus as a supra-film made up of all the films that constitute it and by extension all their photogrammes. A transversal analysis of the images that circulated through a particular distribution channel means understanding that through the search of specific shared film details, it is possible to write a cinema history that acknowledges the importance not only of these channels but also of how the historical context reflects, absorbs, and shapes images, and vice versa.

In practice: In search of wheels

The corpus that we used for our analyses is composed of 64 films made between the years 1902 and 1952, which were part of a larger corpus built during the project TRANSARCHIVES. Film Heritage and Archival Practices: Past and Present Transcontinental Encounters. Transarchive’s corpus includes titles mentioned in the correspondence between Henri Langlois (French Cinematheque), Paulo Emilio Sales Gomes (Brazilian Cinematheque), Rolando Fustiñana (Argentinean Cinematheque), Danilo Trelles (Uruguay’s SODRE Archive), and Walter Dassori and Eugenio Hintz (Uruguayan Cinematheque) from October 1947 to January 1955. So far, 383 films have been traced, including fiction and nonfiction films of diverse length: feature, medium and short. Although not all the films have circulated between the institutions behind the correspondences, as we have argued elsewhere, we consider that ‘the fact that a film was mentioned by this pioneer generation of film archivists suggests that it was part, at least, of an “imaginary collection” of films that deserves attention’.^[45] Even if only metaphorically, these titles have followed a shared circulation path.

Given the large number of digital films available today, it may be believed that the practical aspects of using AI might not necessarily be complex. However, as we have explained in detail in a previous publication, it is not necessarily easy to have an AI system available at an archive that has digital copies of the films to be analysed.^[46] This is the reason why the original list was reduced. We could only use the titles that were readily available for treatment with Snoop at the INA. The resulting corpus is then composed by films which have the double characteristic of having been included in the correspondences between the aforementioned archivists and of having been broadcast on French television, and therefore available at the INA.^[47] This feature did not have a considerable impact on specific variables of our corpus, such as the ratio between origin and quantity. Although constructed through the interaction between archivists from three Latin American and one European archive, the original corpus does not have a salient presence of Latin American films. In addition, the few Latin American films that are part of the exchanges are difficult to trace today, as it has been analysed elsewhere.^[48] Most of the films from Transarchives were from France, followed by titles from Germany, the URSS and the USA, as can be seen in figure 2. Based on films available at the INA, the corpus we treated with Snoop only contains films from these four origins but reflects the same proportion within the original corpus. The main difference between both was that German films were overall more present in the corpus treated with Snoop than in that of Transarchives, as we can see in Figure 3.

As for the object that would serve as the basis for the search, in trying to find a rich example to put our ideas into practice, we chose to look for wheels. The rationale behind this decision was based on the recurrent presence of the abstract circular shape triggered by the idea of cycle, repetition, and iteration that have permeated this project. In addition, the corpus we had available contained the film La roue (The Wheel, Abel Gance, 1923), which seemed to be a funny and relevant coincidence. However, beyond the symbolic aspects of this choice, the interest in studying wheels also lied in their ubiquity. On the one hand, we wondered whether the fact that wheels were present in so many objects would provide us with a general idea of the corpus and allow us to come up with unforeseen connections. On the other hand, we thought that the search for such a ubiquitous object would allow us to test the technology and reflect on the possibilities and limitations of using an AI system like Snoop for studying cinema history.

After conducting approximately 30 rounds of searches, we ended up with a file of 482 frames showing wheels. Before starting, we decided not to engage in a quantitative analysis. Therefore, during our interaction with Snoop, we did not always mark positive examples from the same sequence. This decision aimed to reduce the number of similar photogrammes from the same sequence in the final file. For each project, the researcher needs to decide how to strike a balance between the online training of Snoop and the research project’s needs. Having repeated images in the search results is unavoidable because, statistically, the most similar images to a given photogramme are always other photogrammes from the same sequence. The ‘novelty’ parameter favours images from other sequences, but excerpts from a single sequence are bound to appear again. If we only keep one and mark the rest as negative, Snoop will be confused, and this will have an impact on further findings. These are all methodological choices that affect results.

As expected, the photogrammes resulting from our experiments contained plenty of means of transport which included cars, carriages, and carts both pulled by horses and humans. This allowed us to quickly ‘watch’ our corpus and examine its heterogeneity in terms of subjects and locations. The train, predictably, was very present in the retrieved images. In this sense, Snoop served to validate previous research which has highlighted the fascination that cinema had with this means of transport from the very beginning.^[49] This fascination is visible in the close ups of train wheels that Snoop identified in Die Liebe der Jeanne Ney (The Love of Jeanne Ney, Georg Wilhelm Pabst, 1927), La roue, and Goluboy ekspress (The Blue Express, Ilya Trauberg, 1929) (Figure 4). The tension between the inescapable interaction between humans and machines can be perceived in frames from The Wheel and The Blue Express, which show the human figure by the side of enormous train wheels (Figure 5).

 

There were other findings which showed aesthetic coincidences between stills from different films that caught our attention. We noticed that the way both J’accuse (I Accuse, Abel Gance, 1919) and Le ciel est à vous (The Woman Who Dared, Jean Grémillon, 1944) portrayed the arrival of trains at stations bore a striking similarity between them and also with L’Arrivée d’un train en gare de La Ciotat (The arrival of a train at La Ciotat Station, Louis Lumière, 1985), which was not part of our corpus but was an unavoidable reference (Figure 6). These coincidences between French films made roughly every twenty years can be interpreted as a punctuation which acknowledges previous generations of filmmakers.

However, when seeing these photogrammes we wondered in what ways these coincidences were relevant in relation to our corpus. We side with Amanda Wasielewski’s observation that ‘[a] computational technique can determine whether certain images match one another, but it cannot tell you why they match or whether the fact that they match is relevant information’.^[50] For our case, we thought this match was relevant inasmuch it obliged us to reflect on the meaning of European trains arriving to Latin American screens in the 1950s.

Getting back to scholars that have studied trains in cinema, we found it relevant that Leo Charney and Vanessa R. Schwartz claimed that both cinema and the railroad are part of a ‘few talismanic innovations’ that allow us to grasp ‘“[m]odernity,” as an expression of changes in so-called subjective experience or as a shorthand for broad social, economic, and cultural transformations’.^[51] In the same vein, Lynne Kirby establishes connections between train, cinema, and modernity, and has argued that ‘[b]y incorporating the perceptual disorientation associated with the rapid rush of movement bound up with the train, these films [railroad films from the 1920s] celebrated modernity as a liberating force and aimed to free film from its status as an instrument of bourgeois domination’.^[52] These appreciations seem relevant to explore the context of our corpus, set at the beginning of the Cold War, a time of economic, social, and political disruption in which discourses related to ‘bourgeois domination’ would abound. In addition, as Kirby’s research argues, ‘railways outside the West appeared initially as an extension of Western imperialism’.^[53] This observation, again, emphasised that our corpus was constructed between Europe and Latin America. The images triggered by Snoop allow us to think about the ways ‘the West’ – Europe and the United States – have interacted with Latin America and have expanded modernity.

Urbanisation, one of the stamps of modernity, was a salient characteristic of the early 1950s, when the capitals of Argentina (Buenos Aires) and Uruguay (Montevideo), and the largest industrial city of southeast Brazil (Sao Paulo), saw the growth of a strong middle class thanks to the wealth accumulated during the Second World War. Intellectual elites from these cities would often travel to Europe and consume lots of culture, cinema included. This exceptional period of prosperity did not change the (neo)colonial order and, during the Cold War, the interference of the United States into Latin America had important political consequences. The social, political, and economic crises that began growing in the mid- and late-1950s led to repressive regimes that governed roughly from the mid-1960s to the late 1980s, depending on the country.

These connections with the idea of conflict, initially triggered by photogrammes portraying trains, emerged again when realising that in other photogrammes rendered by Snoop, wheels were also present to move cannons and carriages of soldiers carrying rifles (see Figure 7). We anticipate that we could have harvested similar images through a search of specific guns. However, the fact that we did so through wheels highlights the mobility of armed conflicts, most of the time caused by displacements, invasions, colonialism. This idea, again, brings connections to the specificities of the context of the Cold War in which the corpus was constructed.

These examples show possible ways of analysing the relevance of Snoop’s retrievals, avoiding a focus on the individual film narratives and paying attention, instead, to some of the characteristics of the corpus. In a follow-up project, our analyses could be further expanded by considering, for instance, when these films were digitised, if they were restored, if/when they were screened recently, and interrogate further the context and channels that today gather them together in relation to the images retrieved by Snoop.

Conclusions: Possibilities and limitations of a ubiquitous object

This article followed a technological cycle from still to moving images and vice versa to reflect on how cinema historians can benefit from an object retrieval AI system like Snoop,

to write a cinema history that departs from the analyses of objects, through details in photogrammes and aesthetic associations, in order to explore the corpus’ historical context to better apprehend its characteristics. The choice of the corpus as well as that of the objects to be retrieved is crucial when using a tool like Snoop, and we still have a lot to learn about making the right choices. Although we are convinced that our corpus is extremely rich and we should continue exploring it through Snoop, we realised that searching for a single ubiquitous object like a wheel had its limitations.

As anticipated, in its search for wheels, Snoop retrieved too many examples. On the one hand, this was very practical because we managed to have an idea of the corpus in a few hundred images rather than engaging in hours of film viewing. Although it was rather tedious to organise the images, this became a fascinating endeavour, especially as we began finding aesthetic coincidences that triggered fruitful readings. Methodologically, we realised that focusing on a single object was good to get further insights into the possibilities offered by Snoop but that it was rather limiting as far as the analyses were concerned. As we have suggested above, crossing the images of wheels with photogrammes containing guns, for example, would be a productive way of approaching our corpus to reflect on armed conflicts. The ease at which searches can be conducted with Snoop is a feature that would enrich further comparisons and should certainly be exploited by researchers. This very attractive feature allows serendipitous searches based on unexpected matches that can contribute to better apprehend the corpus’ characteristics. Unless the object is very specific, we realised that constraining a project to one object does not allow us to benefit from Snoop’s full potential.

While not yet available to any spectator, based on the speed at which generative AI has been made accessible in the last year,^[54] it seems likely that an AI system like Snoop will be largely available sooner than later. Although for this article our reflections are thought in terms of an interactive researcher who encapsulates a research question in an image which launches a search, it seems likely that, in a very short time, other kinds of spectators will be accessing, enjoying, and interpreting films likewise. This should motivate us to continue exploring the possibilities and limitations of this technology and reflecting on which is the best way to incorporate it for our research.

Acknowledgements

This project has received funding from the 2023 FIAT/IFTA Medias Studies Grant and has been hosted at INA-LeLab since the beginning of 2023. The Project TRANSARCHIVES, which enabled the tracing of the films included in this article, was funded by the European Union’s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No 746257.

Authors

Beatriz Tadeo Fuica is an associate fellow at the IRCAV, Université Sorbonne Nouvelle, where she developed the project TRANSARCHIVES, supported by a Marie Sklodowska-Curie Fellowship. Her current project, which explores ways of using AI for writing the history of cinema, is hosted at INA Le Lab (France). She holds a PhD from the University of St Andrews (UK).

Arthur Lezer is a research engineer, computational social scientist, and coordinator of INA Le Lab, the data lab of the French National Audiovisual Institute (INA). He holds degrees in cultural policymaking from Sciences Po (France) and data science & sociology from Université Gustave Eiffel (France).

References

Acland, C. ‘“Opening Everywhere”. Multiplexes and the Speed of Cinema Culture’ in Going to the movies: Hollywood and the social experience of cinema, edited by R. Maltby, M. Stokes, and R. Allen. Exeter: University of Exeter Press, 2007: 364-382.

Allen, R. ‘Reimagining the History of the Experience of Cinema in a Post-Moviegoing Age’ in Explorations in new cinema history: Approaches and case studies, edited by R. Maltby, D. Biltereyst, and P. Meers. Malden: Wiley-Blackwell, 2011: 41-57.

Bakels, J., Grotkopp, M., Scherer, T., and Stratil, J. ‘Matching Computational Analysis and Human Experience: Performative Arts and the Digital Humanities’, DHQ: Digital Humanities Quarterly, Vol. 14, No. 4, 2020.

Balsom, E. After uniqueness: A history of film and video art in circulation. New York: Columbia University Press, 2017.

Barthes, R. ‘The Third Meaning. Research Notes on Some Eisenstein Stills’ in Image, music, text: Essays, translated by S. Heath. London: Fontana, 1977: 52-68.

Bellour, R. ‘Cine-Repetitions’, Screen, Vol. 20, No. 2, 1979: 65-72.

Bernardi, J., Usai, P. Williams, T., and Yumibe, J. (eds) Provenance and early cinema. Bloomington: Indiana University Press, 2021.

Bordwell, D. ‘Intensified Continuity Visual Style in Contemporary American Film’, Film Quarterly, Vol. 55, No. 3, 2002: 16-28.

Burges, J., Dimmock, N., and Romphf, J. ‘Collective Reading: Shot Analysis and Data Visualization in the Digital Humanities’, Cinema Journal Teaching Dossier, 3, January 2016.

Burghardt, M., Hafner, K., Edel, L., Kenaan, S., and Wolff, C. ‘An Information System for the Analysis of Color Distributions in MovieBarcodes’ in ISI, 2017.

Burghardt, M., Heftberger, A., Pause, J., Walkowski, N., and Zeppelzauer, M. ‘Film and Video Analysis in the Digital Humanities – An Interdisciplinary Dialog’, Digital Humanities Quarterly, Vol. 14, No. 4, 2020.

Byszuk, J. ‘The Voices of Doctor Who – How Stylometry Can Be Useful in Revealing New Information About TV Series’, Digital Humanities Quarterly, Vol. 14, No. 4, 2020.

Charney, L. and Schwartz, V. (eds) Cinema and the invention of modern life. Berkeley: University of California Press, 1995a.

_____. ‘Introduction’ in Cinema and the invention of modern life. Berkeley: University of California Press, 1995b: 1-12.

Cherchi Usai, P. The death of cinema: History, cultural memory, and the digital dark age. London: BFI Pub, 2001.

Cutting, J. and Candan, A. ‘Shot Durations, Shot Classes, and the Increased Pace of Popular Movies’, Projections, Vol. 9, No. 2, 2015: 40-62.

Dicker, B. ‘Editorial: “Starting, Stopping and Synthesizing Cinematographic Art”’, Art in Translation, Vol. 8, No. 1, 2016: 1-10.

Doukhan, D., Poels, G., Rezgui, Z., and Carrive, J. ‘Describing Gender Equality in French Audiovisual Streams with a Deep Learning Approach’, VIEW Journal of European Television History and Culture, Vol. 7, No. 14, 2018: 103-122.

Elsaesser, T. ‘Is Nothing New? Turn-of-the-Century Epistemes in Film History’ in A companion to early cinema, edited by A. Gaudreault, N. Dulac, and S. Hidalgo. Malden: Wiley-Blackwell, 2012: 587-609.

Elsaesser, T. and Hagener, M. Film theory: An introduction through the senses, second edition. New York: Routledge, 2015.

Ezra, E. The cinema of things: Globalization and the posthuman object. New York-London-Oxford: Bloomsbury Academic, 2018.

Ezra, E. and Wheatley, C. ‘Introduction to Special Issue: Film Objects’, Film-Philosophy, Vol. 27, No. 1, 2023: 1-6.

Ferguson, K. ‘Digital Surrealism: Visualizing Walt Disney Animation Studios’, Digital Humanities Quarterly, Vol. 11, No. 1, 2017.

Fickers, A., Snickars, P., and Williams, M. ‘Editorial: Special Issue Audiovisual Data in Digital Humanities’, VIEW Journal of European Television History and Culture, Vol. 7, No. 14, 2018: 1-4.

Flueckiger, B. and Halter, G. ‘Methods and Advanced Tools for the Analysis of Film Colors in Digital Humanities’, Digital Humanities Quarterly, Vol. 14, No. 4, 2020.

Fossati, G. From grain to pixel: The archival life of film in transition, third revised edition. Amsterdam: Amsterdam University Press, 2018.

Frick, C. Saving cinema: The politics of preservation. New York: Oxford University Press, 2011.

Gambarato, R. ‘Methodology for Film Analysis: The Role of Objects in Films’, Revista Fronteiras – Estudos Midiáticos, Vol. 12, No. 2, 2010: 105-115.

Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F., and Brendel, W. ‘ImageNet-Trained CNNs Are Biased towards Texture; Increasing Shape Bias Improves Accuracy and Robustness’, arXiv, 2018.

Godard, J. Introduction to a true history of cinema and television, first edition, translated by T. Montreal: Caboose, 2014.

Grant, C. ‘Film and Moving Image Studies: Re-Born Digital? Some Participant Observations’, Frames Cinema Journal, Vol. 1, No. 1, 2012.

Halter, G., Ballester-Ripoll, R., Flückiger, B., and Pajarola, R. ‘VIAN – a Visual Annotation Tool for Film Analysis’. Wiley-Blackwell Publishing, Inc, 2019.

Heftberger, A. Digital humanities and film studies: Visualising Dziga Vertov’s work. Cham: Springer International Publishing, 2018.

Hielscher, E. ‘The Phenomenon of Interwar City Symphonies: A Combined Methodology of Digital Tools and Traditional Film Analysis Methods to Study Visual Motifs and Structural

Patterns of Experimental-Documentary City Films’, Digital Humanities Quarterly, Vol. 14, No. 4, 2020.

Holobut, A. and Rybicki, J. ‘The Stylometry of Film Dialogue: Pros and Pitfalls’, Digital Humanities Quarterly, Vol. 14, No. 4, 2020.

Hoyt, E. ‘Lenses for Lantern: Data Mining, Visualization, and Excavating Film History’s Neglected Sources’, Film History, Vol. 26, No. 2, 2014: 146.

Jacobs, S. ‘The History and Aesthetics of the Classical Film Still’, History of Photography, Vol. 34, No. 4, 2014: 373-386.

Joly, A. and Buisson, O. ‘A Posteriori Multi-Probe Locality Sensitive Hashing’ in Proceeding of the 16th ACM International Conference on Multimedia – MM ’08, 209. Vancouver: ACM Press, 2008.

Kirby, L. Parallel tracks: The railroad and silent cinema. Durham: Duke University Press, 1997.

Latsis, D. and Ingravalle, G. ‘Guest Editors’ Foreword: Digital Humanities and/in Film Archives’, The Moving Image: The Journal of the Association of Moving Image Archivists, Vol. 17, No. 2, 2017: xi-xv.

Lewit, I. ‘Rethinking Inside and Outside: The Door in Ernst Lubitsch’s When I Was Dead and Charlie Chaplin’s The Adventurer’, Film-Philosophy, Vol. 27, No. 1, 2023: 79-97.

Li, J. and Allinson, N. ‘Relevance Feedback in Content-Based Image Retrieval: A Survey’ in Handbook on neural information processing, edited by M. Bianchini, M. Maggini, and L. Jain. Berlin-Heidelberg: Springer Berlin Heidelberg, 2013: 433-469.

Lobato, R. Shadow economies of cinema: Mapping informal film distribution. London: Palgrave Macmillan / BFI, 2012.

Long, D., Hoyt, E., Ponto, K., Tran, T., and Hughes, K. ‘Who’s Trending in 1910s American Cinema? Exploring ECHO and MHDL at Scale with Arclight’, The Moving Image: The Journal of the Association of Moving Image Archivists, Vol. 16, No. 1, 2016: 57-81.

Maltby, R., Biltereyst, D., and Meers, P. (eds) Explorations in new cinema history: Approaches and case studies. Malden: Wiley-Blackwell, 2011.

Manovich, L. Cultural analytics. Cambridge: The MIT Press, 2020.

Melgar Estrada, L., Hielscher, E., Koolen, M., Olesen, C., Noordegraaf, J., and Blom, J. ‘Film Analysis as Annotation: Exploring Current Tools’, The Moving Image: The Journal of the Association of Moving Image Archivists, Vol. 17, No. 2, 2017: 40-70.

Mitrovic, D., Zeppelzauer, M., Zaharieva, M., and Breiteneder, C. ‘Retrieval of Visual Composition in Film’ in WIAMIS 2011, 2011.

Mueller, M. ‘Scalable Reading – Scalable Reading’, 2012:https://scalablereading.northwestern.edu/?page_id=22 (accessed on 29 November 2023).

Mulvey, L. Death 24x a second: Stillness and the moving image. London: Reaktion Books, 2006.

Musik, C. and Zeppelzauer, M. ‘Computer Vision and the Digital Humanities: Adapting Image Processing Algorithms and Ground Truth through Active Learning’, VIEW Journal of European Television History and Culture, Vol. 7, No. 14, 2018: 1-14.

Olesen, C., Masson, E., van Gorp, J., Fossati, G., and Noordegraaf, J. ‘Data-Driven Research for Film History: Exploring the Jean Desmet Collection’, The Moving Image: The Journal of the Association of Moving Image Archivists, Vol. 16, No. 1, 2016: 82-105.

Peucker, B. ‘Hitchcock’s Undertexts: Objects and Language’, Film-Philosophy, Vol. 27, No. 1, 2023: 50-63.

Pierre Ulmann, S. ‘Elements for a Theory of the Photogramme’, translated by R. Elliott, Art in Translation, Vol. 8, No. 1, 2016 (orig. in 1971): 108-127.

Pustu-Iren, K., Sittel, J., Mauer, R., Bulgakowa, O., and Ewerth, R. ‘Automated Visual Content Analysis for Film Studies: Current Status and Challenges’, Digital Humanities Quarterly, Vol. 14, No. 4, 2020.

Reyes-García, E. ‘Explorations in Media Visualization’ in Hypertext2014 Extended Proceedings. Santiago de Chile: 2014.

Rombes, N. Cinema in the digital age, revised edition. London-New York: Wallflower Press, 2017.

Ross, M., Grauer, M., and Freisleben, B. (eds) Digital tools in media studies: Analysis and research, an overview. Bielefeld: transcript Verlag, 2009.

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z. et al. ‘ImageNet Large Scale Visual Recognition Challenge’, arXiv, 2014.

Salt, B. ‘Statistical Style Analysis of Motion Pictures’, Film Quarterly, Vol. 28, No. 1, 1974: 13-22.

_____. Film style and technology: History and analysis, second edition. London: Starword, 1992.

_____. Moving into pictures: More on film history, style, and analysis, Starword-London, 2006.

Sanders, E. ‘White Material: Ivory Tiles, White Womanhood, and White Supremacy in

Jean-Marc Vallée’s Sharp Objects’, Film-Philosophy, Vol. 27, No. 1, 2023: 98-114.

Shafto, S. ‘Les objets de Hitchcock, ou le monde devenu solide’ in Hitchcock et l’ art: coïncidences fatales, edited by D. Païni and G. Cogeval. Paris: Centre Pompidou, 2000.

Svanera, M., Savardi, M., Signoroni, A., Kovacs, A., and Benini, S. ‘Who Is the Film’s Director? Authorship Recognition Based on Shot Features’, IEEE MultiMedia, Vol. 26, No. 4, 2019: 43-54.

Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. ‘Rethinking the Inception Architecture for Computer Vision’, arXiv, 2015.

Tadeo Fuica, B. ‘Tracing Past Exchanges between European and South American Cinematheques: A Key to Understanding the Impact of Sharing’, Iluminace. The Journal of Film Theory, History, and Aesthetics, Vol. 31, No. 1, 2019: 27-43.

Tadeo Fuica, B., Buisson, O., and Mussou, C. ‘Watching Historical Films Through AI: Reflections on Image Retrieval from Heritage Collections’, Cinergie – Il Cinema e Le Altre Arti, No. 20, 2021: 97-112.

Tortajada, M. ‘Photographie/cinéma : paradigmes complémentaires du début du XXe siècle’ in Fixe, animé: croisement de la photographie et du cinéma au XXe siècle, edited by L. Guido and L. Lugon. Lausanne: Éd. L’Age d’Homme, 2010: 47-61.

Tsivian, Y. ‘Cinemetrics, Part of the Humanities’ Cyberinfrastructure’ in Digital tools in media studies: Analysis and research, an overview, edited by M. Ross, M. Grauer, and B. Freisleben. Bielefeld: transcript, 2009: 93-100.

Tzelepi, M. and Tefas, A. ‘Relevance Feedback in Deep Convolutional Neural Networks for Content Based Image Retrieval’ in Proceedings of the 9th Hellenic Conference on Artificial Intelligence. Thessaloniki: ACM, 2016: 1-7.

Walton, S. ‘Poetic Objects: Bachelardian Reverie, Reverberation and Repose in Claire Denis’ 35 Shots of Rum’, Film-Philosophy, Vol. 27, No. 1, 2023: 7-28.

Wasielewski, A. Computational formalism: Art history and machine learning. Cambridge-London: The MIT Press, 2023.

Witt, M. Jean-Luc Godard, cinema historian. Bloomington-Indianapolis: Indiana University Press, 2013.

---

^[1] Elsaesser & Hagener 2015, p. 194

^[2] Fossati 2018.

^[3] Acland 2007; Allen 2011; Balsom 2017; Bernardi et al. 2021; Cherchi Usai 2001; Elsaesser 2012; Frick 2011; Lobato 2012.

^[4] Burghardt et al. 2020; Heftberger 2018; Latsis & Ingravalle 2017; Grant 2012.

^[5] Hoyt 2014; Long et al. 2016; Ross & Grauer & Freisleben 2009; Maltby & Biltereyst & Meers 2011.

^[6] Bordwell 2002; Burges & Dimmock & Romphf 2016; Cutting & Candan 2015; Ferguson 2017; Heftberger 2018; Svanera et al. 2019; Tsivian 2009; Salt 1992, 2006, 1974.

^[7] Bakels et al. 2020; Halter et al. 2019; Pustu-Iren et al. 2020; Heftberger 2018; Hielscher 2020; Melgar Estrada et al. 2017

^[8] Burghardt et al. 2017; Ferguson 2017; Flueckiger & Halter 2020; Heftberger 2018; Hielscher 2020; Manovich 2020; Olesen et al. 2016; Reyes-García 2014.

^[9] Byszuk 2020; Doukhan et al. 2018; Holobut & Rybicki 2020; Mitrovic et al. 2011.

^[10] Fickers & Snickars & Williams 2018; Mueller 2012.

^[11] Jacobs 2010; Pierre Ulmann 2016 (orig. in 1971); Tortajada 2010.

^[12] Mulvey 2006, p. 26.

^[13] Bellour 1979, p. 66.

^[14] Dicker 2016.

^[15] Mulvey 2006, p. 22.

^[16] Rombes 2017, p. 108.

^[17] At the time of writing this article, this tool is made available to scholars who request INA lab’s consultancy (https://inalelab.hypotheses.org/acces-et-reservation) or answer INA lab’s yearly CFP (https://inalelab.hypotheses.org/6621) to conduct research on corpora including, but not limited to, INA audiovisual collections at the INA premises.

^[18] Li & Allinson 2013; Tzelepi & Tefas 2016.

^[19] Musik & Zeppelzauer 2018.

^[20] Ibid.

^[21] Szegedy et al. 2015.

^[22] Russakovsky et al. 2014.

^[23] Geirhos et al. 2018.

^[24] Joly & Buisson 2008.

^[25] Barthes 1977.

^[26] Ibid., p. 53.

^[27] Ibid., p. 57.

^[28] Ibid., p. 58.

^[29] Ibid., p. 58.

^[30] Ibid., p. 61.

^[31] Godard 2014, p. 211.

^[32] Witt 2013.

^[33] Shafto 2000, pp. 137-138; Mulvey 2006, p. 147.

^[34] Mulvey 2006, p. 146.

^[35] Ezra 2018; Ezra & Wheatley 2023.

^[36] Peucker 2023; Walton 2023.

^[37] Lewit 2023; Mulvey 2006, pp. 144-160; Sanders 2023.

^[38] Gambarato 2010.

^[39] Mulvey 2006, p. 149.

^[40] Ibid., p. 151.

^[41] Ibid., p. 154.

^[42] Balsom 2017, p. 3.

^[43] Ibid., p. 8.

^[44] Tadeo Fuica 2019, 2020, 2022.

^[45] Tadeo Fuica & Buisson & Mussou 2021, p. 101.

^[46] Ibid.

^[47] Ibid., pp. 100-102.

^[48] Tadeo Fuica 2019.

^[49] Charney & Schwartz 1995a; Kirby 1997.

^[50] Wasielewski 2023, p. 19.

^[51] Charney & Schwartz 1995b, p. 1.

^[52] Kirby 1997, pp. 8-9.

^[53] Ibid., p. 5.

^[54] Gefen 2023.