Unseen Screens: Eye Tracking, Magic and Misdirection

Video by Tessa Dwyer and Jenny Robinson

 

 

Creator's Statement

By Tessa Dwyer and Jenny Robinson (Monash University and RMIT University)

‘Seeing’ is no simple matter. In fact, researchers from various disciplines from psychology to philosophy still grapple to understand many of the processes and variables involved in this imprecise, catch-all concept. ‘Seeing’ has so many moving parts, as does the relationship between visual attention and visual processing or cognition. This video essay provides an introduction to the eye tracking of moving-image screens that canvasses these complexities, starting with the idea that ‘seeing’ depends upon numerous instances of not-seeing and inattention. We explore this idea in relation to magic films and filmed magic, drawing on concepts from screen theory and psychocinematics.

As Dan North details in Performing Illusions (2005), the influence of magic traditions on the development of early cinema is profound. Magic films hark back to cinema’s earliest roots and are particularly well represented in the prolific output of conjurer and director George Méliès. While magic on screen is quite distinct from magic that unfolds in front of a live, active audience, this difference does not defuse its power or fascination. The magic film constitutes an enduring genre that boasts many recent entrants including Now You See Me (Leterrier, 2013) and sequel Now You See Me 2 (Chu, 2016).

What can eye tracking technology bring to an understanding of screen media and screen culture? Eye tracking is a process whereby eye movements are charted in order to record which parts of the visual field are attended to by audiences, for how long and in which order. Eye tracking also provides a means of drilling down into attention, exploring it as a multi-layered process that involves peripheral pre-viewing, fixations (or resting spots), information encoding and extraction, and saccades (the scanning movements that occur in-between fixations).

For this essay, we collected eye tracking data from twenty-one participants who viewed a short one-minute scene from the first Now You See Me. From this data, we produced heat-mapped footage collating the gaze patterns of all participants and displaying as hottest those areas that were most fixated upon. We also produced inverse footage of this same data. Instead of overlaying the image with heat maps that obscure points of high interest, areas of the screen that were fixated upon remain visible while all else is blackened. Additionally, we produced aggregate gaze plot footage indicating the individuated order and length of each participant’s fixations, differentiated by colour.

Via these varied outputs, this data makes tangible thresholds of seeing and not-seeing. Through eye tracking technology, we pinpoint moments of misdirection, attentional splitting, and lag, connecting these findings to film editing processes described by Arthur P. Shimamura et al. (2014) as “a sort of magician’s sleight-of-hand.” Magicians have long understood the limitations inherent within visual processing, using shortfalls to manipulate and misdirect even the most attentive of audiences. In this video essay, we consider how misdirection, sleight-of-hand, and attentional blindness play out in relation to mediated experiences and screen interfaces.

Of course, there are also blind spots in eye tracking research and technologies. For rigorous eye tracking research, the viewing condition needs to be the same for each participant, yet people experience cinema in a variety of contexts. How would this change their attention? Syncing of the visual scene with the eye tracking device is critical, with any misalignment leading to faulty interpretations of attention lag or objects of fixation. Disparities can even occur between film length and total time in fixations, leaving a gap of time unaccounted for where viewers may be distracted, looking off-screen, searching without fixating, or perhaps searching for peripheral cues.

As Jason Farman (2012) states, “the behind-the-scenes, the off-stage, and the hidden-from-view often serve as the foundations for the perceptive world.” Framing always engenders and depends upon the out-of-frame, whilst “our sense of being-in-the-world is dependent on much of the world not being noticed.” That is, we can only ever gain focus and visual clarity by blocking out or simply not noticing most of our visual field. Pushing ‘against the grain’ of much eye tracking research, this videographic essay plots moments of inattention and forms of blindness to explore the ways in which moving-image screens are at once both seen and unseen.

 

References:

Farman, Jason(2012). Mobile Interface Theory: Embodied Space and Locative Media. London and New York: Routledge.

North, Dan (2008). Performing Illusions. London: Wallflower.

Potter, Robert and Paul Bolls (2012). Psychophysiological Measurement and Meaning: Cognitive and Emotional Processing of Media. NY: Routledge.

Shimamura, Arthur P. et al. (2014). ‘Perceiving Movement Across Film Edits: A Psychocinematic Analysis’. Psychology of Aesthetics, Creativity, and the Arts 8.1: 77-80.

The video develops out of some fascinating questions on the theme of misdirection and magic films (those which depict theatrical magicians, as opposed to those that deal with fantastical abilities). Primarily, it links the concept of misdirection to broader forms of filmic continuity editing that attempt to render invisible their mechanics in a way that is, at least allegorically, comparable to the techniques used by magicians to conceal the workings of their conjuring tricks. 

The essay makes striking use of eye tracking footage and close analysis of key film sequences. It could serve as a shopfront for more in-depth research, and it certainly provoked my curiosity to hear and think more about the links between misdirection, magic, and editing, or about how eye tracking might be able to catch spectators in the act of being misdirected: that’s what I thought it was about - an assertion of the value of eye tracking research in explaining how we watch, and how our watching is influenced and conditioned by the cinematic apparatus. Eye tracking usually tells us what we should be interested in analyzing - the areas that spectators are naturally attuned to seek out. This video shifts the emphasis to the zones of disinterest, the places in the frame that are left unscanned or ignored.

Dwyer and Robinson raise significant questions about spectacle and illusionism. Visual effects, and the display of imaging technologies prompted by their use, seem to belong in a separate category from films about stage magicians, where there is a closer connection to be drawn between the way a magician attracts or diverts scopic attention and the way editing and the composition of the frame do something similar. At times the essay was summarizing very briefly some quite complex concepts that obviously can’t be fully explored in this format, so it works best when reading closely the clips from Now You See Me as novel attempts to integrate tricks that are aimed directly at the spectator/camera. The comparisons with Méliès’ demonstrative magic films are clear. Méliès arranged the frame of his filmic compositions in much the same way as he designed stage illusions, pre-empting, managing ,and directing the spectator’s gaze to conceal the solution to each trick. This makes me wonder if there have been any eye tracking studies of Méliès’ films, or indeed any other early films. Such studies might reveal how viewers respond differently to films where the visual cues might be less “conditioned” by contemporaneous viewing habits.

The example from Now You See Me, in which Jesse Eisenberg practices his prestidigitation, is an interesting one, since it compares most explicitly the action of a conjuror to the editing of a film. But we might also point to another layer of trickery in this scene: Eisenberg himself is not really performing these tricks. Like his co-stars, he is not a professional magician, and his sleights-of-hand have been achieved through a combination of judicious editing, hand doubles, and CGI. I tried to find out exactly how much CGI was used, but the producers were not very forthcoming (and in some cases deliberately misleading) on the subject of how authentic the film’s magic tricks were (another layer of extratextual misdirection to account for). The point might be that, while misdirection can point the spectator’s attention away from the location of a trick’s secret, fooling an audience is also an act of narration, telling a whole story that nudges the spectator towards interpreting what they’re seeing as one thing (say, an assistant has entered a box and then been sawn in half) and not another (two people are squeezed into two adjacent boxes and then the boxes are separated). Eye tracking studies might confirm that spectators’ eyes are drawn by particular compositional strategies, or by evolved responses to human faces. Misdirection is an active attempt to divert spectator attention and to give one an alternative narrative to follow. So, while eye tracking can tell us how viewers are conditioned to respond to portions of the frame across several shots or scenes, can it tell us how the viewer constructs meaningful responses to the scenario in which a scene or trick plays out? That is to say, we are misdirected in a conjuring trick not just by where a magician tells us to look, but also by the expectations we bring with us of spatial continuity (e.g., visibly empty hats don’t usually contain rabbits). 

The significance of connections between magic, cinema, and special effects for my own research has always resided in the way stage illusions are designed to carefully structure the spectator’s experience: this is comparable to how mise-en-scene might be deployed to help organize our understanding of filmic space. I have long thought of magic as instructive and instructional: while magicians ostensibly set out to deceive their audience, we come away from magic tricks with a resolve to be better, more attentive, and more critical observers. After all, if we can be duped by something that is so obviously, visually present before us, our faith in so many other apparently obvious phenomenological experiences should be similarly shaken. I like that this video treats magic as a testing ground, as well as an analogy for the interplay between the viewer’s powers of discernment and the filmmakers’ efforts to engage them. Unseen Screens stands as an intriguing proposition about the state-of-play in eye tracking research, and as a productive play with the familiar trope of “movie magic.” It helps us to think about how the fundamental illusions of cinema, such as spatiotemporal continuity, may be manufactured as much through constructive technique as through the flaws in the human visual faculties. 

I’m still waiting for the pay-off from eye tracking research: I find it fascinating, not least for how it tests longstanding theories about how spectatorial attention might be solicited through compositional strategies. The exegesis for this essay notes some of the potential problems with the research, all focused on viewing conditions, but I want to find out how it will eventually inform, complicate, or collide with bigger questions of narrative comprehension, interpretation, and spectator subjectivity. I also look forward to seeing further investigation of how visual effects sequences work to conceal or distract from their mechanisms. Since filmmakers so often manipulate profilmic space, and spectators so often choose to accept those manipulated spaces as cohesive and believable fictions, eye tracking might help us to understand how that relationship between manipulation and acceptance operates. I hope this video will serve as an instructive prompt for discussions of these and other issues. 

This essay expertly applies the video format to its research area, and the exploration of gaze and [mis]direction is one of the most apt and rich uses of the video essay format so far. Exploring a relatively unfamiliar field, Tessa Dwyer and Jenny Robinson use video to explain the methodology of eye tracking to an unfamiliar audience exceptionally well, providing a clear overview of terminology, functions, and modes of analysis. In doing so, it convincingly illustrates the impact eye tracking analysis can make to film interpretation and cognitive analysis, and how such methods may contribute to our understanding of the relationship between film spectatorship and textual construction.

In its key subject of thinking about ‘the unseen’ screen, the alignment of film text and visual processing is revealed, and the juxtapositions in turn enable the creation of vision itself.  The magic film genre articulates the complexity of this dynamic and the extended analysis of watching Now You See Me is artfully employed, showing the formal tenets of framing, timing, and editing as forced direction (or misdirection) that pauses cognition in accepted ‘momentary blindness’. In doing so, paradoxes of the cinematic form are considered. That we take viewing practices learnt from ‘the live’ into ‘the screen’. That the possibility of magic films - and therefore film magic - should be impossible with constructions denying the pleasures of the live staged event. But this disavowal rarely occurs and instead is enabled by textual self-reflexivity or where the camera stands in for the immediate viewer experience. But as Dwyer and Robinson show, the camera is always observer and [en]actor; it disguises the ‘magic’ process otherwise supposedly being revealed. In using eye tracking to examine what ‘is not seen’ the authors offer a tantalising  parallel between formal creativity and the actuality of gaze and response; of textual form and process of interpretation. From this, they explore the conscious and sub conscious fractures that viewers make with directed and misdirected film form across the unseen frame.

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Helvetica; color: #000000; -webkit-text-stroke: #000000} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Helvetica; color: #000000; -webkit-text-stroke: #000000; min-height: 13.0px} span.s1 {font-kerning: none} p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Helvetica; color: #000000; -webkit-text-stroke: #000000} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Helvetica; color: #000000; -webkit-text-stroke: #000000; min-height: 13.0px} span.s1 {font-kerning: none}