Special Effectivities

Creator's Statement

The space of a videogame hums with potential action. When playing a platformer, I feel the potential to mount a particular ledge, or leap over a ravine. When I play a first-person shooter, I relish the opportunity to take out multiple enemies by shooting a well-placed explosive barrel. When I play a puzzle game, I latch on to chain of possibilities posed by a single state change. Game theorists and game designers have adopted the term "affordances" from ecological psychology to name players’ sensed possibilities for action within game environments (Mateas 2004; Pinchbeck 2009; Linderoth 2010; Yang 2013).

However, speaking of only “affordances” misses something. Affordances are not neutral. They hinge upon the bodily capabilities of a given perceiver. In ecological psychology, the sister term “effectivities” refers to these capabilities (Turvey and Shaw 1979). Game avatars are not created equal, and the perceived possibilities for action in a game (affordances) necessarily depend upon the potentials for bodily action (effectivities) of a given avatar. Naming this reciprocal relationship lets us better understand how videogame space is shaped by the bodily abilities of a player’s in-game player-character. It is a first step toward introducing the rich bank of vocabulary and empirical data around the areas of perception, embodiment, and action that the ecological approach has accumulated since its inception. This bank holds enormous potential for the analysis of visual learning, the conveying of action opportunities, and the relationship between user interface and proxied embodiment (to borrow a term from Swink 2009).

Captured video has played a crucial role in crystalizing my thoughts on this issue, inspiring my search for new vocabularies. In film analysis, still frames are useful because they strip away the dimension of motion, arresting the image so that one can pay closer attention to aspects of framing. In videogame analysis, video captured from play sessions performs a similar stripping away. Once I am no longer in the midst of controlling my avatar, I can better observe my own behaviors. It was only once I started watching footage of captured play that I realized, for instance, that I tend to move back and forth when approaching a large gap, judging the environment and gauging my in-game character’s own effectivities through exactly those forms of exploratory action that have been studied within a lab setting by ecological psychologists (Mark et al. 1999).

A secondary aim of the essay is to offer an alternative way of thinking through the relationship between videogame players and their player-characters, one that does not rely on importing the term identification from film studies into videogame studies. In the mid-2000s, there were robust debates over whether games studies should borrow the concept of “identification” from film theory, or whether the term obscured, rather than clarified,  key facets of the player-avatar relationship (Morris 2002; Klevjer 2006; Poster 2007). A strong consensus never emerged—rather, things fell into a sort of armistice, with game studies practitioners begrudgingly using the term, while swearing off most of the particularities film studies had imbued it with (e.g. Shaw 2014).

I find this armistice unsatisfying. Identification—across all of its various definitions, and deployments—has always seemed to presuppose too much psychological interiority to adequately address the basic satisfactions and frustrations of controlling an in-game body. As an alternative framework, I turn in this essay to the concept of pre-reflective self-consciousness, as described by the contemporary phenomenologist Dorotheé Legrand. Legrand characterizes pre-reflective self-consciousness as our basic capability to “experience action and perception as coherent”: to understand the changes in one’s perception as a consequence of bodily movement, without wading into the messy psychological tasks of attribution and individuation(Legrand 2006, 108). I find Legrand’s account of pre-reflective self-consciousness—which Legrand herself insists does “not involve any identification of the subject”—to be a much more apt description of the mechanisms by which we become tethered to in-game characters (Legrand 2006, 92). The player-avatar relationship is built from understanding the loop between bodily action and perceptual response. Less psychological than “identification,” and more sensorimotor, Legrand’s conception of pre-reflective self-consciousness provides a firmer bedrock for thinking through things like the control schemes players use to access their avatar’s bodily abilities (a consistently undertheorized aspect of game design, which I have made a point of emphasizing in the middle portion of this video essay).

“We built our knowledge of movement and of the world,” Maxine Sheets-Johnstone writes, describing our process of acclimating to the environment as infants, “on the basis of having learned our bodies and learned to move ourselves. We accomplished such learning by thinking in movement”(Sheets-Johnstone 2010, 172). Arriving at these ideas has been a process of thinking in movement, which makes a video essay an ideal form for sharing them. This notion of thinking in movement, of learning how to move in our bodies, also deeply influenced by decision to include the voices of multiple narrators. The experience of playing a game is highly personalized, more so than that of watching a film. Although the geometry of a game’s level is the same for all players, no two players will move through it in the same way. Each player will happen upon their own discoveries, make their own mistakes, enact their own micro-narratives of movement (I spawned; I mis-judged my jumping abilities; I died). The process of acclimating to the effectivities of a player-character is predominantly composed of these specific, personalized, and unrepeatable moments, with players asking an implicit question with their controller, and being answered by an onscreen consequence. It seemed wrong to subsume these sorts of moments under the umbrella of a collective “we,” so I opted instead for a chorus of “I”s, each voicing their own moments of inquiry and revelation as they build their knowledge of movement and the world. Eventually, my own vocal performance as an “authoritative” narrator drops out completely, as the visuals become more viscerally proprioceptive, and the remaining chorus echoes Legrand’s insistence that pre-reflective consciousness does not attribute actions to any singular psychological identity.

Bibliography

Klevjer, Rune. 2006. “What Is the Avatar? Fiction and Embodiment in Avatar-Based Singleplayer Computer Games.” Doctoral Thesis, Bergen, Norway: University of Bergen.

Legrand, Dorothée. 2006. “The Bodily Self: The Sensori-Motor Roots of Pre-Reflective Self-Consciousness.” Phenomenology and the Cognitive Sciences 5(1): 89–118.

Linderoth, Jonas. 2010. “Why Gamers Don’t Learn More: An Ecological Approach to Games as Learning Environments.” Paper presented at DiGRA Nordic Conference 2010: “Experiencing Games: Games, Play, and Players,” August 16–17, 2010, Stockholm, Sweden. http://www.digra.org/dl/db/10343.51199.pdf

Mark, Leonard S., Yang Jiang, Sally Steinbach King, and Janina Paasch. 1999. “The Impact of Visual Exploration on Judgments of Whether a Gap Is Crossable.” Journal of Experimental Psychology: Human Perception and Performance 25(1): 287–95.

Mateas, Michael. 2004. “A Preliminary Poetics for Interactive Drama and Games.” In First Person: New Media as Story, Performance, and Game, edited by Noah Wardrip-Fruin and Pat Harrigan. Cambridge, MA: MIT Press.
 
Merleau-Ponty, Maurice. 1945/2002. Phenomenology of Perception. Translated by Colin Smith. New York, NY: Routledge.
 
Morris, Sue. 2002. “First-Person Shooters—A Game Apparatus.” In ScreenPlay: Cinema/Videogames/Interfaces, edited by Geoff King and Tanya Krzywinska. New York, NY: Wallflower Press.
 
Nagel, Thomas. 1974. “What Is It Like to Be a Bat?” The Philosophical Review 83(4): 435–450.
 

Norman, Donald A. The Psychology of Everyday Things. New York, NY: Basic Books, 1988.

 

Pinchbeck, Dan. 2009. “An Affordance Based Model for Gameplay.” Paper presented at DiGRA  Conference 2009: “Breaking New Ground: Innovation in Games, Play, Practice, and Theory,” September 1–7, 2009, West London, UK. http://www.digra.org/dl/db/09287.31155.pdf

Poster, Jamie M. 2007. “Looking and Acting in Computer Games: Cinematic ‘Play’ and New Media Interactivity.” Quarterly Review of Film and Video 24(4): 325–229.

Shaw, Adrienne. 2014. Gaming at the Edge: Sexuality and Gender at the Margins of Gamer Culture. Minneapolis, MN: University of Minnesota Press.
 
Sheets-Johnstone, Maxine. 2010. “Thinking in Movement: Further Analyses and Validations.” In Enaction: Toward a New Paradigm for Cognitive Science, edited by John Stewart, Olivier Gapenne, and Ezequiel A. Di Paolo. Cambridge, MA: MIT Press.
 
Swink, Steve. 2009. Game Feel: A Game Designer's Guide to Virtual Sensation. Boston: Morgan Kaufmann Publishers.
 
Turvey, Michael T. 1992. “Affordances and Prospective Control: An Outline of the Ontology.” Ecological Psychology 4(3): 173–187.
 
Turvey, Michael T., and Robert E. Shaw. 1979. “The Primacy of Perceiving: An Ecological Reformulation of Perception and Memory.” In Perspectives on Memory Research: Essays in Honor of Uppsala Universityʼs 500th Anniversary, edited by Lars-Göran Nilsson. Hillsdale, NJ: Lawrence Erlbaum Associates.
 
Yang, Robert. 2013. “The Unportable: Games as Paratexts and Products.” Radiator Design Blog. January 11, 2013. http://www.blog.radiator.debacle.us/2013/01/the-unportalable-games-as-paratexts-and.html
 

At the risk of beginning this commentary with a phenomenological pun, Ian Bryce Jones’ insightful video essay “Special Effectivities” thoroughly “embodies” its subject by showing us, via the very repetition of trial, error, and (hopefully) eventual success at the heart of videogame play, how we come to understand, utilize and, finally, absorb the mechanics of an interactive cultural form. This video merits close viewing by anyone interested in games studies, for it exhibits phenomenologically what an increasingly large swath of literature from various fields—philosophy, ecological psychology, cognitive science, interface design, among others—theorizes about everyday life as an agentive, embodied, being-in-the-world experience. As a medium both informed by and informing our play lives on an enactive, embodied level, videogames are ripe for phenomenological investigation, and Jones’ video essay offers an excellent introduction to this kind of analysis.

As Jones explores, videogame play necessitates player agency via controller input and onscreen representation—the ability to run, jump, crawl, shoot, etc.—and to do these things through meaningful, motivated, and consequential choices or actions. This agency can be environmentally situated using J.J. Gibson’s formulation of perception as the product of biological beings interacting with the world, and the affordances (what the environment provides) that may couple with (or can be made to submit to) human effectivities (what our bodies—our first interface, perhaps—enable us to do). These concepts were later popularized by Donald Norman, et al., and are now influential in human-computer interface and user design/experience fields. But as Paul Dourish reminds us in Where the Action Is: The Foundations of Embodied Ineraction, Gibson’s affordances and effectivities are met by a third consideration—the very actions we choose to take, and how we go about doing them. Chairs afford sitting, ample posteriors and bending knees effect sitting—but the action itself must still be executed (i.e., sitting as a lazy flop or as a graceful descent, and all styles in between). I would argue that the “how” of player action is at the heart of game aesthetics, and perhaps one of the medium’s defining characteristics.

The video certainly makes the case for the aesthetic analysis of games as playable, interactive media, perhaps most definitively in a split-screen segment contrasting the jumping styles found in Assassin’s Creed versus Tomb Raider (around 7:12). Jones helpfully employs musical terms for tempo (allegro vs. largo, for example) to capture the difference in “game feel” (to use Steve Swink’s term) in how a player executes and experiences a jump—in this case, the forgiveness and fluidity (“a glissando”) of Assassin’s Creed versus Tomb Raider’s more exacting or “staccato” jump mechanic. Whatever else these games bring to bear on the aesthetic experience of their gameplay, their respective jumping mechanics shape at a foundational and pre-reflective, self-conscious level (more on that below) what these games can experientially and aesthetically afford.

This brings me to a point not always articulated in game studies. The aesthetic qualities of games—our agentive experience in playing them—are often discussed and evaluated in an idealized state: the player is skillful at the game, but has not yet exhausted the experience of playing it. This “critical position” presents a reduction—a tidying up—of both a game as a playable object and the player experience of navigating it. As the wealth of excellent gameplay examples in Jones’ video illustrates, acquiring a “peak” relationship with a game is a process: it has a development lifespan all its own—a play “life” consisting of birth and discovery, growth and mastery, boredom and death (i.e., abandoning the game altogether). Thus, any analytical or theoretical claims about games or gameplay are always complicated by and contingent upon play perspectives that are highly elastic, emergent, and evolving—and upon gameplay that at one moment may be full of pleasure, flow, and accomplishment, and the next descend into confusion, tedium, idiosyncrasy, frustration, boredom, even anger (the controller thrown in disgust expresses perhaps a temporary rejection of the agentive, embodied relationship with the screen—a kind of momentary alienation or anti-identification).

Of course, if we eventually “git gud” (in Dark Souls terms) at playing a game, we absorb the game and the controller’s affordances, as well as our avatar’s virtual effectivities. It’s at this level of engagement that actions can enter and take root in a player’s pre-reflective self-consciousness—in other words, we can begin to play without really thinking about how we play. Jones’ use of Legrand’s scholarship on the subject of embodiment and consciousness provides a fascinating investigation into an issue that can be traced back to at least Franz Brentano and questions of intentionality as a feature of consciousness. If consciousness is always conscious of something, what is that something from moment to moment? Shaun Gallagher and Dan Zahavi (whom Legrand cites) in The Phenomenological Mind (2008) use the example of deep reading to help us understand that many learned actions or activities have a pre-reflective, self-conscious component. The act of reading requires scanning arrangements of letters and spaces, but what we’re really conscious of are the ideas encountered. Once we have mastered reading, or walking, or playing a musical instrument, or using a game controller, or any other routinized activity, we do these things pre-reflectively. In fact, to do them consciously would be cognitively challenging; to think about walking while in the act is to risk falling on your face.

There is much more to tease apart here regarding the relationship between actions and consciousness (see N. Katherine Hayles’ recent book, Unthought: The Power of the Cognitive Nonconscious, for more on the current state of theorization about consciousness, non-consciousness, and the self). But I’d rather run with Jones’ useful formulations to explore further the curious relationship between player and avatar: specifically, the avatar as, first and foremost, a tool for gameplay. Martin Heidegger thought a great deal about how tool-use moves into pre-reflective self-consciousness; objects present-at-hand convey their potential affordance, but objects ready-to-hand have been met by human effectivity and are put into actual, skillful, pre-reflective use. To articulate this in videogame terms: a controller map of afforded actions (as often found in a game’s main menu) offers present-at-hand possibilities, but a tutorial level about jumping and running strives to cultivate ready-to-hand, pre-reflective, self-conscious interaction. Controllers, too, as industrialized hardware, pursue design standardization in an attempt to leverage ready-to-handedness—perhaps hard-won by a player in one game—across all gameplay experiences (e.g., the “X” button is always for jumping). Ultimately, this ready-to-handedness becomes our passport to compelling videogame play experiences, and it lays a foundation for affective play, for emergent story-telling, and for reflective or hermeneutic meaning. (Note that ready-to-handedness is largely true for mainstream gaming; artistic and indie games often disturb ready-to-hand interaction in pursuit of other aesthetic experiences and meanings.)

Turning to Jones’ written comments, he begins to explore what an analysis of gaming as a series of pre-reflective, self-conscious actions means for game studies—and, in particular, for the overuse of identification as an analytical tool imported from other academic fields. Psychological or emotional identification seems irrepressible for us humans, even though narratology generally advances that it’s not an aesthetic requirement; we can enjoy the stories of evil protagonists and not have to “identify” with them. But when identification (or its lack) is trafficking in an ideological or political register (as it often is), then perhaps what we are really talking about is representation and inclusivity. And in this vein, videogames—like most other media—suffer from a chronic, historical, structural, and systemic lack of diversity (both onscreen and behind it).

The issue of videogame inclusivity aside, I heartily agree with Jones that conventional media analyses preoccupied with identification are not the best fit for explaining gameplay agency and aesthetics in media res (though game studies remains indebted to film phenomenology scholarship and its explorations of enactive, embodied viewing). In videogaming, my “involvement” with my avatar (whether depicted onscreen or elided) is very much a byproduct of consequential actions that escape their digital representations to become extensions of myself, re-embodied in a “structural coupling” (to use Maturana and Varela’s term for symbiotic organisms) between myself and my avatar. This observation is consistent with the statements made by Jones’ players in the video. The feelings they recount—the stories they tell—are not about their avatars; they are fundamentally about themselves, reconfigured by their tools. “I am fatigued,” says one player. “I am injured.” “I am afraid.” “I’m a toddler” (about 11:11). Of course, language and experience are not synonymous, but our descriptions of gameplay—and the absorption of gameplay affordance and effectivities into pre-reflective self-conscious action—strongly lead us to the conclusion that avatars become, fundamentally, ready-to-hand tools that we strive to wield without a thought about identification. We just do it.

Despite these observations—and the excellent demonstration “Special Effectivities” stages in support of phenomenological analysis—I’m still left wondering what to make of identification, and player psychology more broadly, in game studies. Thanks to Jones’ video essay, I’ve had some stimulating conversations about this topic with colleagues at the UCLA Game Lab, which led to far-ranging riffs about player motivation, emotional state or affect, the pleasure (and pain) of action/interaction, our psychological entanglement (even identification) with the tools we like to use, and how various states of being may actually predispose and therefore color our actions. In the end, I can’t seem to escape this conclusion: even if pre-reflective, self-conscious action operates independently, pre-emptively, or even transcends identification and other psychological affects, it seems impossible to contemplate or evaluate such actions fully without dragging them into consciousness—and therefore back into the realm of identification and other psychological considerations.

I’m therefore provisionally (and maybe constitutionally) inclined to seek a complementary, comprehensive perspective that reconciles the pre-reflective to the reflective, while also accounting for the ever-fluctuating state of play consciousness we find ourselves in from moment to moment in many games. Perhaps twitch play on one end of the spectrum maximizes pre-reflective, self-conscious action, a trance-like state of “pure” algorithmic execution. But many other games find us cycling between evaluation and execution, a symbiotic dynamic that places ready-to-hand action as the figure against a ground of identification that motivates, contextualizes, and gives meaning to those actions. Extending outward, this ground includes such things as game trailers, advertising, iconography, fan cosplay… all the paratextual representations that may predispose, propagandize, or prejudice us in choosing a game to play in the first place. And afterwards, when we’re done playing a game, we can reflect on the experience of our play—its feeling as well as its meaning—a conscious act that “separates” the player from the avatar/tool and places pre-reflective, self-conscious actions back within the envelope of interpretation and identification. At least for me, it is from this perspective—the game no longer ready-to-hand, but merely present-at-hand—that gameplay lives on in my memory.

Affect, identification, action… these concepts and their various registers of consciousness are all swirling about in the experience of gameplay. If we think of games as affective systems (Aubrey Anable’s Playing with Feelings offers a robust investigation), then perhaps pre-reflective, self-conscious action is an essential, even defining, element of gaming that may work both with and against identification to create a wide range of affective experiences. Regardless of where one lands on such questions, the role of pre-reflective, self-conscious action in gameplay is undeniable—and now all the more tangible—thanks to “Special Effectivities.” Jones’ video essay ultimately and productively brings to mind (for conscious reflection and contemplation) why videogame analysis should proceed from an enactive, embodied position—and why videogame play is such a personal and intimate experience. To play at being someone else onscreen always seems to return us to ourselves.

In Special Effectivities Ian Bryce Jones makes artful use of the video essay mode of critique to explore the dynamic and befuddling experience of gameplay embodiment. Scholars from a variety of disciplines have attempted to make sense of how users adopt video game pseudo-physicality, a type of engagement that is deeply interactive but within obvious algorithmic limits. Formally, Jones works with scads of rich game footage of avatar actions. Most imagery shows struggle: jumping only to fall, almost catching a railing but not quite making it, or unsuccessfully leaping toward the next platform. Splat, after splat, after splat. By editing together shot after shot of numerous avatars getting acquainted with their fluctuating physical abilities, while also studying and testing the game environment itself, Jones showcases a comprehensive view of the awkward side of gameplay physicality. Viewers are rewarded at the end with a graceful and indulgent series of shots containing successful leaps, jumps, and other movements; perhaps the most poetic aspect of Jones’ video essay, the adroit, deft movement at the end, conveys the earned satisfaction of gameplay mastery.

Jones’ purpose in delivering the repetition and succession of physical acts from numerous games is to demonstrate how concepts from ecological psychology may be effective for theorizing game physicality. Namely, Jones introduces the concepts “affordances” and “effectivities.” Noting that game designers utilize ecological psychology, Jones’ application of it seems logical. It makes sense to compare how we might apply our sense of what our physical bodies can do (affordances), and base our gameplay actions on those abilities/limitations within factors evident in the environment around us (effectivities). Jones also introduces the concept of pre-reflexive self-consciousness to explain, as I understand it, the quasi-self-awareness required to determine one’s physical abilities and environmental factors, or rather, affordances and effectivities. Compared to the other two concepts, pre-reflexive self-consciousness receives less verbal explanation. This may be due to the poetic nature of the final portion of the video: more words written or spoken would prevent the viewer from meditating solely on the aforementioned delight of mastering the algorithm: effortless, successful movement. Dropping language here seems worth it, given the effectiveness of the essay’s narrative arc and a satisfying indulgence in observing a game players’ hard-won coherence. The term emerges only after the players appear to have mastered both the affordances and effectivities of the game environments and avatars, which suggests that a more stable state of cognition follows all of that trial and error. Coherence, successful embodiment in the game environment, is the goal. It is possible that “pre-reflexive self-consciousness” is Jones’s way of accounting for the mediating role of thought in any physical act, that our cognitive processes encourage us to “experience action and perception as coherent” (Legrand in Jones).

In the accompanying written text, Jones juxtaposes his use of ecological psychology with other approaches, one being cinematic identification. He dismisses the latter in part because of the less interactive nature of film spectatorship. I cannot disagree with him in this regard, and my previous writings on the subject acknowledge this limitation. Yet there are aspects of cinematic identification that, I believe, account for issues not addressed by ecological psychology, namely, mediation. Theories of identification in cinema studies acknowledge that there are many layers of mediation standing between the subject and a text or “reality.” In my view this extends even further; the constructedness of everything, starting with language itself, precludes access to any kind of unmediated, natural experience. I contend that there are layers mediating our encounters with the “real,” which further complicates the application of fields like ecological psychology to textual interactions which could be considered hyperreal. Games are, after all, simulations: exaggerated, wholly unreal worlds that are in many ways modeled after the physical universe we inhabit. Where cinematic identification cannot account for gameplay interactivity, ecological psychology does not account for mediation. This becomes more evident and further complicated when assumptions around instinct and nature are applied to simulations that are pleasurable for both their departure from and simulation of “reality.”

Jones does not address the potential challenges of applying a scientific concept which seeks to understand perception of real world physicality to a highly constructed and unreal simulation where no material bodies reside. Perhaps such a line of inquiry is outside the scope of this project. Exploring the disciplinary/paradigmatic differences that complicate the application of ecological psychology to gameplay embodiment would be a worthy undertaking for sure. In fact, I believe that this is precisely the work we should be doing: combining, clashing, and commingling methods as we continue to encounter new technologized perceptual experiences.

Undergirding disciplinary clashes aside, the poetic effectiveness of Jones’ succession of shots following a narrative arc from action failures to successes is productive and illuminating. The concepts of affordances and effectivities serve well to provide an analytical framework for tracing players’ path from splats to gameplay mastery and coherence. The extraordinary critical value of Special Effectivities lies in its poetic exploration of the interactive dialogues players have with game designers and algorithms.