Brief audiovisually intensified media help shape our experiences. TikTok, Instagram posts, music videos, political ads, prosumer mashups, and film trailers each have different aesthetics and formal properties, but they’re also siblings. Unfortunately our ways of understanding their relations of sound and image are guided by film theories. These theories don’t work. In narrative and much experimental film, the soundtrack, along with elements like cinematography, lighting, and editing, play subservient roles so that a viewer’s focus can remain with the narrative. Theorists like Claudia Gorbman and Michel Chion have claimed that film music and sound become a property of the image;[1] these claims don’t apply to brief media.

With brief audiovisualities, the soundtrack often comes to the fore. Our attention can shift rapidly among parameters, from an engaging timbre, hook, or gesture to an edit. So what other approaches do we have instead? I think three modalities can be useful. First, a more traditional approach to close readings, like David Bordwell’s consideration of parameters; second, an “atmospheres approach;” and third, an approach drawn from neuroscience, especially from research on multisensory integration. What do each of these help us grasp, and what do they leave out? Do we get conflicting interpretations? I hope a consideration of these approaches’ strengths and limitations, and the ways they connect to film and soundtrack studies, can spark a debate that will encourage people to think more deeply about sound and image.

I value all three approaches, though I have preferences, which remain in flux. While I’ve worked across several forms of brief media, from TikTok to “Carpool Karaoke,” I’ll focus here on my specialty, music video.

A parametric approach works well for music video, especially since the genre’s moment-to-moment experiences are so changeable. Clips need to accomplish many things quickly: highlight the star, showcase performances, draw attention to the lyrics, and underscore the music. To teach listeners what’s memorable about a song, the image might emphasize the movement from verse to chorus, or highlight an unusual timbre, melodic contour, or striking rhythm. The visual track might point to one or two musical features at a time. For while the music envelops us, visual features more often momentarily focus our attention acoustically, especially to showcase the song.

Through decades devoted to parametric analysis, I’ve come to appreciate the strangeness of music video. Gaps between lyrics, music, and image are common: images and lyrics often suggest targets, but contradict them. Performers often exist within mutating environments. Tethered to and sometimes at a distance from the performer’s body, a viewer navigates rapid editing, changing speeds, and shifting senses of scale. She’s not locked in, but she might stay close, so as not to skip an edit and jump into an unknown space. Clips often suggest that the music is the ultimate cause of its inner dynamics. Music videos have their own odd ways of creating form, including building archipelago-like connections among audiovisual moments, gridding the space, intimating that performers and objects resonate with or listen to the music, and showcasing what I’ve called “the audiovisual seam.”[2] Music videos are open, brief, heterogeneous, elliptical, poetic, and strange. They’re capable of eliciting a sense of being “alongside,” or creating dancelike engagements that differ from narrative cinema’s modes of absorption, immersion, and forward drive.

I’m still learning about atmospheres and I’m hesitant to publish something that I’ll need to retract.[3] Steffen Hven draws on Gernot Böhme to define atmosphere as “the mediating force between the objective factors of the environment and the aesthetic feelings of (human) beings, such that the atmosphere is what defines the way we feel about ourselves (with) in an environment.”[4] Theorists gloss atmospheres differently. Some argue that atmospheres are about forms and space in motion over time, and that these create a mood, independent of the viewer, that the receptive viewer is touched by. Objects, sounds, and people are all subject to the same type of analysis because they are all “motion-forms.”[5] Focusing on these, analyses can foreground an interest in line, shape, and mood. Some analyses with atmospheric tendencies—by Tomas Jirsa and Eugenie Brinkema, for example—can project a sense of speed and even feel stripped down.[6] As an audiovisual object unfolds, one can wonder how much a viewer has access to these more immediate affective experiences that reflect subject/object attunement (or Stimmung), and whether participating in this analytic mode changes how we experience things.

Music videos demonstrate how atmospheres can be revealed through a parametric approach. My analyses focus on settings and space, and how they are changed through music with respect to weight, texture, and mood; how the image shapes what we hear in the song and vice versa; the intimate and varied connections between a viewer and a medium; and a video’s forms and spaces as they suggest movement, connect with the past, and more palpably drive forward. But I also ask how space, props, gestures, settings, timbre, harmony, melody, and so on operate within isolated, changeable moments, and I’m engaged with cultural and semantic associations.

Neuroscience is good at capturing music video’s (and the viewer’s) moment-by-moment dynamics. It can help me reconsider an intense audiovisual moment, slowing down the experience and revealing its features. It can provide markers and signposts, and help me relate fine details to large-scale form. William James talked about the world as “one great blooming, buzzing confusion.”[7] The world is filled with visual, sonic, olfactory, tactile, and proprioceptive events. How does the brain decide what goes with what? We know that temporal and spatial concurrences and semantic connections (like a dog and its bark) encourage neurons to integrate events. We may also not bind everything in the frame (sound may not correlate with everything in the image).[8]

Multisensory integration is also shaped through Bayesian predictions (the present event mathematically weighted against previous experiences). Neuroscience has its own ways of explaining how color, size and placement, and emotion work, as well as processes like inverse effectiveness (when neurons boost both a low-res and higher-res signal, with the most perceptible often for sound), superadditive multisensory effects, the attentional blink, the ventriloquist effect, congruence and incongruity, memory, distractors, sound before image and vice versa, and other features. Neuroscience can help explain why I feel a particular moment of pleasure or joy or why I can or cannot remember a passage. It helps me track how my experience moment-by-moment shifts.

My work with parametric analysis is dispersed across three books, Experiencing Music Video, Unruly Media, and The Media Swirl.[9] It’s also present in my collected volumes.[10] Steffen Hven and Tomas Jirsa, among others, have published on atmospheres.[11] My collected volume, Cybermedia and chapters in The Media Swirl explore neuroscience in relation to media.

In my The Media Swirl, I suggest some of us may wish to participate in social media. I’m tempted to propose: Why not borrow from the formats of TV shows like Iron Chef and TikTok battles to produce a video? Or instead contract articles for a journal’s special issue? We might choose some music videos and secure several weeks, to then ask the scholars who work on the three modalities to apply the theories to the analyses. This would create a body of content we all could draw on. Hopefully neuroscientists, Bordwellians, and atmospheres scholars will come together to enrich cinema and media studies.

I hope we teach more courses on close readings of media, with modules inspired by this material. And let us all share conversations about these shared readings and a commitment to media literacy for the commons.