Video Data Analysis



Video Data Analysis (VDA) is a curated multi-disciplinary collection of tools, techniques, and quality criteria intended for analyzing the content of visuals to study driving dynamics of social behavior and events in real-life settings. It often uses visual data in combination with other data types. VDA is employed across the social sciences such as sociology, psychology, criminology, business research, and education research.

General approach
VDA makes use of technological and social developments in relation to video recordings. Mobile phone cameras, CCTV surveillance cameras, body-worn cameras, and other types of cameras generate an ever-expanding pool of recordings from real-life situations. More and more of these videos are uploaded to internet platforms such as Snapchat, TikTok, Instagram, LiveLeak, YouTube, Facebook, and many others. Others can be accessed through collaboration with public and private institutions, such as police departments or CCTV providers. Parallel to this increase in third-hand video data, advances in camera and data storage technology also enabled new ways of collecting first-hand videos for research, by researchers. In short, humans find themselves in a new era of how social life is captured. These new sources of video data support researchers in unobtrusively collecting video recordings that depict real-life situations even of extremely rare events that would be otherwise impossible for researchers to observe first hand. VDA relies on these types of videos to analyze real-life social processes and events—tracing them step-by-step to explain how they unfold.

To do so, VDA draws on methodological approaches such as visual studies, ethnography, video-based experimental psychology, and multimodal interaction analysis to provide a multi-disciplinary approach to using video data. Foci of such analyses include sequences of peoples’ interactions, movements, fields of vision, exchanges of glances or gestures, and actors’ facial expressions and body postures. The goals of VDA studies are to further our understanding of the rules, processes, and sequential patterns that govern social life on the micro level, both in everyday encounters and extreme situations. The method can also be used to trace influence of structural factors in social interactions and events, or study how patterns in social interactions and events produce macro-level phenomena. At the core of this perspective lies the question: How do social actions and situational dynamics impact social outcomes? Video data offers the possibility to study situational patterns in unprecedented detail and rigor by allowing researchers to replay situations, watch them in slow motion and fast forward, and share primary data of situations with colleagues and readers. VDA outlines a toolkit of analytic dimensions and procedures, introduces criteria of validity, and discusses challenges and limitations.

Areas of application
VDA is employed in disciplines such as sociology, psychology, criminology, business research, and education research to study a variety of phenomena, including armed store robberies or unattended package theft, the situational dynamics of protests  and uprisings,  or physical violence,  such as street fights and massacres. Others have used the approach to study polarization among politicians, YouTuber staged health practices, teacher competence, school yard fights, or consoling behaviors. VDA has also been applied to study military negotiations, the unfolding of emergency evacuations, as well as police use of force and police training

Analytical dimensions and procedures
Analytical dimensions refer to the content of visual data that are of interest when analyzing situations: facial expressions and body posture, interactions, and context. Facial expressions and body postures are any nonverbal information that a person’s face and body convey. Interactions refer to anything people do or say that is geared toward or affects their environment or people within. Context means information on the physical and social setting of a situation. These dimensions should be understood as lenses that help deriving information from visual recordings and that might help to understand situational dynamics, provided they draw on a thorough theoretical reflection and employ clear, detailed coding schemes.

VDA can be used in indicative and deductive approaches, qualitative in-depth and quantitative large-N, or even computational analyses.

Although these approaches differ in many ways, VDA approaches are united by a number of analytical procedures. First, coding of video data plays a central role in analysis. Coding means to tag a section of data with labels that synthesize content as relevant to a given research project. Some researchers conduct coding in their analysis without using the term itself, and studies differ in whether they develop a coding scheme first, then code data (a deductive approach), or whether they use an iterative approach of data collection, coding, and analysis (an inductive or abductive approach). Still, all types of qualitative and quantitative video analysis include some type of data coding in order to make sense of it and identify patterns.

Second, six analytic lenses can move researchers from labeling the data to identifying and interpreting patterns or driving dynamics: counts and quantifications, timing and sequence, rhythm and turn-taking, actors, networks and relations, and spacing. These procedures can help in analyzing video data, regardless of whether the aim is to describe patterns at the micro level, or to study causal links within situations or events. The six procedures all build on coding of the data and they are all interconnected. For instance, one could produce counts and quantifications based on video data that help studying social relations and networks. In other words, the six procedures should not be understood as discrete analytical steps or mutually exclusive ways to analyze video data. Rather, they are a non-exhaustive toolbox from which researchers can pick any combination of tools that work well for what they try to accomplish in their VDA.

VDA’s detailed case reconstructions and analyses of patterns may be combined with any method of data analysis that helps identifying patterns in a formalized and systematic way. Depending on the research question, regression analysis, configurational comparative methods, grounded theory, and sequence analysis can be especially fruitful additions to VDA-type analyses.

Quality criteria
Criteria for validity include neutral or balanced data sources, optimal capture, and natural behavior. Neutral or balanced data sources means that researchers should reflect on possible vested interests of data providers that could lead to biased data; if sources that demonstrate a propensity for specific interests are used, researchers should seek to triangulate various sources representing divergent interests. Optimal capture means visual data should cover the duration of a situation or event, its space, and all actors involved. Natural behavior refers to an actor’s unaltered behavior in a given situation, that is, the researcher should consider the degree to which actors recorded in visual data behave the same way that they would have otherwise behaved, were a camera not present.

Limitations
VDA is not suited for all types of research questions and theoretical approaches and, like all methodological approaches, it entails limitations and challenges. First, the type of data used by VDA implies limited access to video recordings from private events, such as funerals in Western societies. Second, VDA does not offer the tacit knowledge and immersion in a social context that comes with continuous direct participant observation, and it does not offer the same potential as ethnography for studying the cultural knowledge or narratives of a specific community or group of people. Third, interpretation of certain elements, such as gestures, may be context dependent, making VDA less suitable to study social contexts that a researcher is unfamiliar with. Fourth, a number of research ethics questions remain unclear with the new types of video data VDA often employs; e.g., what types of video from which platforms are admissible to use as research data.

Related approaches

 * Derry, Sharon J. et al. 2010. “Conducting Video Research in the Learning Sciences: Guidance on Selection, Analysis, Technology, and Ethics.” Journal of the Learning Sciences 19(1):3–53.
 * LeBaron, Curtis, Paula Jarzabkowski, Michael G. Pratt, and Greg Fetzer. 2018. “An Introduction to Video Methods in Organizational Research.” Organizational Research Methods 21(2):239–60.
 * Lindegaard, Marie Rosenkranz & Bernasco, Wim (2018). “Lessons Learned from Crime Caught on Camera.” Journal of Research in Crime and Delinquency. 55(1): 155-186.
 * Margolis, Eric and Luc Pauwels, eds. 2011. “The SAGE Handbook of Visual Research Methods.” Los Angeles: Sage Publications Ltd.
 * Norris, Sigrid. 2004. “Analyzing Multimodal Interaction: A Methodological Framework.” New York/London: Routledge.
 * Pauwels, Luc. 2015. “Reframing Visual Social Science: Towards a More Visual Sociology and Anthropology.” Cambridge University Press.
 * Sampson Robert, Raudenbush Stephen. 1999. “Systematic Social Observation of Public Spaces: A New Look at Disorder in Urban Neighborhoods.” American Journal of Sociology 105:603–51.