User:Bcd53

Brandon Dunlap

Intro to Computer Graphics

Computational Videography & Data-Driven Character Animation

Data-driven character animation: Facial Animation in Motion Capture Technology

= Introduction & Motivation = Animation is one of the most powerful applications of Computer Graphics, and Data-Driven animation has become an incredibly popular method of animation since computer based animation was pioneered in the early 1970s. The principle behind data-driven animation is to avoid redundant work by creating animations from pre-existing ones. The process of data-driven animation involves the use of large amounts of data to create realistic, controllable character motion. This data is primarily produced through Motion capture technology. While keyframe animations can also be used in data-driven character animation, the process of making keyframe animations from the ground up is generally time-consuming and expensive; requiring animators to work through potentially thousands of frames of work, only to produce minutes of actual content. Modern motion capture techniques allow creators to efficiently produce high-quality animations, although the data produced typically needs to be edited in a key-frame fashion regardless. Whether an animation library consists of motion captured or hand crafted pieces, for data-driven animation, the animations are executed it is used to finally produce Motion Graphs; which systems of states composed of one, or several animations that are blended together, and transitions to other states.

= History & Application =

While motion capture technology became prominent with the in modern society with the introduction of Gollum, from Lord of the Rings the Two Towers 2002, motion capture techniques have been in practice since the old animated films of the early 1900s. Rotoscoping, invented in 1915 by max Fleischer, is an animation technique in which artists drew over live-acted film frames. The method was pioneered and popularised by Disney's 1938 film Snow White & the Seven Dwarfs, and since then and the turn of the century many productions utilised the method. Another notable example being Ralph Bakshi's Lord of the Rings, released in 1978. Even some video games used Rotoscoping to make their avatars look more realistic, like the 1989 release of Prince of Persia.

= Facial Animation =

Today animators are able to map based on direct point correlation or via a system that relies on an intermediate blendshape and matching system, with tracking leading to similar blendshapes in source and target models. While there are powerful industry level systems in use, there are methods of Facial animation that have become increasingly accessible with markerless capture via depth sensing with inexpensive cameras.

Faceshift
Faceshift is a realtime facial animation software package allowing facial animation input or digital puppeteering via a depth and colour sensitive camera.

Setup
First, there is calibration; or the camera's setup, allowing user to see what the camera observes as a depth map, surface, textured surface, or simply video. Next the user must train the camera to recognise a physical actor, by having said actor enter camera's view and perform a series of FACS style poses; such as rotating their head or lifting their eye brows for a complete 3D capture.

Tracking
Faceshift is capable of calculating and exporting an actor's; head position, eye direction, blendshape activations, virtual marker positions, and a custom avatar to produce animation clips from the recorded video. The software works by using the feed of animation blendshape instructions produced by the rig on the actor's face, and then matching those parameters to the virtual face. Andis capable of producing the 3D motion capture live in real time.

The system works though the use of RGBD, or color and depth. Depth acquisition is done by what is known as "light coding" technology, by which the software codes the scene with near-IR light. The system uses standard CMOS image sensor technology to read the returning, distorted, IR light from the scene to compare the captured images to known 3D depth maps and produce the 3D data. The system's use of light and infrared also allow it to operate in less than perfect lighting conditions, granted there isn't strong direct sunlight to interfere with the IR sensors

After tracking is complete is possible to manually edit and adjust the captured animations

Accessability
The sensor cameras need to use Faceshift only cost around $100, meaning that a studio budget is not necessary for someone to produce facial motion capture data using this system.

Motivation
First showcased at E3 2010, MotionScan is a motion capture technology system developed by Australian company Depth Analysis. The system was made for the development of 2011 video game title L.A. Noire, developed by Depth Analysis's sister company Team Bondi, and published by RockStar Entertainment. The system aimed to create animated characters that could clearly portray the emotions and body language from the actors' performances.

Not only was this a major selling point for the game, but it was an important part of the gameplay. L.A. Noire is a game where the players take the role of a detective, and as such need to read the tone and facial expressions of those they interview to determine if they are telling the truth.

The System
The setup consists of the actor seated in the centre of evenly lit room, leaving no dark spots to interfere to the capture. The actor doesn't require any markers to be worn on their face, only one on their chest and back of their necks to provide a point of reference for the virtual model. The defining feature of the MotionScan system is it's use of 32, 2-megapixel cameras, to record from many angles. This is done to get a complete profile of the actor during their performance. The cameras, recording at 30 frames per second and valued at about $6000 a piece, send their footage to nine powerful computer servers for immediate creation of the 3D avatar; including all the necessary animated textures and shapes. The final result is animated clip of a fully realised virtual character head, fully accessible to animators for use in any scene, or for editing. Another major benefit of MotionScan is the time it saves on filming and subsequent programming; saving the developer valuable money and time.

Alternative Facial Animation methods

 * Interpolation: basis for the earliest methods of facial animation, keyframes of facial expression were designed, and transitioned between using linear interpolation.
 * Free-form deformation: animates by using control vertices to create a bounding volume about the face and drive the animation. Involves dividing the face into multiple sections. Can lead to distortions, this method does not consider the topology of the facial mesh.
 * Physical modeling: the approximation of anatomical structures of the face, the skull, muscles, skin, to simulate how a face actually animates. Is very expensive method.
 * Expression cloning: to map the displacement of the face mesh's vertices. The motion displacements are scaled and rotated with respect the local detail geometry of source and target meshes for pre-processing. While the method works well between similar target meshes, this practice of applying sparse facial motion data to static face models remains unpopular.

= References =