Motion Capture: The Virtual Secret

The once famous Max Headroom, the animated head that took over American TV years ago, has been replaced by a multitude of artificial beings of all kinds, who nevertheless retain the vocation of their predecessor: they are almost always in charge of bad jokes ...

The motion capture technology that allows artificial characters to be included in virtual scenery appeared about fifteen years ago in the graphic studios of video game production. The technological apparatus of these games is, in the end, the same that allows to animate in real time a "doll" that can interact with real characters in a more or less plausible way.

Its basic resource is the sensor system that records and analyzes in real time the movements and displacements of an actor. Initially, completely mechanical systems were used, which took the form of a kind of metal skeleton that provided information about changes in the position of the actor's limbs, flexion of joints and modifications of trunk posture.

This technology, developed by robotics researchers to analyze the natural movements of real beings, proved insufficient to capture the subtleties of organic motion, which ultimately make animation work believable. Although mechanical systems have been perfected over time, their use proved to be excessively complicated and very expensive.

In the late eighties alternatives such as the optical analysis of Gremlin Interactive appeared, which requires a scenario with visible marks and reflective elements in various parts of the actor's body, so that several video cameras record his actions, which are digitized and stored in a graphics station.

A computerized analysis of the images is converted into series of coordinates that accurately describe the displacements in three-dimensional space of the original scenario, and can be easily transferred to animation software packages.

This technology was responsible for the boom of video games with movements captured in the early 90s, and its use has been introduced in the production of graphics for special effects in commercials and films.

However, optical systems have several problems that limit their use. For example, to deliver optimal results, in some cases it is necessary to mask the actor's body with a uniform color, or use special lighting (usually black light).

Subsequently, specialized capture systems in facial movements appeared, which allowed the appearance of software modules for face modeling and the animation of "talking heads" with lip-sync. The development of these packages by firms such as SoftImage allowed a certain popularization of this technology.

New options are currently being explored such as capture systems based on optical sensor arrays, something like a "cube" of light beams and photocells that records movements in limited space, and systems based on ultrasound beams and radio frequency sensors. In the last case, they all fulfill the same task: to deliver coherent series of data that allow to move with some precision the 3D models that represent the body of the actor in a virtual space.

The integration of the artificial character in the virtual scenario is a problem of a different order, which would be materially impossible to solve without the enormous processing capacity of the latest models of graphic stations from manufacturers such as Silicon Graphics or Intergraph.

Normally the 3D models that represent the character are integrated into the scenery, and the processor of the virtual set receives in real time the data to make the corresponding render . In most cases an additional graphics processor delivers a preview to the actor, who constantly monitors him to debug his movements.

An additional problem is that the motion capture system restricts the movements of the actor, who is forced to "simulate" the movements of the character in the space of the virtual scenery. Perhaps for this reason a well-known American critic of children's television programs said, not so long ago, that all artificial characters seem to be experts in moonwalking ... or they simply prefer to remain still.

In cases where it is necessary to preserve the lip-sync or facial expressions of the artificial character, a completely different motion capture system may be necessary than that of body movements, which may eventually be used by the same actor or by a different "puppeteer".

In practice it has come to different characters handling all parts of the "doll" in a coordinated effort: one actor handles the body and perhaps does the voice, another handles the hands, or the parts of the character's body that are completely artificial...

At the moment all manufacturers of virtual scenography systems work intensively on the improvement of their character animation systems in real time, and try to solve the inevitable problems in the production of live programs.

However, with the current state of this technology, it is still necessary to resort to carefully studied and rehearsed scripts to make viable the use of "lie" characters in live productions.

Many of these technical problems will be solved over time, as the relationship between costs and processing capacity of the graphics equipment continues to vary. In fact, most of the problems presented in the graphic part can be solved with a greater contribution of digital brute force by the computers involved, or by an intelligent redistribution of their processing capacity. And in this case the first option – for costs – is to quit the live job.

In fact, in applications that allow post-production, such as the realization of commercials or programs that use virtual off-line scenographies, it is still common for the "live" animation of artificial characters to be completely eliminated and conventional animation and rotoscopy techniques to be applied to generate the case sequence.

And very often conventional 2D animation techniques are still used, where movement and perspective continue to depend on the skill of the artists. That also do not charge much ...

However, motion capture has established itself as an important tool for 3D animation work, and is usually used to generate "mockups" or "drafts" of the animated sequences that are then debugged by the most experienced animators.

Either way, the small details that make the most sophisticated visual effects believable still require human intervention... for now!