Motion Capture for Home Use

I'm no expert when it comes to motion capture, but I'm aware of some of the technical challenges.  In the past, body suits with potentiometers at the joints sent angle information to rigged characters.  Later, multiple cameras were able to triangulate reference points pasted onto body suits to figure out where the points were in 3D space.  Facial capture usually meant wearing headgear with an attached camera pointed at the performer's face, which had dots drawn on it for reference points.

In each case, specialized hardware was necessary and somebody had to write software to translate the raw data into usable positions or angles that could drive a character.

All in all, not something the average person could do at home.

Technology has a habit of taking things that were once difficult and expensive and making them simple for anyone to use.  It's now happening to motion capture.

What you're seeing here is a home motion capture system to work with a webcam and allow a person to drive an animation-style character in real time.  I can't tell if the headphones are part of the necessary hardware or just headphones, but in any case, the system couldn't be much simpler for an average user.  Admittedly, it isn't perfect and the lip synch is probably the weakest part, but like all applications, it will improve in future versions.

This is being built by a team of Romanian software developers and they're raising money on Indiegogo.  The most basic package can be had for $5 U.S. and they've already reached their financial goal.

Technology has put a lot of people out of business and reduced the viability of various fields.  Good luck finding a typesetter and there are fewer graphic designers than there once were now that software has enabled anybody to do it.  True, a good designer brings experience and taste to a project that an amateur will not, but the tools are in reach for anyone who wants them.  And with templates available for blogs, websites, documents and presentations, the bread and butter work that used to cover a graphic designer's overhead has pretty much vanished.

I'm wondering if we're not witnessing something very similar happening to the role of the animator and possibly the role of the storyboard artist as well.

Motion capture isn't animation, but it can look like animation.  The general audience cares less about technique than about being entertained.  Knowing how to act for motion capture can be learned, the same way that comedians in silent films or mimes developed styles of movement that met their needs.  While undoubtedly there will be a lot of junk produced, the democratization of the tools will result in motion captured films that attempt to resemble animation from the major studios.

There's an indie film world where live action features are sometimes made for as little as $100,000.  The evolution of motion capture tools like FaceRig may make "animated" features possible at the same budget level.  Animators would not be necessary.

Possibly neither would storyboard artists.  The board exists to nail down the presentation of the visuals, but many live action directors don't use them.  If you can direct your characters in real time, boards are not as necessary.  In addition, once the performance exists in the virtual 3D world, you're free to direct the film after the performances are captured by placing the camera and deciding when to cut.  It will be easier than ever for people who know how to entertain an audience and communicate a story visually to create a film inexpensively.

Will this affect the animation industry as we know it or is it just a toy?  I don't know.  But I am amazed at how far motion capture has come technologically, when $5 can buy you a facial capture system and a bunch of avatars.  After seeing what happened to record companies and newspapers when technology upended them, the one thing I know is that we should not be complacent.

Steven Spielberg Needs His Eyes Examined

"According to [Spielberg], the biggest challenge was 'getting it to look like the drawings in the Hergé books. We love the art so much that we used animation to get it as close to the art as we can.'"

Full story here.

Tintin Trailer

The trailer for the mocapped The Adventures of Tintin is now online. Note how little the trailer focuses on the characters and especially the faces. Are they hiding something?

ImageMovers Digital to Close

The Wrap is reporting that Disney is closing down ImageMovers Digital, the studio responsible for the production of the motion captured films directed by Bob Zemeckis, including The Polar Express, Beowulf and A Christmas Carol. Disney purchased the studio in 2007. The article implies that Disney is still interested in going ahead with the Yellow Submarine remake, though it's unclear if the closing means that they will subcontract future motion capture or use other techniques instead.

UPDATE: There are some interesting comments from the industry perspective at the Animation Guild Blog.

Be Careful What You Wish For

I'm a little surprised at Cartoon Brew's insistence than Avatar be considered an animated film. I don't know if the reason is because it is the highest grossing film in history and they want animation to have some of the glory or if it's because James Cameron is so insistent that Avatar should not be tainted with the 'A' word. In any case, there are reasons (beyond whatever anyone thinks of Avatar) why I don't think considering it an animated film is a good thing.

Those outside the film business may not be aware of the distinction between production and post-production. In a live action film - one with no animation or special effects - production is the shooting of the film. Post-production is what happens after the film is shot. Those things typically include editing, music, sound effects, dialogue looping, the sound mix and titling.

When a film does include special effects, unless they are done in camera during the regular shoot, they are considered post-production. In the past, certain effects like in-camera matting, hanging miniatures and glass shots were done during production, but most effects were done during post.

In what we would all acknowledge as typical animated films (Snow White, Toy Story), animation is production, not post-production. In films that have animated elements added (Jurassic Park), animation is done in post-production. This may seem like an esoteric distinction, but it's the difference between what is central to a production and what sweetens a production. I am not in any way dismissing the importance of post-production. A film's music score has a huge impact on how the film affects audiences and certainly Jurassic Park's impact depended tremendously on the quality of the dinosaur animation, but in each case, the post-production elements are driven by what has already been shot.

Animators may have worked over Avatar's motion capture and added creatures, but their work was driven by what had been shot (or in this case, recorded). To pretend otherwise is disingenuous. There is no question that Jack Pierce's Frankenstein make-up added to the audience's perception of Boris Karloff as the monster. However, many actors at Universal played the monster (Bela Lugosi, Glen Strange), yet Karloff is generally considered the definitive performance. While Avatar's animators supplied more than digital make-up, it's still the underlying motion captured performance that counts.

I've written extensively on how fragmented the process of making an animated film is and how so many of the acting decisions are made before the animator starts work. The character designs, the storyboard and the voice performance all make acting decisions that constrain the animator's interpretation. There is no question that motion capture is yet another constraint, probably larger than all the others. To insist that Avatar is an animated film is to marginalize animators even more than they are in what are generally considered animated films. Is this the direction we want things to go? Better to agree with James Cameron and focus our attention on films where animators create, not enhance, performances.

"This Is Not An Animated Film"

As a follow up to my earlier post, here's James Cameron's take on what he's doing.

Schism

Updated at the bottom.

(There are no spoilers below.)

I saw Avatar and quite enjoyed it. Many people have pointed out similarities to Pocahontas, Ferngully and Dances with Wolves. There are also elements of Tarzan and Anne McCaffrey's Dragonriders of Pern. I'm sure that some enterprising person will set up a website showing all those references and many more. James Cameron's strong suit isn't originality; what he's best at is taking existing elements, weaving them into a solidly structured screenplay and then kicking it up a notch with his directing ability. Avatar doesn't break new ground from the standpoint of content, but it does deliver that content in a very satisfying manner.

It's basically a fish out of water story combined with a romantic vision of simpler societies. Edgar Rice Burroughs (author of the Tarzan, John Carter and Carson of Venus stories) specialized in stories like this. In Burroughs' case, there was an underlying racism; white men were always destined to rule the natives. There's no racism in Cameron's point of view in Avatar (though there is in many of the characters'), but there's still the arrogant conceit that allows the hero to admire a foreign society and then rise to the top of it. Heaven forbid that the hero could admire another society but fail to dominate it, which is much truer to the immigrant experience.

Cameron adds a strong criticism of capitalist exploitation to the mix, which has apparently raised the ire of the Fox News folks. However, given the current economy, people who have seen their jobs downsized or outsourced, who can't afford health care and who have walked away from their mortgages don't need a movie to tell them that capitalism can be brutal. In this regard, Avatar has a lot in common with Up in the Air, a low-tech film that is built around laying people off. Fox News can complain all it wants, but Hollywood follows the zeitgeist, not the other way around. And the final irony is that while one division of Fox is condemning the film, Fox itself produced it, which proves that Rupert Murdoch is only concerned with profit, not ideology.

Avatar crystallized something for me that I should have realized years ago. There has been a lot of discussion of mocap and its relationship to keyframed animation. I now realize that this is a symptom of a larger division within the film industry. When James Cameron or Peter Jackson use mocap, it's for almost-but-not-quite-human creatures that have to share the screen with human actors. The goal is for these characters to be believable within the confines of a film that has a realistic surface.

The schism isn't between mocap and keyframing; it's between realism and caricature. James Cameron's goals are very different than those of Pixar, Disney, Dreamworks or Blue Sky. Cameron's use of mocap is an attempt to extend reality. The film wants to fool us into believing what we're seeing is real. The all-cgi features have become more detailed and lush in their visuals, but looking at the characters, it's clear that they aren't real. The character designs prevent the audience from being fooled.

This presents an interesting problem. Caricature has never been taken as seriously as realism. The history of Western art, with the exception of the dark ages and the 20th century, has always been derived from realism, and the art of the dark ages probably had more to do with the loss of knowledge and craft than with a conscious artistic choice. Caricature might be seen as clever, but except for artists, nobody values caricature as more than a lightweight diversion. Disney moved more towards illustration when he went into features. The all-cgi features have pushed their visuals towards greater complexity (which sometimes clashes with their character designs). Video games have also gravitated towards realism. I believe that this has been motivated by a desire to be taken more seriously by getting closer to what Western eyes value in art.

Caricature can be serious. The early Disney features prove it and Pixar hasn't done too badly itself making that point. But there's the gravitational pull towards realism, one proven by Avatar's box office to be satisfying to audiences. The move to stereoscopic 3D is another aspect of that pull towards realism. The challenge for animation is to find the sweet spot between the realism that computers are capable of and caricature, which strips away detail to get to the essence of something. It's not a problem for comedy; if caricature is thought of as lightweight, then it's perfectly suited to getting laughs. But just as all comedians yearn to play Hamlet, all animators yearn to be taken seriously, if not in terms of subject matter then in terms of respect.

As the success of cgi features, with their greater dimensionality and visual complexity, suggested to some that drawn animation was old hat, I wonder if Avatar will suggest that caricature is fit only for children's films and comedy. Should the schism between realism and caricature be narrowed or made wider? I think the executive decisions in the wake of Avatar will have a big impact on the future of keyframing and while I admire the film, I'm afraid that its influence won't be wholly good.

Update: David Brooks has an interesting article in the N.Y. Times, referring to the story formula used in Avatar as the "White Messiah fable."

It rests on the stereotype that white people are rationalist and technocratic while colonial victims are spiritual and athletic. It rests on the assumption that nonwhites need the White Messiah to lead their crusades. It rests on the assumption that illiteracy is the path to grace. It also creates a sort of two-edged cultural imperialism. Natives can either have their history shaped by cruel imperialists or benevolent ones, but either way, they are going to be supporting actors in our journey to self-admiration.

His entire article can be found here.

More on Motion Capture and Beowulf

The differences between motion capture and animation are wider than people think. It's not simply a question of technique.

With motion capture, motion exists in the real world. It gets sampled and then applied to a computer character. With animation, the motion does not exist in the real world. It is constructed and only exists when the images are rapidly displayed, creating the illusion of motion. That describes the process.

There are several differences philosophically, however. Motion capture seeks to convince an audience through the accumulation of detail. When it falls short -- when it is criticized for looking like a waxworks -- it is due to insufficient detail in the motion. Therefore, the goal of motion capture is to increase detail to the point where it is indistinguishable from live action. As Ken Ralston says,

Trust me, when you’re sitting in dailies talking about Anthony Hopkins’s armpit hair for an hour, you know there’s a lot of effort that goes into every pore of the skin, into every eye and eyebrow. It’s a massive puzzle that has to be broken apart and then put back together again.

By contrast, good animation seeks to eliminate unnecessary detail in order to arrive at the expressive essence of a motion. Motion capture concerns itself with addition; animation with subtraction.

If animators look down on motion capture technicians, it is because of the relationship that these two groups have with essence. If there is an essence in a motion captured performance, it comes from the actor. The job of the motion capture technicians is to accurately reproduce it. Any animated additions they make are a result of the technology's shortcomings or the director's change of heart. By contrast, an animator, if successful, creates the essence of the motion. This is no small distinction. There is a world of difference between reproduction and creation.

* * *

(Spoilers below.)

By the time Beowulf kills Grendel, we know that lust leads him to lie. His account of the swimming race is verbally different from the evidence offered on screen. He claims to have lost due to sea monsters, but the visuals indicate that he lost due to a sexual dalliance with a mermaid. It is clear that Beowulf also lusts after the queen. Based on what the audience has already seen, it isn't necessary for Grendel's mother to promise Beowulf a crown. He would have sex with her for no other reason than her beauty.

When Beowulf returns from the cave, claiming to have killed Grendel's mother, he learns the truth from the king before the king commits suicide. Now Beowulf should understand that his lust has led him and his kingdom towards further disaster. Does Beowulf agonize over this? No. He doesn't return to Grendel's mother until after the kingdom is attacked. Does he rein in his lust as a result? No, he takes a mistress. There is no self-awareness in Beowulf's actions, only in his dreams of more monsters.

When Anthony Hopkins' king confronts Grendel, neither can kill the other. While they are enemies, their blood relationship complicates their situation and renders action impossible. When Beowulf confronts the dragon who is his son, he has no emotional conflict as to what he must do. The only emotion between Beowulf and the dragon is hatred.

Beowulf's lack of self-knowledge (not realizing how lust overwhelms his best interests and destroys his integrity) and lack of complexity (no qualms over the need to kill his only son) are what make him, and the resulting film, so empty-headed. The question is not whether Beowulf is successful as an example of motion capture; the question is whether he is a fully realized character. The answer to this question is not the level of detail reproduced; it is the nature of the character's essence.

Some may be tempted to blame the script by Neil Gaiman and Roger Avary, but I blame Robert Zemeckis. Any director who understands a scene can add subtext to it by directing the performances, choosing who the camera should be looking at and how a character should be reacting at any given moment. Beowulf is a character with little self-knowledge, even as he dies. The other characters in the film are cut from the same cloth. This is why the technique, even if flawless, is worthless in this case. Ultimately, the detail must lead us to some essence, but Beowulf hasn't any.

B.O. Wulf

Beowulf is a weak film whose motion capture technique and stereoscopic 3D will distract reviewers and audiences from realizing how empty it is. Director Robert Zemeckis is far more involved with his camera than his characters. Some may mistake Zemeckis use of motion capture as a quest for realism, but it's not. Motion capture is just a way for Zemeckis to exert more control over the film; the problem is that control is a disadvantage in the hands of a second-rate director.

The story is about powerful men who are seduced by evil but who can't admit to their failings. This can be the stuff of great drama, but the film rarely rises beyond a high school production of Macbeth. Instead of Zemeckis burrowing into his characters and providing them with conflicting needs and desires, they succumb all too easily. While they are eventually forced to confront their sins, they never feel the full weight of them.

Zemeckis uses motion capture for two reasons. One is to avoid going on location and the other to give him increased freedom with the camera. Neither reason justifies the effort involved.

Locations inform an actor's performance. Lawrence of Arabia would not be the same film if it was shot on a soundstage, unless you believe that the desert heat, the great expanse and the sand that gets blown into every crevice and orifice did not influence the performances. For all of Zemeckis's obsession with skin pores, body hair and saliva, he hasn't bothered to show the breath of characters when they are standing in the snow.

Zemeckis is more besotted with stereoscopic 3D and a computer animated camera than a first year film student. He can't resist throwing things at the audience or using mile-long camera moves. The camera is constantly calling attention to itself, never more so than in the sequence where Beowulf decides to fight Grendel while naked. Besides being questionable from a tactical standpoint, this results in some of the most contrived camera compositions imaginable. While the audience should be getting emotionally involved in the battle, it's constantly distracted by the ways that Zemeckis uses the camera and props to hide Beowulf's genitals. In one shot, Zemeckis hides them behind a sword stuck in the floor, one of many obvious pieces of sexual symbolism sure to raise snickers from the twelve year olds who are the film's target audience.

While debates about motion capture and the relative success of it in this film will dominate the discussion, it's a pointless argument. Had the technical work been flawless, convincing the audience that they were watching flesh and blood creatures on the screen, it would not compensate for the fact that Beowulf is a colossally dumb movie. The characters are so simplistic, the drama so uninvolving, the direction so crass that no technique could elevate this film beyond mediocrity.

Compare the flight of arrows in this film to that of Olivier's Henry V. Compare the battle scenes with Eisenstein's Alexander Nevsky. Compare the monsters to those in Peter Jackson's The Lord of the Rings. Compare the complexity of the characters with George Stevens' Shane. In every case, Zemeckis falls short and by a wide margin.

What's going to kill this kind of film making is that reputable actors will avoid it like the plague. Anthony Hopkins and John Malkovich should be horrified with the results. No one will equate their "performances" in Beowulf with films where these actors appear in the flesh. Why would performers concerned with their reputations lend themselves to a process that doesn't present them at their best?

There will be debates as to whether this film is animated or not. Should it be eligible for the Best Animated Film Oscar? Let the debate rage, but I won't bother with it. A film like Beowulf is a waste of time regardless of how you classify it. Technique and novelty are never enough; they're just distractions that eventually lose their appeal. I demand more from movies than skin pores, big camera moves and spears pointed at my nose. A movie should have a heart and a mind, and Beowulf has neither.

Tintin Done Right

Steven Spielberg and Peter Jackson have announced that they're making several films based on Herge's Tintin using motion capture. Variety reports that,

Jackson said WETA will stay true to Remi's original designs in bringing the cast of Tintin to life, but that the characters won't look cartoonish.

"Instead," Jackson said, "we're making them look photorealistic; the fibers of their clothing, the pores of their skin and each individual hair. They look exactly like real people — but real Herge people!"

Show of hands. How many people choose which movies to go to based on whether they can see skin pores?

DreamWorks bought the film rights from Herge Studios in Brussels, Belgium. Company is led by prexy Fanny Rodwell, Remi's wife when he died in 1983.

"We couldn't think of a better way to honor Herge's legacy that this announcement within days of the 100th anniversary of his birth, May 22, 1907," Rodwell said.

Now I understand. There's no better way to celebrate Herge's 100th birthday than by taking the cartooning style he developed, the "ligne clair" school which has influenced generations of cartoonists, and adding all the details that Herge was too stupid to include. What a shame that Herge didn't live long enough to see his work corrected. I look forward to the day when Spielberg and Jackson have their work corrected as well.

Machinima

I've been aware of the existence of Machinima for a while, but as I'm not into video games, I haven't bothered to search any out. If you're not aware, Machinima are films where the visuals have been created using videogame graphics.

Slate.com has story on Machinima that includes several samples. It's a good, quick, primer as to what it's all about. Personally, I found the music videos based on songs from Avenue Q and by Avril Lavigne to be the most satisfying pieces, mainly because the music created a structured narrative for the visuals.

The most interesting thing about Machinima to me is the democratic nature of it. Not everyone can draw or create cgi characters and environments, which in the past prevented many people at trying their hands at animated films. Machinima provides people with ready-made characters and sets and game play educates them as to how to get visual results on the screen.

I don't want to re-open a can of worms, but I have to add that in their way, these films are motion-captured. They're definitely not created by keyframes. While the Machinima creators are not wearing body suits with tracking markers, their real-time manipulation of controlling devices puts these films in the mocap category.

It's only a matter of time, if it hasn't happened already, before there's a TV series or low budget feature created using this approach.

Don Graham on Rotoscope

"The only thing an artist has is the fact that he can do something that can't be done with a machine. He should say to himself, "Am I going to let a machine work me out of a job and a profession?" That is what it is going to do, unless the artist keeps moving. It is up to him. As long as jobs have to be done, and have to be done economically, the rotoscope is the quickest way to do it, the challenge will always be there. So it is up to the artist to deliver. He must do a drawing and say, "See here - my drawing may not be perfect as far as realism is concerned, but it has a spirit the rotoscope can never give you." -Don Graham, July 26, 1937

Hans Perk has posted a series of transcriptions (starting February 5) of Disney action analysis classes from 1937, while Snow White was still in production. The above quote is only one of many interesting quotes about rotoscoping in the transcripts. As Hans points out, these same issues are now relevant to the use of motion capture.