Saturday, June 30, 2007

Six Authors in Search of a Character: Part 13, Rotoscoping and Motion Capture

Rotoscoping was invented by Max Fleischer in 1915 (Cabarga 19). The process involved taking live action film and projecting it onto glass from underneath. The images would be traced on paper placed over the glass. Fleischer’s invention was created specifically to turn live action into animation and it was first used to trace images of Max’s brother Dave wearing a clown suit (Cabarga 21). The appeal of the process was more lifelike motion for cartoon characters. The film led to the Fleischers being hired by J.R. Bray, for whom they did a series of films using the clown, later named Ko-Ko, as their main character (Cabarga 23).

A drawing from Max Fleischer’s patent application for the rotoscope taken from The Fleischer Story by Leslie Cabarga.

In the 1930’s, the Fleischers used the rotoscope in several Betty Boop cartoons featuring the performer Cab Calloway. Calloway and his band provided the soundtrack for the cartoons, appearing in brief live action clips during the titles. Within the cartoons (Minnie the Moocher, Snow White, The Old Man of the Mountain), Calloway was photographed dancing and the animators rotoscoped the footage turning Calloway into a walrus, an old man, and other characters (Cabarga 63-69).

Later in the 1930’s, Disney used rotoscoping during the production of Snow White and the Seven Dwarfs. The studio was not experienced drawing and moving realistic human characters, so Marge Belcher (later known as Marge Champion after her marriage to Gower Champion) was photographed as Snow White under the direction of Ham Luske, the animator assigned to supervise the character (Barrier, Hollywood 194-95).
“It was only by using live action as a guide, or so Disney must have thought, that he could give to Snow White the consistency – what Marc Davis called the “unity of acting” – that was possible in a short cartoon when a single animator handled a character all the way through. Ham Luske, by directing the live action for Snow White, thus assumed control over the character greater than any he might have enjoyed if he had been only the lead animator” (Barrier, Hollywood 195).
Another reason for the use of live footage was as a communication tool in production. David Hand, the supervising director of Snow White, said, “The value of live action is the working agreement between the animator and the director” (Barrier, Hollywood 215)

How much the animators used this footage seems to have varied. Grim Natwick, one of the animators assigned to the Snow White character, would often discard some number of rotoscoped drawings in a sequence before starting his own work. He remarked, “you always had to carry [the rotoscoping] further, and you always had to be very careful that you didn’t depend on the rotoscope” (Barrier, Hollywood 196). Some animators made it a point in later years to say that they used it as reference, but did not use it as the basis for their motion. Rotoscoping had the aroma of a cheat and animators wished to distance themselves from it. Art Babbitt was adamant about his work on the Queen in Snow White not being rotoscoped.
“You may have read that a lot of rotoscoping was done, but I have proof that I didn’t rotoscope my Queen. Live action was taken of an actress who acted out the parts. I studied the live action on a Moviola, got it firmly in my brain, then put it away and never touched it again” (Strzyz 83).
The issue came down to who originated the character’s behaviour. Had the animators simply traced the live action, the behaviour would belong to the actor. The animator would be functioning as an assistant animator, altering the appearance of the original to make it resemble the character, but not altering the behaviour. While animators were willing to admit to the use of reference, they understood that rotoscoping usurped their control.

The Fleischers didn’t have any reservations about rotoscoping in their first feature Gulliver’s Travels. The character of Gulliver was blatantly rotoscoped. A comparison of the drawings with a photo of the actor, Sam Parker, shows a strong resemblance between the two (Cabarga 158).

Rotoscoping and filmed reference continued to be used for animated features. The scenes containing human characters in Disney’s Cinderella were shot completely in live action on a bare soundstage before the animation stage started. Cinderella was Disney’s first all-animated feature since Bambi in 1942 and the studio could not afford to exceed the budget or produce a flop. Live action reference was a way to try out visual storytelling and acting possibilities on the cheap before the expensive animation stage (Frank Thomas 330). Frank Thomas recalled,
“When all of the live film was spliced together, this was undeniably a strong base for proving the workability of the scenes before they were animated, but the inventiveness and special touches in the acting that had made our animation so popular were lacking” (330).
Rotoscoping has even been used in stop-motion. For the clay-animated film Closed Mondays, directed by Bob Gardiner and Will Vinton, live action reference was shot of the main character, a drunk who stumbles into a museum (Furniss 175). It’s obvious when the character is in close-up that the facial expressions have been copied from live action.

Motion capture is the computer animation equivalent of rotoscoping. In the beginning, some systems hooked potentiometers to a body suit, so that when the actor moved, the potentiometers would measure the angles of the body’s joints. Newer systems rely on using multiple cameras shooting reference points on a performer’s clothing. Software takes the views from the cameras and is able to calculate the location of the reference point in space. This information is then used to drive the equivalent parts of a computer character (Serkis 36).

Recent examples of motion capture include the character of Gollum in The Lord of the Rings trilogy of films by Peter Jackson, the titular character of Jackson’s version of King Kong, and the character of Davy Jones in Pirates of the Caribbean II: Dead Man’s Chest. In these films, motion capture is used to place computer-generated characters in live action environments and interact with live actors.

Andy Serkis (left) wears a suit covered in markers. Cameras and software take the marker information and use it to drive the computer puppet of Gollum in The Two Towers. From

Other recent films have taken a different approach. Robert Zemeckis’ The Polar Express and Monster House (the first as a director and the second as producer) have used motion capture to imitate the look of computer-animated features. Actors are motion captured and their performances drive caricatures of humans placed inside computer-generated environments. In Happy Feet, dancer Savion Glover was motion captured and his dancing was used for the penguin character Mumble (Sarah Kaufman 1).

Motion capture is a way for live action directors to make films that look like animation without utilizing the process. These directors are used to working in real time with actors.
“[Savion Glover] was fitted with a skintight suit covered with small reflectors. Cameras then captured the motion of the reflectors as Glover danced, and technicians applied the data to the image of Mumble. The result could be seen instantly -- instead of looking at his reflection in a mirror, the way a dancer might practice in a studio, Glover faced a computer screen as he danced, which showed what he looked like as Mumble. [Director George] Miller, also peering into a monitor, could speak to Glover through a headset to keep his performance in line with a penguin's range of motion” (Sarah Kaufman 1).
As both rotoscoping and motion capture record motion that exists in real time, they are not animation, though they may be processed to look like animation. Both these systems utilize artists to take the results of the live action and alter it. In the case of rotoscoping, artists will trace the live action and turn it into drawings, whether they resemble the original live action or are transformed into a cartoon character. In the case of motion capture, the process of capturing the motion data is not perfect. Animators routinely have to clean up the data (Solomon, Penguins 1). In the case of Gollum, animators were also responsible for key-framing the facial animation, though only because the producers didn’t feel confident that they could solve technical problems in time to do facial capture before the film’s delivery date (Serkis 91). However, by the time that Serkis played King Kong for Jackson, the technology had advanced to the point where Serkis face was captured along with the larger body movements, though there are claims that only 25% of the facial animation is by Serkis and the rest is by animators (Rowley 1).

As the motion does not originate with animators, when animators work on rotoscope or motion capture results, they are, in effect, doing the jobs of assistant animators. They are cleaning up someone else’s motion. They may be using the expertise they gained as animators to finesse the live action source, but they are not responsible for doing more than polishing the performance.


Jenny Lerew said...

The roto wasn't a problem for the Fleischers, but it's a bit of a sticky wicket when contemplating Disney animation.

I saw "Peter Pan" recently--over and over again on a 12 hour flight--and I was wondering what part rotoscope played in PP and in "Alice in Wonderland" earlier. Apparently both films were almost completely shot scene for scene using live actors-in costume, no less. I'd heard this and seen a few photostats from those sessions, but when I finally saw on one special or another silent bw footage from "Alice" I was shocked at how close the final animation was to what the actors were doing during their voice track playbacks. Then there's the stat/animation comparison from "Cinderella" where we see I believe Frank Thomas using Eleanor Audley's closeups to great effect for his animation. In the fantastic scenes of Hans Conreid's Hook in PP, was it Frank Thomas coming up with all the acting of Conreid smoking his double-cigar/candelabra-holder, or did Conreid come up with that stuff, or did yet another person stage it and Thomas used the stats for his sequence?
I revere Thomas' work, and I don't want to suggest he "cheated"-I don't think he did-just wondering what the extent of actual use and application of roto was at Disney from 1950 onwards. How much was "finessing" and how much inventing? Likely it's a constant movement between the two.
And I agree--motion capture is an impressive technology, but it is not character animation as I define it--it's something else.

Benjamin De Schrijver said...

Motion capture has its place, but I wouldn't place it in the character animation category either.

However, many of the shots on Gollum were completely handkeyed. In some mocap was only used as reference. Others were indeed completely cleaned-up mocap, but I believe that was the minority.

As for Kong, according to some animators (won't name them because I don't know if they'd like that), King Kong only included 2 complete mocap shots. Others had to be edited, most had to be handkeyed. Halfway into production, Peter Jackson didn't even look at the mocap anymore. Or at least that's what I've been told.

Another interesting story I've heard was about Polar Express: A former Disney 2D animator joined later in the production of that film and was assigned some small shots that had to be completely handkeyed because it wasn't worth going back to the mocap stage for. So on dailies, one of those shots was shown, and supposedly Zemeckis said something like "Now why can't the rest of the movie look like that?!"

I'm eagerly awaiting Beowulf. I've heard it looks good, and I'm excited to see it in motion after the sneak posters that were released. If these mocap projects can find a way in which they're not really trying to copy live action and are not trying to copy how animation looks, finding their own way of doing things and their own style, I think it might have a chance. And it definitly has its place in FX, if it produces the best result.

jason said...

the funny thing about that andy & gollum image you're using is that they're not at all related.. the gollum shot was from a shot animated by Mike Stevens.. keyframed (although referenced off Andy's performance). that whole sequence where he's talking to himself was mocapped, but none of the mocap was used.

Mark Mayerson said...

Jenny, I would never claim that Frank Thomas cheated, but your question is a valid one. I wonder if the animators preferred working on Lady and the Tramp or Dalmatians simply because roto was out of the question and they owned more of the performance.

Jason, I don't doubt you, but can I ask where you got your information?

Your comment just highlights how impossible it is to pin down who has contributed what to a performance. Besides the conflicting evidence or claims made by various people on the production, it's also impossible to know from an audience standpoint where an action or piece of business originated.

Benjamin De Schrijver said...

Jason happened to be where I got my Gollum comment from too. He mentioned it somewhere at Animation Mentor or CGtalk.

jason said...

Heya Mark!

sure, I was one of the animators working on gollum for many years.. I also rigged his animation system & wrote the animation side of the production pipeline for LOTR. :)

I'll be the first to say how amazing Andy's performance was.. but then I'll also be mentioning the phenominal work from the 18 animators who worked on Gollum's performance, the leadership from Randy, Adam, and Richie, and the rendering/roto/comping/dynamics from the rest of the crew.

It was interesting working through the production, watching the media grab onto the idea that gollum was created by this one person.. when in reality it was everyone at Weta who made him who he is.