12.18.2004

The Polar Express: A Virtual Train Wreck (conclusion)

Finally! I'm back with the conclusion to my commentary on THE POLAR EXPRESS. If you missed the first installment, just scroll down a bit or click HERE. Okay, let's get back to it.



If you checked out the Newsweek article that I mentioned last time, you were subjected to the above image, with multiple Oscar-winning actor Tom Hanks all dolled up in hi-tech mo-cap gear. Here is where Warner Bros' marketing was really banking on the prestige of Hanks getting all dirty, showing that he's willing to go the extra mile to give us, the audience, something worth watching. But, unfortunately, we are given the actual image from the movie from where this performance was captured. A "before and after" scenario, I guess. It's all too telling, if you ask me. Do you see what's happened from Point A to Point B? Somehow they spent millions of dollars to literally take the soul out of an Oscar-winning actor's performance. That's quite a feat!

I sat there just staring at this image, trying to figure out what happened. What exactly is going on here? Why does the image on the top look so engaging, so vibrant, so full of life, but the image on the bottom - which is supposed to be the exact same performance of the actor "captured" by the computer - look so dead and puppet-like? I went around the internet searching for anyone who might've tried to explain this. I found nothing. There were many who talked about it, who went on about The Uncanny Valley and such (which I mentioned in my previous post), but never anyone who really sat down and analyzed the movie and its characters, frame-by-frame. This is when I decided to do some investigating on my own.

With a vigilant eye, I studied Hanks and his avatar, nit-picking all the most minute details. I noticed several things. First, his brow and the area around the eyes did not have that same subtle intensity that you see from the original image. (What's happening here in the story is Hank's character, the Engineer, is singing into an intercom, while the children on the train are being served hot chocolate. Oh, and keep in mind that this is not a scientific study, just me having some fun, trying to figure some stuff out. And I feel I should say that all images from the film are copyright Warner Bros. So there.) There's a slight twinge in Hank's eye that you cannot make out in the Engineer's image. I know that the glasses are in the way, but even still. Also, the area around the mouth is not as emotive either. There's some stretching going on with the real Hanks that is not picked up in the fake Hanks, just outside the edge of the mouth. I understand that they added some weight to the Engineer, and it's reflected here, but I also know that bigger people are just as capable of emotion, so I'm not going to let that by. Even with weight, you should be able to feel the intensity of the Engineer singing into the intercom. And I'm not seeing that here.


Also, when Hanks pulls his head up to belt into the mic, there's some straining going on in his neck area. This, again, is lost in the digital version. The way he thrusts his head up and out, with his shoulders going down, and the way his right arm is positioned all get mussed up, and become stiff and weak. There must be something in the way that the characters are created, or "modeled", that no matter how much the digital artists tried to tweak to get the body just right, the CG model would still win. All the characters I've seen in FINAL FANTASY and THE POLAR EXPRESS have a puppet-like, marionette-style of construction. The shoulders, the way they walk, the way the head is held in turns, etc. - even when you are capturing real live action for the performances - all have a stiffness to them that apparently cannot be tamed.

My experimenting did not stop with Tom and the Engineer. I decided to grab some images from the film of the main Hero Boy (yes, that's his official name), and his friends to see if I could do better. Now, I am not a CG animator, nor have I ever done any CG modeling or shading and lighting, so these experiments are just that - experiments. I know that there are many many things involved when the animators and digital artists created these scenes in the movie. I'm very familiar with the CG realm as I see it in action on a daily basis here at Primal Screen. So, I'm just going into Photoshop and tweaking the characters to see what could've been. That's all.


Here we have our Hero Boy, just getting on the train and looking out at his neighborhood. For this scene he's unsure about this strange vision of a huge steam locomotive barreling through his town, and so he's excited, yet very wary. So what's up with his face? He looks like he's pissed off at the Engineer for forcing him to get onboard.


Here is my version. Nothing too major, just some subtlety added to the eyebrows and mouth. Now he looks like he's engaged with what's going on, and not angry.


Here, the boy has gotten his ticket and is apparently mesmerized by the darn thing. He looks so vapid. The one thing that I noticed in the movie, was that it seemed like they wanted to light Hero Boy's eyes with such intensity that we would be in constant awe of them. But in reality, they looked like doll eyes. It's just not natural to be able to see all of the iris all of the time. There will always be some shadows present, even with the most dramatic, intense eyes.


And thus, I softened the intensity of his irises and pupils by adding a little bit of shadow just underneath the upper eyelid. Also, I widened his mouth, and again, tweaked his eyebrows.


This was a doozy for me. I got this image from the teaser trailer, when the boy is gazing out at the lights and sounds of the train going by in front of his home. Since this trailer came out a full year before the movie was released, there's a slight possibility that they had changed some details in this particular scene, so I dunno. The boy looks downright freaky. Here, you see what I was talking about his irises being lit up to the point that they are oddly illuminating. Almost like they're about to vaporize you.


After many attempts, I finally came up with something that was halfway decent to me. After working on this one, I came to the conclusion that this character was simply designed poorly. I understand that the filmmakers used real kids for the look and image scans for all the characters, but sometimes what is real does not transpire well into animation, or rather, digitally. There has to be some tweaking to get it to look right. And my main problem with the Hero Boy (besides his eyes) were his eyebrows and his mouth. Maybe the original boy from whom they used to model for the CG character had downturned eyebrows, but I guarantee you that he did not look pissed off and mad at everybody all day. If the filmmakers could've just raised the eyebrows a bit, especially in this scene as he's watching in amazement at this fantastical event, then there could've been some more humanity put into the boy. Just these subtle touches make all the difference in the world.


I did not like the look of this boy at all. He's supposed to be the lonely boy, with a (possible) broken home, poor, untidy, without any spark of drive left in him. Well, his creators felt that to convey this type of sad sack you should give him a dull haircut that makes him look like he's wearing a nicely round fur helmet, making his ears protude, Dumbo-like. Also, they give him strange, rat-like eyes, with no lips at all. He looks like a death mask. But alive! Terrible character design.


Again, I had to fix the eyebrows. (What's with the eyebrows?) I made them more pronounced, and raised them just slightly. I gave him tussled hair, since he's supposed to be "unkempt," but I think I went a little too far. But the roughing up does give some personality to him now. There needs to be some assymetry to this character and you can do that with the hair. Hair can do wonders for character design, as it's unique for each character. You can do so much with hair to convey any type of personality, even with a simple lock. I fixed the bridge of his nose, as it was too strong. I gave him a turned-up nose, similar to a childhood friend of mine. Also, I had to do major work around the character's eyes to try to steer away from that mask-like quality. Plus, I gave him some lips. He was a toughie. I'm not sure if he's at tip-top shape, but he's better off than he was, that's for sure.

As for the girl in the background, I did some touch-up, but I decided not to get too far with it on her. She was the better designed of the bunch, but she still had problems with her eyes and her skin. The creators made a costly mistake in rendering her skin for this movie, making her seem "ashy," or having seemingly very dry skin.

It doesn't take a rocket scientist to conclude that THE POLAR EXPRESS has some terrible character designs. If they could have just hired somebody with an inkling of aesthetic value to determine what needed to be changed and fixed, then all this could have been avoided. But here we are.

Some other thoughts about motion-capture:

One thing I admired Peter Jackson and his co-horts on THE LORD OF THE RINGS trilogy, was his uncanny ability to notice the details. And when something wasn't looking right in his eyes, he acknowledged it right away and worked hard in fixing the problem. Sometimes things don't go the way you imagined them and with Gollum, he proved the hardest character to design and depict on screen. They originally were going to use actor Andy Serkis's voice for Gollum, and some performance actor do the mo-cap bits, but when they checked out video of Andy in the soundbooth, performing Gollum's voice by twisting and wriggling his face and contorting his body to get the voice right, they felt that maybe he would be the perfect guy to do the performance for the character. And boy, did he get into the character. (If you ever get the chance, read Gollum: A Behind the Scenes Guide of the Making of Gollum, written by Andy Serkis himself. It's a fascinating read.) The boys at Weta Digital created a program that dealt with facial expressions, lining up digital skin to Andy's facial details when he created a particular expression.

As they grabbed Andy's performance via mo-cap, Weta had actual animators go in and tweak the performances to make sure they fit the appropriate scenes. Some scenes required more animation, using "keys" (main poses) and having the animator actually manipulate the digital character to give it that extra umph. Just the same, some scenes were mostly mo-cap, with hardly any animation involved. The face was the only place where they used animators entirely. So yes, motion-capture can be a good thing, like picking up on subtle things that the body does for a particular motion that maybe the animator would not realize. Mo-cap helped further the production, simplifying some motions that animators could've had a hard time working on. Here on LOTR, animation and mo-cap worked hand-in-hand: a happy symbiotic relationship. Peter Jackson was recorded to have said that even with all the technology used, they found out that what Disney did in the 30's was still the most reliable way of getting personality into the character.

And as I recall, there was possible talk of Oscar for Andy Serkis's performance, remember? I'm not hearing the same for Oscar-winning Tom Hanks, I'm afraid.


More thoughts. Looking through that wonderful book, The Art of The Incredibles, I came across some interesting quotes:

"In my opinion it's always been a fallacy, the notion that human characters have to look photo-realistic in CG. You can do so much more with stylized human characters. Audiences innately know how humans move and gravity works, so if a human character doesn't feel right, they'll feel something's wrong. But if the weight works for stylized characters, the audience doesn't question it - like the Dwarfs in SNOW WHITE AND THE SEVEN DWARFS, which were so cartoony and stylized. In THE INCREDIBLES, the characters are cartoony heroes but they can be hurt and they have this family dynamic that makes them believable." Ralph Eggleston, Artistic Director for THE INCREDIBLES

There is so much leeway given to stylized human characters, so when you get into that uncanny valley of photo-real digital humans, the audience will notice even the slightest of quirks. Here's another quote:

"From the beginning, we all wanted the cast of characters to look like cartoon people instead of photo-realistic people. In animation, it really takes a bit of exaggeration to make something look convincing. The great caricaturist Al Hirshfeld most typified this. He could perfectly capture a person's identity by simply sketching curlicues for hair and pinholes for eyes. The faces and attitudes he drew were often more recognizable in the abstract than if they had been rendered out realistically." Teddy Newton, Character Designer on THE INCREDIBLES

That's what I was talking about in giving a certain motion that extra umph. Even though every little detail is recorded by motion-capture technology, the performance up on screen still looks stiff and robotic. As an animator, I understand that real life looks dull if rendered exact, and I've had to pump up some scenes to give them that extra bit of life to an otherwise banal movement.

So now, we see that THE POLAR EXPRESS is doing quite well at the box office of recent. I know why. It's got that feel-good Christmas-is-in-your-heart pap that the general public seems to enjoy. And I guess that most audiences will wow and awe at the "magic" of the setting of this film. They'll be amazed by the look and feel of it, because to the unknowing eye, it's something that's new and exciting. That's fine. But we know better, right?

What I find incredibly ironic with the film is that in the story, our Hero Boy has a hard time believing in Santa and in Christmas as a whole. It takes a roller-coaster ride of a trip to the North Pole to finally understand what it means to really believe in Christmas. It's even stamped on his ticket: BELIEVE. Only when he truly believes can he finally hear Santa's sleigh bell that is given to him as the first gift of Christmas. As I watched the screen, I could not help but laugh as I somehow knew that we, the audience, were really the Boy, wanting to truly believe in these characters. But, alas, in the end, we still cannot hear that darn sleigh bell ring.

(This commentary was written in December '04. I am currently working on a follow-up to all this Polar Express hubbub and will post it sometime in February '05. If you are a new visitor, don't forget to check out The Ward-O-Matic's main page for the latest. Thanks. --Ward.)

71 comments:

  1. I agree with you about the eyes of the characters in the Polar Express, somehow they didn't get that quite right. I went in expecting to give them some license and so I didn't feel the need to pick everything apart to the nth degree. I agree with your earlier post about perhaps they could have placed real actors in their incredible CG environments.( I was really impressed by how the snow looked illuminated by the lights coming from the train car interior) Or, perhaps they could have taken the LOTR approach, and not relying so heavily on mo-cap alone. Anyway, great post!

    ReplyDelete
  2. I still haven't gone to see The Polar Express. I just can't bring myself to do it, even though friends have told me that it was better than they expected. Next time someone asks me if I've seen it, I'll simply answer "No" and then point them in the direction of this blog. Great post, Ward!

    ReplyDelete
  3. Frederick Remington, the painter, once said that he wanted his epitaph to be, "He Knew The Horse." Well, I want mine to be, "He Knew the face." Faces, hands and feet are the hardest things to render believably which is why I spend SOOOOO much time drawing faces hands and feet! I think that, based on what I know of animation--which is really just what I've read and observed, it's much the same thing. If any of those elements look off, the whole image suffers. I have done some work in 3D however and have long abandoned the idea of creating a 3D character that will have the same flair and performance potential as something I've drawn. By the time you build a character, rig it, light it, and actually ANIMATE it, you stand the risk of over-cooking the stew--of watering down all that's spontaneous and vital...

    I don't know, I think that animating must be 50% drawing and 50% acting. In know I'm rambling so I'll stop now. Cheers Ward!!!

    ReplyDelete
  4. That's one of the hardest things to do in animation is to keep that spontaneity in your performance. I think that's one of the alluring things about mo-cap: that you're getting a top-notch performance from the source itself. But unfortunately something about the digital zeroes and ones kills that notion entirely. Similar to when CDs first came out in the early to mid-80's, a lot of the audiophiles complained that the sound, even though, to the general public, it was touted as crystal clear and sharp, felt a little "off" and sounded tinny and shallow at times. The bass was not as deep as it should've been, the high tones were too high, etc. The quality of sound on CDs are much much better now, but it took about 10 to 15 years for the technology to catch up. (Notice how many albums are being remasted nowadays?)

    Maybe the technology is not there yet for this sort of "animation." That's why it's better for animators to go in and tweak the motion-capture performance where needed. Giving that real human touch to the action, you cannot beat. Animation is such a unique and individualistic art-form that you cannot cheat it on screen!

    Hey, thanks for the comments, guys! This is good discussion.

    ReplyDelete
  5. Wow!! What a fantastic piece Ward! Excellent excellent insight!

    ReplyDelete
  6. Brilliant! Lets hope Polar Express can be "re-mastered" in the future,like the CD's, to make it the movie we all hoped it would be.

    ReplyDelete
  7. This would be a rare moment where I would actually encourage the director, Zemeckis, to pull a George Lucas and do a major overhaul and release a "special edition," complete with entirely new kids.

    ReplyDelete
  8. Ward;
    now that you've shown what re-touching can do for a piece of excrement like Polar Poop...er, Express, how about getting out the photoshop tools and trying to fix all the dull, vacant expressions in "National Treasure?" Oh, wait-those were "real" actors! How 'bout "Superbabies?" "Shall we Dance?"

    ReplyDelete
  9. Well i haven`t seen it too just like Jared Chapman said. I don`t want to :) But i would like to say something from my experience as character animator. Sometimes you know that something must be done in certain way and you do it, but the director or someone else tells you that it`s not good that way. At the same time you`re pretty sure that it`s right... but he is the director :)

    Marin Petrov

    ReplyDelete
  10. I believe that that's the main issue here, as well. That Zemeckis, the director, assumed that the character designs for the kids and Conductor were stunning. That the lighting and the rendering of the flesh and clothing were absolute amazing, that there couldn't possibly be anything wrong with what his crew was doing on this film. That all of the technological wizardry in this film was going to be the saving grace for the film, when in fact, it was it's thorn in the side. Aesthetics is very important when it comes to computer graphics, especially when it comes to depicting human beings. If you don't get the characters down right, then it's hard for the audience to follow along with your vision. Thanks for the comment, Marin.

    ReplyDelete
  11. I just saw Polar Express in IMAX (at the insistence of an executive friend in Disney marketing, who called it a "life-changing experience"). I can immediately say two things:

    1) The "flying ticket" sequence is the most viscerally astounding thing I've ever seen at the movies.

    2) There are songs in this movie that are just south of "Santa's Super Sleigh" in their saccharine-drenched awfulness.

    So you can imagine how conflicted I am over the experience. The character animation was very disappointing, too. It’s intriguing how you say that “even though every little detail is recorded by motion-capture technology, the performance up on screen still looks stiff and robotic”. I think the heart of the problem with this film was in the recording. They needed every detail, and they didn’t get it.

    I’ve heard from friends in the industry that Zemeckis was holding back his animators, preferring to let the mo-cap performances speak for themselves whenever possible. I understand the reasoning behind this. If you have good actors, and you’ve captured their performance – on film or on a computer – you want to keep that pure. The problem is the data set.

    Consider a live-action movie like Sideways. Actors created the performances in real time. Those performances were captured on film, which is a very big data set. (Film, as AMPAS preservationist Michael Friend says, is so good “we literally don’t know how much resolution there is”). All of Paul Giamatti's gestures and facial expression survive the journey from the camera to the projector to you.

    Consider The Incredibles. The vocal performances come from actors, and the images come from a data set created by animators on a computer. The acting data set is limited to a few hundred variables of body movement. Yet that’s enough to convey real emotion, because the actors doing the animating are working with enough variables, particularly in the eyes and face, to simulate realistic expressions.

    Now look at that tragic before-and-after diptych of Tom Hanks bellowing into the P.A. In the analogue version, he has no costume and no makeup and a gaggle of mo-cap pockmarks all over his dermis, but the facial expression is telling us everything we need to know about his state of mind. In the digital version he’s costumed and stuck into a set, but that liveliness is gone. Why? The data was lost. Ironically, there’s more emotional information in the raw production still than in the final product. There just weren’t enough mo-cap sensors on his face, and that wild pinched expression was never recorded.

    Bummer. Anyway, I loved your improvements!

    Taylor Jessen (AWN.com)

    (Here’s a great article on film vs. digital, six years old but still relevant: http://www.laweekly.com/ink/99/20/cyber-chute.php )

    ReplyDelete
  12. What great feedback, Taylor. And thanks for visiting. I had read somewheres that Zemeckis told the animators to hold back on the movements and just let the actions speak for themselves. Too bad it did not help. I was going to do a follow-up to this continuing commentary mentioning that. And regarding the data set: very interesting. I completely agree with that. I'm still not sure even if they did have the capabilites to place sensors on every possible nook and cranny of Hank's face that they'd be able to capture his pure performance. Without animators going in and tweaking and key-framing, the digital performance will still be soulless, in my opinion. And thanks for the article. I have to check it out now.

    ReplyDelete
  13. In your search for why Polar Express looked so creepy, you may have come across my website (http://sterlingnorth.livejournal.com) and a post about The Uncanny Valley. Yeah, it did explain what it is, but not why it happens... well it may have but I didn't really know why.

    Since then, I've been thinking a bit about it, and I have a hypothesis. If you're tracing something off a photograph, most people draw a line just outside the contours of the image. When you lift that drawing up, people will include the contour as part of the body of the image. The outline is now slightly fatter. I think the exact same thing happens with motion capturing -- whether CG or rotoscoping. They're contouring a milimeter or centimeter just above the skin. That must have the affect of dampening all movements and expressions, thus making it look like all the characters are covered in carbonite.

    ReplyDelete
  14. This points out the eternal battle between artists and consultants, I deal with this kind of mindset all the time. I'm sure there was some Harvard-grad in a suit in the preproduction meetings saying things like "Hey, let's give him a robot arm! Kids love that!" Sheesh. Anyway, good on you Ward for having the guts to show that, once again, the emperor has no clothes. - mh
    http://www.matthaley.com

    ReplyDelete
  15. I think The Polar Express animators had plenty of motion capture data, but they didn't do (for whatever reason) what Peter Jackson's animators did with Gollum, augmenting the movements to correct shortcomings in the technology, the data, or the performance.

    If you're going to rely completely on motion capture, it's probably a good idea to try to capture the body and face motion during the separate actor performances. The facial expressions alone need to be captured with much greater detail than the rest of the body movements.

    Ward said it - it would be interesting if, say in about a decade, a studio (though not necessarily with Zemeckis) could take the existing motion-capture data and voiceovers from The Polar Express and make a completely different rendition of the story, using lessons learned and using newer technology. They'd also have the option to make the Tom Hanks character not look like Tom Hanks wearing a Tom Hanks mask.

    ReplyDelete
  16. I recall reading an article after Shrek's release that was similar to "The Incredibles" comments regarding photorealistic characters. The Shrek animators actually had very photorealistic characters (the villagers, but especially Fiona), but found that if they scaled back on the realism and the characters were more believable and viewer friendly as well as matching the tone and setting of the story.

    ReplyDelete
  17. I'm confused-I've seen very little mention of the Imax 3D -- that is, 3d where you're wearing glasses? -- but if you see THAT version, it will at least be apparent that it is the salient version of the film, even if it's only playing in 60 theaters. Some of the problems are instantly ameliorated by the 3d process - in fact seem to be CAUSED by watching it in 2D. Some of the creepiness persists, but the experience is so astounding -- so far superior to any 3D movie I've ever seen (and no doubt to the 2D version) -- that to waste all this time discussing the 2D version strikes me as pointless. Go see it in 3D and then we'll talk.

    ReplyDelete
  18. I saw the movie in 3D Imax. The most obvious flaw was the mouth movements; they all had a very restricted, very fake, very "Planet of the Apes" quality.

    Great work on this page!

    ReplyDelete
  19. I wouldn't say it's pointless at all considering that it's the exact same film you're watching, just that you're having that illusion of that third dimension. I've heard from others that the movie is much better in the IMAX 3D (and whenever I talk about 3D in my above commentary, it should be referred to as CG, not the actual 3D with glasses), and I plan on seeing it, but I can tell you now that it wouldn't matter to me if these characters would be moving in the third dimension because the actual DESIGNS of the children and people are awkward. The movements, the faces, the weight distribution, etc. would still be as awkward as they are in regular, flat screens. Only now they would be more realistic in depth and space to my eyes. Big deal.

    Remember, I did like the film. Just that I could not get past the characters and their movements. Once I see the film in an IMAX 3D setting, I'll post about that too. (With even MORE "fixed" images!)

    And regarding about that extra layer concept, I'm not exactly sure if that would be the reason for the oddity of structure and the overall movement in the characters, because the same process was used in LOTR with Gollum, and I did not see much problems in that character's movements, if at all. I really think it goes to the director and his minions and how much time, money and energy they wanted to put into that aspect of the film. Apparently, they were more concerned with the overall look of the entire film, the train, the snow, the mountainscapes, the North Pole, etc, than they were with the characters.

    Keep it coming, people! And thanks for the comments. Great discussion.

    ReplyDelete
  20. I actually saw the movie, and the film itself contains a telling contrast between mo-cap and animation. If you ever do see it, look for the characters of the actual Engineer (not the Conductor) and the Fireman. They are grotesquely obese and ridiculously thin, respectively, and are so stylized that I believe they must have been the work of animators. The Engineer's fat rolls around and around his body, the Fireman moves like a frantic walking-stick and both of them have fully-animated, expressive faces. It's like the animators took their one chance to really show off. I remember being struck by how those two characters made everyone else seem even more lifeless, and was sad that they had such a small part in the movie.

    Interestingly, I read an article about the creation of the CGI stunt-double for the little girl in Lemony Snickett, where a big deal was made of the lead animator using his experience painting portraits. Apparently he hand-tooled the model based on hundreds of stills and hours of video, since the girls used were too young and wiggly for motion capture. I haven't seen that film, but I found it interesting to read that they made a special effort to animate the eyes properly. Can anyone attest to the results?

    Troy Lissoway

    ReplyDelete
  21. Hey Ward. I am an animator in Atlanta as well... On the show I work for all of our stuff is done in 2D in After Effects, with pre-drawn heads, faces, hands, etc. This takes a little bit of the spontaneity out of it, but it's super cheap and fast. So, when I am animating a character, pretty much the only way I can convey emotion in their face, is using the eybrows. Because of this experience, I can tell you that eyebrows are one of the most important ways an animator can show emotion and feelings. It seemed like the changes you made all involved some kind of eyebrow change.

    Also, I took a motion capture class when I was in college and we learned about the limitations of motion capture in the first 3 weeks of the course. That Zemeckis DIDN'T have animators go back in and polish the facial expressions and movements, the way Jackson did with Gollum, is shocking, frankly. To me, it seems like that would be common sense. Oh well, great post.

    mack williams
    mackwilliams.com

    ReplyDelete
  22. All he did was give them worried looks .He's hardly a genius or needing to explain the brilliant technique.

    ReplyDelete
  23. Anonymous, you've just strengthened my point: that it didn't take much for the "super-geniuses" behind Polar Express to just go in and do some tweaking to fix the bizarre dead faces that the characters possess. And I've never said that I was a genius. I'm just a concerned citizen who happens to be an animator and who enjoys to look at aesthetically pleasing things.

    And if you read the commentary, I gave the children the looks based on what was happening in the story: wonder, amazement, interest, and yes, concern. You're generalizing here.

    ReplyDelete
  24. This is a different person from above FYI.
    I have always had a keen eye for detail and although I am not a professional artist I have always been fascinated by the details of the body. There is a lesson most 2d artists do early on that involves drawing your hand without looking at the paper while drawing. You only look at the hand. You end up with a very stylized version of the hand and it strongly draws your attention to the details you are noticing on the hand. It also gets you in touch with how a “too perfect” hand looks “not quite right”.

    I think part of what is lost in much animation and motion capture is that skin has layers and flaws. The reason animation looks like puppets is that only the outer layer is being rendered. So it looks like the characters are solid (like a puppet). The body is a (mostly solid) skeleton with a layer of muscles and tendons moving over it and then the skin moving over that and all filled with varying viscosities of liquid. Not to mention all these are being subjected to gravity. Usually the skin will distort as it moves over the bones and the muscles and tendons. This actually is why people who have had face- lifts and such look like their expressions are “plastic”. The skin has lost some of its ability to respond to the bones and such moving under it.

    The color of the skin also changes subtly as it is stretched, as it is not as deep. It is important to remember that skin is translucent (hold a flash light up and cover it with your hand completely in a dark room to see this.) The color of skin is really the color of light moving down through it’s layers and then being reflected back out again (or passing through it, like in the lobe of your ear with a strong back light). You saw this with the eyes and corrected for it nicely with your redraws but it is still a “rendering” of it. Eyes are DEEP! And have varying levels of clarity to the surface. For example the area around the cornea is mostly transparent with the “color” of the eye actually below the reflective surface. Take a moment to look very closely at someone’s eye to see how it is almost like a gem in it’s depth and reflective qualities (and flaws).

    Another thing that is lost is the wrinkles at the edge of the eyes, and the pinch between the eyes. I think the “feel of realism” can be linked especially the part between the eyes. You’ll notice that both Gollum and Mr. Incredible have it and that the others pictures do not.

    The result of these details missing is one of the most glaring problems with motion capture that I see. You can only ever get the opaque surface of an object and there is more to it than that that your eye sees even though you might not register that you have seen it.

    Anyway those are the parts that struck me the most. It would be interesting to see what else would come up with more detailed examination. I have background in theater lighting and you should see how the skin reacts to different colors of filtered light! Even different actors look different under the same light! Part of theater makeup is just trying to get everyone looking the same.

    Great article and as a life long Fan of CG (I went nuts when I saw the old CG tin can commercial the first time and have been hooked since) I would love to see the art push forward!

    Shadowgolem@yahoo.com

    ReplyDelete
  25. I was the anon. recommending the 3d Imax version above, and in fact I share many of your concerns about the film (the kids still do look faintly bizarre, esp. the young girl played by Nona Gaye) (or is that Enola Gaye?), as well as the clunky narrative - although, again, the "cleverness" of the 3D gives even the escalating jeopardy a playful quality; the succeeding levels of "can-you-top-this" make more sense. Overall, though, and despite its flaws, it's my enthusiasm for this version that I wanted to express. I think, when you see it, that you'll be surprised that more people aren't declaiming just how spectacular an experience it i. So I'm saying go tonight. Cheers and thanks for the lively forum...

    ReplyDelete
  26. Wonderful article.

    I'm not an animator. I'm an fantasy illustrator and a friend sent me to this article. Thanks RC ADler!

    I was turned off by the Polar express by the herky-jerky movements of the dance sequence of the waiters. Just didn't feel/look right. And beyond the actual animation, I thought the scene was too busy. Just too much movement and crap on the screen at the same time. Too much detail and no sense of focus. That is just simple design theory. That is inexcusable for a director of Zemeckis's talent. That was in the preview, never went to the movie.

    I thought movmeent was better in Final Fantasy. And that was a much older movie. Much more dated. Maybe herky-jerky "works" for bodies in death threatening combat, compared to mass dance numbers.

    But one positive thing I can take from this... sometimes it takes a few maintaining-status-quo movies before the next big leap is made. I'm glad that CGI is continually being pressed into service. I think great things are around the corner... such as the in the Incredibles. Which was simply wonderful.

    ReplyDelete
  27. In addition to what everyone else has said - Im really glad you touched on the problem of the eyes. Being a 3d animator myself I know its standard practice to create lights that only cast light on the eyeball - to give them life. Everyone has this preconceived notion that eyes arent alive and dont have that "sparkle" if there is no highlight over the iris. The problem comes when it no longer looks like the eyes are sitting in the world. They glow and stand out too much. Your right - in real life often the eyes are lit to darkly to tell the iris from the pupil and there are no highlights, often they are just dark holes. We read emotion from more than that, as you youve pointed out the eyebrows and the areas around the eyes are very important.

    Its so common in cg to hear that the eyes look dead or blank, but I really think everyone is puting so much attention into them that they stand out, and thats the reason they look wrong.

    ReplyDelete
  28. What a brilliant post! I'm from the UK and never actually seen 'The Polar Express' since I don't feel that film wasn't that great to see. Mind you, I went to see 'The Incredibles" and actually lost myself into that film. I was somehow 'tricked' into thinking 'this isn't really computer graphics, it's a film after all'. (If you get my drift).

    ReplyDelete
  29. I agree this page is brilliant. It is easy for me to be creeped out by the animation from P.E. -- it's quite another for someone to go in and demonstrate what was wrong by making the characters look SO human with a few subtle changes. I hope you get a job out of this.

    ReplyDelete
  30. Great article and discussion.

    I haven't seen the film, and probably won't. But, the comments about a "director's cut" made me wonder if the character based parts of the film might be more successful if they were shown as a one man play, just Tom's motion capture footage, edited as in the final form. Okay, perhaps the MC zits might be disturbing, etc, but it's amazing what you can get used to. And what you can't.

    My $.02

    ReplyDelete
  31. Hey look, the guys probably did a great job at making THE POLAR EXPRESS. The fact is that making a person in CG is very hard because everybody knows in exact detail how a human being looks like, moves and shows emotion, so it's practcally impossible to recreate a person, this doesn't apply to Gollum, because nobody knows how he should look exactly so everybody accepts his character. I do agree that te changes you've made make the characters look a bit more human. But i believe i can even retouch my own photo's so they look more human, that's just how virtual reality works. but to make the changes you've made to a character troughout the movie thats the hard. It's practically impossible to keep those changes consistent through a hole scene. So it might still feel 'fake'.

    Greetings Jim

    ReplyDelete
  32. Totally new anon, here. Bless you for pinning down most of what was SO wrong about the Polar Distress. I think 70% is in the eyebrows alone. The proper mo-cap scheme would probably have many, many dots around the eyes and eyebrows.

    My question, for those who've seen Fahrenheit 9/11, in the clip showing Asscroft singing "Let the Eagle Soar," was that live action, or did Michael Moore do some kinda Mo-cap mojo???

    ReplyDelete
  33. Your alterations are GREAT! :)

    ReplyDelete
  34. Firstly I'll say "great blog!".
    I read an article - can't find it at the moment - that said that they scanned the real boy then found that they had to change his eyebrows because they did not match Tom Hanks and therefore the mocap did not translate well. That could be the begining of the problem.
    Secondly, I think that putting an adult in a childs skin was always going to fail. Think about all those subtle differences. If you could do it for real you'd expect it to be freaky! you'd expect the result to look possessed!
    And thirdly, I think that mocap is like "putting on a prosthetic suit" especially when the final character is somewhat different from the one being captured. You have to over act to get over the limitations of the gunk between you and the audience - and I think that's what is really wrong. Tom Hanks and the rest of them (probably under the directors guidance) did not have that experience and missed the whole thing.
    i think they got lost in capturing the actor rather than creating a character.
    I prefer the more stylised approach too - I think it will be some time before we see realistic humans animated - and like the result!

    ReplyDelete
  35. This comment has been removed by a blog administrator.

    ReplyDelete
  36. I haven't seen the movie, so I don't know how relevant this comment is, but the thing that worried me when I read about it was that Tom Hanks, a big guy, was being motion captured and scaled down to a small boy. Both physics and simple observation say that when you scale something down in size, it moves faster, relatively speaking. A kid's arm doesn't weigh nearly as much as a grown adult's, so it has less inertia, and can wiggle faster. When kids walk, they take faster steps of shorter distance, compared to an adult.

    Think of Alvin and the Chipmunks. To get that kind of sound quality, you have to sing slowly, then speed up the tape. To map an adult to a child (never mind that the limb proportions are all wrong), Hanks would have to act at normal speed, but lip-synch with a slowed down version of the vocal track. The speed up required wouldn't be anything tremendous, but without it, I'd imagine you'd get rendered children that look either like they're a little tired, or walking around on a planet with slightly less gravity than the Earth.

    Again, I haven't seen the film, so I don't know whether any of this applies to the animated children, but I was really surprised when I heard that they weren't using children (or at least small adults) for the appropriate motion capture.

    ReplyDelete
  37. I'm a new Anonymous...

    Mikekosim writes: "Suffice it to say that the detail around the eyes seems the most subtle (equated to highest frequency) and that the nuances needed for capture around the eyes are actually far greater than the rest of the face. In the depiction of Tom Hanks in his MO-CAP gear, the area around the eyes, having more subtle nuances, should have far greater MO-CAP detail."

    I agree with the frequency comment.

    Unfortunately, I'm not sure it's possible to get enough reflectors or sensors on a person's face. The Nyquist frequency would require that they be placed at half-wrinkle-width intervals. That's why it's necessary to do it by hand, at least until some kind of 3D laser scanner is available that would work for this application.

    Rich D. writes: "They'd also have the option to make the Tom Hanks character not look like Tom Hanks wearing a Tom Hanks mask"

    I love that!

    Another anonymous writes: "Interestingly, I read an article about the creation of the CGI stunt-double for the little girl in Lemony Snickett, where a big deal was made of the lead animator using his experience painting portraits. Apparently he hand-tooled the model based on hundreds of stills and hours of video, since the girls used were too young and wiggly for motion capture. I haven't seen that film, but I found it interesting to read that they made a special effort to animate the eyes properly. Can anyone attest to the results?"

    I never had any clue that she was CGI at any point in the film. I had no idea. Perhaps they used CGI for shots where the girl's face wasn't visible? Now that I think about it, those were pretty common, and I can think of two particular scenes that would qualify as stunts.

    One, for example, where the girl is shown hanging from the table, supported by her teeth, which are clamped on the edge. First she's shown from behind, feet dangling. Then she's shown from the front, when her feet are not visible and could be supported.

    In the other scene, she catches something (looks like a wooden throwing knife) in her mouth after it's tossed by her brother. If I recall correctly, she is shown from behind until she's caught it, at which point she turns to the camera with it in her teeth.

    ReplyDelete
  38. For all the truth in your points, if you see The Polar Express at an Imax 3D you will shit your pants. It is seriously good.

    ReplyDelete
  39. I have heard this, and so I'm thinking on seeing it in IMAX 3D, once I finish up with this one project at Primal (I've been working on this at night to do so). I'll be doing a follow-up to all this, as well. Thanks for posting, Brian.

    ReplyDelete
  40. I'd like to start by saying, you've done a great job improving "The Polar Express" stills.

    Perhaps the creators fell short of adding expressive details such as folding skin, because they couldn't spare the time or expense to build such a complex digital puppet. By no means is this an excuse for the stiff, unappealing animation in "The Polar Express", but it might be part of the problem.

    Having said this, I'd like to point out that adding such details in a 2D animated character is much easier. If you want little crinkles to appear around the eyes, you just... draw them in. In a 3D animation project, you have to pull off major technical feats to incorporate that feature into your character.

    Granted, 2D animation doesn't have as much capacity for photorealism as 3D, but it's just as captivating. Why does so much of the animation universe believe that 2D is dead?

    Photorealistic 3D characters and effects have a place in the movies; Gollum is an excellent example. He's a digital character in a live action film. But why try to make a photoreal 3D cartoon? In my view, that's a contradiction in terms.

    ReplyDelete
  41. As a mocap artist, I can honestly agree with you on the fact that motion capture (please don't give me that "performance capture" crap) will not stand alone in an entertainment piece. Mocap is great for two things in a film...

    1) It saves your animators time by doing the large motions for them, leaving them to focus their energies on putting subtle expression and life into the character.

    2) It captures small movements of the body most people don't think about. Do this right now. Stand up. Stand fairly still. Become aware of how much your body moves regardless of how still you try to stay. Notice how much your leg muscles slightly contract and expand to keep your self upright and balanced. A living thing is rarely ever still, and mocap will add those ever-so-slight twitches and weight-shifts to a character.

    Mocap is a tool. It is a tool for animators to use, to accelerate production and help them deliver more believable performances by supply more believable general motions. But just as a hammer will not build a house, mocap will not make a film. Mocap is only one of many tools an animator can reach for when creating something.

    That is why it is called Motion Capture and not Performance Capture. Motion is something you put into animation. Performance is what you get out of animation.

    ReplyDelete
  42. http://vfxworld.com/?sa=adv&code=319b255d&atype=articles&id=2306&page=1

    I'm with you Ward but I really think that, bottom line, it was poorly executed. I ended up watching TPE because my neice turned up with when she was staying at our house. She loved it and to her the odness of the characters was down to it being a cartoon - pure and simple.

    It's easy, if you're related in anway to art/design/animation and see the merits in a film like the Incredibles and the poorness of TPE but like someone has almost suggested already, it's just symptomatic of the whole movie making business.

    I was fortunate to attend a workshop with Ed Hooks and he pointed out the article above in one of his news letters - it's well worth the read and points out many of your points - particularly the 'eyes'

    TPE was not a bad film, neither was it close to great. It's certainly not the future of movie making but i really think the best thing any of us can do is accept it, ignore it and go out there and do a Brad Bird, or what ever it is you do. For all the rubbish out there, there are peopple in darkened rooms burning the midnight oil on projects that may never get seen - projects that urinate from an enormous high on the likes of TPE.

    Zemekis has already proved his worth as a movie maker - he's made a few dud's perhaps, and now we'd consider this another - meanwhile the rest of us need to just prove ourselves.

    Cheers,

    gila

    www.gilagraphics.co.uk

    ReplyDelete
  43. It's true, Zemekas held back the animators. They had to match the performance that was 'captured' on stage. What the mocap did not capture correctly had to be hand animated to match. All the movement had to be 'Tom'... (Which is sad since adults just do not move or react like children. Facially or physically. But that is another story.)

    In my opinion what was wrong with PE was that some key people forgot that Motion Capture is just a tool, and nothing more. It still requires a character animators' talent to make it look 'real'.

    But, that being said, what the client wants, the client gets. It doesn't matter what you might personally think, they are paying the bills. It's their film.

    ReplyDelete
  44. I don't blame the animators. I did know that Zemeckis was the one responsible for holding back the animators and wanted to get the "pure performance" of Hanks and others. However, it DOES matter what I think, because, in essence, WE ARE THE CLIENTS. We pay THEIR bills by the number of our butts in theater seats. And once it's out on the screen, it becomes our film. (That's why I feel that Lucas has forgotten this very important aspect of the film-going experience. Once a movie has been made, it becomes a part of the culture, a part of our lives, our experiences. We make it ours, to a degree. And that's why there's so much hubbub over Lucas tweaking the original STAR WARS trilogy. But I digress...)

    To the mocap artist, I would love for you to get in contact with me, as I plan on doing a follow-up post about the film, once I see it in IMAX 3D and have gathered so much new information about all this. I'd love to get some feedback from a mocap guy him/herself. Email me by going to my profile. Thanks.

    ReplyDelete
  45. Your touch ups certainly gave more emotion in the characters faces. I hope the makers of the film see this and offer you a job!

    Mark. UK.

    ReplyDelete
  46. i think that you did a great job touching up those characters. i think that you should have a job in the movies for graphics. :)

    from,
    E.H.

    ReplyDelete
  47. Yes, I completely agree with you. In fact, I mention this theory, the "uncanny valley," in the first installment of this commentary, HERE. Thanks for the comment, by the way.

    ReplyDelete
  48. Excellent modifications there! They just tweaked those images up a notch, closer to photorealism. But that's just a one frame in a movie that contains way over 100.000 frames.

    It sure is a shame that almost all of Hanks's true characteristics and charisma has gone with the wind while they zapped him into digital world. I believe there are few main reasons for that to happen. And most of them are caused by "blindness", which is caused by technological issues.

    Let's say a director knows always what he (or she) wants and makes sure gets what he wants. He always spots a wrong facial expression in a single second of a 30 second take. Now when we stare at the same take created by a computer, we are quite amazed what we can do with computers nowadays. All looks so life-like and real. All looks so perfect and polished. All looks just too fine to be true. And made with this "box" on the table? So, could it be that the director can't say how to make those animations more life-like or he is happy what he sees on the screen and thinks that it cannot be made any better? "Real movie" directors are used to work with live humans and not with old men and little boys made from ones and zeros, but on the other hand, facial expressions and other movements should be the same in real-life and in digital world and quite easy to spot.

    However, it's a totally different world. If we'd like to have a perfect CG-movie, we should take each frame in a movie under special care. Have a good look at it: what kind of an atmosphere it has, does it give you the right feeling and warmth. But when that frame is modified and tweaked to a point that it could be sold as a masterpiece next to Van Gogh's paintings, and it's shown to you with 23 (depending on frame-rate, eh) other images in a time of a second, you really can't see the major difference.

    This is a long path to walk. Photorealism. Maybe in year 2200 you don't need real actors. Every other child who just got his first computer could re-make Matrix Trilogy in a weekend - and make it look even better than the original movies. Maybe. But what's the point? I'll say, it's nice to have that sort of a CG-feel in a movie that is all about CG. Without the certain CG-feel it looks like a real movie shot with real cameras and actors.

    As earlier said, you can tweak and tweak, but with those schedules it certainly aren't possible. Some people are just expected to see the finished movie in theaters.

    And don't get me wrong, those adjustments you made were just the thing that was needed to spice up the scene - or the frame. Keep up the good work.

    ReplyDelete
  49. We can gain a lot by looking to the Nine - the Nine Old Men. This is why Mo-Cap is inadequate to the task:

    "The camera certainly records what is there but it records everything that is there, with an impartial lack of emphasis... On the other hand, an artist shows what he sees is there, especially that which might not be perceived by others. His drawings [or poses, in the case of 3D animation] can be closer to the true realism of an object because he can be selective and personal in what he chooses to show... The point is: a work of art is never a copy; for it to have meaning to people of many generations and numerous cultures, it must be the personal statement of the artist."
    - Frank Thomas
    Chapter 13. The Illusion of Life.

    The technique that Ward-O-Matic (wwwwaaaarrrrddddd) describes from LOTR was looking at Andy Serkis' filmed performance, roto-ing it, exaggerating important subtleties, and animating the face traditionally (they also had to animate fingers and toes). This is the technique they used more often than not and it's the same way Walt and the Nine captured performances for "Snow White". The scene where Gollum's face changes when Frodo first calls him "Smeagol" was traditionally-animated. There is emotion a $3-mil mo-cap rig would be challenged to reproduce.

    Watch the Two Towers' Platinum Edition third disc, titled "The Taming of Smeagol" to see a reflection of what is happening in all of Hollywood. I'm particularly satisfied with Jackson's discerning eye when it comes to the 'flavor of the month' effects versus what looks great (i.e., he doesn't always use CG).

    However, motion capture is being heavily pursued by Hollywood. They hope with enough investment now, it will be quite cheap later and they won't have to bother with us cruddy 'ole animators anymore. It's like buying McAnimation. Give me a shake with that. Hollywood is ultimately a business - we can't forget that. Right now, they aren't willing to pay the traditionally- animated toll.

    If you haven't seen Polar Express - don't.

    When you pay for a movie, that's your "vote". When you buy tickets to the Ice-capades performance of the Polar Express, you're voting. When you buy that insulting stuffed Santa Claus that appeals to the largest possible existentialist audience by saying "Christmas... is in your heart" you are voting. To vote "NO" you don't buy! Let your voice... not be heard.

    ReplyDelete
  50. Your right about those boy's eyebrows in the Polar Express, but you didn't do him any favors by giving him African American lips, and I couldn't really see any differences you might have made to the eyes. Oh and I think you made that girl’s skin even drier. And from a 2D and 3D animator’s point of view, I think they’ve done well in The Polar Express, the cartoonish images look great they have served their purpose.

    ReplyDelete
  51. Anonymous, you are entitled to your opinion, just as I am to mine. However, there are a few things I must point out that you bring up. First of all, that is a rather general viewpoint about me making the boy's lips look "African-American." No one else has made this assumption, thus I feel that you are in the wrong. And even if I did, why would that be wrong? Sounds like you have some issues there that you need to work out.

    Most of the tweaking I did on all the eyes (yes, I worked on ALL the eyes to all the characters in the examples I fixed) were in the pupils and iris. Look closer and you'll see.

    Another thing, I did not even do one tweak on the African-American girl's skin, so, what you see in my new version must be completely in your mind.

    For the record, I am a traditional animation director. I've been doing this for 9 years. I think I know what looks right when it comes to animation or not, so, in my opinion, THE POLAR EXPRESS was a huge waste of time, energy and money. Again, in my OPINION. And based on the majority of the responses here, I am not alone.

    Purpose? What purpose are you talking about? To freak audiences out? They certainly did that. But "cartoonish?" No. If you thought that the filmmakers were trying to go for a "cartoonish" look in the characters, then they were WAY WAY off. Think about it -- they were trying to go for REALISM, here. They put millions of dollars in capturing the live-action performances of human actors. No over-the-top expressions or takes. There was no indication of trying to go "cartoonish," as the filmmakers worked feverishly on getting all the characters, settings, environments to look as close to real life as possible. From the freckles on the boy's skin down to the millions of strands of hair on all the children's heads. There was absolutely no hint of trying to go "cartoonish," in THE POLAR EXPRESS. So, you are terribly wrong in your assessment there.

    ReplyDelete
  52. Take it easy buddy, u expressed your opinion and I expressed mine, so did a dozen other people here, but I’m the only one being attacked.

    If your implying to my ‘issues’ as racism I can assure you I’m not racist, I grew up listening to MC Hammer, and please no one hold that against me, I’m also a fan of Will Smith, Whoopi godberg, and my all time favorite classic is The Cosby Show.

    Any ways back to the matter at hand, I may be mistaken about the African American girl, but I guess I have better things to do with my time than to stare at images picking out imperfections. The Majority here didn’t like The Polar Express, but I’m sure not all of them disliked it for the same reasons you didn’t like it. I mean I’m not a fan of South Park and their illustrations are kept to a minimum and I’m sure even amateur artists can draw better than the South Park Illustrations, but you got to love the time and effort they put into these animations. I hope I’m not getting off track too much here but my point is, alot of people here didn’t like The Polar Express, but if you search the web a lot of other people liked it and would agree with me.

    I seriously doubt ‘The Polar Express’ freaked anybody out, if any animated character should freak people out I think Homer Simpson gets Gold Medal and yet the whole world loves Homer. He’s not perfect, I mean any one can jump onto Photoshop and do wonders with imperfect pictures, biggest evidence are playboy magazines.

    You stressed a lot on my term ‘cartoonish’, well I’m certain they weren’t aiming for a realistic style, I mean the whole animation is supposed to be surreal. If you’d told me Final Fantasy wasn’t going for cartoonish I would have agreed.
    Just because they’ve used Motion Capture doesn’t mean they cant aim for cartoonish, I’m sure if motion capture existed back in the days of Roger Rabbit (And you cant tell me Roger Rabbit wasn’t a cartoon) they sure would have used it rather than spending hours manually animating it.

    Cheers Anonymous :)

    ReplyDelete
  53. Before we continue any further on this argument, anonymous, you should read the first part of my commentary of The Polar Express HERE. In it, I talk about the issues raised in trying to emulate realism in art and/or film, which is what the filmmakers were trying to do here. I mention the theory of The Uncanny Valley, wherein non-human characters (robots, animated characters, etc.) are strangely rejected by humans the closer they resemble real humans.

    This is NO WAY related to Homer Simpson or South Park characters. Your argument here is not even in the same ballpark. In those shows, they are very much indeed cartoon characters and we accept them as such. They are stylized representations of humans, with no emphasis on trying to suggest realistic human behavior or mannerisms. Because of this, the viewer can easily associate and relate to these characters. It's just not even an issue here.

    When the filmmakers of Polar Express established the setting to being as close to real life as possible, they had a big task to tackle, because of the Uncanny Valley theory and with animation in general. Do they stylize the movements of the characters or not? Should they stylize the look and feel of the children, or not? They decided to go the ultra-realism route, thereby making it close to impossible to emulate. It was a challenge.

    Believe me, if you check out all the reviews on The Polar Express (you can see them HERE), the number one issue these people had with the film (and with all the people I talked with personally) were how creeped out they were by the zombie-like children. They simply were not buying the fact that these kids were believable as characters. Funny, since Pixar's The Incredibles had some of the most believable characters in recent animation history (heck, even in live-action films too).

    If you had read the first part to the commentary, you will see that I did, in fact, like The Polar Express as a whole. The major problem I had with it was the fact that the filmmakers were touting the animation and technology as "ground-breaking" and "revolutionary," which they were wrong in both accounts. I hate the fact that millions of dollars were put in a film that could've been pushed further toward having believable characters. My Photoshop experimentation done here was just an indication that all they had to do was spend a little more energy on tweaking the eyes and faces of the characters, and that perhaps more people would've been more open and forgiving of the film, ensuring a bigger box-office return.

    It also goes deeper in the fact that filmmakers can't just simply be complacent in thinking that audiences will go see anything that, on the outside, looks cool. It's all about the characters and the story. Point blank. That's why The Simpsons work and why South Park works. People love the characters. Great writing. It doesn't matter if they made a show with twigs and glue, if there's a decent story with some lovable characters in it, people will dig it. All the high-tech wizardry in The Polar Express was just a cheap facade, if you ask me. It was not worth it. The filmmakers were audacious in thinking that they had a sure thing. And they didn't.

    ReplyDelete
  54. Quite an interesting discussion, but many of the
    posts are missing something obvious:

    much of the discussion is about Mocap versus
    Animation. But note, the Polar faces look creepy
    in STILL pictures, and Ward has made improvments
    to the STILL images.

    Mocap versus animation is an issue of movement.
    While the mocap may be an additional problem,
    the "creepy face" phemomenon shows up in still
    pictures and thus is an issue of modeling or rendering.

    Unrelated,
    a couple other comments: Final Fantasy had
    hand-animated faces and mocap'd bodies,
    but the faces still looked creepy. Another indication that animating itself is not the solution.

    R.e. the Nyquist commentary (correct), the face
    tracking in the Matrix sequels used optic flow,
    meaning that every pixel was being tracked,
    not just a hundred markers. The results
    looked great (not creepy), as seen from their
    presentations at Siggraph. However they had
    a different(?) problem in that the results looked worse as the face got further away. The distant shots in the movies do _not_ look good.

    On Lemony Snickett you do see the CG baby's face
    in a number of shots, e.g. when she is biting
    the table. This is great stuff, the rendering
    is perfect.

    ReplyDelete
  55. Anonymous, I can see what you're talking about, but it doesn't hold up. Still images or moving, the characters in POLAR EXPRESS were creepy all the way around. And I discuss this already in my initial post, that I know that these are still images, that there is a lot that goes on in the modeling and the shading of the characters in the CG realm. BUT It doesn't matter if the kids are still or moving, they were horribly designed and the mo-cap did not suit the situation at all.

    What I did in these still frames was for aesthetics. And my point in doing it was to show that the simple tweaking in Photoshop could've been done in the modeling and shading of the characters. I will say this again: It doesn't matter if these characters are stationary or moving, they are still creepy. And believe me, I've seen the film twice, and I'll attest to that statement time and again.

    FINAL FANTASY is not worth talking about because NOTHING in that film worked. Just because there were bad animators (and I use that term loosely when talking about this particular film) working on the character's faces in that film gives you the notion that animation is not the solution? That's a very limited and very narrow-minded viewpoint.

    I will go back to the RINGS trilogy, and say that it was with Gollum where everything worked like it should've: both animators and performance capture from actors working in tandem with each other. In order for everything to work right, there needs to be communication between everyone, and Peter Jackson, WETA and Co. made sure of this, in my opinion.

    But I'm afraid that Zemeckis just doesn't get this, and so now we'll have to endure more creepy characters in BEOWULF and MONSTER HOUSE, although the creepiness might work in his favor for both films.

    ReplyDelete
  56. I'm very late to comment on this post but can't resist as it's in the area of my research.
    Apart from the Matsuhiro Mori Uncanny Valley stuff I hugely recommend "Superportraits: Caricature and Recognition" by Dr Gillian Rhodes of the UWA Face Lab.
    It turns out that the photoreal CGI is not just inferior because we can't get it to look as real as real life: we are actually not looking for real life when we go about our daily business. Rhodes explains that caricature often communicates the identity of a person better than a photo (and by extension a photorealistic facsimile) of that person because of the way we store the image of that person in our memory. Each new face we experience is mapped against a stored norm that we each have. That norm is derived by averaging all the faces we have experienced. When we remember GW Bush for example we probably remember that his eyes are closer together than those of most faces we have experienced. We store this knowledge, not realistically, but in an exaggerated (or caricatured) fashion. When a good cartoonist exaggerates this difference from the norm (by sticking Bush's eyes right next to each other) we instantly identify with it because the caricature more accurately reflects our memory of Bush than a photo of Bush.
    The thing that I find endlessly fascinating, and am trying to explore in my graphic design PhD, is that the human visual system, the eyes and the brain, should be set up to appreciate exaggeration and distillation despite having evolved through hundreds of thousands of years of looking out on the real (and unexaggerated) visual world. I have reached the conclusion, through a variety of good psychophysical and sociological reasons, that in visual communication terms, illustration (which allows for distillation and exaggeration) and drawn, non-representational, animation are superior means for conveying most messages. Meanwhile the advertising industry and the movie industry stumble blindly on using photography, film and poorly conceived CG design for 9 out of 10 stories.
    Compounding the problem is that most graphic design courses emphasise typography over image and even illustrators themselves have trouble articulating just what the strengths of their chosen medium are (I'm thinking here of books like "the Education of an Illustrator" by Heller and Arisman: 200 + pages and about a half dozen pictures!).

    ReplyDelete
  57. This whole post has been fascinating to read. As well as an artist, art educator and film fan, I'm a mom of a 2-year old who is absolutely devoted to this movie, so I've seen it no fewer than fifty times on DVD, once in 3-D IMAX. I'm in general agreement with most of the posts about the "creepy" characters, and the real need for some serious eyebrow adjustments on these characters and a heck of a lot more data-capture. I'm wondering if anyone thinks another factor in the oddness of it is the inconsistent and badly placed lighting - the center of every face seems bizarrely dark around the nose and eyes, as if they've had to work so hard to get the nose and cheeks to bump out they've covered everything in charcoal and wiped the high points off instead of modeling everything in the correct lighting for the surroundings. And at every point, the interior of their mouths is like a huge, dark hole with no back - once you pass the teeth you just go on in infinite darkness. For me, that was the creepiest of the creepy. And personally, I wasn't blown away by the over-animated, contorted conductor and engineer - they went too far the other way, and they had no reference in the book to make their identities mesh with the otherwise gentle and poetic reality of the images. A teeny bit of restraint there would have gone a long way.

    I do think there needs to be a little bit of correction when one says that they were going for "ultra-realism" in TPE- they're not doing photorealism that would make it look like a live action movie, but rather an imitation of the "realism" of the children's book illustration that is indeed focused on drawing every single hair but simplifies forms, overexaggerates, and has a very still, magical quality (even as they're rushing down the sides of mountains). It's a very 40's kind of look. Realism has changed a lot over centuries, the Dutch Masters' realism looks nothing like the American Realism of the '70s. Now, that doesn't excuse some very out-of-joint shoulders and elbows periodically, which either set of realists would have tut-tutted. And what was up with that Santa? He's really a piece of wood. That beard is made of foam, I'm sure.

    And I agree with one poster, the experience of the film is somehow much, much better in 3-D. Cuts out some of the odd head positioning in the girl character, especially, and begins to make sense of the unusual angles.

    I also have a little request for the gallery - if you don't have little kids, or if you're not deeply involved in your kids' lives, I can see why you think it's so horrible that they'd waste money on goody-two-shoes "pap" like this. But I'm so grateful for just one movie that doesn't include people making fart jokes, punching each other, and blowing each other away. Most of the pure animation movies do those things even if they are designed for and marketed to kids. And I'm saying this as an enormous Quentin Tarantino fan myself, so I'm not at all opposed to major bloodshed, twisted sex, inappropriate behaviour of the naughtiest kind, and the creative use of 4- and more-letter words in adult movies. I just think there's a way to make entertainment that doesn't include any of those things, and I'd be happy for people to spend a heck of a lot more millions on discovering what those ways are. Animated or not.

    ReplyDelete
  58. This is a very good read and it was nice to read varied options. But out of all the comments this one was spot on.

    “Mocap is great for two things in a film...

    1) It saves your animators time by doing the large motions for them, leaving them to focus their energies on putting subtle expression and life into the character.

    2) It captures small movements of the body most people don't think about. Do this right now. Stand up. Stand fairly still. Become aware of how much your body moves regardless of how still you try to stay. Notice how much your leg muscles slightly contract and expand to keep your self upright and balanced. A living thing is rarely ever still, and mocap will add those ever-so-slight twitches and weight-shifts to a character.

    Mocap is a tool. It is a tool for animators to use, to accelerate production and help them deliver more believable performances by supply more believable general motions. But just as a hammer will not build a house, mocap will not make a film. Mocap is only one of many tools an animator can reach for when creating something.”

    I would just like to add that Gollums expressions were basically rotoscoped form the test plates of Andy Serkis acting with minor tweaks from the animators. I feel that this is a good technique as the animators have hindsight on the acting so the can help improve the performance of Andy Serkis. It seems to me that The Polar Express was just bad use of motion capture and there is a lot to learn from its mistakes. Motion capture should not be dismissed and animators should not be afraid of it as they should not be afraid of computers or for that matter a pencil (that’s me) they are all tools to be used, there is a time and a place for every thing.

    ReplyDelete
  59. Who really cares about whether or not the characters are identical to the actors? It is a movie for children. I think the movie was great and the conductor looks just like Hanks! You obviously have too much time on your hands if you sit and analyze every detail in a kid's movie. My children love this film as do I.

    ReplyDelete
  60. What I wonder is- why did they decide on rotoscoping and 3D? If they had just done rotoscoping and traditional animation, or live-action, costuming, and the usual amount of special effects, things would have been much less complicated, and probably less expensive. I don't think they really thought this through very well.

    ReplyDelete
  61. Motion Capture that was magic and expensive back when they did this is now cheap and better quality. Look at
    http://www.phasespace.com/gallery.php?movie=7 for an example of what can be done in real time, instead of weeks later, and doesn't require more than a few minutes to clean up. This leaves more time for story and interpretation rather than just a CG technique.

    ReplyDelete
  62. You can't hear the bell because you're too damned cynical. You've forgotten what it means to be a child. Children don't waste their time being critical details. Find a more constructive way to spend your time and stop wasting mine.

    ReplyDelete
  63. Hey I got the perfect solution for you "anonymous": don't bother stopping by. If you don't like what I say then so be it. You decided to take the time to visit this blog, read through the post -- not me. Oh, and DID you REALLY read my post here? If you did, then you would've read that I actually liked the film, just not the attitude of the filmmakers touting this "new technology" as being the end-all be-all answer to animation. They were sloppy in their execution of the characters and motion and if they just paid attention to the aesthetics of human beings and such, then it would've been a bigger hit.

    But no, you probably just wanted to stir something up here by wasting your own time away by submitting a wasteful comment, empty and useless.

    Cynical? Who's cynical here? It's part of who I am when it comes to analyzing animation. In order to make better cartoons, we animators are going to be more critical than a general audience. True, kids are not going to pick up on the details like I am, but that doesn't mean we have to give the audience a lesser product. I did this to help us, as artists and animators, try and figure out how we could do better.

    BTW, I have two kids. Check out my artwork -- I animate cartoons for a living. I love animation and comics and cartoons. I think that I know more about what it is to be a kid than you'll ever know.

    Okay? Okay.

    ReplyDelete
  64. I thoroughly enjoyed your post. I do a fair bit of work in 3D (Lightwave, mostly) and I avoid character animation of human characters at all costs for this very reason. It's much easier (not easy, but easier) to pull off a Gollum or other stylized character because people don't have preprogrammed expectations. While they may have imagined in general what Gollum should look like, we instantly recognize when a human looks anything less than human because our brains process all the little details from every face we've ever seen in all our life experiences.

    Yes, children are more forgiving, but part of this is because their brains have a smaller database of details to process.

    One commenter blamed the "data set" for the lack of character depth. I mostly blame the director. Zemeckis is quite good at the "big picture," but, in my opinion, he frequently falters when it comes to detail. Some good examples of where Zemeckis' lack of detail kept films from reaching their full potential include "Contact", "Forrest Gump" and "Castaway". Polar Express needed someone to pay more attention to detail. Giving the animators more control could easily have compensated for the small data set. While you can't really make
    the characters "more human," good 3D artists and animators, given enough latitude, have become very, very good at "fooling" the human brain and making things seem more natural.

    I will say that while I found the characters visually lacking (and the elves downright frightening), the overall experience of seeing The Polar Express in IMAX 3D was, at the time, incomparable to anything I had seen prior. I haven't seen any other CG films in IMAX 3D since, so I have no comparison for how they may have gotten even better. But since my first viewing of Polar Express was in IMAX 3D, watching it in 2D is like watching a completely different movie.

    And by the way, a comment to "Anonymous". Ward is not a cynic. He is a skeptic. In film, animation, and similar lines of work, it is skepticism that drives the art form forward. Cynicism would just hold it back. Now if Ward were, say, an accountant or a doctor writing a blog about how how much the 3D work in Polar Express sucked, that would be cynicism. But artists in this field only get better by analyzing a work and figuring out how to improve upon it. I can assure you that it's unlikely an artist that worked on Polar Express would take offense to this, but would instead view it as a useful tool to make the next one even better.

    If CG artists and animators sat on their "laurels" and didn't continue to advance the state of the art, they would be out of work. The film goers who pay for the tickets will settle for nothing less.

    ReplyDelete
  65. Hey, great post.

    Now that the "sequel" to the Polar Express is out - Beowulf - I'm curious to hear what you thought of that.

    I never saw the Polar Express, but I did recently see Beowulf and found myself thinking of the Uncanny Valley and how inhuman the characters often looked in the film. The best things in the film were those that were completely animated, like the dragon, or the scenery. Not really a glowing recommendation for the team that worked on it, although I think there's a lot of potential for someone to do some really interesting things with it.

    ReplyDelete
  66. This is commenting on an old article, but...

    The Polar Express, its secondary star was the steam engine. Steam Engines continue to inspire awe and emotion in the people that observe these behemoths on the railroad.

    This is the Pere Marquette 1225. A Lima Locomotive Works 2-8-4 (wheel arrangement) steam engine built during World War II.

    http://www.youtube.com/watch?v=V2hln47cP78&NR=1

    And yes, this is the actual locomotive that was used as a template (and co-starred in certain segments) in the Polar Express.

    ReplyDelete
  67. Excellent article. Why bother paying Tom Hanks a fortune for that. It just seems pointless.
    Mocap is fantastic when used in computer games but this sort of thing is just using technology for technology's sake

    ReplyDelete
  68. you should do the same for "beowulf." also directed by zemeckis. the big problem with these films is the total lack of taste. as if the "animators" were really just technicians who never took an art class.

    ReplyDelete
  69. "In animation, it really takes a bit of exaggeration to make something look convincing."

    This quote speaks volumes as to why I didn't like Happy Feet. Whenever I saw the penguin dance, it felt stiff. I kept wanting him to make bigger movements. If you look at the movements of Red Hot Riding Hood, they are exaggerated, but they look right. If MoCap "animators" were allowed to make adjustments to the captured footage, then maybe it would look better. I think eventually people will get tired of badly executed MoCap and it will die a slow silent death. Oddly enough, I love MoCap when used in games.

    ReplyDelete
  70. For that matter Hanks played a conductor not an engineer, the only thing that I didn't like about the film is that there was no coupler on the front of the locomotive, though it was based on Pere Marquette 1225.
    http://www.youtube.com/watch?v=5CbKzozQs90

    ReplyDelete
  71. This is bad necro, I know, but I thought it was funny that I found this while searching for info on the process used in Avatar.

    Funny because I saw a picture today showing one of the actors in full mo-cap rig and side-by-side with the finished CGI 'actor' at the same point in the film.

    MY first reaction was that the CGI version just didn't have the same expressiveness as her real face. Exactly the same complaint you are making here and other people agree with.

    My next comment was that Gollum did a much better job of the intricate facial details, and now I see the same idea, with an explanation that they had hand-animators working the face 90% of the time.

    What's interesting is the progress or lack of in the 6 or so years since the Gollum work was done. In Avatar, the actor's face is painted with a fine grid of green dots and a camera is set up as part of the mocap gear, pointing directly into the face of the actor. So they can track facial movements with far greater detail than before. But this *hasn't* translated into better expressions. The tiny movements that convey humanity are still smaller than the grid of dots.

    Anyway, shame on my for reviving this old thread but the coincidence of my conclusions matching these for a different film was worth it, I figured.

    Here is the Avatar picture:

    http://www-movieline-com.vimg.net/images/assets_c/2009/12/avatar_zoe_mocap-thumb-585xauto-7962.jpg

    You can't tell me that CGI has captured her full range of expression, it's positively flat.

    ReplyDelete