Sign in to follow this  
Followers 0
DF Oliver

Programming Update #4: Animating the Jack

266 posts in this topic

Hi! It’s time for another programming update! This time I’ll tell you guys about our character animation system. We’ll even have to use some (simple) math in the end. So let’s get going.

A few weeks ago we met with our amazing artists Ray and Lee to talk about the goals for our character animation tech. Expressive characters are obviously very important for this game, but as a programmer I know that we have to fit them into memory, too. What we need is an awesome system that will be expressive and efficient. How will we do that? Good question!

In the old days of SCUMM, characters were basically animated using flipbooks (a slight simplification for the sake of clarity). That means a moving character was represented by a series of individual images called animation frames that were hand drawn by talented artists. The game would then show these images at the correct speed which would give the impression of a moving character. Redbot was done in a similar way and this is what the flipbook of his walk cycle looks like (as you can see I’m not really a good animator):

1.png

While the flipbook technique is flexible and makes it possible to create very expressive animations, it is also very slow to produce and change. It’s quite a lot of work for an artist to draw a series of consistent frames. Fifteen years ago, this wasn’t so much of a problem because the games were running at a much lower resolution, which means that a few animation frames were usually enough to create a convincingly moving character. For example, Guybrush’s walk cycle in The Secret of Monkey Island consisted of only 6 frames.

Things have changed since then. In fact, we are aiming for smooth high-definition animation which requires 30 frames per second! It would take a long time or way more artists to draw that many (consistent) images. But even if that weren't a problem, we still have limited graphics memory, so completely hand drawn character animations are not an option. (I’ll come back to the memory argument at the end of this post.)

We have to find an alternative solution in order to get the game done. The standard technique for character animation these days is called skeletal animation. This makes it possible to separate the visual representation of a character from its animations and therefore is more memory efficient. It is almost as expressive as the (pure) flipbook approach, and I’ll describe later how we got around this limitation, but first let’s have a closer look at how skeletal animation works.

The initial step in creating a new character is to draw its body parts: shirt, sleeves, head, and so on. Then 2.5D geometric representations of the individual parts are constructed (in our case in Autodesk Maya) and the images from the previous stage are mapped onto the geometry. After that, bones (or joints) are assembled into a virtual skeleton. Finally, the character gets a skin by attaching the geometry created earlier to the individual bones. This is called a rig. An image will make this much clearer, so here is the character rig of the Lumberjack:

2_thumb.png

But how does this help us create expressive characters? Similar to a real skin, the virtual skin follows the movement of the underlying bones. Since the body parts are attached to the bones in a non-rigid way the skin will deform when the joints move. This means that an artist can concentrate on animating the bones of the skeleton rather than manually drawing the different states of a character movement.

In contrast to traditional flipbook animation, this technique makes it possible to change the look of the character after it has been animated or to use the movement of an existing character for another one, which saves a lot time and enables us to create a richer world.

Skeletal animation is great for moving legs and arms, so it’s very easy to make a character walk or wave, for example. It doesn’t, however, work very well for facial animation or similar cases, because the face of a 2.5D character really is just a flat plane. So unless we come up with an alternative technique, this is bad news for the Lumberjack. He wouldn’t be able to talk. Ever.

As I mentioned earlier, the visual representation of a character is assembled from different parts. In the game, we can show and hide these elements dynamically. This solves our mute Lumberjack problem, because a rig can thus have multiple mouth shapes. An artist can simply animate the visibility of these parts in order to give the impression of a talking mouth. Our animation tech combines both skeletal and flipbook animation into an awesome hybrid system that allows us to benefit from the advantages of both techniques!

The hands of the Lumberjack are another great example where localized flipbooks work better than skeletal animation. Even though we could create bones for all the fingers, the fact that the hand geometry is flat is a problem. It’s pretty easy to draw a few different hand states and animate their visibility. Here is an example:

3.png

So far I talked about how characters and their animations are authored. In the last two weeks I've spent a lot of programming time on writing an exporter for Autodesk Maya that converts the rig and movement data into something our engine can read. In addition, I added support for skinned meshes to Moai, our development framework. We've already shared the code with Moai developer Zipline Games, so with a bit of luck this should appear in the tech's main line soon.

I have explained in detail what kind of data the character rig contains, but what exactly is stored in the animation files? While a flipbook needs to store all the pixels of the individual frames (which is a lot of data), in a skeletal animation we only need to store the position, rotation, and scale of the bones and the visibility of the body parts. If we want to make the Lumberjack wave we simply need to store data describing the rotation of the shoulder joint for each frame.

Here is our Lumberjack in the game. The purple lines represent the bones of the virtual skeleton:

4.png

Here is a bit of the Lumberjack's walk cycle captured from the game, as well as Redbot's:

5_1.gif5_2.gif5_3.gif

(If Redbot isn't animating, try viewing the image directly.)

I want to close this post with a bit of math! So you may want to stop reading now! Basically, I want to show you guys that skeletal animation can be seen as form of data compression.

Let’s assume we have our Lumberjack one a single 512x512 pixel image, which results in 1024kB of raw data (512 x 512 x 4 bytes per pixel). We can reduce its GPU memory footprint by using DxT compression (256kB) or PVR compression (128kB). I’ll use the PVR compressed version for the remainder of the post.

As I said earlier, we are aiming for 30 frames per second, so that means a flipbook containing 1 second of animation requires 3840kB GPU memory.

For our character rig we need the image as well as the geometry and skeleton data. The Lumberjack rig requires around 125kB of combined main and GPU memory right now. So that’s around 253kB (128kB + 125kB) of data for the entire rig.

The walk cycle of the Lumberjack character is one second long and in its binarized form requires around 24kB of main memory. Obviously , we need the rig in order to be able to play the animation, therefore 1 second of skeletal animation results in about 277kB of data.

So if we compare this to the 3840kB required for the flipbook this represents a compression ratio of 13:1 which is pretty sweet!

At this point I should mention that we haven’t optimized the size of the rig or animation files, so the compression ratio will be even better in the end. And since the same rig gets used for many animations its cost will effectively be distributed over all of them.

That’s it from me for now! I hope you found this interesting. As always, please feel free to ask questions and let me know what you guys think.

Share this post


Link to post
Share on other sites

Wow, that's a smooth walk cycle!

Can't wait to see this tech in action!

Share this post


Link to post
Share on other sites

... and I'm still trying to teach myself the flipbook method. A very interesting post.

Share this post


Link to post
Share on other sites

Thanks for the post! Its very interesting to read details like this.

I think you linked the wrong redbot link at the very end of the post, its not animating like the others :)

Share this post


Link to post
Share on other sites

cool. :)

just curious how the animation system would handle transitions between different versions of the character (e.g. when the character is drawn from a different perspective to walk toward or away from the screen). Are the "puppets" just swapped out between two frames or would you use a number of non-skeletal animation frames for the transition or some kind of blending magic?

ps

tu-dresden shoutout

(couldnt resist :))

Share this post


Link to post
Share on other sites

Very interesting approach, but I have a question: what if the artists need to make an animation that is supposed to look more organic than what a skeleton can do? Like, suppose the Lumberjack eats something big and his stomach instantly expands. How can that be done using the skeleton? Around the stomach area there are only nodes on the spine, so apparently moving the nodes can't make the stomach puff up. Also resizing the torso part might not look so good.

So would flipbooks be applied in these special cases for this game? (special cases are common in adventure games, I imagine)

I think, for example, Machinarium doesn't use skeletons, it uses sprites (flipbooks), because some movements of the robots are really organic, they stretch in many ways. I find it really interesting to give an organic feeling to the animations. Skeletons give a really smooth animation, but sprites open many opportunities to the animators, so the end result is normally really expressive and organic. I mean, how important is it to have 30 frames per second?

Share this post


Link to post
Share on other sites

I love these posts and this amount of insight. Thanks a ton, and keep up the great work!

Share this post


Link to post
Share on other sites

Very cool! Thanks for the update.

A couple questions because I don't know/understand enough and want to learn more!

Is there a reason there's 3 joints in the stomach for the skeleton?

Also the head seems to be connected through a center point at the chin then down the rest of the body. Is there a reason it doesn't have a central point in middle of the head? Does that stuff matter in a virtual skeleton? Or is that just how it all landed to put this update together?

Share this post


Link to post
Share on other sites

In the last art update, Lee said that you guys were still figuring out how to do character rim highlighting in-game... what would be the pros/cons of a bump-map layer for this purpose?

Also, all of a sudden, I have an irrational desire to make a 2.5D skeletal animation system in HTML/jQuery.

Share this post


Link to post
Share on other sites

hmm, the walking animation for the extreme lumberjack still looks a bit wierd. i suppose its that part with the knee popping forward on the leg that has just landed. he looks almost like hes going to sit there.

anyways. 30 fps isnt smooth :PPPP. you have to add 10-20, ideally 60 to make it smooth.

Share this post


Link to post
Share on other sites
Thanks for the post! Its very interesting to read details like this.

I think you linked the wrong redbot link at the very end of the post, its not animating like the others :)

There seems to be a problem with Chrome were it doesn't animate Redbot for some reason. We are trying to figure out if there is a fix, but in the meantime you can simply check the page in Firefox or IE.

Share this post


Link to post
Share on other sites
Thanks for the post! Its very interesting to read details like this.

I think you linked the wrong redbot link at the very end of the post, its not animating like the others :)

There seems to be a problem with Chrome were it doesn't animate Redbot for some reason. We are trying to figure out if there is a fix, but in the meantime you can simply check the page in Firefox or IE.

You can also view the image directly--Chrome doesn't seem to have a problem with that, for whatever reason: http://www.doublefine.com/assets/forum_updates/programming4/5_3.gif

Share this post


Link to post
Share on other sites
cool. :)

just curious how the animation system would handle transitions between different versions of the character (e.g. when the character is drawn from a different perspective to walk toward or away from the screen). Are the "puppets" just swapped out between two frames or would you use a number of non-skeletal animation frames for the transition or some kind of blending magic?

ps

tu-dresden shoutout

(couldnt resist :))

That is a very good question. Right now we are simply replacing the rig, so there is a pop (blending wouldn't really work in this case). The animators will have to figure out whether or not that looks good enough. We could try to connect transitions like these with a special animation, but that might be a lot of work.

Share this post


Link to post
Share on other sites
Very interesting approach, but I have a question: what if the artists need to make an animation that is supposed to look more organic than what a skeleton can do? Like, suppose the Lumberjack eats something big and his stomach instantly expands. How can that be done using the skeleton? Around the stomach area there are only nodes on the spine, so apparently moving the nodes can't make the stomach puff up. Also resizing the torso part might not look so good.

So would flipbooks be applied in these special cases for this game? (special cases are common in adventure games, I imagine)

I think, for example, Machinarium doesn't use skeletons, it uses sprites (flipbooks), because some movements of the robots are really organic, they stretch in many ways. I find it really interesting to give an organic feeling to the animations. Skeletons give a really smooth animation, but sprites open many opportunities to the animators, so the end result is normally really expressive and organic. I mean, how important is it to have 30 frames per second?

You are absolutely right, some effects are notoriously difficult to animate using skeletal animations. Our plan right now is to use flipbook animations for these cases. Given the scope of the game and the manpower we have to make it my feeling is that this won't happen too often, but we'll see. I'm sure the artists will talk about it in more detail at some point in the future.

Share this post


Link to post
Share on other sites
Very cool! Thanks for the update.

A couple questions because I don't know/understand enough and want to learn more!

Is there a reason there's 3 joints in the stomach for the skeleton?

Also the head seems to be connected through a center point at the chin then down the rest of the body. Is there a reason it doesn't have a central point in middle of the head? Does that stuff matter in a virtual skeleton? Or is that just how it all landed to put this update together?

I'm not an animator so I can't really answer this question from a creative perspective, but technically it doesn't matter where the joints are. For example the head geometry is attached to the neck joint, so when it rotates the geometry will follow. Generally the animator use the minimal set of bones they need for an expressive character and that may or may not follow the human anatomy... :-)

Share this post


Link to post
Share on other sites
In the last art update, Lee said that you guys were still figuring out how to do character rim highlighting in-game... what would be the pros/cons of a bump-map layer for this purpose?

Excellent comment Hoatzin. We already thought about it and having actual geometry (maybe even including normals) will make it much much easier to achieve good looking rim lighting.

Share this post


Link to post
Share on other sites
hmm, the walking animation for the extreme lumberjack still looks a bit wierd. i suppose its that part with the knee popping forward on the leg that has just landed. he looks almost like hes going to sit there.

anyways. 30 fps isnt smooth :PPPP. you have to add 10-20, ideally 60 to make it smooth.

It is pretty smooth in the game, but we had to drop a couple of frames when creating the GIF. So some artifacts come from that. :-)

Share this post


Link to post
Share on other sites

One way of saving memory with the old "flipbook" style of animation was that you would just do a single series of images for the character walking in one direction---say, right---and then when you wanted the player to walk left, the game would just flip the images horizontally, so you get two walk animations stored as a single image set.

The skeleton animations are not based on image sets, but I assume you can still just create a single walk animation and then flip it or rotate it?

Share this post


Link to post
Share on other sites

´Don't you think he looks too much like a pupett when mooving? Or is it simply done rapidly to show how it works.

Share this post


Link to post
Share on other sites

Hey, have you seen a Kickstarter project called Spriter? It's a program to make animated sprites which doesn't consume that much memory.

Here it is: http://www.kickstarter.com/projects/539087245/spriter

Do you know what technology does this use to consume less memory? I didn't understand much (they don't explain that very well in the video), but in the end I backed the project.

Share this post


Link to post
Share on other sites

This is a neat way of animating. It feels a lot livelier than standard after effects puppet animation, the slight 3D feel makes it look like it's been built out of cardboard or something.

hmm, the walking animation for the extreme lumberjack still looks a bit wierd. i suppose its that part with the knee popping forward on the leg that has just landed. he looks almost like hes going to sit there.

anyways. 30 fps isnt smooth :PPPP. you have to add 10-20, ideally 60 to make it smooth.

The human eye can only take in around 24 frames per second so going much higher than 30 is pretty much a waste of time. I know they tend to use higher frame rates for a lot of 3D games but that's just to compensate for frames that might get dropped during live rendering as far as I understand it.

Share this post


Link to post
Share on other sites

Mr. Hipster Lumberjack is looking much more expressive this time around, I like his slight slouch and expression, but he sort of resembles a paper cutout. I assume that's pretty much because this is only demoing the animation system and not much effort was put into cleaning it up and no lighting/effects were applied. I assume, for the final product, that the joints will be nicely cleaned up and not quite as visible, correct?

Edit: The difference between 30 and 60 FPS isn't huge, but it is noticeable. This website shows examples of an animation at different framerates: http://boallen.com/fps-compare.html

Ultra-smooth animation wouldn't be necessary for an adventure game, but more fast-paced action-oriented titles do benefit from higher FPS.

Share this post


Link to post
Share on other sites
hmm, the walking animation for the extreme lumberjack still looks a bit wierd. i suppose its that part with the knee popping forward on the leg that has just landed. he looks almost like hes going to sit there.

anyways. 30 fps isnt smooth :PPPP. you have to add 10-20, ideally 60 to make it smooth.

It is pretty smooth in the game, but we had to drop a couple of frames when creating the GIF. So some artifacts come from that. :-)

i was talking about 30 fps in general. of course, the gif is going to look like a rotoscope flip through by a 100 year old person and not an upcoming video game designer who will become famous and create one of the best modern day game trilogies :). 30 fps isnt really what "smooth" is.

but that is just a minor thing. i wont scream "shut up ang give me back my money!", because of that :))).

Share this post


Link to post
Share on other sites

I like mr. Lumberjack much more now, because the animation has made him so expressive.

And very much thank you for this update!

Share this post


Link to post
Share on other sites
Sign in to follow this  
Followers 0