Stardate 20031010.1830

(On Screen): When it comes to art, I have no talent whatever. I can't draw to save my life. So one of the reasons I'm fascinated by computer 3D rendering is that the computer wields the pen, thus relieving me of that burden. I'm also fascinated by the underlying technology, because the image which is created is only as good as the algorithm which produces it. And none of that happens by accident, and a lot of it is preposterously difficult.

I recently bought a copy of Carrerra and I'm working on trying to learn it, but I've probably spent more time working with Poser than anything else. The most recent revision of Poser included a new simulator for cloth, which I've had a lot of fun with. Before that, clothing for Poser figures were actually articulated figures in their own right with body parts corresponding to the parts of the figure wearing them. A woman's skirt actually had an abdomen section, a hip section, and two thigh sections. With the new cloth simulation, what happens is that the cloth adapts to the shape of the figure, and the result is a lot better looking.

The most recent version of Poser also included an entirely new renderer which they call Firefly and in many ways it's more powerful than the previous one. The new renderer uses raytracing, and it can produce better shadows. It is also capable of handling reflection and refraction. But there are curious lapses: Mirrors correctly reflect for the camera, but they don't reflect light sources. In cases where that should take place, the resulting image can be rather jarring, though it may take a while to figure out why it doesn't look right. (One of the first scenes I created using a mirror depended on the mirror for illumination, and it took me a while to figure out why it wasn't coming out the way I thought it should.)

The original renderer has an algorithm for shadows but it doesn't work very well. And there's a lot else it doesn't do well, or doesn't even attempt to do.

Poser's fundamental method for describing human figures is quite primitive. The figure is constructed of body parts and each of those is described with polygons. Each body part contains a "bone" which has length but no thickness, and every joint between body parts connects two bones, with a "normal bend", a "sideways bend" which is 90° away, and a "twist" (which distorts the entire body part rotationally). Most of the figures I've seen divide the body trunk into three pieces (the chest, the abdomen, and the hip) with only two flex points overall. The actual human spine doesn't act that way.

Poser also doesn't handle the joints all that well. When the figure is created, the model maker will include info about how the polys bend and stretch when flexing. Depending on the skill of the model maker, some figures are better than others, but when it comes to knees and elbows, Poser is notoriously bad, and it isn't possible for the model maker to overcome Poser's flaws. It also tends to handle the shoulders extremely badly, and a figure whose arms are raised over its head will usually look extremely distorted. Here's an image I just put together which demonstrates most of the kinds of things that users of Poser soon learn to avoid at all cost:

No one would ever mistake this for a photograph, or an actual woman. The "conforming" right shoe didn't bend the way her toes did. The knees and elbows have crinkled, as has the skirt, causing some polys to turn over and be rendered from behind (which looks black). The shoulder looks terrible, and the strongly twisted upper arms don't look right. The hair looks like a solid object (which it is). The halter-top hasn't changed shape the same way as she has, and in a couple of places her polys are on top of the halter-top's polys. The lighting and shadows only vaguely resemble real light and real shadows. (And you don't want to see what her right leg did to the polys of the hip under that skirt.)

The new renderer handles shadows much better, and the "cloth" algorithm makes clothing look far more realistic, and a lot of material textures look better, but it doesn't solve the fundamental problem with how Poser describes human figures. Poser treats them as if they had exoskeletons, like arthropods.

To Poser, the skin is what is real. There's nothing inside it. But that's not how we non-arthropods actually work.

Realistically simulating a human figure even in static posing is a difficult problem. (Doing so in motion is even more exciting.) The skin conforms to what's inside, and what's inside changes shape in a variety of extremely complex ways.

Take the biceps muscle, for example. It lays on top of the humerus (the upper arm bone) and is the flexor for the elbow. Many muscles come in pairs, rigged to oppose one another, and in many such cases they're called flexors and extensors. The triceps muscle is on the opposite side of the humerus and it is the extensor for the elbow. (And I just got my money's worth out of the anatomy segment of the Zoology course I took in college.)

When the elbow bends, the biceps becomes shorter, and it bunches up and bulges. But it has a different shape if it's actively working than it does if it is relaxed, even with the elbow at the same angle. It also has a different shape depending on how the arm is turned.

Our shoulders and hips are ball-joints, capable of bending in many directions and of rotating. When we twist our arms, sometimes part of that twist happens in the shoulder. And though the humerus is stiff and can be treated as if it doesn't flex, when it turns the lower anchor point of the biceps moves relative to the upper one. In the picture above, the woman's right humerus is rotated 90° inward, while her left humerus is rotated 60° the opposite direction (which is probably a greater range of motion that is actually physically possible). That should make the shapes of the biceps muscles entirely different, but Poser doesn't know from "biceps".

Calculating the true shape of the shoulder requires knowing the size and shape and anchorage points of at least three major muscles (biceps, triceps and deltoid), how much load the muscles are sustaining, how thick the fat and skin is over those muscles, how those muscles change as they flex and as a function of load, and how the fat and skin changes, the shape of the upper part of the humerus and how it joins the other bones, and where the other bones are and what their shapes are, and even that doesn't complete the list. (Which is why you don't need to send me any mail telling me what I left out.)

I was wandering the web today and followed a link to this site, which is in Japan. Its owner is as talented in computer rendering as I am not. It's not just that he is better at scene composition, camera placement, lighting, inspiration and artistry, though he is.

He designs his own human figures, mostly young women. And they actually look like real people. If someone encountered one of his pictures and wasn't alert, they might think it was a photograph.

Of course, they're idealized. They look like young women, but also like particularly beautiful ones. Since he seems to be using them commercially, that's completely reasonable. But they don't look generic. In fact, they look as if they were modeled from specific real young women.

There was one picture in particular in this gallery which I won't fully reproduce here so as to honor his copyright. But I did want to reproduce a small piece of it, because of what it demonstrates about the model and the tool:

The Futaba model is wearing a string bikini and is bending forward towards the camera. She has quite large breasts, and the limiting factor on "sag" is the bikini top, supported by the upper strap around her neck.

And that's damned difficult to do. It's easy for nature, but preposterously hard for a computer program. The calculations involved have got to be fiendish.

The edge of the bra cup is pushing into the breast tissue, and above that other tissue is sagging down and out around it, but only a bit. Skin is flexible, but not infinitely so (and a good thing, too). Adipose tissue is soft and even more flexible than skin, but it isn't liquid. On the other hand, the bra strap is not making an equivalent dent on the collar bone. All of which means that the underlying model is identifying not only what Futaba is shaped like, but also what she's made of, because it's modifying her shape based on the effects of gravity and her clothing. It is identifying her breast tissue as being soft and flexible, while identifying the collar bone as hard and unyielding. The program has figured out how much the breast tissue wants to sag, simulated the forces involved between breast and bra (which flexes easily but doesn't stretch) and between adipose tissue and skin, and has performed an iterative "brute force" calculation which ultimately yielded a shape in that area which looks completely reasonable. It doesn't jar the viewer as being unrealistic and flawed, the way my picture does.

Two of his pictures for "Hiroki" confirm that the program is handling soft tissue properly. The breasts of a large-breasted woman who is laying on her back are not shaped as they are when she's standing or leaning forward, and there's a picture of Hiroki supine and another where she's leaning forward, and her breasts are indeed shaped differently, as they should be.

I knew that such things had finally become possible in computer rendering, if you had enough compute power. The Final Fantasy movie used extremely realistic computer rendering. I haven't seen that film, so I don't know if their system actually got to the point of properly handling tissues underneath skin (such things as muscle bulges). Regardless, their stuff didn't scale down to the level where one guy working alone could reasonably use it. Their models were immensely complicated, involving thousands of polys, and they had a huge rendering farm. And their software was proprietary.

But it looks as if packages with that kind of capability are now available for desktop computers, and it looks as if one extremely talented guy (as Oishi-san clearly is), working alone, can create models which are so realistic that they could potentially fool an unwary viewer into thinking they were photographs. And it didn't require him to drown in polys, either. One of the pages has a picture showing the basic Futaba mesh, and it's less complicated than the Poser figure I used above (for all the good that did Poser).

I wish I knew what package Oishi-san has been using. I wrote to him this afternoon to ask him, but there's no way of knowing whether he knows enough English to be able to read my fawning letter, or whether he might be inclined to answer it.

But maybe he'll notice all the refer traffic I'm about to send him! (Not that that's the reason I'm writing this post, of course.)

Update: Reader Jack can read Japanese, and he says he found a link on Oishi-san's site pointing to this package: Shade 6. That's probably what's being used.

William writes in to say that there isn't actually anything available that can do what I described above, and suggests that the shape was manually manipulated. Unless Oishi-san writes back, there's really no way to know how he did it.

Update 20031013: Roger L. Simon has an interesting take on these kinds of tools.