Brooklyn, New York
I’m operating mostly from intuition. I know which controls to futz with and generally I achieve what’s needed, but at this point I don’t know enough yet to make a comprehensive plan. There’s still a mighty learning curve to climb.
Are you tired of reading about rotoscoping yet? This will be the third week of roto work and there are probably several more yet to go – at least. For the red hallway, I’ve repaired about 6 of the 12 reverse scenes, the shots from the hallway into Christina’s cabin with the over-exposed window in the background. In last week’s update, I thought there were only five to fix – oops. Warning – technical descriptions ahead.
To review, there are three critical layers to this scene, each defined by one or more masks. Let’s start with the window and water. The problem in the original shot is that the water is overexposed, so it has to be replaced completely. The window is square in shape, partially covered by a venetian blind. I could get away with just a simple rectangle mask for this window, but I’ve added a horizontal sliver mask where one of the venetian blind slats was slightly askew. The idea is to get away from a strictly square window into a more haphazard and accidental shape. This gives the new window a stronger presence and integrates it into the rest of the shot.
Not only do the outline of the actors change as they move in front of the window, but light from the over-exposed windows spills onto the edges of their outlines. The actor masks have to cut in closer where there is excessive light spill, slicing off the bright edges.
One layer defines the new background (water) and another defines the foreground (clothing). That gives a consistent edge between the actor’s clothing and the water. However, the actors shoulder is not completely in front of the window, but is also passing in front of the window frame. Even though there is no problem with the window frame, it also has to also be replaced by a new image. Otherwise the original line of the actor’s clothing will change as it moves across the border between the window (new image) and frame (old image with old outline of clothing). The edges of the clothing that I cut away will reappear when shoulder is in front of the original window frame. So our third layer is a new window frame.
Here cut outs between the shoulder mask and the original image show how much the outline of the clothing changes, especially over the over-exposed water.
The camera doesn’t move in this shot but the various takes differ slightly in angle and zoom. I grabbed still images of the window frame from later on in each of the 5 takes, when no actors were standing in front of it, (thank god!)
Even when nothing is moving in a moving image, there is usually dancing grain as the pixels (or for film, bits of silver) quaver between frames. So when compositing a still image like the window frame, I have to add grain that matches the surrounding image.
The layers are stacked from foreground to background in a slightly different order than real life. Since the window layer’s mask doesn’t change shape, it’s placed in front of the window frame layer and enlarged slightly to cover the light spill that is on the edges of the window frame. The window frame is the transition from old image to new image. It’s mask shape must change to surround the entire shoulder mask while avoiding other unmasked parts of the actor’s body like hair and skin. On top of these two is the shoulder layer. These three layers are built up into a composition, which then becomes the foreground of the original scene.
Think of these layers as an irregular blob with new image in the middle and a bit of original image at the edges. When the blob is stuck on top of the original image, the edges disappear. There’s no more blob, either. It’s a spaceship landing in a cornfield, a kitten talking on the telephone, a bowl of live crickets and cream. The illusion is accomplished.
Each layer is created like an Eternal’s MNC or minimum necessary change (Isaac Asimov, The End of Eternity) – we alter only what we must to achieve the desired result. Building animated masks to isolate background and foreground (rotoscoping) can be incredibly time consuming and tedious work. You don’t want to roto more than you have to.
Which brings us to key frames. An airplane high in the sky moving from left to right at a constant speed could be defined by two keyframes. A ball thrown up in the air is a different story. The ball moves up slowing until it stops and then falls down going faster and faster until it hits the ground, all the while moving from left to right. I could still define the left to right movement with 2 keyframes, but the acceleration requires more information.
The movement of a human body is full of acceleration and de-acceleration. Body parts rotate and articulate while moving, changing their outlines. Rotoscoping requires an ability to recognize where the crucial changes are – where a moving hand has begun to de-accelerate or when a swinging arm has reached the top of it’s parabola. These are keyframes and once identified a mask can be made to follow the movement pretty nearly. So a lot of what I am doing is actually the opposite of traditional animation in that I am finding inherent keyframes rather than creating them from scratch.
Here’s a strip of images illustrating the masks changing over time to follow the movement of the actor and the de-acceleration as his head turns from left to right. Notice the fuzziness of the head in the top frames resulting from motion blur and the corresponding fuzzy edge of the mask. As the head finishes turning and becomes sharper, the mask is also becoming sharper. The head is partially masked because it passes in front of the window in earlier frames and also interacts with the coat mask in these frames. A better masking solution for these frames would have been to mask the jacket up to the skin only. That’s the MNC discussed above.
In July I joined fxphd.com which offers advanced production courses. I am way behind on the homework at this point, in fact I haven’t even finalized on which courses I want. I am leaning toward red production night and day, miniatures and perhaps a basic sound course.
Nothing like staying home all summer and looking at screens. You couldn’t pay me to do this by the way, it’s all about the vision! I’m about to miss the WHO farm bus which is going to Burning Man and points farther west for the next three months. Rats. They stopped by the Prospect Heights Community Farm today!
So where am I? How far away is the finish? I can’t really say exactly, but I am going to try and repair the entire red hallway by next week’s update, that’s about 25 shots in all, a good hunk of the first act and the very start of the second. I might know enough now to really crank. Next week I’ll take a shot at presenting an inventory of what’s left and project a schedule, though that will probably not be pretty. Keep your fingers crossed.
Until next week,
Dan Kelly