Home · Lab · CG Multipass & Relighting

CG8 Holdouts and merge order

Whatever is nearer the camera goes on later

CG Multipass & Relighting · ~7 min · Free

1 The idea

In over, A covers B, so layers stack from far to near. The orange pole in the plate is 7.0 m from the camera and the robot 9.0 m. When something in the plate is closer than the CG like this, the CG must be cut away there (a holdout) or it floats in front.

Reorder the layers with the ↑/↓ buttons, or switch the robot holdout between “none”, “stencil by the pole matte” and “3D holdout render”. Watch the order line and the error in the pole area against Blender’s render of the whole scene. At the start the robot is in front (last in the list), so it covers the pole.

over = A + B(1 − a)

2 Hands-on

Task: put the robot behind the pole, like Blender’s render of the whole scene, with the error in the pole area under the goal in the panel.

In the list the top item is at the back and the bottom one in front; in the order line the left is the back. Ask yourself what is nearest the camera. Try moving the pole to the front, and try a holdout.

SkyVFX Lab · CG8interactive · 1001

3 Deep dive

A covers B: order means depth

over is A + B·(1 − a). Where A’s alpha is 1, B is gone completely. So the layer merged later always reads as nearer the camera, wherever it really is in the scene.

Before wiring nodes, read the scene from far to near and stack the Merges that way. Here the plate is farthest, the robot is in the middle and the pole is nearest.

  1. 1Plate (farthest)
  2. 2Robot over
  3. 3Pole cutout over (nearest)

Two ways: cut the CG, or put the pole back on top

stencil is B·(1 − a), with the pole matte as A. The robot is cut wherever the matte is white; over the plate, the real pole shows through the hole.

The other way cuts the pole out of the plate with the same matte and puts it over everything, last in the stack. Both rely on the pole matte: the edge is only as good as the matte.

MethodWhat it usesWatch for
Stencil the CGThe pole matte as A of a stencil MergeAn inverted matte, or the robot’s alpha by mistake
Pole back on topThe plate cut by the pole matte, premultipliedIt must stay last in the stack

Where a holdout can come from

Plate objects usually get roto. If both sides have depth you can compare depths; if you have the occluder’s geometry the renderer can cut it; with deep, order stops mattering.

Z depth stores only the front surface per pixel. Deep merges per sample by depth, so it needs no fixed order and no re-render, which makes it the robust choice for fur, volumes and motion blur.

Holdout sourceGood for
RotoSolid plate objects like the pole, drawn and tracked by hand
ZMerge (depth compare)Layers that both carry depth; picks by the depth channel you set
3D holdout renderWhen the occluder’s geometry exists and a re-render is acceptable
Deep (DeepHoldout2)Fur, smoke, volumes, motion blur; no re-render

Hold out every layer, with a soft edge

If the element carries other layers, such as passes for a later glow or relight, hold out all of them, not just rgba. A layer missing from the foreground element is not occluded, so add it with AddChannels first. Passes added after the holdout skip the cut too.

A hard, aliased matte leaves a thin fringe of robot around the pole. The matte edge should be as soft as the pole’s edge in the plate.

Step through every layer in the pole area: each one you’ll use later must have the same hole.

4 In real work

In Nuke

  1. Roto: draw the pole matte on the plate and track it to the camera.
  2. Merge2 (operation stencil): A = pole matte, B = robot; then Merge2 (over) the robot onto the plate.
  3. Or: Copy the pole matte into the plate’s alpha → Premult → Merge2 (over) as the last layer.
  4. ZMerge: when both sides carry depth, set the depth channel and use depth that isn’t premultiplied.
  5. Deep: DeepHoldout2 cuts the element by the scene’s deep, then merge it in 2D.
  6. AddChannels: add any missing layers before the holdout.

Common mistakes → the fix

  • ✗ Merging the CG last, over plate objects that are nearer✓ Stack back to front with nearer things later, or give the CG a holdout.
  • ✗ Stencilling with the wrong matte: inverted, or the robot’s own alpha✓ Use the pole matte and view it first: it should be white on the pole.
  • ✗ A hard, aliased holdout matte that leaves a fringe around the pole✓ Make the matte edge as soft as the pole’s edge in the plate.
  • ✗ Holding out the beauty but not the other layers riding with it✓ Hold out every layer, and add missing ones with AddChannels.
  • ✗ Using depth that is premultiplied at the edges for a depth holdout✓ Use raw depth: not premultiplied, not graded.

Before you send the shot

  • The Merge order runs far to near, matching the scene.
  • Wiping against the plate around the pole shows no robot over the pole.
  • With gain up, no thin robot fringe shows around the pole.
  • The pole matte follows the pole for the whole shot.
  • Every layer used downstream is held out the same way.

5 Check yourself

One question to check yourself

The robot floats over a plate pole that is nearer the camera. How do you fix it?

What you learned

🔒 Answer the question above and this card opens

More questions

1. Without a stencil, which stack (bottom to top, B → A) puts the robot behind the pole?

2. Motion-blurred smoke weaves in and out of the robot. What should the holdout use?

Where the numbers come from: Every picture is a Blender 5.2 Cycles render of our own desert and robot (tools/assets/passes_cg.py), computed in your browser. Renders: the plate with an orange pole we added, the pole’s matte, the robot with the pole as a 3D holdout, and the whole scene rendered together as the reference. The robot’s shadow comes from the shadow catcher. The error is measured only where the pole overlaps the robot, and the goal is measured live when the images load.