1 The idea
This Keyer is set very hard. The body is solid, but the thin antennas on the head and the soft edges are gone, just like hair or motion blur in real work.
This screen is uneven like a real stage (a hot spot, fabric seams, tape along the foot of the wall), but one well-set key still reaches the goal. On harder footage, studios split it into a soft key (keeps the edges) and a core key (fills the body), combined with max. See this lesson’s deep dive and K4 for details.
2 Hands-on in the mini compositor
Task: bring the fine detail back (edge error ≤ 0.163) while the whole matte stays accurate (frame error ≤ 1.1‰)
Lower the gain and loosen the clips a little at a time; watch the antennas in alpha
3 Deep dive
Why thin detail vanishes
An antenna thinner than a pixel covers only part of it, so that pixel’s colour is a mix of antenna and screen. If the antenna covers 30%, alpha should be about 0.3. Set clip black to 0.3 and that becomes 0: the antenna is gone.
Hair, defocused edges and motion blur are the same thing: partly transparent pixels. A key hard enough to make the body solid is usually too hard for the edges.
Split the job: one matte, one task
Real footage doesn’t rely on one key. A soft key keeps every bit of edge; a hard core key, shrunk a few pixels inside the edge, fills holes; a garbage matte removes everything that is neither subject nor screen; a holdout forces known-solid areas to 1.
- 1Soft keys
- 2Core (max)
- 3Garbage
- 4Holdout
| Matte | Job | Combined with |
|---|---|---|
| Soft / edge key | Edges, fine detail, transparency | Base |
| Core key | Fills holes, a few px inside the edge | Merge max |
| Garbage matte | Removes lights, stands, screen edges | Merge stencil / mask |
| Holdout | Areas that must be solid | Merge max |
Dark or bright edges: the fix order
Over = A + B(1 − a). With the Grade on alpha after the Premult, right before the Over, the robot’s rgb stays fixed and alpha only decides how much background gets added. A gamma below 1 there adds more background, so the edge brightens; above 1 darkens it. It keeps 0 and 1 where they are: fine detail may get a little fainter, but it is never wiped out the way an erode wipes it, so try it before any erode. (With the Grade before the Premult, the edge moves towards the background colour instead: see K5.)
Before fixing a dark edge, check two things: a double premult, and a despill that’s darker than the background. Only then edge extend just the edge that needs it, and erode as the last resort. This is the short version: despill is covered in K3, erode in K4, and the full fix order with alpha gamma in K5.
- 1Check premult / despill
- 2Alpha gamma (after Premult, before Over)
- 3Edge extend
- 4Erode (last)
Check edges against the plate
Wipe between plate and alpha: every soft area in the alpha must match something visible in the plate, like an antenna or motion blur. Soft where it should be solid means the core needs work; hard and chunky on the antenna means the core reaches too far.
Holes are the core’s job. Don’t crunch the edge key to fill them.
4 In real work
In Nuke
- Edge key: IBKColour + IBKGizmo, or a lightly tuned Keylight (Screen Gain near 1)
- Core key: a hard Primatte or Keylight → FilterErode (box) inwards → a little Blur → Merge (max) with the edge key (see K4)
- Garbage Roto → Merge (stencil or mask); holdout Roto → Merge (max)
- Different keys per area: Keymix through a soft Roto
- Grade (channels = alpha, gamma) after the Premult, right before the Merge over; EdgeExtend only where needed; Erode/FilterErode last
Common mistakes → the fix
- ✗ Crunching the edge key to fill holes in the body✓ Let a core key fill holes, combined with max
- ✗ Recombining two masked mattes with Over, leaving a grey seam✓ Use Keymix
- ✗ Eroding first to kill a dark edge, losing the antennas✓ Despill first, then try an alpha gamma right before the Over (see K5)
- ✗ A core matte that isn’t shrunk, making edges hard✓ Erode the core a few pixels inside and keep it off the antennas
- ✗ Edge extending everything, or motion-blurred edges✓ Only where needed; on motion blur, paint colour on the unpremultiplied image
Before you send the shot
- Antennas and soft edges are still in the alpha
- The robot body is solid 1, no holes
- The core stays inside the edge, away from fine detail
- No dark or bright rim over the real background
- Every soft area in the alpha has a reason in the plate
5 Check yourself
On a real shot, hair disappears in the key. Which approach?
🔒 Answer the question above and this card opens
1. An antenna pixel has alpha 0.25 before the clips. Clip black 0.2, clip white 1. What does it become?
2. A thin dark rim around the robot over a bright desert sky. The matte shape is right. What do you try first?
Where the numbers come from: The keyer here is a simple green-difference keyer (not Keylight or IBK). The despill formulas are the standard limit methods. Scores compare against the true matte rendered in Blender alongside the green-screen plate (the same robot and camera as the plate). The goals are measured on these renders, relative to the best key this Keyer can reach.