1 The idea
Inference processes each frame on its own; the model doesn’t know how the previous frame came out. So results can flicker from frame to frame even when each still looks fine, and the model may touch pixels it shouldn’t.
Your QC tools: output − input shows everything the model changed, output − previous frame shows flicker, and a graph of the fixed area’s mean shows which frame jumps. Push the gain to see clearly.
2 Hands-on
Task: find the 3 frames that fail QC and flag exactly those three
There is no real model in this lesson: the result is simulated with 3 common kinds of failure
3 Deep dive
Why ML output needs its own QC
Inference processes each frame separately with no memory across frames, so stability over time depends on how well the model was trained. A frame whose input differs slightly can get a noticeably different result.
Three problems are common: incomplete fixes on some frames, flicker, and changes where nothing should change. All three are hard to see on a still.
| Problem | Tool that catches it |
|---|---|
| Incomplete fix | Output at high gain, output − input |
| Flicker | Output − previous frame, plot over time, playback |
| Changes elsewhere | Output − input over the whole frame |
| Missing grain in the fix | Output − previous frame (that area goes still) |
Reading a difference view
A good output − input shows only the shape of what you meant to remove, with black everywhere else. Marks elsewhere mean the model touched those areas; a shape that is fainter on some frames means those frames were only partly fixed.
Differences are often tiny, so push the Viewer gain or multiply the result to see them. Showing positive and negative in different colours tells you brighter from darker.
A good difference = black everywhere except where you meant to fix
Catching flicker over time
Flicker is change over time, so compare across frames: play it at real speed, view the difference with the previous frame, or measure the fixed area’s average on every frame and plot it. The frames that jump off the line are the ones to fix.
- 1Play it through
- 2Output − previous frame
- 3Plot the average over time
- 4Jumping frames → fix
4 In real work
In Nuke
- A Merge set to difference (or minus to keep the sign) between the Inference result and the input plate, with the Viewer gain pushed up.
- Frame-to-frame difference: a TimeOffset of +1 on the Inference result (which gives the previous frame), then a Merge (difference) against the un-offset result.
- Plot over time: a CurveTool set to Avg Intensities over the fixed area, then read the curve in the Curve Editor.
- Limit the result to the fixed area with a Keymix and a mask, and regrain before delivery.
Common mistakes → the fix
- ✗ QC from stills, one frame at a time✓ Play it through and view frame-to-frame differences; flicker only shows over time.
- ✗ Checking only the fixed area✓ View output − input over the whole frame; the model may touch other areas.
- ✗ Delivering without regraining the model’s fill✓ A smooth area looks unnaturally still; add grain to match the plate.
- ✗ Assuming a low loss means every frame passes (see AI2)✓ The loss covers the dataset only; check the whole shot yourself.
Before you send the shot
- Output − input checked over the whole frame: nothing outside the fixed area
- Played at real speed with no flicker
- The fixed area’s average plotted over time: no jumping frames
- Grain in the fixed area matches the plate
- Failed frames fixed by hand or fed back into the dataset
5 Check yourself
What catches flicker in ML output best?
🔒 Answer the question above and this card opens
1. Output − input shows a faint square on the ground, far from the removed marker. What does it mean?
2. Plotting the fixed area’s average, one frame sits clearly above the rest, while the same area on the input plate stays steady. What is the problem?
Where the numbers come from: A simulated result: we planted 3 faults ourselves (marker left, the fill brighter on one frame, damage outside the fixed area) on a crop of the Blender plate with code-made grain. Nothing is trained here. That Inference works frame by frame comes from the CopyCat guide in our knowledge library.