1 The idea
CopyCat is supervised learning: you give it pairs of Input and Ground Truth (the correct result you made by hand), and it adjusts its internal numbers until its output gets as close to the Ground Truth as it can. It doesn’t know what is in the picture; it only learns “this comes in, that goes out”.
Here is the smallest possible model: 12 numbers (a 3×3 matrix plus a bias) learning from the before/after pixels in 3 boxes. Every step it measures the loss (the mean squared error) and nudges the numbers in the direction that lowers it. That is gradient descent.
out = M · rgb + b · loss = mean((out − target)²) · w ← w − lr · ∂loss/∂w2 Hands-on
Task: train until the loss is below 2×10⁻⁶, then check how well the model works on the whole frame
Try a learning rate of 2.9 or more: steps that long make the loss explode instead of falling
3 Deep dive
One training step
Every supervised model runs the same loop: feed the input, get an output, compare it with the target to get a loss, work out which way each number should move to lower the loss (the gradient), then move a little, scaled by the learning rate.
The lesson has 12 numbers. CopyCat has vastly more and learns from image crops, but the loop is the same.
- 1Input
- 2Model → output
- 3Compare with Ground Truth = loss
- 4Gradient
- 5Nudge the numbers
Ground Truth is the quality ceiling
The model only tries to copy the Ground Truth. If the roto or paint in your example frames is off in the same way every time, edges a little too tight or a smudge left in, the model learns that too and repeats it on every frame. Errors that change randomly from frame to frame, such as chattering edges, tend to come out as soft or unstable edges instead.
ML doesn’t turn poor work into good work; it just produces more of the same, faster. Time spent making the Ground Truth clean is the best time you spend.
Make the Ground Truth as good as the final delivery, not “good enough”
Learning rate: how long a step
Too low a learning rate and the loss falls very slowly; too high and each step overshoots until it explodes. In the lesson that happens around 2.9 because it is a small linear problem; real tools use completely different values.
| Loss graph shows | Likely means |
|---|---|
| Falls steadily, then flattens | Normal, healthy training |
| Barely moves | Steps too small, or the task is too hard for the model |
| Jumps around or shoots up | Steps too big, or bad data |
From 12 numbers to a .cat file
Training produces a set of numbers (the weights). In the lesson you see all 12. In Nuke, CopyCat writes them to a .cat file, which the Inference node then runs over the whole sequence.
PyTorch models trained outside Nuke can be converted to .cat with CatFileCreator, and the Cattery is a library of ready-made models already converted.
4 In real work
In Nuke
- Prepare pairs: the plate goes into CopyCat’s Input, the hand-made frames (roto/paint/grade) into its Ground Truth.
- Use AppendClip to gather the example frames into one clip, on both the Input and Ground Truth sides, in exactly the same order.
- Set CopyCat’s output folder and start training; watch the loss graph and contact sheets (random crops of input, result and Ground Truth; see AI3) as it runs.
- Load the resulting .cat into an Inference node on the full plate.
- Training needs NukeX or Nuke Studio; Inference runs in Nuke.
Common mistakes → the fix
- ✗ Input and Ground Truth don’t line up (offset frames, different sizes, wrong order)✓ Check each pair matches frame for frame, size and alignment before you train.
- ✗ Making a rough Ground Truth and hoping the model cleans it up✓ The model copies the roughness too. Make the examples delivery quality.
- ✗ Calling it done because the loss went down (see AI2)✓ The loss only says it fits the examples; look at results on other frames too.
Before you send the shot
- Every Input/Ground Truth pair matches in frame, size and position
- Ground Truth is delivery quality
- The loss graph falls and doesn’t explode
- Run Inference on frames outside the dataset and check them by eye
5 Check yourself
What does CopyCat learn from?
🔒 Answer the question above and this card opens
1. The roto in your example frames is consistently about 2 px too tight. What will the model produce?
2. The loss shoots up and swings wildly from the start. What do you try first?
Where the numbers come from: A 3×3 matrix + bias trained with real gradient descent on 192 pixel pairs from the plate. The “after” grade is one we set (saturation 1.25 with Rec.709 luma, gain 1.15/1.02/0.85, offset (add) 0.01). The point where the loss explodes (around learning rate 2.9) belongs to this example only. Teaching version: real CopyCat trains a convolutional neural network on image crops inside Nuke.