1 The idea
A flat surface, like the green screen on its pole, can be tracked as a plane with four corners even when the camera moves in 3D, because a plane’s motion is exactly a corner pin (a homography).
Drag the four corners onto the green screen’s corners on frame 1, set Track type to Planar (corner pin) and play. Watch the worst corner error over the shot, then switch back to One point (translate): however well it is placed, its corners still slide by about 4.9 px on a screen 103 px wide.
plane motion = corner pin (homography, 4 corners)2 Hands-on
Task: replace the screen with a planar track, corners on the screen’s corners, with the worst corner error over the whole shot at most 2.5 px.
While you drag a corner, a magnified view of it appears top left, and the arrow buttons nudge the selected corner by half a pixel. Read the worst error over the shot, not only this frame’s.
3 Deep dive
Why a plane is exactly a corner pin
With an ideal, distortion-free lens, however the camera turns or moves, a flat surface in two frames is related by a single homography, and four corners are enough to define it. That’s why a 2.5D or planar track works on monitors, phones, murals and floors, even under a 3D camera move. On a real plate with lens distortion, undistort first or the corners will drift.
A Transform is affine: parallel lines stay parallel, so it can’t do perspective. A CornerPin is projective, so it can squeeze one side of the screen narrower than the other, the way the camera really sees it.
| Type | What it can follow |
|---|---|
| One point | Translation only |
| Transform (affine) | Translate, rotate, scale; parallel lines stay parallel |
| CornerPin (projective) | All of the above, plus a plane’s perspective |
One point is not enough for a screen
As the camera moves, the screen’s shape in the frame changes: the near side grows, the far side shrinks. One point only says where the screen moved, not where each of its four corners went.
Here the screen is about 103 px wide on frame 1, and a perfectly placed one-point pin still misses at the corners by up to about 4.9 px over the shot. The centre may hold, but the sliding edges are what the eye catches.
If the centre holds and the corners slide, suspect perspective before blaming the tracker.
The screen replacement workflow
Bring the replacement in at its real resolution and set the CornerPin’s from corners to the input’s edges. Then put the to corners on the screen’s corners on the reference frame, the frame with no correction, before you export.
Spread the points as far apart as the surface allows. If they are clustered, any small mistrack is magnified at the far corners.
- 1Picture at its own resolution
- 2from = set to input
- 3Track four corners / planar
- 4Align on the reference frame
- 5Export CornerPin
- 6Check with a stabilise
Planar tracker options
The PlanarTracker’s motion type goes from Translation up to TSR + Shear and Perspective. Choose Perspective to follow a plane’s full keystone; a simpler type averages the area into fewer kinds of motion. A planar tracker is four corners, not a mesh, so it is for flat surfaces only, and the surface must itself be trackable.
| Export mode | What you get |
|---|---|
| CornerPin2D (relative) | from is filled in; no change on the reference frame |
| CornerPin2D (absolute) | from left empty; for an element at its own format |
| CornerPin2D (stabilize) | A CornerPin that holds the plane still |
| Tracker | A Tracker with four points |
Check with a stabilise, and track the right surface
Copy the CornerPin, flip it to stabilise, zoom in and scrub fast. The screen should sit dead still; if a corner moves, fix that corner. The stabilise adds one more resample, so it exaggerates jitter from filtering: make the final call on the un-stabilised result in motion.
Track the screen surface, not the reflections or the people passing in front of it, and don’t use a planar track on something that isn’t flat, such as the robot’s body or a hill with rocks at different depths.
A stabilised check is strict: if a corner drifts there, it drifts in the shot too.
4 In real work
In Nuke
- Read the replacement at its own resolution; don’t Reformat it to the plate format first.
- Tracker: track the screen’s four corners with T/R/S on and choose Export > CornerPin2D (use reference frame), or use the PlanarTracker (NukeX) and export a CornerPin2D (relative).
- CornerPin2D: in the from tab, set no animation, then set to input.
- Set the reference frame and line the corners up on that frame before exporting.
- Copy the CornerPin2D and turn on invert to stabilise, add a Reformat (resize none, slightly larger) so nothing is cropped, then scrub fast.
Common mistakes → the fix
- ✗ Bringing the replacement in at the plate format, so the from corners are wrong✓ Use the picture’s real resolution and set to input in the from tab.
- ✗ Lining the corners up on a frame that isn’t the reference frame✓ Set the reference frame first, then align on that frame.
- ✗ Tracking the reflections or the people in front of the screen✓ Track the screen surface itself and keep moving things on it out of the track.
- ✗ Using a planar track on something that isn’t flat✓ Keep planar for flat surfaces; with depth differences use a 2D or 3D track.
- ✗ Clustering the track points in the middle of the screen✓ Spread them as far apart as the surface allows.
Before you send the shot
- The replacement is at its own resolution and from matches the input’s edges.
- The corners sit on the screen on the reference frame.
- Stabilised on the pin and scrubbed fast, every corner holds still.
- Played through un-stabilised, the corners don’t slide anywhere in the shot.
- People or objects in front of the screen still sit in front of the new picture.
5 Check yourself
A one-point pin is placed exactly, yet the picture’s corners still slide as the camera moves. Why?
🔒 Answer the question above and this card opens
1. The replacement looks distorted from the first frame and doesn’t fill the screen. It had been reformatted to the plate format. What’s wrong?
2. Which of these suits a planar track best?
Where the numbers come from: A 48-frame handheld shot (640×360) rendered in Blender (Cycles) through our desert set, with a green screen on a pole and a drone flying across. Frame 1 is the still plate’s camera. Every true position comes from the scene’s own 3D (truth.json, dist.f32). The plane’s motion comes from the scene’s true screen corners (Blender camera), so the figure shows what a perfect planar track gives; a real planar tracker estimates the same corner pin from the picture and adds small errors. The test card is SkyVFX’s own; computed in your browser.