Attach a Graphic to a Moving Surface: Track It or Rebuild the Shot?

Choose the route from the motion the surface actually performs, not from what the tracker offers to do automatically. A card that only travels across the frame is a position problem. A card that also turns flat-side to flat-side is a rotation problem. A card turning away from the lens is a perspective problem, and perspective is not a larger serving of the first two. It is a different question, solved from different information.

The short version: four visible, stable, coplanar corners can carry a graphic through translation, rotation, scale and foreshortening at once. Fewer than four, and you are estimating. None, and you are guessing. Whether to repair the gap or rebuild the shot comes down to where the graphic is visible, not to where the track has a hole. Those are different frames more often than you'd expect, and separating them is most of the skill.

Two limits before anything else. The shot below is a designed demonstration, not a performed test: its geometry and frame budget are laid out so the reasoning can be checked, and no part of it has been tracked, repaired or timed. The tracking observation I'm relying on comes from Adobe's documentation of the motion-tracking workflow, which records that point and corner-pin approaches estimate different motion relationships, and that feature visibility, coplanarity and drift affect whether the result is usable. That describes a mechanism. It says nothing about your plate, and nothing here establishes that a finished commercial shot is achievable on a schedule.

Inspect the interval before choosing a tracker

The first pass is not tracking. It's classification.

Ask what the surface does. Translation only means every point on the surface moves by the same vector. Add rotation in the plane of the image or a uniform change of size and you need two or three more numbers. Add a turn away from the lens and the surface stops being rigid in the image: one edge stays wide while the other shortens. That last change cannot be produced by any amount of position, rotation or uniform scale, which is why the choice of route is settled before a single frame is solved.

Then separate this task from camera reconstruction. Solving a camera answers where the lens was. Solving a surface answers where the card is. They fail differently, they need different evidence, and a treatment test needs the second. If you find yourself building a point cloud, you have left the job.

Now take the passage apart frame by frame. Here is the demonstration's budget: 96 frames at 24 fps, four seconds. A card roughly 15 cm across is held in a hand, travels left to right, turns in the flat plane and then begins to yaw, and is crossed by the other hand, which passes closer to the lens than the card does.

Add those up and the shot divides into 60 frames with a full set of corners, 5 frames with two or three, 9 frames with one or none, and 22 frames with four again. That table is the whole argument, because it tells you which frames could possibly be solved and which ones will have to be invented.

Notice where the invention sits. Frames 66–74 are nine frames, 0.375 seconds and 9.4% of the shot, in which the card is largely or entirely behind the hand. The graphic sits on the card's right portion, so it is behind the hand too. For those frames the composite should hide the graphic. A track that has no idea where the card is during 66–74 is not a compositing failure, provided the graphic is genuinely covered and the pin lands correctly when the hand clears.

Frames 61–65 are the opposite. The hand is only partly across, the graphic's left edge is still visible, and two corners are already gone. Those five frames are not moot. They are the hardest five frames in the shot, and they are where a repair earns or loses its credibility.

The distinction to carry forward: a track gap inside a matte is free. A track gap at the edge of a matte is not.

Match the required motion to a bounded route

Three routes are available for this motion, and only one of them matches it.

Position only. One pair of numbers. It reproduces travel and nothing else. By frame 24 the card is about 5° off-square and the graphic, carried by position alone, is still perfectly square to the frame. Five degrees is not a subtle error; it is a graphic visibly rotating against the surface it's supposed to be printed on. If the graphic has a straight edge meant to run parallel to the card's edge, the tolerance is well under a degree and position-only tracking is dead within the first dozen frames.

Position, rotation and scale. Four numbers. This handles the first 60 frames' in-plane work, and it is genuinely sufficient for a great many shots. Scale belongs to the route's general reach rather than to anything this demonstration exercises: the card travels, turns in plane and yaws, but never approaches the lens. The route fails at the yaw. By frame 60 the card sits roughly 28° off-axis, which foreshortens its width by around 12%. On a card that fills 300 px, the far edge is about 36 px shorter than the near edge, and a uniform scale has no way to reproduce that: it can grow or shrink the graphic as a whole, but it cannot narrow one side more than the other. The error lands on the graphic's far corners, which is exactly where a person checking alignment looks first.

Four-corner pinning. Four points define a quadrilateral, and a perspective transform maps the graphic into it. Translation, in-plane rotation, uniform scale and foreshortening all fall out of the same solve, because the solve is for the plane rather than for a handful of motion values. This is the route the recorded documentation is pointing at when it distinguishes point approaches from corner-pin approaches: they estimate different relationships. Pinning isn't a more thorough version of point tracking. It answers a different question.

Two conditions come attached, and both are recorded in the source's caution about coplanarity. The surface must actually be flat. A card held in a hand bends. Four perfectly placed corners with a bowed middle still puts the graphic's centre on the wrong part of the card, and no amount of corner accuracy fixes it. Backing the card with something stiff is cheaper than repairing a bow in a composite. Second, the corners must be visible, which is what the frame budget above is really measuring.

One more decision belongs here rather than later: what the pin is attached to. Track the corners of the card, not a passing feature on the hand and not a highlight sliding across a table. A graphic pinned to the wrong reference will move beautifully and sit in the wrong place, and that failure is harder to see in motion than a slip.

Finally, set the tolerance before you track. What in the graphic reveals misalignment? A hard border parallel to a card edge shows a fraction of a degree. A soft blob shows several. Decide now, because it decides which of the three routes is sufficient and whether the repairs you're about to attempt are worth attempting.

Apply the data without losing the working controls

Duplicate the plate and leave the source untouched. Apply the result to an editable control — a transform, a corner pin, a null — and parent the graphic to that, rather than baking the graphic down in place.

The reason is not tidiness. The graphic will change. Moving it two centimetres on the card should be a re-render, not a re-track. A revisable control also lets you flip between the tracked result and a hand-adjusted alternative on the same frame, which is the only way to settle an argument about whether a two-pixel slip is real.

Then check placement separately from movement. A pin can follow the card flawlessly and still put the graphic in the wrong spot on it. Verify the placement on a frame where the card is flat and fully visible — near the start of the interval, not during the turn, where a placement error and a perspective error look alike.

Occlusion order is the next thing to get right. The hand's foreground matte goes above the graphic. If the graphic sits over everything, it rides across the hand, and that single failure gives away an attached element faster than any amount of edge softness. While you're there, decide what the graphic does about sharpness. A crisp overlay on a card blurred by its own motion at a 1/48-second shutter reads as a sticker laid on the picture. Either match the blur through the fast interval or keep the graphic's most legible feature out of it.

Inspect the frames that make the illusion fail

The first and last frames of an interval can both be perfect while frame 44 swims. This is the sense in which a convincing tracked frame is not a convincing track: a still proves that one solve landed, and says nothing about the other 95.

Step through the whole interval once, then go frame by frame in the doubtful regions. Scrub in both directions. A slip that hides moving forward often shows moving backward.

Look for three specific failures. First, sliding edges: the graphic's border creeping against a fixed feature of the card by a few pixels and then creeping back. That is corner-estimate jitter rather than a wrong route, and it is usually treated with smoothing, a tighter search region, or manual keys on the offending corner. Second, wrong occlusion: the graphic crossing the hand, or a hard graphic edge butting a soft hand edge. Third, blur that no longer belongs to the plate — the graphic sharper or softer than the surface it sits on.

Watch for drift across the whole 96 frames. Small per-frame errors accumulate, and the documentation names drift as a limit on usable results, not as a curiosity. Give the last third of the interval as much attention as the first.

Then check the two occlusion boundaries on their own. Frame 60 is the last frame with a full set of corners and a visible graphic. Frame 75 is the first re-emerged frame. Solve frame 75 fresh and compare it with whatever the pin was doing at 74. If the two don't agree, the transition will pop, and a pop at a boundary is far more noticeable than a static error somewhere in the middle.

Only after all that, watch the interval at speed. Motion hides small errors and exposes large ones; stepping does the reverse. You need both readings, and neither one substitutes for the other.

Choose limited repair or a different demonstration

Three routes now, with what each costs and what each leaves unproved.

Bounded repair. Solve corners on every frame with three or four visible corners — that covers 1–63. For 64–70, estimate the hidden corners and cross-check the visible ones against the estimate rather than trusting either alone. For 71–74, interpolate. Then solve 75–96 from fresh corners and verify that the interpolation arriving at 74 hands off cleanly to the solve at 75. Finally, check frames 61–65 against the hand's own edge, because that is where a partially visible graphic and a partly visible card have to agree.

The cost is manual work on roughly fourteen frames plus the two boundaries. What it leaves unproved is whether the interpolated path matches what the card actually did in the nine hidden frames. That is untestable from this plate, and it also doesn't matter, because the graphic is behind the hand there. What the repair does establish is that attachment survives translation, in-plane rotation, yaw, a partial occlusion and a full one.

The test for legitimacy is blunt: does any frame with a wrong pin also have a visible graphic? If yes, the repair isn't finished. If no, the gap is inside a matte and you can stop.

Rebuild the shot. Re-block so the four corner regions stay clear of the hand. The hand can still cross the card; it just has to cross the middle and leave the corners exposed. If the corners stay visible, all four solve through the occlusion and the problem disappears. The cost is a take, and a slightly less sweeping gesture. If the plate already exists, the same logic applies to a re-shoot, with the honest caveat that a new path for the hand changes how the shot feels. The narrower promise: attachment through translation, rotation and yaw, with an occlusion the tracker never had to solve. It says nothing about a shot where the corners themselves disappear.

Simpler demonstration. Card travels across the frame, fronto-parallel, no turn, no hand. Position tracking alone is sufficient and can be verified in minutes. It proves that the graphic is attached to the surface rather than laid over the frame. It leaves unproved everything about perspective, everything about occlusion, and everything about the original proposal's look. If the treatment's argument is "the mark lives on the card," this may carry it. If the argument is "the mark survives a real hand turning a real card," it doesn't, and it shouldn't be described as though it did.

Line them up: repair preserves the shot's ambition and spends editing time. The re-block preserves the shot's look and spends a take. The simpler demonstration preserves neither and costs least. None of the three establishes final VFX feasibility, and a tracked composite's job here is to make a proposed frame discussable — not to certify that the finished shot is achievable.

Where this ends

Finish with one of two artifacts. Either a tracked interval you can play while naming its limits frame by frame, or a written decision to rebuild, with the reason and the replacement's narrower claim stated plainly.

A rejected track is a result. "The pin holds through the turn and drifts at the occlusion boundary at frame 74; the re-block keeps the corners clear" is a complete answer to the question the treatment asked. A single beautiful frame with no interval behind it is not an answer at all.

Frequently asked questions

How do you choose between position tracking, position-rotation-scale, and four-corner pinning?

Choose from the motion the surface actually performs. Translation only is a position problem. Adding in-plane rotation or uniform scale needs more numbers. A turn away from the lens is a perspective problem, not a larger serving of the first two. Four visible, stable, coplanar corners can carry translation, rotation, scale, and foreshortening at once. Fewer than four and you are estimating; none and you are guessing. The route is settled before a single frame is solved.

Why can position, rotation, and scale not handle a card that yaws?

Uniform scale can grow or shrink the graphic as a whole, but it cannot narrow one side more than the other. In the demonstration, by frame 60 the card sits roughly 28 degrees off-axis, foreshortening its width by around 12 percent. On a card that fills 300 px, the far edge is about 36 px shorter than the near edge. The error lands on the graphic's far corners, which is where a person checking alignment looks first.

When is a tracking gap not a compositing failure?

A track gap inside a matte is free; a track gap at the edge of a matte is not. In the article's demonstration, frames 66 to 74 put the graphic largely or entirely behind the hand, so the composite should hide it there. Frames 61 to 65 are the opposite: the hand is only partly across, the graphic's left edge is still visible, and two corners are already gone. Those five frames are the hardest, and they are where a repair earns or loses credibility. The test is blunt: does any frame with a wrong pin also have a visible graphic? If yes, the repair is not finished.

What should be checked after the tracking data is applied?

Duplicate the plate and leave the source untouched. Apply the result to an editable control, such as a transform, corner pin, or null, and parent the graphic to that rather than baking it down. Check placement separately from movement on a frame where the card is flat and fully visible. Put the hand's foreground matte above the graphic, and match the graphic's blur to the plate. Step through the whole interval, scrub both directions, and look for sliding edges, wrong occlusion, and blur that no longer belongs. Check the occlusion boundaries, especially frame 60 and frame 75, solving 75 fresh and comparing it with the pin at 74. Then watch at speed, because motion hides small errors and exposes large ones.

What are the three route choices and their limits?

Bounded repair solves corners on frames with three or four visible corners, estimates hidden corners, interpolates through the full cover, and solves the re-emerged frames fresh. It costs manual work and leaves unproved whether the interpolated path matches what the card did, but the graphic is behind the hand there. Rebuilding the shot re-blocks so the four corner regions stay clear of the hand; it preserves the look and costs a take, with the narrower promise that attachment survives translation, rotation, and yaw with an occlusion the tracker never had to solve. A simpler demonstration uses position tracking alone and proves only that the graphic is attached to the surface, leaving perspective, occlusion, and the original proposal's look unproved. None of the three establishes final VFX feasibility.

Further reading

• Prototype a Projection-Mapped Commercial Scene Without Promising the Installation — Read when four corners are your route: it explains how four point correspondences define a plane-to-plane map and why a bow, lens distortion or non-planar surface defeats it.
• Prepare a VFX Shot Turnover for a Commercial Pitch Test — Read when the tracked interval must survive a handoff; it turns frame ranges, handles and temporary treatments into notes a compositor can check without asking.