All work

Keyframes as a control surface

The one place in a generative video pipeline where an art director still has direct authority.

Year
2026
Role
Keyframe strategy, continuity testing

The ask

Get deliberate composition into a pipeline whose generation stage does not take composition notes.

Constraints

Constraints are what make this engineering rather than decoration.

No direct frame authoring
The generation stage cannot be told "put the subject here". Influence has to be indirect.
Continuity across boundaries
Anything anchored has to survive the seam where one segment meets the next.
Cheap to iterate
A keyframe strategy that requires a full render to evaluate is not a strategy.
Robust to bad candidates
The automatic keyframe choice is sometimes the worst frame in the shot. Assume it will be.

Where style lives

Claiming the whole pipeline would be less credible, not more.

PipelineMine to change (2)Not mine (3)
  1. 01

    Shot detection

    Sets the candidate boundaries. Its mistakes become my problem.

  2. 02

    Keyframe candidate scoring

    Which frame gets to define the shot. The decision everything downstream inherits.

  3. 03

    Keyframe treatment

    The one stage with genuine frame-level control.

  4. 04

    Interpolation / generation

    Fills between anchors. Respects them only as far as conditioning strength allows.

  5. 05

    Seam handling

    Where two anchored segments meet, and where continuity most often dies.

What broke

Four ways this failed before it worked.

  1. Failure 01

    The automatic keyframe is a motion-blurred frame

    Cause

    Frame scoring that optimises for sharpness or coverage will happily pick a frame mid-whip-pan. Every downstream decision then inherits a smeared anchor, and the whole segment is built on it.

    The lever

    Score candidates against composition and stability, not just sharpness — and always allow a fallback window.

    Evidence

    Anchor frame selected mid-motion, degrading the whole segment

  2. Failure 02

    Anchors are respected at the start and abandoned by the end

    Cause

    Conditioning influence decays over a segment's length. A composition that is honoured in the first second has quietly dissolved by the fourth, so short tests pass and real clips fail.

    The lever

    Shorten segments and re-anchor, rather than asking one anchor to hold longer than it can.

    AtlasThis failure class, in general

    Evidence missing

    This failure has no clip yet. A named failure with video evidence is far more convincing than one described in prose — add one.

  3. Failure 03

    Visible seam where segments meet

    Cause

    Two segments each internally consistent, anchored independently, produce a discontinuity precisely at the cut — the moment a viewer is already paying attention.

    The lever

    Overlap the generation windows and let them share frames rather than meeting at a hard boundary.

    Evidence

    Discontinuity at a segment boundary

The fix

The reframe: a keyframe is not a shortcut to skip generation work. It is the last stage where a compositional decision survives literally — where “the subject sits here, the light comes from there, this is the frame” is honoured exactly rather than interpreted.

Everything after it is negotiation. The generation stage will take an anchor as a strong suggestion and drift from it in proportion to how long it has to run and how hard the conditioning pushes. So the practical question stops being “what should this shot look like” and becomes “what is the most influential thing I can fix in place, and how long will it hold?”

Two changes followed from that:

  • Score candidates on stability and composition, not sharpness. A slightly softer frame with clean structure is a far better anchor than a sharp frame caught mid-motion. Sharpness is easy to measure, which is exactly why the naive scoring optimises for it.
  • Re-anchor more often than feels necessary. Anchor influence decays. If a segment is longer than its anchor’s reach, the back half is unanchored whether or not that was the plan — and short test clips will never show you this.

The part that transfers

The general lesson is about where to look for leverage. The interesting stage of a pipeline — the model — is usually the stage you control least. The boring stages around it are often where you have real authority and are not using it.

Auditing a pipeline for where influence is actually possible, before trying to improve the part that seems most important, has saved me more time than any other habit.

Result

Same source, same generation settings. The only difference is which frames were chosen as anchors and how they were treated.

Automatic anchorsScored anchors

Drag to compare. Arrow keys also work; space toggles playback.

Receipts

Usable-first-pass rate
TK — unfilled
Reduction in manual reselection
TK — unfilled

What I'd do differently

I treated keyframe selection as a preprocessing detail for far too long, because it sits before the interesting part of the pipeline. It turned out to be the single highest-leverage stage available to me, and it was leverage I already had — no model change, no new tooling, just a better decision about which frame gets to define a shot. Worth asking, on any pipeline: where do I already have authority that I am not using?

Next case study

A style is a contract

Turning one art-directed look into a spec that survives thousands of inputs nobody planned for.