FIELD NOTES · AUTOMATIC EDITING
Browser Auto-Edit: Why a Short Shot Can Disappear
Four controlled videos expose sampling and selection misses in a local execution of Timeline Studio commit 68980d1. Compare clean cuts, dissolves and a six-frame insert, then use the actual scores to decide what needs a manual edit. This case does not verify the current hosted editor or generate AI captions.
THE OBSERVED RESULT
Both clean cuts survived selection. A 0.2-second shot disappeared twice: first between samples, then through score ranking and minimum-gap suppression. The candidate gallery needs review against the original footage.
RECORDED CONTROLS
The four clips share one sampling schedule
- 12 seconds, 224 × 224, 30 fps
- 19 supplied pictures per clip
- Adaptive threshold: 0.1 in all four cases
- Same-segment candidate gap: 1.2 seconds
- Native FFmpeg decode and real production selectors
- No AI inference or live browser export in this case
EXECUTED SOURCE CASE · SEPTEMBER 7, 2026
A six-frame insert disappeared in two different ways
A candidate gallery is useful only if it preserves the moments you need to edit. We made four short videos with known changes and ran Timeline Studio's actual visual-analysis worker and selection helpers. Clean cuts were detected. A 0.2-second insert was lost both when it fell between samples and when a stronger neighboring candidate displaced its sampled picture. That distinction tells you where to look before changing a threshold.
- Inputs and environment
- Four original 12-second geometric videos; 224 × 224, 30 fps, 360 H.264 frames each, no audio. ffmpeg version 8.0.1 Copyright (c) 2000-2025 the FFmpeg developers; Node v22.14.0 on darwin arm64.
- What actually ran
- Native FFmpeg encoded and decoded the MP4s; the source-checked sampling schedule supplied 19 pictures to the unmodified production worker, followed by the production selectors. Source: commit 68980d1. DOM seeking and browser inference were outside this local experiment.
1. Establish a case that should work
The default selector kept pictures at 0, 4 and 8 seconds. The 0-second picture is always retained as the segment's start; the other two match our known cuts. For this duration, the source requests 1.5 samples per second and includes both timeline endpoints, giving 19 sample labels. The final 12-second label reads a source request clamped to 11.95 seconds. These labels are sampling coordinates, not independently measured edit boundaries.
| Controlled input | Selected timeline times | Editing consequence |
|---|---|---|
| Hard cuts at 4 and 8 seconds | 0, 4, 8 s | Both known cuts have a candidate. |
| Two-second dissolves at 3–5 and 7–9 seconds | 0, 4, 5.333, 7.333, 8.667 s | Two dissolves produce four candidates after the initial picture. |
| 0.2-second insert at 4.1–4.3 seconds | 0 s | No sampled picture contains the short insert. |
| Same 0.2-second insert moved to 4.6–4.8 seconds | 0, 5.333 s | The insert is sampled, but the selected later picture already shows A again. |
2. Separate a sampling miss from a selection miss
Samples fall at 4.000 and 4.667 seconds. The entire 4.1–4.3-second insert sits between them, so all supplied pictures show A. The selected list contains only the initial picture. Re-running selection with explicit thresholds 0.03, 0.1, 0.3 and 0.6 changes nothing: the detector never received the orange picture.
Moving the insert onto the sampling grid did not make its picture survive selection. The B picture at 4.667 seconds had weighted score 0.67036; the return to A at 5.333 seconds scored 0.67414. Both exceeded 0.1. The selector ranked the later A picture first, then rejected B because the two pictures are only 0.667 seconds apart and the default minimum gap is 1.2 seconds.
A separate developer control with a 0.3-second gap kept both sampled transitions. It could not recover the between-samples insert. These controls are function arguments in the published harness, not settings claimed to be available in the live interface. The initial hypothesis and failed check are retained: we first expected that moving the shot onto a sample would be enough, and the observed result disproved it.
3. Treat dissolve candidates as pictures to review
These two transitions produced four selected pictures after the initial one: 4, 5.333, 7.333 and 8.667 seconds. One is already after the first dissolve. Multiple candidates can therefore describe one transition; their count is not a count of edits. A threshold of 0.6 suppressed all four in this fixture and also lost the control's cut at 8 seconds. Raising it is a tradeoff, not a universal cleanup rule.
The original explanation also overstated the local-peak filter. This snapshot computes a localPeak value but does not use it in the filter. It ranks quality-weighted differences, applies a quality gate and enforces same-segment spacing. Segment starts are retained before those tests. Keep this distinction when interpreting candidate scores or implementing a different sampler.
4. Review timing after the draft has been normalized
Candidate spacing does not guarantee readable caption duration. As a separate timing check, we supplied three explicitly hand-authored caption objects for a three-second clip: 0–4, 1–2 and 1.5–3 seconds. The real timing normalizer returned 0–0.5, 0.5–1.5 and 1.5–3 seconds. The first caption is only half a second long after overlap adjustment. The exact input strings and returned ranges are in run.json; none is represented as an AI-generated caption.
The reviewed Apply handler passes the reviewed caption array to its commit callback. It is a caption draft workflow, with no narrative trimming of the source video. Preserve a project version before applying, inspect how existing captions are handled, and preview the new timing at normal playback speed. Model wording, browser model availability and the visible review step require their own live check.
Repeat the case and choose the next editing step
Download the complete reproduction kit, verify it with shasum -a 256 -c SHA256SUMS, then run node reproduce.mjs ./my-run with Node and native FFmpeg/ffprobe installed. No model, API key, account or package installation is required. The harness regenerates the videos and verifies the recorded observations, including the misses. Different encoder or decoder builds can change exact scores; preserve the new report if results differ.
For a short product reveal, gesture or action, first confirm that the important picture exists in the sampled input. If it is missing there, mark it manually or use a denser sampling strategy. If it was sampled but lost during selection, inspect neighboring scores and spacing. For a dissolve, review the transition as a whole before keeping several similar captions. Finish by watching the source and the caption track together.
Full method and limits · Measured outputs and controls · Executed FFmpeg commands · Pinned source and SHA-256 fingerprints · Artifact manifest.
This original fixture establishes specific behavior in four controlled clips, not a real-footage accuracy rate. Native decoding, Node execution and a square input bypass browser seek, canvas scaling and model-download behavior. No Chrome LanguageModel, Translator, generated-caption quality, timeline application or browser export was tested. The bundled public source retains its MIT license.
REFERENCE
Questions the case can answer
Why can a short shot disappear from automatic-edit candidates?
A shot can fall entirely between sampled frames. Even when sampled, its picture can be suppressed by a higher-scoring nearby frame and the 1.2-second minimum gap. Both happened in the published controlled cases.
Will lowering the scene threshold recover a missed shot?
Not necessarily. Thresholds of 0.03, 0.1, 0.3 and 0.6 did not recover the short inserts in this experiment. No threshold can recover a picture absent from the sampled input, and a lower threshold does not remove minimum-gap suppression.
Does a scene candidate mark the exact moment of a cut?
No. It is a sampled picture selected for visual change. The two-second dissolves produced multiple candidates, including a picture after a transition. A candidate gallery is not a frame-accurate edit decision list.
Were the videos and AI captions generated in the live browser?
No. The videos are original procedural fixtures encoded and decoded by native FFmpeg. The pinned production worker and selection helpers ran locally in Node. No browser AI inference, model download, review dialog, caption application or final browser export was executed.
Does the automatic-edit workflow shorten my source clips?
The reviewed automatic-edit hook prepares candidate pictures and draft captions, then passes the reviewed caption array to its commit callback on Apply. It does not perform narrative trimming or reorder the source video. Preserve an earlier project and review existing captions before applying a new draft.