FIELD NOTES · AUTOMATIC EDITING
From Scene Changes to a Reviewable Rough Cut in the Browser
Timeline Studio's automatic editing workflow turns a long video into a compact set of scene candidates, asks an image-capable browser language model to describe them, and presents the result for review before anything reaches the caption track. It is a local-first rough-cut assistant—not an opaque system that silently deletes or rearranges clips.
THE DIRECT ANSWER
Timeline Studio samples a video, measures visual change in a Web Worker, selects high-quality scene peaks, and asks the browser's built-in multimodal LanguageModel to draft timed captions. The editor then shows every candidate for human review. Only approved captions are written to the timeline; source clips are not automatically cut, deleted, or reordered.
IMPLEMENTATION LEDGER
Six facts the interface can defend
- 1.5 fps sampling for videos up to 120 seconds
- 224 × 224 contained JPEG analysis frames
- Adaptive threshold: median + max(0.035, MAD × 2.5)
- At least 1.2 seconds between selected scene candidates
- Browser LanguageModel requires image and text input support
- Human approval is required before captions reach the timeline
Act I — Turn motion into evidence
The analyzer does not send the whole video to a remote editing service. It samples the active visual clips at a duration-aware rate: 1.5 frames per second up to two minutes, 1 frame per second up to ten minutes, and 0.75 frames per second for longer material. Each frame is contained inside a 224 by 224 canvas and encoded as JPEG for analysis. A dedicated Worker measures frame difference and image quality so decoding and scoring do not freeze the editing interface.
Act II — Select change, not noise
A fixed sensitivity value breaks across screen recordings, talking heads and handheld footage. Timeline Studio derives an adaptive threshold from the score distribution: the median plus the larger of 0.035 or 2.5 times the median absolute deviation, with a floor of 0.1. It keeps clip boundaries and local peaks that clear the threshold, rejects candidates below a 0.28 quality score, and enforces a 1.2-second minimum gap. This favors meaningful visual transitions without pretending that every pixel change is an edit point.
Act III — Let multimodal AI describe the candidates
The browser's built-in LanguageModel is probed for both image and text input support before a session is created. Candidate frames are captioned in small overlapping windows so the model receives enough neighboring context to describe what changes while keeping prompts bounded. The workflow requests structured JSON, normalizes timing and falls back to per-frame generation when a batch omits a candidate. Ten output languages are supported, including Chinese, English, Japanese and Korean.
The review dialog is the editorial boundary
The result is not committed immediately. A review dialog shows candidate images, timestamps, aspect ratios, visual-change percentages, per-clip progress and the generated text. This is where an editor can catch a wrong subject, an invented detail or a caption that describes a transition rather than the story. Only the approved result creates caption clips on the timeline; the original video clips are not deleted, shortened or reordered by this feature.
Why model setup starts on the click
Browsers may require transient user activation before downloading or initializing a built-in AI model. Timeline Studio therefore begins session creation directly from the automatic-edit button action instead of waiting until after asynchronous frame analysis. This small product constraint is easy to miss: if initialization starts later, the same API call can fail even though the device otherwise supports the model.
Where this approach works—and where it does not
The workflow is strongest for travel footage, demos, interviews with cutaways and other material with visible scene changes. It is less useful for a single unbroken talking-head shot, and visual difference alone cannot understand narrative priority. Browser AI availability also varies by browser, device and model-download state. The system responds by exposing capability checks and reviewable evidence instead of claiming universal one-click editing.
EDITOR'S NOTES
Before you apply the rough cut
- 01Use a video with meaningful shot or composition changes; a static recording produces fewer useful peaks.
- 02Keep the browser tab active during initial model setup and allow the built-in model to download if required.
- 03Review names, numbers, brand references and scene intent because captions are model-generated estimates.
- 04Check adjacent captions for pacing even though candidate frames are separated by at least 1.2 seconds.
- 05Treat the result as a caption rough cut: source clips remain unchanged until you edit them yourself.
REFERENCE
Questions with precise boundaries
Does automatic editing cut or delete my video clips?
No. The current workflow detects scene candidates and drafts timed captions. It changes the timeline only after review, and it does not automatically delete, shorten or reorder source clips.
Does the video have to be uploaded to an AI service?
The analysis and supported built-in language-model workflow run in the browser. Application and model files may need to download first, but the source video is not sent to a remote editing service by this feature.
Which browser AI capability is required?
The browser must expose a built-in LanguageModel that accepts both image and text input. Availability depends on browser version, device support and model-download state.
Why are the generated captions reviewable?
Visual captions can misidentify objects, names or intent. Showing candidates and scores lets the editor verify evidence before the result becomes part of the project.
Is this the same as a finished automatic edit?
No. It is a reviewable rough-cut assistant for scene selection evidence and captions. Narrative trimming, ordering, pacing and final editorial judgment remain with the user.