Source review · Commit 3684ec4 · September 2026

A generation plugin should return media, not control the timeline.

Timeline Studio now connects Puter.js, ComfyUI and Stable Diffusion WebUI or Forge through one generation-provider contract. The interesting part is not the catalog. It is the boundary that prevents provider code from becoming a second, less accountable editor.

Published September 3, 2026
By Martin Delophy
Engineering field note · Source-reviewed
BY Martin DelophyPUBLISHED REVIEWED ENGINEERING FIELD NOTETESTING METHOD
Diagram showing Puter.js, ComfyUI and Stable Diffusion WebUI adapters passing media through the Timeline Studio host before it enters My assets

The implemented path is provider → isolated adapter → host validation → My assets. No provider receives timeline mutation functions, and a completed output is not placed on a track automatically.

THE SHORT VERSION

The plugin contract is useful because it defines what provider code cannot do. Generation remains replaceable; the host keeps ownership of media truth and editor state.

01

“Plugin” currently means a reviewed connector

The name can easily imply a marketplace where arbitrary packages are downloaded and executed at runtime. That is not what has shipped. The current implementation keeps manifests and adapters in the Timeline Studio source tree, registers them during the build and reviews them with the rest of the application. There is no third-party code loader, runtime permission prompt or compatibility negotiation yet.

That limitation is a security feature until those missing pieces exist. A generation connector can touch remote accounts, local AI servers and media bytes. Calling any JavaScript bundle a plugin before defining its permissions would turn an extension label into ambient authority. The development contract is explicit: today, adding a plugin means contributing a reviewed provider adapter to the repository.

Puter.jsbrowser-session
provider-session auth
text-to-image · text-to-video
ComfyUIloopback at 127.0.0.1:8188
workflow-image · workflow-video
SD WebUI / Forgeloopback at 127.0.0.1:7860
text-to-image · image-to-image
02

A manifest declares capability; an adapter performs transport

Each provider has two deliberately different responsibilities. Its manifest describes stable identity, runtime class, authentication mode, capabilities, output types and default endpoint. Its adapter connects to the actual SDK or HTTP API, converts the shared request into provider-specific calls and returns normalized outputs. React state is not part of that transport layer.

The shared registry contains exactly three manifests. The hook looks up the corresponding adapter, coordinates connection and generation attempts with AbortController, and passes the result to the host. Adding a fourth provider therefore does not require another provider-specific network branch inside the shared hook.

manifest = identity + runtime + capabilities + auth
adapter  = connect() + generate() + cancel() + normalizeError()
host     = validate bytes + decode media + commit My assets
hook     = lifecycle + supersession + visible job state

The separation matters during failure. A malformed provider response should fail in its adapter or at the host boundary. It should not leave half-created React state, a guessed timeline object or a success message backed only by a URL.

03

Three providers, three real connection models

Puter.js owns a browser session and popup authorization flow. Connection waits for the provider's actual sign-in result instead of optimistically changing a badge. Image generation calls puter.ai.txt2img; video generation calls puter.ai.txt2vid. The adapter then resolves or downloads the returned media. The Grok image path uses a temporary provider file and removes it after the bytes have been read.

ComfyUI and Stable Diffusion WebUI are local services, but “local” is verified rather than assumed. Their endpoint parser accepts only localhost, 127.0.0.1 or ::1 over HTTP or HTTPS. ComfyUI checks /system_stats; WebUI checks /sdapi/v1/samplers. A network failure is translated into a message that tells the user to check whether the service is running and whether CORS permits the editor origin.

Once connected, ComfyUI submits a workflow to /prompt, polls /history/{prompt_id} and downloads every reported output through /view. WebUI calls either /sdapi/v1/txt2img or /sdapi/v1/img2img and decodes every returned base64 image. The adapter does not silently keep the first file when a provider returns a batch.

04

A temporary URL is not a completed asset

Generation APIs often return URLs that are signed, session-bound or short-lived. If an editor stores that URL as its result, the thumbnail may work during the session and disappear after a refresh—or expire before export. Timeline Studio defines completion more strictly: the provider must return usable media bytes, and the host must be able to inspect them.

Download every outputVerify non-empty BlobDecode image or videoCreate My assets entries

For images, the host reads the file signature, reconciles the declared MIME type, creates an ImageBitmap and rejects a file with no dimensions. PNG, JPEG, WebP and AVIF signatures are recognized. For video, the host creates a temporary object URL and waits for browser metadata so duration and dimensions can be recorded. Object URLs used only for inspection are revoked.

Only after that validation does the host assign asset IDs, create browser-local URLs and prepend the entire result batch to My assets. This is a small but important definition of truth: “complete” means the editor owns decodable media, not that a provider once returned something that looked like a file location.

05

Generation is not permission to edit

The host commits results to My assets and selects the latest imported item. It does not create a clip, choose a track, move the playhead or infer how long a generated image should remain visible. Those are editorial decisions, even when the prompt was entered from inside an editor.

This boundary also makes batch results legible. If ComfyUI emits several images or videos, all valid outputs become library assets. A user can compare them, reject them, rename them and deliberately insert the chosen media. Automatic placement would collapse generation and editing into one irreversible-looking action and make failure recovery harder.

Provider adapter mayProvider adapter may notHost remains responsible for
Connect to its declared runtimeRead or mutate timeline stateArbitrating the active job
Translate shared requestsCall setUserAssets directlyValidating all returned media
Report provider-backed stateFabricate progress or successCreating asset IDs and object URLs
Return normalized outputsInsert clips automaticallyCommitting every valid output
06

Cancellation needs an honest verb

The shared hook creates an AbortController for each connection or generation attempt, ignores late callbacks from superseded attempts and prevents two jobs from owning the same inspector surface. Cancelling always stops the editor from waiting. It only claims remote cancellation when the provider exposes a real operation for it.

ComfyUI has an /interrupt endpoint, so its adapter can request cancellation from the local server. Other adapters may only stop client-side work. The difference belongs in user-facing language because “cancelled” can mean two materially different things: the compute stopped, or the editor stopped listening while remote compute may continue.

DISCONNECTEDCONNECTINGCONNECTEDRUNNINGCANCELLEDERROR

Progress follows the same discipline. A percentage is shown only when a provider supplies meaningful progress; otherwise the running state remains indeterminate. A polished progress animation is not evidence that an external system knows how much work remains.

07

What this architecture proves—and what it does not

The code establishes a narrow provider architecture: validated manifests, isolated adapters, loopback restrictions, host-owned output validation, lifecycle coordination and a My assets destination. It provides a repeatable place to add another generation provider without moving transport logic back into a monolithic hook.

It does not prove that every model is available in every region, that Puter account terms or model prices will remain unchanged, that a user's local ComfyUI workflow is safe, or that generated media is accurate or publishable. Provider availability, cost, browser support and content restrictions remain provider-specific. Local endpoints also remain software running on the user's machine and should not be exposed beyond loopback without a separate security design.

Most importantly, this is not yet a general plugin marketplace. Runtime installation would require signed or otherwise attributable packages, permission declarations, version negotiation, sandboxing, revocation and a review model. Until those controls exist, source integration is the honest name for what ships.

Source trail for this field note

Questions people ask

Can I install a third-party plugin package?

No. The current providers are source-integrated connectors reviewed and built with Timeline Studio. There is no runtime for downloading and executing arbitrary third-party plugin code.

Does a generated image appear on the timeline automatically?

No. Every valid output is added to My assets. The user decides which result to place on a track and how it should be edited.

Can the editor connect to ComfyUI on another computer?

The current loopback boundary accepts only localhost, 127.0.0.1 or ::1. It intentionally does not encourage LAN exposure or wildcard CORS.

Why validate the Blob if the provider says it is an image?

Response labels can be wrong, empty or temporary. The host checks file signatures and decodes the media so completion reflects a usable asset.

AUTHOR

Martin Delophy

Independent full-stack and algorithm engineer in China with 10 years of frontend, AI and audio/video development experience, including 5 years focused on AI. His open-source work includes Timeline Studio, browser AI pipelines, ONNX, WebGPU and agent-compatible creative workflows.

About the author and testing method →

Generate media. Keep the edit intentional.

Inspect the sourceOpen Timeline Studio