HISTORICAL PORT RECORD · SOURCE REVIEW

LivePortrait WebGPU Port: Evidence and Limits

The earlier Timeline Studio port record describes how two unsupported 5D GridSample nodes were rewritten for browser WebGPU and compared against fixed-input outputs. This page preserves that numerical evidence. Our September 2026 source review also identifies the limits of the record: its timing is historical, its audiovisual result was not reproduced here, and current frame checks do not establish convincing speech or identity.

BY Martin DelophyPUBLISHED REVIEWED SOURCE-VERIFIED FIELD NOTETESTING METHOD
LivePortrait WebGPU Port: Evidence and Limits visual guide
7.8976e-6Max absolute error316Rewritten graph nodes512²Validated output
01

Decompose the avatar pipeline first

The browser workflow is not one model call. JoyVASA converts audio into motion, while LivePortrait performs appearance extraction, motion extraction, lip retargeting, stitching, feature warping and SPADE image generation. Keeping those stages separate creates useful validation boundaries: audio-driver error, keypoint error, warping error and final generator error can be measured independently rather than diagnosed from one incorrect-looking face.

02

Locate the real WebGPU blocker

The opset-20 combined generator had 277 nodes, including 13 rank-5 convolutions and two rank-5 GridSample nodes. A Linux shared-library plugin from the source repository could not run in the browser, and ONNX Runtime Web's WebGPU execution provider could not directly execute the 5D samplers. Removing the depth dimension, treating it as an arbitrary batch or selecting the nearest slice would all change the function computed by the model.

03

Rewrite trilinear sampling with supported operations

A target depth coordinate lies between slices d0 and d1. The rewrite performs supported 4D bilinear GridSample on both planes, then interpolates the two results using the fractional depth coordinate alpha. Depth slices are arranged as batches so WebGPU can process them in parallel and then restored to the original spatial layout. A verified shared-XY structure in the sparse-motion grid removes redundant sampling. The rewritten graph grew from 277 to 316 nodes, but every operation could run in the browser.

04

Read the recorded numerical comparison

The earlier port record reports a comparison of the original 512 × 512 generator and rewritten graph on fixed inputs: maximum absolute error 7.8976e-6, mean absolute error 1.3853e-7 and 99.9th-percentile error 1.7881e-6. It also records maximum ONNX-versus-PyTorch errors of 7.75e-6 for JoyVASA's HuBERT export and 6.44e-6 for its diffusion denoiser. These are preserved development measurements, not values rerun during the September 2026 review. They support the specific recorded numerical comparison; they do not establish quality across different portraits or speech.

05

Solve the browser runtime around the graph

The WebGPU workers configure matching ONNX Runtime Asyncify JavaScript and WASM files and create execution-provider sessions serially. The render worker retains the portrait appearance feature for reuse. Summing the JoyVASA and LivePortrait configured artifacts gives 738,879,580 bytes for preview, or 949,593,258 bytes for both resolution tiers: approximately 739 MB and 950 MB respectively, equivalent to 705 MiB and 906 MiB. These are file-size calculations, excluding the application, runtime and inference memory; they are not measured downloads. The model files are loaded from a pinned hosted revision.

06

Treat the timing as a historical cost warning

The configured mixed-FP16 tiers produce 256-pixel preview or 512-pixel quality images. The earlier development note reports roughly 25–60 seconds per 512 × 512 neural frame on its test browser and GPU. It does not name that browser/GPU configuration or attach a complete timing log. We did not rerun that benchmark in this review and cannot transfer its speed to the current device or the 256-pixel tier.

The current implementation renders sparse neural keyframes and copies the nearest available image into each encoded timestamp. An eight-fps WebM therefore does not mean eight neural images per second or motion interpolation. Downloads, session initialization, JoyVASA motion generation and encoding add costs outside the historical per-frame rendering figure.

07

Separate the historical media report from a current demonstration

The development note reports that model-part loading, WebGPU setup, 50-step JoyVASA motion, LivePortrait rendering, WebM encoding, browser playback and visual-track replacement completed. It describes a playable 3.56-second WebM, changing mouth states, head and eye movement, and no browser console errors after completion. The original voiceover, output video and full run log were not supplied with that note and were not revalidated in this assessment. The report establishes what was recorded during development; it is not a new synchronized audiovisual example a reader can inspect here.

A fresh practical acceptance run should preserve the exact portrait, speech, intermediate video and final export with sound. The current avatar encoder adds only a video track to the intermediate WebM. Audio must be verified after the timeline export rather than inferred from a successful video encode.

08

Keep tensor contracts available for a future rerun

A useful graph-port record should identify tensor names, rank, axis order, numeric type and output ownership. Batch, channel, depth, height and width have distinct meanings in the feature volume; reshapes and transposes must preserve them. The preserved port note supplies baseline graph hashes and fixed-input error statistics, while the smoke script documents a small renderer path.

That smoke script runs WASM and requests fixed lip ratios; it is not a JoyVASA audio-driven browser benchmark. Dynamic-shape coverage, mixed-precision regression, current WebGPU speed and audiovisual alignment would need separate tests and retained artifacts. We did not execute those tests in this review and do not claim that the recorded fixed-input comparison covers them.

09

The current temporal checks leave a perceptual gap

The implemented guards inspect sampled finite values, extreme magnitude, near-flat variance and distance from a reference frame. The worker retries a rejected keyframe once and can repeat the previous valid frame if both attempts fail. This can keep an obviously damaged image out of the encoded sequence, but a held frame is still a visible loss of motion.

No phoneme alignment, speech-to-mouth score or face-identity metric is implemented by those numerical guards. The current nearest-keyframe reuse also provides no generated intermediate motion whose identity preservation has been measured. To judge a performance, retain a final export with the actual voiceover and review lip closure, teeth, eyes, jaw, start/end timing and any fallback holds. The talking-portrait assessment proposes a one-second preview and a stopping budget before a more expensive run; that perceptual trial has not yet been performed for this review.

ENGINEERING RECORD / NUMERICAL VALIDATION

The graph rewrite is measured, not described by resemblance

The port record preserves the pinned graph hashes and numerical comparison used to accept the WebGPU rewrite. Those measurements are stronger evidence than a single favorable portrait frame, while still leaving speed and generalization as explicit limits.

REVIEWED SNAPSHOT
Timeline Studio commit 68980d1 · LivePortrait baseline e6c5d24
WHAT THE SOURCE SUPPORTS
The recorded opset-20 generator grows from 277 to 316 nodes after two rank-5 GridSample operations are rewritten as supported 4D sampling plus depth interpolation. On fixed 512 × 512 inputs, the recorded maximum absolute error is 7.8976e-6 and mean absolute error is 1.3853e-7.
HOW TO REPEAT THE CHECK
Use the pinned original and rewritten graph hashes, run both with the same fixed tensors, compare full-frame absolute-error statistics, then run the smoke script and the browser worker. Accept the media path only after encoding, playback and timeline insertion also complete.
CLAIM BOUNDARY
The earlier note records a fixed-input numerical comparison, not identity preservation across portraits. Its reported 25–60 seconds per 512 × 512 neural frame is historical; no current speed or synchronized audiovisual benchmark was run for this review.

SEPTEMBER 7, 2026 · SOURCE REVIEW AND CORRECTIONS

The source files behind this revision

We checked the implementation at commit 68980d1 to narrow claims that the earlier article could not support. This revision records source findings and model-size arithmetic. The related portrait assessment now adds a reproducible non-neural encoder experiment with three test cards. Neither page includes new neural inference, a dynamic portrait-quality test or a current neural-speed benchmark.

Playback and sound
The preserved media-function excerpt shows nearest-keyframe reuse and a video-only intermediate WebM. The new test-card experiment executes that encoder and preserves an eight-frame, one-second WebM with three source images and no audio track. This verifies media packaging, not portrait motion or final audio mixing.
What a frame guard checks
The numerical guard and render-worker excerpt document sampled values, frame distance, retries and fallback holds. They do not score identity or speech-to-mouth alignment.
Historical versus calculated evidence
The earlier port note is the source of the reported 3.56-second clip and 25–60 seconds per 512-pixel frame. The JoyVASA and LivePortrait configurations supply the declared bytes used in the model budget.

The structured source review records these boundaries and the proposed one-second trial. The SHA-256 manifest identifies every preserved source file and excerpt; the method and file guide explains their scope. A current final video with sound, perceptual review and complete timing remains future evidence.

REFERENCE

Frequently asked questions

Why not use a custom native plugin in the browser?

Browser runtimes cannot load the original Linux shared library, so the computation must use portable supported operators or a browser-specific implementation.

Is the rewritten GridSample exact?

The earlier fixed-input comparison records errors around 1e-6 to 1e-7. Those measurements were not rerun in this review and do not establish equivalence for every input.

Is the 512-pixel avatar path real time?

No current speed benchmark is established here. The older port note reports 25–60 seconds per 512-pixel neural frame; current eight-fps encoding reuses sparse keyframes and is not real-time neural generation.

Why pin the model revision?

A fixed revision keeps cached graph files, tensor contracts and runtime assumptions synchronized across sessions.

AUTHOR

Martin Delophy

Independent full-stack and algorithm engineer in China with 10 years of frontend, AI and audio/video development experience, including 5 years focused on AI. His open-source work covers browser AI, ONNX, WebGPU, Transformers, Stable Diffusion and local-first creative tools.

About the author →