PORTRAIT DIRECTION · BROWSER WEBGPU

How a Portrait and Voiceover Become a Talking Digital Human in the Browser

A useful digital human begins with ordinary editorial decisions: a readable portrait, clean speech, believable timing and a careful review of identity. Timeline Studio combines the current front-facing image with the current voiceover, generates audio-driven motion with JoyVASA, renders the portrait with LivePortrait and places the resulting WebM directly on the visual track. The workflow runs in browser Workers and exposes its progress, quality choices and failure recovery instead of hiding them behind a one-click promise.

How a Portrait and Voiceover Become a Talking Digital Human in the Browser visual guide

CHECKLIST

Practical review checklist

  1. 01Use a front-facing or slight three-quarter portrait with visible eyes, mouth, jaw and shoulders.
  2. 02Choose clean speech without music, clipping or long reverberation when lip timing matters.
  3. 03Run the browser support check and leave enough storage for the first model download.
  4. 04Start with the 256px preview path before committing time to the 512px quality render.
  5. 05Review identity, lip closure, eye motion, head stability and audio synchronization before export.
  6. 06Use only portraits and voices you own or have explicit permission to animate.
01

Start with a portrait that can survive motion

The system needs a current image on the visual track, not an abstract character description. A level, front-facing portrait is the most reliable input because both eyes, the mouth contour and the jaw remain visible. Leave space around the head and shoulders so small pose changes do not immediately collide with the crop. Hard side light, hair covering the lips, extreme expression, profile views and heavy beauty filters give the renderer less trustworthy evidence. High resolution helps only until the face is clear; a sharp but badly framed portrait is still a poor driver. The source should look like a neutral frame from the video you want to make, with an expression that can plausibly move into speech.

02

The voiceover supplies motion, not just sound

Timeline Studio decodes the current voiceover to 16 kHz samples and sends it to a dedicated JoyVASA Worker. The audio model and motion denoiser translate speech rhythm into a sequence of mouth, eye and head-motion parameters. The configured runtime uses a 64,000-sample window, 25 motion frames per second and 50 diffusion steps. This is why clean speech matters: music, clipping, aggressive noise reduction and room echo can blur the temporal cues that should become lip closure and emphasis. If the same audio blob is used again, its motion result is cached during the session, so a second portrait render can reuse the expensive audio-motion stage instead of repeating it.

03

LivePortrait turns motion into rendered frames

The portrait and motion buffer move to a separate rendering Worker. LivePortrait extracts appearance and motion features, applies the audio-driven keypoint changes, performs the verified 3D warping path and synthesizes new portrait frames. The split between motion and rendering is useful: one voiceover can drive another approved portrait without confusing audio analysis with identity appearance. It also lets the application release one group of GPU sessions before initializing another, reducing competition for WebGPU memory. The final result is encoded as a WebM asset with its dimensions, duration and model path recorded in metadata, then inserted into the media library and used to replace the current visual track.

04

Preview and quality modes answer different questions

Fast preview renders at 256 pixels with FP16 models and adaptive neural keyframes, while the high-quality path renders at 512 pixels. Both target an interpolated playback rate of up to eight frames per second rather than claiming real-time neural generation. Preview is for checking the portrait crop, identity, mouth response and overall stability. Quality mode is for a result that has already passed that editorial test. Rendering only the higher tier first wastes time when the source image or audio needs correction. A sensible workflow is to preview a short phrase, listen and watch at normal speed, adjust the inputs, and use the larger path only after the speaking behavior feels credible.

05

Progress reporting is part of the product

A large browser avatar pipeline has several distinct waits: downloading model parts, initializing HuBERT and the motion model, generating diffusion motion, extracting portrait features, rendering FP16 keyframes, interpolating motion and encoding video. Timeline Studio names these stages and reports completed frames and elapsed seconds where possible. The first run is slower because model files are delivered in manageable parts and cached by the service worker; later visits can reuse cached artifacts. The support check also verifies WebGPU availability, requests a high-performance adapter, inspects maximum GPU buffer size and estimates whether roughly 1.2 GB of cache space remains. A warning is evidence for a choice, not a guarantee that every device will finish the quality tier.

06

Bad frames are handled before they become a video

A numerically valid tensor can still be visually unusable. The renderer samples output values, rejects non-finite data, extreme values and nearly flat frames, and compares temporal distance with the previous accepted frame. The first frame allows a sampled-distance threshold of 0.34; later frames use the tighter 0.22 threshold to discourage abrupt identity jumps. When a neural keyframe looks abnormal, the workflow retries it in place. If it remains corrupt, the renderer drops that keyframe and holds the previous valid frame rather than inserting a broken image into the sequence. This policy favors a brief pause over a flash of damaged identity, but the finished clip still requires human review.

07

Review the result as a performance

Do not judge a talking portrait from one attractive still. Play the encoded clip with sound and inspect complete phrases. Look for closed lips on consonants, reasonable openings on stressed vowels, stable teeth, consistent eye shape, natural blinking and head movement that does not fight the crop. Watch the beginning and end, where neutral pose transitions are easiest to notice. Compare the output with the source portrait at the same size to catch changes in age, face shape, skin detail or accessories. If the person is recognizable, disclose that the footage is synthetic when context could mislead viewers. Consent, voice rights and the intended audience matter as much as render quality.

08

Know the present boundary

This feature creates a short talking portrait from the current image and voiceover; it is not a full-body actor, live webcam puppet or identity-proofing system. Production duration is bounded in the current workflow, and high-quality frames can take substantial time on modest hardware. Browser support, GPU memory, storage quota and model-download state all affect availability. The pipeline is local-first, but model artifacts must be downloaded from pinned revisions before inference can run. A successful render does not grant permission to animate a person or clone a voice. The honest use case is a reviewable portrait performance for authorized narration, education, prototyping or creative production—not an invisible replacement for consent or editorial judgment.

REFERENCE

Frequently asked questions

What do I need before generating a digital human?

Place a front-facing image on the visual track and generate or add a voiceover. The browser also needs working WebGPU and sufficient storage for the model cache.

Does Timeline Studio upload the portrait and voiceover for avatar inference?

The JoyVASA and LivePortrait inference path runs in browser Workers. Model files download from pinned hosted revisions, but the portrait and current voiceover are processed locally by this workflow.

What is the difference between preview and high quality?

Preview uses a 256px FP16 generator for faster editorial checks. High quality uses the 512px FP16 generator and should be chosen after the portrait and motion have been approved.

Is talking-avatar generation real time?

No. Neural keyframes, motion interpolation and encoding reduce the amount of work, but high-quality generation can still take significant time depending on the device.

What happens if one generated frame is corrupt?

The renderer checks numerical and temporal quality, retries an abnormal keyframe once and can hold the previous valid frame if the retry still fails.

Can I animate another person's photo or voice?

Only with appropriate consent and rights. A technically successful render does not provide permission to impersonate, mislead or publish someone else's likeness or voice.