LivePortrait Tutorial: Animating Portraits With Control

LivePortrait is a fast, expressive open-source model for animating still portraits. Unlike pure lip-sync tools, it gives you fine control over expression and head motion, often by transferring movement from a driving video. This tutorial explains what it does, how it works, what you need to run it, and how it differs from audio-driven talking-avatar tools — so you can decide where it fits in your workflow.

As with any research project, LivePortrait's exact commands and dependencies shift between versions. This tutorial focuses on the concepts and workflow; always follow the current official repository for precise steps.

What LivePortrait does

LivePortrait (2024) animates a single still portrait, making the face move, change expression, and turn — efficiently and with notable control. Its defining traits:

• Motion transfer. It commonly drives the portrait using a driving video: the expressions and head movements from the driving clip are transferred onto your still image. The portrait "performs" what the driver does.
• Speed. It's designed to be efficient, producing lively animation without enormous compute.
• Control. It exposes handles for expression and motion, so you can dial the result rather than accept a single fixed output.

This makes it different from tools like Wav2Lip or MuseTalk, which are primarily audio-driven lip-sync — they move the mouth to match a sound. LivePortrait is primarily motion-driven portrait animation — it transfers a performance. For the lip-sync side, see lip sync AI explained; for a model overview, best lip sync AI models.

Driving video vs. audio: the key distinction

Understanding this distinction is the most important part of using LivePortrait well.

• Audio-driven (Wav2Lip, MuseTalk): you provide sound, and the tool generates mouth movement to match. Best when you want a portrait to speak specific words.
• Motion-driven (LivePortrait): you provide a driving video of someone moving and emoting, and the tool transfers that motion to your portrait. Best when you want a portrait to perform — expressions, head turns, liveliness — mirroring a reference.

You can combine approaches in a larger pipeline: use LivePortrait for expressive head and face motion, and a lip-sync model to nail the mouth to specific audio. But out of the box, LivePortrait's strength is expressive transfer, not phoneme-accurate speech.

What you'll need

• A capable GPU. LivePortrait is efficient but still benefits from a decent GPU, typically NVIDIA with CUDA.
• A Python environment. An isolated environment for dependencies.
• The repository and model weights, downloaded and placed as instructed.
• Inputs: a source portrait (the face you want to animate) and, for motion transfer, a driving video.

The setup workflow

The general shape, version specifics aside:

1. Clone the official repository.
2. Create and activate a fresh Python environment.
3. Install dependencies, including a PyTorch build matching your CUDA version. As always, version matching is the step most likely to cause trouble.
4. Download the model weights into the expected directories.
5. Run a provided example to confirm everything works before using your own media.

Running an animation

Conceptually:

1. Prepare a source portrait — clear, front-facing, well-lit, full face visible.
2. Prepare a driving video (for motion transfer) showing the expressions and head movement you want transferred.
3. Run the inference, pointing at your source and driver, with an output path.
4. Review and adjust using the available expression and motion controls.

The result is your portrait performing the driver's motion. Swap the driving video and the same portrait performs something different.

Tips for natural results

• Match the framing. Source and driving video should have similar head sizes and framing for the cleanest transfer.
• Choose a clean driver. A driving video with clear, well-lit, front-facing motion transfers better than a chaotic one.
• Start subtle. Extreme driving motion can look uncanny on a still source. Gentle, natural movement reads best.
• Mind the source quality. As with all portrait animation, a sharp, front-facing source is the foundation — see how to animate a portrait.
• Combine for speech. If you need the portrait to say specific words, pair the motion with an audio-driven lip-sync pass.

Common pitfalls

• CUDA/PyTorch mismatch. The usual culprit behind setup failures.
• Mismatched framing. Source and driver with very different head sizes produce awkward transfers.
• Over-driven motion. Too much movement from the driver looks unnatural on a still.
• Expecting speech accuracy. LivePortrait transfers motion; it isn't a phoneme-accurate lip-sync tool on its own.

Choosing and preparing a driving video

Because LivePortrait's signature workflow is motion transfer, the driving video is half the result — choose it carefully.

A good driving clip shows a face with clear, natural movement: gentle head turns, blinking, eyebrow motion, and expression changes, all front-facing and well-lit. The framing should roughly match your source portrait — similar head size and position — so the transfer maps cleanly. Avoid driving clips with extreme motion, rapid cuts, or a face that leaves the frame, since those translate into jarring, unnatural movement on your still.

It also helps to keep the driving motion appropriate to the message. Calm, conversational movement suits a presenter; lively, animated movement suits expressive content. Since LivePortrait transfers whatever the driver does, the personality of your output is really the personality of your driving clip. Preparing or recording a good driver is often where the quality is won or lost.

Combining LivePortrait with lip-sync

LivePortrait gives you expressive head and face motion, but on its own it isn't a phoneme-accurate speech tool. For a portrait that needs to say specific words, the strong pattern is to combine: use LivePortrait (or its motion) to add lifelike head movement and expression, then apply an audio-driven lip-sync pass to lock the mouth to your audio.

The result is a portrait that both moves naturally and speaks accurately — each stage doing what it's best at. This chaining is common in serious pipelines and is a big part of why people learn multiple tools. If that sounds like more assembly than you want, a packaged app that does audio-driven talking avatars directly skips the chaining entirely.

Performance and batch tips

• Match framing between source and driver for the cleanest transfer — it's the number-one quality lever.
• Keep drivers short when testing; iterate on a few seconds before committing to a long clip.
• Update GPU drivers and close competing GPU apps to free memory.
• Reuse a driver across multiple source portraits to give a series a consistent feel.
• Render at a sensible resolution for your hardware rather than maxing it by default.

Frequently asked questions

Does LivePortrait do lip-sync from audio? Its core strength is motion transfer from a driving video, not phoneme-accurate audio lip-sync. For speech to specific words, pair it with an audio-driven lip-sync pass.

Do I always need a driving video? For its signature motion-transfer workflow, yes — the driver supplies the movement. That's the main difference from audio-driven tools that need only a sound track.

Why does my result look unnatural? Usually mismatched framing between source and driver, or a driver with too much motion. Match head sizes and keep the driving movement gentle.

What hardware does it need? A reasonable GPU, typically NVIDIA with CUDA. It's efficient for its capabilities but still benefits from a dedicated card.

I just want my photo to talk — is LivePortrait the right tool? Not the most direct one. For "make a photo speak from audio or text," a packaged talking-avatar app is simpler than a driving-video workflow.

Can I reuse one driving video across many portraits? Yes, and it's a smart efficiency trick. A single well-chosen driving clip can animate several different source portraits, giving a set of avatars a consistent feel and movement style. This is handy when you want a series of characters or presenters that all share the same calm, natural motion — you prepare the driver once and apply it across the set, rather than sourcing fresh motion for each.

Where LivePortrait fits — and where a packaged app does

LivePortrait is fantastic for expressive, controllable portrait animation, and running it locally keeps your media private. The costs are the familiar ones: Python setup, CUDA matching, a driving-video workflow, and ongoing maintenance — covered more broadly in open source talking avatar projects. And the usual caution applies: skipping setup via a hosted demo means uploading your footage, undoing the local-privacy benefit.

If your actual goal is simpler — make a portrait speak from audio or text, without managing repos and driving videos — a packaged local app is the faster path. ClapClip turns a single photo into a talking video with audio- or text-driven lip-sync, locally on Windows, with nothing uploaded and no setup. It's a different emphasis from LivePortrait (speech vs. expressive transfer), but for most "make my photo talk" needs it's the more direct route.

The takeaway

LivePortrait shines at expressive, motion-driven portrait animation with real control, especially via driving-video transfer. Run it locally if you're technical and want that expressiveness and customizability. If you mainly want a portrait to speak specific words with minimal setup, an audio-driven local app gets you there faster.

To try the speak-from-a-photo path with no setup, download ClapClip for Windows and open the Talking Avatar workflow.