InfiniteTalk AI

Sparse-frame video dubbing is a new paradigm introduced by the InfiniteTalk research.

Instead of editing every frame or only inpainting the mouth, we:

Select a sparse set of keyframes from the original video, and Use those keyframes as high-level anchors, while Letting the model freely generate all in-between frames in sync with the new audio. These keyframes act as control points for:

Identity — who the person is (face structure, hairstyle, clothing). Emotional cadence — where the original performance speeds up, slows down, or hits strong emotional beats. Iconic gestures — recognisable hand movements or poses that define the scene. Camera trajectory — framing, zoom, and rough camera path across the shot. The model then generates the full dubbed video such that:

Lips, face, head, and body are driven by the new speech. Keyframes ensure the actor still looks like themselves and stays consistent across time. The camera still feels like the original shot, not a completely new recording.