Stability AI unveils Stable Video Diffusion, a foundation model for generative video

Stability AI, the company responsible for the well-known deep learning model Stable Diffusion, has recently announced the launch of Stable Video Diffusion. This marks the first foundation model for generative video, based on the image model Stable Diffusion.

The newly released video model can be adapted to a range of downstream tasks. Notably, it can be fine-tuned on multi-view datasets to achieve multi-view synthesis from a single image. Stability AI intends to develop a variety of models that build upon and expand this base, aiming to create an ecosystem akin to the one that surrounds Stable Diffusion.

Stability AI has released Stable Video Diffusion in the form of two image-to-video models. These models can generate 14 and 25 frames respectively, with customizable frame rates ranging from 3 to 30 frames per second.

Upon release, external evaluation revealed that these models outperform the leading closed models (like Pika Labs and Runway) in user preference studies, according to Stability AI. However, the company has clarified that this model is not intended for real-world or commercial applications at the current stage.

The code for Stable Video Diffusion is accessible on GitHub. The necessary weights for running the model locally are available on the Hugging Face page.