2023-07-03 00:32:41 +08:00
|
|
|
|
# AnimateDiff
|
|
|
|
|
|
|
2023-07-03 01:09:13 +08:00
|
|
|
|
This repository is the official implementation of [AnimateDiff]().
|
2023-07-03 00:32:41 +08:00
|
|
|
|
|
|
|
|
|
|
**[AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning]()**
|
|
|
|
|
|
</br>
|
|
|
|
|
|
Yuwei Guo,
|
|
|
|
|
|
Ceyuan Yang,
|
|
|
|
|
|
Anyi Rao,
|
|
|
|
|
|
Yaohui Wang,
|
|
|
|
|
|
Yu Qiao,
|
|
|
|
|
|
Dahua Lin,
|
|
|
|
|
|
Bo Dai
|
|
|
|
|
|
<!-- [Humphrey Shi](https://www.humphreyshi.com) -->
|
2023-07-03 01:09:13 +08:00
|
|
|
|
<!-- </br> -->
|
2023-07-03 00:32:41 +08:00
|
|
|
|
|
|
|
|
|
|
[Paper]() | [Project](https://animatediff.github.io/)
|
|
|
|
|
|
|
2023-07-03 01:09:13 +08:00
|
|
|
|
Code will be release soon, stay tuned!
|
|
|
|
|
|
|
|
|
|
|
|
<!-- ## Setup
|
2023-07-03 00:32:41 +08:00
|
|
|
|
Install the required packages:
|
|
|
|
|
|
```
|
|
|
|
|
|
pip install -r requirements.txt
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## Inference
|
|
|
|
|
|
|
|
|
|
|
|
1. Download personalized Stable Diffusion (LoRA or DreamBooth are currently supported) from [CivitAI](https://civitai.com/) or Huggingface;
|
|
|
|
|
|
|
|
|
|
|
|
2. Download our motion module's pretrained weights from [this link]();
|
|
|
|
|
|
|
|
|
|
|
|
3. Edit config files in
|
|
|
|
|
|
```
|
|
|
|
|
|
configs/prompts/lora.yaml
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
4. Execute animation generation:
|
|
|
|
|
|
```
|
|
|
|
|
|
python -m scripts.animate --prompt configs/prompts/lora.yaml
|
2023-07-03 01:09:13 +08:00
|
|
|
|
``` -->
|
|
|
|
|
|
|
|
|
|
|
|
## Results
|
|
|
|
|
|
|
|
|
|
|
|
<table class="center">
|
|
|
|
|
|
<tr>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_01/01.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_01/02.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_01/03.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_01/04.gif"></td>
|
|
|
|
|
|
</tr>
|
|
|
|
|
|
</table>
|
|
|
|
|
|
<p style="margin-left: 2em; margin-top: -1em">Model：<a href="https://civitai.com/models/30240/toonyou">ToonYou</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
<table>
|
|
|
|
|
|
<tr>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_02/01.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_02/02.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_02/03.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_02/04.gif"></td>
|
|
|
|
|
|
</tr>
|
|
|
|
|
|
</table>
|
|
|
|
|
|
<p style="margin-left: 2em; margin-top: -1em">Model：<a href="https://civitai.com/models/4468/counterfeit-v30">Counterfeit V3.0</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
<table>
|
|
|
|
|
|
<tr>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_03/01.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_03/02.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_03/03.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_03/04.gif"></td>
|
|
|
|
|
|
</tr>
|
|
|
|
|
|
</table>
|
|
|
|
|
|
<p style="margin-left: 2em; margin-top: -1em">Model：<a href="https://civitai.com/models/4201/realistic-vision-v20">Realistic Vision V2.0</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
<table>
|
|
|
|
|
|
<tr>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_04/01.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_04/02.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_04/03.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_04/04.gif"></td>
|
|
|
|
|
|
</tr>
|
|
|
|
|
|
</table>
|
|
|
|
|
|
<p style="margin-left: 2em; margin-top: -1em">Model： <a href="https://civitai.com/models/43331/majicmix-realistic">majicMIX Realistic</a></p>
|
|
|
|
|
|
|
|
|
|
|
|
<table>
|
|
|
|
|
|
<tr>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_05/01.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_05/02.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_05/03.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_05/04.gif"></td>
|
|
|
|
|
|
</tr>
|
|
|
|
|
|
</table>
|
|
|
|
|
|
<p style="margin-left: 2em; margin-top: -1em">Model：<a href="https://civitai.com/models/66347/rcnz-cartoon-3d">RCNZ Cartoon</a>></p>
|
|
|
|
|
|
|
|
|
|
|
|
<table>
|
|
|
|
|
|
<tr>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_06/01.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_06/02.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_06/03.gif"></td>
|
|
|
|
|
|
<td><img src="__assets__/animations/model_06/04.gif"></td>
|
|
|
|
|
|
</tr>
|
|
|
|
|
|
</table>
|
|
|
|
|
|
<p style="margin-left: 2em; margin-top: -1em">Model：<a href="https://civitai.com/models/33208/filmgirl-film-grain-lora-and-loha">FilmVelvia</a></p>
|
