Mirrors/AnimateDiff
1
0
Fork 0
mirror of https://github.com/guoyww/AnimateDiff.git synced 2025-12-16 08:27:51 +01:00
Code Issues Packages Projects Releases Wiki Activity

add explain

Browse Source
This commit is contained in:
Yuwei Guo
2023-12-16 01:20:48 +08:00
parent 4825080465
commit 212ebb6a80
2 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -66,6 +66,10 @@ Manually download the AnimateDiff modules. The download links can be found in ea
In this version, we did the image model finetuning through **Domain Adapter LoRA** for more flexiblity at inference time.
- **Explanation:** Domain Adapter is a LoRA module trained on static frames of the training video dataset. This process is done before training the motion module, and helps the motion module focus on motion modeling, as shown in the figure below. At inference, By adjusting the LoRA scale of the Domain Adapter, some visual attributes of the training video, e.g., the watermarks, can be removed. To utilize the SparseCtrl encoder, it's necessary to use a full Domain Adapter in the pipeline.
<div style="text-align:center"><img src="__assets__/figs/adapter_explain.png" style="width:60%"></div>
Additionally, we implement two (RGB image/scribble) [SparseCtrl](https://arxiv.org/abs/2311.16933) Encoders, which can take abitary number of condition maps to control the generation process.
Technical details of SparseCtrl can be found in this research paper:

BIN
__assets__/figs/adapter_explain.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 374 KiB