2020-11-12 17:31:40 +08:00
# RIFE
2020-11-13 11:06:48 +08:00
## [arXiv](https://arxiv.org/pdf/2011.06294.pdf)
**We are working on arranging our training code and other scripts.**
2020-11-13 19:18:57 +08:00
Our model can run 30+FPS for 2X interpolation on 2080Ti GPU. We also support CPU device including MacOS.
2020-11-12 20:27:43 +08:00
2020-11-13 20:24:27 +08:00
16X interpolation results from two input images:
2020-11-13 18:41:50 +08:00
2020-11-13 18:24:04 +08:00
2020-11-12 17:31:40 +08:00
## Abstract
We propose a real-time intermediate flow estimation algorithm (RIFE) for video frame interpolation (VFI). Most existing methods first estimate the bi-directional optical flows, and then linearly combine them to approximate intermediate flows, leading to artifacts around motion boundaries. We design an intermediate flow model named IFNet that can directly estimate the intermediate flows from coarse to fine. We then warp the input frames according to the estimated intermediate flows and employ a fusion process to compute final results. Based on our proposed leakage distillation, RIFE can be trained end-to-end and achieve excellent performance. Experiments demonstrate that RIFE is significantly faster than existing flow-based VFI methods and achieves state-of-the-art index on several benchmarks.
2020-11-12 20:27:43 +08:00
## Dependencies
```
2020-11-12 23:45:48 +08:00
pip3 install torch==1.6.0
2020-11-12 20:27:43 +08:00
pip3 install numpy
2020-11-12 23:45:48 +08:00
pip3 install opencv-python
2020-11-12 20:27:43 +08:00
```
## Inference and Testing
* Download the pretrained models from [here ](https://drive.google.com/file/d/1c1R7iF-ypN6USo-D2YH_ORtaH3tukSlo/view?usp=sharing )
2020-11-13 01:44:56 +08:00
(We also provide 百度网盘 source. 链接: https://pan.baidu.com/s/17tHd-syovvRGP2C6UVPsIw 提取码: 5ha7)
2020-11-13 00:06:58 +08:00
* Unzip and move the pretrained parameters to train_log/\*.pkl
2020-11-12 23:39:28 +08:00
2020-11-13 02:36:37 +08:00
**Inference**
2020-11-12 23:39:28 +08:00
```
2020-11-13 17:42:43 +08:00
$ python3 inference.py --img /path/to/image_0 /path/to/image_1 --times=5
(You will get 32x interpolation results in default.)
$ ffmpeg -r 10 -f image2 -i output/img%d.png -s 448x256 -c:v libx264 -pix_fmt yuv420p output/slomo.mp4 -q:v 0 -q:a 0
(You will get a slomo video based on two input images.)
2020-11-12 23:39:28 +08:00
```
2020-11-13 10:27:30 +08:00
2020-11-13 11:16:57 +08:00
## Citation
2020-11-13 10:27:30 +08:00
```
@article {huang2020rife,
title={RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation},
author={Huang, Zhewei and Zhang, Tianyuan and Heng, Wen and Shi, Boxin and Zhou, Shuchang},
journal={arXiv preprint arXiv:2011.06294},
year={2020}
}
```
## Reference
2020-11-13 11:16:57 +08:00
Optical Flow:
2020-11-13 17:42:43 +08:00
[ARFlow ](https://github.com/lliuz/ARFlow ) [pytorch-liteflownet ](https://github.com/sniklaus/pytorch-liteflownet ) [RAFT ](https://github.com/princeton-vl/RAFT )
2020-11-13 11:16:57 +08:00
Video Interpolation:
[DAIN ](https://github.com/baowenbo/DAIN ) [CAIN ](https://github.com/myungsub/CAIN ) [AdaCoF-pytorch ](https://github.com/HyeongminLEE/AdaCoF-pytorch )
