[
  {
    "path": "LICENSE",
    "content": "MIT License\n\nCopyright (c) 2023 Zhixing Zhang\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "README.md",
    "content": "# AVID <br><sub> Any-Length Video Inpainting with Diffusion Model </sub>\n\n\n[![arXiv](https://img.shields.io/badge/arXiv-2312.03816-b31b1b)](https://arxiv.org/abs/2312.03816)\n[![Project Page](https://img.shields.io/badge/Project-Website-orange)](https://zhang-zx.github.io/AVID)\n[![Static Badge](https://img.shields.io/badge/supplementary-blue)](https://zhang-zx.github.io/AVID/supp/index.html)\n\nThis respository contains the code for the CVPR 2024 paper [AVID: <ins>A</ins>ny-Length <ins>V</ins>ideo <ins>I</ins>npainting with <ins>D</ins>iffusion Model](https://arxiv.org/pdf/2312.03816.pdf).\nFor more visualization results, please check our [supplementary materils](https://zhang-zx.github.io/AVID/supp/index.html).\n\n> **[AVID: <ins>A</ins>ny-Length <ins>V</ins>ideo <ins>I</ins>npainting with <ins>D</ins>iffusion Model](https://arxiv.org/abs/2312.03816)** \\\n> [Zhixing Zhang](https://zhang-zx.github.io/) <sup>1</sup>,\n> [Bichen Wu](https://scholar.google.com/citations?user=K3QJPdMAAAAJ&hl) <sup>2</sup>,\n> [Xiaoyan Wang](https://xiaoyan.horizonian.com/) <sup>2</sup>,\n> [Yaqiao Luo](https://scholar.google.com/citations?user=be4sU3cAAAAJ) <sup>2</sup>,\n> [Luxin Zhang](https://lucinezhang.github.io/) <sup>2</sup>,\n> [Yinan Zhao](https://yinan-zhao.github.io/) <sup>2</sup>,\n> [Peter Vajda](https://sites.google.com/site/vajdap) <sup>2</sup>,\n> [Dimitris Metaxas](https://people.cs.rutgers.edu/~dnm/) <sup>1</sup>,\n> and [Licheng Yu](https://lichengunc.github.io/) <sup>2</sup> \\\n> <sup>1</sup> Rutgers University\n> <sup>2</sup> Meta AI\n\n<div align=\"center\">\n    <a><img src=\"assets/overview.png\"  width=\"100%\" ></a>\n</div>\n\n**AVID** is a a video inpainting method versatile across a spectrum of video durations and tasks.\n\n> Recent advances in diffusion models have successfully enabled text-guided image inpainting.\nWhile it seems straightforward to extend such editing capability into video domain, there has been fewer works regarding text-guided video inpainting.\nGiven a video, a masked region at its initial frame, and an editing prompt, it requires a model to do infilling at each frame following the editing guidance while keeping the out-of-mask region intact.\nThere are three main challenges in text-guided video inpainting: ($i$) temporal consistency of the edited video, ($ii$) supporting different inpainting types at different structural fidelity level, and ($iii$) dealing with variable video length.\nTo address these challenges, we introduce Any-Length Video Inpainting with Diffusion Model, dubbed as AVID.\nAt its core, our model is equipped with effective motion modules and adjustable structure guidance, for fixed-length video inpainting.\nBuilding on top of that, we propose a novel Temporal MultiDiffusion sampling pipeline with an middle-frame attention guidance mechanism, facilitating the generation of videos with any desired duration. \nOur comprehensive experiments show our model can robustly deal with various inpainting types at different video duration range, with high quality.\n\n## Acknowledgement\n\nThis codebase is built on top of [diffusers](https://github.com/huggingface/diffusers), [AnimateDiff](https://github.com/guoyww/AnimateDiff), and [Tune-A-Video](https://github.com/showlab/Tune-A-Video).\nWe thank the authors for their great works.\n\n## Reference\n\nIf our work helps you, please consider to cite our paper. Thanks!\n\n```BibTeX\n@article{zhang2023avid,\n  title={AVID: Any-Length Video Inpainting with Diffusion Model},\n  author={Zhang, Zhixing and Wu, Bichen and Wang, Xiaoyan and Luo, Yaqiao and Zhang, Luxin and Zhao, Yinan and Vajda, Peter and Metaxas, Dimitris and Yu, Licheng},\n  journal={arXiv preprint arXiv:2312.03816},\n  year={2023}\n}\n```\n"
  }
]