[
  {
    "path": ".github/CODE_OF_CONDUCT.md",
    "content": "# Contributor Covenant Code of Conduct\n\n## Our Pledge\n\nWe as members, contributors, and leaders pledge to make participation in our\ncommunity a harassment-free experience for everyone, regardless of age, body\nsize, visible or invisible disability, ethnicity, sex characteristics, gender\nidentity and expression, level of experience, education, socio-economic status,\nnationality, personal appearance, race, caste, color, religion, or sexual\nidentity and orientation.\n\nWe pledge to act and interact in ways that contribute to an open, welcoming,\ndiverse, inclusive, and healthy community.\n\n## Our Standards\n\nExamples of behavior that contributes to a positive environment for our\ncommunity include:\n\n* Demonstrating empathy and kindness toward other people\n* Being respectful of differing opinions, viewpoints, and experiences\n* Giving and gracefully accepting constructive feedback\n* Accepting responsibility and apologizing to those affected by our mistakes,\n  and learning from the experience\n* Focusing on what is best not just for us as individuals, but for the overall\n  community\n\nExamples of unacceptable behavior include:\n\n* The use of sexualized language or imagery, and sexual attention or advances of\n  any kind\n* Trolling, insulting or derogatory comments, and personal or political attacks\n* Public or private harassment\n* Publishing others' private information, such as a physical or email address,\n  without their explicit permission\n* Other conduct which could reasonably be considered inappropriate in a\n  professional setting\n\n## Enforcement Responsibilities\n\nCommunity leaders are responsible for clarifying and enforcing our standards of\nacceptable behavior and will take appropriate and fair corrective action in\nresponse to any behavior that they deem inappropriate, threatening, offensive,\nor harmful.\n\nCommunity leaders have the right and responsibility to remove, edit, or reject\ncomments, commits, code, wiki edits, issues, and other contributions that are\nnot aligned to this Code of Conduct, and will communicate reasons for moderation\ndecisions when appropriate.\n\n## Scope\n\nThis Code of Conduct applies within all community spaces, and also applies when\nan individual is officially representing the community in public spaces.\nExamples of representing our community include using an official e-mail address,\nposting via an official social media account, or acting as an appointed\nrepresentative at an online or offline event.\n\n## Enforcement\n\nInstances of abusive, harassing, or otherwise unacceptable behavior may be\nreported to the community leaders responsible for enforcement at\n[wangluting@buaa.edu.cn](mailto:wangluting@buaa.edu.cn).\nAll complaints will be reviewed and investigated promptly and fairly.\n\nAll community leaders are obligated to respect the privacy and security of the\nreporter of any incident.\n\n## Enforcement Guidelines\n\nCommunity leaders will follow these Community Impact Guidelines in determining\nthe consequences for any action they deem in violation of this Code of Conduct:\n\n### 1. Correction\n\n**Community Impact**: Use of inappropriate language or other behavior deemed\nunprofessional or unwelcome in the community.\n\n**Consequence**: A private, written warning from community leaders, providing\nclarity around the nature of the violation and an explanation of why the\nbehavior was inappropriate. A public apology may be requested.\n\n### 2. Warning\n\n**Community Impact**: A violation through a single incident or series of\nactions.\n\n**Consequence**: A warning with consequences for continued behavior. No\ninteraction with the people involved, including unsolicited interaction with\nthose enforcing the Code of Conduct, for a specified period of time. This\nincludes avoiding interactions in community spaces as well as external channels\nlike social media. Violating these terms may lead to a temporary or permanent\nban.\n\n### 3. Temporary Ban\n\n**Community Impact**: A serious violation of community standards, including\nsustained inappropriate behavior.\n\n**Consequence**: A temporary ban from any sort of interaction or public\ncommunication with the community for a specified period of time. No public or\nprivate interaction with the people involved, including unsolicited interaction\nwith those enforcing the Code of Conduct, is allowed during this period.\nViolating these terms may lead to a permanent ban.\n\n### 4. Permanent Ban\n\n**Community Impact**: Demonstrating a pattern of violation of community\nstandards, including sustained inappropriate behavior, harassment of an\nindividual, or aggression toward or disparagement of classes of individuals.\n\n**Consequence**: A permanent ban from any sort of public interaction within the\ncommunity.\n\n## Attribution\n\nThis Code of Conduct is adapted from the [Contributor Covenant][homepage],\nversion 2.1, available at\n[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].\n\nCommunity Impact Guidelines were inspired by\n[Mozilla's code of conduct enforcement ladder][Mozilla CoC].\n\nFor answers to common questions about this code of conduct, see the FAQ at\n[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at\n[https://www.contributor-covenant.org/translations][translations].\n\n[homepage]: https://www.contributor-covenant.org\n[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html\n[Mozilla CoC]: https://github.com/mozilla/diversity\n[FAQ]: https://www.contributor-covenant.org/faq\n[translations]: https://www.contributor-covenant.org/translations\n"
  },
  {
    "path": ".github/CONTRIBUTING.md",
    "content": "# Contribution Guidelines\n\nYou’re more than welcome to contribute to the project!\nTo keep things organized and smooth, please follow these guidelines.\n\n## Awesomeness\n\nAwesome works in the field of detection KD are those with a strong influence.\nThat said, the field is still under rapid development, so newly published works with high potential are also acceptable, for example works from top conferences like CVPR.\nBefore adding a new work to the list, the awesomeness should be discussed in the pull request.\n\n## Formatting\n\nEach work should be formatted as below:\n\n```md\nMethodName. *ConfName YYYY*.\n\\[[source](URL)\\]\n<[repo](URL)>\n\\- Description\n\n- Paper Title\n- FirstName LastName and FirstName LastName\n```\n\nThe rendered output should look like:\n\n> KD. *NeurIPS 2015*.\n> \\[[arXiv](http://arxiv.org/abs/1503.02531)\\]\n> \\- Description\n>\n> - Distilling the Knowledge in a Neural Network\n> - Geoffrey Hinton and Oriol Vinyals and Jeff Dean\n\nIf the work does not have a `MethodName`, the field can be omitted.\nA work can have multiple `source` fields.\n\n## More Information\n\nFor more information, please refer to the [Contribution Guidelines of Awesome](https://github.com/sindresorhus/awesome/blob/main/contributing.md).\n"
  },
  {
    "path": ".github/workflows/lint.yaml",
    "content": "name: lint\non:\n  pull_request:\n  push:\n    branches:\n      - main\n\njobs:\n  lint:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v2\n      - uses: DavidAnson/markdownlint-cli2-action@v11\n"
  },
  {
    "path": ".markdownlint-cli2.yaml",
    "content": "config:\n  MD013: false\n"
  },
  {
    "path": ".tool-versions",
    "content": "nodejs 16.5.0\n"
  },
  {
    "path": ".vscode/settings.json",
    "content": "{\n    \"cSpell.words\": [\n        \"DRKD\",\n        \"DSIG\",\n        \"FGFI\",\n        \"SSIM\",\n        \"TADF\",\n        \"Xplore\"\n    ]\n}\n"
  },
  {
    "path": "LICENSE",
    "content": "Creative Commons Legal Code\n\nCC0 1.0 Universal\n\n    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE\n    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN\n    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS\n    INFORMATION ON AN \"AS-IS\" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES\n    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS\n    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM\n    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED\n    HEREUNDER.\n\nStatement of Purpose\n\nThe laws of most jurisdictions throughout the world automatically confer\nexclusive Copyright and Related Rights (defined below) upon the creator\nand subsequent owner(s) (each and all, an \"owner\") of an original work of\nauthorship and/or a database (each, a \"Work\").\n\nCertain owners wish to permanently relinquish those rights to a Work for\nthe purpose of contributing to a commons of creative, cultural and\nscientific works (\"Commons\") that the public can reliably and without fear\nof later claims of infringement build upon, modify, incorporate in other\nworks, reuse and redistribute as freely as possible in any form whatsoever\nand for any purposes, including without limitation commercial purposes.\nThese owners may contribute to the Commons to promote the ideal of a free\nculture and the further production of creative, cultural and scientific\nworks, or to gain reputation or greater distribution for their Work in\npart through the use and efforts of others.\n\nFor these and/or other purposes and motivations, and without any\nexpectation of additional consideration or compensation, the person\nassociating CC0 with a Work (the \"Affirmer\"), to the extent that he or she\nis an owner of Copyright and Related Rights in the Work, voluntarily\nelects to apply CC0 to the Work and publicly distribute the Work under its\nterms, with knowledge of his or her Copyright and Related Rights in the\nWork and the meaning and intended legal effect of CC0 on those rights.\n\n1. Copyright and Related Rights. A Work made available under CC0 may be\nprotected by copyright and related or neighboring rights (\"Copyright and\nRelated Rights\"). Copyright and Related Rights include, but are not\nlimited to, the following:\n\n  i. the right to reproduce, adapt, distribute, perform, display,\n     communicate, and translate a Work;\n ii. moral rights retained by the original author(s) and/or performer(s);\niii. publicity and privacy rights pertaining to a person's image or\n     likeness depicted in a Work;\n iv. rights protecting against unfair competition in regards to a Work,\n     subject to the limitations in paragraph 4(a), below;\n  v. rights protecting the extraction, dissemination, use and reuse of data\n     in a Work;\n vi. database rights (such as those arising under Directive 96/9/EC of the\n     European Parliament and of the Council of 11 March 1996 on the legal\n     protection of databases, and under any national implementation\n     thereof, including any amended or successor version of such\n     directive); and\nvii. other similar, equivalent or corresponding rights throughout the\n     world based on applicable law or treaty, and any national\n     implementations thereof.\n\n2. Waiver. To the greatest extent permitted by, but not in contravention\nof, applicable law, Affirmer hereby overtly, fully, permanently,\nirrevocably and unconditionally waives, abandons, and surrenders all of\nAffirmer's Copyright and Related Rights and associated claims and causes\nof action, whether now known or unknown (including existing as well as\nfuture claims and causes of action), in the Work (i) in all territories\nworldwide, (ii) for the maximum duration provided by applicable law or\ntreaty (including future time extensions), (iii) in any current or future\nmedium and for any number of copies, and (iv) for any purpose whatsoever,\nincluding without limitation commercial, advertising or promotional\npurposes (the \"Waiver\"). Affirmer makes the Waiver for the benefit of each\nmember of the public at large and to the detriment of Affirmer's heirs and\nsuccessors, fully intending that such Waiver shall not be subject to\nrevocation, rescission, cancellation, termination, or any other legal or\nequitable action to disrupt the quiet enjoyment of the Work by the public\nas contemplated by Affirmer's express Statement of Purpose.\n\n3. Public License Fallback. Should any part of the Waiver for any reason\nbe judged legally invalid or ineffective under applicable law, then the\nWaiver shall be preserved to the maximum extent permitted taking into\naccount Affirmer's express Statement of Purpose. In addition, to the\nextent the Waiver is so judged Affirmer hereby grants to each affected\nperson a royalty-free, non transferable, non sublicensable, non exclusive,\nirrevocable and unconditional license to exercise Affirmer's Copyright and\nRelated Rights in the Work (i) in all territories worldwide, (ii) for the\nmaximum duration provided by applicable law or treaty (including future\ntime extensions), (iii) in any current or future medium and for any number\nof copies, and (iv) for any purpose whatsoever, including without\nlimitation commercial, advertising or promotional purposes (the\n\"License\"). The License shall be deemed effective as of the date CC0 was\napplied by Affirmer to the Work. Should any part of the License for any\nreason be judged legally invalid or ineffective under applicable law, such\npartial invalidity or ineffectiveness shall not invalidate the remainder\nof the License, and in such case Affirmer hereby affirms that he or she\nwill not (i) exercise any of his or her remaining Copyright and Related\nRights in the Work or (ii) assert any associated claims and causes of\naction with respect to the Work, in either case contrary to Affirmer's\nexpress Statement of Purpose.\n\n4. Limitations and Disclaimers.\n\n a. No trademark or patent rights held by Affirmer are waived, abandoned,\n    surrendered, licensed or otherwise affected by this document.\n b. Affirmer offers the Work as-is and makes no representations or\n    warranties of any kind concerning the Work, express, implied,\n    statutory or otherwise, including without limitation warranties of\n    title, merchantability, fitness for a particular purpose, non\n    infringement, or the absence of latent or other defects, accuracy, or\n    the present or absence of errors, whether or not discoverable, all to\n    the greatest extent permissible under applicable law.\n c. Affirmer disclaims responsibility for clearing rights of other persons\n    that may apply to the Work or any use thereof, including without\n    limitation any person's Copyright and Related Rights in the Work.\n    Further, Affirmer disclaims responsibility for obtaining any necessary\n    consents, permissions or other rights required for any use of the\n    Work.\n d. Affirmer understands and acknowledges that Creative Commons is not a\n    party to this document and has no duty or obligation with respect to\n    this CC0 or use of the Work.\n"
  },
  {
    "path": "README.md",
    "content": "# Awesome Knowledge Distillation for Object Detection [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)\n\n[![lint](https://github.com/LutingWang/awesome-knowledge-distillation-for-object-detection/actions/workflows/lint.yaml/badge.svg)](https://github.com/LutingWang/awesome-knowledge-distillation-for-object-detection/actions/workflows/lint.yaml)\n\nA curated list of **awesome** distillation techniques designed for object detectors.\n\nParameters compression and accuracy boosting are core problems for object detection towards practical application, where knowledge distillation (KD) is one of the most popular solutions.\nKD aims at training the compact model (student) by transferring knowledge from a high-capacity model (teacher).\nPapers and codes are listed.\n\n## Contents\n\n- [Knowledge Distillation for General Object Detectors](#knowledge-distillation-for-general-object-detectors)\n  - [Feature Distillation](#feature-distillation)\n    - [Foreground Masks](#foreground-masks)\n      - [Ground Truth Guided](#ground-truth-guided)\n      - [Prediction Guided](#prediction-guided)\n      - [Attention Guided](#attention-guided)\n      - [Miscellaneous Foreground Masks](#miscellaneous-foreground-masks)\n    - [Miscellaneous Feature Distillation](#miscellaneous-feature-distillation)\n  - [Instance Distillation](#instance-distillation)\n  - [Label Assignment Distillation](#label-assignment-distillation)\n  - [Balancing between Tasks](#balancing-between-tasks)\n  - [Miscellaneous Knowledge Distillation for General Object Detectors](#miscellaneous-knowledge-distillation-for-general-object-detectors)\n- [Knowledge Distillation for Specific Object Detectors](#knowledge-distillation-for-specific-object-detectors)\n  - [Knowledge Distillation for GFL](#knowledge-distillation-for-gfl)\n  - [Knowledge Distillation for DETR](#knowledge-distillation-for-detr)\n- [Knowledge Distillation for Heterogeneous Object Detectors](#knowledge-distillation-for-heterogeneous-object-detector-pairs)\n- [Multi Teacher Knowledge Distillation for Object Detectors](#multi-teacher-knowledge-distillation-for-object-detectors)\n- [Teacher Free Knowledge Distillation for Object Detectors](#teacher-free-knowledge-distillation-for-object-detectors)\n- [Miscellaneous](#miscellaneous)\n- [Newly Published Papers](#newly-published-papers)\n\n## Knowledge Distillation for General Object Detectors\n\n*NeurIPS 2017*.\n\\[[NeurIPS](https://proceedings.neurips.cc/paper/2017/hash/e1e32e235eee1f970470a3a6658dfdd5-Abstract.html)\\]\n\\- A new framework to learn compact and fast object detection networks with improved accuracy using knowledge distillation and hint learning.\n\n- Learning Efficient Object Detection Models with Knowledge Distillation\n- Guobin Chen and Wongun Choi and Xiang Yu and Tony Han and Manmohan Chandraker\n\nMimic. *CVPR 2017*.\n\\[[CVF](http://openaccess.thecvf.com/content_cvpr_2017/html/Li_Mimicking_Very_Efficient_CVPR_2017_paper.html)\\]\n\\[[IEEE Xplore](http://ieeexplore.ieee.org/abstract/document/8100259/)\\]\n\\- A fully convolutional feature mimic framework to train very efficient CNN based detectors, which do not need ImageNet pre-training and achieve competitive performance as the large and slow models.\n\n- Mimicking Very Efficient Network for Object Detection\n- Quanquan Li and Shengying Jin and Junjie Yan\n\n### Feature Distillation\n\n#### Foreground Masks\n\n##### Ground Truth Guided\n\nFGFI. *CVPR 2019*.\n\\[[CVF](http://openaccess.thecvf.com/content_CVPR_2019/html/Wang_Distilling_Object_Detectors_With_Fine-Grained_Feature_Imitation_CVPR_2019_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/8953432/)\\]\n\\[[arXiv](http://arxiv.org/abs/1906.03609)\\]\n<[GitHub](https://github.com/twangnh/Distilling-Object-Detectors)>\n\\- A fine-grained feature imitation method exploiting the cross-location discrepancy of feature response.\n\n- Distilling Object Detectors With Fine-Grained Feature Imitation\n- Tao Wang and Li Yuan and Xiaopeng Zhang and Jiashi Feng\n\nDeFeat. *CVPR 2021*.\n\\[[CVF](http://openaccess.thecvf.com/content/CVPR2021/html/Guo_Distilling_Object_Detectors_via_Decoupled_Features_CVPR_2021_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/9578919/)\\]\n\\[[arXiv](http://arxiv.org/abs/2103.14475)\\]\n\\- A novel distillation algorithm via decoupled features for learning a better student detector.\n\n- Distilling Object Detectors via Decoupled Features\n- Jianyuan Guo and Kai Han and Yunhe Wang and Han Wu and Xinghao Chen and Chunjing Xu and Chang Xu\n\n##### Prediction Guided\n\nFRS. *NeurIPS 2021*.\n\\[[NeurIPS](https://proceedings.neurips.cc/paper_files/paper/2021/hash/29c0c0ee223856f336d7ea8052057753-Abstract.html)\\]\n\\[[OpenReview](https://openreview.net/forum?id=_bOfK2k_7R)\\]\n\\[[arXiv](http://arxiv.org/abs/2111.00674)\\]\n\\- A novel Feature-Richness Score (FRS) method to choose important features that improve generalized detectability during distilling.\n\n- Distilling Object Detectors with Feature Richness\n- Zhixing Du and Rui Zhang and Ming Chang and Xishan Zhang and Shaoli Liu and Tianshi Chen and Yunji Chen\n\nPGD. *ECCV 2022*.\n\\[[ECVA](https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136690123.pdf)\\]\n\\[[Springer](https://link.springer.com/chapter/10.1007/978-3-031-20077-9_8)\\]\n\\[[arXiv](http://arxiv.org/abs/2203.05469)\\]\n<[GitHub](https://github.com/ChenhongyiYang/PGD)>\n\\- Distill on the key predictive regions of the teacher.\n\n- Prediction-Guided Distillation for Dense Object Detection\n- Chenhongyi Yang and Mateusz Ochal and Amos Storkey and Elliot J Crowley\n\nTBD. *PR*.\n\\[[ScienceDirect](https://www.sciencedirect.com/science/article/abs/pii/S0031320323000213)\\]\n\\[[arXiv](https://arxiv.org/abs/2208.03006)\\]\n\\- Alleviates the misalignment between classification score and localization quality via Harmony Score and Task-Balanced Distillation.\n\n- Task-balanced distillation for object detection\n- Ruining Tang and Zhenyu Liu and Yangguang Li and Yiguo Song and Hui Liu and Qide Wang and Jing Shao and Guifang Duan and Jianrong Tan\n\n##### Attention Guided\n\nFKD. *ICLR 2021*.\n\\[[OpenReview](https://openreview.net/forum?id=uKhGRvM8QNH)\\]\n<[GitHub](https://github.com/ArchipLab-LinfengZhang/Object-Detection-Knowledge-Distillation-ICLR2021)>\n\\- Attention-guided distillation and non-local distillation.\n\n- Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors\n- Chunting Zhou and Graham Neubig and Jiatao Gu\n\nFKD. *TPAMI*.\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/10198386/)\\]\n\\- A structured knowledge distillation scheme, including attention-guided distillation and non-local distillation.\n\n- Structured Knowledge Distillation for Accurate and Efficient Object Detection\n- Linfeng Zhang and Kaisheng Ma\n\nFGD. *CVPR 2022*.\n\\[[CVF](https://openaccess.thecvf.com/content/CVPR2022/html/Yang_Focal_and_Global_Knowledge_Distillation_for_Detectors_CVPR_2022_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/9879869/)\\]\n\\[[arXiv](http://arxiv.org/abs/2111.11837)\\]\n<[GitHub](https://github.com/yzd-v/FGD)>\n\\- Focal distillation separates the fore-ground and background, while global distillation rebuilds the relation between different pixels and transfers it from teachers to students.\n\n- Focal and Global Knowledge Distillation for Detectors\n- Zhendong Yang and Zhe Li and Xiaohu Jiang and Yuan Gong and Zehuan Yuan and Danpei Zhao and Chun Yuan\n\nGLAMD. *ECCV 2022*.\n\\[[ECVA](https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136700456.pdf)\\]\n\\[[Springer](https://link.springer.com/chapter/10.1007/978-3-031-20080-9_27)\\]\n\\- Divide the feature maps into several patches and apply an attention mechanism for both the entire feature area and each patch.\n\n- GLAMD: Global and Local Attention Mask Distillation for Object Detectors\n- Younho Jang and Wheemyung Shin and Jinbeom Kim and Simon Woo and Sung-Ho Bae\n\n##### Miscellaneous Foreground Masks\n\nCD. *ICCV 2021*.\n\\[[CVF](http://openaccess.thecvf.com/content/ICCV2021/html/Shu_Channel-Wise_Knowledge_Distillation_for_Dense_Prediction_ICCV_2021_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/9710762/)\\]\n\\[[arXiv](http://arxiv.org/abs/2011.13256)\\]\n<[GitHub](https://github.com/irfanICMLL/TorchDistiller/tree/main/SemSeg-distill)>\n\\- Normalize the activation map of each channel to obtain a soft probability map.\n\n- Channel-wise Knowledge Distillation for Dense Prediction\n- Changyong Shu and Yifan Liu and Jianfei Gao and Zheng Yan and Chunhua Shen\n\n#### Miscellaneous Feature Distillation\n\nDRKD. *IJCAI 2023*.\n\\[[arXiv](https://arxiv.org/abs/2302.05637)\\]\n\\- Dual relation knowledge distillation, including pixel-wise relation distillation and instance-wise relation distillation\n\n- Dual Relation Knowledge Distillation for Object Detection\n- Zhenliang Ni and Fukui Yang and Shengzhao Wen and Gang Zhang\n\n### Instance Distillation\n\nGID. *CVPR 2021*.\n\\[[CVF](http://openaccess.thecvf.com/content/CVPR2021/html/Dai_General_Instance_Distillation_for_Object_Detection_CVPR_2021_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/9577671/)\\]\n\\[[arXiv](http://arxiv.org/abs/2103.02340)\\]\n\\- A novel distillation method for detection tasks based on discriminative instances without considering the positive or negative distinguished by GT.\n\n- General Instance Distillation for Object Detection\n- Xing Dai and Zeren Jiang and Zhao Wu and Yiping Bao and Zhicheng Wang and Si Liu and Erjin Zhou\n\nDSIG. *ICCV 2021*.\n\\[[CVF](http://openaccess.thecvf.com/content/ICCV2021/html/Chen_Deep_Structured_Instance_Graph_for_Distilling_Object_Detectors_ICCV_2021_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/9711100/)\\]\n\\[[arXiv](http://arxiv.org/abs/2109.12862)\\]\n<[GitHub](https://github.com/dvlab-research/Dsig)>\n\\- A simple knowledge structure to exploit and encode information inside the detection system to  facilitate detector knowledge distillation.\n\n- Deep Structured Instance Graph for Distilling Object Detectors\n- Yixin Chen and Pengguang Chen and Shu Liu and Liwei Wang and Jiaya Jia\n\nICD. *NeurIPS 2021*.\n\\[[NeurIPS](https://proceedings.neurips.cc/paper_files/paper/2021/hash/892c91e0a653ba19df81a90f89d99bcd-Abstract.html)\\]\n\\[[OpenReview](https://openreview.net/forum?id=k7aeAz4Vbb)\\]\n\\[[arXiv](http://arxiv.org/abs/2110.12724)\\]\n<[GitHub](https://github.com/MegEngine/ICD)>\n\\- An instance-conditional distillation framework to find desired knowledge.\n\n- Instance-Conditional Knowledge Distillation for Object Detection\n- Zijian Kang and Peizhen Zhang and Xiangyu Zhang and Jian Sun and Nanning Zheng\n\n### Label Assignment Distillation\n\nLAD. *WACV 2022*.\n\\[[CVF](https://openaccess.thecvf.com/content/WACV2022/html/Nguyen_Improving_Object_Detection_by_Label_Assignment_Distillation_WACV_2022_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/9706993/)\\]\n\\[[arXiv](http://arxiv.org/abs/2108.10520)\\]\n<[MMDet](https://github.com/open-mmlab/mmdetection/tree/master/configs/lad)>\n\\- Use the teacher network to generate labels for the student, through the hard labels dynamically  assigned by the teacher.\n\n- Improving Object Detection by Label Assignment Distillation\n- Chuong H. Nguyen and Thuy C. Nguyen and Tuan N. Tang and Nam L. H. Phan\n\n### Balancing between Tasks\n\nTADF.\n\\[[arXiv](http://arxiv.org/abs/2006.13108)\\]\n\\- A general distillation framework that adaptively transfers knowledge from teacher to student  according to the task specific prior.\n\n- Distilling Object Detectors with Task Adaptive Regularization\n- Ruoyu Sun and Fuhui Tang and Xiaopeng Zhang and Hongkai Xiong and Qi Tian\n\nBCKD. *ICCV 2023*\n\\[[CVF](https://openaccess.thecvf.com/content/ICCV2023/html/Yang_Bridging_Cross-task_Protocol_Inconsistency_for_Distillation_in_Dense_Object_Detection_ICCV_2023_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/10377607)\\]\n\\[[arXiv](http://arxiv.org/abs/2308.14286)\\]\n<[GitHub](https://github.com/TinyTigerPan/BCKD)>\n\\- A novel distillation method with cross-task consistent protocols, tailored for the dense object detection.\n\n- Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection\n- Longrong Yang and Xianpan Zhou and Xuewei Li and Liang Qiao and Zheyang Li and Ziwei Yang and Gaoang Wang and Xi Li\n\n### Miscellaneous Knowledge Distillation for General Object Detectors\n\n*AAAI 2022*.\n\\[[AAAI](https://ojs.aaai.org/index.php/AAAI/article/view/20018)\\]\n\\[[arXiv](http://arxiv.org/abs/2112.04840)\\]\n\\- RM takes the rank of candidate boxes from teachers as a new form of knowledge to distill. PFI attempts to correlate feature differences with prediction differences.\n\n- Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-Guided Feature Imitation\n- Gang Li and Xiang Li and Yujie Wang and Shanshan Zhang and Yichao Wu and Ding Liang\n\n*NeurIPS 2022*.\n\\[[OpenReview](https://openreview.net/forum?id=O3My0RK9s_R)\\]\n\\[[arXiv](https://arxiv.org/abs/2211.13133v1)\\]\n<[GitHub](https://github.com/kornia/kornia)>\n\\- By taking into account additional contrast and structural cues, feature importance, correlation, and spatial dependence in the feature space are considered in the loss formulation.\n\n- Structural Knowledge Distillation for Object Detection\n- Philip De Rijk and Lukas Schneider and Marius Cordts and Dariu M Gavrila\n\nCrossKD. CVPR 2024.\n\\[[CVF](https://openaccess.thecvf.com/content/CVPR2024/html/Wang_CrossKD_Cross-Head_Knowledge_Distillation_for_Object_Detection_CVPR_2024_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/document/10654891)\\]\n\\[[arXiv](https://arxiv.org/abs/2306.11369)\\]\n<[GitHub](https://github.com/jbwang1997/CrossKD)>\n\\- Delivers the intermediate features of the student's detection head to the teacher's detection head\n\n- CrossKD: Cross-Head Knowledge Distillation for Dense Object Detection\n- Jiabao Wang and Yuming Chen and Zhaohui Zheng and Xiang Li and Ming-Ming Cheng and Qibin Hou\n\n## Knowledge Distillation for Specific Object Detectors\n\n### Knowledge Distillation for GFL\n\nLD. *CVPR 2022*.\n\\[[CVF](https://openaccess.thecvf.com/content/CVPR2022/html/Zheng_Localization_Distillation_for_Dense_Object_Detection_CVPR_2022_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/9878414/)\\]\n\\[[arXiv](http://arxiv.org/abs/2102.12252)\\]\n<[GitHub](https://github.com/HikariTJU/LD)>\n<[MMDet](https://github.com/open-mmlab/mmdetection/tree/master/configs/ld)>\n\\- Standard KD by adopting the general localization representation of bounding box.\n\n- Localization Distillation for Dense Object Detection\n- Zhaohui Zheng and Rongguang Ye and Ping Wang and Jun Wang and Dongwei Ren and Wangmeng Zuo\n\n### Knowledge Distillation for DETR\n\nDETRDistill. *ICCV 2023*.\n\\[[CVF](https://openaccess.thecvf.com/content/ICCV2023/html/Chang_DETRDistill_A_Universal_Knowledge_Distillation_Framework_for_DETR-families_ICCV_2023_paper.html)\\]\n\\[[arXiv](http://arxiv.org/abs/2211.10156)\\]\n\\- A novel knowledge distillation dedicated to DETR-families.\n\n- DETRDistill: A Universal Knowledge Distillation Framework for DETR-families\n- Jiahao Chang and Shuo Wang and Guangkai Xu and Zehui Chen and Chenhongyi Yang and Feng Zhao\n\nD^3^ETR.\n\\[[arXiv](http://arxiv.org/abs/2211.09768)\\]\n\\- Distills knowledge in decoder predictions and attention maps from the teachers to students.\n\n- D^3^ETR: Decoder Distillation for Detection Transformer\n- Xiaokang Chen and Jiahui Chen and Yan Liu and Gang Zeng\n\nKD-DETR.\n\\[[arXiv](http://arxiv.org/abs/2211.08071)\\]\n\\- A general knowledge distillation paradigm for DETR with consistent distillation points sampling.\n\n- Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling\n- Yu Wang and Xin Li and Shengzhao Wen and Fukui Yang and Wanping Zhang and Gang Zhang and Haocheng Feng and Junyu Han and Errui Ding\n\n## Knowledge Distillation for Heterogeneous Object Detector Pairs\n\nG-DetKD. *ICCV 2021*.\n\\[[CVF](http://openaccess.thecvf.com/content/ICCV2021/html/Yao_G-DetKD_Towards_General_Distillation_Framework_for_Object_Detectors_via_Contrastive_ICCV_2021_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/9711293/)\\]\n\\[[arXiv](http://arxiv.org/abs/2108.07482)\\]\n\\- A novel semantic-guided feature imitation technique, which automatically performs soft matching between feature pairs across all pyramid levels to provide the optimal guidance to the student.\n\n- G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation\n- Lewei Yao and Renjie Pi and Hang Xu and Wei Zhang and Zhenguo Li and Tong Zhang\n\nHEAD. *ECCV 2022*.\n\\[[ECVA](https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136690310.pdf)\\]\n\\[[Springer](https://link.springer.com/chapter/10.1007/978-3-031-20077-9_19)\\]\n\\[[arXiv](https://arxiv.org/abs/2207.05345)\\]\n<[GitHub](https://github.com/LutingWang/HEAD)>\n\\- HEtero-Assists Distillation leveraging heterogeneous detection heads as assistants to guide the optimization of the student detector.\n\n- HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectors\n- Luting Wang and Xiaojie Li and Yue Liao and Zeren Jiang and Jianlong Wu and Fei Wang and Chen Qian and Si Liu\n\nPKD. *NeurIPS 2022*.\n\\[[OpenReview](https://openreview.net/forum?id=Q9dj3MzY1o7)\\]\n\\[[arXiv](https://arxiv.org/abs/2207.02039v2)\\]\n<[GitHub](https://github.com/open-mmlab/mmrazor)>\n\\- Imitate features with Pearson Correlation Coefficient to focus on the relational information from the teacher and relax constraints on the magnitude of the features.\n\n- PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient\n- Weihan Cao and Yifan Zhang and Jianfei Gao and Anda Cheng and Ke Cheng and Jian Cheng\n\nUniKD. ICCV 2023.\n\\[[CVF](https://openaccess.thecvf.com/content/ICCV2023/html/Lao_UniKD_Universal_Knowledge_Distillation_for_Mimicking_Homogeneous_or_Heterogeneous_Object_ICCV_2023_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/document/10376876)\\]\n\\- Proposes a universal knowledge distillation framework that extracts detection-relevant knowledge embeddings via Adaptive Knowledge Extractor (AKE), enabling efficient knowledge transfer between homogeneous/heterogeneous detectors.\n\n- UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors\n- Shanshan Lao and Guanglu Song and Boxiao Liu and Yu Liu and Yujiu Yang\n\nDetKDS. ICML 2024.\n\\[[ICML](https://icml.cc/virtual/2024/poster/34018)\\]\n<[GitHub](https://github.com/lliai/DetKDS)>\n\\- The first distillation search framework for object detection, automatically adapting to homogeneous/heterogeneous teacher-student pairs by searching for optimal distillation strategies to enhance model generalization.\n\n- DetKDS: Knowledge Distillation Search for Object Detectors\n- Lujun Li and Yufan Bao and Peijie Dong and Chuanguang Yang and Anggeng Li and Wenhan Luo and Qifeng Liu and Wei Xue and Yike Guo\n\n## Multi Teacher Knowledge Distillation for Object Detectors\n\nLLOD. ICML 2023.\n\\[[ICML](https://icml.cc/virtual/2023/poster/23867)\\]\n\\[[arXiv](https://arxiv.org/abs/2308.09105)\\]\n<[GitHub](https://github.com/Shengcao-Cao/MTPD)>\n\\- Proposes a multi-teacher progressive distillation strategy, sequentially distilling knowledge from multiple teachers to bridge model capacity gaps, especially effective for knowledge transfer between heterogeneous architectures.\n\n- Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation\n- Shengcao Cao and Mengtian Li and James Hays and Deva Ramanan and Yu-Xiong Wang and Liangyan Gui\n\n## Teacher Free Knowledge Distillation for Object Detectors\n\nMimicDet. *ECCV 2020*.\n\\[[ECVA](https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123590528.pdf)\\]\n\\[[Springer](https://link.springer.com/chapter/10.1007/978-3-030-58568-6_32)\\]\n\\[[arXiv](http://arxiv.org/abs/2009.11528)\\]\n\\- A novel and efficient framework to train a one-stage detector by directly mimic the two-stage features, aiming to bridge the accuracy gap between one-stage and two-stage detector.\n\n- MimicDet: Bridging the Gap Between One-Stage and Two-Stage Object Detection\n- Xin Lu and Quanquan Li and Buyu Li and Junjie Yan\n\nLabelEnc. *ECCV 2020*.\n\\[[ECVA](https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123700528.pdf)\\]\n\\[[Springer](https://link.springer.com/chapter/10.1007/978-3-030-58595-2_32)\\]\n\\[[arXiv](http://arxiv.org/abs/2007.03282)\\]\n<[GitHub](https://github.com/megvii-model/LabelEnc)>\n\\- A new intermediate supervision method to boost the training of object detection systems.\n\n- LabelEnc: A New Intermediate Supervision Method for Object Detection\n- Miao Hao and Yitao Liu and Xiangyu Zhang and Jian Sun\n\nHEAD. *ECCV 2022*.\n\\[[ECVA](https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136690310.pdf)\\]\n\\[[Springer](https://link.springer.com/chapter/10.1007/978-3-031-20077-9_19)\\]\n\\[[arXiv](https://arxiv.org/abs/2207.05345)\\]\n<[GitHub](https://github.com/LutingWang/HEAD)>\n\\- HEtero-Assists Distillation leveraging heterogeneous detection heads as assistants to guide the optimization of the student detector.\n\n- HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectors\n- Luting Wang and Xiaojie Li and Yue Liao and Zeren Jiang and Jianlong Wu and Fei Wang and Chen Qian and Si Liu\n\nLGD. *AAAI 2022*.\n\\[[AAAI](https://ojs.aaai.org/index.php/AAAI/article/view/20240)\\]\n\\[[arXiv](http://arxiv.org/abs/2109.11496)\\]\n\\- The first self-distillation framework for general object detection.\n\n- LGD: Label-Guided Self-Distillation for Object Detection\n- Peizhen Zhang and Zijian Kang and Tong Yang and Xiangyu Zhang and Nanning Zheng and Jian Sun\n\nSSD-Det. *ICCV 2023*.\n\\[[CVF](https://openaccess.thecvf.com/content/ICCV2023/html/Wu_Spatial_Self-Distillation_for_Object_Detection_with_Inaccurate_Bounding_Boxes_ICCV_2023_paper.html)\\]\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/10377611)\\]\n\\[[arXiv](http://arxiv.org/abs/2307.12101)\\]\n\\- Mine spatial information to refine the inaccurate box in a self-distillation fashion.\n\n- Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxes\n- Di Wu and Pengfei Chen and Xuehui Yu and Guorong Li and Zhenjun Han and Jianbin Jiao\n\n## Miscellaneous\n\n*TPAMI*.\n\\[[IEEE Xplore](https://ieeexplore.ieee.org/abstract/document/10070820)\\]\n\\- A comprehensive survey of KD-based OD models.\n\n- When Object Detection Meets Knowledge Distillation: A Survey\n- Zhihui Li and Pengfei Xu and Xiaojun Chang and Luyao Yang and Yuanyuan Zhang and Lina Yao and Xiaojiang Chen\n\nScaleKD. *CVPR 2023*.\n\\[[CVF](https://openaccess.thecvf.com/content/CVPR2023/html/Zhu_ScaleKD_Distilling_Scale-Aware_Knowledge_in_Small_Object_Detector_CVPR_2023_paper.html)\\]\n\\- Consists of a Scale-Decoupled Feature distillation module and a Cross-Scale Assistant.\n\n- ScaleKD: Distilling Scale-Aware Knowledge in Small Object Detector\n- Yichen Zhu and Qiqi Zhou and Ning Liu and Zhiyuan Xu and Zhicai Ou and Xiaofeng Mou and Jian Tang\n\n## Newly Published Papers\n"
  }
]