[ { "path": ".gitignore", "content": "*.log\n*.pyc\n*.xml\n*.json\n*.mat\n*.eps\n*.cpython-37.pyc\n/DLPan-Toolbox/01-DL-toolbox(Pytorch)/results/*\n/bak/*\n" }, { "path": "01-DL-toolbox(Pytorch)/LICENSE", "content": "GNU GENERAL PUBLIC LICENSE\n Version 2, June 1991\n\n Copyright (C) 1989, 1991 Free Software Foundation, Inc.,\n 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA\n Everyone is permitted to copy and distribute verbatim copies\n of this license document, but changing it is not allowed.\n\n Preamble\n\n The licenses for most software are designed to take away your\nfreedom to share and change it. By contrast, the GNU General Public\nLicense is intended to guarantee your freedom to share and change free\nsoftware--to make sure the software is free for all its users. This\nGeneral Public License applies to most of the Free Software\nFoundation's software and to any other program whose authors commit to\nusing it. (Some other Free Software Foundation software is covered by\nthe GNU Lesser General Public License instead.) You can apply it to\nyour programs, too.\n\n When we speak of free software, we are referring to freedom, not\nprice. Our General Public Licenses are designed to make sure that you\nhave the freedom to distribute copies of free software (and charge for\nthis service if you wish), that you receive source code or can get it\nif you want it, that you can change the software or use pieces of it\nin new free programs; and that you know you can do these things.\n\n To protect your rights, we need to make restrictions that forbid\nanyone to deny you these rights or to ask you to surrender the rights.\nThese restrictions translate to certain responsibilities for you if you\ndistribute copies of the software, or if you modify it.\n\n For example, if you distribute copies of such a program, whether\ngratis or for a fee, you must give the recipients all the rights that\nyou have. You must make sure that they, too, receive or can get the\nsource code. And you must show them these terms so they know their\nrights.\n\n We protect your rights with two steps: (1) copyright the software, and\n(2) offer you this license which gives you legal permission to copy,\ndistribute and/or modify the software.\n\n Also, for each author's protection and ours, we want to make certain\nthat everyone understands that there is no warranty for this free\nsoftware. If the software is modified by someone else and passed on, we\nwant its recipients to know that what they have is not the original, so\nthat any problems introduced by others will not reflect on the original\nauthors' reputations.\n\n Finally, any free program is threatened constantly by software\npatents. We wish to avoid the danger that redistributors of a free\nprogram will individually obtain patent licenses, in effect making the\nprogram proprietary. To prevent this, we have made it clear that any\npatent must be licensed for everyone's free use or not licensed at all.\n\n The precise terms and conditions for copying, distribution and\nmodification follow.\n\n GNU GENERAL PUBLIC LICENSE\n TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION\n\n 0. This License applies to any program or other work which contains\na notice placed by the copyright holder saying it may be distributed\nunder the terms of this General Public License. The \"Program\", below,\nrefers to any such program or work, and a \"work based on the Program\"\nmeans either the Program or any derivative work under copyright law:\nthat is to say, a work containing the Program or a portion of it,\neither verbatim or with modifications and/or translated into another\nlanguage. (Hereinafter, translation is included without limitation in\nthe term \"modification\".) Each licensee is addressed as \"you\".\n\nActivities other than copying, distribution and modification are not\ncovered by this License; they are outside its scope. The act of\nrunning the Program is not restricted, and the output from the Program\nis covered only if its contents constitute a work based on the\nProgram (independent of having been made by running the Program).\nWhether that is true depends on what the Program does.\n\n 1. You may copy and distribute verbatim copies of the Program's\nsource code as you receive it, in any medium, provided that you\nconspicuously and appropriately publish on each copy an appropriate\ncopyright notice and disclaimer of warranty; keep intact all the\nnotices that refer to this License and to the absence of any warranty;\nand give any other recipients of the Program a copy of this License\nalong with the Program.\n\nYou may charge a fee for the physical act of transferring a copy, and\nyou may at your option offer warranty protection in exchange for a fee.\n\n 2. You may modify your copy or copies of the Program or any portion\nof it, thus forming a work based on the Program, and copy and\ndistribute such modifications or work under the terms of Section 1\nabove, provided that you also meet all of these conditions:\n\n a) You must cause the modified files to carry prominent notices\n stating that you changed the files and the date of any change.\n\n b) You must cause any work that you distribute or publish, that in\n whole or in part contains or is derived from the Program or any\n part thereof, to be licensed as a whole at no charge to all third\n parties under the terms of this License.\n\n c) If the modified program normally reads commands interactively\n when run, you must cause it, when started running for such\n interactive use in the most ordinary way, to print or display an\n announcement including an appropriate copyright notice and a\n notice that there is no warranty (or else, saying that you provide\n a warranty) and that users may redistribute the program under\n these conditions, and telling the user how to view a copy of this\n License. (Exception: if the Program itself is interactive but\n does not normally print such an announcement, your work based on\n the Program is not required to print an announcement.)\n\nThese requirements apply to the modified work as a whole. If\nidentifiable sections of that work are not derived from the Program,\nand can be reasonably considered independent and separate works in\nthemselves, then this License, and its terms, do not apply to those\nsections when you distribute them as separate works. But when you\ndistribute the same sections as part of a whole which is a work based\non the Program, the distribution of the whole must be on the terms of\nthis License, whose permissions for other licensees extend to the\nentire whole, and thus to each and every part regardless of who wrote it.\n\nThus, it is not the intent of this section to claim rights or contest\nyour rights to work written entirely by you; rather, the intent is to\nexercise the right to control the distribution of derivative or\ncollective works based on the Program.\n\nIn addition, mere aggregation of another work not based on the Program\nwith the Program (or with a work based on the Program) on a volume of\na storage or distribution medium does not bring the other work under\nthe scope of this License.\n\n 3. You may copy and distribute the Program (or a work based on it,\nunder Section 2) in object code or executable form under the terms of\nSections 1 and 2 above provided that you also do one of the following:\n\n a) Accompany it with the complete corresponding machine-readable\n source code, which must be distributed under the terms of Sections\n 1 and 2 above on a medium customarily used for software interchange; or,\n\n b) Accompany it with a written offer, valid for at least three\n years, to give any third party, for a charge no more than your\n cost of physically performing source distribution, a complete\n machine-readable copy of the corresponding source code, to be\n distributed under the terms of Sections 1 and 2 above on a medium\n customarily used for software interchange; or,\n\n c) Accompany it with the information you received as to the offer\n to distribute corresponding source code. (This alternative is\n allowed only for noncommercial distribution and only if you\n received the program in object code or executable form with such\n an offer, in accord with Subsection b above.)\n\nThe source code for a work means the preferred form of the work for\nmaking modifications to it. For an executable work, complete source\ncode means all the source code for all modules it contains, plus any\nassociated interface definition files, plus the scripts used to\ncontrol compilation and installation of the executable. However, as a\nspecial exception, the source code distributed need not include\nanything that is normally distributed (in either source or binary\nform) with the major components (compiler, kernel, and so on) of the\noperating system on which the executable runs, unless that component\nitself accompanies the executable.\n\nIf distribution of executable or object code is made by offering\naccess to copy from a designated place, then offering equivalent\naccess to copy the source code from the same place counts as\ndistribution of the source code, even though third parties are not\ncompelled to copy the source along with the object code.\n\n 4. You may not copy, modify, sublicense, or distribute the Program\nexcept as expressly provided under this License. Any attempt\notherwise to copy, modify, sublicense or distribute the Program is\nvoid, and will automatically terminate your rights under this License.\nHowever, parties who have received copies, or rights, from you under\nthis License will not have their licenses terminated so long as such\nparties remain in full compliance.\n\n 5. You are not required to accept this License, since you have not\nsigned it. However, nothing else grants you permission to modify or\ndistribute the Program or its derivative works. These actions are\nprohibited by law if you do not accept this License. Therefore, by\nmodifying or distributing the Program (or any work based on the\nProgram), you indicate your acceptance of this License to do so, and\nall its terms and conditions for copying, distributing or modifying\nthe Program or works based on it.\n\n 6. Each time you redistribute the Program (or any work based on the\nProgram), the recipient automatically receives a license from the\noriginal licensor to copy, distribute or modify the Program subject to\nthese terms and conditions. You may not impose any further\nrestrictions on the recipients' exercise of the rights granted herein.\nYou are not responsible for enforcing compliance by third parties to\nthis License.\n\n 7. If, as a consequence of a court judgment or allegation of patent\ninfringement or for any other reason (not limited to patent issues),\nconditions are imposed on you (whether by court order, agreement or\notherwise) that contradict the conditions of this License, they do not\nexcuse you from the conditions of this License. If you cannot\ndistribute so as to satisfy simultaneously your obligations under this\nLicense and any other pertinent obligations, then as a consequence you\nmay not distribute the Program at all. For example, if a patent\nlicense would not permit royalty-free redistribution of the Program by\nall those who receive copies directly or indirectly through you, then\nthe only way you could satisfy both it and this License would be to\nrefrain entirely from distribution of the Program.\n\nIf any portion of this section is held invalid or unenforceable under\nany particular circumstance, the balance of the section is intended to\napply and the section as a whole is intended to apply in other\ncircumstances.\n\nIt is not the purpose of this section to induce you to infringe any\npatents or other property right claims or to contest validity of any\nsuch claims; this section has the sole purpose of protecting the\nintegrity of the free software distribution system, which is\nimplemented by public license practices. Many people have made\ngenerous contributions to the wide range of software distributed\nthrough that system in reliance on consistent application of that\nsystem; it is up to the author/donor to decide if he or she is willing\nto distribute software through any other system and a licensee cannot\nimpose that choice.\n\nThis section is intended to make thoroughly clear what is believed to\nbe a consequence of the rest of this License.\n\n 8. If the distribution and/or use of the Program is restricted in\ncertain countries either by patents or by copyrighted interfaces, the\noriginal copyright holder who places the Program under this License\nmay add an explicit geographical distribution limitation excluding\nthose countries, so that distribution is permitted only in or among\ncountries not thus excluded. In such case, this License incorporates\nthe limitation as if written in the body of this License.\n\n 9. The Free Software Foundation may publish revised and/or new versions\nof the General Public License from time to time. Such new versions will\nbe similar in spirit to the present version, but may differ in detail to\naddress new problems or concerns.\n\nEach version is given a distinguishing version number. If the Program\nspecifies a version number of this License which applies to it and \"any\nlater version\", you have the option of following the terms and conditions\neither of that version or of any later version published by the Free\nSoftware Foundation. If the Program does not specify a version number of\nthis License, you may choose any version ever published by the Free Software\nFoundation.\n\n 10. If you wish to incorporate parts of the Program into other free\nprograms whose distribution conditions are different, write to the author\nto ask for permission. For software which is copyrighted by the Free\nSoftware Foundation, write to the Free Software Foundation; we sometimes\nmake exceptions for this. Our decision will be guided by the two goals\nof preserving the free status of all derivatives of our free software and\nof promoting the sharing and reuse of software generally.\n\n NO WARRANTY\n\n 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY\nFOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN\nOTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES\nPROVIDE THE PROGRAM \"AS IS\" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED\nOR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF\nMERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS\nTO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE\nPROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,\nREPAIR OR CORRECTION.\n\n 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING\nWILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR\nREDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,\nINCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING\nOUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED\nTO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY\nYOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER\nPROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE\nPOSSIBILITY OF SUCH DAMAGES.\n\n END OF TERMS AND CONDITIONS\n\n How to Apply These Terms to Your New Programs\n\n If you develop a new program, and you want it to be of the greatest\npossible use to the public, the best way to achieve this is to make it\nfree software which everyone can redistribute and change under these terms.\n\n To do so, attach the following notices to the program. It is safest\nto attach them to the start of each source file to most effectively\nconvey the exclusion of warranty; and each file should have at least\nthe \"copyright\" line and a pointer to where the full notice is found.\n\n \n Copyright (C) \n\n This program is free software; you can redistribute it and/or modify\n it under the terms of the GNU General Public License as published by\n the Free Software Foundation; either version 2 of the License, or\n (at your option) any later version.\n\n This program is distributed in the hope that it will be useful,\n but WITHOUT ANY WARRANTY; without even the implied warranty of\n MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n GNU General Public License for more details.\n\n You should have received a copy of the GNU General Public License along\n with this program; if not, write to the Free Software Foundation, Inc.,\n 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.\n\nAlso add information on how to contact you by electronic and paper mail.\n\nIf the program is interactive, make it output a short notice like this\nwhen it starts in an interactive mode:\n\n Gnomovision version 69, Copyright (C) year name of author\n Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.\n This is free software, and you are welcome to redistribute it\n under certain conditions; type `show c' for details.\n\nThe hypothetical commands `show w' and `show c' should show the appropriate\nparts of the General Public License. Of course, the commands you use may\nbe called something other than `show w' and `show c'; they could even be\nmouse-clicks or menu items--whatever suits your program.\n\nYou should also get your employer (if you work as a programmer) or your\nschool, if any, to sign a \"copyright disclaimer\" for the program, if\nnecessary. Here is a sample; alter the names:\n\n Yoyodyne, Inc., hereby disclaims all copyright interest in the program\n `Gnomovision' (which makes passes at compilers) written by James Hacker.\n\n , 1 April 1989\n Ty Coon, President of Vice\n\nThis General Public License does not permit incorporating your program into\nproprietary programs. If your program is a subroutine library, you may\nconsider it more useful to permit linking proprietary applications with the\nlibrary. If this is what you want to do, use the GNU Lesser General\nPublic License instead of this License.\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/AutoDL/__init__.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nfrom UDL.Basis.python_sub_class import PanSharpeningModel, TaskDispatcher, ModelDispatcher\nimport UDL.Basis.option\n\ndef build_model(arch, task, cfg=None):\n\n if task == \"pansharpening\":\n from UDL.pansharpening.models import PanSharpeningModel as MODELS\n\n return MODELS.build_model(cfg)\n else:\n raise NotImplementedError(f\"It's not supported in {task}\")\n\n\ndef getDataSession(cfg):\n\n task = cfg.task\n\n if task in [\"pansharpening\"]:\n from UDL.pansharpening.common.psdata import PansharpeningSession as DataSession\n else:\n raise NotImplementedError\n\n return DataSession(cfg)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/AutoDL/trainer.py", "content": "# GPL License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport argparse\nimport copy\nimport os\nimport os.path as osp\nimport warnings\nimport random\nimport numpy as np\nimport torch\nimport torch.distributed as dist\nimport time\n\ntic = time.time()\n# 1.14s\n\nimport sys\nsys.path.append('../..')\n\nsys.path.append('../mmcv')\n\nfrom UDL.AutoDL import build_model, getDataSession, ModelDispatcher\nfrom UDL.Basis.auxiliary import init_random_seed, set_random_seed\nfrom mmcv.utils.logging import print_log, create_logger\n# 1.5s\nfrom mmcv.runner import init_dist, find_latest_checkpoint\nfrom mmcv.parallel import MMDataParallel, MMDistributedDataParallel\nfrom mmcv.runner import (DistSamplerSeedHook, EpochBasedRunner,\n Fp16OptimizerHook, OptimizerHook, build_optimizer,\n build_runner, get_dist_info)\n\n\n# 10s\n# from mmdet.datasets import (build_dataloader, build_dataset,\n# replace_ImageToTensor)\n\ndef trainer(cfg, logger,\n distributed=False,\n meta=None):\n\n model, criterion, optimizer, scheduler = build_model(cfg.arch, cfg.task, cfg)\n\n\n if hasattr(model, 'init_weights'):\n model.init_weights()\n\n\n sess = getDataSession(cfg)\n\n if cfg.eval:\n cfg.workflow = [('val', 1)]\n if not any('train' in mode for mode, _ in cfg.workflow):\n cfg.eval = True\n\n # put model on gpus\n if distributed:\n find_unused_parameters = cfg.get('find_unused_parameters', False)\n # Sets the `find_unused_parameters` parameter in\n # torch.nn.parallel.DistributedDataParallel\n model = MMDistributedDataParallel(\n model.cuda(),\n device_ids=[torch.cuda.current_device()],\n broadcast_buffers=False,\n find_unused_parameters=find_unused_parameters)\n else:\n if not hasattr(model, 'train'):\n if isinstance(model.model, dict):\n for name, m in model.model.items():\n model.model[name] = MMDataParallel(m, device_ids=cfg.gpu_ids)\n else:\n model.model = MMDataParallel(model.model, device_ids=cfg.gpu_ids)\n else:\n model = MMDataParallel(model, device_ids=cfg.gpu_ids)\n\n if cfg.get('optimizer', None) is not None:\n optimizer = build_optimizer(model, cfg.optimizer)\n\n if 'runner' not in cfg:\n cfg.runner = {\n 'type': 'EpochBasedRunner',\n 'max_epochs': cfg.epochs # argparser\n }\n warnings.warn(\n 'config is now expected to have a `runner` section, '\n 'please set `runner` in your config.', UserWarning)\n else:\n if 'epochs' in cfg and 'max_iters' not in cfg.runner:\n cfg.runner['max_epochs'] = cfg.epochs\n # assert cfg.epochs == cfg.runner['max_epochs'], print(cfg.epochs, cfg.runner['max_epochs'])\n\n runner = build_runner(\n cfg.runner,\n default_args=dict(\n model=model,\n optimizer=optimizer,\n work_dir=cfg.work_dir,\n logger=logger,\n meta=meta,\n opt_cfg={'print_freq': cfg.print_freq,\n 'accumulated_step': cfg.accumulated_step,\n 'clip_max_norm': cfg.clip_max_norm,\n 'dataset': cfg.dataset,\n 'img_range': cfg.img_range,\n 'metrics': cfg.metrics,\n 'save_fmt': cfg.save_fmt,\n 'mode': cfg.mode,\n 'eval': cfg.eval,\n 'save_dir': cfg.work_dir + \"/results\"}))\n\n # an ugly workaround to make .log and .log.json filenames the same\n # runner.timestamp = timestamp\n\n # fp16 setting\n fp16_cfg = cfg.get('fp16', None)\n if fp16_cfg is not None:\n optimizer_config = Fp16OptimizerHook(\n **cfg.optimizer_config, **fp16_cfg, distributed=distributed)\n elif distributed and 'type' not in cfg.optimizer_config:\n optimizer_config = OptimizerHook(**cfg.optimizer_config)\n else:\n optimizer_config = cfg.get('optimizer_config', None)\n\n ############################################################\n # register training hooks\n ############################################################\n if cfg.get('config', None) is not None:\n '''\n optimizer = dict(type='SGD', lr=0.1, momentum=0.9, weight_decay=0.0001)\n optimizer_config = dict(grad_clip=None)\n lr_config = dict(policy='step', step=[100, 150])\n checkpoint_config = dict(interval=1)\n log_config = dict(\n interval=100,\n hooks=[\n dict(type='TextLoggerHook'),\n # dict(type='TensorboardLoggerHook')\n ])\n '''\n runner.register_training_hooks(\n cfg.lr_config,\n optimizer_config,\n cfg.checkpoint_config,\n cfg.log_config,\n cfg.get('momentum_config', None),\n custom_hooks_config=cfg.get('custom_hooks', None))\n\n elif cfg.get('log_config', None) is None and len(cfg.workflow) and cfg.workflow[0][0] != 'simple_train':\n if cfg.mode == 'nni':\n runner.register_custom_hooks({'type': 'NNIHook', 'priority': 'very_low'})\n if scheduler is not None:\n runner.register_lr_hook(dict(policy=scheduler.__class__.__name__[:-2], step=scheduler.step_size))\n runner.register_checkpoint_hook(\n dict(type='ModelCheckpoint', indicator='loss', save_top_k=cfg.save_top_k, print_freq=cfg.save_print_freq))\n runner.register_optimizer_hook(dict(grad_clip=10)) # ExternOptimizer\n runner.register_timer_hook(dict(type='IterTimerHook'))\n log_config = [dict(type='TextLoggerHook')]\n if cfg.use_tfb:\n log_config.append(dict(type='TensorboardLoggerHook'))\n runner.register_logger_hooks(dict(\n interval=cfg.print_freq,\n hooks=log_config))\n\n else:\n runner.register_checkpoint_hook(dict(type='ModelCheckpoint', indicator='loss'))\n\n if distributed:\n if isinstance(runner, EpochBasedRunner):\n runner.register_hook(DistSamplerSeedHook())\n\n data_loaders = {}\n\n ############################################################\n # load data\n ############################################################\n\n for flow in cfg.workflow:\n mode, _ = flow\n if 'val' in mode:\n # cfg.dataset = cfg.dataset + '_OrigScale_multiExm1.h5'\n # cfg.dataset = cfg.dataset + '_multiExm1.h5'\n\n eval_loader, eval_sampler = sess.get_eval_dataloader(cfg.dataset[mode], distributed)\n\n eval_cfg = cfg.get('evaluation', {})\n eval_cfg['by_epoch'] = cfg.runner['type'] != 'IterBasedRunner'\n from mmcv.runner import EvalHook, DistEvalHook\n eval_hook = DistEvalHook if distributed else EvalHook\n # In this PR (https://github.com/open-mmlab/mmcv/pull/1193), the\n # priority of IterTimerHook has been modified from 'NORMAL' to 'LOW'.\n if mode != 'simple_val':\n runner.register_hook(\n eval_hook(eval_loader, **eval_cfg), priority='LOW')\n\n data_loaders[mode] = eval_loader\n # if len(cfg.workflow) == 0:\n # cfg.workflow.append(('val', 1))\n\n if 'train' in mode:\n train_loader, train_sampler = sess.get_dataloader(cfg.dataset[mode], distributed)\n if cfg.once_epoch:\n train_loader = iter(list(train_loader))\n data_loaders[mode] = train_loader\n\n if len(cfg.workflow) == 0:\n cfg.workflow.append(('simple_train', 1))\n ############################################################\n # load model\n ############################################################\n\n resume_from = None\n if cfg.get('resume_from', None) is None and cfg.get('auto_resume'):\n resume_from = find_latest_checkpoint(cfg.work_dir)\n if resume_from is not None:\n cfg.resume_from = resume_from\n\n # if cfg.get('resume_from', None):\n runner.resume(cfg.resume_from, cfg.resume_mode, cfg.reset_lr, cfg.lr)\n if cfg.get('load_from', None) and cfg.get('resume_from', None) is not None:\n runner.load_checkpoint(cfg.load_from, cfg.resume_mode)\n\n ############################################################\n # run train/val/test\n ############################################################\n runner.run(data_loaders, cfg.workflow)\n\n\ndef main(cfg):\n # init distributed env first, since logger depends on the dist info.\n if cfg.launcher == 'none':\n distributed = False\n else:\n distributed = True\n init_dist(cfg.launcher, **cfg.dist_params)\n # re-set gpu_ids with distributed training mode\n _, world_size = get_dist_info()\n cfg.gpu_ids = range(world_size)\n\n logger, out_dir, model_save_dir, tfb_dir = create_logger(cfg, cfg.experimental_desc, 0)\n cfg.out_dir = cfg.work_dir = model_save_dir\n seed = init_random_seed(cfg.seed)\n print_log(f'Set random seed to {seed}', logger=logger)\n\n set_random_seed(seed)\n\n # if cfg.checkpoint_config is not None:\n # # save mmdet version, config file content and class names in\n # # checkpoints as meta data\n # cfg.checkpoint_config.meta = dict(\n # mmdet_version=__version__ + get_git_hash()[:7],\n # CLASSES=datasets[0].CLASSES)\n # add an attribute for visualization convenience\n\n trainer(\n cfg,\n logger,\n distributed=distributed,\n meta={})\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/__init__.py", "content": "from ..auxiliary.utils import AverageMeter, accuracy, MetricLogger, SmoothedValue, set_random_seed, init_random_seed, show_memory_info" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/base.py", "content": "from nvidia.dali.plugin.pytorch import DALIGenericIterator\n\n\nclass DALIDataloader(DALIGenericIterator):\n def __init__(self, pipeline, size, batch_size, output_map=[\"data\", \"label\"], auto_reset=True, onehot_label=False):\n # self.size = size\n self.batch_size = batch_size\n self.onehot_label = onehot_label\n self.output_map = output_map\n super().__init__(pipelines=pipeline, size=size, auto_reset=auto_reset, output_map=output_map)\n\n def __next__(self):\n if self._first_batch is not None:\n batch = self._first_batch\n self._first_batch = None\n return batch\n data = super().__next__()[0]\n if self.onehot_label:\n return [data[self.output_map[0]], data[self.output_map[1]].squeeze().long()]\n else:\n return [data[self.output_map[0]], data[self.output_map[1]]]\n\n def __len__(self):\n if self.size % self.batch_size == 0:\n return self.size // self.batch_size\n else:\n return self.size // self.batch_size + 1" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/fp16_utils.py", "content": "import torch\nimport torch.nn as nn\nfrom torch.autograd import Variable\nfrom torch._utils import _flatten_dense_tensors, _unflatten_dense_tensors\n\n\nclass tofp16(nn.Module):\n \"\"\"\n Utility module that implements::\n def forward(self, input):\n return input.half()\n \"\"\"\n\n def __init__(self):\n super(tofp16, self).__init__()\n\n def forward(self, input):\n return input.half()\n\n\ndef BN_convert_float(module):\n \"\"\"\n Utility function for network_to_half().\n Retained for legacy purposes.\n \"\"\"\n if isinstance(module, torch.nn.modules.batchnorm._BatchNorm) and module.affine is True:\n module.float()\n for child in module.children():\n BN_convert_float(child)\n return module\n\n\ndef network_to_half(network):\n \"\"\"\n Convert model to half precision in a batchnorm-safe way.\n Retained for legacy purposes. It is recommended to use FP16Model.\n \"\"\"\n return nn.Sequential(tofp16(), BN_convert_float(network.half()))\n\n\ndef convert_module(module, dtype):\n \"\"\"\n Converts a module's immediate parameters and buffers to dtype.\n \"\"\"\n for param in module.parameters(recurse=False):\n if param is not None:\n if param.data.dtype.is_floating_point:\n param.data = param.data.to(dtype=dtype)\n if param._grad is not None and param._grad.data.dtype.is_floating_point:\n param._grad.data = param._grad.data.to(dtype=dtype)\n\n for buf in module.buffers(recurse=False):\n if buf is not None and buf.data.dtype.is_floating_point:\n buf.data = buf.data.to(dtype=dtype)\n\n\ndef convert_network(network, dtype):\n \"\"\"\n Converts a network's parameters and buffers to dtype.\n \"\"\"\n for module in network.modules():\n if isinstance(module, torch.nn.modules.batchnorm._BatchNorm) and module.affine is True:\n continue\n convert_module(module, dtype)\n if isinstance(module, torch.nn.RNNBase) or isinstance(module, torch.nn.modules.rnn.RNNBase):\n module.flatten_parameters()\n return network\n\n\nclass FP16Model(nn.Module):\n \"\"\"\n Convert model to half precision in a batchnorm-safe way.\n \"\"\"\n\n def __init__(self, network):\n super(FP16Model, self).__init__()\n self.network = convert_network(network, dtype=torch.half)\n\n def forward(self, *inputs):\n inputs = tuple(t.half() for t in inputs)\n return self.network(*inputs)\n\n\ndef backwards_debug_hook(grad):\n raise RuntimeError(\"master_params recieved a gradient in the backward pass!\")\n\ndef prep_param_lists(model, flat_master=False):\n \"\"\"\n Creates a list of FP32 master parameters for a given model, as in\n `Training Neural Networks with Mixed Precision: Real Examples`_.\n Args:\n model (torch.nn.Module): Existing Pytorch model\n flat_master (bool, optional, default=False): Flatten the master parameters into a single tensor, as a performance optimization.\n Returns:\n A tuple (``model_params``, ``master_params``). ``model_params`` is a list of the model's parameters for later use with :func:`model_grads_to_master_grads` and :func:`master_params_to_model_params`. ``master_params`` is a list of FP32 master gradients. If ``flat_master=True``, ``master_params`` will be a list with one element.\n Example::\n model_params, master_params = prep_param_lists(model)\n .. warning::\n Currently, if ``flat_master=True``, all the model's parameters must be the same type. If the model has parameters of different types, use ``flat_master=False``, or use :class:`FP16_Optimizer`.\n .. _`Training Neural Networks with Mixed Precision: Real Examples`:\n http://on-demand.gputechconf.com/gtc/2018/video/S81012/\n \"\"\"\n model_params = [param for param in model.parameters() if param.requires_grad]\n\n if flat_master:\n # Give the user some more useful error messages\n try:\n # flatten_dense_tensors returns a contiguous flat array.\n # http://pytorch.org/docs/master/_modules/torch/_utils.html\n master_params = _flatten_dense_tensors([param.data for param in model_params]).float()\n except:\n print(\"Error in prep_param_lists: model may contain a mixture of parameters \"\n \"of different types. Use flat_master=False, or use F16_Optimizer.\")\n raise\n master_params = torch.nn.Parameter(master_params)\n master_params.requires_grad = True\n # master_params.register_hook(backwards_debug_hook)\n if master_params.grad is None:\n master_params.grad = master_params.new(*master_params.size())\n return model_params, [master_params]\n else:\n master_params = [param.clone().float().detach() for param in model_params]\n for param in master_params:\n param.requires_grad = True\n return model_params, master_params\n\n\ndef model_grads_to_master_grads(model_params, master_params, flat_master=False):\n \"\"\"\n Copy model gradients to master gradients.\n Args:\n model_params: List of model parameters created by :func:`prep_param_lists`.\n master_params: List of FP32 master parameters created by :func:`prep_param_lists`. If ``master_params`` was created with ``flat_master=True``, ``flat_master=True`` should also be supplied to :func:`model_grads_to_master_grads`.\n \"\"\"\n if flat_master:\n # The flattening may incur one more deep copy than is necessary.\n master_params[0].grad.data.copy_(\n _flatten_dense_tensors([p.grad.data for p in model_params]))\n else:\n for model, master in zip(model_params, master_params):\n if model.grad is not None:\n if master.grad is None:\n master.grad = Variable(master.data.new(*master.data.size()))\n master.grad.data.copy_(model.grad.data)\n else:\n master.grad = None\n\n\ndef master_params_to_model_params(model_params, master_params, flat_master=False):\n \"\"\"\n Copy master parameters to model parameters.\n Args:\n model_params: List of model parameters created by :func:`prep_param_lists`.\n master_params: List of FP32 master parameters created by :func:`prep_param_lists`. If ``master_params`` was created with ``flat_master=True``, ``flat_master=True`` should also be supplied to :func:`master_params_to_model_params`.\n \"\"\"\n if flat_master:\n for model, master in zip(model_params,\n _unflatten_dense_tensors(master_params[0].data, model_params)):\n model.data.copy_(master)\n else:\n for model, master in zip(model_params, master_params):\n model.data.copy_(master.data)\n\n# Backward compatibility fixes\n\ndef to_python_float(t):\n if hasattr(t, 'item'):\n return t.item()\n else:\n return t[0]\n\nTORCH_MAJOR = int(torch.__version__.split('.')[0])\nTORCH_MINOR = int(torch.__version__.split('.')[1])\nif TORCH_MAJOR == 0 and TORCH_MINOR <= 4:\n clip_grad_norm = torch.nn.utils.clip_grad_norm\nelse:\n clip_grad_norm = torch.nn.utils.clip_grad_norm_" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/__init__.py", "content": "__copyright__ = 'Copyright (C) 2018 Swall0w'\n__version__ = '0.0.7'\n__author__ = 'Swall0w'\n__url__ = 'https://github.com/Swall0w/torchstat'\n\nfrom torchstat.compute_memory import compute_memory\nfrom torchstat.compute_madd import compute_madd\nfrom torchstat.compute_flops import compute_flops\nfrom torchstat.stat_tree import StatTree, StatNode\nfrom torchstat.model_hook import ModelHook\nfrom torchstat.reporter import report_format\nfrom torchstat.statistics import stat, ModelStat\n\n__all__ = ['report_format', 'StatTree', 'StatNode', 'compute_madd',\n 'compute_flops', 'ModelHook', 'stat', 'ModelStat', '__main__',\n 'compute_memory']\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/__main__.py", "content": "from torchstat import stat\nimport argparse\nimport importlib.util\nimport torch\n\n\ndef arg():\n parser = argparse.ArgumentParser(description='Torch model statistics')\n parser.add_argument('--file', '-f', type=str,\n help='Module file.')\n parser.add_argument('--model', '-m', type=str,\n help='Model name')\n parser.add_argument('--size', '-s', type=str, default='3x224x224',\n help='Input size. channels x height x width (default: 3x224x224)')\n return parser.parse_args()\n\n\ndef main():\n args = arg()\n try:\n spec = importlib.util.spec_from_file_location('models', args.file)\n module = importlib.util.module_from_spec(spec)\n spec.loader.exec_module(module)\n model = getattr(module, args.model)()\n except Exception:\n import traceback\n print(f'Tried to import {args.model} from {args.file}. but failed.')\n traceback.print_exc()\n\n import sys\n sys.exit()\n\n input_size = tuple(int(x) for x in args.size.split('x'))\n stat(model, input_size, query_granularity=1)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/compute_flops.py", "content": "import torch.nn as nn\nimport torch\nimport numpy as np\nimport inspect\n\n\ndef compute_flops(module, inp, out):\n # print(module.__class__)\n # if 'attn' in module.__name__:\n # print(module.__class__)\n # print(list(filter(lambda m: not m.startswith(\"__\") and not m.endswith(\"__\") and callable(getattr(module, m)), dir(module))))\n if isinstance(module, nn.Conv2d):\n return compute_Conv2d_flops(module, inp, out)\n elif isinstance(module, nn.BatchNorm2d):\n return compute_BatchNorm2d_flops(module, inp, out)\n elif isinstance(module, nn.LayerNorm) or 'LayerNorm' in type(module).__name__:\n return compute_LayerNorm_flops(module, inp, out)\n elif isinstance(module, (nn.AvgPool2d, nn.MaxPool2d)):\n return compute_Pool2d_flops(module, inp, out)\n elif isinstance(module, (nn.ReLU, nn.ReLU6, nn.PReLU, nn.ELU, nn.LeakyReLU)):\n return compute_ReLU_flops(module, inp, out)\n # elif isinstance(module, nn.Upsample):\n # return compute_Upsample_flops(module, inp, out)\n elif isinstance(module, nn.Linear):\n return compute_Linear_flops(module, inp, out)\n elif 'SwinTEB' in module.__class__.__name__:#\n return compute_WindowAttention_flops(module, inp, out)\n elif 'XCTEB' in module.__class__.__name__:\n return compute_XCA_flops(module, inp, out)\n elif 'MSA' in module.__class__.__name__:\n return compute_MSA_flops(module, inp, out)\n elif 'cGCN' == module.__class__.__name__:\n return compute_cGCN_flops(module, inp, out)\n elif 'sGCN' == module.__class__.__name__:\n return compute_sGCN_flops(module, inp, out)\n else:\n print(f\"[Flops]: {module.__class__.__name__} is not supported!\")\n return 0\n pass\n\n\ndef compute_cGCN_flops(module, inp, out):\n batch_size, dim, H, W = inp.size()\n dim = dim // 2\n L = H * W\n\n # N = window_size ** 2\n # num_patches = H * W // N\n\n # calculate flops for 1 window with token length of N\n flops = 0\n # qkv = self.qkv(x)\n # flops += N * dim * 3 * dim\n # attn = (q @ k.transpose(-2, -1)) b head c (h w) b head (h w) c\n flops += dim * (dim//2) * L\n # x = (attn @ v) b head c c b head c (h w)\n flops += L * dim * (dim//2)\n\n return batch_size * flops\n\n\ndef compute_sGCN_flops(module, inp, out):\n\n batch_size, dim, H, W = inp.size()\n dim = dim // 2\n L = H * W\n\n # calculate flops for 1 window with token length of N\n flops = 0\n # qkv = self.qkv(x)\n # flops += N * dim * 3 * dim\n # attn = (q @ k.transpose(-2, -1)) b head c (h w) b head (h w) c\n flops += dim * dim * L\n # x = (attn @ v) b head c c b head c (h w)\n flops += L * dim * dim\n\n return batch_size * flops\n\ndef compute_Conv2d_flops(module, inp, out):\n # Can have multiple inputs, getting the first one\n assert isinstance(module, nn.Conv2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n\n batch_size = inp.size()[0]\n in_c = inp.size()[1]\n k_h, k_w = module.kernel_size\n out_c, out_h, out_w = out.size()[1:]\n groups = module.groups\n\n filters_per_channel = out_c // groups\n conv_per_position_flops = k_h * k_w * in_c * filters_per_channel\n active_elements_count = batch_size * out_h * out_w\n\n total_conv_flops = conv_per_position_flops * active_elements_count\n\n bias_flops = 0\n if module.bias is not None:\n bias_flops = out_c * active_elements_count\n # k * k * c * H * W * o = (乘法 + 加法 + bias) * active_elements_count\n total_flops = total_conv_flops + bias_flops\n return total_flops\n\n\ndef compute_BatchNorm2d_flops(module, inp, out):\n assert isinstance(module, nn.BatchNorm2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n in_c, in_h, in_w = inp.size()[1:]\n batch_flops = np.prod(inp.shape)\n if module.affine:\n batch_flops *= 2\n return batch_flops\n\ndef compute_LayerNorm_flops(module, inp, out):\n # assert isinstance(module, nn.LayerNorm)\n if len(inp.size()) == 3:\n inp = inp.unsqueeze(0)\n if len(out.size()) == 3:\n out = out.unsqueeze(0)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n flops = np.prod(inp.shape)\n\n return flops\n\ndef compute_ReLU_flops(module, inp, out):\n assert isinstance(module, (nn.ReLU, nn.ReLU6, nn.PReLU, nn.ELU, nn.LeakyReLU))\n batch_size = inp.size()[0]\n active_elements_count = batch_size\n\n for s in inp.size()[1:]:\n active_elements_count *= s\n\n return active_elements_count\n\n\ndef compute_Pool2d_flops(module, inp, out):\n assert isinstance(module, nn.MaxPool2d) or isinstance(module, nn.AvgPool2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n return np.prod(inp.shape)\n\n\ndef compute_Linear_flops(module, inp, out):\n assert isinstance(module, nn.Linear)\n if len(inp.size()) > 3:\n inp = inp.reshape(inp.size(0), inp.size(1), -1)\n if len(out.size()) > 3:\n out = out.reshape(out.size(0), out.size(1), -1)\n batch_size = inp.size()[0]\n if len(inp.size()) == 3:# and inp.size(0) == 1:\n inp = inp[0, ...]#.squeeze(0)\n if len(out.size()) == 3:# and out.size(0) == 1:\n out = out[0, ...]#.squeeze(0)\n assert len(inp.size()) == 2 and len(out.size()) == 2\n\n return batch_size * inp.size()[1] * out.size()[1]\n\ndef compute_Upsample_flops(module, inp, out):\n assert isinstance(module, nn.Upsample)\n output_size = out[0]\n batch_size = inp.size()[0]\n output_elements_count = batch_size\n # for s in output_size.\n\ndef compute_MSA_flops(module, inp, out):\n # q = inp[0]\n if isinstance(inp, tuple):\n inp = inp[0]\n if module.__class__.__name__ == \"MSA\":\n N, batch_size, dim = inp.size()\n elif module.__class__.__name__ == \"MSA_BNC\":\n batch_size, N, dim = inp.size()\n\n\n # window_size = module.window_size\n if hasattr(module, 'num_heads'):\n num_heads = module.num_heads\n elif hasattr(module, 'n_heads'):\n num_heads = module.num_heads\n num_patches = 1#H * W // N\n # num_patches = module.num_patches\n # batch_size /= num_patches# B*nH*nW\n # assert batch_size == 1, print(f\"{inp.size()} is not compatiable with {num_patches}\")\n\n # print(inp.size(), out.size(), dir(module))\n\n # calculate flops for 1 window with token length of N\n flops = 0\n # qkv = self.qkv(x)\n # flops += N * dim * 3 * dim\n # attn = (q @ k.transpose(-2, -1))\n flops += num_heads * N * (dim // num_heads) * N\n # x = (attn @ v)\n flops += num_heads * N * N * (dim // num_heads)\n # x = self.proj(x)\n # flops += N * dim * dim\n return batch_size * num_patches * flops\n\ndef compute_WindowAttention_flops(module, inp, out):\n # inp = inp[0].permute(0, 3, 1, 2) # B, p, L, C\n # out = out.permute(0, 3, 1, 2)\n\n # dim = out.size(1)\n if isinstance(inp, tuple):\n inp = inp[0]\n # inp = inp[0]\n L = len(inp.size())\n if L == 3:\n batch_size, HW, dim = inp.size()\n H = W = int(np.sqrt(HW))\n elif L == 4:\n batch_size, dim, H, W = inp.size()\n\n window_size = module.window_size\n num_heads = module.num_heads\n N = window_size ** 2\n num_patches = H * W // N\n # num_patches = module.num_patches\n # batch_size /= num_patches# B*nH*nW\n # assert batch_size == 1, print(f\"{inp.size()} is not compatiable with {num_patches}\")\n\n\n # print(inp.size(), out.size(), dir(module))\n\n # calculate flops for 1 window with token length of N\n flops = 0\n # qkv = self.qkv(x)\n # flops += N * dim * 3 * dim\n # attn = (q @ k.transpose(-2, -1))\n flops += num_heads * N * (dim // num_heads) * N\n # x = (attn @ v)\n flops += num_heads * N * N * (dim // num_heads)\n # x = self.proj(x)\n # flops += N * dim * dim\n # module.__base__ = f'{module.__class__.__name__}(dim={dim}, win_size={window_size}, nh={num_heads}, n_p={num_patches}, size=({H}, {W}))'\n # print(f'{module.__class__.__name__}, dim={dim}, win_size={window_size}, num_heads={num_heads},'\n # f'num_patches={num_patches}, img_size=({H}, {W})')\n return batch_size * num_patches * flops\n\n\ndef compute_XCA_flops(module, inp, out):\n\n dim = out.size(1)\n batch_size, _, H, W = inp.size()\n if hasattr(module, \"window_size\"):\n window_size = module.window_size\n N = window_size ** 2\n num_patches = H * W // N\n else:\n num_patches = 1\n window_size = 1\n N = H * W\n # window_size = module.window_size\n num_heads = module.num_heads\n\n # N = window_size ** 2\n # num_patches = H * W // N\n\n # calculate flops for 1 window with token length of N\n flops = 0\n # qkv = self.qkv(x)\n # flops += N * dim * 3 * dim\n # attn = (q @ k.transpose(-2, -1)) b head c (h w) b head (h w) c\n flops += num_heads * (dim // num_heads) * (dim // num_heads) * N\n # x = (attn @ v) b head c c b head c (h w)\n flops += num_heads * N * (dim // num_heads) * (dim // num_heads)\n # x = self.proj(x)\n # flops += N * dim * dim\n return batch_size * num_patches * flops" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/compute_madd.py", "content": "\"\"\"\ncompute Multiply-Adds(MAdd) of each leaf module\n\"\"\"\n\nimport torch.nn as nn\n\n\ndef compute_Conv2d_madd(module, inp, out):\n assert isinstance(module, nn.Conv2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n\n in_c = inp.size()[1]\n k_h, k_w = module.kernel_size\n out_c, out_h, out_w = out.size()[1:]\n groups = module.groups\n\n # ops per output element\n kernel_mul = k_h * k_w * (in_c // groups)\n kernel_add = kernel_mul - 1 + (0 if module.bias is None else 1)\n\n kernel_mul_group = kernel_mul * out_h * out_w * (out_c // groups)\n kernel_add_group = kernel_add * out_h * out_w * (out_c // groups)\n\n total_mul = kernel_mul_group * groups\n total_add = kernel_add_group * groups\n\n return total_mul + total_add\n\n\ndef compute_ConvTranspose2d_madd(module, inp, out):\n assert isinstance(module, nn.ConvTranspose2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n\n in_c, in_h, in_w = inp.size()[1:]\n k_h, k_w = module.kernel_size\n out_c, out_h, out_w = out.size()[1:]\n groups = module.groups\n\n kernel_mul = k_h * k_w * (in_c // groups)\n kernel_add = kernel_mul - 1 + (0 if module.bias is None else 1)\n\n kernel_mul_group = kernel_mul * in_h * in_w * (out_c // groups)\n kernel_add_group = kernel_add * in_h * in_w * (out_c // groups)\n\n total_mul = kernel_mul_group * groups\n total_add = kernel_add_group * groups\n\n return total_mul + total_add\n\ndef compute_LayerNorm_madd(module, inp, out):\n # assert isinstance(module, nn.LayerNorm)\n if len(inp.size()) == 3:\n inp = inp.unsqueeze(0)\n if len(out.size()) == 3:\n out = out.unsqueeze(0)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n\n in_c, in_h, in_w = inp.size()[1:]\n\n # 1. sub mean\n # 2. div standard deviation\n # 3. mul alpha\n # 4. add beta\n return 4 * in_h * in_w\n\ndef compute_BatchNorm2d_madd(module, inp, out):\n assert isinstance(module, nn.BatchNorm2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n\n in_c, in_h, in_w = inp.size()[1:]\n\n # 1. sub mean\n # 2. div standard deviation\n # 3. mul alpha\n # 4. add beta\n return 4 * in_c * in_h * in_w\n\n\ndef compute_MaxPool2d_madd(module, inp, out):\n assert isinstance(module, nn.MaxPool2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n\n if isinstance(module.kernel_size, (tuple, list)):\n k_h, k_w = module.kernel_size\n else:\n k_h, k_w = module.kernel_size, module.kernel_size\n out_c, out_h, out_w = out.size()[1:]\n\n return (k_h * k_w - 1) * out_h * out_w * out_c\n\n\ndef compute_AvgPool2d_madd(module, inp, out):\n assert isinstance(module, nn.AvgPool2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n\n if isinstance(module.kernel_size, (tuple, list)):\n k_h, k_w = module.kernel_size\n else:\n k_h, k_w = module.kernel_size, module.kernel_size\n out_c, out_h, out_w = out.size()[1:]\n\n kernel_add = k_h * k_w - 1\n kernel_avg = 1\n\n return (kernel_add + kernel_avg) * (out_h * out_w) * out_c\n\n\ndef compute_ReLU_madd(module, inp, out):\n assert isinstance(module, (nn.ReLU, nn.ReLU6))\n\n count = 1\n for i in inp.size()[1:]:\n count *= i\n return count\n\n\ndef compute_Softmax_madd(module, inp, out):\n assert isinstance(module, nn.Softmax)\n assert len(inp.size()) > 1\n\n count = 1\n for s in inp.size()[1:]:\n count *= s\n exp = count\n add = count - 1\n div = count\n return exp + add + div\n\n\ndef compute_Linear_madd(module, inp, out):\n assert isinstance(module, nn.Linear)\n if len(inp.size()) > 3:\n inp = inp.reshape(inp.size(0), inp.size(1), -1)\n if len(out.size()) > 3:\n out = out.reshape(out.size(0), out.size(1), -1)\n if len(inp.size()) == 3:# and inp.size(0) == 1\n inp = inp[0, ...]#.squeeze(0)\n if len(out.size()) == 3:# and out.size(0) == 1\n out = out[0, ...]#.squeeze(0)\n\n assert len(inp.size()) == 2 and len(out.size()) == 2, print(inp.size(), out.size())\n\n num_in_features = inp.size()[1]\n num_out_features = out.size()[1]\n\n mul = num_in_features\n add = num_in_features - 1\n return num_out_features * (mul + add)\n\n\ndef compute_Bilinear_madd(module, inp1, inp2, out):\n assert isinstance(module, nn.Bilinear)\n assert len(inp1.size()) == 2 and len(inp2.size()) == 2 and len(out.size()) == 2\n\n num_in_features_1 = inp1.size()[1]\n num_in_features_2 = inp2.size()[1]\n num_out_features = out.size()[1]\n\n mul = num_in_features_1 * num_in_features_2 + num_in_features_2\n add = num_in_features_1 * num_in_features_2 + num_in_features_2 - 1\n return num_out_features * (mul + add)\n\n\ndef compute_madd(module, inp, out):\n if isinstance(module, nn.Conv2d):\n return compute_Conv2d_madd(module, inp, out)\n elif isinstance(module, nn.ConvTranspose2d):\n return compute_ConvTranspose2d_madd(module, inp, out)\n elif isinstance(module, nn.BatchNorm2d):\n return compute_BatchNorm2d_madd(module, inp, out)\n elif isinstance(module, nn.LayerNorm) or 'LayerNorm' in type(module).__name__:\n return compute_LayerNorm_madd(module, inp, out)\n elif isinstance(module, nn.MaxPool2d):\n return compute_MaxPool2d_madd(module, inp, out)\n elif isinstance(module, nn.AvgPool2d):\n return compute_AvgPool2d_madd(module, inp, out)\n elif isinstance(module, (nn.ReLU, nn.ReLU6)):\n return compute_ReLU_madd(module, inp, out)\n elif isinstance(module, nn.Softmax):\n return compute_Softmax_madd(module, inp, out)\n elif isinstance(module, nn.Linear):\n return compute_Linear_madd(module, inp, out)\n elif isinstance(module, nn.Bilinear):\n return compute_Bilinear_madd(module, inp[0], inp[1], out)\n else:\n print(f\"[MAdd]: {type(module).__name__} is not supported!\")\n return 0\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/compute_memory.py", "content": "import torch.nn as nn\nimport torch\nimport numpy as np\n\n\ndef compute_memory(module, inp, out):\n if isinstance(module, (nn.ReLU, nn.ReLU6, nn.ELU, nn.LeakyReLU)):\n return compute_ReLU_memory(module, inp, out)\n elif isinstance(module, nn.PReLU):\n return compute_PReLU_memory(module, inp, out)\n elif isinstance(module, nn.Conv2d):\n return compute_Conv2d_memory(module, inp, out)\n elif isinstance(module, nn.BatchNorm2d):\n return compute_BatchNorm2d_memory(module, inp, out)\n elif isinstance(module, nn.LayerNorm) or 'LayerNorm' in type(module).__name__:\n return compute_LayerNorm_memory(module, inp, out)\n elif isinstance(module, nn.Linear):\n return compute_Linear_memory(module, inp, out)\n elif isinstance(module, (nn.AvgPool2d, nn.MaxPool2d)):\n return compute_Pool2d_memory(module, inp, out)\n else:\n print(f\"[Memory]: {type(module).__name__} is not supported!\")\n return (0, 0)\n pass\n\n\ndef num_params(module):\n return sum(p.numel() for p in module.parameters() if p.requires_grad)\n\n\ndef compute_ReLU_memory(module, inp, out):\n assert isinstance(module, (nn.ReLU, nn.ReLU6, nn.ELU, nn.LeakyReLU))\n batch_size = inp.size()[0]\n mread = batch_size * inp.size()[1:].numel()\n mwrite = batch_size * inp.size()[1:].numel()\n\n return (mread, mwrite)\n\n\ndef compute_PReLU_memory(module, inp, out):\n assert isinstance(module, (nn.PReLU))\n batch_size = inp.size()[0]\n mread = batch_size * (inp.size()[1:].numel() + num_params(module))\n mwrite = batch_size * inp.size()[1:].numel()\n\n return (mread, mwrite)\n\n\ndef compute_Conv2d_memory(module, inp, out):\n # Can have multiple inputs, getting the first one\n assert isinstance(module, nn.Conv2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n\n batch_size = inp.size()[0]\n in_c = inp.size()[1]\n out_c, out_h, out_w = out.size()[1:]\n\n # This includes weighs with bias if the module contains it.\n mread = batch_size * (inp.size()[1:].numel() + num_params(module))\n mwrite = batch_size * out_c * out_h * out_w\n return (mread, mwrite)\n\n\ndef compute_BatchNorm2d_memory(module, inp, out):\n assert isinstance(module, nn.BatchNorm2d)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n batch_size, in_c, in_h, in_w = inp.size()\n\n mread = batch_size * (inp.size()[1:].numel() + 2 * in_c)\n mwrite = inp.size().numel()\n return (mread, mwrite)\n\ndef compute_LayerNorm_memory(module, inp, out):\n # assert isinstance(module, nn.LayerNorm)\n if len(inp.size()) == 3:\n inp = inp.unsqueeze(0)\n if len(out.size()) == 3:\n out = out.unsqueeze(0)\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n batch_size, in_c = inp.size()[:2]\n\n mread = batch_size * (inp.size()[2:].numel() + 2 * in_c)\n mwrite = inp.size().numel()\n return (mread, mwrite)\n\ndef compute_Linear_memory(module, inp, out):\n assert isinstance(module, nn.Linear)\n if len(inp.size()) > 3:\n inp = inp.reshape(inp.size(0), inp.size(1), -1)\n if len(out.size()) > 3:\n out = out.reshape(out.size(0), out.size(1), -1)\n\n batch_size = inp.size()[0]\n if len(inp.size()) == 3:# and inp.size(0) == 1:\n inp = inp[0, ...]#.squeeze(0)\n if len(out.size()) == 3:# and out.size(0) == 1:\n out = out[0, ...]#.squeeze(0)\n assert len(inp.size()) == 2 and len(out.size()) == 2\n\n mread = batch_size * (inp.size()[1:].numel() + num_params(module))\n mwrite = out.size().numel()\n\n return (mread, mwrite)\n\n\ndef compute_Pool2d_memory(module, inp, out):\n assert isinstance(module, (nn.MaxPool2d, nn.AvgPool2d))\n assert len(inp.size()) == 4 and len(inp.size()) == len(out.size())\n batch_size = inp.size()[0]\n mread = batch_size * inp.size()[1:].numel()\n mwrite = batch_size * out.size()[1:].numel()\n return (mread, mwrite)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/model_hook.py", "content": "import time\nfrom collections import OrderedDict\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom functools import partial\nfrom torchstat import compute_madd\nfrom torchstat import compute_flops\nfrom torchstat import compute_memory\n\n\nclass ModelHook(object):\n def __init__(self, model, input_size, device=\"cuda\", debug_layers=[]):\n assert isinstance(model, nn.Module)\n assert isinstance(input_size, (list, tuple))\n self.leaf_modules = []\n self.debug_layers = debug_layers\n self._model = model\n self._input_size = input_size\n self._origin_call = dict() # sub module call hook\n self.hooks = []\n self._hook_model()\n # x = [torch.rand(1, *self._input_size)] # add module duration time\n device = device.lower()\n assert device in [\n \"cuda\",\n \"cpu\",\n ], \"Input device is not valid, please specify 'cuda' or 'cpu'\"\n\n if device == \"cuda\" and torch.cuda.is_available():\n dtype = torch.cuda.FloatTensor\n else:\n dtype = torch.FloatTensor\n x = [torch.rand(*in_size).type(dtype) for in_size in input_size]\n self._model.eval()\n self._model(*x)\n\n # if len(debug_layers) > 0:\n # self.debug_partial_layer(debug_layers)\n\n\n @staticmethod\n def _register_buffer(module):\n assert isinstance(module, nn.Module)\n\n if len(list(module.children())) > 0:\n return\n\n module.register_buffer('input_shape', torch.zeros(3).int())\n module.register_buffer('output_shape', torch.zeros(3).int())\n module.register_buffer('parameter_quantity', torch.zeros(1).int())\n module.register_buffer('inference_memory', torch.zeros(1).long())\n module.register_buffer('MAdd', torch.zeros(1).long())\n module.register_buffer('duration', torch.zeros(1).float())\n module.register_buffer('Flops', torch.zeros(1).long())\n module.register_buffer('Memory', torch.zeros(2).long())\n\n def _sub_module_call_hook(self):\n def wrap_call(module, *input, **kwargs):\n assert module.__class__ in self._origin_call\n # Itemsize for memory\n try:\n itemsize = input[0].detach().numpy().itemsize\n except:\n itemsize = input[0].detach().cpu().numpy().itemsize\n\n start = time.time()\n output = self._origin_call[module.__class__](module, *input, **kwargs) # 都是nn.Conv2D则有相同的_call__不需要重复存储\n end = time.time()\n module.duration = torch.from_numpy(\n np.array([end - start], dtype=np.float32))\n # c, h, w\n module.input_shape = torch.from_numpy(\n np.array(input[0].size()[1:], dtype=np.int32))\n module.output_shape = torch.from_numpy(\n np.array(output.size()[1:], dtype=np.int32))\n # print(module.name)\n parameter_quantity = 0\n inference_memory = 1\n # iterate through parameters and count num params\n if 'XCTEB' in module.__class__.__name__:\n c, h, w = module.input_shape\n num_heads = module.num_heads\n parameter_quantity += c * c * num_heads\n elif 'SwinTEB' in module.__class__.__name__:\n if len(module.input_shape) == 3:\n # c, h, w = module.input_shape # c, h, w\n _, N, c = module.input_shape\n # N = h * w\n elif len(module.input_shape) == 2:\n N = module.input_shape[0]\n num_heads = module.num_heads\n # hh = nH * h WindowAttention只减少了flops并没有减少显存占用，因此参数量按照图像大小算\n parameter_quantity += N * N * num_heads\n print(parameter_quantity, N, module.input_shape)\n elif 'MSA' == module.__class__.__name__:\n # L, B, D\n # if hasattr(module, '__name__'):\n # print('model.body.decoder.layers.0.self_attn')\n # print(module.__name__, module.input_shape)\n module.input_shape = torch.from_numpy(\n np.array(input[0].permute(1, 2, 0).size()[1:], dtype=np.int32))\n c, L = module.input_shape\n num_heads = module.num_heads\n parameter_quantity += L * L * num_heads\n # print(L, c)\n elif 'MSA_BNC' == module.__class__.__name__:\n # B, L, C\n module.input_shape = torch.from_numpy(\n np.array(input[0].permute(0, 2, 1).size()[1:], dtype=np.int32))\n c, L = module.input_shape\n num_heads = module.num_heads\n parameter_quantity += L * L * num_heads\n # print(L, c)\n elif 'sGCN' == module.__class__.__name__:\n module.input_shape = torch.from_numpy(\n np.array(input[0][0].permute(0, 2, 1).size(), dtype=np.int32))\n c, H, W = module.input_shape\n c = c // 2\n parameter_quantity += c * c\n elif 'cGCN' == module.__class__.__name__:\n module.input_shape = torch.from_numpy(\n np.array(input[0][0].permute(0, 2, 1).size(), dtype=np.int32))\n c, H, W = module.input_shape\n c = c // 2\n parameter_quantity += c * c // 2\n else:\n for s in output.size()[1:]:\n inference_memory *= s\n # memory += parameters_number # exclude parameter memory\n for name, p in module._parameters.items():\n parameter_quantity += (0 if p is None else torch.numel(p.data))\n module.parameter_quantity = torch.from_numpy(\n np.array([parameter_quantity], dtype=np.long))\n\n inference_memory = inference_memory * 4 / (1024 ** 2) # shown as MB unit\n module.inference_memory = torch.from_numpy(\n np.array([inference_memory], dtype=np.float32))\n\n if len(input) == 1:\n madd = compute_madd(module, input[0], output)\n flops = compute_flops(module, input[0], output)\n Memory = compute_memory(module, input[0], output)\n elif len(input) > 1:\n madd = compute_madd(module, input, output)\n flops = compute_flops(module, input, output)\n Memory = compute_memory(module, input, output)\n else: # error\n madd = 0\n flops = 0\n Memory = (0, 0)\n module.MAdd = torch.from_numpy(\n np.array([madd], dtype=np.int64))\n module.Flops = torch.from_numpy(\n np.array([flops], dtype=np.int64))\n Memory = np.array(Memory, dtype=np.int64) * itemsize\n module.Memory = torch.from_numpy(Memory)\n\n return output\n\n leaf_modules = self.leaf_modules\n # for m in self._model.modules():\n # print(m.__class__)\n\n for name, module in self._model.named_modules():\n if len(list(module.children())) == 0:\n module.name = name\n leaf_modules.append((name, module))\n if module.__class__ not in self._origin_call:\n # 只记录一类与具体实例无关的__call__\n self._origin_call[module.__class__] = module.__class__.__call__\n module.__class__.__call__ = wrap_call\n elif name != '' and len(list(module.children())) > 0 and any([L in module.__class__.__name__ for L in self.debug_layers]):\n #name in self.debug_layers:# module.__class__.__name__ in self.debug_layers\n # if module.__class__.__name__ in self.debug_layers:\n # print(\"111\")\n leaf_modules.append((name, module))\n if module.__class__ not in self._origin_call:\n self._origin_call[module.__class__] = module.__class__.__call__\n module.__class__.__call__ = wrap_call\n print(name, module.__class__.__name__)\n\n # for module in self._model.modules():\n # if len(list(module.children())) == 0 and module.__class__ not in self._origin_call:\n # self.hooks.append(module.register_forward_hook(wrap_call))\n\n def _hook_model(self):\n self._model.apply(self._register_buffer)\n self._sub_module_call_hook()\n\n def clear_hooks(self) -> None:\n \"\"\"Clear model hooks\"\"\"\n\n # for handle in self.hook_handles:\n # handle.pop()\n def unwarp_calls(module):\n if module.__class__ in self._origin_call:\n module.__class__.__call__ = self._origin_call[module.__class__]\n # module.__delattr__('__name__')\n\n calls = list(map(unwarp_calls, self._model.modules()))\n del calls\n # for module in self._model.modules():\n # if module.__class__ in self._origin_call:\n # module.__class__.__call__ = self._origin_call[module.__class__]\n\n # @staticmethod\n # def _retrieve_leaf_modules(model):\n # leaf_modules = []\n # for name, m in model.named_modules():\n # if len(list(m.children())) == 0:\n # leaf_modules.append((name, m))\n # return leaf_modules\n\n def retrieve_leaf_modules(self):\n return OrderedDict(self.leaf_modules)\n # return OrderedDict(self._retrieve_leaf_modules(self._model))\n\n def debug_partial_layer(self, target_keys):\n target_layers = []\n submodule_name = dict(list(self._model.named_modules())[1:]).keys()\n for t in target_keys:\n for name in submodule_name:\n if t in name:\n target_layers.append(name)\n\n return target_layers\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/reporter.py", "content": "import pandas as pd\n\n\npd.set_option('display.width', 1000)\npd.set_option('display.max_rows', 10000)\npd.set_option('display.max_columns', 10000)\n\n\ndef round_value(value, binary=False):\n divisor = 1024. if binary else 1000.\n\n if value // divisor**4 > 0:\n return str(round(value / divisor**4, 2)) + 'T'\n elif value // divisor**3 > 0:\n return str(round(value / divisor**3, 2)) + 'G'\n elif value // divisor**2 > 0:\n return str(round(value / divisor**2, 2)) + 'M'\n elif value // divisor > 0:\n return str(round(value / divisor, 2)) + 'K'\n return str(value)\n\n\ndef report_format(collected_nodes):\n data = list()\n properties = list()\n for node in collected_nodes:\n name = node.name\n mtype = node.mtype\n input_shape = ' '.join(['{:>3d}'] * len(node.input_shape)).format(\n *[e for e in node.input_shape])\n output_shape = ' '.join(['{:>3d}'] * len(node.output_shape)).format(\n *[e for e in node.output_shape])\n parameter_quantity = node.parameter_quantity\n inference_memory = node.inference_memory\n MAdd = node.MAdd\n Flops = node.Flops\n mread, mwrite = [i for i in node.Memory]\n duration = node.duration\n data.append([name, input_shape, output_shape, parameter_quantity,\n inference_memory, MAdd, duration, Flops, mread,\n mwrite])\n properties.append(mtype)\n pd.set_option('display.max_columns', None)\n df = pd.DataFrame(data)\n df_properties = pd.DataFrame(properties)\n df.columns = ['module name', 'input shape', 'output shape',\n 'params', 'memory(MB)',\n 'MAdd', 'duration', 'Flops', 'MemRead(B)', 'MemWrite(B)']\n df['duration[%]'] = df['duration'] / (df['duration'].sum() + 1e-7)\n df['MemR+W(B)'] = df['MemRead(B)'] + df['MemWrite(B)']\n df['type'] = df_properties\n total_parameters_quantity = df['params'].sum()\n total_memory = df['memory(MB)'].sum()\n total_operation_quantity = df['MAdd'].sum()\n total_flops = df['Flops'].sum()\n total_duration = df['duration[%]'].sum()\n total_mread = df['MemRead(B)'].sum()\n total_mwrite = df['MemWrite(B)'].sum()\n total_memrw = df['MemR+W(B)'].sum()\n del df['duration']\n\n # Add Total row\n total_df = pd.Series([total_parameters_quantity, total_memory,\n total_operation_quantity, total_flops,\n total_duration, mread, mwrite, total_memrw],\n index=['params', 'memory(MB)', 'MAdd', 'Flops', 'duration[%]',\n 'MemRead(B)', 'MemWrite(B)', 'MemR+W(B)'],\n name='total')\n # df_properties = pd.DataFrame(properties, columns=['type'])\n df = df.append([total_df])\n\n df = df.fillna(' ')\n df['memory(MB)'] = df['memory(MB)'].apply(\n lambda x: '{:.2f}'.format(x))\n df['duration[%]'] = df['duration[%]'].apply(lambda x: '{:.2%}'.format(x))\n df['MAdd'] = df['MAdd'].apply(lambda x: '{:,}'.format(x))\n df['Flops'] = df['Flops'].apply(lambda x: '{:,}'.format(x))\n\n summary = str(df) + '\\n'\n summary += \"=\" * len(str(df).split('\\n')[0])\n summary += '\\n'\n summary += \"Total params: {:,}\\n\".format(total_parameters_quantity)\n\n summary += \"-\" * len(str(df).split('\\n')[0])\n summary += '\\n'\n summary += \"Total memory: {:.2f}MB\\n\".format(total_memory)\n summary += \"Total MAdd: {}MAdd\\n\".format(round_value(total_operation_quantity))\n summary += \"Total Flops: {}Flops\\n\".format(round_value(total_flops))\n summary += \"Total MemR+W: {}B\\n\".format(round_value(total_memrw, True))\n return summary\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/stat_tree.py", "content": "import queue\n\n\nclass StatTree(object):\n def __init__(self, root_node):\n assert isinstance(root_node, StatNode)\n\n self.root_node = root_node\n\n def get_same_level_max_node_depth(self, query_node):\n if query_node.name == self.root_node.name:\n return 0\n same_level_depth = max([child.depth for child in query_node.parent.children])\n return same_level_depth\n\n def update_stat_nodes_granularity(self):\n q = queue.Queue()\n q.put(self.root_node)\n while not q.empty():\n node = q.get()\n node.granularity = self.get_same_level_max_node_depth(node)\n for child in node.children:\n q.put(child)\n\n def get_collected_stat_nodes(self, debug_layers, query_granularity):\n self.update_stat_nodes_granularity()\n\n collected_nodes = []\n stack = list()\n stack.append(self.root_node)\n while len(stack) > 0:\n node = stack.pop()\n if any([L in node.mtype for L in debug_layers]): #node.name\n collected_nodes.append(node)\n for child in reversed(node.children):\n stack.append(child)\n if node.depth == query_granularity:\n collected_nodes.append(node)\n if node.depth < query_granularity <= node.granularity:\n collected_nodes.append(node)\n return collected_nodes\n\n\nclass StatNode(object):\n def __init__(self, name=str(), mtype=str(), parent=None):\n self._name = name\n self._mtype = mtype\n self._input_shape = None\n self._output_shape = None\n self._parameter_quantity = 0\n self._inference_memory = 0\n self._MAdd = 0\n self._Memory = (0, 0)\n self._Flops = 0\n self._duration = 0\n self._duration_percent = 0\n\n self._granularity = 1\n self._depth = 1\n self.parent = parent\n self.children = list()\n\n @property\n def name(self):\n return self._name\n\n @name.setter\n def name(self, name):\n self._name = name\n\n @property\n def mtype(self):\n return self._mtype\n\n @mtype.setter\n def mtype(self, mtype):\n self._mtype = mtype\n\n @property\n def granularity(self):\n return self._granularity\n\n @granularity.setter\n def granularity(self, g):\n self._granularity = g\n\n @property\n def depth(self):\n d = self._depth\n if len(self.children) > 0:\n d += max([child.depth for child in self.children])\n return d\n\n @property\n def input_shape(self):\n if len(self.children) == 0: # leaf\n return self._input_shape\n else:\n return self.children[0].input_shape\n\n @input_shape.setter\n def input_shape(self, input_shape):\n assert isinstance(input_shape, (list, tuple))\n self._input_shape = input_shape\n\n @property\n def output_shape(self):\n if len(self.children) == 0: # leaf\n return self._output_shape\n else:\n return self.children[-1].output_shape\n\n @output_shape.setter\n def output_shape(self, output_shape):\n assert isinstance(output_shape, (list, tuple))\n self._output_shape = output_shape\n\n @property\n def parameter_quantity(self):\n # return self.parameters_quantity\n total_parameter_quantity = self._parameter_quantity\n # for child in self.children:\n # total_parameter_quantity += child.parameter_quantity\n return total_parameter_quantity\n\n @parameter_quantity.setter\n def parameter_quantity(self, parameter_quantity):\n assert parameter_quantity >= 0\n self._parameter_quantity = parameter_quantity\n\n @property\n def inference_memory(self):\n total_inference_memory = self._inference_memory\n for child in self.children:\n total_inference_memory += child.inference_memory\n return total_inference_memory\n\n @inference_memory.setter\n def inference_memory(self, inference_memory):\n self._inference_memory = inference_memory\n\n @property\n def MAdd(self):\n total_MAdd = self._MAdd\n # for child in self.children:\n # total_MAdd += child.MAdd\n return total_MAdd\n\n @MAdd.setter\n def MAdd(self, MAdd):\n self._MAdd = MAdd\n\n @property\n def Flops(self):\n total_Flops = self._Flops\n # for child in self.children:\n # total_Flops += child.Flops\n return total_Flops\n\n @Flops.setter\n def Flops(self, Flops):\n self._Flops = Flops\n\n @property\n def Memory(self):\n total_Memory = self._Memory\n # for child in self.children:\n # total_Memory[0] += child.Memory[0]\n # total_Memory[1] += child.Memory[1]\n # print(total_Memory)\n return total_Memory\n\n @Memory.setter\n def Memory(self, Memory):\n assert isinstance(Memory, (list, tuple))\n self._Memory = Memory\n\n @property\n def duration(self):\n total_duration = self._duration\n # for child in self.children:\n # total_duration += child.duration\n return total_duration\n\n @duration.setter\n def duration(self, duration):\n self._duration = duration\n\n def find_child_index(self, child_name):\n assert isinstance(child_name, str)\n\n index = -1\n for i in range(len(self.children)):\n if child_name == self.children[i].name:\n index = i\n return index\n\n def add_child(self, node):\n assert isinstance(node, StatNode)\n\n if self.find_child_index(node.name) == -1: # not exist\n self.children.append(node)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/torchstat/statistics.py", "content": "import torch\nimport torch.nn as nn\nfrom torchstat import ModelHook\nfrom collections import OrderedDict\nfrom torchstat import StatTree, StatNode, report_format\n\n\ndef get_parent_node(root_node, stat_node_name):\n assert isinstance(root_node, StatNode)\n\n node = root_node\n names = stat_node_name.split('.')\n for i in range(len(names) - 1):\n node_name = '.'.join(names[0:i+1])\n child_index = node.find_child_index(node_name)\n assert child_index != -1\n node = node.children[child_index]\n return node\n\n\ndef convert_leaf_modules_to_stat_tree(leaf_modules):\n assert isinstance(leaf_modules, OrderedDict)\n\n create_index = 1\n root_node = StatNode(name='root', parent=None)\n for leaf_module_name, leaf_module in leaf_modules.items():\n if 'model.body.decoder.layers.0.self_attn' in leaf_module_name:\n print(\"111\", leaf_module_name, leaf_module.__class__.__name__)\n names = leaf_module_name.split('.')\n for i in range(len(names)):\n create_index += 1\n stat_node_name = '.'.join(names[0:i+1])\n parent_node = get_parent_node(root_node, stat_node_name)\n node = StatNode(name=stat_node_name, mtype=leaf_module.__base__ if hasattr(leaf_module, '__base__') else leaf_module.__class__.__name__, parent=parent_node)#.__class__.__name__\n parent_node.add_child(node)\n if i == len(names) - 1: # leaf module itself\n input_shape = leaf_module.input_shape.numpy().tolist()\n output_shape = leaf_module.output_shape.numpy().tolist()\n node.input_shape = input_shape\n node.output_shape = output_shape\n node.parameter_quantity = leaf_module.parameter_quantity.numpy()[0]\n node.inference_memory = leaf_module.inference_memory.numpy()[0]\n node.MAdd = leaf_module.MAdd.numpy()[0]\n node.Flops = leaf_module.Flops.numpy()[0]\n node.duration = leaf_module.duration.numpy()[0]\n node.Memory = leaf_module.Memory.numpy().tolist()\n return StatTree(root_node)\n\n\nclass ModelStat(object):\n def __init__(self, model, input_size, query_granularity=1, debug_layers=[]):\n assert isinstance(model, nn.Module)\n # assert isinstance(input_size, (tuple, list)) and len(input_size) == 3\n self._model = model\n self._input_size = input_size\n self._query_granularity = query_granularity\n self.debug_layers = debug_layers\n\n def _analyze_model(self):\n model_hook = ModelHook(self._model, self._input_size, debug_layers=self.debug_layers)\n leaf_modules = model_hook.retrieve_leaf_modules()\n stat_tree = convert_leaf_modules_to_stat_tree(leaf_modules)\n collected_nodes = stat_tree.get_collected_stat_nodes(self.debug_layers, self._query_granularity)\n model_hook.clear_hooks()\n return collected_nodes\n\n def show_report(self):\n collected_nodes = self._analyze_model()\n report = report_format(collected_nodes)\n print(report)\n\ndef stat(model, input_size, query_granularity=1, debug_layers=[\"MSA\", \"SwinTEB\", \"XCTEB\", \"MSA_BNC\", 'cGCN', 'sGCN']):\n ms = ModelStat(model, input_size, query_granularity, debug_layers)\n ms.show_report()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/auxiliary/utils.py", "content": "import os\nimport datetime\nimport torch\nimport psutil\nfrom collections import defaultdict, deque\nimport time\nimport sys\nsys.path.append('../..')\nsys.path.append('../mmcv')\nfrom mmcv.utils.logging import print_log\nimport numpy as np\nimport random\nimport torch.backends.cudnn as cudnn\nimport torch.distributed as dist\nfrom functools import partial\n\ndef get_dist_info():\n if dist.is_available() and dist.is_initialized():\n rank = dist.get_rank()\n world_size = dist.get_world_size()\n else:\n rank = 0\n world_size = 1\n return rank, world_size\n\ndef init_random_seed(seed=None, device='cuda'):\n \"\"\"Initialize random seed.\n\n If the seed is not set, the seed will be automatically randomized,\n and then broadcast to all processes to prevent some potential bugs.\n\n Args:\n seed (int, Optional): The seed. Default to None.\n device (str): The device where the seed will be put on.\n Default to 'cuda'.\n\n Returns:\n int: Seed to be used.\n \"\"\"\n if seed is not None:\n return seed\n\n # Make sure all ranks share the same random seed to prevent\n # some potential bugs. Please refer to\n # https://github.com/open-mmlab/mmdetection/issues/6339\n rank, world_size = get_dist_info()\n seed = np.random.randint(2**31)\n if world_size == 1:\n return seed\n\n if rank == 0:\n random_num = torch.tensor(seed, dtype=torch.int32, device=device)\n else:\n random_num = torch.tensor(0, dtype=torch.int32, device=device)\n dist.broadcast(random_num, src=0)\n return random_num.item()\n\ndef set_random_seed(seed, deterministic=True):\n \"\"\"Set random seed.\n\n Args:\n seed (int): Seed to be used.\n deterministic (bool): Whether to set the deterministic option for\n CUDNN backend, i.e., set `torch.backends.cudnn.deterministic`\n to True and `torch.backends.cudnn.benchmark` to False.\n Default: False.\n \"\"\"\n random.seed(seed)\n np.random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.cuda.manual_seed_all(seed)\n if deterministic:\n torch.backends.cudnn.deterministic = True\n torch.backends.cudnn.benchmark = False\n\n# def set_random_seed(seed):\n# np.random.seed(seed)\n# random.seed(seed)\n# torch.manual_seed(seed)\n# torch.cuda.manual_seed(seed)\n# torch.cuda.manual_seed_all(seed)\n# cudnn.deterministic = True\n\n\ndef show_memory_info(hint):\n pid = os.getpid()\n p = psutil.Process(pid)\n\n info = p.memory_full_info()\n memory = info.uss / 1024. / 1024\n print('{} memory used: {} MB'.format(hint, memory))\n\n\n# class OrderedAverageMeter(object):\n# def __init__(self):\n\n\nclass AverageMeter(object):\n \"\"\"Computes and stores the average and current value\"\"\"\n\n def __init__(self, name=None, fmt=\":f\"):\n # self.name = name\n # self.fmt = fmt\n self.reset()\n\n def reset(self):\n self.val = 0\n self.avg = 0\n self.sum = 0\n self.count = 0\n\n def update(self, val, n=1):\n self.val = val\n self.sum += val * n\n self.count += n\n self.avg = self.sum / self.count\n\n # def __str__(self):\n # fmtstr = '{name} {val' + self.fmt + '} ({avg' + self.fmt + '})'\n # return fmtstr.format(**self.__dict__)\n\n\nclass ProgressMeter(object):\n def __init__(self, num_batches, meters, prefix=\"\"):\n self.batch_fmtstr = self._get_batch_fmtstr(num_batches)\n self.meters = meters\n self.prefix = prefix\n\n def display(self, batch):\n entries = [self.prefix + self.batch_fmtstr.format(batch)]\n entries += [str(meter) for meter in self.meters]\n print('\\t'.join(entries))\n\n def _get_batch_fmtstr(self, num_batches):\n num_digits = len(str(num_batches // 1))\n fmt = '{:' + str(num_digits) + 'd}'\n return '[' + fmt + '/' + fmt.format(num_batches) + ']'\n\n\ndef accuracy(output, target, topk=(1,)):\n \"\"\"Computes the precision@k for the specified values of k\"\"\"\n with torch.no_grad():\n maxk = max(topk)\n batch_size = target.size(0)\n\n _, pred = output.topk(maxk, 1, True, True)\n pred = pred.t()\n correct = pred.eq(target.view(1, -1).expand_as(pred))\n\n res = []\n for k in topk:\n correct_k = correct[:k].reshape(-1).float().sum(0, keepdim=True)\n res.append(correct_k.mul_(100.0 / batch_size))\n return res\n\n\n# class logger():\n# def __init__(self, obj, LOG_DIR, parser):\n# logname = 'log_train' + datetime.datetime.now().strftime('%Y_%m_%d-%H_%M_%S')+'.txt'\n# self.LOG_FOUT = open(os.path.join(LOG_DIR, logname), 'w')\n# self.LOG_FOUT.write(str(parser)+'\\n')\n# def __call__(self, out_str):\n# self.LOG_FOUT.write(out_str+'\\n')\n# self.LOG_FOUT.flush()\n# print(out_str)\n\ndef is_dist_avail_and_initialized():\n if not dist.is_available():\n return False\n if not dist.is_initialized():\n return False\n return True\n\n\nclass SmoothedValue(object):\n \"\"\"Track a series of values and provide access to smoothed values over a\n window or the global series average.\n \"\"\"\n\n def __init__(self, window_size=20, fmt=None, eval=False):\n if fmt is None:\n if not eval:\n fmt = \"{value:.7f} (avg:{avg:.7f})\"\n else:\n fmt = \"{value:.7f} (avg:{avg:.7f}, std:{std:.7f})\"\n self.reset(window_size)\n self.fmt = fmt\n\n def reset(self, window_size):\n self.deque = deque(maxlen=window_size)\n self.val = 0\n self.avg = 0\n self.total = 0\n self.count = 0\n\n def update(self, value, n=1):\n self.deque.append(value)\n self.val = value\n self.count += n\n self.total += value * n\n self.avg = self.total / self.count\n\n def synchronize_between_processes(self):\n \"\"\"\n Warning: does not synchronize the deque!\n \"\"\"\n if not is_dist_avail_and_initialized():\n return\n t = torch.tensor([self.val, self.count, self.total], dtype=torch.float64, device='cuda')\n dist.barrier()\n dist.all_reduce(t)\n t = t.tolist()\n self.val = t[0]\n self.count = int(t[1])\n self.total = t[2]\n self.avg = self.total / self.count\n\n @property\n def median(self):\n d = torch.tensor(list(self.deque))\n return d.median().item()\n\n @property\n def std(self):\n return torch.tensor(list(self.deque)).std().item()\n\n # @property\n # def avg(self):\n # d = torch.tensor(list(self.deque), dtype=torch.float32)\n # return d.mean().item()\n\n # @property\n # def global_avg(self):\n # return self.total / self.count\n\n @property\n def max(self):\n return max(self.deque)\n\n #\n # @property\n # def value(self):\n # return self.deque[-1]\n\n def __str__(self):\n # return self.fmt.format(\n # median=self.median,\n # avg=self.avg,\n # global_avg=self.global_avg,\n # max=self.max,\n # value=self.value)\n return self.fmt.format(\n median=self.median,\n avg=self.avg,\n max=self.max,\n value=self.val,\n std=self.std)\n\n\nclass MetricLogger(object):\n\n\n\n def __init__(self, logger=None, delimiter=\"\\t\", dist_print=0, window_size=20, eval=False):\n self.meters = defaultdict(partial(SmoothedValue, window_size=window_size, eval=eval))\n self.delimiter = delimiter\n self.dist_print = dist_print\n # self.log = get_root_logger(\"UDL\")\n # self.logger = logger\n\n # {k:v}打印，对每个k都有val、avg、max、deque属性\n def update(self, **kwargs):\n # dist.barrier()\n for k, v in kwargs.items():\n if isinstance(v, torch.Tensor):\n v = torch.mean(v)\n if hasattr(v, 'item'):\n v = v.item()\n assert isinstance(v, (float, int, str)), print(\"type: \", type(v))\n self.meters[k].update(v)\n\n # {k:v}打印，对每个k都有val、avg、max、deque属性\n def update_dict(self, kwargs: dict):\n # dist.barrier()\n for k, v in kwargs.items():\n if isinstance(v, torch.Tensor):\n v = torch.mean(v)\n if hasattr(v, 'item'):\n v = v.item()\n assert isinstance(v, (float, int, str)), print(\"type: \", type(v))\n self.meters[k].update(v)\n\n def __getattr__(self, attr):\n if attr in self.meters:\n return self.meters[attr]\n if attr in self.__dict__:\n return self.__dict__[attr]\n raise AttributeError(\"'{}' object has no attribute '{}'\".format(\n type(self).__name__, attr))\n\n def __str__(self):\n loss_str = []\n for name, meter in self.meters.items():\n loss_str.append(\n \"{}: {}\".format(name, str(meter))\n )\n return self.delimiter.join(loss_str)\n\n def synchronize_between_processes(self):\n for meter in self.meters.values():\n meter.synchronize_between_processes()\n\n def add_meter(self, name, meter):\n self.meters[name] = meter\n\n def clear(self):\n self.meters.clear()\n\n def log_every(self, iterable, print_freq, header=None):\n i = 1\n if not header:\n header = ''\n start_time = time.time()\n end = time.time()\n iter_time = SmoothedValue(fmt='{avg:.4f}')\n data_time = SmoothedValue(fmt='{avg:.4f}')\n space_fmt = ':' + str(len(str(len(iterable)))) + 'd'\n if torch.cuda.is_available():\n log_msg = self.delimiter.join([\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}',\n 'max mem: {memory:.0f}MB'\n ])\n else:\n log_msg = self.delimiter.join([\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}'\n ])\n MB = 1024.0 * 1024.0\n # log_string = self.logger.info\n for obj in iterable:\n data_time.update(time.time() - end)\n yield obj, i\n iter_time.update(time.time() - end)\n if i % print_freq == 0 or i == len(iterable):\n eta_seconds = iter_time.avg * (len(iterable) - i)\n eta_string = str(datetime.timedelta(seconds=int(eta_seconds)))\n if torch.cuda.is_available():\n if self.dist_print == 0:\n print_log(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time),\n memory=torch.cuda.max_memory_allocated() / MB))\n\n else:\n print_log(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time)))\n i += 1\n end = time.time()\n total_time = time.time() - start_time\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n if self.dist_print == 0:\n print_log('{} Total time: {} ({:.4f} s / it)'.format(\n header, total_time_str, total_time / len(iterable)))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/cal_ssim.py", "content": "# GPL License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn.functional as F\nfrom torch.autograd import Variable\nimport numpy as np\nfrom math import exp\n\ndef gaussian(window_size, sigma):\n gauss = torch.Tensor([exp(-(x - window_size//2)**2/float(2*sigma**2)) for x in range(window_size)])\n return gauss/gauss.sum()\n\ndef create_window(window_size, channel, sigma=1.5):\n _1D_window = gaussian(window_size, sigma).unsqueeze(1)\n _2D_window = _1D_window.mm(_1D_window.t()).float().unsqueeze(0).unsqueeze(0)\n window = Variable(_2D_window.expand(channel, 1, window_size, window_size).contiguous())\n return window\n\ndef _ssim(img1, img2, window, window_size, channel, size_average = True):\n mu1 = F.conv2d(img1, window, padding = window_size//2, groups = channel)\n mu2 = F.conv2d(img2, window, padding = window_size//2, groups = channel)\n\n mu1_sq = mu1.pow(2)\n mu2_sq = mu2.pow(2)\n mu1_mu2 = mu1*mu2\n\n sigma1_sq = F.conv2d(img1*img1, window, padding = window_size//2, groups = channel) - mu1_sq\n sigma2_sq = F.conv2d(img2*img2, window, padding = window_size//2, groups = channel) - mu2_sq\n sigma12 = F.conv2d(img1*img2, window, padding = window_size//2, groups = channel) - mu1_mu2\n\n C1 = 0.01**2\n C2 = 0.03**2\n\n ssim_map = ((2*mu1_mu2 + C1)*(2*sigma12 + C2))/((mu1_sq + mu2_sq + C1)*(sigma1_sq + sigma2_sq + C2))\n\n if size_average:\n return ssim_map.mean()\n else:\n return ssim_map.mean(1).mean(1).mean(1)\n\nclass SSIM(torch.nn.Module):\n def __init__(self, win_size=11, win_sigma=1.5, data_range=1, size_average=True, channel=3):\n super(SSIM, self).__init__()\n self.window_size = win_size\n self.size_average = size_average\n self.channel = channel\n self.window = create_window(win_size, self.channel, win_sigma)\n self.win_sigma = win_sigma\n\n def forward(self, img1, img2):\n #print(img1.size())\n (_, channel, _, _) = img1.size()\n\n if channel == self.channel and self.window.data.type() == img1.data.type():\n window = self.window\n else:\n window = create_window(self.window_size, channel, self.win_sigma)\n \n if img1.is_cuda:\n window = window.cuda(img1.get_device())\n window = window.type_as(img1)\n \n self.window = window\n self.channel = channel\n\n\n return _ssim(img1, img2, window, self.window_size, channel, self.size_average)\n\ndef ssim(img1, img2, win_size = 11, data_range=1, size_average = True):\n (_, channel, _, _) = img1.size()\n window = create_window(win_size, channel)\n \n if img1.is_cuda:\n window = window.cuda(img1.get_device())\n window = window.type_as(img1)\n \n return _ssim(img1, img2, window, win_size, channel, size_average)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/config.py", "content": "# Copyright (c) Open-MMLab. All rights reserved.\n# GPL License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport ast\nimport copy\nimport os\nimport os.path as osp\nimport platform\nimport shutil\nimport sys\nimport tempfile\nimport uuid\nimport warnings\nfrom argparse import Action, ArgumentParser, Namespace\nfrom collections import abc\nfrom importlib import import_module\n\nfrom addict import Dict\nfrom yapf.yapflib.yapf_api import FormatCode\n\n# from .misc import import_modules_from_strings\n# from .path import check_file_exist\n\n\ndef import_modules_from_strings(imports, allow_failed_imports=False):\n \"\"\"Import modules from the given list of strings.\n\n Args:\n imports (list | str | None): The given module names to be imported.\n allow_failed_imports (bool): If True, the failed imports will return\n None. Otherwise, an ImportError is raise. Default: False.\n\n Returns:\n list[module] | module | None: The imported modules.\n\n Examples:\n >>> osp, sys = import_modules_from_strings(\n ... ['os.path', 'sys'])\n >>> import os.path as osp_\n >>> import sys as sys_\n >>> assert osp == osp_\n >>> assert sys == sys_\n \"\"\"\n if not imports:\n return\n single_import = False\n if isinstance(imports, str):\n single_import = True\n imports = [imports]\n if not isinstance(imports, list):\n raise TypeError(\n f'custom_imports must be a list but got type {type(imports)}')\n imported = []\n for imp in imports:\n if not isinstance(imp, str):\n raise TypeError(\n f'{imp} is of type {type(imp)} and cannot be imported.')\n try:\n imported_tmp = import_module(imp)\n except ImportError:\n if allow_failed_imports:\n warnings.warn(f'{imp} failed to import and is ignored.',\n UserWarning)\n imported_tmp = None\n else:\n raise ImportError\n imported.append(imported_tmp)\n if single_import:\n imported = imported[0]\n return imported\n\n\ndef check_file_exist(filename, msg_tmpl='file \"{}\" does not exist'):\n if not osp.isfile(filename):\n raise FileNotFoundError(msg_tmpl.format(filename))\n\nif platform.system() == 'Windows':\n import regex as re\nelse:\n import re\n\nBASE_KEY = '_base_'\nDELETE_KEY = '_delete_'\nRESERVED_KEYS = ['filename', 'text', 'pretty_text']\n\n\nclass ConfigDict(Dict):\n\n def __missing__(self, name):\n raise KeyError(name)\n\n def __getattr__(self, name):\n try:\n value = super(ConfigDict, self).__getattr__(name)\n except KeyError:\n ex = AttributeError(f\"'{self.__class__.__name__}' object has no \"\n f\"attribute '{name}'\")\n except Exception as e:\n ex = e\n else:\n return value\n raise ex\n\n\ndef add_args(parser, cfg, prefix=''):\n for k, v in cfg.items():\n if isinstance(v, str):\n parser.add_argument('--' + prefix + k)\n elif isinstance(v, int):\n parser.add_argument('--' + prefix + k, type=int)\n elif isinstance(v, float):\n parser.add_argument('--' + prefix + k, type=float)\n elif isinstance(v, bool):\n parser.add_argument('--' + prefix + k, action='store_true')\n elif isinstance(v, dict):\n add_args(parser, v, prefix + k + '.')\n elif isinstance(v, abc.Iterable):\n parser.add_argument('--' + prefix + k, type=type(v[0]), nargs='+')\n else:\n print(f'cannot parse key {prefix + k} of type {type(v)}')\n return parser\n\n\nclass Config:\n \"\"\"A facility for config and config files.\n\n It supports common file formats as configs: python/json/yaml. The interface\n is the same as a dict object and also allows access config values as\n attributes.\n\n Example:\n >>> cfg = Config(dict(a=1, b=dict(b1=[0, 1])))\n >>> cfg.a\n 1\n >>> cfg.b\n {'b1': [0, 1]}\n >>> cfg.b.b1\n [0, 1]\n >>> cfg = Config.fromfile('tests/data/config/a.py')\n >>> cfg.filename\n \"/home/kchen/projects/mmcv/tests/data/config/a.py\"\n >>> cfg.item4\n 'test'\n >>> cfg\n \"Config [path: /home/kchen/projects/mmcv/tests/data/config/a.py]: \"\n \"{'item1': [1, 2], 'item2': {'a': 0}, 'item3': True, 'item4': 'test'}\"\n \"\"\"\n\n @staticmethod\n def _validate_py_syntax(filename):\n with open(filename, 'r', encoding='utf-8') as f:\n # Setting encoding explicitly to resolve coding issue on windows\n content = f.read()\n try:\n ast.parse(content)\n except SyntaxError as e:\n raise SyntaxError('There are syntax errors in config '\n f'file {filename}: {e}')\n\n @staticmethod\n def _substitute_predefined_vars(filename, temp_config_name):\n file_dirname = osp.dirname(filename)\n file_basename = osp.basename(filename)\n file_basename_no_extension = osp.splitext(file_basename)[0]\n file_extname = osp.splitext(filename)[1]\n support_templates = dict(\n fileDirname=file_dirname,\n fileBasename=file_basename,\n fileBasenameNoExtension=file_basename_no_extension,\n fileExtname=file_extname)\n with open(filename, 'r', encoding='utf-8') as f:\n # Setting encoding explicitly to resolve coding issue on windows\n config_file = f.read()\n for key, value in support_templates.items():\n regexp = r'\\{\\{\\s*' + str(key) + r'\\s*\\}\\}'\n value = value.replace('\\\\', '/')\n config_file = re.sub(regexp, value, config_file)\n with open(temp_config_name, 'w') as tmp_config_file:\n tmp_config_file.write(config_file)\n\n @staticmethod\n def _pre_substitute_base_vars(filename, temp_config_name):\n \"\"\"Substitute base variable placehoders to string, so that parsing\n would work.\"\"\"\n with open(filename, 'r', encoding='utf-8') as f:\n # Setting encoding explicitly to resolve coding issue on windows\n config_file = f.read()\n base_var_dict = {}\n regexp = r'\\{\\{\\s*' + BASE_KEY + r'\\.([\\w\\. ]+)\\s*\\}\\}'\n base_vars = set(re.findall(regexp, config_file))\n for base_var in base_vars:\n randstr = f'_{base_var}_{uuid.uuid4().hex.lower()[:6]}'\n base_var_dict[randstr] = base_var\n regexp = r'\\{\\{\\s*' + BASE_KEY + r'\\.' + base_var + r'\\s*\\}\\}'\n config_file = re.sub(regexp, f'\"{randstr}\"', config_file)\n with open(temp_config_name, 'w') as tmp_config_file:\n tmp_config_file.write(config_file)\n return base_var_dict\n\n @staticmethod\n def _substitute_base_vars(cfg, base_var_dict, base_cfg):\n \"\"\"Substitute variable strings to their actual values.\"\"\"\n cfg = copy.deepcopy(cfg)\n\n if isinstance(cfg, dict):\n for k, v in cfg.items():\n if isinstance(v, str) and v in base_var_dict:\n new_v = base_cfg\n for new_k in base_var_dict[v].split('.'):\n new_v = new_v[new_k]\n cfg[k] = new_v\n elif isinstance(v, (list, tuple, dict)):\n cfg[k] = Config._substitute_base_vars(\n v, base_var_dict, base_cfg)\n elif isinstance(cfg, tuple):\n cfg = tuple(\n Config._substitute_base_vars(c, base_var_dict, base_cfg)\n for c in cfg)\n elif isinstance(cfg, list):\n cfg = [\n Config._substitute_base_vars(c, base_var_dict, base_cfg)\n for c in cfg\n ]\n elif isinstance(cfg, str) and cfg in base_var_dict:\n new_v = base_cfg\n for new_k in base_var_dict[cfg].split('.'):\n new_v = new_v[new_k]\n cfg = new_v\n\n return cfg\n\n @staticmethod\n def _file2dict(filename, use_predefined_variables=True):\n filename = osp.abspath(osp.expanduser(filename))\n check_file_exist(filename)\n fileExtname = osp.splitext(filename)[1]\n if fileExtname not in ['.py', '.json', '.yaml', '.yml']:\n raise IOError('Only py/yml/yaml/json type are supported now!')\n\n with tempfile.TemporaryDirectory() as temp_config_dir:\n temp_config_file = tempfile.NamedTemporaryFile(\n dir=temp_config_dir, suffix=fileExtname)\n if platform.system() == 'Windows':\n temp_config_file.close()\n temp_config_name = osp.basename(temp_config_file.name)\n # Substitute predefined variables\n if use_predefined_variables:\n Config._substitute_predefined_vars(filename,\n temp_config_file.name)\n else:\n shutil.copyfile(filename, temp_config_file.name)\n # Substitute base variables from placeholders to strings\n base_var_dict = Config._pre_substitute_base_vars(\n temp_config_file.name, temp_config_file.name)\n\n if filename.endswith('.py'):\n temp_module_name = osp.splitext(temp_config_name)[0]\n sys.path.insert(0, temp_config_dir)\n Config._validate_py_syntax(filename)\n mod = import_module(temp_module_name)\n sys.path.pop(0)\n cfg_dict = {}\n for name, value in mod.__dict__.items():\n if not name.startswith('__'):\n if callable(value):\n name = 'data'\n cfg_dict.update({\n name: value\n })\n # cfg_dict = {name: value for name, value in mod.__dict__.items() if not name.startswith('__')}\n # delete imported module\n del sys.modules[temp_module_name]\n elif filename.endswith(('.yml', '.yaml', '.json')):\n import mmcv\n cfg_dict = mmcv.load(temp_config_file.name)\n # close temp file\n temp_config_file.close()\n\n cfg_text = filename + '\\n'\n with open(filename, 'r', encoding='utf-8') as f:\n # Setting encoding explicitly to resolve coding issue on windows\n cfg_text += f.read()\n\n if BASE_KEY in cfg_dict:\n cfg_dir = osp.dirname(filename)\n base_filename = cfg_dict.pop(BASE_KEY)\n base_filename = base_filename if isinstance(\n base_filename, list) else [base_filename]\n\n cfg_dict_list = list()\n cfg_text_list = list()\n for f in base_filename:\n _cfg_dict, _cfg_text = Config._file2dict(osp.join(cfg_dir, f))\n cfg_dict_list.append(_cfg_dict)\n cfg_text_list.append(_cfg_text)\n\n base_cfg_dict = dict()\n for c in cfg_dict_list:\n if len(base_cfg_dict.keys() & c.keys()) > 0:\n raise KeyError('Duplicate key is not allowed among bases')\n base_cfg_dict.update(c)\n\n # Subtitute base variables from strings to their actual values\n cfg_dict = Config._substitute_base_vars(cfg_dict, base_var_dict,\n base_cfg_dict)\n\n base_cfg_dict = Config._merge_a_into_b(cfg_dict, base_cfg_dict)\n cfg_dict = base_cfg_dict\n\n # merge cfg_text\n cfg_text_list.append(cfg_text)\n cfg_text = '\\n'.join(cfg_text_list)\n\n return cfg_dict, cfg_text\n\n @staticmethod\n def _merge_a_into_b(a, b, allow_list_keys=False):\n \"\"\"merge dict ``a`` into dict ``b`` (non-inplace).\n\n Values in ``a`` will overwrite ``b``. ``b`` is copied first to avoid\n in-place modifications.\n\n Args:\n a (dict): The source dict to be merged into ``b``.\n b (dict): The origin dict to be fetch keys from ``a``.\n allow_list_keys (bool): If True, int string keys (e.g. '0', '1')\n are allowed in source ``a`` and will replace the element of the\n corresponding index in b if b is a list. Default: False.\n\n Returns:\n dict: The modified dict of ``b`` using ``a``.\n\n Examples:\n # Normally merge a into b.\n >>> Config._merge_a_into_b(\n ... dict(obj=dict(a=2)), dict(obj=dict(a=1)))\n {'obj': {'a': 2}}\n\n # Delete b first and merge a into b.\n >>> Config._merge_a_into_b(\n ... dict(obj=dict(_delete_=True, a=2)), dict(obj=dict(a=1)))\n {'obj': {'a': 2}}\n\n # b is a list\n >>> Config._merge_a_into_b(\n ... {'0': dict(a=2)}, [dict(a=1), dict(b=2)], True)\n [{'a': 2}, {'b': 2}]\n \"\"\"\n b = b.copy()\n for k, v in a.items():\n if allow_list_keys and k.isdigit() and isinstance(b, list):\n k = int(k)\n if len(b) <= k:\n raise KeyError(f'Index {k} exceeds the length of list {b}')\n b[k] = Config._merge_a_into_b(v, b[k], allow_list_keys)\n elif isinstance(v,\n dict) and k in b and not v.pop(DELETE_KEY, False):\n allowed_types = (dict, list) if allow_list_keys else dict\n if not isinstance(b[k], allowed_types):\n raise TypeError(\n f'{k}={v} in child config cannot inherit from base '\n f'because {k} is a dict in the child config but is of '\n f'type {type(b[k])} in base config. You may set '\n f'`{DELETE_KEY}=True` to ignore the base config')\n b[k] = Config._merge_a_into_b(v, b[k], allow_list_keys)\n else:\n b[k] = v\n return b\n\n @staticmethod\n def fromfile(filename,\n use_predefined_variables=True,\n import_custom_modules=True):\n cfg_dict, cfg_text = Config._file2dict(filename,\n use_predefined_variables)\n if import_custom_modules and cfg_dict.get('custom_imports', None):\n import_modules_from_strings(**cfg_dict['custom_imports'])\n return Config(cfg_dict, cfg_text=cfg_text, filename=filename)\n\n @staticmethod\n def fromstring(cfg_str, file_format):\n \"\"\"Generate config from config str.\n\n Args:\n cfg_str (str): Config str.\n file_format (str): Config file format corresponding to the\n config str. Only py/yml/yaml/json type are supported now!\n\n Returns:\n obj:`Config`: Config obj.\n \"\"\"\n if file_format not in ['.py', '.json', '.yaml', '.yml']:\n raise IOError('Only py/yml/yaml/json type are supported now!')\n if file_format != '.py' and 'dict(' in cfg_str:\n # check if users specify a wrong suffix for python\n warnings.warn(\n 'Please check \"file_format\", the file format may be .py')\n with tempfile.NamedTemporaryFile(\n 'w', suffix=file_format, delete=False) as temp_file:\n temp_file.write(cfg_str)\n # on windows, previous implementation cause error\n # see PR 1077 for details\n cfg = Config.fromfile(temp_file.name)\n os.remove(temp_file.name)\n return cfg\n\n @staticmethod\n def auto_argparser(description=None):\n \"\"\"Generate argparser from config file automatically (experimental)\"\"\"\n partial_parser = ArgumentParser(description=description)\n partial_parser.add_argument('--config', help='config file path', default=\"../../dev/config_detr.yml\")\n cfg_file = partial_parser.parse_known_args()[0].config\n cfg = Config.fromfile(cfg_file)\n parser = ArgumentParser(description=description)\n parser.add_argument('config', help='config file path')\n add_args(parser, cfg)\n return parser, cfg\n\n @staticmethod\n def fromargparse(args):\n cfg_dict = {}\n for k, v in args._get_kwargs():\n cfg_dict.update({k: v})\n return cfg_dict\n\n def merge_args2cfg(self, args, allow_list_keys=True):\n\n cfg_dict = super(Config, self).__getattribute__('_cfg_dict')\n option_cfg_dict = self.fromargparse(args) #cfg_dict\n super(Config, self).__setattr__(\n '_cfg_dict',\n Config._merge_a_into_b(\n option_cfg_dict, cfg_dict, allow_list_keys=allow_list_keys))\n\n\n def __init__(self, cfg_dict=None, cfg_text=None, filename=None):\n if cfg_dict is None:\n cfg_dict = dict()\n elif isinstance(cfg_dict, Namespace):\n cfg_dict = self.fromargparse(cfg_dict)\n elif not isinstance(cfg_dict, (dict, Namespace)):\n raise TypeError('cfg_dict must be a dict or Namespace, but '\n f'got {type(cfg_dict)}')\n for key in cfg_dict:\n if key in RESERVED_KEYS:\n raise KeyError(f'{key} is reserved for config file')\n\n super(Config, self).__setattr__('_cfg_dict', ConfigDict(cfg_dict))\n super(Config, self).__setattr__('_filename', filename)\n if cfg_text:\n text = cfg_text\n elif filename:\n with open(filename, 'r') as f:\n text = f.read()\n else:\n text = ''\n super(Config, self).__setattr__('_text', text)\n\n @property\n def filename(self):\n return self._filename\n\n @property\n def text(self):\n return self._text\n\n @property\n def pretty_text(self):\n\n indent = 4\n\n def _indent(s_, num_spaces):\n s = s_.split('\\n')\n if len(s) == 1:\n return s_\n first = s.pop(0)\n s = [(num_spaces * ' ') + line for line in s]\n s = '\\n'.join(s)\n s = first + '\\n' + s\n return s\n\n def _format_basic_types(k, v, use_mapping=False):\n if isinstance(v, str):\n v_str = f\"'{v}'\"\n else:\n v_str = str(v)\n\n if use_mapping:\n k_str = f\"'{k}'\" if isinstance(k, str) else str(k)\n attr_str = f'{k_str}: {v_str}'\n else:\n attr_str = f'{str(k)}={v_str}'\n attr_str = _indent(attr_str, indent)\n\n return attr_str\n\n def _format_list(k, v, use_mapping=False):\n # check if all items in the list are dict\n if all(isinstance(_, dict) for _ in v):\n v_str = '[\\n'\n v_str += '\\n'.join(\n f'dict({_indent(_format_dict(v_), indent)}),'\n for v_ in v).rstrip(',')\n if use_mapping:\n k_str = f\"'{k}'\" if isinstance(k, str) else str(k)\n attr_str = f'{k_str}: {v_str}'\n else:\n attr_str = f'{str(k)}={v_str}'\n attr_str = _indent(attr_str, indent) + ']'\n else:\n attr_str = _format_basic_types(k, v, use_mapping)\n return attr_str\n\n def _contain_invalid_identifier(dict_str):\n contain_invalid_identifier = False\n for key_name in dict_str:\n contain_invalid_identifier |= \\\n (not str(key_name).isidentifier())\n return contain_invalid_identifier\n\n def _format_dict(input_dict, outest_level=False):\n r = ''\n s = []\n\n use_mapping = _contain_invalid_identifier(input_dict)\n if use_mapping:\n r += '{'\n for idx, (k, v) in enumerate(input_dict.items()):\n is_last = idx >= len(input_dict) - 1\n end = '' if outest_level or is_last else ','\n if isinstance(v, dict):\n v_str = '\\n' + _format_dict(v)\n if use_mapping:\n k_str = f\"'{k}'\" if isinstance(k, str) else str(k)\n attr_str = f'{k_str}: dict({v_str}'\n else:\n attr_str = f'{str(k)}=dict({v_str}'\n attr_str = _indent(attr_str, indent) + ')' + end\n elif isinstance(v, list):\n attr_str = _format_list(k, v, use_mapping) + end\n else:\n attr_str = _format_basic_types(k, v, use_mapping) + end\n\n s.append(attr_str)\n r += '\\n'.join(s)\n if use_mapping:\n r += '}'\n return r\n\n cfg_dict = self._cfg_dict.to_dict()\n text = _format_dict(cfg_dict, outest_level=True)\n # copied from setup.cfg\n yapf_style = dict(\n based_on_style='pep8',\n blank_line_before_nested_class_or_def=True,\n split_before_expression_after_opening_paren=True)\n text, _ = FormatCode(text.replace('\\\\', '/'), style_config=yapf_style, verify=True)\n\n return text\n\n def __repr__(self):\n return f'Config (path: {self.filename}): {self._cfg_dict.__repr__()}'\n\n def __len__(self):\n return len(self._cfg_dict)\n\n def __getattr__(self, name):\n return getattr(self._cfg_dict, name)\n\n def __delattr__(self, name):\n return delattr(self._cfg_dict, name)\n\n def __getitem__(self, name):\n return self._cfg_dict.__getitem__(name)\n\n def __setattr__(self, name, value):\n if isinstance(value, dict):\n value = ConfigDict(value)\n self._cfg_dict.__setattr__(name, value)\n\n def __setitem__(self, name, value):\n if isinstance(value, dict):\n value = ConfigDict(value)\n self._cfg_dict.__setitem__(name, value)\n\n def __iter__(self):\n return iter(self._cfg_dict)\n\n def __getstate__(self):\n return (self._cfg_dict, self._filename, self._text)\n\n def __setstate__(self, state):\n _cfg_dict, _filename, _text = state\n super(Config, self).__setattr__('_cfg_dict', _cfg_dict)\n super(Config, self).__setattr__('_filename', _filename)\n super(Config, self).__setattr__('_text', _text)\n\n def dump(self, file=None):\n cfg_dict = super(Config, self).__getattribute__('_cfg_dict').to_dict()\n if self.filename.endswith('.py'):\n if file is None:\n return self.pretty_text\n else:\n with open(file, 'w') as f:\n f.write(self.pretty_text)\n else:\n import mmcv\n if file is None:\n file_format = self.filename.split('.')[-1]\n return mmcv.dump(cfg_dict, file_format=file_format)\n else:\n mmcv.dump(cfg_dict, file)\n\n def merge_from_dict(self, options, allow_list_keys=True):\n \"\"\"Merge list into cfg_dict.\n\n Merge the dict parsed by MultipleKVAction into this cfg.\n\n Examples:\n >>> options = {'model.backbone.depth': 50,\n ... 'model.backbone.with_cp':True}\n >>> cfg = Config(dict(model=dict(backbone=dict(type='ResNet'))))\n >>> cfg.merge_from_dict(options)\n >>> cfg_dict = super(Config, self).__getattribute__('_cfg_dict')\n >>> assert cfg_dict == dict(\n ... model=dict(backbone=dict(depth=50, with_cp=True)))\n\n # Merge list element\n >>> cfg = Config(dict(pipeline=[\n ... dict(type='LoadImage'), dict(type='LoadAnnotations')]))\n >>> options = dict(pipeline={'0': dict(type='SelfLoadImage')})\n >>> cfg.merge_from_dict(options, allow_list_keys=True)\n >>> cfg_dict = super(Config, self).__getattribute__('_cfg_dict')\n >>> assert cfg_dict == dict(pipeline=[\n ... dict(type='SelfLoadImage'), dict(type='LoadAnnotations')])\n\n Args:\n options (dict): dict of configs to merge from.\n allow_list_keys (bool): If True, int string keys (e.g. '0', '1')\n are allowed in ``options`` and will replace the element of the\n corresponding index in the config if the config is a list.\n Default: True.\n \"\"\"\n option_cfg_dict = {}\n for full_key, v in options.items():\n d = option_cfg_dict\n key_list = full_key.split('.')\n for subkey in key_list[:-1]:\n d.setdefault(subkey, ConfigDict())\n d = d[subkey]\n subkey = key_list[-1]\n d[subkey] = v\n\n cfg_dict = super(Config, self).__getattribute__('_cfg_dict')\n super(Config, self).__setattr__(\n '_cfg_dict',\n Config._merge_a_into_b(\n option_cfg_dict, cfg_dict, allow_list_keys=allow_list_keys))\n\n\nclass DictAction(Action):\n \"\"\"\n argparse action to split an argument into KEY=VALUE form\n on the first = and append to a dictionary. List options can\n be passed as comma separated values, i.e 'KEY=V1,V2,V3', or with explicit\n brackets, i.e. 'KEY=[V1,V2,V3]'. It also support nested brackets to build\n list/tuple values. e.g. 'KEY=[(V1,V2),(V3,V4)]'\n \"\"\"\n\n @staticmethod\n def _parse_int_float_bool(val):\n try:\n return int(val)\n except ValueError:\n pass\n try:\n return float(val)\n except ValueError:\n pass\n if val.lower() in ['true', 'false']:\n return True if val.lower() == 'true' else False\n return val\n\n @staticmethod\n def _parse_iterable(val):\n \"\"\"Parse iterable values in the string.\n\n All elements inside '()' or '[]' are treated as iterable values.\n\n Args:\n val (str): Value string.\n\n Returns:\n list | tuple: The expanded list or tuple from the string.\n\n Examples:\n >>> DictAction._parse_iterable('1,2,3')\n [1, 2, 3]\n >>> DictAction._parse_iterable('[a, b, c]')\n ['a', 'b', 'c']\n >>> DictAction._parse_iterable('[(1, 2, 3), [a, b], c]')\n [(1, 2, 3), ['a', 'b'], 'c']\n \"\"\"\n\n def find_next_comma(string):\n \"\"\"Find the position of next comma in the string.\n\n If no ',' is found in the string, return the string length. All\n chars inside '()' and '[]' are treated as one element and thus ','\n inside these brackets are ignored.\n \"\"\"\n assert (string.count('(') == string.count(')')) and (\n string.count('[') == string.count(']')), \\\n f'Imbalanced brackets exist in {string}'\n end = len(string)\n for idx, char in enumerate(string):\n pre = string[:idx]\n # The string before this ',' is balanced\n if ((char == ',') and (pre.count('(') == pre.count(')'))\n and (pre.count('[') == pre.count(']'))):\n end = idx\n break\n return end\n\n # Strip ' and \" characters and replace whitespace.\n val = val.strip('\\'\\\"').replace(' ', '')\n is_tuple = False\n if val.startswith('(') and val.endswith(')'):\n is_tuple = True\n val = val[1:-1]\n elif val.startswith('[') and val.endswith(']'):\n val = val[1:-1]\n elif ',' not in val:\n # val is a single value\n return DictAction._parse_int_float_bool(val)\n\n values = []\n while len(val) > 0:\n comma_idx = find_next_comma(val)\n element = DictAction._parse_iterable(val[:comma_idx])\n values.append(element)\n val = val[comma_idx + 1:]\n if is_tuple:\n values = tuple(values)\n return values\n\n def __call__(self, parser, namespace, values, option_string=None):\n options = {}\n for kv in values:\n key, val = kv.split('=', maxsplit=1)\n options[key] = self._parse_iterable(val)\n setattr(namespace, self.dest, options)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/criterion_metrics.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nfrom torch import nn\nimport torch\nfrom torch import distributed as dist\nfrom collections import OrderedDict\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n\n def __init__(self, losses, weight_dict):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: n able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relatiumber of object categories, omitting the special no-object category\n matcher: moduleve classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.weight_dict = weight_dict\n self.losses = losses\n self.loss_dicts = {}\n\n def forward(self, outputs, targets, *args, **kwargs):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n # Compute all the requested losses\n for k in self.losses.keys():\n # k, loss = loss_dict\n if k == 'loss':\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets))\n else:\n self.loss_dicts.update({k: loss(outputs, targets)})\n else:\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets, *args)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets, *args))\n else:\n self.loss_dicts.update({k: loss(outputs, targets, *args)})\n\n return self.loss_dicts#self._parse_losses(self.loss_dicts)\n\n def _parse_losses(self, losses):\n log_vars = OrderedDict()\n for loss_name, loss_value in losses.items():\n if isinstance(loss_value, torch.Tensor):\n log_vars[loss_name] = loss_value.mean()\n elif isinstance(loss_value, list):\n log_vars[loss_name] = sum(_loss.mean() for _loss in loss_value)\n # top-1, top-5 both belong to accuracy\n elif isinstance(loss_value, dict):\n # log_vars[loss_name] = {}\n for name, value in loss_value.items():\n log_vars[name] = value\n # log_vars[loss_name].update({name: value.item()})\n else:\n raise TypeError(\n f'{loss_name} is not a tensor or list of tensors')\n\n assert 'loss' not in log_vars.keys(), KeyError(\"key: 'loss' can't be set from cfg_file.\")\n loss = sum(_value for _key, _value in log_vars.items()\n if 'top' not in _key)#if 'loss' in _key\n log_vars['loss'] = loss\n # output = log_vars.pop('acc') #get\n for loss_name, loss_value in log_vars.items():\n # reduce loss when distributed training\n if dist.is_available() and dist.is_initialized():\n loss_value = loss_value.data.clone()\n dist.all_reduce(loss_value.div_(dist.get_world_size()))\n log_vars[loss_name] = loss_value.item()\n # log_vars.update(acc=output)\n\n return loss, log_vars" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/dist_utils.py", "content": "import os\nimport subprocess\nimport torch\nfrom torch import nn\nimport torch.multiprocessing as mp\nfrom torch import distributed as dist\nfrom torch.nn.parallel.distributed import DistributedDataParallel\nfrom logging import info as log_string\ntry:\n from apex.parallel.distributed import DistributedDataParallel as DDP\nexcept:\n Warning(\"No module named 'apex\")\n\ndef scaled_all_reduce(tensors):\n \"\"\"Performs the scaled all_reduce operation on the provided tensors.\n The input tensors are modified in-place. Currently supports only the sum\n reduction operator. The reduced values are scaled by the inverse size of the\n process group.\n \"\"\"\n # There is no need for reduction in the single-proc case\n gpus = dist.get_world_size()\n if gpus == 1:\n return tensors\n # Queue the reductions\n reductions = []\n for tensor in tensors:\n reduction = dist.all_reduce(tensor, async_op=True)\n reductions.append(reduction)\n # Wait for reductions to finish\n for reduction in reductions:\n reduction.wait()\n # Scale the results\n for tensor in tensors:\n tensor.mul_(1.0 / gpus)\n return tensors\n\ndef init_dist(launcher, args, backend='nccl', **kwargs):\n if 'LOCAL_RANK' not in os.environ:\n os.environ['LOCAL_RANK'] = str(args.local_rank)\n if mp.get_start_method(allow_none=True) is None:\n mp.set_start_method('spawn')\n if launcher == 'pytorch':\n _init_dist_pytorch(backend, **kwargs)\n elif launcher == 'mpi':\n _init_dist_mpi(backend, **kwargs)\n elif launcher == 'slurm':\n _init_dist_slurm(backend, **kwargs)\n else:\n raise ValueError(f'Invalid launcher type: {launcher}')\n\ndef get_dist_info():\n if dist.is_available():\n initialized = dist.is_initialized()\n else:\n initialized = False\n if initialized:\n rank = dist.get_rank()\n world_size = dist.get_world_size()\n else:\n rank = 0\n world_size = 1\n # print(f\"DDP: {dist.is_available()} {world_size}\")\n return rank, world_size\n\ndef _init_dist_pytorch(backend, **kwargs):\n # TODO: use local_rank instead of rank % num_gpus\n rank = int(os.environ['RANK'])\n num_gpus = torch.cuda.device_count()\n torch.cuda.set_device(rank % num_gpus)\n dist.init_process_group(backend=backend, **kwargs)\n\n\ndef _init_dist_mpi(backend, **kwargs):\n # TODO: use local_rank instead of rank % num_gpus\n rank = int(os.environ['OMPI_COMM_WORLD_RANK'])\n num_gpus = torch.cuda.device_count()\n torch.cuda.set_device(rank % num_gpus)\n dist.init_process_group(backend=backend, **kwargs)\n\n\ndef _init_dist_slurm(backend, port=None, **kwargs):\n \"\"\"Initialize slurm distributed training environment.\n\n If argument ``port`` is not specified, then the master port will be system\n environment variable ``MASTER_PORT``. If ``MASTER_PORT`` is not in system\n environment variable, then a default port ``29500`` will be used.\n\n Args:\n backend (str): Backend of torch.distributed.\n port (int, optional): Master port. Defaults to None.\n \"\"\"\n proc_id = int(os.environ['SLURM_PROCID'])\n ntasks = int(os.environ['SLURM_NTASKS'])\n node_list = os.environ['SLURM_NODELIST']\n num_gpus = torch.cuda.device_count()\n torch.cuda.set_device(proc_id % num_gpus)\n addr = subprocess.getoutput(\n f'scontrol show hostname {node_list} | head -n1')\n # print(proc_id, ntasks, node_list, addr)\n # specify master port\n if port is not None:\n os.environ['MASTER_PORT'] = str(port)\n elif 'MASTER_PORT' in os.environ:\n pass # use MASTER_PORT in the environment variable\n else:\n # 29500 is torch.distributed default port\n os.environ['MASTER_PORT'] = '29500'\n # use MASTER_ADDR in the environment variable if it already exists\n if 'MASTER_ADDR' not in os.environ:\n os.environ['MASTER_ADDR'] = addr\n os.environ['WORLD_SIZE'] = str(ntasks)\n os.environ['LOCAL_RANK'] = str(proc_id % num_gpus)\n os.environ['RANK'] = str(proc_id)\n # print(os.environ)\n dist.init_process_group(backend=backend)\n\ndef reduce_mean(tensor, nprocs=None):\n if nprocs is None:\n _, nprocs = get_dist_info()\n if nprocs == 1:\n return tensor\n # print(\"reduce_mean\", tensor)\n rt = tensor.clone()\n dist.all_reduce(rt, op=dist.ReduceOp.SUM)\n # print(rt, nprocs)\n rt /= nprocs\n # print(rt)\n return rt\n\nclass MMDistributedDataParallel(DistributedDataParallel):\n\n def __init__(self, model, device_ids):\n super(MMDistributedDataParallel, self).__init__(model, device_ids, find_unused_parameters=True)\n\n self.ddp = model\n\n def reduce_mean(self, tensor, nprocs=None):\n if nprocs is None:\n _, nprocs = get_dist_info()\n rt = tensor.clone()\n dist.all_reduce(rt, op=dist.ReduceOp.SUM)\n rt /= nprocs\n return rt\n\n def ddp_step(self, loss_dicts):\n losses = {}\n _, world_size = get_dist_info()\n if world_size == 1:\n return loss_dicts\n dist.barrier()\n # keys = loss_dicts.keys()\n # reduced_loss = scaled_all_reduce(loss_dicts.values())\n # losses = {k: v for k, v in zip(keys, reduced_loss)}\n for k, loss in loss_dicts.items():\n reduced_loss = self.reduce_mean(loss)\n losses.update({k: reduced_loss})\n return losses\n\ndef dist_train_v1(args, model):\n if args.mode == \"DDP\":\n if args.global_rank == 0:\n log_string(f'Distributed training: {args.distributed}')\n if args.distributed:\n if args.amp is not None:\n if not args.amp:\n # delay_allreduce delays all communication to the end of the backward pass.\n log_string(\"IN apex DistributedDataParallel mode.\")\n model = DDP(model, delay_allreduce=True)\n else:\n # model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.local_rank])\n model = MMDistributedDataParallel(model, device_ids=[args.local_rank])\n # train_sampler = torch.auxiliary.data.distributed.DistributedSampler(train_dataset)\n # val_sampler = torch.auxiliary.data.distributed.DistributedSampler(val_dataset)\n elif args.mode == \"DP\":\n log_string(f'DataParallel training')\n model = nn.DataParallel(model, device_ids=args.device_ids)\n\n return model\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/kill_dist.sh", "content": "kill -9 $(ps aux | grep main.py | grep -v grep | awk '{print $2}')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/launch.py", "content": "r\"\"\"\n`torch.distributed.launch` is a module that spawns up multiple distributed\ntraining processes on each of the training nodes.\nThe utility can be used for single-node distributed training, in which one or\nmore processes per node will be spawned. The utility can be used for either\nCPU training or GPU training. If the utility is used for GPU training,\neach distributed process will be operating on a single GPU. This can achieve\nwell-improved single-node training performance. It can also be used in\nmulti-node distributed training, by spawning up multiple processes on each node\nfor well-improved multi-node distributed training performance as well.\nThis will especially be benefitial for systems with multiple Infiniband\ninterfaces that have direct-GPU support, since all of them can be utilized for\naggregated communication bandwidth.\nIn both cases of single-node distributed training or multi-node distributed\ntraining, this utility will launch the given number of processes per node\n(``--nproc_per_node``). If used for GPU training, this number needs to be less\nor equal to the number of GPUs on the current system (``nproc_per_node``),\nand each process will be operating on a single GPU from *GPU 0 to\nGPU (nproc_per_node - 1)*.\n**How to use this module:**\n1. Single-Node multi-process distributed training\n::\n >>> #python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3 and all other\n arguments of your training script)\n2. Multi-Node multi-process distributed training: (e.g. two nodes)\nNode 1: *(IP: 192.168.1.1, and has a free port: 1234)*\n::\n >>> #python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n --nnodes=2 --node_rank=0 --master_addr=\"192.168.1.1\"\n --master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3\n and all other arguments of your training script)\nNode 2:\n::\n >>> #python -m torch.distributed.launch --nproc_per_node=NUM_GPUS_YOU_HAVE\n --nnodes=2 --node_rank=1 --master_addr=\"192.168.1.1\"\n --master_port=1234 YOUR_TRAINING_SCRIPT.py (--arg1 --arg2 --arg3\n and all other arguments of your training script)\n3. To look up what optional arguments this module offers:\n::\n >>> #python -m torch.distributed.launch --help\n**Important Notices:**\n1. This utility and multi-process distributed (single-node or\nmulti-node) GPU training currently only achieves the best performance using\nthe NCCL distributed backend. Thus NCCL backend is the recommended backend to\nuse for GPU training.\n2. In your training program, you must parse the command-line argument:\n``--local_rank=LOCAL_PROCESS_RANK``, which will be provided by this module.\nIf your training program uses GPUs, you should ensure that your code only\nruns on the GPU device of LOCAL_PROCESS_RANK. This can be done by:\nParsing the local_rank argument\n::\n # >>> import argparse\n # >>> parser = argparse.ArgumentParser()\n # >>> parser.add_argument(\"--local_rank\", type=int)\n # >>> args = parser.parse_args()\n# Set your device to local rank using either\n# ::\n# >>> torch.cuda.set_device(args.local_rank) # before your code runs\n# or\n# ::\n# >>> with torch.cuda.device(args.local_rank):\n# >>> # your code to run\n# 3. In your training program, you are supposed to call the following function\nat the beginning to start the distributed backend. You need to make sure that\nthe init_method uses ``env://``, which is the only supported ``init_method``\nby this module.\n::\n torch.distributed.init_process_group(backend='YOUR BACKEND',\n init_method='env://')\n4. In your training program, you can either use regular distributed functions\nor use :func:`torch.nn.parallel.DistributedDataParallel` module. If your\ntraining program uses GPUs for training and you would like to use\n:func:`torch.nn.parallel.DistributedDataParallel` module,\nhere is how to configure it.\n::\n model = torch.nn.parallel.DistributedDataParallel(model,\n device_ids=[args.local_rank],\n output_device=args.local_rank)\nPlease ensure that ``device_ids`` argument is set to be the only GPU device id\nthat your code will be operating on. This is generally the local rank of the\nprocess. In other words, the ``device_ids`` needs to be ``[args.local_rank]``,\nand ``output_device`` needs to be ``args.local_rank`` in order to use this\nutility\n5. Another way to pass ``local_rank`` to the subprocesses via environment variable\n``LOCAL_RANK``. This behavior is enabled when you launch the script with\n``--use_env=True``. You must adjust the subprocess example above to replace\n``args.local_rank`` with ``os.environ['LOCAL_RANK']``; the launcher\nwill not pass ``--local_rank`` when you specify this flag.\n.. warning::\n ``local_rank`` is NOT globally unique: it is only unique per process\n on a machine. Thus, don't use it to decide if you should, e.g.,\n write to a networked filesystem. See\n https://github.com/pytorch/pytorch/issues/12042 for an example of\n how things can go wrong if you don't do this correctly.\n\"\"\"\n\n\nimport time\nimport signal\nimport sys\nimport subprocess\nimport os\nfrom argparse import ArgumentParser, REMAINDER\nfrom typing import Optional, IO, List, Any\n\nnode_local_rank_stdout_filename = \"node_{}_local_rank_{}_stdout\"\nnode_local_rank_stderr_filename = \"node_{}_local_rank_{}_stderr\"\n\ndef parse_args():\n \"\"\"\n Helper function parsing the command line options\n @retval ArgumentParser\n \"\"\"\n parser = ArgumentParser(description=\"PyTorch distributed training launch \"\n \"helper utility that will spawn up \"\n \"multiple distributed processes\")\n\n # Optional arguments for the launch helper\n parser.add_argument(\"--nnodes\", type=int, default=1,\n help=\"The number of nodes to use for distributed \"\n \"training\")\n parser.add_argument(\"--node_rank\", type=int, default=0,\n help=\"The rank of the node for multi-node distributed \"\n \"training\")\n parser.add_argument(\"--nproc_per_node\", type=int, default=1,\n help=\"The number of processes to launch on each node, \"\n \"for GPU training, this is recommended to be set \"\n \"to the number of GPUs in your system so that \"\n \"each process can be bound to a single GPU.\")\n parser.add_argument(\"--master_addr\", default=\"127.0.0.1\", type=str,\n help=\"Master node (rank 0)'s address, should be either \"\n \"the IP address or the hostname of node 0, for \"\n \"single node multi-proc training, the \"\n \"--master_addr can simply be 127.0.0.1\")\n parser.add_argument(\"--master_port\", default=29500, type=int,\n help=\"Master node (rank 0)'s free port that needs to \"\n \"be used for communication during distributed \"\n \"training\")\n parser.add_argument(\"--use_env\", default=False, action=\"store_true\",\n help=\"Use environment variable to pass \"\n \"'local rank'. For legacy reasons, the default value is False. \"\n \"If set to True, the script will not pass \"\n \"--local_rank as argument, and will instead set LOCAL_RANK.\")\n parser.add_argument(\"-m\", \"--module\", default=False, action=\"store_true\",\n help=\"Changes each process to interpret the launch script \"\n \"as a python module, executing with the same behavior as\"\n \"'python -m'.\")\n parser.add_argument(\"--no_python\", default=False, action=\"store_true\",\n help=\"Do not prepend the training script with \\\"python\\\" - just exec \"\n \"it directly. Useful when the script is not a Python script.\")\n parser.add_argument(\n \"--logdir\",\n default=None,\n type=str,\n help=f\"\"\"Relative path to write subprocess logs to. Passing in a relative\n path will create a directory if needed, and write the stdout and stderr to files\n {node_local_rank_stdout_filename} and {node_local_rank_stderr_filename}. Note that\n successive runs with the same path to write logs to will overwrite existing logs,\n so be sure to save logs as needed.\"\"\",\n )\n\n # positional\n parser.add_argument(\"training_script\", type=str,\n help=\"The full path to the single GPU training \"\n \"program/script to be launched in parallel, \"\n \"followed by all the arguments for the \"\n \"training script\")\n\n # rest from the training program\n parser.add_argument('training_script_args', nargs=REMAINDER)\n return parser.parse_args()\n\ndef main():\n args = parse_args()\n\n # world size in terms of number of processes\n dist_world_size = args.nproc_per_node * args.nnodes\n\n # set PyTorch distributed related environmental variables\n current_env = os.environ.copy()\n current_env[\"MASTER_ADDR\"] = args.master_addr\n current_env[\"MASTER_PORT\"] = str(args.master_port)\n current_env[\"WORLD_SIZE\"] = str(dist_world_size)\n\n processes: List[Any] = []\n\n if 'OMP_NUM_THREADS' not in os.environ and args.nproc_per_node > 1:\n current_env[\"OMP_NUM_THREADS\"] = str(1)\n print(\"*****************************************\\n\"\n \"Setting OMP_NUM_THREADS environment variable for each process \"\n \"to be {} in default, to avoid your system being overloaded, \"\n \"please further tune the variable for optimal performance in \"\n \"your application as needed. \\n\"\n \"*****************************************\".format(current_env[\"OMP_NUM_THREADS\"]))\n\n if args.logdir:\n # Possibly create the directory to write subprocess log output to.\n if os.path.exists(args.logdir):\n if not os.path.isdir(args.logdir):\n raise ValueError(\"argument --logdir must be a path to a directory.\")\n else:\n # create the relative directory\n os.mkdir(os.path.join(os.getcwd(), args.logdir))\n\n subprocess_file_handles = []\n\n for local_rank in range(0, args.nproc_per_node):\n # each process's rank\n dist_rank = args.nproc_per_node * args.node_rank + local_rank\n current_env[\"RANK\"] = str(dist_rank)\n current_env[\"LOCAL_RANK\"] = str(local_rank)\n\n # spawn the processes\n with_python = not args.no_python\n cmd = []\n if with_python:\n cmd = [sys.executable, \"-u\"]\n if args.module:\n cmd.append(\"-m\")\n else:\n if not args.use_env:\n raise ValueError(\"When using the '--no_python' flag, you must also set the '--use_env' flag.\")\n if args.module:\n raise ValueError(\"Don't use both the '--no_python' flag and the '--module' flag at the same time.\")\n\n cmd.append(args.training_script)\n\n if not args.use_env:\n cmd.append(\"--local_rank={}\".format(local_rank))\n\n cmd.extend(args.training_script_args)\n\n stdout_handle: Optional[IO]\n stderr_handle: Optional[IO]\n if args.logdir:\n directory_path = os.path.join(os.getcwd(), args.logdir)\n node_rank = args.node_rank\n stdout_file_name = node_local_rank_stdout_filename.format(node_rank, local_rank)\n stderr_file_name = node_local_rank_stderr_filename.format(node_rank, local_rank)\n stdout_handle = open(os.path.join(directory_path, stdout_file_name), \"w\")\n stderr_handle = open(os.path.join(directory_path, stderr_file_name), \"w\")\n subprocess_file_handles.append((stdout_handle, stderr_handle))\n stdout_name = stdout_handle.name\n stderr_name = stderr_handle.name\n print(f\"\"\"Note: Stdout and stderr for node {node_rank} rank {local_rank} will\n be written to {stdout_name}, {stderr_name} respectively.\"\"\")\n\n sig_names = {2: \"SIGINT\", 15: \"SIGTERM\"}\n last_return_code = None\n\n def sigkill_handler(signum, frame):\n for process in processes:\n print(f\"Killing subprocess {process.pid}\")\n try:\n process.kill()\n except Exception as e:\n pass\n if last_return_code is not None:\n raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)\n if signum in sig_names:\n print(f\"Main process received {sig_names[signum]}, exiting\")\n sys.exit(1)\n\n # pass SIGINT/SIGTERM to children if the parent is being terminated\n signal.signal(signal.SIGINT, sigkill_handler)\n signal.signal(signal.SIGTERM, sigkill_handler)\n\n stdout_handle = None if not subprocess_file_handles else subprocess_file_handles[local_rank][0]\n stderr_handle = None if not subprocess_file_handles else subprocess_file_handles[local_rank][1]\n process = subprocess.Popen(cmd, env=current_env, stdout=stdout_handle, stderr=stderr_handle)\n processes.append(process)\n\n try:\n alive_processes = set(processes)\n while len(alive_processes):\n finished_processes = []\n for process in alive_processes:\n if process.poll() is None:\n # the process is still running\n continue\n else:\n if process.returncode != 0:\n last_return_code = process.returncode # for sigkill_handler\n sigkill_handler(signal.SIGTERM, None) # not coming back\n else:\n # exited cleanly\n finished_processes.append(process)\n alive_processes = set(alive_processes) - set(finished_processes)\n\n time.sleep(1)\n finally:\n # close open file descriptors\n for (stdout_handle, stderr_handle) in subprocess_file_handles:\n stdout_handle.close()\n stderr_handle.close()\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/logger.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport json\nfrom collections import defaultdict\nimport logging\nimport os\nimport functools\nimport torch.distributed as dist\nimport colorlog\nimport time\nfrom pathlib import Path\n\nlogger_initialized = {}\n\nlog_colors_config = {\n 'DEBUG': 'cyan',\n 'INFO': 'white',\n 'WARNING': 'yellow',\n 'ERROR': 'red',\n 'CRITICAL': 'red',\n}\n\n\n# def get_root_logger(name, log_file=None, log_level=logging.INFO):\n# return get_logger('mmcls', log_file, log_level)\ndef get_root_logger(name=None, cfg=None, cfg_name=None, log_level=logging.INFO):\n return get_logger(name, cfg, cfg_name, log_level)\n# TODO: Depre\n# the same as \"get_root_logger\"\ndef create_logger(cfg=None, cfg_name=None, dist_print=0, log_level=logging.INFO):\n return get_logger(None, cfg, cfg_name, log_level)\n\n@functools.lru_cache() # so that calling setup_logger multiple times won't add many handlers\ndef setup_logger(name, final_log_file, color=True):\n # LOG_DIR = cfg.log_dir\n # LOG_FOUT = open(final_log_file, 'w')\n # head = '%(asctime)-15s %(message)s'\n\n logging.basicConfig(filename=str(final_log_file).replace('\\\\', '/'), format='%(message)s', level=logging.INFO)\n # logger = logging.getLogger()\n # logger.setLevel(logging.INFO)\n # console = logging.StreamHandler()\n # logging.getLogger('').addHandler(console)\n\n logger = logging.getLogger(name)\n # if name in logger_initialized:\n # return logger\n\n for handler in logger.root.handlers:\n if type(handler) is logging.StreamHandler:\n handler.setLevel(logging.ERROR)\n\n # stream_handler = logging.StreamHandler()\n console = colorlog.StreamHandler()\n handlers = [console]\n\n # logger.setLevel(logging.INFO)\n # formatter = colorlog.ColoredFormatter(\n # '%(log_color)s[%(asctime)s] [%(filename)s:%(lineno)d] [%(module)s:%(funcName)s] [%(levelname)s]- %(message)s',\n # log_colors=log_colors_config) # 日志输出格式\n\n if dist.is_available() and dist.is_initialized():\n rank = dist.get_rank()\n else:\n rank = 0\n\n if rank == 0:\n # console = colorlog.StreamHandler()\n # console.setLevel(logging.DEBUG)\n handlers.append(console)\n # if color:\n # formatter = _ColorfulFormatter(\n # colored(\"%(message)s\", \"green\")\n # )\n # else:\n formatter = colorlog.ColoredFormatter(\n '%(log_color)s- %(message)s',\n log_colors=log_colors_config) # 日志输出格式\n\n # console.setFormatter(formatter)\n # logger.addHandler(console)\n for handler in handlers:\n handler.setFormatter(formatter)\n handler.setLevel(logging.INFO) # log_level\n logger.addHandler(handler)\n\n # if rank == 0:\n # logger.setLevel(logging.INFO) # log_level\n # else:\n # logger.setLevel(logging.ERROR)\n\n logger_initialized[name] = True\n\n return logger\n\n\ndef get_logger(name=None, cfg=None, cfg_name=None, phase='train', log_level=logging.INFO, file_mode='w'): # log_file=None,\n \"\"\"Initialize and get a logger by name.\n\n If the logger has not been initialized, this method will initialize the\n logger by adding one or two handlers, otherwise the initialized logger will\n be directly returned. During initialization, a StreamHandler will always be\n added. If `log_file` is specified and the process rank is 0, a FileHandler\n will also be added.\n\n Args:\n name (str): Logger name.\n log_file (str | None): The log filename. If specified, a FileHandler\n will be added to the logger.\n log_level (int): The logger level. Note that only the process of\n rank 0 is affected, and other processes will set the level to\n \"Error\" thus be silent most of the time.\n file_mode (str): The file mode used in opening log file.\n Defaults to 'w'.\n\n Returns:\n logging.Logger: The expected logger.\n \"\"\"\n if name in logger_initialized:\n if cfg is None: # cfg.use_log\n return logging.getLogger(name)\n else:\n return None\n # handle hierarchical names\n # e.g., logger \"a\" is initialized, then logger \"a.b\" will skip the\n # initialization since it is a child of \"a\".\n for logger_name in logger_initialized:\n if name.startswith(logger_name):\n if cfg.use_log:\n return logging.getLogger(name)\n else:\n return None\n\n logger = None\n tensorboard_log_dir = None\n root_output_dir = Path(cfg.out_dir)\n # set up logger in root_path\n if not root_output_dir.exists():\n # if not dist_print: #rank 0-N, 0 is False\n print('=> creating {}'.format(root_output_dir))\n root_output_dir.mkdir(parents=True, exist_ok=True)\n\n dataset = cfg.dataset\n model = cfg.arch\n cfg_name = os.path.basename(cfg_name).split('.')[0]\n time_str = time.strftime('%Y-%m-%d-%H-%M-%S')\n\n # store all output except tb_log file\n final_output_dir = root_output_dir / dataset / model / cfg_name\n if cfg.eval:\n model_save_tmp = os.path.dirname(cfg.resume).split('/')[-1]\n else:\n model_save_tmp = \"model_{}\".format(time_str)\n\n model_save_dir = final_output_dir / model_save_tmp\n # if not dist_print:\n log_string('=> creating {}'.format(final_output_dir))\n final_output_dir.mkdir(parents=True, exist_ok=True)\n model_save_dir.mkdir(parents=True, exist_ok=True)\n\n\n if cfg.use_log:\n cfg_name = '{}_{}'.format(cfg_name, time_str)\n # a logger to save results\n log_file = '{}_{}.log'.format(cfg_name, phase)\n if cfg.eval:\n final_log_file = model_save_dir / log_file\n else:\n final_log_file = final_output_dir / log_file\n # tensorboard_log\n tensorboard_log_dir = root_output_dir / Path(cfg.log_dir) / dataset / model / cfg_name\n # if not dist_print:\n print('=> creating tfb logs {}'.format(tensorboard_log_dir))\n tensorboard_log_dir.mkdir(parents=True, exist_ok=True)\n logger = setup_logger(name, final_log_file)\n\n return logger, str(final_output_dir), str(model_save_dir), str(\n tensorboard_log_dir) # logger,\n\ndef print_log(msg, logger=None, level=logging.INFO):\n \"\"\"Print a log message.\n\n Args:\n msg (str): The message to be logged.\n logger (logging.Logger | str | None): The logger to be used.\n Some special loggers are:\n - \"silent\": no message will be printed.\n - other str: the logger obtained with `get_root_logger(logger)`.\n - None: The `print()` method will be used to print log messages.\n level (int): Logging level. Only available when `logger` is a Logger\n object or \"root\".\n \"\"\"\n if logger is None:\n print(msg)\n elif isinstance(logger, logging.Logger):\n logger.log(level, msg)\n elif logger == 'silent':\n pass\n elif isinstance(logger, str):\n _logger = get_logger(logger)\n _logger.log(level, msg)\n else:\n raise TypeError(\n 'logger should be either a logging.Logger object, str, '\n f'\"silent\" or None, but got {type(logger)}')\n\n\ndef load_json_log(json_log):\n \"\"\"load and convert json_logs to log_dicts.\n\n Args:\n json_log (str): The path of the json log file.\n\n Returns:\n dict[int, dict[str, list]]:\n Key is the epoch, value is a sub dict. The keys in each sub dict\n are different metrics, e.g. memory, bbox_mAP, and the value is a\n list of corresponding values in all iterations in this epoch.\n\n .. code-block:: python\n\n # An example output\n {\n 1: {'iter': [100, 200, 300], 'loss': [6.94, 6.73, 6.53]},\n 2: {'iter': [100, 200, 300], 'loss': [6.33, 6.20, 6.07]},\n ...\n }\n \"\"\"\n log_dict = dict()\n with open(json_log, 'r') as log_file:\n for line in log_file:\n log = json.loads(line.strip())\n # skip lines without `epoch` field\n if 'epoch' not in log:\n continue\n epoch = log.pop('epoch')\n if epoch not in log_dict:\n log_dict[epoch] = defaultdict(list)\n for k, v in log.items():\n log_dict[epoch][k].append(v)\n return log_dict\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/metrics.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch\nimport numpy as np\nimport math\nimport torch.nn as nn\n\n\ndef rgb2ycbcr(img, y_only=True):\n \"\"\"metrics\"\"\"\n img.astype(np.float32)\n if y_only:\n rlt = np.dot(img, [65.481, 128.553, 24.966]) / 255.0 + 16.0\n return rlt\n\n\ndef quantize(img, rgb_range):\n pixel_range = 255.0 / rgb_range\n return img.mul(pixel_range).clamp(0, 255).round().div(pixel_range)\n\n\ndef calc_psnr(sr, hr, scale, rgb_range):\n \"\"\"metrics\"\"\"\n hr = np.float32(hr)\n sr = np.float32(sr)\n diff = (sr - hr) / rgb_range\n # .reshape((1, 1, 3)) / 256#\n gray_coeffs = np.array([65.738, 129.057, 25.064]).reshape((1, 3, 1, 1)) / 256\n diff = np.multiply(diff, gray_coeffs).sum(1) # (1)\n if hr.size == 1:\n return 0\n if scale != 1:\n shave = scale\n else:\n shave = scale + 6\n if scale == 1:\n valid = diff\n else:\n valid = diff[..., shave:-shave, shave:-shave]\n # valid = diff[shave:-shave, shave:-shave, ...]\n # mse = np.mean(np.mean(pow(valid, 2), axis=[1, 2, 3]), axis=0)\n mse = np.mean(pow(valid, 2))\n if mse == 0:\n return 100\n try:\n psnr = -10 * math.log10(mse)\n except Exception:\n print(mse)\n\n return psnr\n\n\nclass PSNR_ycbcr(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.gray_coeffs = torch.tensor([65.738, 129.057, 25.064],\n requires_grad=False).reshape((1, 3, 1, 1)) / 256\n\n def quantize(self, img, rgb_range):\n \"\"\"metrics\"\"\"\n pixel_range = 255 / rgb_range\n img = torch.multiply(img, pixel_range)\n img = torch.clip(img, 0, 255)\n img = torch.round(img) / pixel_range\n return img\n\n @torch.no_grad()\n def forward(self, sr, hr, scale, rgb_range):\n \"\"\"metrics\"\"\"\n sr = self.quantize(sr, rgb_range)\n gray_coeffs = self.gray_coeffs.to(sr.device)\n\n hr = hr.float()\n sr = sr.float()\n diff = (sr - hr) / rgb_range\n\n diff = torch.multiply(diff, gray_coeffs).sum(1)\n if hr.size == 1:\n return 0\n if scale != 1:\n shave = scale\n else:\n shave = scale + 6\n if scale == 1:\n valid = diff\n else:\n valid = diff[..., shave:-shave, shave:-shave]\n mse = torch.mean(torch.pow(valid, 2))\n return -10 * torch.log10(mse)\n\n\ndef sub_mean(x):\n x = x * 255.0\n red_channel_mean = 0.4488 * 255\n green_channel_mean = 0.4371 * 255\n blue_channel_mean = 0.4040 * 255\n x[:, 0, :, :] -= red_channel_mean\n x[:, 1, :, :] -= green_channel_mean\n x[:, 2, :, :] -= blue_channel_mean\n return x / 255.0\n\n\ndef add_mean(x):\n x = x * 255.0\n red_channel_mean = 0.4488 * 255\n green_channel_mean = 0.4371 * 255\n blue_channel_mean = 0.4040 * 255\n x[:, 0, :, :] += red_channel_mean\n x[:, 1, :, :] += green_channel_mean\n x[:, 2, :, :] += blue_channel_mean\n return x" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/optim.py", "content": "import torch\nfrom torch import nn\nimport torch.optim as optim\nimport matplotlib.pyplot as plt\n\n\nclass lr_scheduler(object):\n\n def __init__(self, lr, epochs):\n self.epochs = epochs\n self.lr = lr\n self.lr_scheduler = None\n\n # 六大学习率调整策略，lr = lr * gamma\n '''\n ReduceLROnPlateau:\n mode(str)- 模式选择，有 min 和 max 两种模式， min 表示当指标不再降低(如监测loss)， max 表示当指标不再升高(如监测 accuracy)。\n factor(float)- 学习率调整倍数(等同于其它方法的 gamma)，即学习率更新为 lr = lr * factor\n patience(int)- 忍受该指标多少个 step 不变化，当忍无可忍时，调整学习率。\n verbose(bool)- 是否打印学习率信息， print(‘Epoch {:5d}: reducing learning rate of group {} to {:.4e}.’.format(epoch, i, new_lr))\n threshold_mode(str)- 选择判断指标是否达最优的模式，有两种模式， rel 和 abs。\n 当 threshold_mode == rel，并且 mode == max 时， dynamic_threshold = best * ( 1 +threshold )；\n 当 threshold_mode == rel，并且 mode == min 时， dynamic_threshold = best * ( 1 -threshold )；\n 当 threshold_mode == abs，并且 mode== max 时， dynamic_threshold = best + threshold ；\n 当 threshold_mode == rel，并且 mode == max 时， dynamic_threshold = best - threshold；\n threshold(float)- 配合 threshold_mode 使用。\n cooldown(int)- “冷却时间“，当调整学习率之后，让学习率调整策略冷静一下，让模型再训练一段时间，再重启监测模式。\n min_lr(float or list)- 学习率下限，可为 float，或者 list，当有多个参数组时，可用 list 进行设置\n '''\n\n def set_optimizer(self, optimizer, lr_scheduler):\n self.optimizer = optimizer\n # self.lr_scheduler = lr_scheduler\n # self.scheduler = []\n if lr_scheduler == torch.optim.lr_scheduler.StepLR:\n # 等间距阶段式衰减\n self.lr_scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=0.1)\n elif lr_scheduler == optim.lr_scheduler.ReduceLROnPlateau:\n # Reduce learning rate when validation accuarcy plateau.\n self.lr_scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='max', patience=5, verbose=True)\n elif lr_scheduler == optim.lr_scheduler.MultiStepLR:\n # milestones=[epoch1,epoch2,...] 阶段式衰减\n self.lr_scheduler = optim.lr_scheduler.MultiStepLR(optimizer, [100, 200, 300],\n gamma=0.5) # [50, 100, 150, 200, 250, 300, 350, 400], gamma=0.5)\n elif lr_scheduler == optim.lr_scheduler.ExponentialLR:\n # 指数衰减x, 0.1,0.01,0.001,...\n self.lr_scheduler = optim.lr_scheduler.ExponentialLR(optimizer=optimizer, gamma=0.1)\n elif lr_scheduler == optim.lr_scheduler.CosineAnnealingLR:\n # Cosine annealing learning rate.\n self.lr_scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=30, eta_min=1e-7)\n elif lr_scheduler == optim.lr_scheduler.CyclicLR:\n self.lr_scheduler = optim.lr_scheduler.CyclicLR(optimizer, base_lr=1e-3, max_lr=1e-4, step_size_down=30,\n step_size_up=150, cycle_momentum=False)\n elif lr_scheduler == optim.lr_scheduler.LambdaLR:\n # 学习率 = 初始学习率 * lr_lambda(last_epoch）\n curves = lambda epoch: epoch // 30\n # lambda2 = lambda epoch: 0.95 ** epoch\n # lr_lambda对应optimizer中的keys，model.parameters()就只有一个lambda函数\n self.lr_scheduler = optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=[curves])\n elif lr_scheduler == optim.lr_scheduler.CosineAnnealingWarmRestarts:\n # To 初始周期\n # T_mult 每次循环 周期改变倍数 T_0 = T_0*T_mult\n # Learning rate warmup by 10 epochs.\n self.lr_scheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=2, eta_min=0)\n else:\n print(\"self.lr_scheduler not in pytorch\")\n\n def adjust_2_learning_rate(self, epoch):\n \"\"\"编写2种形式的学习率衰减策略的组合\"\"\"\n param_groups = self.optimizer.param_groups\n if epoch <= 5:\n lr = [param_groups[0]['lr'] * 0.9]\n for param_group, val in zip(param_groups, lr):\n param_group['lr'] = val\n else:\n for param_group in param_groups:\n if epoch % 5 == 0:\n # 0.09 0.009 0.0009\n param_group['lr'] *= 0.9\n # print(param_group['lr'])\n\n def adjust_1_learning_rate(self, epoch, mini_lr=1e-6):\n \"\"\"Sets the learning rate to the initial LR decayed by 10 every 30 epochs\"\"\"\n if self.optimizer.param_groups[0][\"lr\"] < mini_lr:\n lr = 1e-5\n for param_group in self.optimizer.param_groups:\n param_group['lr'] = lr\n return\n if epoch <= 40: # 40 20 80\n # lr = self.lr\n lr = self.lr * (0.1 ** (epoch // 20))\n for param_group in self.optimizer.param_groups:\n param_group['lr'] = lr\n return\n elif epoch == 81: # 41\n self.lr = self.optimizer.param_groups[0][\"lr\"]\n # for param_group in self.optimizer.param_groups:\n # param_group['lr'] = 1e-4\n # if epoch >= 42 and epoch % 5 ==0:\n if epoch >= 81:\n lr = self.lr * (0.9 ** (epoch // 20))\n for param_group in self.optimizer.param_groups:\n param_group['lr'] = lr\n return\n elif epoch == 81:\n lr = 1e-5\n else:\n lr = 1e-5\n # self.lr = self.lr * (0.9 ** (epoch // 50))\n # #if epoch <= 120:\n # lr = self.lr * (0.9 ** (epoch // 50))\n # elif epoch == 121:\n # self.lr = self.optimizer.param_groups[0][\"lr\"]\n # lr = self.lr * (0.9 ** (epoch // 50))\n # else:\n # lr = 0.01\n for param_group in self.optimizer.param_groups:\n param_group['lr'] = lr\n\n def step(self, epoch):\n # if not end:\n # self.optimizer.step()\n # else:\n if self.lr_scheduler is None:\n # self.optimizer.step()\n self.adjust_1_learning_rate(epoch)\n else:\n # self.optimizer.step()\n self.lr_scheduler.step(epoch)\n\n # preprint lr_map\n def get_lr_map(self, title, out_file=None, viz=False):\n plt.figure()\n lr = []\n print(\"preprint lr_scheduler\")\n tmp = self.optimizer.param_groups[0]['lr']\n if self.lr_scheduler is None:\n for epoch in range(self.epochs):\n self.step(epoch)\n # TODO:按层绘制\n # print(self.optimizer.param_groups[0]['lr'])\n lr.append(self.optimizer.param_groups[0]['lr'])\n else:\n for epoch in range(self.epochs):\n self.step(epoch)\n try:\n lr.append(self.lr_scheduler.get_last_lr())\n # lr.append(self.lr_scheduler.get_lr())\n except:\n # ReduceLROnPlateau没有get_lr方法\n lr.append(self.optimizer.param_groups[0]['lr'])\n plt.plot(list(range(self.epochs)), lr)\n plt.xlabel(\"epoch\")\n plt.ylabel(\"learning rate\")\n plt.title(title)\n if out_file is not None:\n plt.savefig(out_file)\n if viz:\n plt.show()\n self.optimizer.param_groups[0]['lr'] = tmp\n self.lr = tmp\n\n\ndef tune_param():\n ...\n\n\ndef partial_train(model, layers: list):\n # forzen layers\n for param in model.parameters():\n if layers is not None and layers in param:\n continue\n param.requires_grad = False\n\n # Replace the last fc layer\n model.fc = nn.Linear(512, 100)\n return model\n\n\nif __name__ == \"__main__\":\n from torchvision.models import AlexNet\n import matplotlib.pyplot as plt\n\n model = AlexNet(num_classes=2)\n\n\n class LinearRegression(nn.Module):\n def __init__(self):\n super(LinearRegression, self).__init__()\n self.linear1 = nn.Linear(1, 5) # input and output is 1 dimension\n self.linear2 = nn.Linear(5, 1)\n\n def forward(self, x):\n out = self.linear1(x)\n out = self.linear2(out)\n return out\n\n\n glm = LinearRegression()\n\n optimizer = optim.SGD(params=glm.parameters(), lr=0.1)\n\n epochs = 450\n # 构造一个带warmup小学习率的optimizer，再上升到标准值，再正常周期下降\n lrs = lr_scheduler(0.1, epochs)\n # lrs.set_optimizer(optimizer, optim.lr_scheduler.MultiStepLR)\n # lrs.get_lr_map(\"MultiStepLR\")\n # lrs.set_optimizer(optimizer, optim.lr_scheduler.ExponentialLR)\n # lrs.get_lr_map(\"ExponentialLR\")\n # lrs.set_optimizer(optimizer, optim.lr_scheduler.StepLR)\n # lrs.get_lr_map(\"StepLR\")\n # lrs.set_optimizer(optimizer, optim.lr_scheduler.CyclicLR)\n # lrs.get_lr_map(\"CyclicLR\")\n # # lrs.set_optimizer(optimizer, optim.lr_scheduler.ReduceLROnPlateau)\n # # lrs.get_lr_map(\"ReduceLROnPlateau\")\n lrs.set_optimizer(optimizer, None)\n lrs.get_lr_map(\"LambdaLR\")\n # lrs.set_optimizer(optimizer, optim.lr_scheduler.CosineAnnealingLR)\n # lrs.get_lr_map(\"CosineAnnealingLR\")\n # lrs.set_optimizer(optimizer, optim.lr_scheduler.CosineAnnealingWarmRestarts)\n # lrs.get_lr_map(\"CosineAnnealingWarmRestarts\")\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/option.py", "content": "# GPL License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport argparse\nimport platform\n# import warnings\nimport os\nfrom UDL.AutoDL import TaskDispatcher\nfrom UDL.Basis.config import Config\nimport warnings\n\ndef common_cfg():\n parser = argparse.ArgumentParser(description='PyTorch Training')\n # * Logger\n parser.add_argument('--use-log', default=True\n , type=bool)\n parser.add_argument('--log-dir', metavar='DIR', default='logs',\n help='path to save log')\n parser.add_argument('--tfb-dir', metavar='DIR', default=None,\n help='useless in this script.')\n parser.add_argument('--use-tfb', default=False, type=bool)\n\n # * DDP\n parser.add_argument(\n '--launcher',\n choices=['none', 'pytorch', 'slurm', 'mpi'],\n default='none',\n help='job launcher')\n parser.add_argument('--local_rank', default=0, type=int,\n help=\"host rank must be 0 and python -m torch.distributed.launch main.py need args.local_rank\")\n parser.add_argument('--backend', default='nccl', type=str, # gloo\n help='distributed backend')\n parser.add_argument('--dist-url', default='env://',\n type=str, # 'tcp://224.66.41.62:23456'\n help='url used to set up distributed training')\n # * AMP\n parser.add_argument('--amp', default=None, type=bool,\n help=\"False is apex, besides True is torch1.6+, which has supports amp ops to reduce gpu memory and speed up training\")\n parser.add_argument('--amp-opt-level', type=str, default='O1', choices=['O0', 'O1', 'O2'],\n help='mixed precision opt level, if O0, no amp is used')\n\n # * Training\n parser.add_argument('--accumulated-step', default=1, type=int)\n parser.add_argument('--clip_max_norm', default=0, type=float,\n help='gradient clipping max norm')\n\n # * extra\n parser.add_argument('--seed', default=10, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--device', default='cuda',\n help='device to use for training / testing')\n parser.add_argument('--reg', type=bool, default=True,\n help='loss with l2 reguliarization for nn.Conv2D, '\n 'which is very important for classical panshrapening!!! ')\n\n parser.add_argument('--crop_batch_size', type=int, default=128,\n help='input batch size for-'\n ' training')\n parser.add_argument('--rgb_range', type=int, default=255,\n help='maximum value of RGB')\n parser.add_argument('--model_style', type=str, default=None,\n help='model_style is used to recursive/cascade or GAN training')\n parser.add_argument('--mode', type=str, default=None,\n help='dataset file extension')\n parser.add_argument('--task', type=str, default=None,\n help='dataset file extension')\n parser.add_argument('--arch', type=str, default='',\n help='arch')\n args = parser.parse_args()\n args.global_rank = 0\n args.once_epoch = False\n args.reset_lr = False\n args.amp_opt_level = 'O0' if args.amp == None else args.amp_opt_level\n args.save_top_k = 5\n args.save_print_freq = 10\n args.start_epoch = 1\n assert args.accumulated_step > 0\n args.load_model_strict = True\n args.resume_mode = 'best'\n args.validate = False\n args.gpu_ids = [0]\n # args.workflow = []\n\n return Config(args)\n\nclass panshaprening_cfg(TaskDispatcher, name='pansharpening'):\n\n\n def __init__(self, cfg=None, arch=None):\n super(panshaprening_cfg, self).__init__()\n\n import UDL.pansharpening.configs\n import UDL.pansharpening.models\n\n if cfg is None:\n cfg = common_cfg()\n\n cfg.scale = [1]\n if platform.system() == 'Linux':\n cfg.data_dir = '/Datasets/pansharpening/DLPan'\n if platform.system() == \"Windows\":\n cfg.data_dir = 'D:/Datasets/pansharpening/DLPan'\n \n cfg.best_prec1 = 10000\n cfg.best_prec5 = 10000\n cfg.metrics = 'loss'\n cfg.task = \"pansharpening\"\n cfg.save_fmt = \"mat\" # fmt is mat or not mat\n cfg.taskhead = \"pansharpening\"\n\n # * Importantly\n warning = f\"Note: FusionNet, DiCNN, PNN don't have high-pass filter\"\n warnings.warn(warning)\n if arch is not None:\n cfg = self.new(cfg=cfg, arch=cfg.arch)\n self.merge_from_dict(cfg)\n\ndef nni_cfg(args):\n if args.mode == 'nni':\n import nni\n tuner_params = nni.get_next_parameter()\n print(\"launcher: nni is running. \\n\", tuner_params)\n args.merge_from_dict(tuner_params)\n return args\n\nclass get_cfg(TaskDispatcher, name='entrypoint'):\n def __init__(self, task=None, arch=None):\n super(get_cfg, self).__init__()\n args = common_cfg()\n # args.mode = 'nni'\n if arch is not None:\n args.arch = arch\n if args.mode == 'nni':\n args = nni_cfg(args)\n # args.__delattr__('workflow')\n\n if hasattr(args, 'task'):\n cfg = TaskDispatcher.new(cfg=args, task=task, arch=args.arch)\n cfg.merge_from_dict(args)\n elif task in TaskDispatcher._task.keys():\n cfg = TaskDispatcher.new(cfg=args, task=task, arch=args.arch)\n cfg.merge_from_dict(args)\n else:\n raise ValueError(f\"nni starter don't have task={task} but expected\"\n f\"one of {super()._task.keys()} in TaskDispatcher\")\n # cfg.setdefault('workflow', [])\n cfg = data_cfg(cfg)\n print(cfg.pretty_text)\n\n self.merge_from_dict(cfg)\n\n\ndef data_cfg(cfg):\n if cfg.get('config', None) is not None:\n\n if not os.path.isfile(cfg.config):\n raise IOError(f\"reading {cfg.config} failed\")\n\n cfg.merge_from_dict(cfg.fromfile(cfg.config))\n if cfg.get('data', None) is not None and callable(cfg.data):\n data_func = cfg.pop('data')\n cfg.merge_from_dict(Config(data_func(cfg.data_dir)))\n\n cfg.workflow = cfg.get('workflow', [])\n if cfg.get('norm_cfg', None) is not None and cfg.launcher == 'none':\n cfg.norm_cfg = 'BN'\n\n # modify loading COCO from extern\n # if hasattr(cfg, 'data'):\n # cfg.data.train['ann_file'] = cfg.data.train['ann_file'].replace('data', cfg.data_dir)\n # cfg.data.train['img_prefix'] = cfg.data.train['img_prefix'].replace('data', cfg.data_dir)\n # cfg.data.val['ann_file'] = cfg.data.val['ann_file'].replace('data', cfg.data_dir)\n # cfg.data.val['img_prefix'] = cfg.data.val['img_prefix'].replace('data', cfg.data_dir)\n # cfg.samples_per_gpu = cfg.data.samples_per_gpu\n # cfg.workers_per_gpu = cfg.data.workers_per_gpu\n\n return cfg\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/postprocess.py", "content": "# GPL License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nfrom typing import Union, Optional, List, Tuple, Text, BinaryIO\nfrom PIL import Image\nimport cv2\nimport os\nimport numpy as np\nimport io\nimport pathlib\nimport torch\nimport math\n\nirange = range\n\n\ndef format_np_output(np_arr):\n \"\"\"\n This is a (kind of) bandaid fix to streamline saving procedure.\n It converts all the outputs to the same format which is 3xWxH\n with using sucecssive if clauses.\n Args:\n im_as_arr (Numpy array): Matrix of shape 1xWxH or WxH or 3xWxH\n \"\"\"\n # Phase/Case 1: The np arr only has 2 dimensions\n # Result: Add a dimension at the beginning\n if len(np_arr.shape) == 2:\n np_arr = np.expand_dims(np_arr, axis=0)\n # Phase/Case 2: Np arr has only 1 channel (assuming first dim is channel)\n # Result: Repeat first channel and convert 1xWxH to 3xWxH\n if np_arr.shape[0] == 1:\n np_arr = np.repeat(np_arr, 3, axis=0)\n # Phase/Case 3: Np arr is of shape 3xWxH\n # Result: Convert it to WxHx3 in order to make it saveable by PIL\n if np_arr.shape[0] == 3:\n np_arr = np_arr.transpose(1, 2, 0)\n # Phase/Case 4: NP arr is normalized between 0-1\n # Result: Multiply with 255 and change type to make it saveable by PIL\n if np.max(np_arr) <= 1:\n np_arr = (np_arr * 255).astype(np.uint8)\n return np_arr\n\n\ndef save_image(im, path):\n \"\"\"\n Saves a numpy matrix or PIL image as an image\n Args:\n im_as_arr (Numpy array): Matrix of shape DxWxH\n path (str): Path to the image\n \"\"\"\n if isinstance(im, (np.ndarray, np.generic)):\n im = format_np_output(im)\n im = Image.fromarray(im)\n im.save(path)\n\n\ndef norm_image(image, factor=255.):\n \"\"\"\n 标准化图像\n :param factor:\n :param image: [H,W,C]\n :return:\n \"\"\"\n image = image.copy()\n image -= np.max(np.min(image), 0)\n image /= np.max(image)\n if factor == 255. or factor == 255:\n image *= factor\n return np.uint8(image)\n else:\n return image\n\n\ndef convert_to_grayscale(im_as_arr):\n \"\"\"\n Converts 3d image to grayscale\n\n Args:\n im_as_arr (numpy arr): RGB image with shape (D,W,H)\n\n returns:\n grayscale_im (numpy_arr): Grayscale image with shape (1,W,D)\n \"\"\"\n # grayscale_im = np.sum(np.abs(im_as_arr), axis=0)\n # im_max = np.percentile(grayscale_im, 99)\n # im_min = np.min(grayscale_im)\n # grayscale_im = (np.clip((grayscale_im - im_min) / (im_max - im_min), 0, 1))\n # grayscale_im = np.expand_dims(grayscale_im, axis=0)\n # return grayscale_im\n\n grayscale_im = np.sum(np.abs(im_as_arr), axis=-1)\n im_max = np.percentile(grayscale_im, 99)\n im_min = np.min(grayscale_im)\n grayscale_im = (np.clip((grayscale_im - im_min) / (im_max - im_min), 0, 1))\n grayscale_im = np.expand_dims(grayscale_im, axis=-1)\n return grayscale_im\n\n\ndef apply_gradient_images(gradient, file_name, is_save=False):\n \"\"\"\n Exports the original gradient image\n\n Args:\n gradient (np arr): Numpy array of the gradient with shape (3, 224, 224)\n file_name (str): File name to be exported\n \"\"\"\n if not os.path.exists('../results'):\n os.makedirs('../results')\n # Normalize\n gradient = gradient - gradient.min()\n gradient /= gradient.max()\n # Save image\n if is_save:\n path_to_file = os.path.join('../results', file_name + '.jpg')\n save_image(gradient, path_to_file)\n return None\n else:\n return gradient\n\n\n# misc_function\nimport matplotlib.cm as mpl_color_map\nimport copy\nimport PIL\n\n\ndef gen_colormap(input_image, feature, gradient, factor=255):\n if feature.size(0) == 1:\n feature = feature.squeeze(0)\n if gradient.size(0) == 1:\n gradient = gradient.squeeze(0)\n gradient = gradient.cpu().data.numpy() # [C,H,W]\n weight = np.mean(gradient, axis=(1, 2)) # [C]\n\n feature = feature.cpu().data.numpy() # [C,H,W]\n\n cam = feature * weight[:, np.newaxis, np.newaxis] # [C,H,W]\n cam = np.sum(cam, axis=0) # [H,W]\n cam = np.maximum(cam, 0) # ReLU\n\n # 数值归一化\n # cam -= np.min(cam)\n # cam /= np.max(cam)\n\n # cam = np.maximum(cam, 0)\n cam = (cam - np.min(cam)) / (np.max(cam) - np.min(cam)) # Normalize between 0-1\n cam = np.uint8(cam * factor) # Scale between 0-255 to visualize\n cam = np.uint8(Image.fromarray(cam).resize((input_image.shape[2],\n input_image.shape[3]), Image.ANTIALIAS)) / factor\n return cam\n\n\ndef apply_colormap_on_image(org_im, activation, colormap_name):\n \"\"\"\n Apply heatmap on image\n Args:\n org_img (PIL img): Original image\n activation_map (numpy arr): Activation map (grayscale) 0-255\n colormap_name (str): Name of the colormap\n \"\"\"\n\n if not isinstance(org_im, np.ndarray):\n org_im = org_im[0, :3, ...].permute(1, 2, 0)\n org_im = org_im.cpu().numpy() * 255\n org_im = PIL.Image.fromarray(org_im.astype(np.uint8))\n else:\n org_im = org_im * 255\n org_im = PIL.Image.fromarray(org_im.astype(np.uint8))\n # Get colormap\n '''\n \n '''\n color_map = mpl_color_map.get_cmap(colormap_name)\n no_trans_heatmap = color_map(activation)\n # Change alpha channel in colormap to make sure original image is displayed\n heatmap = copy.copy(no_trans_heatmap)\n heatmap[..., 3] = 0.4\n heatmap = Image.fromarray((heatmap * 255).astype(np.uint8))\n heatmap = heatmap.resize(org_im.size, Image.ANTIALIAS)\n no_trans_heatmap = Image.fromarray((no_trans_heatmap * 255).astype(np.uint8))\n\n # Apply heatmap on image\n heatmap_on_image = Image.new(\"RGBA\", org_im.size)\n heatmap_on_image = Image.alpha_composite(heatmap_on_image, org_im.convert('RGBA'))\n heatmap_on_image = Image.alpha_composite(heatmap_on_image, heatmap)\n return no_trans_heatmap, heatmap_on_image\n\n\ndef get_positive_negative_saliency(gradient):\n \"\"\"\n Generates positive and negative saliency maps based on the gradient\n Args:\n gradient (numpy arr): Gradient of the operation to visualize\n\n returns:\n pos_saliency ( )\n \"\"\"\n pos_saliency = (np.maximum(0, gradient) / gradient.max())\n neg_saliency = (np.maximum(0, -gradient) / -gradient.min())\n return pos_saliency, neg_saliency\n\n\n# hook_test/Viz/main\n\ndef gen_grad_cam(image, feature, gradient):\n if feature.size(0) == 1:\n feature = feature.squeeze(0)\n if gradient.size(0) == 1:\n gradient = gradient.squeeze(0)\n gradient = gradient # .cpu().data.numpy() # [C,H,W]\n weight = torch.mean(gradient, dim=(1, 2)) # [C]\n\n # feature = feature.cpu().data.numpy() # [C,H,W]\n\n # cam = feature * weight[:, np.newaxis, np.newaxis]\n cam = feature * weight[:, np.newaxis, np.newaxis] # [C,H,W]\n cam = torch.maximum(cam, torch.zeros_like(cam)) # ReLU\n cam = torch.sum(cam, dim=0) # [H,W]\n # cam = torch.mean(feature, dim=0)\n # cam = torch.maximum(cam, torch.zeros_like(cam)) # ReLU\n\n # 数值归一化\n cam -= torch.min(cam)\n cam /= (torch.max(cam) - torch.min(cam) + 1e-8)\n\n return cam.cpu().data.numpy()\n\n\ndef apply_heatmap(image, mask, factor=255):\n \"\"\"\n 生成CAM图\n :param image: [H,W,C],原始图像\n :param mask: [H,W],范围0~1\n :return: tuple(cam,heatmap)\n \"\"\"\n\n # mask转为heatmap\n if not isinstance(image, np.ndarray):\n image = image[0, :3, ...].permute(1, 2, 0)\n image = image.cpu().numpy()\n if not isinstance(mask, np.ndarray):\n mask = mask.cpu().numpy()\n # heatmaps = np.tile(np.zeros_like(mask)[..., np.newaxis], [1, 1, 1, 3])\n # for c_idx in range(mask.shape[0]):\n # c_mask = mask[c_idx, ..., np.newaxis]\n # heatmap = cv2.applyColorMap(np.uint8(255 * c_mask), cv2.COLORMAP_JET)\n # heatmaps[c_idx, ...] = np.float32(heatmap) / 255\n # heatmaps = heatmaps[..., ::-1] # gbr to rgb\n # mask转为heatmap\n heatmap = cv2.applyColorMap(np.uint8(255 * mask), cv2.COLORMAP_JET)\n heatmap = np.float32(heatmap) / 255\n heatmap = heatmap[..., ::-1] # gbr to rgb\n # 合并heatmap到原始图像\n cam = cv2.resize(heatmap, image.shape[:2]) + np.float32(image)\n return norm_image(cam, 2048), (heatmap * 255).astype(np.uint8)\n # cam = heatmaps[np.newaxis, ...] + np.float32(image)\n # return norm_image(cam), (heatmaps * 255).astype(np.uint8)\n\n\ndef showimage8(images, unnormlize=2047.0, first_channel=False):\n assert images.shape[1] >= 3, print(\"input images format is not suitable\")\n\n if isinstance(images, torch.Tensor):\n unnormlize = np.where(max(np.float(torch.max(images)), 1.0) > 1.0, 1.0, unnormlize)\n if first_channel:\n images = images.permute(1, 2, 0)\n output = images[..., [0, 2, 4]] * torch.tensor(unnormlize)\n output = torch.clamp(output, 0, 2047)\n output = output.cpu().detach().numpy()\n\n norm_image = linstretch(output)\n return norm_image[:, :, ::-1]\n\n\ndef linstretch(images, tol=None):\n '''\n NM = N*M;\n for i=1:3\n b = reshape(double(uint16(ImageToView(:,:,i))),NM,1);\n [hb,levelb] = hist(b,max(b)-min(b));\n chb = cumsum(hb);#沿第一个非单一维运算。matlab矩阵顺序 HxWxC,列的累计和\n t(1)=ceil(levelb(find(chb>NM*tol(i,1), 1 )));\n t(2)=ceil(levelb(find(chbt(2))=t(2);\n b = (b-t(1))/(t(2)-t(1));\n ImageToView(:,:,i) = reshape(b,N,M);\n end\n '''\n # images = np.random.randn(64, 64, 3) * 2047.0\n if tol is None:\n tol = [0.01, 0.995]\n if images.ndim == 3:\n h, w, channels = images.shape\n else:\n images = np.expand_dims(images, axis=-1)\n h, w, channels = images.shape\n N = h * w\n for c in range(channels):\n image = np.float32(np.round(images[:, :, c])).reshape(N, 1)\n hb, levelb = np.histogram(image, bins=math.ceil(image.max() - image.min()))\n chb = np.cumsum(hb, 0)\n levelb_center = levelb[:-1] + (levelb[1] - levelb[0]) / 2\n lbc_min, lbc_max = levelb_center[chb > N * tol[0]][0], levelb_center[chb < N * tol[1]][-1]\n image = np.clip(image, a_min=lbc_min, a_max=lbc_max)\n image = (image - lbc_min) / (lbc_max - lbc_min)\n images[..., c] = np.reshape(image, (h, w))\n\n images = np.squeeze(images)\n\n return images\n\n\ndef make_grid(\n tensor: Union[torch.Tensor, List[torch.Tensor]],\n mode: str = \"grey\",\n nrow: int = 8,\n padding: int = 2,\n normalize: bool = False,\n range: Optional[Tuple[int, int]] = None,\n scale_each: bool = False,\n pad_value: int = 0,\n) -> torch.Tensor:\n \"\"\"Make a grid of images.\n\n Args:\n tensor (Tensor or list): 4D mini-batch Tensor of shape (B x C x H x W)\n or a list of images all of the same size.\n mode (str, optional): 人为设定通道模式\n nrow (int, optional): Number of images displayed in each row of the grid.\n The final grid size is ``(B / nrow, nrow)``. Default: ``8``.\n padding (int, optional): amount of padding. Default: ``2``.\n normalize (bool, optional): If True, shift the image to the range (0, 1),\n by the min and max values specified by :attr:`range`. Default: ``False``.\n range (tuple, optional): tuple (min, max) where min and max are numbers,\n then these numbers are used to normalize the image. By default, min and max\n are computed from the tensor.\n scale_each (bool, optional): If ``True``, scale each image in the batch of\n images separately rather than the (min, max) over all images. Default: ``False``.\n pad_value (float, optional): Value for the padded pixels. Default: ``0``.\n\n Example:\n See this notebook `here `_\n\n \"\"\"\n if not (torch.is_tensor(tensor) or\n (isinstance(tensor, list) and all(torch.is_tensor(t) for t in tensor))):\n raise TypeError('tensor or list of tensors expected, got {}'.format(type(tensor)))\n\n # if list of tensors, convert to a 4D mini-batch Tensor\n if isinstance(tensor, list):\n tensor = torch.stack(tensor, dim=0)\n\n if tensor.dim() == 2: # single image H x W\n tensor = tensor.unsqueeze(0)\n if tensor.dim() == 3: # single image\n if tensor.size(0) == 1: # if single-channel, convert to 3-channel\n tensor = torch.cat((tensor, tensor, tensor), 0)\n tensor = tensor.unsqueeze(0)\n\n if tensor.dim() == 4 and tensor.size(1) == 1: # single-channel images\n if mode == \"RGB\":\n tensor = torch.cat((tensor, tensor, tensor), 1)\n\n if normalize is True:\n tensor = tensor.clone() # avoid modifying tensor in-place\n if range is not None:\n assert isinstance(range, tuple), \\\n \"range has to be a tuple (min, max) if specified. min and max are numbers\"\n\n def norm_ip(img, min, max):\n img.clamp_(min=min, max=max)\n img.add_(-min).div_(max - min + 1e-5)\n\n def norm_range(t, range):\n if range is not None:\n norm_ip(t, range[0], range[1])\n else:\n norm_ip(t, float(t.min()), float(t.max()))\n\n if scale_each is True:\n for t in tensor: # loop over mini-batch dimension\n norm_range(t, range)\n else:\n norm_range(tensor, range)\n\n if tensor.size(0) == 1:\n return tensor.squeeze(0)\n\n # make the mini-batch of images into a grid\n nmaps = tensor.size(0)\n xmaps = min(nrow, nmaps)\n ymaps = int(math.ceil(float(nmaps) / xmaps))\n height, width = int(tensor.size(2) + padding), int(tensor.size(3) + padding)\n num_channels = tensor.size(1)\n grid = tensor.new_full((num_channels, height * ymaps + padding, width * xmaps + padding), pad_value)\n k = 0\n for y in irange(ymaps):\n for x in irange(xmaps):\n if k >= nmaps:\n break\n # Tensor.copy_() is a valid method but seems to be missing from the stubs\n # https://pytorch.org/docs/stable/tensors.html#torch.Tensor.copy_\n grid.narrow(1, y * height + padding, height - padding).narrow( # type: ignore[attr-defined]\n 2, x * width + padding, width - padding\n ).copy_(tensor[k])\n k = k + 1\n return grid\n\n\ndef tensor_save_image(\n tensor: Union[torch.Tensor, List[torch.Tensor]],\n fp: Union[Text, pathlib.Path, BinaryIO],\n nrow: int = 8,\n padding: int = 2,\n normalize: bool = False,\n range: Optional[Tuple[int, int]] = None,\n scale_each: bool = False,\n pad_value: int = 0,\n format: Optional[str] = None,\n) -> None:\n \"\"\"Save a given Tensor into an image file.\n\n Args:\n tensor (Tensor or list): Image to be saved. If given a mini-batch tensor,\n saves the tensor as a grid of images by calling ``make_grid``.\n fp (string or file object): A filename or a file object\n format(Optional): If omitted, the format to use is determined from the filename extension.\n If a file object was used instead of a filename, this parameter should always be used.\n **kwargs: Other arguments are documented in ``make_grid``.\n \"\"\"\n from PIL import Image\n grid = make_grid(tensor, nrow=nrow, padding=padding, pad_value=pad_value,\n normalize=normalize, range=range, scale_each=scale_each)\n # Add 0.5 after unnormalizing to [0, 255] to round to nearest integer\n ndarr = grid.mul(255).add_(0.5).clamp_(0, 255).permute(1, 2, 0).to('cpu', torch.uint8).numpy()\n im = Image.fromarray(ndarr)\n im.save(fp, format=format)\n\n\nif __name__ == \"__main__\":\n a = np.random.randn(3, 3)\n linstretch(a)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/python_sub_class.py", "content": "# GPL License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch\nfrom UDL.pansharpening.evaluation.ps_evaluate import analysis_accu, save_results\nfrom UDL.Basis.config import Config\nimport warnings\nimport scipy.io as sio\n\nclass TaskDispatcher(Config):\n _task = dict()\n\n def __init_subclass__(cls, name='', **kwargs):\n super().__init_subclass__(**kwargs)\n\n if name != '':\n cls._task[name] = cls\n cls._name = name\n # print(cls.__repr__, cls..__repr__)\n else:\n # warnings.warn(f'Creating a subclass of MetaModel {cls.__name__} with no name.')\n cls._task[cls.__name__] = cls\n cls._name = cls.__name__\n\n def __new__(cls, *args, **kwargs):\n if cls is TaskDispatcher:\n task = kwargs.get('task')\n try:\n cls = cls._task[task]\n except KeyError:\n raise ValueError(f'Got task={task} but expected'\n f'one of {cls._task.keys()}')\n\n instance = super().__new__(cls)\n\n return instance\n\n # def __len__(self):\n # return len(self._cfg_dict)\n #\n # def __getattr__(self, name):\n # return getattr(self._cfg_dict, name)\n #\n # def __delattr__(self, name):\n # return delattr(self._cfg_dict, name)\n #\n # def __getitem__(self, name):\n # return self._cfg_dict.__getitem__(name)\n #\n # def __iter__(self):\n # return iter(self._cfg_dict)\n #\n # def __repr__(self):\n # return f'TaskDispatcher {self._cfg_dict.__repr__()}'\n\n # def __setattr__(self, name, value):\n # if isinstance(value, dict):\n # value = ConfigDict(value)\n # print(\"__setattr__\")\n # self._cfg_dict.__setattr__(name, value)\n\n # def __setitem__(self, name, value):\n # if isinstance(value, dict):\n # value = ConfigDict(value)\n # print(\"__setitem__\")\n # self._cfg_dict.__setitem__(name, value)\n\n @classmethod\n def new(cls, **kwargs):\n # 需要从外部启动和从任务启动，但参数不同\n key = 'mode'\n value = kwargs.setdefault('mode', None)\n print('111', value)\n if value is None:\n # 第二、三调用层进入此函数\n key = 'task'\n if kwargs.get('task', None):\n # 二\n value = kwargs.pop('task')\n print('222', value)\n elif kwargs.get('arch', None):\n # 三\n key = 'arch'\n value = kwargs.pop('arch')\n print('333', value)\n else:\n key = 'arch'\n\n kwargs.pop('mode')\n\n try:\n cls = cls._task[value]\n except KeyError:\n warning = f'Got {key}={value} but expected ' \\\n f'one of {cls._task.keys()}'\n warnings.warn(warning)\n return Config()\n\n return cls(**kwargs)\n\nclass ModelDispatcher(object):\n _task = dict()\n\n def __init_subclass__(cls, name='', **kwargs):\n super().__init_subclass__(**kwargs)\n if name != '':\n cls._task[name] = cls\n cls._name = name\n # print(cls.__repr__, cls..__repr__)\n else:\n # warnings.warn(f'Creating a subclass of MetaModel {cls.__name__} with no name.')\n cls._task[cls.__name__] = cls\n cls._name = cls.__name__\n\n def __new__(cls, *args, **kwargs):\n if cls is ModelDispatcher:\n task = kwargs.get('task')\n try:\n cls = cls._task[task]\n except KeyError:\n raise ValueError(f'Got task={task} but expected'\n f'one of {cls._task.keys()}')\n\n instance = super().__new__(cls)\n\n return instance\n\n @classmethod\n def build_model(cls, cfg):\n\n arch = cfg.arch\n task = cfg.task\n model_style = cfg.model_style\n\n try:\n # 获得PansharpeningModel,进行分发\n cls = cls._task[task](None, None)\n except KeyError:\n raise ValueError(f'Got task={task} but expected '\n f'one of {cls._task.keys()} in {cls}')\n try:\n # 获得具体的模型\n cls_arch = cls._models[arch]()\n except KeyError:\n raise ValueError(f'Got arch={arch} but expected '\n f'one of {cls._models.keys()} in {cls}')\n\n model, criterion, optimizer, scheduler = cls_arch(cfg)\n\n if model_style is None:\n # 获得PansharpeningModel,model+head\n model_style = task\n\n if model_style is not None:\n try:\n # 获得具体的模型\n model = cls._task[model_style](model, criterion)\n except KeyError:\n raise ValueError(f'Got model_style={model_style} but expected '\n f'one of {cls._models.keys()} (merged in _models) in {cls}')\n\n return model, criterion, optimizer, scheduler\n\nclass PanSharpeningModel(ModelDispatcher, name='pansharpening'):\n\n _models = {}\n\n def __init__(self, model=None, criterion=None):\n super(PanSharpeningModel, self).__init__()\n self.model = model\n self.criterion = criterion\n self.reg = False\n if hasattr(self.model, 'reg'):\n self.reg = self.model.reg\n\n def __init_subclass__(cls, name='', **kwargs):\n\n # print(name, cls)\n if name != '':\n cls._models[name] = cls\n cls._name = name\n else:\n cls._models[cls.__name__] = cls\n cls._name = cls.__name__\n # warnings.warn(f'Creating a subclass of MetaModel {cls.__name__} with no name.')\n\n def l2_regularization(self, loss_dict, weight_decay=1e-5, flag=False):\n regularizations = []\n for k, v in self.model.named_parameters():\n if 'conv' in k and 'weight' in k:\n # print(k)\n penality = weight_decay * ((v.data ** 2).sum() / 2)\n regularizations.append(penality)\n if flag:\n print(\"{} : {}\".format(k, penality))\n # r = torch.sum(regularizations)\n if isinstance(loss_dict, dict):\n loss_dict['loss'] = loss_dict['loss'] + sum(regularizations)\n loss_dict['log_vars'].update(reg_loss=loss_dict['loss'])\n else:\n loss_dict = loss_dict + sum(regularizations)\n\n return loss_dict\n\n def train_step(self, *args, **kwargs):\n\n loss_dict = self.model.train_step(args[0], **kwargs)\n\n if self.reg:\n return self.l2_regularization(loss_dict)\n\n return loss_dict\n\n def val_step(self, *args, **kwargs):\n sr, gt = self.model.val_step(*args, **kwargs)\n result_our = torch.squeeze(sr).permute(1, 2, 0)\n result_our = torch.clip(result_our, 0, 1)\n metrics = analysis_accu(gt.cuda().squeeze(0), result_our, 4)\n result_our = result_our * kwargs['img_range']\n\n if kwargs['save_fmt'] is not None:\n save_results(kwargs['idx'], kwargs['save_dir'], kwargs['filename'], kwargs['save_fmt'], result_our)\n\n return {'log_vars': metrics}\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/slurm_train.sh", "content": "#!/usr/bin/env bash\n\nset -x\n\n#cd projects/derain\n\nPARTITION=defq\nJOB_NAME=task\n#CONFIG=$3\n#WORK_DIR=$4\nGPUS=${GPUS:-16}\nGPUS_PER_NODE=${GPUS_PER_NODE:-8}\nCPUS_PER_TASK=${CPUS_PER_TASK:-8}\nSRUN_ARGS=${SRUN_ARGS:-\"\"}\n#NNODE=${NNODE:-'node[004]'}\n#PY_ARGS=${@:5}\n\n#PYTHONPATH=\"$(dirname $0)/..\":$PYTHONPATH \\\nsrun -p ${PARTITION} \\\n --job-name=${JOB_NAME} \\\n --gres=gpu:0 \\\n --ntasks=${GPUS} \\\n --ntasks-per-node=${GPUS_PER_NODE} \\\n --cpus-per-task=${CPUS_PER_TASK} \\\n --kill-on-bad-exit=1 \\\n --nodelist=node[004-005] \\\n ${SRUN_ARGS} \\\n python -u main.py --launcher=\"slurm\" #${PY_ARGS}\n\n#srun -p defq -J test -n 2 --nodelist=node[004-005] --ntasks-per-node=2 --export=cuda_home python -u test_slurm.py\n#srun --partition=defq --job-name=rain -n 1 --nodelist=node004 --gres=gpu:8 --ntasks-per-node=8 python -u derain_main.py --launcher=\"slurm\n#sed -i \"s/\\r//\" slurm_train.sh\n# srun -p defq -J test -n 1 --nodelist=node[004] --ntasks-per-node=1 python -u derain_main.py --launcher slurm\n#srun -p defq -J test -n 2 --nodelist=node[004-005] --ntasks-per-node=1 python -u test_slurm.py\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/snmn_d.sh", "content": "python -m launch --nproc_per_node 8 --master_port 27890 main.py\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Basis/variance_sacling_initializer.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport math\n\n\ndef truncated_normal_(tensor, mean=0.0, std=1.0):\n with torch.no_grad():\n size = tensor.shape\n tmp = tensor.new_empty(size + (4,)).normal_()\n valid = (tmp < 2) & (tmp > -2)\n ind = valid.max(-1, keepdim=True)[1]\n tensor.data.copy_(tmp.gather(-1, ind).squeeze(-1))\n tensor.data.mul_(std).add_(mean)\n return tensor\n\n\ndef variance_scaling_initializer(tensor):\n from scipy.stats import truncnorm\n def calculate_fan(shape, factor=2.0, mode='FAN_IN', uniform=False):\n # 64 9 3 3 -> 3 3 9 64\n # 64 64 3 3 -> 3 3 64 64\n if shape:\n # fan_in = float(shape[1]) if len(shape) > 1 else float(shape[0])\n # fan_out = float(shape[0])\n fan_in = float(shape[-2]) if len(shape) > 1 else float(shape[-1])\n fan_out = float(shape[-1])\n else:\n fan_in = 1.0\n fan_out = 1.0\n for dim in shape[:-2]:\n fan_in *= float(dim)\n fan_out *= float(dim)\n if mode == 'FAN_IN':\n # Count only number of input connections.\n n = fan_in\n elif mode == 'FAN_OUT':\n # Count only number of output connections.\n n = fan_out\n elif mode == 'FAN_AVG':\n # Average number of inputs and output connections.\n n = (fan_in + fan_out) / 2.0\n if uniform:\n raise NotImplemented\n # # To get stddev = math.sqrt(factor / n) need to adjust for uniform.\n # limit = math.sqrt(3.0 * factor / n)\n # return random_ops.random_uniform(shape, -limit, limit,\n # dtype, seed=seed)\n else:\n # To get stddev = math.sqrt(factor / n) need to adjust for truncated.\n trunc_stddev = math.sqrt(1.3 * factor / n)\n return fan_in, fan_out, trunc_stddev\n\n def variance_scaling(x, scale=1.0, mode=\"fan_in\", distribution=\"truncated_normal\", seed=None):\n # fan_in, fan_out = torch.nn.init._calculate_fan_in_and_fan_out(x)\n x = x.permute(3, 2, 1, 0) # .permute(2, 3, 1, 0)\n fan_in, fan_out, trunc_stddev = calculate_fan(x.shape)\n # print(trunc_stddev) # debug\n # if mode == \"fan_in\":\n # scale /= max(1., fan_in)\n # elif mode == \"fan_out\":\n # scale /= max(1., fan_out)\n # else:\n # scale /= max(1., (fan_in + fan_out) / 2.)\n # if distribution == \"normal\" or distribution == \"truncated_normal\":\n # # constant taken from scipy.stats.truncnorm.std(a=-2, b=2, loc=0., scale=1.)\n # stddev = math.sqrt(scale) / .87962566103423978\n # print(fan_in,fan_out,scale,stddev)#100,100,0.01,0.1136\n truncated_normal_(x, 0.0, trunc_stddev) # 0.001)\n x = x.permute(3, 2, 0, 1)\n # print(x.min(), x.max())) # debug\n return x # /10*1.28\n\n variance_scaling(tensor)\n\n return tensor\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Data/pansharpening/test_data/readme-test.txt", "content": "" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Data/pansharpening/training_data/readme-test.txt", "content": "" }, { "path": "01-DL-toolbox(Pytorch)/UDL/Data/pansharpening/validation_data/readme-test.txt", "content": "" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/CITATION.cff", "content": "cff-version: 1.2.0\nmessage: \"If you use this software, please cite it as below.\"\nauthors:\n - name: \"MMCV Contributors\"\ntitle: \"OpenMMLab Computer Vision Foundation\"\ndate-released: 2018-08-22\nurl: \"https://github.com/open-mmlab/mmcv\"\nlicense: Apache-2.0\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/CONTRIBUTING.md", "content": "## Contributing to OpenMMLab\n\nAll kinds of contributions are welcome, including but not limited to the following.\n\n- Fix typo or bugs\n- Add documentation or translate the documentation into other languages\n- Add new features and components\n\n### Workflow\n\n1. fork and pull the latest OpenMMLab repository\n2. checkout a new branch (do not use master branch for PRs)\n3. commit your changes\n4. create a PR\n\n```{note}\nIf you plan to add some new features that involve large changes, it is encouraged to open an issue for discussion first.\n```\n### Code style\n\n#### Python\n\nWe adopt [PEP8](https://www.python.org/dev/peps/pep-0008/) as the preferred code style.\n\nWe use the following tools for linting and formatting:\n\n- [flake8](http://flake8.pycqa.org/en/latest/): A wrapper around some linter tools.\n- [yapf](https://github.com/google/yapf): A formatter for Python files.\n- [isort](https://github.com/timothycrosley/isort): A Python utility to sort imports.\n- [markdownlint](https://github.com/markdownlint/markdownlint): A linter to check markdown files and flag style issues.\n- [docformatter](https://github.com/myint/docformatter): A formatter to format docstring.\n\nStyle configurations of yapf and isort can be found in [setup.cfg](./setup.cfg).\n\nWe use [pre-commit hook](https://pre-commit.com/) that checks and formats for `flake8`, `yapf`, `isort`, `trailing whitespaces`, `markdown files`,\nfixes `end-of-files`, `double-quoted-strings`, `python-encoding-pragma`, `mixed-line-ending`, sorts `requirments.txt` automatically on every commit.\nThe config for a pre-commit hook is stored in [.pre-commit-config](./.pre-commit-config.yaml).\n\nAfter you clone the repository, you will need to install initialize pre-commit hook.\n\n```shell\npip install -U pre-commit\n```\n\nFrom the repository folder\n\n```shell\npre-commit install\n```\n\nTry the following steps to install ruby when you encounter an issue on installing markdownlint\n\n```shell\n# install rvm\ncurl -L https://get.rvm.io | bash -s -- --autolibs=read-fail\n[[ -s \"$HOME/.rvm/scripts/rvm\" ]] && source \"$HOME/.rvm/scripts/rvm\"\nrvm autolibs disable\n\n# install ruby\nrvm install 2.7.1\n```\n\nOr refer to [this repo](https://github.com/innerlee/setup) and take [`zzruby.sh`](https://github.com/innerlee/setup/blob/master/zzruby.sh) according its instruction.\n\nAfter this on every commit check code linters and formatter will be enforced.\n\n>Before you create a PR, make sure that your code lints and is formatted by yapf.\n\n#### C++ and CUDA\n\nWe follow the [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html).\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/Dockerfile", "content": "FROM python:3.7\n\nWORKDIR /mmcv\n\nCOPY . /mmcv\n\nRUN pip install -e .\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/Jenkinsfile", "content": "def docker_images = [\"registry.cn-hangzhou.aliyuncs.com/sensetime/openmmlab:cuda10.1-cudnn7-devel-ubuntu18.04-py37-pt1.3\",\n \"registry.cn-hangzhou.aliyuncs.com/sensetime/openmmlab:cuda10.2-cudnn7-devel-ubuntu18.04-py37-pt1.5\"]\ndef torch_versions = [\"1.3.0\", \"1.5.0\"]\ndef torchvision_versions = [\"0.4.2\", \"0.6.0\"]\n\n\ndef get_stages(docker_image, folder) {\n def pip_mirror = \"-i https://mirrors.aliyun.com/pypi/simple\"\n stages = {\n docker.image(docker_image).inside('-u root --gpus all --net host') {\n sh \"rm -rf ${env.WORKSPACE}-${folder} ${env.WORKSPACE}-${folder}@tmp\"\n sh \"cp -r ${env.WORKSPACE} ${env.WORKSPACE}-${folder}\"\n try {\n dir(\"${env.WORKSPACE}-${folder}\") {\n stage(\"before_install\") {\n sh \"apt-get update && apt-get install -y ninja-build\"\n }\n stage(\"dependencies\") {\n // torch and torchvision are pre-installed in dockers\n sh \"pip list | grep torch\"\n sh \"apt-get install -y ffmpeg libturbojpeg\"\n sh \"pip install pytest coverage lmdb PyTurboJPEG Cython ${pip_mirror}\"\n }\n stage(\"build\") {\n sh \"MMCV_WITH_OPS=1 pip install -e . ${pip_mirror}\"\n }\n stage(\"test\") {\n sh \"coverage run --branch --source=mmcv -m pytest tests/\"\n sh \"coverage xml\"\n sh \"coverage report -m\"\n }\n }\n } finally {\n sh \"rm -rf ${env.WORKSPACE}-${folder} ${env.WORKSPACE}-${folder}@tmp\"\n }\n }\n }\n return stages\n}\n\n\nnode('master') {\n // fetch latest change from SCM (Source Control Management)\n checkout scm\n\n def stages = [:]\n for (int i = 0; i < docker_images.size(); i++) {\n def docker_image = docker_images[i]\n def torch = torch_versions[i]\n def torchvision = torchvision_versions[i]\n def tag = docker_image + '_' + torch + '_' + torchvision\n def folder = \"${i}\"\n stages[tag] = get_stages(docker_image, folder)\n }\n parallel stages\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/LICENSE", "content": "Copyright (c) OpenMMLab. All rights reserved\n\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright 2018-2020 Open-MMLab. All rights reserved.\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/LICENSES.md", "content": "# Licenses for special operations\n\nIn this file, we list the operations with other licenses instead of Apache 2.0. Users should be careful about adopting these operations in any commercial matters.\n\n| Operation | Files | License |\n| :--------------: | :---------------------------------------------------------------------------------------------------------------------------------------------------: | :------------: |\n| upfirdn2d | [mmcv/ops/csrc/pytorch/cuda/upfirdn2d_kernel.cu](https://github.com/open-mmlab/mmcv/blob/master/mmcv/ops/csrc/pytorch/cuda/upfirdn2d_kernel.cu) | NVIDIA License |\n| fused_leaky_relu | [mmcv/ops/csrc/pytorch/cuda/fused_bias_leakyrelu_cuda.cu](https://github.com/open-mmlab/mmcv/blob/master/mmcv/ops/csrc/pytorch/cuda/fused_bias_leakyrelu_cuda.cu) | NVIDIA License |\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/MANIFEST.in", "content": "include requirements/runtime.txt\ninclude mmcv/model_zoo/open_mmlab.json mmcv/model_zoo/deprecated.json mmcv/model_zoo/mmcls.json\ninclude mmcv/ops/csrc/common/cuda/*.cuh mmcv/ops/csrc/common/cuda/*.hpp mmcv/ops/csrc/common/*.hpp\ninclude mmcv/ops/csrc/pytorch/*.cpp mmcv/ops/csrc/pytorch/cuda/*.cu mmcv/ops/csrc/pytorch/cuda/*.cpp mmcv/ops/csrc/pytorch/cpu/*.cpp\ninclude mmcv/ops/csrc/parrots/*.h mmcv/ops/csrc/parrots/*.cpp\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/README_zh-CN.md", "content": "
\n \n
 
\n
\n OpenMMLab 官网\n \n \n HOT\n \n \n     \n OpenMMLab 开放平台\n \n \n TRY IT OUT\n \n \n
\n
 
\n
\n\n[![docs](https://img.shields.io/badge/docs-latest-blue)](https://mmcv.readthedocs.io/zh_CN/latest/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mmcv)](https://pypi.org/project/mmcv/)\n[![PyPI](https://img.shields.io/pypi/v/mmcv)](https://pypi.org/project/mmcv)\n[![badge](https://github.com/open-mmlab/mmcv/workflows/build/badge.svg)](https://github.com/open-mmlab/mmcv/actions)\n[![codecov](https://codecov.io/gh/open-mmlab/mmcv/branch/master/graph/badge.svg)](https://codecov.io/gh/open-mmlab/mmcv)\n[![license](https://img.shields.io/github/license/open-mmlab/mmcv.svg)](https://github.com/open-mmlab/mmcv/blob/master/LICENSE)\n\n[English](README.md) | 简体中文\n\n## 简介\n\nMMCV 是一个面向计算机视觉的基础库，它支持了很多开源项目，例如：\n\n- [MIM](https://github.com/open-mmlab/mim): OpenMMLab 项目、算法、模型的统一入口\n- [MMClassification](https://github.com/open-mmlab/mmclassification): OpenMMLab 图像分类工具箱与测试基准\n- [MMDetection](https://github.com/open-mmlab/mmdetection): OpenMMLab 检测工具箱与测试基准\n- [MMDetection3D](https://github.com/open-mmlab/mmdetection3d): OpenMMLab 新一代通用3D目标检测平台\n- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation): OpenMMLab 语义分割工具箱与测试基准\n- [MMAction2](https://github.com/open-mmlab/mmaction2): OpenMMLab 新一代视频理解工具箱与测试基准\n- [MMTracking](https://github.com/open-mmlab/mmtracking): OpenMMLab 一体化视频目标感知平台\n- [MMPose](https://github.com/open-mmlab/mmpose): OpenMMLab 姿态估计工具箱与测试基准\n- [MMEditing](https://github.com/open-mmlab/mmediting): OpenMMLab 图像视频编辑工具箱\n- [MMOCR](https://github.com/open-mmlab/mmocr): OpenMMLab 全流程文字检测识别理解工具包\n- [MMGeneration](https://github.com/open-mmlab/mmgeneration): OpenMMLab 新一代生成模型工具箱\n- [MMFlow](https://github.com/open-mmlab/mmflow): OpenMMLab 光流估计工具箱与测试基准\n- [MMFewShot](https://github.com/open-mmlab/mmfewshot): OpenMMLab 少样本学习工具箱与测试基准\n- [MMHuman3D](https://github.com/open-mmlab/mmhuman3d): OpenMMLab 人体参数化模型工具箱与测试基准\n- [MMSelfSup](https://github.com/open-mmlab/mmselfsup): OpenMMLab 自监督学习工具箱与测试基准\n- [MMRazor](https://github.com/open-mmlab/mmrazor): OpenMMLab 模型压缩工具箱与测试基准\n- [MMDeploy](https://github.com/open-mmlab/mmdeploy): OpenMMLab 模型部署框架\n\nMMCV 提供了如下众多功能：\n\n- 通用的 IO 接口\n- 图像和视频处理\n- 图像和标注结果可视化\n- 常用小工具（进度条，计时器等）\n- 基于 PyTorch 的通用训练框架\n- 多种 CNN 网络结构\n- 高质量实现的常见 CUDA 算子\n\n如想了解更多特性和使用，请参考[文档](http://mmcv.readthedocs.io/zh_CN/latest)。\n\n提示: MMCV 需要 Python 3.6 以上版本。\n\n## 安装\n\nMMCV 有两个版本：\n\n- **mmcv-full**: 完整版，包含所有的特性以及丰富的开箱即用的 CUDA 算子。注意完整版本可能需要更长时间来编译。\n- **mmcv**: 精简版，不包含 CUDA 算子但包含其余所有特性和功能，类似 MMCV 1.0 之前的版本。如果你不需要使用 CUDA 算子的话，精简版可以作为一个考虑选项。\n\n**注意**: 请不要在同一个环境中安装两个版本，否则可能会遇到类似 `ModuleNotFound` 的错误。在安装一个版本之前，需要先卸载另一个。`如果CUDA可用，强烈推荐安装mmcv-full`。\n\na. 安装完整版\n\n在安装 mmcv-full 之前，请确保 PyTorch 已经成功安装在环境中，可以参考 PyTorch [官方文档](https://pytorch.org/)。\n\n我们提供了不同 PyTorch 和 CUDA 版本的 mmcv-full 预编译包，可以大大简化用户安装编译过程。强烈推荐通过预编译包来安装。另外，安装完成后可以运行 [check_installation.py](.dev_scripts/check_installation.py) 脚本检查 mmcv-full 是否安装成功。\n\ni. 安装最新版本\n\n如下是安装最新版 ``mmcv-full`` 的命令\n\n```shell\npip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html\n```\n\n请将链接中的 ``{cu_version}`` 和 ``{torch_version}`` 根据自身需求替换成实际的版本号，例如想安装和 ``CUDA 11.1``、``PyTorch 1.9.0`` 兼容的最新版 ``mmcv-full``，使用如下替换过的命令\n\n```shell\npip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html\n```\n\n**注意**: PyTorch 在 1.x.0 和 1.x.1 之间通常是兼容的，故 mmcv-full 只提供 1.x.0 的编译包。如果你的 PyTorch 版本是 1.x.1，你可以放心地安装在 1.x.0 版本编译的 mmcv-full。例如，如果你的 PyTorch 版本是 1.8.1、CUDA 版本是 11.1，你可以使用以下命令安装 mmcv-full。\n\n```shell\npip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.8.0/index.html\n```\n\n如果想知道更多 CUDA 和 PyTorch 版本的命令，可以参考下面的表格，将链接中的 ``=={mmcv_version}`` 删去即可。\n\nii. 安装特定的版本\n\n如下是安装特定版本 ``mmcv-full`` 的命令\n\n```shell\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html\n```\n\n首先请参考版本发布信息找到想要安装的版本号，将 ``{mmcv_version}`` 替换成该版本号，例如 ``1.3.9``。\n然后将链接中的 ``{cu_version}`` 和 ``{torch_version}`` 根据自身需求替换成实际的版本号，例如想安装和 ``CUDA 11.1``、``PyTorch 1.9.0`` 兼容的 ``mmcv-full`` 1.3.9 版本，使用如下替换过的命令\n\n```shell\npip install mmcv-full==1.3.9 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html\n```\n\n对于更多的 PyTorch 和 CUDA 版本组合，请参考下表：\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
CUDA torch 1.10torch 1.9torch 1.8torch 1.7torch 1.6torch 1.5
11.3
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
11.1
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.10.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.8.0/index.html
11.0
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html
10.2
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.10.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.9.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.7.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.6.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.5.0/index.html
10.1
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.8.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.7.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.6.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.5.0/index.html
9.2
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.7.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.6.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.5.0/index.html
cpu
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.10.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.9.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.8.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.7.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.6.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.5.0/index.html
\n\n**注意**：以上提供的预编译包并不囊括所有的 mmcv-full 版本，你可以点击对应链接查看支持的版本。例如，点击 [cu102-torch1.8.0](https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html)，可以看到 `cu102-torch1.8.0` 只提供了 1.3.0 及以上的 mmcv-full 版本。另外，从 `mmcv v1.3.17` 开始，我们不再提供`PyTorch 1.3 & 1.4` 对应的 mmcv-full 预编译包。你可以在 [这](./docs/zh_cn/get_started/previous_versions.md) 找到 `PyTorch 1.3 & 1.4` 对应的预编包。虽然我们不再提供 `PyTorch 1.3 & 1.4` 对应的预编译包，但是我们依然在 CI 中保证对它们的兼容持续到下一年。\n\n除了使用预编译包之外，另一种方式是在本地进行编译，直接运行下述命令\n\n```python\npip install mmcv-full\n```\n\n但注意本地编译可能会耗时 10 分钟以上。\n\nb. 安装精简版\n\n```python\npip install mmcv\n```\n\nc. 安装完整版并且编译 onnxruntime 的自定义算子\n\n- 详细的指南请查看[这里](docs/zh_cn/deployment/onnxruntime_op.md)。\n\n如果想从源码编译 MMCV，请参考[该文档](https://mmcv.readthedocs.io/zh_CN/latest/get_started/build.html)。\n\n## FAQ\n\n如果你遇到了安装问题，CUDA 相关的问题或者 RuntimeErrors，可以首先参考[问题解决页面](https://mmcv.readthedocs.io/zh_CN/latest/faq.html) 看是否已经有解决方案。\n\n## 贡献指南\n\n我们感谢所有的贡献者为改进和提升 MMCV 所作出的努力。请参考[贡献指南](CONTRIBUTING.md)来了解参与项目贡献的相关指引。\n\n## 许可证\n\n`MMCV` 目前以 Apache 2.0 的许可证发布，但是其中有一部分功能并不是使用的 Apache2.0 许可证，我们在 [许可证](LICENSES.md) 中详细地列出了这些功能以及他们对应的许可证，如果您正在从事盈利性活动，请谨慎参考此文档。\n## 欢迎加入 OpenMMLab 社区\n\n扫描下方的二维码可关注 OpenMMLab 团队的 [知乎官方账号](https://www.zhihu.com/people/openmmlab)，加入 OpenMMLab 团队的 [官方交流 QQ 群](https://jq.qq.com/?_wv=1027&k=GJP18SjI)\n\n
\n \n
\n\n我们会在 OpenMMLab 社区为大家\n\n- 📢 分享 AI 框架的前沿核心技术\n- 💻 解读 PyTorch 常用模块源码\n- 📰 发布 OpenMMLab 的相关新闻\n- 🚀 介绍 OpenMMLab 开发的前沿算法\n- 🏃 获取更高效的问题答疑和意见反馈\n- 🔥 提供与各行各业开发者充分交流的平台\n\n干货满满 📘，等你来撩 💗，OpenMMLab 社区期待您的加入 👬\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/TERMINOLOGY.md", "content": "# English-Chinese terminology comparison (英汉术语对照)\n\nThis document is used as a reference for English-Chinese terminology translation.\n\n该文档用作中英文翻译对照参考。\n\n| English | 中文 |\n| :-----: | :---:|\n| annotation | 标注 |\n| backbone | 主干网络 |\n| benchmark | 基准测试 |\n| checkpoint | 模型权重文件 |\n| classifier | 分类器 |\n| cls_head | 分类头 |\n| decoder | 解码器 |\n| detector | 检测器 |\n| encoder | 编码器 |\n| finetune | 微调 |\n| ground truth | 真实标签 |\n| hook | 钩子 |\n| localizer | 定位器 |\n| neck | 模型颈部 |\n| pipeline | 流水线 |\n| recognizer | 识别器 |\n| register | 注册器 |\n| schedule | 调整 |\n| scheduler | 调度器 |\n| segmentor | 分割器 |\n| tensor | 张量 |\n| training schedule | 训练策略 |\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/Makefile", "content": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD = sphinx-build\nSOURCEDIR = .\nBUILDDIR = _build\n\n# Put it first so that \"make\" without argument is like \"make help\".\nhelp:\n\t@$(SPHINXBUILD) -M help \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n\n.PHONY: help Makefile\n\n# Catch-all target: route all unknown targets to Sphinx using the new\n# \"make mode\" option. $(O) is meant as a shortcut for $(SPHINXOPTS).\n%: Makefile\n\t@$(SPHINXBUILD) -M $@ \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/_static/css/readthedocs.css", "content": ".header-logo {\n background-image: url(\"../image/mmcv-logo.png\");\n background-size: 85px 40px;\n height: 40px;\n width: 85px;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/api.rst", "content": "fileio\n-------\n.. automodule:: mmcv.fileio\n :members:\n\nimage\n------\n.. automodule:: mmcv.image\n :members:\n\nvideo\n------\n.. automodule:: mmcv.video\n :members:\n\narraymisc\n---------\n.. automodule:: mmcv.arraymisc\n :members:\n\nvisualization\n--------------\n.. automodule:: mmcv.visualization\n :members:\n\nutils\n-----\n.. automodule:: mmcv.utils\n :members:\n\ncnn\n----\n.. automodule:: mmcv.cnn\n :members:\n\nrunner\n------\n.. automodule:: mmcv.runner\n :members:\n\nengine\n------\n.. automodule:: mmcv.engine\n :members:\n\nops\n------\n.. automodule:: mmcv.ops\n :members:\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/community/pr.md", "content": "## Pull Request (PR)\n\n### What is PR\n\n`PR` is the abbreviation of `Pull Request`. Here's the definition of `PR` in the [official document](https://docs.github.com/en/github/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests) of Github.\n\n```\nPull requests let you tell others about changes you have pushed to a branch in a repository on GitHub. Once a pull request is opened, you can discuss and review the potential changes with collaborators and add follow-up commits before your changes are merged into the base branch.\n```\n\n### Basic Workflow\n\n1. Get the most recent codebase\n2. Checkout a new branch from the master branch\n3. Commit your changes\n4. Push your changes and create a PR\n5. Discuss and review your code\n6. Merge your branch to the master branch\n\n### Procedures in detail\n\n#### 1. Get the most recent codebase\n\n+ When you work on your first PR\n\n Fork the OpenMMLab repository: click the **fork** button at the top right corner of Github page\n ![avatar](../_static/community/1.png)\n\n Clone forked repository to local\n\n ```bash\n git clone git@github.com:XXX/mmcv.git\n ```\n\n Add source repository to upstream\n\n ```bash\n git remote add upstream git@github.com:open-mmlab/mmcv\n ```\n\n+ After your first PR\n\n Checkout master branch of the local repository and pull the latest master branch of the source repository\n\n ```bash\n git checkout master\n git pull upstream master\n ```\n\n#### 2. Checkout a new branch from the master branch\n\n```bash\ngit checkout -b branchname\n```\n\n```{tip}\nTo make commit history clear, we strongly recommend you checkout the master branch before create a new branch.\n```\n\n#### 3. Commit your changes\n\n```bash\n# coding\ngit add [files]\ngit commit -m 'messages'\n```\n\n#### 4. Push your changes to the forked repository and create a PR\n\n+ Push the branch to your forked remote repository\n\n ```bash\n git push origin branchname\n ```\n\n+ Create a PR\n![avatar](../_static/community/2.png)\n\n+ Revise PR message template to describe your motivation and modifications made in this PR. You can also link the related issue to the PR manually in the PR message (For more information, checkout the [official guidance](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue)).\n\n#### 5. Discuss and review your code\n\n+ After creating a pull request, you can ask a specific person to review the changes you've proposed\n![avatar](../_static/community/3.png)\n\n+ Modify your codes according to reviewers' suggestions and then push your changes\n\n#### 6. Merge your branch to the master branch and delete the branch\n\n```bash\ngit branch -d branchname # delete local branch\ngit push origin --delete branchname # delete remote branch\n```\n\n### PR Specs\n\n1. Use [pre-commit](https://pre-commit.com) hook to avoid issues of code style\n\n2. One short-time branch should be matched with only one PR\n\n3. Accomplish a detailed change in one PR. Avoid large PR\n\n + Bad: Support Faster R-CNN\n + Acceptable: Add a box head to Faster R-CNN\n + Good: Add a parameter to box head to support custom conv-layer number\n\n4. Provide clear and significant commit message\n\n5. Provide clear and meaningful PR description\n\n + Task name should be clarified in title. The general format is: [Prefix] Short description of the PR (Suffix)\n + Prefix: add new feature [Feature], fix bug [Fix], related to documents [Docs], in developing [WIP] (which will not be reviewed temporarily)\n + Introduce main changes, results and influences on other modules in short description\n + Associate related issues and pull requests with a milestone\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/compatibility.md", "content": "### v1.3.18\n\nSome ops have different implementations on different devices. Lots of macros and type checks are scattered in several files, which makes the code hard to maintain. For example:\n\n```c++\n if (input.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n CHECK_CUDA_INPUT(input);\n CHECK_CUDA_INPUT(rois);\n CHECK_CUDA_INPUT(output);\n CHECK_CUDA_INPUT(argmax_y);\n CHECK_CUDA_INPUT(argmax_x);\n\n roi_align_forward_cuda(input, rois, output, argmax_y, argmax_x,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n#else\n AT_ERROR(\"RoIAlign is not compiled with GPU support\");\n#endif\n } else {\n CHECK_CPU_INPUT(input);\n CHECK_CPU_INPUT(rois);\n CHECK_CPU_INPUT(output);\n CHECK_CPU_INPUT(argmax_y);\n CHECK_CPU_INPUT(argmax_x);\n roi_align_forward_cpu(input, rois, output, argmax_y, argmax_x,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n }\n```\n\nRegistry and dispatcher are added to manage these implementations.\n\n```c++\n\nvoid ROIAlignForwardCUDAKernelLauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nvoid roi_align_forward_cuda(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n ROIAlignForwardCUDAKernelLauncher(\n input, rois, output, argmax_y, argmax_x, aligned_height, aligned_width,\n spatial_scale, sampling_ratio, pool_mode, aligned);\n}\n\n// register cuda implementation\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\nREGISTER_DEVICE_IMPL(roi_align_forward_impl, CUDA, roi_align_forward_cuda);\n\n// roi_align.cpp\n// use the dispatcher to invoke different implementation depending on device type of input tensors.\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n DISPATCH_DEVICE_IMPL(roi_align_forward_impl, input, rois, output, argmax_y,\n argmax_x, aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\n```\n\n### v1.3.11\n\nIn order to flexibly support more backends and hardwares like `NVIDIA GPUs` and `AMD GPUs`, the directory of `mmcv/ops/csrc` is refactored. Note that this refactoring will not affect the usage in API. For related information, please refer to [PR1206](https://github.com/open-mmlab/mmcv/pull/1206).\n\nThe original directory was organized as follows.\n\n```\n.\n├── common_cuda_helper.hpp\n├── ops_cuda_kernel.cuh\n├── pytorch_cpp_helper.hpp\n├── pytorch_cuda_helper.hpp\n├── parrots_cpp_helper.hpp\n├── parrots_cuda_helper.hpp\n├── parrots_cudawarpfunction.cuh\n├── onnxruntime\n│   ├── onnxruntime_register.h\n│   ├── onnxruntime_session_options_config_keys.h\n│   ├── ort_mmcv_utils.h\n│   ├── ...\n│   ├── onnx_ops.h\n│   └── cpu\n│ ├── onnxruntime_register.cpp\n│      ├── ...\n│      └── onnx_ops_impl.cpp\n├── parrots\n│   ├── ...\n│   ├── ops.cpp\n│   ├── ops_cuda.cu\n│   ├── ops_parrots.cpp\n│   └── ops_pytorch.h\n├── pytorch\n│   ├── ...\n│   ├── ops.cpp\n│   ├── ops_cuda.cu\n│   ├── pybind.cpp\n└── tensorrt\n ├── trt_cuda_helper.cuh\n ├── trt_plugin_helper.hpp\n ├── trt_plugin.hpp\n ├── trt_serialize.hpp\n ├── ...\n ├── trt_ops.hpp\n └── plugins\n    ├── trt_cuda_helper.cu\n    ├── trt_plugin.cpp\n    ├── ...\n    ├── trt_ops.cpp\n    └── trt_ops_kernel.cu\n```\n\nAfter refactored, it is organized as follows.\n\n```\n.\n├── common\n│ ├── box_iou_rotated_utils.hpp\n│ ├── parrots_cpp_helper.hpp\n│ ├── parrots_cuda_helper.hpp\n│ ├── pytorch_cpp_helper.hpp\n│ ├── pytorch_cuda_helper.hpp\n│   └── cuda\n│   ├── common_cuda_helper.hpp\n│   ├── parrots_cudawarpfunction.cuh\n│   ├── ...\n│   └── ops_cuda_kernel.cuh\n├── onnxruntime\n│   ├── onnxruntime_register.h\n│   ├── onnxruntime_session_options_config_keys.h\n│   ├── ort_mmcv_utils.h\n│   ├── ...\n│   ├── onnx_ops.h\n│   └── cpu\n│ ├── onnxruntime_register.cpp\n│      ├── ...\n│      └── onnx_ops_impl.cpp\n├── parrots\n│   ├── ...\n│   ├── ops.cpp\n│   ├── ops_parrots.cpp\n│   └── ops_pytorch.h\n├── pytorch\n│   ├── info.cpp\n│   ├── pybind.cpp\n│   ├── ...\n│   ├── ops.cpp\n│   └── cuda\n│      ├── ...\n│      └── ops_cuda.cu\n└── tensorrt\n ├── trt_cuda_helper.cuh\n ├── trt_plugin_helper.hpp\n ├── trt_plugin.hpp\n ├── trt_serialize.hpp\n ├── ...\n ├── trt_ops.hpp\n └── plugins\n    ├── trt_cuda_helper.cu\n    ├── trt_plugin.cpp\n    ├── ...\n    ├── trt_ops.cpp\n    └── trt_ops_kernel.cu\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/conf.py", "content": "#\n# Configuration file for the Sphinx documentation builder.\n#\n# This file does only contain a selection of the most common options. For a\n# full list see the documentation:\n# http://www.sphinx-doc.org/en/master/config\n\n# -- Path setup --------------------------------------------------------------\n\n# If extensions (or modules to document with autodoc) are in another directory,\n# add these directories to sys.path here. If the directory is relative to the\n# documentation root, use os.path.abspath to make it absolute, like shown here.\n#\nimport os\nimport sys\n\nimport pytorch_sphinx_theme\nfrom sphinx.builders.html import StandaloneHTMLBuilder\n\nsys.path.insert(0, os.path.abspath('../..'))\n\nversion_file = '../../mmcv/version.py'\nwith open(version_file, 'r') as f:\n exec(compile(f.read(), version_file, 'exec'))\n__version__ = locals()['__version__']\n\n# -- Project information -----------------------------------------------------\n\nproject = 'mmcv'\ncopyright = '2018-2021, OpenMMLab'\nauthor = 'MMCV Authors'\n\n# The short X.Y version\nversion = __version__\n# The full version, including alpha/beta/rc tags\nrelease = __version__\n\n# -- General configuration ---------------------------------------------------\n\n# If your documentation needs a minimal Sphinx version, state it here.\n#\n# needs_sphinx = '1.0'\n\n# Add any Sphinx extension module names here, as strings. They can be\n# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom\n# ones.\n\nextensions = [\n 'sphinx.ext.autodoc',\n 'sphinx.ext.napoleon',\n 'sphinx.ext.viewcode',\n 'sphinx_markdown_tables',\n 'myst_parser',\n 'sphinx_copybutton',\n] # yapf: disable\n\nautodoc_mock_imports = ['mmcv._ext', 'mmcv.utils.ext_loader', 'torchvision']\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = ['_templates']\n\n# The suffix(es) of source filenames.\n# You can specify multiple suffix as a list of string:\n#\nsource_suffix = {\n '.rst': 'restructuredtext',\n '.md': 'markdown',\n}\n\n# The master toctree document.\nmaster_doc = 'index'\n\n# The language for content autogenerated by Sphinx. Refer to documentation\n# for a list of supported languages.\n#\n# This is also used if you do content translation via gettext catalogs.\n# Usually you set \"language\" from the command line for these cases.\nlanguage = None\n\n# List of patterns, relative to source directory, that match files and\n# directories to ignore when looking for source files.\n# This pattern also affects html_static_path and html_extra_path.\nexclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']\n\n# The name of the Pygments (syntax highlighting) style to use.\npygments_style = 'sphinx'\n\n# -- Options for HTML output -------------------------------------------------\n\n# The theme to use for HTML and HTML Help pages. See the documentation for\n# a list of builtin themes.\n#\n# html_theme = 'sphinx_rtd_theme'\nhtml_theme = 'pytorch_sphinx_theme'\nhtml_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]\n\n# Theme options are theme-specific and customize the look and feel of a theme\n# further. For a list of options available for each theme, see the\n# documentation.\n#\nhtml_theme_options = {\n 'menu': [\n {\n 'name': 'GitHub',\n 'url': 'https://github.com/open-mmlab/mmcv'\n },\n ],\n # Specify the language of shared menu\n 'menu_lang': 'en',\n}\n\n# Add any paths that contain custom static files (such as style sheets) here,\n# relative to this directory. They are copied after the builtin static files,\n# so a file named \"default.css\" will overwrite the builtin \"default.css\".\nhtml_static_path = ['_static']\nhtml_css_files = ['css/readthedocs.css']\n\n# Custom sidebar templates, must be a dictionary that maps document names\n# to template names.\n#\n# The default sidebars (for documents that don't match any pattern) are\n# defined by theme itself. Builtin themes are using these templates by\n# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',\n# 'searchbox.html']``.\n#\n# html_sidebars = {}\n\n# -- Options for HTMLHelp output ---------------------------------------------\n\n# Output file base name for HTML help builder.\nhtmlhelp_basename = 'mmcvdoc'\n\n# -- Options for LaTeX output ------------------------------------------------\n\nlatex_elements = {\n # The paper size ('letterpaper' or 'a4paper').\n #\n # 'papersize': 'letterpaper',\n\n # The font size ('10pt', '11pt' or '12pt').\n #\n # 'pointsize': '10pt',\n\n # Additional stuff for the LaTeX preamble.\n #\n # 'preamble': '',\n\n # Latex figure (float) alignment\n #\n # 'figure_align': 'htbp',\n}\n\n# Grouping the document tree into LaTeX files. List of tuples\n# (source start file, target name, title,\n# author, documentclass [howto, manual, or own class]).\nlatex_documents = [\n (master_doc, 'mmcv.tex', 'mmcv Documentation', 'MMCV Contributors',\n 'manual'),\n]\n\n# -- Options for manual page output ------------------------------------------\n\n# One entry per manual page. List of tuples\n# (source start file, name, description, authors, manual section).\nman_pages = [(master_doc, 'mmcv', 'mmcv Documentation', [author], 1)]\n\n# -- Options for Texinfo output ----------------------------------------------\n\n# Grouping the document tree into Texinfo files. List of tuples\n# (source start file, target name, title, author,\n# dir menu entry, description, category)\ntexinfo_documents = [\n (master_doc, 'mmcv', 'mmcv Documentation', author, 'mmcv',\n 'One line description of project.', 'Miscellaneous'),\n]\n\n# -- Options for Epub output -------------------------------------------------\n\n# Bibliographic Dublin Core info.\nepub_title = project\n\n# The unique identifier of the text. This can be a ISBN number\n# or the project homepage.\n#\n# epub_identifier = ''\n\n# A unique identification for the text.\n#\n# epub_uid = ''\n\n# A list of files that should not be packed into the epub file.\nepub_exclude_files = ['search.html']\n\n# set priority when building html\nStandaloneHTMLBuilder.supported_image_types = [\n 'image/svg+xml', 'image/gif', 'image/png', 'image/jpeg'\n]\n# -- Extension configuration -------------------------------------------------\n# Ignore >>> when copying code\ncopybutton_prompt_text = r'>>> |\\.\\.\\. '\ncopybutton_prompt_is_regexp = True\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/deployment/mmcv_ops_definition.md", "content": "# MMCV Operators\n\nTo make custom operators in MMCV more standard, precise definitions of each operator are listed in this document.\n\n\n- [MMCV Operators](#mmcv-operators)\n - [MMCVBorderAlign](#mmcvborderalign)\n - [Description](#description)\n - [Parameters](#parameters)\n - [Inputs](#inputs)\n - [Outputs](#outputs)\n - [Type Constraints](#type-constraints)\n - [MMCVCARAFE](#mmcvcarafe)\n - [Description](#description-1)\n - [Parameters](#parameters-1)\n - [Inputs](#inputs-1)\n - [Outputs](#outputs-1)\n - [Type Constraints](#type-constraints-1)\n - [MMCVCAWeight](#mmcvcaweight)\n - [Description](#description-2)\n - [Parameters](#parameters-2)\n - [Inputs](#inputs-2)\n - [Outputs](#outputs-2)\n - [Type Constraints](#type-constraints-2)\n - [MMCVCAMap](#mmcvcamap)\n - [Description](#description-3)\n - [Parameters](#parameters-3)\n - [Inputs](#inputs-3)\n - [Outputs](#outputs-3)\n - [Type Constraints](#type-constraints-3)\n - [MMCVCornerPool](#mmcvcornerpool)\n - [Description](#description-4)\n - [Parameters](#parameters-4)\n - [Inputs](#inputs-4)\n - [Outputs](#outputs-4)\n - [Type Constraints](#type-constraints-4)\n - [MMCVDeformConv2d](#mmcvdeformconv2d)\n - [Description](#description-5)\n - [Parameters](#parameters-5)\n - [Inputs](#inputs-5)\n - [Outputs](#outputs-5)\n - [Type Constraints](#type-constraints-5)\n - [MMCVModulatedDeformConv2d](#mmcvmodulateddeformconv2d)\n - [Description](#description-6)\n - [Parameters](#parameters-6)\n - [Inputs](#inputs-6)\n - [Outputs](#outputs-6)\n - [Type Constraints](#type-constraints-6)\n - [MMCVDeformRoIPool](#mmcvdeformroipool)\n - [Description](#description-7)\n - [Parameters](#parameters-7)\n - [Inputs](#inputs-7)\n - [Outputs](#outputs-7)\n - [Type Constraints](#type-constraints-7)\n - [MMCVMaskedConv2d](#mmcvmaskedconv2d)\n - [Description](#description-8)\n - [Parameters](#parameters-8)\n - [Inputs](#inputs-8)\n - [Outputs](#outputs-8)\n - [Type Constraints](#type-constraints-8)\n - [MMCVPSAMask](#mmcvpsamask)\n - [Description](#description-9)\n - [Parameters](#parameters-9)\n - [Inputs](#inputs-9)\n - [Outputs](#outputs-9)\n - [Type Constraints](#type-constraints-9)\n - [NonMaxSuppression](#nonmaxsuppression)\n - [Description](#description-10)\n - [Parameters](#parameters-10)\n - [Inputs](#inputs-10)\n - [Outputs](#outputs-10)\n - [Type Constraints](#type-constraints-10)\n - [MMCVRoIAlign](#mmcvroialign)\n - [Description](#description-11)\n - [Parameters](#parameters-11)\n - [Inputs](#inputs-11)\n - [Outputs](#outputs-11)\n - [Type Constraints](#type-constraints-11)\n - [MMCVRoIAlignRotated](#mmcvroialignrotated)\n - [Description](#description-12)\n - [Parameters](#parameters-12)\n - [Inputs](#inputs-12)\n - [Outputs](#outputs-12)\n - [Type Constraints](#type-constraints-12)\n - [grid_sampler*](#grid_sampler)\n - [Description](#description-13)\n - [Parameters](#parameters-13)\n - [Inputs](#inputs-13)\n - [Outputs](#outputs-13)\n - [Type Constraints](#type-constraints-13)\n - [cummax*](#cummax)\n - [Description](#description-14)\n - [Parameters](#parameters-14)\n - [Inputs](#inputs-14)\n - [Outputs](#outputs-14)\n - [Type Constraints](#type-constraints-14)\n - [cummin*](#cummin)\n - [Description](#description-15)\n - [Parameters](#parameters-15)\n - [Inputs](#inputs-15)\n - [Outputs](#outputs-15)\n - [Type Constraints](#type-constraints-15)\n - [Reminders](#reminders)\n\n\n## MMCVBorderAlign\n\n### Description\n\nApplies `border_align` over the input feature based on predicted bboxes.\n\nFor each border line (e.g. top, left, bottom or right) of each box,\nborder_align does the following:\n\n- uniformly samples `pool_size`+1 positions on this line, involving the start and end points.\n- the corresponding features on these points are computed by bilinear interpolation.\n- max pooling over all the `pool_size`+1 positions are used for computing pooled feature.\n\nRead [BorderDet: Border Feature for Dense Object Detection](ttps://arxiv.org/abs/2007.11056) for more detailed information.\n\n### Parameters\n\n| Type | Parameter | Description |\n|-------|-------------|-------------------------------------------------------------------------------------|\n| `int` | `pool_size` | number of positions sampled over the boxes' borders(e.g. top, bottom, left, right). |\n\n### Inputs\n\n
\n
input: T
\n
Features with shape [N,4C,H,W]. Channels ranged in [0,C), [C,2C), [2C,3C), [3C,4C) represent the top, left, bottom, right features respectively
\n
boxes: T
\n
Boxes with shape [N,H*W,4]. Coordinate format (x1,y1,x2,y2).
\n
\n\n### Outputs\n\n
\n
output: T
\n
Pooled features with shape [N,C,H*W,4]. The order is(top,left,bottom,right) for the last dimension.
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## MMCVCARAFE\n\n### Description\n\nCARAFE operator performs feature upsampling.\n\nRead [CARAFE: Content-Aware ReAssembly of FEatures](https://arxiv.org/abs/1905.02188) for more detailed information.\n\n### Parameters\n\n| Type | Parameter | Description |\n|---------|----------------|-----------------------------------------------|\n| `int` | `kernel_size` | reassemble kernel size, should be odd integer |\n| `int` | `group_size` | reassemble group size |\n| `float` | `scale_factor` | upsample ratio(>=1) |\n\n### Inputs\n\n
\n
features: T
\n
Input features. 4-D tensor of shape (N, C, H, W). N is the batch size.
\n
masks: T
\n
The input mask
\n
\n\n### Outputs\n\n
\n
output: T
\n
The upsampled features. 4-D tensor of shape (N, C, H * scale_factor, W * scale_factor). N is the batch size.
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## MMCVCAWeight\n\n### Description\n\nOperator for Criss-Cross Attention\nRead [CCNet: Criss-Cross Attention for SemanticSegmentation](https://arxiv.org/pdf/1811.11721.pdf) for more detailed information.\n\n### Parameters\n\nNone\n\n### Inputs\n\n
\n
t: T
\n
The query matrix of shape (N, C', H, W).
\n
f: T
\n
The key matrix of shape (N, C', H, W).
\n
\n\n### Outputs\n\n
\n
weight: T
\n
The attention map of shape (N, H+W-1, H, W).
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## MMCVCAMap\n\n### Description\n\nOperator for Criss-Cross Attention\nRead [CCNet: Criss-Cross Attention for SemanticSegmentation](https://arxiv.org/pdf/1811.11721.pdf) for more detailed information.\n\n### Parameters\n\nNone\n\n### Inputs\n\n
\n
weight: T
\n
Output from the operator MMCVCAWeight.
\n
value: T
\n
The value matrix of shape (N, C, H, W).
\n
\n\n### Outputs\n\n
\n
output: T
\n
Output tensor of aggregated contextual information
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n\n## MMCVCornerPool\n\n### Description\n\nPerform CornerPool on `input` features. Read [CornerNet -- Detecting Objects as Paired Keypoints](https://arxiv.org/abs/1808.01244) for more details.\n\n### Parameters\n\n| Type | Parameter | Description |\n|-------|-----------|------------------------------------------------------------------|\n| `int` | `mode` | corner pool mode, (0: `top`, 1: `bottom`, 2: `left`, 3: `right`) |\n\n### Inputs\n\n
\n
input: T
\n
Input features. 4-D tensor of shape (N, C, H, W). N is the batch size.
\n
\n\n### Outputs\n\n
\n
output: T
\n
The pooled features. 4-D tensor of shape (N, C, H, W).
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## MMCVDeformConv2d\n\n### Description\n\nApplies a deformable 2D convolution over an input signal composed of several input planes.\n\nRead [Deformable Convolutional Networks](https://arxiv.org/pdf/1703.06211.pdf) for detail.\n\n### Parameters\n\n| Type | Parameter | Description |\n|----------------|---------------------|-------------------------------------------------------------------------------------------------------------------|\n| `list of ints` | `stride` | The stride of the convolving kernel, (sH, sW). Defaults to `(1, 1)`. |\n| `list of ints` | `padding` | Paddings on both sides of the input, (padH, padW). Defaults to `(0, 0)`. |\n| `list of ints` | `dilation` | The spacing between kernel elements (dH, dW). Defaults to `(1, 1)`. |\n| `int` | `groups` | Split input into groups. `input_channel` should be divisible by the number of groups. Defaults to `1`. |\n| `int` | `deformable_groups` | Groups of deformable offset. Defaults to `1`. |\n| `int` | `bias` | Whether to add a learnable bias to the output. `0` stands for `False` and `1` stands for `True`. Defaults to `0`. |\n| `int` | `im2col_step` | Groups of deformable offset. Defaults to `32`. |\n\n### Inputs\n\n
\n
input: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the number of channels, inH and inW are the height and width of the data.
\n
offset: T
\n
Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW are the height and width of weight, outH and outW is the height and width of offset and output.
\n
weight: T
\n
Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
\n
\n\n### Outputs\n\n
\n
output: T
\n
Output feature; 4-D tensor of shape (N, output_channel, outH, outW).
\n
\n\n### Type Constraints\n\n- T:tensor(float32, Linear)\n\n## MMCVModulatedDeformConv2d\n\n### Description\n\nPerform Modulated Deformable Convolution on input feature, read [Deformable ConvNets v2: More Deformable, Better Results](https://arxiv.org/abs/1811.11168?from=timeline) for detail.\n\n### Parameters\n\n| Type | Parameter | Description |\n|----------------|---------------------|---------------------------------------------------------------------------------------|\n| `list of ints` | `stride` | The stride of the convolving kernel. (sH, sW) |\n| `list of ints` | `padding` | Paddings on both sides of the input. (padH, padW) |\n| `list of ints` | `dilation` | The spacing between kernel elements. (dH, dW) |\n| `int` | `deformable_groups` | Groups of deformable offset. |\n| `int` | `groups` | Split input into groups. `input_channel` should be divisible by the number of groups. |\n\n### Inputs\n\n
\n
feature: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the number of channels, inH and inW are the height and width of the data.
\n
offset: T
\n
Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW are the height and width of weight, outH and outW are the height and width of offset and output.
\n
mask: T
\n
Input mask; 4-D tensor of shape (N, deformable_group* kH* kW, outH, outW), where kH and kW are the height and width of weight, outH and outW are the height and width of offset and output.
\n
weight]: T
\n
Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
\n
bias: T, optional
\n
Input bias; 1-D tensor of shape (output_channel).
\n
\n\n### Outputs\n\n
\n
output: T
\n
Output feature; 4-D tensor of shape (N, output_channel, outH, outW).
\n
\n\n### Type Constraints\n\n- T:tensor(float32, Linear)\n\n## MMCVDeformRoIPool\n\n### Description\n\nDeformable roi pooling layer\n\n### Parameters\n\n| Type | Parameter | Description |\n|---------|------------------|---------------------------------------------------------------------------------------------------------------|\n| `int` | `output_height` | height of output roi |\n| `int` | `output_width` | width of output roi |\n| `float` | `spatial_scale` | used to scale the input boxes |\n| `int` | `sampling_ratio` | number of input samples to take for each output sample. `0` means to take samples densely for current models. |\n| `float` | `gamma` | gamma |\n\n### Inputs\n\n
\n
input: T
\n
Input feature map; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
\n
rois: T
\n
RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of input.
\n
offset: T
\n
offset of height and width. Defaults to a tensor of zero
\n
\n\n### Outputs\n\n
\n
feat: T
\n
RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element feat[r-1] is a pooled feature map corresponding to the r-th RoI RoIs[r-1].
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## MMCVMaskedConv2d\n\n### Description\n\nPerforms a masked 2D convolution from PixelRNN\nRead [Pixel Recurrent Neural Networks](https://arxiv.org/abs/1601.06759) for more detailed information.\n\n### Parameters\n\n| Type | Parameter | Description |\n|----------------|-----------|----------------------------------------------------------------------------------|\n| `list of ints` | `stride` | The stride of the convolving kernel. (sH, sW). **Only support stride=1 in mmcv** |\n| `list of ints` | `padding` | Paddings on both sides of the input. (padH, padW). Defaults to `(0, 0)`. |\n\n### Inputs\n\n
\n
features: T
\n
Input features; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
\n
mask: T
\n
Input mask; 3D tensor of shape (N, H, W)
\n
weight: T
\n
The learnable weights of the module
\n
bias: T
\n
The learnable bias of the module
\n
\n\n### Outputs\n\n
\n
output: T
\n
The output convolved feature
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## MMCVPSAMask\n\n### Description\n\nAn operator from PSANet.\n\nRead [PSANet: Point-wise Spatial Attention Network for Scene Parsing](https://hszhao.github.io/papers/eccv18_psanet.pdf) for more detailed information.\n\n### Parameters\n\n| Type | Parameter | Description |\n|----------------|-------------|----------------------------------------------|\n| `int` | `psa_type` | `0` means collect and `1` means `distribute` |\n| `list of ints` | `mask_size` | The size of mask |\n\n### Inputs\n\n
\n
input: T
\n
Input feature; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
\n
\n\n### Outputs\n\n
\n
output: T
\n
Output tensor of shape (N, H * W, H, W)
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## NonMaxSuppression\n\n### Description\n\nFilter out boxes has high IoU overlap with previously selected boxes or low score. Output the indices of valid boxes.\n\nNote this definition is slightly different with [onnx: NonMaxSuppression](https://github.com/onnx/onnx/blob/master/docs/Operators.md#nonmaxsuppression)\n\n### Parameters\n\n| Type | Parameter | Description |\n|---------|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|\n| `int` | `center_point_box` | 0 - the box data is supplied as [y1, x1, y2, x2], 1-the box data is supplied as [x_center, y_center, width, height]. |\n| `int` | `max_output_boxes_per_class` | The maximum number of boxes to be selected per batch per class. Default to 0, number of output boxes equal to number of input boxes. |\n| `float` | `iou_threshold` | The threshold for deciding whether boxes overlap too much with respect to IoU. Value range [0, 1]. Default to 0. |\n| `float` | `score_threshold` | The threshold for deciding when to remove boxes based on score. |\n| `int` | `offset` | 0 or 1, boxes' width or height is (x2 - x1 + offset). |\n\n### Inputs\n\n
\n
boxes: T
\n
Input boxes. 3-D tensor of shape (num_batches, spatial_dimension, 4).
\n
scores: T
\n
Input scores. 3-D tensor of shape (num_batches, num_classes, spatial_dimension).
\n
\n\n### Outputs\n\n
\n
indices: tensor(int32, Linear)
\n
Selected indices. 2-D tensor of shape (num_selected_indices, 3) as [[batch_index, class_index, box_index], ...].
\n
num_selected_indices=num_batches* num_classes* min(max_output_boxes_per_class, spatial_dimension).
\n
All invalid indices will be filled with -1.
\n
\n\n### Type Constraints\n\n- T:tensor(float32, Linear)\n\n## MMCVRoIAlign\n\n### Description\n\nPerform RoIAlign on output feature, used in bbox_head of most two-stage detectors.\n\n### Parameters\n\n| Type | Parameter | Description |\n|---------|------------------|---------------------------------------------------------------------------------------------------------------|\n| `int` | `output_height` | height of output roi |\n| `int` | `output_width` | width of output roi |\n| `float` | `spatial_scale` | used to scale the input boxes |\n| `int` | `sampling_ratio` | number of input samples to take for each output sample. `0` means to take samples densely for current models. |\n| `str` | `mode` | pooling mode in each bin. `avg` or `max` |\n| `int` | `aligned` | If `aligned=0`, use the legacy implementation in MMDetection. Else, align the results more perfectly. |\n\n### Inputs\n\n
\n
input: T
\n
Input feature map; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
\n
rois: T
\n
RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of input.
\n
\n\n### Outputs\n\n
\n
feat: T
\n
RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element feat[r-1] is a pooled feature map corresponding to the r-th RoI RoIs[r-1].
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## MMCVRoIAlignRotated\n\n### Description\n\nPerform RoI align pooling for rotated proposals\n\n### Parameters\n\n| Type | Parameter | Description |\n|---------|------------------|---------------------------------------------------------------------------------------------------------------|\n| `int` | `output_height` | height of output roi |\n| `int` | `output_width` | width of output roi |\n| `float` | `spatial_scale` | used to scale the input boxes |\n| `int` | `sampling_ratio` | number of input samples to take for each output sample. `0` means to take samples densely for current models. |\n| `str` | `mode` | pooling mode in each bin. `avg` or `max` |\n| `int` | `aligned` | If `aligned=0`, use the legacy implementation in MMDetection. Else, align the results more perfectly. |\n| `int` | `clockwise` | If `aligned=0`, use the legacy implementation in MMDetection. Else, align the results more perfectly. |\n\n### Inputs\n\n
\n
features: T
\n
Input feature map; 4D tensor of shape (N, C, H, W)
\n
rois: T
\n
RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of input.
\n
\n\n### Outputs\n\n
\n
RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element feat[r-1] is a pooled feature map corresponding to the r-th RoI RoIs[r-1].
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## grid_sampler*\n\n### Description\n\nPerform sample from `input` with pixel locations from `grid`.\n\nCheck [torch.nn.functional.grid_sample](https://pytorch.org/docs/stable/generated/torch.nn.functional.grid_sample.html?highlight=grid_sample#torch.nn.functional.grid_sample) for more information.\n\n### Parameters\n\n| Type | Parameter | Description |\n|-------|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `int` | `interpolation_mode` | Interpolation mode to calculate output values. (0: `bilinear` , 1: `nearest`) |\n| `int` | `padding_mode` | Padding mode for outside grid values. (0: `zeros`, 1: `border`, 2: `reflection`) |\n| `int` | `align_corners` | If `align_corners=1`, the extrema (`-1` and `1`) are considered as referring to the center points of the input's corner pixels. If `align_corners=0`, they are instead considered as referring to the corner points of the input's corner pixels, making the sampling more resolution agnostic. |\n\n### Inputs\n\n
\n
input: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the numbers of channels, inH and inW are the height and width of the data.
\n
grid: T
\n
Input offset; 4-D tensor of shape (N, outH, outW, 2), where outH and outW are the height and width of offset and output.
\n
\n\n### Outputs\n\n
\n
output: T
\n
Output feature; 4-D tensor of shape (N, C, outH, outW).
\n
\n\n### Type Constraints\n\n- T:tensor(float32, Linear)\n\n## cummax*\n\n### Description\n\nReturns a tuple (`values`, `indices`) where `values` is the cumulative maximum elements of `input` in the dimension `dim`. And `indices` is the index location of each maximum value found in the dimension `dim`. Read [torch.cummax](https://pytorch.org/docs/stable/generated/torch.cummax.html) for more details.\n\n### Parameters\n\n| Type | Parameter | Description |\n|-------|-----------|----------------------------------------|\n| `int` | `dim` | the dimension to do the operation over |\n\n### Inputs\n\n
\n
input: T
\n
The input tensor with various shapes. Tensor with empty element is also supported.
\n
\n\n### Outputs\n\n
\n
output: T
\n
Output the cumulative maximum elements of `input` in the dimension `dim`, with the same shape and dtype as `input`.
\n
indices: tensor(int64)
\n
Output the index location of each cumulative maximum value found in the dimension `dim`, with the same shape as `input`.
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## cummin*\n\n### Description\n\nReturns a tuple (`values`, `indices`) where `values` is the cumulative minimum elements of `input` in the dimension `dim`. And `indices` is the index location of each minimum value found in the dimension `dim`. Read [torch.cummin](https://pytorch.org/docs/stable/generated/torch.cummin.html) for more details.\n\n### Parameters\n\n| Type | Parameter | Description |\n|-------|-----------|----------------------------------------|\n| `int` | `dim` | the dimension to do the operation over |\n\n### Inputs\n\n
\n
input: T
\n
The input tensor with various shapes. Tensor with empty element is also supported.
\n
\n\n### Outputs\n\n
\n
output: T
\n
Output the cumulative minimum elements of `input` in the dimension `dim`, with the same shape and dtype as `input`.
\n
indices: tensor(int64)
\n
Output the index location of each cumulative minimum value found in the dimension `dim`, with the same shape as `input`.
\n
\n\n### Type Constraints\n\n- T:tensor(float32)\n\n## Reminders\n\n- Operators endwith `*` are defined in Torch and are included here for the conversion to ONNX.\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/deployment/onnx.md", "content": "## Introduction of mmcv.onnx module\n\n### register_extra_symbolics\n\nSome extra symbolic functions need to be registered before exporting PyTorch model to ONNX.\n\n#### Example\n\n```python\nimport mmcv\nfrom mmcv.onnx import register_extra_symbolics\n\nopset_version = 11\nregister_extra_symbolics(opset_version)\n```\n\n#### Reminder\n\n- *Please note that this feature is experimental and may change in the future.*\n\n#### FAQs\n\n- None\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/deployment/onnxruntime_custom_ops.md", "content": "## ONNX Runtime Custom Ops\n\n\n\n- [ONNX Runtime Custom Ops](#onnx-runtime-custom-ops)\n - [SoftNMS](#softnms)\n - [Description](#description)\n - [Parameters](#parameters)\n - [Inputs](#inputs)\n - [Outputs](#outputs)\n - [Type Constraints](#type-constraints)\n - [RoIAlign](#roialign)\n - [Description](#description-1)\n - [Parameters](#parameters-1)\n - [Inputs](#inputs-1)\n - [Outputs](#outputs-1)\n - [Type Constraints](#type-constraints-1)\n - [NMS](#nms)\n - [Description](#description-2)\n - [Parameters](#parameters-2)\n - [Inputs](#inputs-2)\n - [Outputs](#outputs-2)\n - [Type Constraints](#type-constraints-2)\n - [grid_sampler](#grid_sampler)\n - [Description](#description-3)\n - [Parameters](#parameters-3)\n - [Inputs](#inputs-3)\n - [Outputs](#outputs-3)\n - [Type Constraints](#type-constraints-3)\n - [CornerPool](#cornerpool)\n - [Description](#description-4)\n - [Parameters](#parameters-4)\n - [Inputs](#inputs-4)\n - [Outputs](#outputs-4)\n - [Type Constraints](#type-constraints-4)\n - [cummax](#cummax)\n - [Description](#description-5)\n - [Parameters](#parameters-5)\n - [Inputs](#inputs-5)\n - [Outputs](#outputs-5)\n - [Type Constraints](#type-constraints-5)\n - [cummin](#cummin)\n - [Description](#description-6)\n - [Parameters](#parameters-6)\n - [Inputs](#inputs-6)\n - [Outputs](#outputs-6)\n - [Type Constraints](#type-constraints-6)\n - [MMCVModulatedDeformConv2d](#mmcvmodulateddeformconv2d)\n - [Description](#description-7)\n - [Parameters](#parameters-7)\n - [Inputs](#inputs-7)\n - [Outputs](#outputs-7)\n - [Type Constraints](#type-constraints-7)\n - [MMCVDeformConv2d](#mmcvdeformconv2d)\n - [Description](#description-8)\n - [Parameters](#parameters-8)\n - [Inputs](#inputs-8)\n - [Outputs](#outputs-8)\n - [Type Constraints](#type-constraints-8)\n\n\n\n### SoftNMS\n\n#### Description\n\nPerform soft NMS on `boxes` with `scores`. Read [Soft-NMS -- Improving Object Detection With One Line of Code](https://arxiv.org/abs/1704.04503) for detail.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|---------|-----------------|----------------------------------------------------------------|\n| `float` | `iou_threshold` | IoU threshold for NMS |\n| `float` | `sigma` | hyperparameter for gaussian method |\n| `float` | `min_score` | score filter threshold |\n| `int` | `method` | method to do the nms, (0: `naive`, 1: `linear`, 2: `gaussian`) |\n| `int` | `offset` | `boxes` width or height is (x2 - x1 + offset). (0 or 1) |\n\n#### Inputs\n\n
\n
boxes: T
\n
Input boxes. 2-D tensor of shape (N, 4). N is the number of boxes.
\n
scores: T
\n
Input scores. 1-D tensor of shape (N, ).
\n
\n\n#### Outputs\n\n
\n
dets: T
\n
Output boxes and scores. 2-D tensor of shape (num_valid_boxes, 5), [[x1, y1, x2, y2, score], ...]. num_valid_boxes is the number of valid boxes.
\n
indices: tensor(int64)
\n
Output indices. 1-D tensor of shape (num_valid_boxes, ).
\n
\n\n#### Type Constraints\n\n- T:tensor(float32)\n\n### RoIAlign\n\n#### Description\n\nPerform RoIAlign on output feature, used in bbox_head of most two-stage detectors.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|---------|------------------|---------------------------------------------------------------------------------------------------------------|\n| `int` | `output_height` | height of output roi |\n| `int` | `output_width` | width of output roi |\n| `float` | `spatial_scale` | used to scale the input boxes |\n| `int` | `sampling_ratio` | number of input samples to take for each output sample. `0` means to take samples densely for current models. |\n| `str` | `mode` | pooling mode in each bin. `avg` or `max` |\n| `int` | `aligned` | If `aligned=0`, use the legacy implementation in MMDetection. Else, align the results more perfectly. |\n\n#### Inputs\n\n
\n
input: T
\n
Input feature map; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
\n
rois: T
\n
RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of input.
\n
\n\n#### Outputs\n\n
\n
feat: T
\n
RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element feat[r-1] is a pooled feature map corresponding to the r-th RoI RoIs[r-1].
\n
\n\n#### Type Constraints\n\n- T:tensor(float32)\n\n### NMS\n\n#### Description\n\nFilter out boxes has high IoU overlap with previously selected boxes.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|---------|-----------------|------------------------------------------------------------------------------------------------------------------|\n| `float` | `iou_threshold` | The threshold for deciding whether boxes overlap too much with respect to IoU. Value range [0, 1]. Default to 0. |\n| `int` | `offset` | 0 or 1, boxes' width or height is (x2 - x1 + offset). |\n\n#### Inputs\n\n
\n
bboxes: T
\n
Input boxes. 2-D tensor of shape (num_boxes, 4). num_boxes is the number of input boxes.
\n
scores: T
\n
Input scores. 1-D tensor of shape (num_boxes, ).
\n
\n\n#### Outputs\n\n
\n
indices: tensor(int32, Linear)
\n
Selected indices. 1-D tensor of shape (num_valid_boxes, ). num_valid_boxes is the number of valid boxes.
\n
\n\n#### Type Constraints\n\n- T:tensor(float32)\n\n### grid_sampler\n\n#### Description\n\nPerform sample from `input` with pixel locations from `grid`.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|-------|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `int` | `interpolation_mode` | Interpolation mode to calculate output values. (0: `bilinear` , 1: `nearest`) |\n| `int` | `padding_mode` | Padding mode for outside grid values. (0: `zeros`, 1: `border`, 2: `reflection`) |\n| `int` | `align_corners` | If `align_corners=1`, the extrema (`-1` and `1`) are considered as referring to the center points of the input's corner pixels. If `align_corners=0`, they are instead considered as referring to the corner points of the input's corner pixels, making the sampling more resolution agnostic. |\n\n#### Inputs\n\n
\n
input: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the numbers of channels, inH and inW are the height and width of the data.
\n
grid: T
\n
Input offset; 4-D tensor of shape (N, outH, outW, 2), where outH and outW is the height and width of offset and output.
\n
\n\n#### Outputs\n\n
\n
output: T
\n
Output feature; 4-D tensor of shape (N, C, outH, outW).
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### CornerPool\n\n#### Description\n\nPerform CornerPool on `input` features. Read [CornerNet -- Detecting Objects as Paired Keypoints](https://arxiv.org/abs/1808.01244) for more details.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|-------|-----------|------------------------------------------------------------------|\n| `int` | `mode` | corner pool mode, (0: `top`, 1: `bottom`, 2: `left`, 3: `right`) |\n\n#### Inputs\n\n
\n
input: T
\n
Input features. 4-D tensor of shape (N, C, H, W). N is the batch size.
\n
\n\n#### Outputs\n\n
\n
output: T
\n
Output the pooled features. 4-D tensor of shape (N, C, H, W).
\n
\n\n#### Type Constraints\n\n- T:tensor(float32)\n\n### cummax\n\n#### Description\n\nReturns a tuple (`values`, `indices`) where `values` is the cumulative maximum elements of `input` in the dimension `dim`. And `indices` is the index location of each maximum value found in the dimension `dim`. Read [torch.cummax](https://pytorch.org/docs/stable/generated/torch.cummax.html) for more details.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|-------|-----------|----------------------------------------|\n| `int` | `dim` | the dimension to do the operation over |\n\n#### Inputs\n\n
\n
input: T
\n
The input tensor with various shapes. Tensor with empty element is also supported.
\n
\n\n#### Outputs\n\n
\n
output: T
\n
Output the cumulative maximum elements of `input` in the dimension `dim`, with the same shape and dtype as `input`.
\n
indices: tensor(int64)
\n
Output the index location of each cumulative maximum value found in the dimension `dim`, with the same shape as `input`.
\n
\n\n#### Type Constraints\n\n- T:tensor(float32)\n\n### cummin\n\n#### Description\n\nReturns a tuple (`values`, `indices`) where `values` is the cumulative minimum elements of `input` in the dimension `dim`. And `indices` is the index location of each minimum value found in the dimension `dim`. Read [torch.cummin](https://pytorch.org/docs/stable/generated/torch.cummin.html) for more details.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|-------|-----------|----------------------------------------|\n| `int` | `dim` | the dimension to do the operation over |\n\n#### Inputs\n\n
\n
input: T
\n
The input tensor with various shapes. Tensor with empty element is also supported.
\n
\n\n#### Outputs\n\n
\n
output: T
\n
Output the cumulative minimum elements of `input` in the dimension `dim`, with the same shape and dtype as `input`.
\n
indices: tensor(int64)
\n
Output the index location of each cumulative minimum value found in the dimension `dim`, with the same shape as `input`.
\n
\n\n#### Type Constraints\n\n- T:tensor(float32)\n\n### MMCVModulatedDeformConv2d\n\n#### Description\n\nPerform Modulated Deformable Convolution on input feature, read [Deformable ConvNets v2: More Deformable, Better Results](https://arxiv.org/abs/1811.11168?from=timeline) for detail.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|----------------|---------------------|---------------------------------------------------------------------------------------|\n| `list of ints` | `stride` | The stride of the convolving kernel. (sH, sW) |\n| `list of ints` | `padding` | Paddings on both sides of the input. (padH, padW) |\n| `list of ints` | `dilation` | The spacing between kernel elements. (dH, dW) |\n| `int` | `deformable_groups` | Groups of deformable offset. |\n| `int` | `groups` | Split input into groups. `input_channel` should be divisible by the number of groups. |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the number of channels, inH and inW are the height and width of the data.
\n
inputs[1]: T
\n
Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
\n
inputs[2]: T
\n
Input mask; 4-D tensor of shape (N, deformable_group* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
\n
inputs[3]: T
\n
Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
\n
inputs[4]: T, optional
\n
Input bias; 1-D tensor of shape (output_channel).
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
Output feature; 4-D tensor of shape (N, output_channel, outH, outW).
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### MMCVDeformConv2d\n\n#### Description\n\nPerform Deformable Convolution on input feature, read [Deformable Convolutional Network](https://arxiv.org/abs/1703.06211) for detail.\n\n#### Parameters\n\n| Type | Parameter | Description |\n|----------------|--------------------|-----------------------------------------------------------------------------------------------------------------------------------|\n| `list of ints` | `stride` | The stride of the convolving kernel. (sH, sW) |\n| `list of ints` | `padding` | Paddings on both sides of the input. (padH, padW) |\n| `list of ints` | `dilation` | The spacing between kernel elements. (dH, dW) |\n| `int` | `deformable_group` | Groups of deformable offset. |\n| `int` | `group` | Split input into groups. `input_channel` should be divisible by the number of groups. |\n| `int` | `im2col_step` | DeformableConv2d use im2col to compute convolution. im2col_step is used to split input and offset, reduce memory usage of column. |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the numbers of channels, inH and inW are the height and width of the data.
\n
inputs[1]: T
\n
Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
\n
inputs[2]: T
\n
Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
Output feature; 4-D tensor of shape (N, output_channel, outH, outW).
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/deployment/onnxruntime_op.md", "content": "## ONNX Runtime Deployment\n\n### Introduction of ONNX Runtime\n\n**ONNX Runtime** is a cross-platform inferencing and training accelerator compatible with many popular ML/DNN frameworks. Check its [github](https://github.com/microsoft/onnxruntime) for more information.\n\n### Introduction of ONNX\n\n**ONNX** stands for **Open Neural Network Exchange**, which acts as *Intermediate Representation(IR)* for ML/DNN models from many frameworks. Check its [github](https://github.com/onnx/onnx) for more information.\n\n### Why include custom operators for ONNX Runtime in MMCV\n\n- To verify the correctness of exported ONNX models in ONNX Runtime.\n- To ease the deployment of ONNX models with custom operators from `mmcv.ops` in ONNX Runtime.\n\n### List of operators for ONNX Runtime supported in MMCV\n\n| Operator | CPU | GPU | MMCV Releases |\n|:-------------------------------------------------------|:---:|:---:|:-------------:|\n| [SoftNMS](onnxruntime_custom_ops.md#softnms) | Y | N | 1.2.3 |\n| [RoIAlign](onnxruntime_custom_ops.md#roialign) | Y | N | 1.2.5 |\n| [NMS](onnxruntime_custom_ops.md#nms) | Y | N | 1.2.7 |\n| [grid_sampler](onnxruntime_custom_ops.md#grid_sampler) | Y | N | 1.3.1 |\n| [CornerPool](onnxruntime_custom_ops.md#cornerpool) | Y | N | 1.3.4 |\n| [cummax](onnxruntime_custom_ops.md#cummax) | Y | N | 1.3.4 |\n| [cummin](onnxruntime_custom_ops.md#cummin) | Y | N | 1.3.4 |\n\n### How to build custom operators for ONNX Runtime\n\n*Please be noted that only **onnxruntime>=1.8.1** of CPU version on Linux platform is tested by now.*\n\n#### Prerequisite\n\n- Clone repository\n\n```bash\ngit clone https://github.com/open-mmlab/mmcv.git\n```\n\n- Download `onnxruntime-linux` from ONNX Runtime [releases](https://github.com/microsoft/onnxruntime/releases/tag/v1.8.1), extract it, expose `ONNXRUNTIME_DIR` and finally add the lib path to `LD_LIBRARY_PATH` as below:\n\n```bash\nwget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-1.8.1.tgz\n\ntar -zxvf onnxruntime-linux-x64-1.8.1.tgz\ncd onnxruntime-linux-x64-1.8.1\nexport ONNXRUNTIME_DIR=$(pwd)\nexport LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH\n```\n\n#### Build on Linux\n\n```bash\ncd mmcv ## to MMCV root directory\nMMCV_WITH_OPS=1 MMCV_WITH_ORT=1 python setup.py develop\n```\n\n### How to do inference using exported ONNX models with custom operators in ONNX Runtime in python\n\nInstall ONNX Runtime with `pip`\n\n```bash\npip install onnxruntime==1.8.1\n```\n\nInference Demo\n\n```python\nimport os\n\nimport numpy as np\nimport onnxruntime as ort\n\nfrom mmcv.ops import get_onnxruntime_op_path\n\nort_custom_op_path = get_onnxruntime_op_path()\nassert os.path.exists(ort_custom_op_path)\nsession_options = ort.SessionOptions()\nsession_options.register_custom_ops_library(ort_custom_op_path)\n## exported ONNX model with custom operators\nonnx_file = 'sample.onnx'\ninput_data = np.random.randn(1, 3, 224, 224).astype(np.float32)\nsess = ort.InferenceSession(onnx_file, session_options)\nonnx_results = sess.run(None, {'input' : input_data})\n```\n\n### How to add a new custom operator for ONNX Runtime in MMCV\n\n#### Reminder\n\n- *Please note that this feature is experimental and may change in the future. Strongly suggest users always try with the latest master branch.*\n\n- The custom operator is not included in [supported operator list](https://github.com/microsoft/onnxruntime/blob/master/docs/OperatorKernels.md) in ONNX Runtime.\n- The custom operator should be able to be exported to ONNX.\n\n#### Main procedures\n\nTake custom operator `soft_nms` for example.\n\n1. Add header `soft_nms.h` to ONNX Runtime include directory `mmcv/ops/csrc/onnxruntime/`\n2. Add source `soft_nms.cpp` to ONNX Runtime source directory `mmcv/ops/csrc/onnxruntime/cpu/`\n3. Register `soft_nms` operator in [onnxruntime_register.cpp](../../mmcv/ops/csrc/onnxruntime/cpu/onnxruntime_register.cpp)\n\n ```c++\n #include \"soft_nms.h\"\n\n SoftNmsOp c_SoftNmsOp;\n\n if (auto status = ortApi->CustomOpDomain_Add(domain, &c_SoftNmsOp)) {\n return status;\n }\n ```\n\n4. Add unit test into `tests/test_ops/test_onnx.py`\n Check [here](../../tests/test_ops/test_onnx.py) for examples.\n\n**Finally, welcome to send us PR of adding custom operators for ONNX Runtime in MMCV.** :nerd_face:\n\n### Known Issues\n\n- \"RuntimeError: tuple appears in op that does not forward tuples, unsupported kind: `prim::PythonOp`.\"\n 1. Note generally `cummax` or `cummin` is exportable to ONNX as long as the torch version >= 1.5.0, since `torch.cummax` is only supported with torch >= 1.5.0. But when `cummax` or `cummin` serves as an intermediate component whose outputs is used as inputs for another modules, it's expected that torch version must be >= 1.7.0. Otherwise the above error might arise, when running exported ONNX model with onnxruntime.\n 2. Solution: update the torch version to 1.7.0 or higher.\n\n### References\n\n- [How to export Pytorch model with custom op to ONNX and run it in ONNX Runtime](https://github.com/onnx/tutorials/blob/master/PyTorchCustomOperator/README.md)\n- [How to add a custom operator/kernel in ONNX Runtime](https://github.com/microsoft/onnxruntime/blob/master/docs/AddingCustomOp.md)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/deployment/tensorrt_custom_ops.md", "content": "## TensorRT Custom Ops\n\n\n\n- [TensorRT Custom Ops](#tensorrt-custom-ops)\n - [MMCVRoIAlign](#mmcvroialign)\n - [Description](#description)\n - [Parameters](#parameters)\n - [Inputs](#inputs)\n - [Outputs](#outputs)\n - [Type Constraints](#type-constraints)\n - [ScatterND](#scatternd)\n - [Description](#description-1)\n - [Parameters](#parameters-1)\n - [Inputs](#inputs-1)\n - [Outputs](#outputs-1)\n - [Type Constraints](#type-constraints-1)\n - [NonMaxSuppression](#nonmaxsuppression)\n - [Description](#description-2)\n - [Parameters](#parameters-2)\n - [Inputs](#inputs-2)\n - [Outputs](#outputs-2)\n - [Type Constraints](#type-constraints-2)\n - [MMCVDeformConv2d](#mmcvdeformconv2d)\n - [Description](#description-3)\n - [Parameters](#parameters-3)\n - [Inputs](#inputs-3)\n - [Outputs](#outputs-3)\n - [Type Constraints](#type-constraints-3)\n - [grid_sampler](#grid_sampler)\n - [Description](#description-4)\n - [Parameters](#parameters-4)\n - [Inputs](#inputs-4)\n - [Outputs](#outputs-4)\n - [Type Constraints](#type-constraints-4)\n - [cummax](#cummax)\n - [Description](#description-5)\n - [Parameters](#parameters-5)\n - [Inputs](#inputs-5)\n - [Outputs](#outputs-5)\n - [Type Constraints](#type-constraints-5)\n - [cummin](#cummin)\n - [Description](#description-6)\n - [Parameters](#parameters-6)\n - [Inputs](#inputs-6)\n - [Outputs](#outputs-6)\n - [Type Constraints](#type-constraints-6)\n - [MMCVInstanceNormalization](#mmcvinstancenormalization)\n - [Description](#description-7)\n - [Parameters](#parameters-7)\n - [Inputs](#inputs-7)\n - [Outputs](#outputs-7)\n - [Type Constraints](#type-constraints-7)\n - [MMCVModulatedDeformConv2d](#mmcvmodulateddeformconv2d)\n - [Description](#description-8)\n - [Parameters](#parameters-8)\n - [Inputs](#inputs-8)\n - [Outputs](#outputs-8)\n - [Type Constraints](#type-constraints-8)\n\n\n\n### MMCVRoIAlign\n\n#### Description\n\nPerform RoIAlign on output feature, used in bbox_head of most two stage\ndetectors.\n\n#### Parameters\n\n| Type | Parameter | Description |\n| ------- | ---------------- | ------------------------------------------------------------------------------------------------------------- |\n| `int` | `output_height` | height of output roi |\n| `int` | `output_width` | width of output roi |\n| `float` | `spatial_scale` | used to scale the input boxes |\n| `int` | `sampling_ratio` | number of input samples to take for each output sample. `0` means to take samples densely for current models. |\n| `str` | `mode` | pooling mode in each bin. `avg` or `max` |\n| `int` | `aligned` | If `aligned=0`, use the legacy implementation in MMDetection. Else, align the results more perfectly. |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
Input feature map; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
\n
inputs[1]: T
\n
RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of inputs[0].
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element output[0][r-1] is a pooled feature map corresponding to the r-th RoI inputs[1][r-1].
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### ScatterND\n\n#### Description\n\nScatterND takes three inputs `data` tensor of rank r >= 1, `indices` tensor of rank q >= 1, and `updates` tensor of rank q + r - indices.shape[-1] - 1. The output of the operation is produced by creating a copy of the input `data`, and then updating its value to values specified by updates at specific index positions specified by `indices`. Its output shape is the same as the shape of `data`. Note that `indices` should not have duplicate entries. That is, two or more updates for the same index-location is not supported.\n\nThe `output` is calculated via the following equation:\n\n```python\n output = np.copy(data)\n update_indices = indices.shape[:-1]\n for idx in np.ndindex(update_indices):\n output[indices[idx]] = updates[idx]\n```\n\n#### Parameters\n\nNone\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
Tensor of rank r>=1.
\n\n
inputs[1]: tensor(int32, Linear)
\n
Tensor of rank q>=1.
\n\n
inputs[2]: T
\n
Tensor of rank q + r - indices_shape[-1] - 1.
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
Tensor of rank r >= 1.
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear), tensor(int32, Linear)\n\n### NonMaxSuppression\n\n#### Description\n\nFilter out boxes has high IoU overlap with previously selected boxes or low score. Output the indices of valid boxes. Indices of invalid boxes will be filled with -1.\n\n#### Parameters\n\n| Type | Parameter | Description |\n| ------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |\n| `int` | `center_point_box` | 0 - the box data is supplied as [y1, x1, y2, x2], 1-the box data is supplied as [x_center, y_center, width, height]. |\n| `int` | `max_output_boxes_per_class` | The maximum number of boxes to be selected per batch per class. Default to 0, number of output boxes equal to number of input boxes. |\n| `float` | `iou_threshold` | The threshold for deciding whether boxes overlap too much with respect to IoU. Value range [0, 1]. Default to 0. |\n| `float` | `score_threshold` | The threshold for deciding when to remove boxes based on score. |\n| `int` | `offset` | 0 or 1, boxes' width or height is (x2 - x1 + offset). |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
Input boxes. 3-D tensor of shape (num_batches, spatial_dimension, 4).
\n
inputs[1]: T
\n
Input scores. 3-D tensor of shape (num_batches, num_classes, spatial_dimension).
\n
\n\n#### Outputs\n\n
\n
outputs[0]: tensor(int32, Linear)
\n
Selected indices. 2-D tensor of shape (num_selected_indices, 3) as [[batch_index, class_index, box_index], ...].
\n
num_selected_indices=num_batches* num_classes* min(max_output_boxes_per_class, spatial_dimension).
\n
All invalid indices will be filled with -1.
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### MMCVDeformConv2d\n\n#### Description\n\nPerform Deformable Convolution on input feature, read [Deformable Convolutional Network](https://arxiv.org/abs/1703.06211) for detail.\n\n#### Parameters\n\n| Type | Parameter | Description |\n| -------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------- |\n| `list of ints` | `stride` | The stride of the convolving kernel. (sH, sW) |\n| `list of ints` | `padding` | Paddings on both sides of the input. (padH, padW) |\n| `list of ints` | `dilation` | The spacing between kernel elements. (dH, dW) |\n| `int` | `deformable_group` | Groups of deformable offset. |\n| `int` | `group` | Split input into groups. `input_channel` should be divisible by the number of groups. |\n| `int` | `im2col_step` | DeformableConv2d use im2col to compute convolution. im2col_step is used to split input and offset, reduce memory usage of column. |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the numbers of channels, inH and inW are the height and width of the data.
\n
inputs[1]: T
\n
Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
\n
inputs[2]: T
\n
Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
Output feature; 4-D tensor of shape (N, output_channel, outH, outW).
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### grid_sampler\n\n#### Description\n\nPerform sample from `input` with pixel locations from `grid`.\n\n#### Parameters\n\n| Type | Parameter | Description |\n| ----- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `int` | `interpolation_mode` | Interpolation mode to calculate output values. (0: `bilinear` , 1: `nearest`) |\n| `int` | `padding_mode` | Padding mode for outside grid values. (0: `zeros`, 1: `border`, 2: `reflection`) |\n| `int` | `align_corners` | If `align_corners=1`, the extrema (`-1` and `1`) are considered as referring to the center points of the input's corner pixels. If `align_corners=0`, they are instead considered as referring to the corner points of the input's corner pixels, making the sampling more resolution agnostic. |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the numbers of channels, inH and inW are the height and width of the data.
\n
inputs[1]: T
\n
Input offset; 4-D tensor of shape (N, outH, outW, 2), where outH and outW is the height and width of offset and output.
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
Output feature; 4-D tensor of shape (N, C, outH, outW).
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### cummax\n\n#### Description\n\nReturns a namedtuple (`values`, `indices`) where `values` is the cumulative maximum of elements of `input` in the dimension `dim`. And `indices` is the index location of each maximum value found in the dimension `dim`.\n\n#### Parameters\n\n| Type | Parameter | Description |\n| ----- | --------- | --------------------------------------- |\n| `int` | `dim` | The dimension to do the operation over. |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
The input tensor.
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
Output values.
\n
outputs[1]: (int32, Linear)
\n
Output indices.
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### cummin\n\n#### Description\n\nReturns a namedtuple (`values`, `indices`) where `values` is the cumulative minimum of elements of `input` in the dimension `dim`. And `indices` is the index location of each minimum value found in the dimension `dim`.\n\n#### Parameters\n\n| Type | Parameter | Description |\n| ----- | --------- | --------------------------------------- |\n| `int` | `dim` | The dimension to do the operation over. |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
The input tensor.
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
Output values.
\n
outputs[1]: (int32, Linear)
\n
Output indices.
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### MMCVInstanceNormalization\n\n#### Description\n\nCarries out instance normalization as described in the paper https://arxiv.org/abs/1607.08022.\n\ny = scale * (x - mean) / sqrt(variance + epsilon) + B, where mean and variance are computed per instance per channel.\n\n#### Parameters\n\n| Type | Parameter | Description |\n| ------- | --------- | -------------------------------------------------------------------- |\n| `float` | `epsilon` | The epsilon value to use to avoid division by zero. Default is 1e-05 |\n\n#### Inputs\n\n
\n
input: T
\n
Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 ... Dn), where N is the batch size.
\n
scale: T
\n
The input 1-dimensional scale tensor of size C.
\n
B: T
\n
The input 1-dimensional bias tensor of size C.
\n
\n\n#### Outputs\n\n
\n
output: T
\n
The output tensor of the same shape as input.
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n\n### MMCVModulatedDeformConv2d\n\n#### Description\n\nPerform Modulated Deformable Convolution on input feature, read [Deformable ConvNets v2: More Deformable, Better Results](https://arxiv.org/abs/1811.11168?from=timeline) for detail.\n\n#### Parameters\n\n| Type | Parameter | Description |\n| -------------- | ------------------ | ------------------------------------------------------------------------------------- |\n| `list of ints` | `stride` | The stride of the convolving kernel. (sH, sW) |\n| `list of ints` | `padding` | Paddings on both sides of the input. (padH, padW) |\n| `list of ints` | `dilation` | The spacing between kernel elements. (dH, dW) |\n| `int` | `deformable_group` | Groups of deformable offset. |\n| `int` | `group` | Split input into groups. `input_channel` should be divisible by the number of groups. |\n\n#### Inputs\n\n
\n
inputs[0]: T
\n
Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the number of channels, inH and inW are the height and width of the data.
\n
inputs[1]: T
\n
Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
\n
inputs[2]: T
\n
Input mask; 4-D tensor of shape (N, deformable_group* kH* kW, outH, outW), where kH and kW is the height and width of weight, outH and outW is the height and width of offset and output.
\n
inputs[3]: T
\n
Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
\n
inputs[4]: T, optional
\n
Input weight; 1-D tensor of shape (output_channel).
\n
\n\n#### Outputs\n\n
\n
outputs[0]: T
\n
Output feature; 4-D tensor of shape (N, output_channel, outH, outW).
\n
\n\n#### Type Constraints\n\n- T:tensor(float32, Linear)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/deployment/tensorrt_plugin.md", "content": "## TensorRT Deployment\n\n\n\n- [TensorRT Deployment](#tensorrt-deployment)\n - [Introduction](#introduction)\n - [List of TensorRT plugins supported in MMCV](#list-of-tensorrt-plugins-supported-in-mmcv)\n - [How to build TensorRT plugins in MMCV](#how-to-build-tensorrt-plugins-in-mmcv)\n - [Prerequisite](#prerequisite)\n - [Build on Linux](#build-on-linux)\n - [Create TensorRT engine and run inference in python](#create-tensorrt-engine-and-run-inference-in-python)\n - [How to add a TensorRT plugin for custom op in MMCV](#how-to-add-a-tensorrt-plugin-for-custom-op-in-mmcv)\n - [Main procedures](#main-procedures)\n - [Reminders](#reminders)\n - [Known Issues](#known-issues)\n - [References](#references)\n\n\n\n### Introduction\n\n**NVIDIA TensorRT** is a software development kit(SDK) for high-performance inference of deep learning models. It includes a deep learning inference optimizer and runtime that delivers low latency and high-throughput for deep learning inference applications. Please check its [developer's website](https://developer.nvidia.com/tensorrt) for more information.\nTo ease the deployment of trained models with custom operators from `mmcv.ops` using TensorRT, a series of TensorRT plugins are included in MMCV.\n\n### List of TensorRT plugins supported in MMCV\n\n| ONNX Operator | TensorRT Plugin | MMCV Releases |\n|:--------------------------|:--------------------------------------------------------------------------------|:-------------:|\n| MMCVRoiAlign | [MMCVRoiAlign](./tensorrt_custom_ops.md#mmcvroialign) | 1.2.6 |\n| ScatterND | [ScatterND](./tensorrt_custom_ops.md#scatternd) | 1.2.6 |\n| NonMaxSuppression | [NonMaxSuppression](./tensorrt_custom_ops.md#nonmaxsuppression) | 1.3.0 |\n| MMCVDeformConv2d | [MMCVDeformConv2d](./tensorrt_custom_ops.md#mmcvdeformconv2d) | 1.3.0 |\n| grid_sampler | [grid_sampler](./tensorrt_custom_ops.md#grid-sampler) | 1.3.1 |\n| cummax | [cummax](./tensorrt_custom_ops.md#cummax) | 1.3.5 |\n| cummin | [cummin](./tensorrt_custom_ops.md#cummin) | 1.3.5 |\n| MMCVInstanceNormalization | [MMCVInstanceNormalization](./tensorrt_custom_ops.md#mmcvinstancenormalization) | 1.3.5 |\n| MMCVModulatedDeformConv2d | [MMCVModulatedDeformConv2d](./tensorrt_custom_ops.md#mmcvmodulateddeformconv2d) | 1.3.8 |\n\nNotes\n\n- All plugins listed above are developed on TensorRT-7.2.1.6.Ubuntu-16.04.x86_64-gnu.cuda-10.2.cudnn8.0\n\n### How to build TensorRT plugins in MMCV\n\n#### Prerequisite\n\n- Clone repository\n\n```bash\ngit clone https://github.com/open-mmlab/mmcv.git\n```\n\n- Install TensorRT\n\nDownload the corresponding TensorRT build from [NVIDIA Developer Zone](https://developer.nvidia.com/nvidia-tensorrt-download).\n\nFor example, for Ubuntu 16.04 on x86-64 with cuda-10.2, the downloaded file is `TensorRT-7.2.1.6.Ubuntu-16.04.x86_64-gnu.cuda-10.2.cudnn8.0.tar.gz`.\n\nThen, install as below:\n\n```bash\ncd ~/Downloads\ntar -xvzf TensorRT-7.2.1.6.Ubuntu-16.04.x86_64-gnu.cuda-10.2.cudnn8.0.tar.gz\nexport TENSORRT_DIR=`pwd`/TensorRT-7.2.1.6\nexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TENSORRT_DIR/lib\n```\n\nInstall python packages: tensorrt, graphsurgeon, onnx-graphsurgeon\n\n```bash\npip install $TENSORRT_DIR/python/tensorrt-7.2.1.6-cp37-none-linux_x86_64.whl\npip install $TENSORRT_DIR/onnx_graphsurgeon/onnx_graphsurgeon-0.2.6-py2.py3-none-any.whl\npip install $TENSORRT_DIR/graphsurgeon/graphsurgeon-0.4.5-py2.py3-none-any.whl\n```\n\nFor more detailed information of installing TensorRT using tar, please refer to [Nvidia' website](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-721/install-guide/index.html#installing-tar).\n\n#### Build on Linux\n\n```bash\ncd mmcv ## to MMCV root directory\nMMCV_WITH_OPS=1 MMCV_WITH_TRT=1 pip install -e .\n```\n\n### Create TensorRT engine and run inference in python\n\nHere is an example.\n\n```python\nimport torch\nimport onnx\n\nfrom mmcv.tensorrt import (TRTWrapper, onnx2trt, save_trt_engine,\n is_tensorrt_plugin_loaded)\n\nassert is_tensorrt_plugin_loaded(), 'Requires to complie TensorRT plugins in mmcv'\n\nonnx_file = 'sample.onnx'\ntrt_file = 'sample.trt'\nonnx_model = onnx.load(onnx_file)\n\n## Model input\ninputs = torch.rand(1, 3, 224, 224).cuda()\n## Model input shape info\nopt_shape_dict = {\n 'input': [list(inputs.shape),\n list(inputs.shape),\n list(inputs.shape)]\n}\n\n## Create TensorRT engine\nmax_workspace_size = 1 << 30\ntrt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n max_workspace_size=max_workspace_size)\n\n## Save TensorRT engine\nsave_trt_engine(trt_engine, trt_file)\n\n## Run inference with TensorRT\ntrt_model = TRTWrapper(trt_file, ['input'], ['output'])\n\nwith torch.no_grad():\n trt_outputs = trt_model({'input': inputs})\n output = trt_outputs['output']\n\n```\n\n### How to add a TensorRT plugin for custom op in MMCV\n\n#### Main procedures\n\nBelow are the main steps:\n\n1. Add c++ header file\n2. Add c++ source file\n3. Add cuda kernel file\n4. Register plugin in `trt_plugin.cpp`\n5. Add unit test in `tests/test_ops/test_tensorrt.py`\n\n**Take RoIAlign plugin `roi_align` for example.**\n\n1. Add header `trt_roi_align.hpp` to TensorRT include directory `mmcv/ops/csrc/tensorrt/`\n2. Add source `trt_roi_align.cpp` to TensorRT source directory `mmcv/ops/csrc/tensorrt/plugins/`\n3. Add cuda kernel `trt_roi_align_kernel.cu` to TensorRT source directory `mmcv/ops/csrc/tensorrt/plugins/`\n4. Register `roi_align` plugin in [trt_plugin.cpp](https://github.com/open-mmlab/mmcv/blob/master/mmcv/ops/csrc/tensorrt/plugins/trt_plugin.cpp)\n\n ```c++\n #include \"trt_plugin.hpp\"\n\n #include \"trt_roi_align.hpp\"\n\n REGISTER_TENSORRT_PLUGIN(RoIAlignPluginDynamicCreator);\n\n extern \"C\" {\n bool initLibMMCVInferPlugins() { return true; }\n } // extern \"C\"\n ```\n\n5. Add unit test into `tests/test_ops/test_tensorrt.py`\n Check [here](https://github.com/open-mmlab/mmcv/blob/master/tests/test_ops/test_tensorrt.py) for examples.\n\n#### Reminders\n\n- *Please note that this feature is experimental and may change in the future. Strongly suggest users always try with the latest master branch.*\n\n- Some of the [custom ops](https://mmcv.readthedocs.io/en/latest/ops.html) in `mmcv` have their cuda implementations, which could be referred.\n\n### Known Issues\n\n- None\n\n### References\n\n- [Developer guide of Nvidia TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html)\n- [TensorRT Open Source Software](https://github.com/NVIDIA/TensorRT)\n- [onnx-tensorrt](https://github.com/onnx/onnx-tensorrt)\n- [TensorRT python API](https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/index.html)\n- [TensorRT c++ plugin API](https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_plugin.html)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/faq.md", "content": "## Frequently Asked Questions\n\nWe list some common troubles faced by many users and their corresponding solutions here.\nFeel free to enrich the list if you find any frequent issues and have ways to help others to solve them.\n\n### Installation\n\n- KeyError: \"xxx: 'yyy is not in the zzz registry'\"\n\n The registry mechanism will be triggered only when the file of the module is imported.\n So you need to import that file somewhere. More details can be found at https://github.com/open-mmlab/mmdetection/issues/5974.\n\n- \"No module named 'mmcv.ops'\"; \"No module named 'mmcv._ext'\"\n\n 1. Uninstall existing mmcv in the environment using `pip uninstall mmcv`\n 2. Install mmcv-full following the [installation instruction](https://mmcv.readthedocs.io/en/latest/get_started/installation.html) or [Build MMCV from source](https://mmcv.readthedocs.io/en/latest/get_started/build.html)\n\n- \"invalid device function\" or \"no kernel image is available for execution\"\n\n 1. Check the CUDA compute capability of you GPU\n 2. Run `python mmdet/utils/collect_env.py` to check whether PyTorch, torchvision, and MMCV are built for the correct GPU architecture. You may need to set `TORCH_CUDA_ARCH_LIST` to reinstall MMCV. The compatibility issue could happen when using old GPUS, e.g., Tesla K80 (3.7) on colab.\n 3. Check whether the running environment is the same as that when mmcv/mmdet is compiled. For example, you may compile mmcv using CUDA 10.0 bug run it on CUDA9.0 environments\n\n- \"undefined symbol\" or \"cannot open xxx.so\"\n\n 1. If those symbols are CUDA/C++ symbols (e.g., libcudart.so or GLIBCXX), check\n whether the CUDA/GCC runtimes are the same as those used for compiling mmcv\n 2. If those symbols are Pytorch symbols (e.g., symbols containing caffe, aten, and TH), check whether the Pytorch version is the same as that used for compiling mmcv\n 3. Run `python mmdet/utils/collect_env.py` to check whether PyTorch, torchvision, and MMCV are built by and running on the same environment\n\n- \"RuntimeError: CUDA error: invalid configuration argument\"\n\n This error may be caused by the poor performance of GPU. Try to decrease the value of [THREADS_PER_BLOCK](https://github.com/open-mmlab/mmcv/blob/cac22f8cf5a904477e3b5461b1cc36856c2793da/mmcv/ops/csrc/common_cuda_helper.hpp#L10)\n and recompile mmcv.\n\n- \"RuntimeError: nms is not compiled with GPU support\"\n\n This error is because your CUDA environment is not installed correctly.\n You may try to re-install your CUDA environment and then delete the build/ folder before re-compile mmcv.\n\n- \"Segmentation fault\"\n\n 1. Check your GCC version and use GCC >= 5.4. This usually caused by the incompatibility between PyTorch and the environment (e.g., GCC < 4.9 for PyTorch). We also recommend the users to avoid using GCC 5.5 because many feedbacks report that GCC 5.5 will cause \"segmentation fault\" and simply changing it to GCC 5.4 could solve the problem\n 2. Check whether PyTorch is correctly installed and could use CUDA op, e.g. type the following command in your terminal and see whether they could correctly output results\n ```shell\n python -c 'import torch; print(torch.cuda.is_available())'\n ```\n 3. If PyTorch is correctly installed, check whether MMCV is correctly installed. If MMCV is correctly installed, then there will be no issue of the command\n ```shell\n python -c 'import mmcv; import mmcv.ops'\n ```\n 4. If MMCV and PyTorch are correctly installed, you can use `ipdb` to set breakpoints or directly add `print` to debug and see which part leads the `segmentation fault`\n\n- \"libtorch_cuda_cu.so: cannot open shared object file\"\n\n `mmcv-full` depends on the share object but it can not be found. We can check whether the object exists in `~/miniconda3/envs/{environment-name}/lib/python3.7/site-packages/torch/lib` or try to re-install the PyTorch.\n\n- \"fatal error C1189: #error: -- unsupported Microsoft Visual Studio version!\"\n\n If you are building mmcv-full on Windows and the version of CUDA is 9.2, you will probably encounter the error `\"C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v9.2\\include\\crt/host_config.h(133): fatal error C1189: #error: -- unsupported Microsoft Visual Studio version! Only the versions 2012, 2013, 2015 and 2017 are supported!\"`, in which case you can use a lower version of Microsoft Visual Studio like vs2017.\n\n- \"error: member \"torch::jit::detail::ModulePolicy::all_slots\" may not be initialized\"\n\n If your version of PyTorch is 1.5.0 and you are building mmcv-full on Windows, you will probably encounter the error `- torch/csrc/jit/api/module.h(474): error: member \"torch::jit::detail::ModulePolicy::all_slots\" may not be initialized`. The way to solve the error is to replace all the `static constexpr bool all_slots = false;` with `static bool all_slots = false;` at this file `https://github.com/pytorch/pytorch/blob/v1.5.0/torch/csrc/jit/api/module.h`. More details can be found at https://github.com/pytorch/pytorch/issues/39394.\n\n- \"error: a member with an in-class initializer must be const\"\n\n If your version of PyTorch is 1.6.0 and you are building mmcv-full on Windows, you will probably encounter the error `\"- torch/include\\torch/csrc/jit/api/module.h(483): error: a member with an in-class initializer must be const\"`. The way to solve the error is to replace all the `CONSTEXPR_EXCEPT_WIN_CUDA ` with `const` at `torch/include\\torch/csrc/jit/api/module.h`. More details can be found at https://github.com/open-mmlab/mmcv/issues/575.\n\n- \"error: member \"torch::jit::ProfileOptionalOp::Kind\" may not be initialized\"\n\n If your version of PyTorch is 1.7.0 and you are building mmcv-full on Windows, you will probably encounter the error `torch/include\\torch/csrc/jit/ir/ir.h(1347): error: member \"torch::jit::ProfileOptionalOp::Kind\" may not be initialized`. The way to solve the error needs to modify several local files of PyTorch:\n\n - delete `static constexpr Symbol Kind = ::c10::prim::profile;` and `tatic constexpr Symbol Kind = ::c10::prim::profile_optional;` at `torch/include\\torch/csrc/jit/ir/ir.h`\n - replace `explicit operator type&() { return *(this->value); }` with `explicit operator type&() { return *((type*)this->value); }` at `torch\\include\\pybind11\\cast.h`\n - replace all the `CONSTEXPR_EXCEPT_WIN_CUDA` with `const` at `torch/include\\torch/csrc/jit/api/module.h`\n\n- Compatibility issue between MMCV and MMDetection; \"ConvWS is already registered in conv layer\"\n\n Please install the correct version of MMCV for the version of your MMDetection following the [installation instruction](https://mmdetection.readthedocs.io/en/latest/get_started.html#installation). More details can be found at https://github.com/pytorch/pytorch/pull/45956.\n\n### Usage\n\n- \"RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one\"\n\n 1. This error indicates that your module has parameters that were not used in producing loss. This phenomenon may be caused by running different branches in your code in DDP mode. More datails at https://github.com/pytorch/pytorch/issues/55582\n 2. You can set ` find_unused_parameters = True` in the config to solve the above problems or find those unused parameters manually\n\n- \"RuntimeError: Trying to backward through the graph a second time\"\n\n `GradientCumulativeOptimizerHook` and `OptimizerHook` are both set which causes the `loss.backward()` to be called twice so `RuntimeError` was raised. We can only use one of these. More datails at https://github.com/open-mmlab/mmcv/issues/1379.\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/get_started/build.md", "content": "## Build MMCV from source\n\n### Build on Linux or macOS\n\nAfter cloning the repo with\n\n```bash\ngit clone https://github.com/open-mmlab/mmcv.git\ncd mmcv\n```\n\nIt is recommended to install `ninja` to speed up the compilation\n\n```bash\npip install -r requirements/optional.txt\n```\n\nYou can either\n\n- install the lite version\n\n ```bash\n pip install -e .\n ```\n\n- install the full version\n\n ```bash\n MMCV_WITH_OPS=1 pip install -e .\n ```\n\nIf you are on macOS, add the following environment variables before the installing command.\n\n```bash\nCC=clang CXX=clang++ CFLAGS='-stdlib=libc++'\n```\n\ne.g.,\n\n```bash\nCC=clang CXX=clang++ CFLAGS='-stdlib=libc++' MMCV_WITH_OPS=1 pip install -e .\n```\n\n```{note}\nIf you would like to use `opencv-python-headless` instead of `opencv-python`,\ne.g., in a minimum container environment or servers without GUI,\nyou can first install it before installing MMCV to skip the installation of `opencv-python`.\n```\n### Build on Windows\n\nBuilding MMCV on Windows is a bit more complicated than that on Linux.\nThe following instructions show how to get this accomplished.\n\n#### Prerequisite\n\nThe following software is required for building MMCV on windows.\nInstall them first.\n\n- [Git](https://git-scm.com/download/win)\n - During installation, tick **add git to Path**.\n- [Visual Studio Community 2019](https://visualstudio.microsoft.com)\n - A compiler for C++ and CUDA codes.\n- [Miniconda](https://docs.conda.io/en/latest/miniconda.html)\n - Official distributions of Python should work too.\n- [CUDA 10.2](https://developer.nvidia.com/cuda-10.2-download-archive)\n - Not required for building CPU version.\n - Customize the installation if necessary. As a recommendation, skip the driver installation if a newer version is already installed.\n\n```{note}\nYou should know how to set up environment variables, especially `Path`, on Windows. The following instruction relies heavily on this skill.\n```\n\n#### Setup Python Environment\n\n1. Launch Anaconda prompt from Windows Start menu\n\n Do not use raw `cmd.exe` s instruction is based on PowerShell syntax.\n\n2. Create a new conda environment\n\n ```shell\n conda create --name mmcv python=3.7 # 3.6, 3.7, 3.8 should work too as tested\n conda activate mmcv # make sure to activate environment before any operation\n ```\n\n3. Install PyTorch. Choose a version based on your need.\n\n ```shell\n conda install pytorch torchvision cudatoolkit=10.2 -c pytorch\n ```\n\n We only tested PyTorch version >= 1.6.0.\n\n4. Prepare MMCV source code\n\n ```shell\n git clone https://github.com/open-mmlab/mmcv.git\n cd mmcv\n ```\n\n5. Install required Python packages\n\n ```shell\n pip3 install -r requirements/runtime.txt\n ```\n\n6. It is recommended to install `ninja` to speed up the compilation\n\n ```bash\n pip install -r requirements/optional.txt\n ```\n\n#### Build and install MMCV\n\nMMCV can be built in three ways:\n\n1. Lite version (without ops)\n\n In this way, no custom ops are compiled and mmcv is a pure python package.\n\n2. Full version (CPU ops)\n\n Module `ops` will be compiled as a pytorch extension, but only x86 code will be compiled. The compiled ops can be executed on CPU only.\n\n3. Full version (CUDA ops)\n\n Both x86 and CUDA codes of `ops` module will be compiled. The compiled version can be run on both CPU and CUDA-enabled GPU (if implemented).\n\n##### Common steps\n\n1. Set up MSVC compiler\n\n Set Environment variable, add `C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\Hostx86\\x64` to `PATH`, so that `cl.exe` will be available in prompt, as shown below.\n\n ```none\n (base) PS C:\\Users\\xxx> cl\n Microsoft (R) C/C++ Optimizing Compiler Version 19.27.29111 for x64\n Copyright (C) Microsoft Corporation. All rights reserved.\n\n usage: cl [ option... ] filename... [ / link linkoption... ]\n ```\n\n For compatibility, we use the x86-hosted and x64-targeted compiler. note `Hostx86\\x64` in the path.\n\n You may want to change the system language to English because pytorch will parse text output from `cl.exe` to check its version. However only utf-8 is recognized. Navigate to Control Panel -> Region -> Administrative -> Language for Non-Unicode programs and change it to English.\n\n##### Option 1: Build MMCV (lite version)\n\nAfter finishing above common steps, launch Anaconda shell from Start menu and issue the following commands:\n\n```shell\n# activate environment\nconda activate mmcv\n# change directory\ncd mmcv\n# install\npython setup.py develop\n# check\npip list\n```\n\n##### Option 2: Build MMCV (full version with CPU)\n\n1. Finish above common steps\n2. Set up environment variables\n\n ```shell\n $env:MMCV_WITH_OPS = 1\n $env:MAX_JOBS = 8 # based on your available number of CPU cores and amount of memory\n ```\n\n3. Following build steps of the lite version\n\n ```shell\n # activate environment\n conda activate mmcv\n # change directory\n cd mmcv\n # build\n python setup.py build_ext # if success, cl will be launched to compile ops\n # install\n python setup.py develop\n # check\n pip list\n ```\n\n##### Option 3: Build MMCV (full version with CUDA)\n\n1. Finish above common steps\n2. Make sure `CUDA_PATH` or `CUDA_HOME` is already set in `envs` via `ls env:`, desired output is shown as below:\n\n ```none\n (base) PS C:\\Users\\WRH> ls env:\n\n Name Value\n ---- -----\n <... omit some lines ...>\n CUDA_PATH C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.2\n CUDA_PATH_V10_1 C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.1\n CUDA_PATH_V10_2 C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.2\n <... omit some lines ...>\n ```\n\n This should already be done by CUDA installer. If not, or you have multiple version of CUDA toolkit installed, set it with\n\n ```shell\n $env:CUDA_HOME = \"C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.2\"\n # OR\n $env:CUDA_HOME = $env:CUDA_PATH_V10_2 # if CUDA_PATH_V10_2 is in envs:\n ```\n\n3. Set CUDA target arch\n\n ```shell\n # Suppose you are using GTX 1080, which is of capability 6.1\n $env:TORCH_CUDA_ARCH_LIST=\"6.1\"\n # OR build all supported arch, will be slow\n $env:TORCH_CUDA_ARCH_LIST=\"3.5 3.7 5.0 5.2 6.0 6.1 7.0 7.5\"\n ```\n\n```{note}\nCheck your the compute capability of your GPU from [here](https://developer.nvidia.com/cuda-gpus).\n```\n\n4. Launch compiling the same way as CPU\n\n ```shell\n $env:MMCV_WITH_OPS = 1\n $env:MAX_JOBS = 8 # based on available number of CPU cores and amount of memory\n # activate environment\n conda activate mmcv\n # change directory\n cd mmcv\n # build\n python setup.py build_ext # if success, cl will be launched to compile ops\n # install\n python setup.py develop\n # check\n pip list\n ```\n\n```{note}\nIf you are compiling against PyTorch 1.6.0, you might meet some errors from PyTorch as described in [this issue](https://github.com/pytorch/pytorch/issues/42467). Follow [this pull request](https://github.com/pytorch/pytorch/pull/43380/files) to modify the source code in your local PyTorch installation.\n```\n\nIf you meet issues when running or compiling mmcv, we list some common issues in [Frequently Asked Question](../faq.html).\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/get_started/installation.md", "content": "## Installation\n\nThere are two versions of MMCV:\n\n- **mmcv-full**: comprehensive, with full features and various CUDA ops out of box. It takes longer time to build.\n- **mmcv**: lite, without CUDA ops but all other features, similar to mmcv<1.0.0. It is useful when you do not need those CUDA ops.\n\n```{warning}\nDo not install both versions in the same environment, otherwise you may encounter errors like `ModuleNotFound`. You need to uninstall one before installing the other. `Installing the full version is highly recommended if CUDA is avaliable`.\n```\n\na. Install the full version.\n\nBefore installing mmcv-full, make sure that PyTorch has been successfully installed following the [official guide](https://pytorch.org/).\n\nWe provide pre-built mmcv packages (recommended) with different PyTorch and CUDA versions to simplify the building. In addition, you can run [check_installation.py](.dev_scripts/check_installation.py) to check the installation of mmcv-full after running the installation commands.\n\ni. Install the latest version.\n\nThe rule for installing the latest ``mmcv-full`` is as follows:\n\n```shell\npip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html\n```\n\nPlease replace ``{cu_version}`` and ``{torch_version}`` in the url to your desired one. For example,\nto install the latest ``mmcv-full`` with ``CUDA 11.1`` and ``PyTorch 1.9.0``, use the following command:\n\n```shell\npip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html\n```\n\nFor more details, please refer the the following tables and delete ``=={mmcv_version}``.\n\nii. Install a specified version.\n\nThe rule for installing a specified ``mmcv-full`` is as follows:\n\n```shell\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html\n```\n\nFirst of all, please refer to the Releases and replace ``{mmcv_version}`` a specified one. e.g. ``1.3.9``.\nThen replace ``{cu_version}`` and ``{torch_version}`` in the url to your desired versions. For example,\nto install ``mmcv-full==1.3.9`` with ``CUDA 11.1`` and ``PyTorch 1.9.0``, use the following command:\n\n```shell\npip install mmcv-full==1.3.9 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html\n```\n\n```{note}\nmmcv-full is only compiled on PyTorch 1.x.0 because the compatibility\nusually holds between 1.x.0 and 1.x.1. If your PyTorch version is 1.x.1, you\ncan install mmcv-full compiled with PyTorch 1.x.0 and it usually works well.\nFor example, if your PyTorch version is 1.8.1 and CUDA version is 11.1, you\ncan use the following command to install mmcv-full.\n\n`pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.8.0/index.html`\n```\n\nFor more details, please refer the the following tables.\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
CUDA torch 1.10torch 1.9torch 1.8torch 1.7torch 1.6torch 1.5
11.3
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
11.1
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.10.0/index.html
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.8.0/index.html
11.0
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html
10.2
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.10.0/index.html
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.9.0/index.html
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.7.0/index.html
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.6.0/index.html
install 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.5.0/index.html
10.1
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.8.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.7.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.6.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.5.0/index.html
9.2
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.7.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.6.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.5.0/index.html
cpu
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.10.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.9.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.8.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.7.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.6.0/index.html
install 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.5.0/index.html
\n\n```{note}\nThe pre-built packages provided above do not include all versions of mmcv-full, you can click on the corresponding links to see the supported versions. For example, if you click [cu102-torch1.8.0](https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html), you can see that `cu102-torch1.8.0` only provides 1.3.0 and above versions of mmcv-full. In addition, We no longer provide `mmcv-full` pre-built packages compiled with `PyTorch 1.3 & 1.4` since v1.3.17. You can find previous versions that compiled with PyTorch 1.3 & 1.4 [here](./previous_versions.md). The compatibility is still ensured in our CI, but we will discard the support of PyTorch 1.3 & 1.4 next year.\n```\n\nAnother way is to compile locally by running\n\n```python\npip install mmcv-full\n```\n\nNote that the local compiling may take up to 10 mins.\n\nb. Install the lite version.\n\n```python\npip install mmcv\n```\n\nc. Install full version with custom operators for onnxruntime\n\n- Check [here](https://mmcv.readthedocs.io/en/latest/deployment/onnxruntime_custom_ops.html) for detailed instruction.\n\nIf you would like to build MMCV from source, please refer to the [guide](https://mmcv.readthedocs.io/en/latest/get_started/build.html).\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/get_started/introduction.md", "content": "## Introduction\n\nMMCV is a foundational library for computer vision research and supports many\nresearch projects as below:\n\n- [MIM](https://github.com/open-mmlab/mim): MIM Installs OpenMMLab Packages.\n- [MMClassification](https://github.com/open-mmlab/mmclassification): OpenMMLab image classification toolbox and benchmark.\n- [MMDetection](https://github.com/open-mmlab/mmdetection): OpenMMLab detection toolbox and benchmark.\n- [MMDetection3D](https://github.com/open-mmlab/mmdetection3d): OpenMMLab's next-generation platform for general 3D object detection.\n- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation): OpenMMLab semantic segmentation toolbox and benchmark.\n- [MMAction2](https://github.com/open-mmlab/mmaction2): OpenMMLab's next-generation action understanding toolbox and benchmark.\n- [MMTracking](https://github.com/open-mmlab/mmtracking): OpenMMLab video perception toolbox and benchmark.\n- [MMPose](https://github.com/open-mmlab/mmpose): OpenMMLab pose estimation toolbox and benchmark.\n- [MMEditing](https://github.com/open-mmlab/mmediting): OpenMMLab image and video editing toolbox.\n- [MMOCR](https://github.com/open-mmlab/mmocr): A Comprehensive Toolbox for Text Detection, Recognition and Understanding.\n- [MMGeneration](https://github.com/open-mmlab/mmgeneration): OpenMMLab image and video generative models toolbox.\n- [MMFlow](https://github.com/open-mmlab/mmflow): OpenMMLab optical flow toolbox and benchmark.\n- [MMFewShot](https://github.com/open-mmlab/mmfewshot): OpenMMLab FewShot Learning Toolbox and Benchmark.\n- [MMHuman3D](https://github.com/open-mmlab/mmhuman3d): OpenMMLab 3D human parametric model toolbox and benchmark.\n- [MMSelfSup](https://github.com/open-mmlab/mmselfsup): OpenMMLab self-supervised learning Toolbox and Benchmark.\n- [MMRazor](https://github.com/open-mmlab/mmrazor): OpenMMLab Model Compression Toolbox and Benchmark.\n- [MMDeploy](https://github.com/open-mmlab/mmdeploy): OpenMMLab Model Deployment Framework.\n\nIt provides the following functionalities.\n\n- Universal IO APIs\n- Image/Video processing\n- Image and annotation visualization\n- Useful utilities (progress bar, timer, ...)\n- PyTorch runner with hooking mechanism\n- Various CNN architectures\n- High-quality implementation of common CUDA ops\n\n```{note}\nMMCV requires Python 3.6+.\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/get_started/previous_versions.md", "content": "## OTHER VERSIONS OF PYTORCH BUILT FOR MMCV-FULL\n\nWe no longer provide `mmcv-full` packages compiled under lower versions of `PyTorch`, but for your convenience, you can find them below.\n\n### PyTorch 1.4\n\n| 1.0.0 <= mmcv_version <= 1.2.1\n\n#### CUDA 10.1\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.4.0/index.html\n```\n\n#### CUDA 9.2\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.4.0/index.html\n```\n\n#### CPU\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.4.0/index.html\n```\n\n### PyTorch v1.3\n\n| 1.0.0 <= mmcv_version <= 1.3.16\n\n#### CUDA 10.1\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.3.0/index.html\n```\n\n#### CUDA 9.2\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.3.0/index.html\n```\n\n#### CPU\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.3.0/index.html\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/index.rst", "content": "Welcome to MMCV's documentation!\n================================\n\nYou can switch between Chinese and English documents in the lower-left corner of the layout.\n\n.. toctree::\n :maxdepth: 2\n :caption: Get Started\n\n get_started/introduction.md\n get_started/installation.md\n get_started/build.md\n\n.. toctree::\n :maxdepth: 2\n :caption: Understand MMCV\n\n understand_mmcv/config.md\n understand_mmcv/registry.md\n understand_mmcv/runner.md\n understand_mmcv/io.md\n understand_mmcv/data_process.md\n understand_mmcv/visualization.md\n understand_mmcv/cnn.md\n understand_mmcv/ops.md\n understand_mmcv/utils.md\n\n.. toctree::\n :maxdepth: 2\n :caption: Deployment\n\n deployment/mmcv_ops_definition.md\n deployment/onnx.md\n deployment/onnxruntime_op.md\n deployment/tensorrt_plugin.md\n\n.. toctree::\n :maxdepth: 2\n :caption: Compatibility\n\n compatibility.md\n\n.. toctree::\n :maxdepth: 2\n :caption: FAQ\n\n faq.md\n\n.. toctree::\n :maxdepth: 2\n :caption: Community\n\n community/contributing.md\n community/pr.md\n\n.. toctree::\n :maxdepth: 2\n :caption: API Reference\n\n api.rst\n\nIndices and tables\n==================\n\n* :ref:`genindex`\n* :ref:`search`\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/make.bat", "content": "@ECHO OFF\n\npushd %~dp0\n\nREM Command file for Sphinx documentation\n\nif \"%SPHINXBUILD%\" == \"\" (\n\tset SPHINXBUILD=sphinx-build\n)\nset SOURCEDIR=.\nset BUILDDIR=_build\n\nif \"%1\" == \"\" goto help\n\n%SPHINXBUILD% >NUL 2>NUL\nif errorlevel 9009 (\n\techo.\n\techo.The 'sphinx-build' command was not found. Make sure you have Sphinx\n\techo.installed, then set the SPHINXBUILD environment variable to point\n\techo.to the full path of the 'sphinx-build' executable. Alternatively you\n\techo.may add the Sphinx directory to PATH.\n\techo.\n\techo.If you don't have Sphinx installed, grab it from\n\techo.http://sphinx-doc.org/\n\texit /b 1\n)\n\n%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\ngoto end\n\n:help\n%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\n\n:end\npopd\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/cnn.md", "content": "## CNN\n\nWe provide some building bricks for CNNs, including layer building, module bundles and weight initialization.\n\n### Layer building\n\nWe may need to try different layers of the same type when running experiments,\nbut do not want to modify the code from time to time.\nHere we provide some layer building methods to construct layers from a dict,\nwhich can be written in configs or specified via command line arguments.\n\n#### Usage\n\nA simplest example is\n\n```python\ncfg = dict(type='Conv3d')\nlayer = build_conv_layer(cfg, in_channels=3, out_channels=8, kernel_size=3)\n```\n\n- `build_conv_layer`: Supported types are Conv1d, Conv2d, Conv3d, Conv (alias for Conv2d).\n- `build_norm_layer`: Supported types are BN1d, BN2d, BN3d, BN (alias for BN2d), SyncBN, GN, LN, IN1d, IN2d, IN3d, IN (alias for IN2d).\n- `build_activation_layer`: Supported types are ReLU, LeakyReLU, PReLU, RReLU, ReLU6, ELU, Sigmoid, Tanh, GELU.\n- `build_upsample_layer`: Supported types are nearest, bilinear, deconv, pixel_shuffle.\n- `build_padding_layer`: Supported types are zero, reflect, replicate.\n\n#### Extension\n\nWe also allow extending the building methods with custom layers and operators.\n\n1. Write and register your own module.\n\n ```python\n from mmcv.cnn import UPSAMPLE_LAYERS\n\n @UPSAMPLE_LAYERS.register_module()\n class MyUpsample:\n\n def __init__(self, scale_factor):\n pass\n\n def forward(self, x):\n pass\n ```\n\n2. Import `MyUpsample` somewhere (e.g., in `__init__.py`) and then use it.\n\n ```python\n cfg = dict(type='MyUpsample', scale_factor=2)\n layer = build_upsample_layer(cfg)\n ```\n\n### Module bundles\n\nWe also provide common module bundles to facilitate the network construction.\n`ConvModule` is a bundle of convolution, normalization and activation layers,\nplease refer to the [api](api.html#mmcv.cnn.ConvModule) for details.\n\n```python\n# conv + bn + relu\nconv = ConvModule(3, 8, 2, norm_cfg=dict(type='BN'))\n# conv + gn + relu\nconv = ConvModule(3, 8, 2, norm_cfg=dict(type='GN', num_groups=2))\n# conv + relu\nconv = ConvModule(3, 8, 2)\n# conv\nconv = ConvModule(3, 8, 2, act_cfg=None)\n# conv + leaky relu\nconv = ConvModule(3, 8, 3, padding=1, act_cfg=dict(type='LeakyReLU'))\n# bn + conv + relu\nconv = ConvModule(\n 3, 8, 2, norm_cfg=dict(type='BN'), order=('norm', 'conv', 'act'))\n```\n\n### Weight initialization\n\n> Implementation details are available at [mmcv/cnn/utils/weight_init.py](../../mmcv/cnn/utils/weight_init.py)\n\nDuring training, a proper initialization strategy is beneficial to speed up the\ntraining or obtain a higher performance. In MMCV, we provide some commonly used\nmethods for initializing modules like `nn.Conv2d`. Of course, we also provide\nhigh-level APIs for initializing models containing one or more\nmodules.\n\n#### Initialization functions\n\nInitialize a `nn.Module` such as `nn.Conv2d`, `nn.Linear` in a functional way.\n\nWe provide the following initialization methods.\n\n- constant_init\n\n Initialize module parameters with constant values.\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import constant_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # constant_init(module, val, bias=0)\n >>> constant_init(conv1, 1, 0)\n >>> conv1.weight\n ```\n\n- xavier_init\n\n Initialize module parameters with values according to the method\n described in [Understanding the difficulty of training deep feedforward neural networks - Glorot, X. & Bengio, Y. (2010)](http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf)\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import xavier_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # xavier_init(module, gain=1, bias=0, distribution='normal')\n >>> xavier_init(conv1, distribution='normal')\n ```\n\n- normal_init\n\n Initialize module parameters with the values drawn from a normal distribution.\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import normal_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # normal_init(module, mean=0, std=1, bias=0)\n >>> normal_init(conv1, std=0.01, bias=0)\n ```\n\n- uniform_init\n\n Initialize module parameters with values drawn from a uniform distribution.\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import uniform_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # uniform_init(module, a=0, b=1, bias=0)\n >>> uniform_init(conv1, a=0, b=1)\n ```\n\n- kaiming_init\n\n Initialize module parameters with the values according to the method\n described in [Delving deep into rectifiers: Surpassing human-level\n performance on ImageNet classification - He, K. et al. (2015)](https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/He_Delving_Deep_into_ICCV_2015_paper.pdf)\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import kaiming_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # kaiming_init(module, a=0, mode='fan_out', nonlinearity='relu', bias=0, distribution='normal')\n >>> kaiming_init(conv1)\n ```\n\n- caffe2_xavier_init\n\n The xavier initialization is implemented in caffe2, which corresponds to `kaiming_uniform_` in PyTorch.\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import caffe2_xavier_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # caffe2_xavier_init(module, bias=0)\n >>> caffe2_xavier_init(conv1)\n ```\n\n- bias_init_with_prob\n\n Initialize conv/fc bias value according to a given probability, as proposed in [Focal Loss for Dense Object Detection](https://arxiv.org/pdf/1708.02002.pdf).\n\n ```python\n >>> from mmcv.cnn import bias_init_with_prob\n >>> # bias_init_with_prob is proposed in Focal Loss\n >>> bias = bias_init_with_prob(0.01)\n >>> bias\n -4.59511985013459\n ```\n\n#### Initializers and configs\n\nOn the basis of the initialization methods, we define the corresponding initialization classes and register them to `INITIALIZERS`, so we can\nuse the configuration to initialize the model.\n\nWe provide the following initialization classes.\n\n- ConstantInit\n- XavierInit\n- NormalInit\n- UniformInit\n- KaimingInit\n- Caffe2XavierInit\n- PretrainedInit\n\nLet us introduce the usage of `initialize` in detail.\n\n1. Initialize model by `layer` key\n\n If we only define `layer`, it just initialize the layer in `layer` key.\n\n NOTE: Value of `layer` key is the class name with attributes weights and bias of Pytorch, so `MultiheadAttention layer` is not supported.\n\n- Define `layer` key for initializing module with same configuration.\n\n ```python\n import torch.nn as nn\n from mmcv.cnn import initialize\n\n class FooNet(nn.Module):\n def __init__(self):\n super().__init__()\n self.feat = nn.Conv1d(3, 1, 3)\n self.reg = nn.Conv2d(3, 3, 3)\n self.cls = nn.Linear(1, 2)\n\n model = FooNet()\n init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d', 'Linear'], val=1)\n # initialize whole module with same configuration\n initialize(model, init_cfg)\n # model.feat.weight\n # Parameter containing:\n # tensor([[[1., 1., 1.],\n # [1., 1., 1.],\n # [1., 1., 1.]]], requires_grad=True)\n ```\n\n- Define `layer` key for initializing layer with different configurations.\n\n ```python\n import torch.nn as nn\n from mmcv.cnn.utils import initialize\n\n class FooNet(nn.Module):\n def __init__(self):\n super().__init__()\n self.feat = nn.Conv1d(3, 1, 3)\n self.reg = nn.Conv2d(3, 3, 3)\n self.cls = nn.Linear(1,2)\n\n model = FooNet()\n init_cfg = [dict(type='Constant', layer='Conv1d', val=1),\n dict(type='Constant', layer='Conv2d', val=2),\n dict(type='Constant', layer='Linear', val=3)]\n # nn.Conv1d will be initialized with dict(type='Constant', val=1)\n # nn.Conv2d will be initialized with dict(type='Constant', val=2)\n # nn.Linear will be initialized with dict(type='Constant', val=3)\n initialize(model, init_cfg)\n # model.reg.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n ```\n\n2. Initialize model by `override` key\n\n- When initializing some specific part with its attribute name, we can use `override` key, and the value in `override` will ignore the value in init_cfg.\n\n ```python\n import torch.nn as nn\n from mmcv.cnn import initialize\n\n class FooNet(nn.Module):\n def __init__(self):\n super().__init__()\n self.feat = nn.Conv1d(3, 1, 3)\n self.reg = nn.Conv2d(3, 3, 3)\n self.cls = nn.Sequential(nn.Conv1d(3, 1, 3), nn.Linear(1,2))\n\n # if we would like to initialize model's weights as 1 and bias as 2\n # but weight in `cls` as 3 and bias 4, we can use override key\n model = FooNet()\n init_cfg = dict(type='Constant', layer=['Conv1d','Conv2d'], val=1, bias=2,\n override=dict(type='Constant', name='reg', val=3, bias=4))\n # self.feat and self.cls will be initialized with dict(type='Constant', val=1, bias=2)\n # The module called 'reg' will be initialized with dict(type='Constant', val=3, bias=4)\n initialize(model, init_cfg)\n # model.reg.weight\n # Parameter containing:\n # tensor([[[[3., 3., 3.],\n # [3., 3., 3.],\n # [3., 3., 3.]],\n # ...,\n # [[3., 3., 3.],\n # [3., 3., 3.],\n # [3., 3., 3.]]]], requires_grad=True)\n ```\n\n- If `layer` is None in init_cfg, only sub-module with the name in override will be initialized, and type and other args in override can be omitted.\n\n ```python\n model = FooNet()\n init_cfg = dict(type='Constant', val=1, bias=2, override=dict(name='reg'))\n # self.feat and self.cls will be initialized by Pytorch\n # The module called 'reg' will be initialized with dict(type='Constant', val=1, bias=2)\n initialize(model, init_cfg)\n # model.reg.weight\n # Parameter containing:\n # tensor([[[[1., 1., 1.],\n # [1., 1., 1.],\n # [1., 1., 1.]],\n # ...,\n # [[1., 1., 1.],\n # [1., 1., 1.],\n # [1., 1., 1.]]]], requires_grad=True)\n ```\n\n- If we don't define `layer` key or `override` key, it will not initialize anything.\n\n- Invalid usage\n\n ```python\n # It is invalid that override don't have name key\n init_cfg = dict(type='Constant', layer=['Conv1d','Conv2d'],\n val=1, bias=2,\n override=dict(type='Constant', val=3, bias=4))\n\n # It is also invalid that override has name and other args except type\n init_cfg = dict(type='Constant', layer=['Conv1d','Conv2d'],\n val=1, bias=2,\n override=dict(name='reg', val=3, bias=4))\n ```\n\n3. Initialize model with the pretrained model\n\n ```python\n import torch.nn as nn\n import torchvision.models as models\n from mmcv.cnn import initialize\n\n # initialize model with pretrained model\n model = models.resnet50()\n # model.conv1.weight\n # Parameter containing:\n # tensor([[[[-6.7435e-03, -2.3531e-02, -9.0143e-03, ..., -2.1245e-03,\n # -1.8077e-03, 3.0338e-03],\n # [-1.2603e-02, -2.7831e-02, 2.3187e-02, ..., -1.5793e-02,\n # 1.1655e-02, 4.5889e-03],\n # [-3.7916e-02, 1.2014e-02, 1.3815e-02, ..., -4.2651e-03,\n # 1.7314e-02, -9.9998e-03],\n # ...,\n\n init_cfg = dict(type='Pretrained',\n checkpoint='torchvision://resnet50')\n initialize(model, init_cfg)\n # model.conv1.weight\n # Parameter containing:\n # tensor([[[[ 1.3335e-02, 1.4664e-02, -1.5351e-02, ..., -4.0896e-02,\n # -4.3034e-02, -7.0755e-02],\n # [ 4.1205e-03, 5.8477e-03, 1.4948e-02, ..., 2.2060e-03,\n # -2.0912e-02, -3.8517e-02],\n # [ 2.2331e-02, 2.3595e-02, 1.6120e-02, ..., 1.0281e-01,\n # 6.2641e-02, 5.1977e-02],\n # ...,\n\n # initialize weights of a sub-module with the specific part of a pretrained model by using 'prefix'\n model = models.resnet50()\n url = 'http://download.openmmlab.com/mmdetection/v2.0/retinanet/'\\\n 'retinanet_r50_fpn_1x_coco/'\\\n 'retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth'\n init_cfg = dict(type='Pretrained',\n checkpoint=url, prefix='backbone.')\n initialize(model, init_cfg)\n ```\n\n4. Initialize model inherited from BaseModule, Sequential, ModuleList, ModuleDict\n\n `BaseModule` is inherited from `torch.nn.Module`, and the only different between them is that `BaseModule` implements `init_weight`.\n\n `Sequential` is inherited from `BaseModule` and `torch.nn.Sequential`.\n\n `ModuleList` is inherited from `BaseModule` and `torch.nn.ModuleList`.\n\n `ModuleDict` is inherited from `BaseModule` and `torch.nn.ModuleDict`.\n\n `````python\n import torch.nn as nn\n from mmcv.runner import BaseModule, Sequential, ModuleList, ModuleDict\n\n class FooConv1d(BaseModule):\n\n def __init__(self, init_cfg=None):\n super().__init__(init_cfg)\n self.conv1d = nn.Conv1d(4, 1, 4)\n\n def forward(self, x):\n return self.conv1d(x)\n\n class FooConv2d(BaseModule):\n\n def __init__(self, init_cfg=None):\n super().__init__(init_cfg)\n self.conv2d = nn.Conv2d(3, 1, 3)\n\n def forward(self, x):\n return self.conv2d(x)\n\n # BaseModule\n init_cfg = dict(type='Constant', layer='Conv1d', val=0., bias=1.)\n model = FooConv1d(init_cfg)\n model.init_weights()\n # model.conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n\n # Sequential\n init_cfg1 = dict(type='Constant', layer='Conv1d', val=0., bias=1.)\n init_cfg2 = dict(type='Constant', layer='Conv2d', val=2., bias=3.)\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n seq_model = Sequential(model1, model2)\n seq_model.init_weights()\n # seq_model[0].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # seq_model[1].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # inner init_cfg has higher priority\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.)\n seq_model = Sequential(model1, model2, init_cfg=init_cfg)\n seq_model.init_weights()\n # seq_model[0].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # seq_model[1].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # ModuleList\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n modellist = ModuleList([model1, model2])\n modellist.init_weights()\n # modellist[0].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # modellist[1].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # inner init_cfg has higher priority\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.)\n modellist = ModuleList([model1, model2], init_cfg=init_cfg)\n modellist.init_weights()\n # modellist[0].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # modellist[1].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # ModuleDict\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n modeldict = ModuleDict(dict(model1=model1, model2=model2))\n modeldict.init_weights()\n # modeldict['model1'].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # modeldict['model2'].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # inner init_cfg has higher priority\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.)\n modeldict = ModuleDict(dict(model1=model1, model2=model2), init_cfg=init_cfg)\n modeldict.init_weights()\n # modeldict['model1'].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # modeldict['model2'].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n `````\n\n### Model Zoo\n\nBesides torchvision pre-trained models, we also provide pre-trained models of following CNN:\n\n- VGG Caffe\n- ResNet Caffe\n- ResNeXt\n- ResNet with Group Normalization\n- ResNet with Group Normalization and Weight Standardization\n- HRNetV2\n- Res2Net\n- RegNet\n\n#### Model URLs in JSON\n\nThe model zoo links in MMCV are managed by JSON files.\nThe json file consists of key-value pair of model name and its url or path.\nAn example json file could be like:\n\n```json\n{\n \"model_a\": \"https://example.com/models/model_a_9e5bac.pth\",\n \"model_b\": \"pretrain/model_b_ab3ef2c.pth\"\n}\n```\n\nThe default links of the pre-trained models hosted on OpenMMLab AWS could be found [here](https://github.com/open-mmlab/mmcv/blob/master/mmcv/model_zoo/open_mmlab.json).\n\nYou may override default links by putting `open-mmlab.json` under `MMCV_HOME`. If `MMCV_HOME` is not find in the environment, `~/.cache/mmcv` will be used by default. You may `export MMCV_HOME=/your/path` to use your own path.\n\nThe external json files will be merged into default one. If the same key presents in both external json and default json, the external one will be used.\n\n#### Load Checkpoint\n\nThe following types are supported for `filename` argument of `mmcv.load_checkpoint()`.\n\n- filepath: The filepath of the checkpoint.\n- `http://xxx` and `https://xxx`: The link to download the checkpoint. The `SHA256` postfix should be contained in the filename.\n- `torchvision://xxx`: The model links in `torchvision.models`.Please refer to [torchvision](https://pytorch.org/docs/stable/torchvision/models.html) for details.\n- `open-mmlab://xxx`: The model links or filepath provided in default and additional json files.\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/config.md", "content": "## Config\n\n`Config` class is used for manipulating config and config files. It supports\nloading configs from multiple file formats including **python**, **json** and **yaml**.\nIt provides dict-like apis to get and set values.\n\nHere is an example of the config file `test.py`.\n\n```python\na = 1\nb = dict(b1=[0, 1, 2], b2=None)\nc = (1, 2)\nd = 'string'\n```\n\nTo load and use configs\n\n```python\n>>> cfg = Config.fromfile('test.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b1=[0, 1, 2], b2=None),\n... c=(1, 2),\n... d='string')\n```\n\nFor all format configs, some predefined variables are supported. It will convert the variable in `{{ var }}` with its real value.\n\nCurrently, it supports four predefined variables:\n\n`{{ fileDirname }}` - the current opened file's dirname, e.g. /home/your-username/your-project/folder\n\n`{{ fileBasename }}` - the current opened file's basename, e.g. file.ext\n\n`{{ fileBasenameNoExtension }}` - the current opened file's basename with no file extension, e.g. file\n\n`{{ fileExtname }}` - the current opened file's extension, e.g. .ext\n\nThese variable names are referred from [VS Code](https://code.visualstudio.com/docs/editor/variables-reference).\n\nHere is one examples of config with predefined variables.\n\n`config_a.py`\n\n```python\na = 1\nb = './work_dir/{{ fileBasenameNoExtension }}'\nc = '{{ fileExtname }}'\n```\n\n```python\n>>> cfg = Config.fromfile('./config_a.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b='./work_dir/config_a',\n... c='.py')\n```\n\nFor all format configs, inheritance is supported. To reuse fields in other config files,\nspecify `_base_='./config_a.py'` or a list of configs `_base_=['./config_a.py', './config_b.py']`.\nHere are 4 examples of config inheritance.\n\n`config_a.py`\n\n```python\na = 1\nb = dict(b1=[0, 1, 2], b2=None)\n```\n\n### Inherit from base config without overlapped keys\n\n`config_b.py`\n\n```python\n_base_ = './config_a.py'\nc = (1, 2)\nd = 'string'\n```\n\n```python\n>>> cfg = Config.fromfile('./config_b.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b1=[0, 1, 2], b2=None),\n... c=(1, 2),\n... d='string')\n```\n\nNew fields in `config_b.py` are combined with old fields in `config_a.py`\n\n### Inherit from base config with overlapped keys\n\n`config_c.py`\n\n```python\n_base_ = './config_a.py'\nb = dict(b2=1)\nc = (1, 2)\n```\n\n```python\n>>> cfg = Config.fromfile('./config_c.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b1=[0, 1, 2], b2=1),\n... c=(1, 2))\n```\n\n`b.b2=None` in `config_a` is replaced with `b.b2=1` in `config_c.py`.\n\n### Inherit from base config with ignored fields\n\n`config_d.py`\n\n```python\n_base_ = './config_a.py'\nb = dict(_delete_=True, b2=None, b3=0.1)\nc = (1, 2)\n```\n\n```python\n>>> cfg = Config.fromfile('./config_d.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b2=None, b3=0.1),\n... c=(1, 2))\n```\n\nYou may also set `_delete_=True` to ignore some fields in base configs. All old keys `b1, b2, b3` in `b` are replaced with new keys `b2, b3`.\n\n### Inherit from multiple base configs (the base configs should not contain the same keys)\n\n`config_e.py`\n\n```python\nc = (1, 2)\nd = 'string'\n```\n\n`config_f.py`\n\n```python\n_base_ = ['./config_a.py', './config_e.py']\n```\n\n```python\n>>> cfg = Config.fromfile('./config_f.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b1=[0, 1, 2], b2=None),\n... c=(1, 2),\n... d='string')\n```\n\n### Reference variables from base\n\nYou can reference variables defined in base using the following grammar.\n\n`base.py`\n\n```python\nitem1 = 'a'\nitem2 = dict(item3 = 'b')\n```\n\n`config_g.py`\n\n```python\n_base_ = ['./base.py']\nitem = dict(a = {{ _base_.item1 }}, b = {{ _base_.item2.item3 }})\n```\n\n```python\n>>> cfg = Config.fromfile('./config_g.py')\n>>> print(cfg.pretty_text)\nitem1 = 'a'\nitem2 = dict(item3='b')\nitem = dict(a='a', b='b')\n```\n\n### Add deprecation information in configs\n\nDeprecation information can be added in a config file, which will trigger a `UserWarning` when this config file is loaded.\n\n`deprecated_cfg.py`\n\n```python\n_base_ = 'expected_cfg.py'\n\n_deprecation_ = dict(\n expected = 'expected_cfg.py', # optional to show expected config path in the warning information\n reference = 'url to related PR' # optional to show reference link in the warning information\n)\n```\n\n```python\n>>> cfg = Config.fromfile('./deprecated_cfg.py')\n\nUserWarning: The config file deprecated.py will be deprecated in the future. Please use expected_cfg.py instead. More information can be found at https://github.com/open-mmlab/mmcv/pull/1275\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/data_process.md", "content": "## Data Process\n\n### Image\n\nThis module provides some image processing methods, which requires `opencv` to be installed.\n\n#### Read/Write/Show\n\nTo read or write images files, use `imread` or `imwrite`.\n\n```python\nimport mmcv\n\nimg = mmcv.imread('test.jpg')\nimg = mmcv.imread('test.jpg', flag='grayscale')\nimg_ = mmcv.imread(img) # nothing will happen, img_ = img\nmmcv.imwrite(img, 'out.jpg')\n```\n\nTo read images from bytes\n\n```python\nwith open('test.jpg', 'rb') as f:\n data = f.read()\nimg = mmcv.imfrombytes(data)\n```\n\nTo show an image file or a loaded image\n\n```python\nmmcv.imshow('tests/data/color.jpg')\n# this is equivalent to\n\nfor i in range(10):\n img = np.random.randint(256, size=(100, 100, 3), dtype=np.uint8)\n mmcv.imshow(img, win_name='test image', wait_time=200)\n```\n\n#### Color space conversion\n\nSupported conversion methods:\n\n- bgr2gray\n- gray2bgr\n- bgr2rgb\n- rgb2bgr\n- bgr2hsv\n- hsv2bgr\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\nimg1 = mmcv.bgr2rgb(img)\nimg2 = mmcv.rgb2gray(img1)\nimg3 = mmcv.bgr2hsv(img)\n```\n\n#### Resize\n\nThere are three resize methods. All `imresize_*` methods have an argument `return_scale`,\nif this argument is `False`, then the return value is merely the resized image, otherwise\nis a tuple `(resized_img, scale)`.\n\n```python\n# resize to a given size\nmmcv.imresize(img, (1000, 600), return_scale=True)\n\n# resize to the same size of another image\nmmcv.imresize_like(img, dst_img, return_scale=False)\n\n# resize by a ratio\nmmcv.imrescale(img, 0.5)\n\n# resize so that the max edge no longer than 1000, short edge no longer than 800\n# without changing the aspect ratio\nmmcv.imrescale(img, (1000, 800))\n```\n\n#### Rotate\n\nTo rotate an image by some angle, use `imrotate`. The center can be specified,\nwhich is the center of original image by default. There are two modes of rotating,\none is to keep the image size unchanged so that some parts of the image will be\ncropped after rotating, the other is to extend the image size to fit the rotated\nimage.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# rotate the image clockwise by 30 degrees.\nimg_ = mmcv.imrotate(img, 30)\n\n# rotate the image counterclockwise by 90 degrees.\nimg_ = mmcv.imrotate(img, -90)\n\n# rotate the image clockwise by 30 degrees, and rescale it by 1.5x at the same time.\nimg_ = mmcv.imrotate(img, 30, scale=1.5)\n\n# rotate the image clockwise by 30 degrees, with (100, 100) as the center.\nimg_ = mmcv.imrotate(img, 30, center=(100, 100))\n\n# rotate the image clockwise by 30 degrees, and extend the image size.\nimg_ = mmcv.imrotate(img, 30, auto_bound=True)\n```\n\n#### Flip\n\nTo flip an image, use `imflip`.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# flip the image horizontally\nmmcv.imflip(img)\n\n# flip the image vertically\nmmcv.imflip(img, direction='vertical')\n```\n\n#### Crop\n\n`imcrop` can crop the image with one or some regions, represented as (x1, y1, x2, y2).\n\n```python\nimport mmcv\nimport numpy as np\n\nimg = mmcv.imread('tests/data/color.jpg')\n\n# crop the region (10, 10, 100, 120)\nbboxes = np.array([10, 10, 100, 120])\npatch = mmcv.imcrop(img, bboxes)\n\n# crop two regions (10, 10, 100, 120) and (0, 0, 50, 50)\nbboxes = np.array([[10, 10, 100, 120], [0, 0, 50, 50]])\npatches = mmcv.imcrop(img, bboxes)\n\n# crop two regions, and rescale the patches by 1.2x\npatches = mmcv.imcrop(img, bboxes, scale_ratio=1.2)\n```\n\n#### Padding\n\nThere are two methods `impad` and `impad_to_multiple` to pad an image to the\nspecific size with given values.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# pad the image to (1000, 1200) with all zeros\nimg_ = mmcv.impad(img, shape=(1000, 1200), pad_val=0)\n\n# pad the image to (1000, 1200) with different values for three channels.\nimg_ = mmcv.impad(img, shape=(1000, 1200), pad_val=[100, 50, 200])\n\n# pad the image on left, right, top, bottom borders with all zeros\nimg_ = mmcv.impad(img, padding=(10, 20, 30, 40), pad_val=0)\n\n# pad the image on left, right, top, bottom borders with different values\n# for three channels.\nimg_ = mmcv.impad(img, padding=(10, 20, 30, 40), pad_val=[100, 50, 200])\n\n# pad an image so that each edge is a multiple of some value.\nimg_ = mmcv.impad_to_multiple(img, 32)\n```\n\n### Video\n\nThis module provides the following functionalities.\n\n- A `VideoReader` class with friendly apis to read and convert videos.\n- Some methods for editing (cut, concat, resize) videos.\n- Optical flow read/write/warp.\n\n#### VideoReader\n\nThe `VideoReader` class provides sequence like apis to access video frames.\nIt will internally cache the frames which have been visited.\n\n```python\nvideo = mmcv.VideoReader('test.mp4')\n\n# obtain basic information\nprint(len(video))\nprint(video.width, video.height, video.resolution, video.fps)\n\n# iterate over all frames\nfor frame in video:\n print(frame.shape)\n\n# read the next frame\nimg = video.read()\n\n# read a frame by index\nimg = video[100]\n\n# read some frames\nimg = video[5:10]\n```\n\nTo convert a video to images or generate a video from a image directory.\n\n```python\n# split a video into frames and save to a folder\nvideo = mmcv.VideoReader('test.mp4')\nvideo.cvt2frames('out_dir')\n\n# generate video from frames\nmmcv.frames2video('out_dir', 'test.avi')\n```\n\n#### Editing utils\n\nThere are also some methods for editing videos, which wraps the commands of ffmpeg.\n\n```python\n# cut a video clip\nmmcv.cut_video('test.mp4', 'clip1.mp4', start=3, end=10, vcodec='h264')\n\n# join a list of video clips\nmmcv.concat_video(['clip1.mp4', 'clip2.mp4'], 'joined.mp4', log_level='quiet')\n\n# resize a video with the specified size\nmmcv.resize_video('test.mp4', 'resized1.mp4', (360, 240))\n\n# resize a video with a scaling ratio of 2\nmmcv.resize_video('test.mp4', 'resized2.mp4', ratio=2)\n```\n\n#### Optical flow\n\n`mmcv` provides the following methods to operate on optical flows.\n\n- IO\n- Visualization\n- Flow warpping\n\nWe provide two options to dump optical flow files: uncompressed and compressed.\nThe uncompressed way just dumps the floating numbers to a binary file. It is\nlossless but the dumped file has a larger size.\nThe compressed way quantizes the optical flow to 0-255 and dumps it as a\njpeg image. The flow of x-dim and y-dim will be concatenated into a single image.\n\n1. IO\n\n```python\nflow = np.random.rand(800, 600, 2).astype(np.float32)\n# dump the flow to a flo file (~3.7M)\nmmcv.flowwrite(flow, 'uncompressed.flo')\n# dump the flow to a jpeg file (~230K)\n# the shape of the dumped image is (800, 1200)\nmmcv.flowwrite(flow, 'compressed.jpg', quantize=True, concat_axis=1)\n\n# read the flow file, the shape of loaded flow is (800, 600, 2) for both ways\nflow = mmcv.flowread('uncompressed.flo')\nflow = mmcv.flowread('compressed.jpg', quantize=True, concat_axis=1)\n```\n\n2. Visualization\n\nIt is possible to visualize optical flows with `mmcv.flowshow()`.\n\n```python\nmmcv.flowshow(flow)\n```\n\n![progress](../_static/flow_visualization.png)\n\n3. Flow warpping\n\n```python\nimg1 = mmcv.imread('img1.jpg')\nflow = mmcv.flowread('flow.flo')\nwarpped_img2 = mmcv.flow_warp(img1, flow)\n```\n\nimg1 (left) and img2 (right)\n\n![raw images](../_static/flow_raw_images.png)\n\noptical flow (img2 -> img1)\n\n![optical flow](../_static/flow_img2toimg1.png)\n\nwarpped image and difference with ground truth\n\n![warpped image](../_static/flow_warp_diff.png)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/io.md", "content": "## File IO\n\nThis module provides two universal API to load and dump files of different formats.\n\n```{note}\nSince v1.3.16, the IO modules support loading (dumping) data from (to) different backends, respectively. More details are in PR [#1330](https://github.com/open-mmlab/mmcv/pull/1330).\n```\n\n### Load and dump data\n\n`mmcv` provides a universal api for loading and dumping data, currently\nsupported formats are json, yaml and pickle.\n\n#### Load from disk or dump to disk\n\n```python\nimport mmcv\n\n# load data from a file\ndata = mmcv.load('test.json')\ndata = mmcv.load('test.yaml')\ndata = mmcv.load('test.pkl')\n# load data from a file-like object\nwith open('test.json', 'r') as f:\n data = mmcv.load(f, file_format='json')\n\n# dump data to a string\njson_str = mmcv.dump(data, file_format='json')\n\n# dump data to a file with a filename (infer format from file extension)\nmmcv.dump(data, 'out.pkl')\n\n# dump data to a file with a file-like object\nwith open('test.yaml', 'w') as f:\n data = mmcv.dump(data, f, file_format='yaml')\n```\n\n#### Load from other backends or dump to other backends\n\n```python\nimport mmcv\n\n# load data from a file\ndata = mmcv.load('s3://bucket-name/test.json')\ndata = mmcv.load('s3://bucket-name/test.yaml')\ndata = mmcv.load('s3://bucket-name/test.pkl')\n\n# dump data to a file with a filename (infer format from file extension)\nmmcv.dump(data, 's3://bucket-name/out.pkl')\n```\n\nIt is also very convenient to extend the api to support more file formats.\nAll you need to do is to write a file handler inherited from `BaseFileHandler`\nand register it with one or several file formats.\n\nYou need to implement at least 3 methods.\n\n```python\nimport mmcv\n\n# To register multiple file formats, a list can be used as the argument.\n# @mmcv.register_handler(['txt', 'log'])\n@mmcv.register_handler('txt')\nclass TxtHandler1(mmcv.BaseFileHandler):\n\n def load_from_fileobj(self, file):\n return file.read()\n\n def dump_to_fileobj(self, obj, file):\n file.write(str(obj))\n\n def dump_to_str(self, obj, **kwargs):\n return str(obj)\n```\n\nHere is an example of `PickleHandler`.\n\n```python\nimport pickle\n\nclass PickleHandler(mmcv.BaseFileHandler):\n\n def load_from_fileobj(self, file, **kwargs):\n return pickle.load(file, **kwargs)\n\n def load_from_path(self, filepath, **kwargs):\n return super(PickleHandler, self).load_from_path(\n filepath, mode='rb', **kwargs)\n\n def dump_to_str(self, obj, **kwargs):\n kwargs.setdefault('protocol', 2)\n return pickle.dumps(obj, **kwargs)\n\n def dump_to_fileobj(self, obj, file, **kwargs):\n kwargs.setdefault('protocol', 2)\n pickle.dump(obj, file, **kwargs)\n\n def dump_to_path(self, obj, filepath, **kwargs):\n super(PickleHandler, self).dump_to_path(\n obj, filepath, mode='wb', **kwargs)\n```\n\n### Load a text file as a list or dict\n\nFor example `a.txt` is a text file with 5 lines.\n\n```\na\nb\nc\nd\ne\n```\n\n#### Load from disk\n\nUse `list_from_file` to load the list from a.txt.\n\n```python\n>>> mmcv.list_from_file('a.txt')\n['a', 'b', 'c', 'd', 'e']\n>>> mmcv.list_from_file('a.txt', offset=2)\n['c', 'd', 'e']\n>>> mmcv.list_from_file('a.txt', max_num=2)\n['a', 'b']\n>>> mmcv.list_from_file('a.txt', prefix='/mnt/')\n['/mnt/a', '/mnt/b', '/mnt/c', '/mnt/d', '/mnt/e']\n```\n\nFor example `b.txt` is a text file with 3 lines.\n\n```\n1 cat\n2 dog cow\n3 panda\n```\n\nThen use `dict_from_file` to load the dict from `b.txt`.\n\n```python\n>>> mmcv.dict_from_file('b.txt')\n{'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n>>> mmcv.dict_from_file('b.txt', key_type=int)\n{1: 'cat', 2: ['dog', 'cow'], 3: 'panda'}\n```\n\n#### Load from other backends\n\nUse `list_from_file` to load the list from `s3://bucket-name/a.txt`.\n\n```python\n>>> mmcv.list_from_file('s3://bucket-name/a.txt')\n['a', 'b', 'c', 'd', 'e']\n>>> mmcv.list_from_file('s3://bucket-name/a.txt', offset=2)\n['c', 'd', 'e']\n>>> mmcv.list_from_file('s3://bucket-name/a.txt', max_num=2)\n['a', 'b']\n>>> mmcv.list_from_file('s3://bucket-name/a.txt', prefix='/mnt/')\n['/mnt/a', '/mnt/b', '/mnt/c', '/mnt/d', '/mnt/e']\n```\n\nUse `dict_from_file` to load the dict from `s3://bucket-name/b.txt`.\n\n```python\n>>> mmcv.dict_from_file('s3://bucket-name/b.txt')\n{'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n>>> mmcv.dict_from_file('s3://bucket-name/b.txt', key_type=int)\n{1: 'cat', 2: ['dog', 'cow'], 3: 'panda'}\n```\n\n### Load and dump checkpoints\n\n#### Load checkpoints from disk or save to disk\n\nWe can read the checkpoints from disk or save to disk in the following way.\n\n```python\nimport torch\n\nfilepath1 = '/path/of/your/checkpoint1.pth'\nfilepath2 = '/path/of/your/checkpoint2.pth'\n# read from filepath1\ncheckpoint = torch.load(filepath1)\n# save to filepath2\ntorch.save(checkpoint, filepath2)\n```\n\nMMCV provides many backends. `HardDiskBackend` is one of them and we can use it to read or save checkpoints.\n\n```python\nimport io\nfrom mmcv.fileio.file_client import HardDiskBackend\n\ndisk_backend = HardDiskBackend()\nwith io.BytesIO(disk_backend.get(filepath1)) as buffer:\n checkpoint = torch.load(buffer)\nwith io.BytesIO() as buffer:\n torch.save(checkpoint, f)\n disk_backend.put(f.getvalue(), filepath2)\n```\n\nIf we want to implement an interface which automatically select the corresponding\nbackend based on the file path, we can use the `FileClient`.\nFor example, we want to implement two methods for reading checkpoints as well as saving checkpoints,\nwhich need to support different types of file paths, either disk paths, network paths or other paths.\n\n```python\nfrom mmcv.fileio.file_client import FileClient\n\ndef load_checkpoint(path):\n file_client = FileClient.infer(uri=path)\n with io.BytesIO(file_client.get(path)) as buffer:\n checkpoint = torch.load(buffer)\n return checkpoint\n\ndef save_checkpoint(checkpoint, path):\n with io.BytesIO() as buffer:\n torch.save(checkpoint, buffer)\n file_client.put(buffer.getvalue(), path)\n\nfile_client = FileClient.infer_client(uri=filepath1)\ncheckpoint = load_checkpoint(filepath1)\nsave_checkpoint(checkpoint, filepath2)\n```\n\n#### Load checkpoints from the Internet\n\n```{note}\nCurrently, it only supports reading checkpoints from the Internet, and does not support saving checkpoints to the Internet.\n```\n\n```python\nimport io\nimport torch\nfrom mmcv.fileio.file_client import HTTPBackend, FileClient\n\nfilepath = 'http://path/of/your/checkpoint.pth'\ncheckpoint = torch.utils.model_zoo.load_url(filepath)\n\nhttp_backend = HTTPBackend()\nwith io.BytesIO(http_backend.get(filepath)) as buffer:\n checkpoint = torch.load(buffer)\n\nfile_client = FileClient.infer_client(uri=filepath)\nwith io.BytesIO(file_client.get(filepath)) as buffer:\n checkpoint = torch.load(buffer)\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/ops.md", "content": "## CUDA ops\n\nWe implement common CUDA ops used in detection, segmentation, etc.\n\n- ActiveRotatedFilter\n- AssignScoreWithK\n- BallQuery\n- BBoxOverlaps\n- CARAFE\n- CrissCrossAttention\n- ContextBlock\n- ConvexIoU\n- CornerPool\n- Deformable Convolution v1/v2\n- Deformable RoIPool\n- DynamicScatter\n- GatherPoints\n- FurthestPointSample\n- FurthestPointSampleWithDist\n- GeneralizedAttention\n- GroupPoints\n- KNN\n- MaskedConv\n- MinAreaPolygon\n- NMS\n- PointsInPolygons\n- PSAMask\n- RiRoIAlignRotated\n- RotatedFeatureAlign\n- RoIPointPool3d\n- RoIPool\n- RoIAlign\n- RoIAwarePool3d\n- SimpleRoIAlign\n- SigmoidFocalLoss\n- SoftmaxFocalLoss\n- SoftNMS\n- Synchronized BatchNorm\n- Voxelization\n- ThreeInterpolate\n- ThreeNN\n- Weight standardization\n- Correlation\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/registry.md", "content": "## Registry\n\nMMCV implements [registry](https://github.com/open-mmlab/mmcv/blob/master/mmcv/utils/registry.py) to manage different modules that share similar functionalities, e.g., backbones, head, and necks, in detectors.\nMost projects in OpenMMLab use registry to manage modules of datasets and models, such as [MMDetection](https://github.com/open-mmlab/mmdetection), [MMDetection3D](https://github.com/open-mmlab/mmdetection3d), [MMClassification](https://github.com/open-mmlab/mmclassification), [MMEditing](https://github.com/open-mmlab/mmediting), etc.\n\n### What is registry\n\nIn MMCV, registry can be regarded as a mapping that maps a class to a string.\nThese classes contained by a single registry usually have similar APIs but implement different algorithms or support different datasets.\nWith the registry, users can find and instantiate the class through its corresponding string, and use the instantiated module as they want.\nOne typical example is the config systems in most OpenMMLab projects, which use the registry to create hooks, runners, models, and datasets, through configs.\nThe API reference could be found [here](https://mmcv.readthedocs.io/en/latest/api.html?highlight=registry#mmcv.utils.Registry).\n\nTo manage your modules in the codebase by `Registry`, there are three steps as below.\n\n1. Create a build method (optional, in most cases you can just use the default one).\n2. Create a registry.\n3. Use this registry to manage the modules.\n\n`build_func` argument of `Registry` is to customize how to instantiate the class instance, the default one is `build_from_cfg` implemented [here](https://mmcv.readthedocs.io/en/latest/api.html?highlight=registry#mmcv.utils.build_from_cfg).\n\n### A Simple Example\n\nHere we show a simple example of using registry to manage modules in a package.\nYou can find more practical examples in OpenMMLab projects.\n\nAssuming we want to implement a series of Dataset Converter for converting different formats of data to the expected data format.\nWe create a directory as a package named `converters`.\nIn the package, we first create a file to implement builders, named `converters/builder.py`, as below\n\n```python\nfrom mmcv.utils import Registry\n# create a registry for converters\nCONVERTERS = Registry('converter')\n```\n\nThen we can implement different converters in the package. For example, implement `Converter1` in `converters/converter1.py`\n\n```python\n\nfrom .builder import CONVERTERS\n\n# use the registry to manage the module\n@CONVERTERS.register_module()\nclass Converter1(object):\n def __init__(self, a, b):\n self.a = a\n self.b = b\n```\nThe key step to use registry for managing the modules is to register the implemented module into the registry `CONVERTERS` through\n`@CONVERTERS.register_module()` when you are creating the module. By this way, a mapping between a string and the class is built and maintained by `CONVERTERS` as below\n\n```python\n'Converter1' -> \n```\n```{note}\nThe registry mechanism will be triggered only when the file where the module is located is imported.\nSo you need to import that file somewhere. More details can be found at https://github.com/open-mmlab/mmdetection/issues/5974.\n```\n\nIf the module is successfully registered, you can use this converter through configs as\n\n```python\nconverter_cfg = dict(type='Converter1', a=a_value, b=b_value)\nconverter = CONVERTERS.build(converter_cfg)\n```\n\n### Customize Build Function\n\nSuppose we would like to customize how `converters` are built, we could implement a customized `build_func` and pass it into the registry.\n\n```python\nfrom mmcv.utils import Registry\n\n# create a build function\ndef build_converter(cfg, registry, *args, **kwargs):\n cfg_ = cfg.copy()\n converter_type = cfg_.pop('type')\n if converter_type not in registry:\n raise KeyError(f'Unrecognized converter type {converter_type}')\n else:\n converter_cls = registry.get(converter_type)\n\n converter = converter_cls(*args, **kwargs, **cfg_)\n return converter\n\n# create a registry for converters and pass ``build_converter`` function\nCONVERTERS = Registry('converter', build_func=build_converter)\n```\n\n```{note}\nIn this example, we demonstrate how to use the `build_func` argument to customize the way to build a class instance.\nThe functionality is similar to the default `build_from_cfg`. In most cases, default one would be sufficient.\n`build_model_from_cfg` is also implemented to build PyTorch module in `nn.Sequentail`, you may directly use them instead of implementing by yourself.\n```\n\n### Hierarchy Registry\n\nYou could also build modules from more than one OpenMMLab frameworks, e.g. you could use all backbones in [MMClassification](https://github.com/open-mmlab/mmclassification) for object detectors in [MMDetection](https://github.com/open-mmlab/mmdetection), you may also combine an object detection model in [MMDetection](https://github.com/open-mmlab/mmdetection) and semantic segmentation model in [MMSegmentation](https://github.com/open-mmlab/mmsegmentation).\n\nAll `MODELS` registries of downstream codebases are children registries of MMCV's `MODELS` registry.\nBasically, there are two ways to build a module from child or sibling registries.\n\n1. Build from children registries.\n\n For example:\n\n In MMDetection we define:\n\n ```python\n from mmcv.utils import Registry\n from mmcv.cnn import MODELS as MMCV_MODELS\n MODELS = Registry('model', parent=MMCV_MODELS)\n\n @MODELS.register_module()\n class NetA(nn.Module):\n def forward(self, x):\n return x\n ```\n\n In MMClassification we define:\n\n ```python\n from mmcv.utils import Registry\n from mmcv.cnn import MODELS as MMCV_MODELS\n MODELS = Registry('model', parent=MMCV_MODELS)\n\n @MODELS.register_module()\n class NetB(nn.Module):\n def forward(self, x):\n return x + 1\n ```\n\n We could build two net in either MMDetection or MMClassification by:\n\n ```python\n from mmdet.models import MODELS\n net_a = MODELS.build(cfg=dict(type='NetA'))\n net_b = MODELS.build(cfg=dict(type='mmcls.NetB'))\n ```\n\n or\n\n ```python\n from mmcls.models import MODELS\n net_a = MODELS.build(cfg=dict(type='mmdet.NetA'))\n net_b = MODELS.build(cfg=dict(type='NetB'))\n ```\n\n2. Build from parent registry.\n\n The shared `MODELS` registry in MMCV is the parent registry for all downstream codebases (root registry):\n\n ```python\n from mmcv.cnn import MODELS as MMCV_MODELS\n net_a = MMCV_MODELS.build(cfg=dict(type='mmdet.NetA'))\n net_b = MMCV_MODELS.build(cfg=dict(type='mmcls.NetB'))\n ```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/runner.md", "content": "## Runner\n\nThe runner class is designed to manage the training. It eases the training process with less code demanded from users while staying flexible and configurable. The main features are as listed:\n\n- Support `EpochBasedRunner` and `IterBasedRunner` for different scenarios. Implementing customized runners is also allowed to meet customized needs.\n- Support customized workflow to allow switching between different modes while training. Currently, supported modes are train and val.\n- Enable extensibility through various hooks, including hooks defined in MMCV and customized ones.\n\n### EpochBasedRunner\n\nAs its name indicates, workflow in `EpochBasedRunner` should be set based on epochs. For example, [('train', 2), ('val', 1)] means running 2 epochs for training and 1 epoch for validation, iteratively. And each epoch may contain multiple iterations. Currently, MMDetection uses `EpochBasedRunner` by default.\n\nLet's take a look at its core logic:\n\n```python\n# the condition to stop training\nwhile curr_epoch < max_epochs:\n # traverse the workflow.\n # e.g. workflow = [('train', 2), ('val', 1)]\n for i, flow in enumerate(workflow):\n # mode(e.g. train) determines which function to run\n mode, epochs = flow\n # epoch_runner will be either self.train() or self.val()\n epoch_runner = getattr(self, mode)\n # execute the corresponding function\n for _ in range(epochs):\n epoch_runner(data_loaders[i], **kwargs)\n```\n\nCurrently, we support 2 modes: train and val. Let's take a train function for example and have a look at its core logic:\n\n```python\n# Currently, epoch_runner could be either train or val\ndef train(self, data_loader, **kwargs):\n # traverse the dataset and get batch data for 1 epoch\n for i, data_batch in enumerate(data_loader):\n # it will execute all before_train_iter function in the hooks registered. You may want to watch out for the order.\n self.call_hook('before_train_iter')\n # set train_mode as False in val function\n self.run_iter(data_batch, train_mode=True, **kwargs)\n self.call_hook('after_train_iter')\n self.call_hook('after_train_epoch')\n```\n\n### IterBasedRunner\n\nDifferent from `EpochBasedRunner`, workflow in `IterBasedRunner` should be set based on iterations. For example, [('train', 2), ('val', 1)] means running 2 iters for training and 1 iter for validation, iteratively. Currently, MMSegmentation uses `IterBasedRunner` by default.\n\nLet's take a look at its core logic:\n\n```python\n# Although we set workflow by iters here, we might also need info on the epochs in some using cases. That can be provided by IterLoader.\niter_loaders = [IterLoader(x) for x in data_loaders]\n# the condition to stop training\nwhile curr_iter < max_iters:\n # traverse the workflow.\n # e.g. workflow = [('train', 2), ('val', 1)]\n for i, flow in enumerate(workflow):\n # mode(e.g. train) determines which function to run\n mode, iters = flow\n # iter_runner will be either self.train() or self.val()\n iter_runner = getattr(self, mode)\n # execute the corresponding function\n for _ in range(iters):\n iter_runner(iter_loaders[i], **kwargs)\n```\n\nCurrently, we support 2 modes: train and val. Let's take a val function for example and have a look at its core logic:\n\n```python\n# Currently, iter_runner could be either train or val\ndef val(self, data_loader, **kwargs):\n # get batch data for 1 iter\n data_batch = next(data_loader)\n # it will execute all before_val_iter function in the hooks registered. You may want to watch out for the order.\n self.call_hook('before_val_iter')\n outputs = self.model.val_step(data_batch, self.optimizer, **kwargs)\n self.outputs = outputs\n self.call_hook('after_val_iter')\n```\n\nOther than the basic functionalities explained above, `EpochBasedRunner` and `IterBasedRunner` provide methods such as `resume`, `save_checkpoint` and `register_hook`. In case you are not familiar with the term Hook mentioned earlier, we will also provide a tutorial about it.(coming soon...) Essentially, a hook is functionality to alter or augment the code behaviors through predefined api. It allows users to have their own code called under certain circumstances. It makes code extensible in a non-intrusive manner.\n\n### A Simple Example\n\nWe will walk you through the usage of runner with a classification task. The following code only contains essential steps for demonstration purposes. The following steps are necessary for any training tasks.\n\n**(1) Initialize dataloader, model, optimizer, etc.**\n\n```python\n# initialize model\nmodel=...\n# initialize optimizer, typically, we set: cfg.optimizer = dict(type='SGD', lr=0.1, momentum=0.9, weight_decay=0.0001)\noptimizer = build_optimizer(model, cfg.optimizer)\n# initialize the dataloader corresponding to the workflow(train/val)\ndata_loaders = [\n build_dataloader(\n ds,\n cfg.data.samples_per_gpu,\n cfg.data.workers_per_gpu,\n ...) for ds in dataset\n ]\n```\n\n**(2) Initialize runner**\n\n```python\nrunner = build_runner(\n # cfg.runner is typically set as:\n # runner = dict(type='EpochBasedRunner', max_epochs=200)\n cfg.runner,\n default_args=dict(\n model=model,\n batch_processor=None,\n optimizer=optimizer,\n logger=logger))\n```\n\n**(3) Register training hooks and customized hooks.**\n\n```python\n# register default hooks necessary for training\nrunner.register_training_hooks(\n # configs of learning rate, it is typically set as:\n # lr_config = dict(policy='step', step=[100, 150])\n cfg.lr_config,\n # configuration of optimizer, e.g. grad_clip\n optimizer_config,\n # configuration of saving checkpoints, it is typically set as:\n # checkpoint_config = dict(interval=1), saving checkpoints every epochs\n cfg.checkpoint_config,\n # configuration of logs\n cfg.log_config,\n ...)\n\n# register customized hooks\n# say we want to enable ema, then we could set custom_hooks=[dict(type='EMAHook')]\nif cfg.get('custom_hooks', None):\n custom_hooks = cfg.custom_hooks\n for hook_cfg in cfg.custom_hooks:\n hook_cfg = hook_cfg.copy()\n priority = hook_cfg.pop('priority', 'NORMAL')\n hook = build_from_cfg(hook_cfg, HOOKS)\n runner.register_hook(hook, priority=priority)\n```\n\nThen, we can use `resume` or `load_checkpoint` to load existing weights.\n\n**(4) Start training**\n\n```python\n# workflow is typically set as: workflow = [('train', 1)]\n# here the training begins.\nrunner.run(data_loaders, cfg.workflow)\n```\n\nLet's take `EpochBasedRunner` for example and go a little bit into details about setting workflow:\n\n- Say we only want to put train in the workflow, then we can set: workflow = [('train', 1)]. The runner will only execute train iteratively in this case.\n- Say we want to put both train and val in the workflow, then we can set: workflow = [('train', 3), ('val',1)]. The runner will first execute train for 3 epochs and then switch to val mode and execute val for 1 epoch. The workflow will be repeated until the current epoch hit the max_epochs.\n- Workflow is highly flexible. Therefore, you can set workflow = [('val', 1), ('train',1)] if you would like the runner to validate first and train after.\n\nThe code we demonstrated above is already in `train.py` in MM repositories. Simply modify the corresponding keys in the configuration files and the script will execute the expected workflow automatically.\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/utils.md", "content": "## Utils\n\n### ProgressBar\n\nIf you want to apply a method to a list of items and track the progress, `track_progress`\nis a good choice. It will display a progress bar to tell the progress and ETA.\n\n```python\nimport mmcv\n\ndef func(item):\n # do something\n pass\n\ntasks = [item_1, item_2, ..., item_n]\n\nmmcv.track_progress(func, tasks)\n```\n\nThe output is like the following.\n\n![progress](../_static/progress.*)\n\nThere is another method `track_parallel_progress`, which wraps multiprocessing and\nprogress visualization.\n\n```python\nmmcv.track_parallel_progress(func, tasks, 8) # 8 workers\n```\n\n![progress](../_static/parallel_progress.*)\n\nIf you want to iterate or enumerate a list of items and track the progress, `track_iter_progress`\nis a good choice. It will display a progress bar to tell the progress and ETA.\n\n```python\nimport mmcv\n\ntasks = [item_1, item_2, ..., item_n]\n\nfor task in mmcv.track_iter_progress(tasks):\n # do something like print\n print(task)\n\nfor i, task in enumerate(mmcv.track_iter_progress(tasks)):\n # do something like print\n print(i)\n print(task)\n```\n\n### Timer\n\nIt is convenient to compute the runtime of a code block with `Timer`.\n\n```python\nimport time\n\nwith mmcv.Timer():\n # simulate some code block\n time.sleep(1)\n```\n\nor try with `since_start()` and `since_last_check()`. This former can\nreturn the runtime since the timer starts and the latter will return the time\nsince the last time checked.\n\n```python\ntimer = mmcv.Timer()\n# code block 1 here\nprint(timer.since_start())\n# code block 2 here\nprint(timer.since_last_check())\nprint(timer.since_start())\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/en/understand_mmcv/visualization.md", "content": "## Visualization\n\n`mmcv` can show images and annotations (currently supported types include bounding boxes).\n\n```python\n# show an image file\nmmcv.imshow('a.jpg')\n\n# show a loaded image\nimg = np.random.rand(100, 100, 3)\nmmcv.imshow(img)\n\n# show image with bounding boxes\nimg = np.random.rand(100, 100, 3)\nbboxes = np.array([[0, 0, 50, 50], [20, 20, 60, 60]])\nmmcv.imshow_bboxes(img, bboxes)\n```\n\n`mmcv` can also visualize special images such as optical flows.\n\n```python\nflow = mmcv.flowread('test.flo')\nmmcv.flowshow(flow)\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/Makefile", "content": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD = sphinx-build\nSOURCEDIR = .\nBUILDDIR = _build\n\n# Put it first so that \"make\" without argument is like \"make help\".\nhelp:\n\t@$(SPHINXBUILD) -M help \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n\n.PHONY: help Makefile\n\n# Catch-all target: route all unknown targets to Sphinx using the new\n# \"make mode\" option. $(O) is meant as a shortcut for $(SPHINXOPTS).\n%: Makefile\n\t@$(SPHINXBUILD) -M $@ \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/_static/css/readthedocs.css", "content": ".header-logo {\n background-image: url(\"../image/mmcv-logo.png\");\n background-size: 85px 40px;\n height: 40px;\n width: 85px;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/api.rst", "content": "fileio\n-------\n.. automodule:: mmcv.fileio\n :members:\n\nimage\n------\n.. automodule:: mmcv.image\n :members:\n\nvideo\n------\n.. automodule:: mmcv.video\n :members:\n\narraymisc\n---------\n.. automodule:: mmcv.arraymisc\n :members:\n\nvisualization\n--------------\n.. automodule:: mmcv.visualization\n :members:\n\nutils\n-----\n.. automodule:: mmcv.utils\n :members:\n\ncnn\n----\n.. automodule:: mmcv.cnn\n :members:\n\nrunner\n------\n.. automodule:: mmcv.runner\n :members:\n\nengine\n------\n.. automodule:: mmcv.engine\n :members:\n\nops\n------\n.. automodule:: mmcv.ops\n :members:\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/community/contributing.md", "content": "## 贡献代码\n\n欢迎任何类型的贡献，包括但不限于\n\n- 修改拼写错误或代码错误\n- 添加文档或将文档翻译成其他语言\n- 添加新功能和新组件\n\n### 工作流\n| 详细工作流见 [拉取请求](pr.md)\n1. 复刻并拉取最新的 OpenMMLab 算法库\n2. 创建新的分支（不建议使用主分支提拉取请求）\n3. 提交你的修改\n4. 创建拉取请求\n\n```{note}\n如果你计划添加新功能并且该功能包含比较大的改动，建议先开 issue 讨论\n```\n### 代码风格\n\n#### Python\n\n[PEP8](https://www.python.org/dev/peps/pep-0008/) 作为 OpenMMLab 算法库首选的代码规范，我们使用以下工具检查和格式化代码\n\n- [flake8](http://flake8.pycqa.org/en/latest/): Python 官方发布的代码规范检查工具，是多个检查工具的封装\n- [yapf](https://github.com/google/yapf): Google 发布的代码规范检查工具\n- [isort](https://github.com/timothycrosley/isort): 自动调整模块导入顺序的工具\n- [markdownlint](https://github.com/markdownlint/markdownlint): 检查 markdown 文件的工具\n- [docformatter](https://github.com/myint/docformatter): 格式化 docstring 的工具\n\nyapf 和 isort 的配置可以在 [setup.cfg](./setup.cfg) 找到\n\n通过配置 [pre-commit hook](https://pre-commit.com/) ，我们可以在提交代码时自动检查和格式化 `flake8`、`yapf`、`isort`、`trailing whitespaces`、`markdown files`，\n修复 `end-of-files`、`double-quoted-strings`、`python-encoding-pragma`、`mixed-line-ending`，调整 `requirments.txt` 的包顺序。\npre-commit 钩子的配置可以在 [.pre-commit-config](./.pre-commit-config.yaml) 找到。\n\n在克隆算法库后，你需要安装并初始化 pre-commit 钩子\n\n```shell\npip install -U pre-commit\n```\n\n切换算法库根目录\n\n```shell\npre-commit install\n```\n\n如果安装 markdownlint 遇到了问题，可以尝试使用以下的步骤安装 ruby\n\n```shell\n# install rvm\ncurl -L https://get.rvm.io | bash -s -- --autolibs=read-fail\n[[ -s \"$HOME/.rvm/scripts/rvm\" ]] && source \"$HOME/.rvm/scripts/rvm\"\nrvm autolibs disable\n\n# install ruby\nrvm install 2.7.1\n```\n\n或者参考 [这个代码库](https://github.com/innerlee/setup) 和 [`zzruby.sh`](https://github.com/innerlee/setup/blob/master/zzruby.sh)。\n\n至此，每一次 commit 修改都会触发 pre-commit 检查代码格式。\n\n>提交拉取请求前，请确保你的代码符合 yapf 的格式\n\n#### C++ and CUDA\n\nC++ 和 CUDA 的代码规范遵从 [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/community/pr.md", "content": "## 拉取请求\n\n### 什么是拉取请求？\n\n`拉取请求` (Pull Request), [GitHub 官方文档](https://docs.github.com/en/github/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests)定义如下。\n\n```\n拉取请求是一种通知机制。你修改了他人的代码，将你的修改通知原来作者，希望他合并你的修改。\n```\n\n### 基本的工作流：\n\n1. 获取最新的代码库\n2. 从主分支创建最新的分支进行开发\n3. 提交修改\n4. 推送你的修改并创建一个 `拉取请求`\n5. 讨论、审核代码\n6. 将开发分支合并到主分支\n\n### 具体步骤\n\n#### 1. 获取最新的代码库\n\n+ 当你第一次提 PR 时\n\n 复刻 OpenMMLab 原代码库，点击 GitHub 页面右上角的 **Fork** 按钮即可\n ![avatar](../../en/_static/community/1.png)\n\n 克隆复刻的代码库到本地\n\n ```bash\n git clone git@github.com:XXX/mmcv.git\n ```\n\n 添加原代码库为上游代码库\n\n ```bash\n git remote add upstream git@github.com:open-mmlab/mmcv\n ```\n\n+ 从第二个 PR 起\n\n 检出本地代码库的主分支，然后从最新的原代码库的主分支拉取更新\n\n ```bash\n git checkout master\n git pull upstream master\n ```\n\n#### 2. 从主分支创建一个新的开发分支\n\n```bash\ngit checkout -b branchname\n```\n\n```{tip}\n为了保证提交历史清晰可读，我们强烈推荐您先检出主分支 (master)，再创建新的分支。\n```\n#### 3. 提交你的修改\n\n```bash\n# coding\ngit add [files]\ngit commit -m 'messages'\n```\n\n#### 4. 推送你的修改到复刻的代码库，并创建一个`拉取请求`\n\n+ 推送当前分支到远端复刻的代码库\n\n ```bash\n git push origin branchname\n ```\n\n+ 创建一个`拉取请求`\n![avatar](../../en/_static/community/2.png)\n\n+ 修改`拉取请求`信息模板，描述修改原因和修改内容。还可以在 PR 描述中，手动关联到相关的`议题` (issue),（更多细节，请参考[官方文档](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue)）。\n\n#### 5. 讨论并评审你的代码\n\n+ 创建`拉取请求`时，可以关联给相关人员进行评审\n![avatar](../../en/_static/community/3.png)\n\n+ 根据评审人员的意见修改代码，并推送修改\n\n#### 6. `拉取请求`合并之后删除该分支\n\n```bash\ngit branch -d branchname # delete local branch\ngit push origin --delete branchname # delete remote branch\n```\n\n### PR 规范\n\n1. 使用 [pre-commit hook](https://pre-commit.com)，尽量减少代码风格相关问题\n\n2. 一个 PR 对应一个短期分支\n\n3. 粒度要细，一个PR只做一件事情，避免超大的PR\n\n + Bad：实现 Faster R-CNN\n + Acceptable：给 Faster R-CNN 添加一个 box head\n + Good：给 box head 增加一个参数来支持自定义的 conv 层数\n\n4. 每次 Commit 时需要提供清晰且有意义 commit 信息\n\n5. 提供清晰且有意义的`拉取请求`描述\n\n + 标题写明白任务名称，一般格式:[Prefix] Short description of the pull request (Suffix)\n + prefix: 新增功能 [Feature], 修 bug [Fix], 文档相关 [Docs], 开发中 [WIP] (暂时不会被review)\n + 描述里介绍`拉取请求`的主要修改内容，结果，以及对其他部分的影响, 参考`拉取请求`模板\n + 关联相关的`议题` (issue) 和其他`拉取请求`\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/compatibility.md", "content": "### v1.3.18\n\n部分自定义算子对于不同的设备有不同实现，为此添加的大量宏命令与类型检查使得代码变得难以维护。例如：\n\n```c++\n if (input.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n CHECK_CUDA_INPUT(input);\n CHECK_CUDA_INPUT(rois);\n CHECK_CUDA_INPUT(output);\n CHECK_CUDA_INPUT(argmax_y);\n CHECK_CUDA_INPUT(argmax_x);\n\n roi_align_forward_cuda(input, rois, output, argmax_y, argmax_x,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n#else\n AT_ERROR(\"RoIAlign is not compiled with GPU support\");\n#endif\n } else {\n CHECK_CPU_INPUT(input);\n CHECK_CPU_INPUT(rois);\n CHECK_CPU_INPUT(output);\n CHECK_CPU_INPUT(argmax_y);\n CHECK_CPU_INPUT(argmax_x);\n roi_align_forward_cpu(input, rois, output, argmax_y, argmax_x,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n }\n```\n\n为此我们设计了注册与分发的机制以更好的管理这些算子实现。\n\n```c++\n\nvoid ROIAlignForwardCUDAKernelLauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nvoid roi_align_forward_cuda(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n ROIAlignForwardCUDAKernelLauncher(\n input, rois, output, argmax_y, argmax_x, aligned_height, aligned_width,\n spatial_scale, sampling_ratio, pool_mode, aligned);\n}\n\n// 注册算子的cuda实现\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\nREGISTER_DEVICE_IMPL(roi_align_forward_impl, CUDA, roi_align_forward_cuda);\n\n// roi_align.cpp\n// 使用dispatcher根据参数中的Tensor device类型对实现进行分发\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n DISPATCH_DEVICE_IMPL(roi_align_forward_impl, input, rois, output, argmax_y,\n argmax_x, aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\n```\n\n### v1.3.11\n\n为了灵活地支持更多的后端和硬件，例如 `NVIDIA GPUs` 、`AMD GPUs`，我们重构了 `mmcv/ops/csrc` 目录。注意，这次重构不会影响 API 的使用。更多相关信息，请参考 [PR1206](https://github.com/open-mmlab/mmcv/pull/1206)。\n\n原始的目录结构如下所示\n\n```\n.\n├── common_cuda_helper.hpp\n├── ops_cuda_kernel.cuh\n├── pytorch_cpp_helper.hpp\n├── pytorch_cuda_helper.hpp\n├── parrots_cpp_helper.hpp\n├── parrots_cuda_helper.hpp\n├── parrots_cudawarpfunction.cuh\n├── onnxruntime\n│   ├── onnxruntime_register.h\n│   ├── onnxruntime_session_options_config_keys.h\n│   ├── ort_mmcv_utils.h\n│   ├── ...\n│   ├── onnx_ops.h\n│   └── cpu\n│ ├── onnxruntime_register.cpp\n│      ├── ...\n│      └── onnx_ops_impl.cpp\n├── parrots\n│   ├── ...\n│   ├── ops.cpp\n│   ├── ops_cuda.cu\n│   ├── ops_parrots.cpp\n│   └── ops_pytorch.h\n├── pytorch\n│   ├── ...\n│   ├── ops.cpp\n│   ├── ops_cuda.cu\n│   ├── pybind.cpp\n└── tensorrt\n ├── trt_cuda_helper.cuh\n ├── trt_plugin_helper.hpp\n ├── trt_plugin.hpp\n ├── trt_serialize.hpp\n ├── ...\n ├── trt_ops.hpp\n └── plugins\n    ├── trt_cuda_helper.cu\n    ├── trt_plugin.cpp\n    ├── ...\n    ├── trt_ops.cpp\n    └── trt_ops_kernel.cu\n```\n\n重构之后，它的结构如下所示\n\n```\n.\n├── common\n│ ├── box_iou_rotated_utils.hpp\n│ ├── parrots_cpp_helper.hpp\n│ ├── parrots_cuda_helper.hpp\n│ ├── pytorch_cpp_helper.hpp\n│ ├── pytorch_cuda_helper.hpp\n│   └── cuda\n│   ├── common_cuda_helper.hpp\n│   ├── parrots_cudawarpfunction.cuh\n│   ├── ...\n│   └── ops_cuda_kernel.cuh\n├── onnxruntime\n│   ├── onnxruntime_register.h\n│   ├── onnxruntime_session_options_config_keys.h\n│   ├── ort_mmcv_utils.h\n│   ├── ...\n│   ├── onnx_ops.h\n│   └── cpu\n│ ├── onnxruntime_register.cpp\n│      ├── ...\n│      └── onnx_ops_impl.cpp\n├── parrots\n│   ├── ...\n│   ├── ops.cpp\n│   ├── ops_parrots.cpp\n│   └── ops_pytorch.h\n├── pytorch\n│   ├── info.cpp\n│   ├── pybind.cpp\n│   ├── ...\n│   ├── ops.cpp\n│   └── cuda\n│      ├── ...\n│      └── ops_cuda.cu\n└── tensorrt\n ├── trt_cuda_helper.cuh\n ├── trt_plugin_helper.hpp\n ├── trt_plugin.hpp\n ├── trt_serialize.hpp\n ├── ...\n ├── trt_ops.hpp\n └── plugins\n    ├── trt_cuda_helper.cu\n    ├── trt_plugin.cpp\n    ├── ...\n    ├── trt_ops.cpp\n    └── trt_ops_kernel.cu\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/conf.py", "content": "#\n# Configuration file for the Sphinx documentation builder.\n#\n# This file does only contain a selection of the most common options. For a\n# full list see the documentation:\n# http://www.sphinx-doc.org/en/master/config\n\n# -- Path setup --------------------------------------------------------------\n\n# If extensions (or modules to document with autodoc) are in another directory,\n# add these directories to sys.path here. If the directory is relative to the\n# documentation root, use os.path.abspath to make it absolute, like shown here.\n#\nimport os\nimport sys\n\nimport pytorch_sphinx_theme\nfrom sphinx.builders.html import StandaloneHTMLBuilder\n\nsys.path.insert(0, os.path.abspath('../..'))\n\nversion_file = '../../mmcv/version.py'\nwith open(version_file, 'r') as f:\n exec(compile(f.read(), version_file, 'exec'))\n__version__ = locals()['__version__']\n\n# -- Project information -----------------------------------------------------\n\nproject = 'mmcv'\ncopyright = '2018-2021, OpenMMLab'\nauthor = 'MMCV Authors'\n\n# The short X.Y version\nversion = __version__\n# The full version, including alpha/beta/rc tags\nrelease = __version__\n\n# -- General configuration ---------------------------------------------------\n\n# If your documentation needs a minimal Sphinx version, state it here.\n#\n# needs_sphinx = '1.0'\n\n# Add any Sphinx extension module names here, as strings. They can be\n# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom\n# ones.\n\nextensions = [\n 'sphinx.ext.autodoc',\n 'sphinx.ext.napoleon',\n 'sphinx.ext.viewcode',\n 'sphinx.ext.autosectionlabel',\n 'sphinx_markdown_tables',\n 'myst_parser',\n 'sphinx_copybutton',\n] # yapf: disable\n\nautodoc_mock_imports = ['mmcv._ext', 'mmcv.utils.ext_loader', 'torchvision']\nautosectionlabel_prefix_document = True\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = ['_templates']\n\n# The suffix(es) of source filenames.\n# You can specify multiple suffix as a list of string:\n#\nsource_suffix = {\n '.rst': 'restructuredtext',\n '.md': 'markdown',\n}\n\n# The master toctree document.\nmaster_doc = 'index'\n\n# The language for content autogenerated by Sphinx. Refer to documentation\n# for a list of supported languages.\n#\n# This is also used if you do content translation via gettext catalogs.\n# Usually you set \"language\" from the command line for these cases.\nlanguage = 'zh_CN'\n\n# List of patterns, relative to source directory, that match files and\n# directories to ignore when looking for source files.\n# This pattern also affects html_static_path and html_extra_path.\nexclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']\n\n# The name of the Pygments (syntax highlighting) style to use.\npygments_style = 'sphinx'\n\n# -- Options for HTML output -------------------------------------------------\n\n# The theme to use for HTML and HTML Help pages. See the documentation for\n# a list of builtin themes.\n#\n# html_theme = 'sphinx_rtd_theme'\nhtml_theme = 'pytorch_sphinx_theme'\nhtml_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]\n\n# Theme options are theme-specific and customize the look and feel of a theme\n# further. For a list of options available for each theme, see the\n# documentation.\n#\nhtml_theme_options = {\n 'menu': [\n {\n 'name': 'GitHub',\n 'url': 'https://github.com/open-mmlab/mmcv'\n },\n ],\n # Specify the language of shared menu\n 'menu_lang': 'cn',\n}\n\n# Add any paths that contain custom static files (such as style sheets) here,\n# relative to this directory. They are copied after the builtin static files,\n# so a file named \"default.css\" will overwrite the builtin \"default.css\".\nhtml_static_path = ['_static']\nhtml_css_files = ['css/readthedocs.css']\n\n# Custom sidebar templates, must be a dictionary that maps document names\n# to template names.\n#\n# The default sidebars (for documents that don't match any pattern) are\n# defined by theme itself. Builtin themes are using these templates by\n# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',\n# 'searchbox.html']``.\n#\n# html_sidebars = {}\n\n# -- Options for HTMLHelp output ---------------------------------------------\n\n# Output file base name for HTML help builder.\nhtmlhelp_basename = 'mmcvdoc'\n\n# -- Options for LaTeX output ------------------------------------------------\n\nlatex_elements = {\n # The paper size ('letterpaper' or 'a4paper').\n #\n # 'papersize': 'letterpaper',\n\n # The font size ('10pt', '11pt' or '12pt').\n #\n # 'pointsize': '10pt',\n\n # Additional stuff for the LaTeX preamble.\n #\n # 'preamble': '',\n\n # Latex figure (float) alignment\n #\n # 'figure_align': 'htbp',\n}\n\n# Grouping the document tree into LaTeX files. List of tuples\n# (source start file, target name, title,\n# author, documentclass [howto, manual, or own class]).\nlatex_documents = [\n (master_doc, 'mmcv.tex', 'mmcv Documentation', 'MMCV Contributors',\n 'manual'),\n]\n\n# -- Options for manual page output ------------------------------------------\n\n# One entry per manual page. List of tuples\n# (source start file, name, description, authors, manual section).\nman_pages = [(master_doc, 'mmcv', 'mmcv Documentation', [author], 1)]\n\n# -- Options for Texinfo output ----------------------------------------------\n\n# Grouping the document tree into Texinfo files. List of tuples\n# (source start file, target name, title, author,\n# dir menu entry, description, category)\ntexinfo_documents = [\n (master_doc, 'mmcv', 'mmcv Documentation', author, 'mmcv',\n 'One line description of project.', 'Miscellaneous'),\n]\n\n# -- Options for Epub output -------------------------------------------------\n\n# Bibliographic Dublin Core info.\nepub_title = project\n\n# The unique identifier of the text. This can be a ISBN number\n# or the project homepage.\n#\n# epub_identifier = ''\n\n# A unique identification for the text.\n#\n# epub_uid = ''\n\n# A list of files that should not be packed into the epub file.\nepub_exclude_files = ['search.html']\n\n# set priority when building html\nStandaloneHTMLBuilder.supported_image_types = [\n 'image/svg+xml', 'image/gif', 'image/png', 'image/jpeg'\n]\n# -- Extension configuration -------------------------------------------------\n# Ignore >>> when copying code\ncopybutton_prompt_text = r'>>> |\\.\\.\\. '\ncopybutton_prompt_is_regexp = True\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/deployment/onnx.md", "content": "## MMCV中ONNX模块简介 (实验性)\n\n### register_extra_symbolics\n\n在将PyTorch模型导出成ONNX时，需要注册额外的符号函数\n\n#### 范例\n\n```python\nimport mmcv\nfrom mmcv.onnx import register_extra_symbolics\n\nopset_version = 11\nregister_extra_symbolics(opset_version)\n```\n\n#### 常见问题\n\n- 无\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/deployment/onnxruntime_custom_ops.md", "content": "## ONNX Runtime自定义算子\n\n\n\n- [ONNX Runtime自定义算子](#onnx-runtime自定义算子)\n - [SoftNMS](#softnms)\n - [描述](#描述)\n - [模型参数](#模型参数)\n - [输入](#输入)\n - [输出](#输出)\n - [类型约束](#类型约束)\n - [RoIAlign](#roialign)\n - [描述](#描述-1)\n - [模型参数](#模型参数-1)\n - [输入](#输入-1)\n - [输出](#输出-1)\n - [类型约束](#类型约束-1)\n - [NMS](#nms)\n - [描述](#描述-2)\n - [模型参数](#模型参数-2)\n - [输入](#输入-2)\n - [输出](#输出-2)\n - [类型约束](#类型约束-2)\n - [grid_sampler](#grid_sampler)\n - [描述](#描述-3)\n - [模型参数](#模型参数-3)\n - [输入](#输入-3)\n - [输出](#输出-3)\n - [类型约束](#类型约束-3)\n - [CornerPool](#cornerpool)\n - [描述](#描述-4)\n - [模型参数](#模型参数-4)\n - [输入](#输入-4)\n - [输出](#输出-4)\n - [类型约束](#类型约束-4)\n - [cummax](#cummax)\n - [描述](#描述-5)\n - [模型参数](#模型参数-5)\n - [输入](#输入-5)\n - [输出](#输出-5)\n - [类型约束](#类型约束-5)\n - [cummin](#cummin)\n - [描述](#描述-6)\n - [模型参数](#模型参数-6)\n - [输入](#输入-6)\n - [输出](#输出-6)\n - [类型约束](#类型约束-6)\n - [MMCVModulatedDeformConv2d](#mmcvmodulateddeformconv2d)\n - [描述](#描述-7)\n - [模型参数](#模型参数-7)\n - [输入](#输入-7)\n - [输出](#输出-7)\n - [类型约束](#类型约束-7)\n\n\n\n### SoftNMS\n\n#### 描述\n\n根据`scores`计算`boxes`的soft NMS。 请阅读[Soft-NMS -- Improving Object Detection With One Line of Code](https://arxiv.org/abs/1704.04503)了解细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ------- | --------------- | ------------------------------------------------------- |\n| `float` | `iou_threshold` | 用来判断候选框重合度的阈值，取值范围[0, 1]。默认值为0 |\n| `float` | `sigma` | 高斯方法的超参数 |\n| `float` | `min_score` | NMS的score阈值 |\n| `int` | `method` | NMS的计算方式, (0: `naive`, 1: `linear`, 2: `gaussian`) |\n| `int` | `offset` | 用来计算候选框的宽高(x2 - x1 + offset)。可选值0或1 |\n\n#### 输入\n\n
\n
boxes: T
\n
输入候选框。形状为(N, 4)的二维张量，N为候选框数量。
\n
scores: T
\n
输入得分。形状为(N, )的一维张量。
\n
\n\n#### 输出\n\n
\n
dets: T
\n
输出的检测框与得分。形状为(num_valid_boxes, 5)的二维张量，内容为[[x1, y1, x2, y2, score], ...]。num_valid_boxes是合法的检测框数量。
\n
indices: tensor(int64)
\n
输出序号。形状为(num_valid_boxes, )的一维张量。
\n
\n\n#### 类型约束\n\n- T:tensor(float32)\n\n### RoIAlign\n\n#### 描述\n\n在特征图上计算RoIAlign，通常在双阶段目标检测模型的bbox_head中使用\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ------- | ---------------- | ------------------------------------------------------- |\n| `int` | `output_height` | roi特征的输出高度 |\n| `int` | `output_width` | roi特征的输出宽度 |\n| `float` | `spatial_scale` | 输入检测框的缩放系数 |\n| `int` | `sampling_ratio` | 输出的采样率。`0`表示使用密集采样 |\n| `str` | `mode` | 池化方式。 `avg`或`max` |\n| `int` | `aligned` | 如果`aligned=1`，则像素会进行-0.5的偏移以达到更好的对齐 |\n\n#### 输入\n\n
\n
input: T
\n
输入特征图；形状为(N, C, H, W)的四维张量，其中N为batch大小，C为输入通道数，H和W为输入特征图的高和宽。
\n
rois: T
\n
需要进行池化的感兴趣区域；形状为(num_rois, 5)的二维张量，内容为[[batch_index, x1, y1, x2, y2], ...]。rois的坐标为输入特征图的坐标系。
\n
\n\n#### 输出\n\n
\n
feat: T
\n
池化的输出；形状为(num_rois, C, output_height, output_width)的四维张量。每个输出特征feat[i]都与输入感兴趣区域rois[i]一一对应。
\n
\n\n#### 类型约束\n\n- T:tensor(float32)\n\n### NMS\n\n#### 描述\n\n根据IoU阈值对候选框进行非极大值抑制。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ------- | --------------- | ----------------------------------------------------- |\n| `float` | `iou_threshold` | 用来判断候选框重合度的阈值，取值范围[0, 1]。默认值为0 |\n| `int` | `offset` | 用来计算候选框的宽高(x2 - x1 + offset)。可选值0或1 |\n\n#### 输入\n\n
\n
boxes: T
\n
输入候选框。形状为(N, 4)的二维张量，N为候选框数量。
\n
scores: T
\n
输入得分。形状为(N, )的一维张量。
\n
\n\n#### 输出\n\n
\n
indices: tensor(int32, Linear)
\n
被选中的候选框索引。形状为(num_valid_boxes, )的一维张量，num_valid_boxes表示被选上的候选框数量。
\n
\n\n#### 类型约束\n\n- T:tensor(float32)\n\n### grid_sampler\n\n#### 描述\n\n根据`grid`的像素位置对`input`进行网格采样。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ----- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `int` | `interpolation_mode` | 计算输出使用的插值模式。(0: `bilinear` , 1: `nearest`) |\n| `int` | `padding_mode` | 边缘填充模式。(0: `zeros`, 1: `border`, 2: `reflection`) |\n| `int` | `align_corners` | 如果`align_corners=1`，则极值(`-1`和`1`)会被当做输入边缘像素的中心点。如果`align_corners=0`，则它们会被看做是边缘像素的边缘点,减小分辨率对采样的影响 |\n\n#### 输入\n\n
\n
input: T
\n
输入特征；形状为(N, C, inH, inW)的四维张量，其中N为batch大小，C为输入通道数，inH和inW为输入特征图的高和宽。
\n
grid: T
\n
输入网格；形状为(N, outH, outW, 2)的四维张量，outH和outW为输出的高和宽。
\n
\n\n#### 输出\n\n
\n
output: T
\n
输出特征；形状为(N, C, outH, outW)的四维张量。
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n\n### CornerPool\n\n#### 描述\n\n对`input`计算CornerPool。请阅读[CornerNet -- Detecting Objects as Paired Keypoints](https://arxiv.org/abs/1808.01244)了解更多细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ----- | ------ | -------------------------------------------------------- |\n| `int` | `mode` | 池化模式。(0: `top`, 1: `bottom`, 2: `left`, 3: `right`) |\n\n#### 输入\n\n
\n
input: T
\n
输入特征；形状为(N, C, H, W)的四维张量，其中N为batch大小，C为输入通道数，H和W为输入特征图的高和宽。
\n
\n\n#### 输出\n\n
\n
output: T
\n
输出特征；形状为(N, C, H, W)的四维张量。
\n
\n\n#### 类型约束\n\n- T:tensor(float32)\n\n### cummax\n\n#### 描述\n\n返回一个元组(`values`, `indices`)，其中`values`为`input`第`dim`维的累计最大值，`indices`为第`dim`维最大值位置。请阅读[torch.cummax](https://pytorch.org/docs/stable/generated/torch.cummax.html)了解更多细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ----- | ------ | ------------------ |\n| `int` | `dim` | 进行累计计算的维度 |\n\n#### 输入\n\n
\n
input: T
\n
输入张量；可以使任意形状；也支持空Tensor
\n
\n\n#### 输出\n\n
\n
output: T
\n
`input`第`dim`维的累计最大值，形状与`input`相同。类型和`input`一致
\n
indices: tensor(int64)
\n
第`dim`维最大值位置，形状与`input`相同。
\n
\n\n#### 类型约束\n\n- T:tensor(float32)\n\n### cummin\n\n#### 描述\n\n返回一个元组(`values`, `indices`)，其中`values`为`input`第`dim`维的累计最小值，`indices`为第`dim`维最小值位置。请阅读[torch.cummin](https://pytorch.org/docs/stable/generated/torch.cummin.html)了解更多细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ----- | ------ | ------------------ |\n| `int` | `dim` | 进行累计计算的维度 |\n\n#### 输入\n\n
\n
input: T
\n
输入张量；可以是任意形状；也支持空Tensor
\n
\n\n#### 输出\n\n
\n
output: T
\n
`input`第`dim`维的累计最小值，形状与`input`相同。类型和`input`一致
\n
indices: tensor(int64)
\n
第`dim`维最小值位置，形状与`input`相同。
\n
\n\n#### 类型约束\n\n- T:tensor(float32)\n\n### MMCVModulatedDeformConv2d\n\n#### 描述\n\n在输入特征上计算Modulated Deformable Convolution，请阅读[Deformable ConvNets v2: More Deformable, Better Results](https://arxiv.org/abs/1811.11168?from=timeline)了解更多细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| -------------- | ------------------- | ------------------------------------------------------------- |\n| `list of ints` | `stride` | 卷积的步长 (sH, sW) |\n| `list of ints` | `padding` | 输入特征填充大小 (padH, padW) |\n| `list of ints` | `dilation` | 卷积核各元素间隔 (dH, dW) |\n| `int` | `deformable_groups` | 可变偏移量的分组，通常置位1即可 |\n| `int` | `groups` | 卷积分组数，`input_channel`会根据这个值被分为数个分组进行计算 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入特征；形状为(N, C, inH, inW)的四维张量，其中N为batch大小，C为输入通道数，inH和inW为输入特征图的高和宽。
\n
inputs[1]: T
\n
输入偏移量；形状为(N, deformable_group* 2* kH* kW, outH, outW)的四维张量，kH和kW为输入特征图的高和宽，outH和outW为输入特征图的高和宽。
\n
inputs[2]: T
\n
输入掩码；形状为(N, deformable_group* kH* kW, outH, outW)的四维张量。
\n
inputs[3]: T
\n
输入权重；形状为(output_channel, input_channel, kH, kW)的四维张量。
\n
inputs[4]: T, optional
\n
输入偏移量；形状为(output_channel)的一维张量。
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
输出特征；形状为(N, output_channel, outH, outW)的四维张量。
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/deployment/onnxruntime_op.md", "content": "## MMCV中的ONNX Runtime自定义算子\n\n### ONNX Runtime介绍\n\n**ONNX Runtime**是一个跨平台的推理与训练加速器，适配许多常用的机器学习/深度神经网络框架。请访问[github](https://github.com/microsoft/onnxruntime)了解更多信息。\n\n### ONNX介绍\n\n**ONNX**是**Open Neural Network Exchange**的缩写，是许多机器学习/深度神经网络框架使用的*中间表示(IR)*。请访问[github](https://github.com/onnx/onnx)了解更多信息。\n\n### 为什么要在MMCV中添加ONNX自定义算子？\n\n- 为了验证ONNX模型在ONNX Runtime下的推理的正确性。\n- 为了方便使用了`mmcv.ops`自定义算子的模型的部署工作。\n\n### MMCV已支持的算子\n\n| 算子 | CPU | GPU | MMCV版本 |\n| :------------------------------------------------------------------------------: | :---: | :---: | :------: |\n| [SoftNMS](onnxruntime_custom_ops.md#softnms) | Y | N | 1.2.3 |\n| [RoIAlign](onnxruntime_custom_ops.md#roialign) | Y | N | 1.2.5 |\n| [NMS](onnxruntime_custom_ops.md#nms) | Y | N | 1.2.7 |\n| [grid_sampler](onnxruntime_custom_ops.md#grid_sampler) | Y | N | 1.3.1 |\n| [CornerPool](onnxruntime_custom_ops.md#cornerpool) | Y | N | 1.3.4 |\n| [cummax](onnxruntime_custom_ops.md#cummax) | Y | N | 1.3.4 |\n| [cummin](onnxruntime_custom_ops.md#cummin) | Y | N | 1.3.4 |\n| [MMCVModulatedDeformConv2d](onnxruntime_custom_ops.md#mmcvmodulateddeformconv2d) | Y | N | 1.3.12 |\n\n### 如何编译ONNX Runtime自定义算子？\n\n*请注意我们仅在**onnxruntime>=1.8.1**的Linux x86-64 cpu平台上进行过测试*\n\n#### 准备工作\n\n- 克隆代码仓库\n\n```bash\ngit clone https://github.com/open-mmlab/mmcv.git\n```\n\n- 从ONNX Runtime下载`onnxruntime-linux`：[releases](https://github.com/microsoft/onnxruntime/releases/tag/v1.8.1)，解压缩，根据路径创建变量`ONNXRUNTIME_DIR`并把路径下的lib目录添加到`LD_LIBRARY_PATH`，步骤如下：\n\n```bash\nwget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-1.8.1.tgz\n\ntar -zxvf onnxruntime-linux-x64-1.8.1.tgz\ncd onnxruntime-linux-x64-1.8.1\nexport ONNXRUNTIME_DIR=$(pwd)\nexport LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH\n```\n\n#### Linux系统下编译\n\n```bash\ncd mmcv ## to MMCV root directory\nMMCV_WITH_OPS=1 MMCV_WITH_ORT=1 python setup.py develop\n```\n\n### 如何在python下使用ONNX Runtime对导出的ONNX模型做编译\n\n使用`pip`安装ONNX Runtime\n\n```bash\npip install onnxruntime==1.8.1\n```\n\n推理范例\n\n```python\nimport os\n\nimport numpy as np\nimport onnxruntime as ort\n\nfrom mmcv.ops import get_onnxruntime_op_path\n\nort_custom_op_path = get_onnxruntime_op_path()\nassert os.path.exists(ort_custom_op_path)\nsession_options = ort.SessionOptions()\nsession_options.register_custom_ops_library(ort_custom_op_path)\n## exported ONNX model with custom operators\nonnx_file = 'sample.onnx'\ninput_data = np.random.randn(1, 3, 224, 224).astype(np.float32)\nsess = ort.InferenceSession(onnx_file, session_options)\nonnx_results = sess.run(None, {'input' : input_data})\n```\n\n### 如何为MMCV添加ONNX Runtime的自定义算子\n\n#### 开发前提醒\n\n- 该算子的ONNX Runtime实现尚未在MMCV中支持[已实现算子列表](https://github.com/microsoft/onnxruntime/blob/master/docs/OperatorKernels.md)。\n- 确保该自定义算子可以被ONNX导出。\n\n#### 添加方法\n\n以`soft_nms`为例：\n\n1. 在ONNX Runtime头文件目录`mmcv/ops/csrc/onnxruntime/`下添加头文件`soft_nms.h`\n2. 在ONNX Runtime源码目录`mmcv/ops/csrc/onnxruntime/cpu/`下添加算子实现`soft_nms.cpp`\n3. 在[onnxruntime_register.cpp](../../mmcv/ops/csrc/onnxruntime/cpu/onnxruntime_register.cpp)中注册实现的算子`soft_nms`\n\n ```c++\n #include \"soft_nms.h\"\n\n SoftNmsOp c_SoftNmsOp;\n\n if (auto status = ortApi->CustomOpDomain_Add(domain, &c_SoftNmsOp)) {\n return status;\n }\n ```\n\n4. 在`tests/test_ops/test_onnx.py`添加单元测试，\n 可以参考[here](../../tests/test_ops/test_onnx.py)。\n\n**最后，欢迎为MMCV添加ONNX Runtime自定义算子** :nerd_face:\n\n### 已知问题\n\n- \"RuntimeError: tuple appears in op that does not forward tuples, unsupported kind: `prim::PythonOp`.\"\n 1. 请注意`cummax`和`cummin`算子是在torch >= 1.5.0被添加的。但他们需要在torch version >= 1.7.0才能正确导出。否则会在导出时发生上面的错误。\n 2. 解决方法：升级PyTorch到1.7.0以上版本\n\n### 引用\n\n- [How to export Pytorch model with custom op to ONNX and run it in ONNX Runtime](https://github.com/onnx/tutorials/blob/master/PyTorchCustomOperator/README.md)\n- [How to add a custom operator/kernel in ONNX Runtime](https://github.com/microsoft/onnxruntime/blob/master/docs/AddingCustomOp.md)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/deployment/tensorrt_custom_ops.md", "content": "## TensorRT自定义算子\n\n\n\n- [TensorRT自定义算子](#tensorrt自定义算子)\n - [MMCVRoIAlign](#mmcvroialign)\n - [描述](#描述)\n - [模型参数](#模型参数)\n - [输入](#输入)\n - [输出](#输出)\n - [类型约束](#类型约束)\n - [ScatterND](#scatternd)\n - [描述](#描述-1)\n - [模型参数](#模型参数-1)\n - [输入](#输入-1)\n - [输出](#输出-1)\n - [类型约束](#类型约束-1)\n - [NonMaxSuppression](#nonmaxsuppression)\n - [描述](#描述-2)\n - [模型参数](#模型参数-2)\n - [输入](#输入-2)\n - [输出](#输出-2)\n - [类型约束](#类型约束-2)\n - [MMCVDeformConv2d](#mmcvdeformconv2d)\n - [描述](#描述-3)\n - [模型参数](#模型参数-3)\n - [输入](#输入-3)\n - [输出](#输出-3)\n - [类型约束](#类型约束-3)\n - [grid_sampler](#grid_sampler)\n - [描述](#描述-4)\n - [模型参数](#模型参数-4)\n - [输入](#输入-4)\n - [输出](#输出-4)\n - [类型约束](#类型约束-4)\n - [cummax](#cummax)\n - [描述](#描述-5)\n - [模型参数](#模型参数-5)\n - [输入](#输入-5)\n - [输出](#输出-5)\n - [类型约束](#类型约束-5)\n - [cummin](#cummin)\n - [描述](#描述-6)\n - [模型参数](#模型参数-6)\n - [输入](#输入-6)\n - [输出](#输出-6)\n - [类型约束](#类型约束-6)\n - [MMCVInstanceNormalization](#mmcvinstancenormalization)\n - [描述](#描述-7)\n - [模型参数](#模型参数-7)\n - [输入](#输入-7)\n - [输出](#输出-7)\n - [类型约束](#类型约束-7)\n - [MMCVModulatedDeformConv2d](#mmcvmodulateddeformconv2d)\n - [描述](#描述-8)\n - [模型参数](#模型参数-8)\n - [输入](#输入-8)\n - [输出](#输出-8)\n - [类型约束](#类型约束-8)\n\n\n\n### MMCVRoIAlign\n\n#### 描述\n\n在特征图上计算RoIAlign，在多数双阶段目标检测模型的bbox_head中使用\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ------- | ---------------- | ------------------------------------------------------- |\n| `int` | `output_height` | roi特征的输出高度 |\n| `int` | `output_width` | roi特征的输出宽度 |\n| `float` | `spatial_scale` | 输入检测框的缩放系数 |\n| `int` | `sampling_ratio` | 输出的采样率。`0`表示使用密集采样 |\n| `str` | `mode` | 池化方式。 `avg`或`max` |\n| `int` | `aligned` | 如果`aligned=1`，则像素会进行-0.5的偏移以达到更好的对齐 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入特征图；形状为(N, C, H, W)的四维张量，其中N为batch大小，C为输入通道数，H和W为输入特征图的高和宽。
\n
inputs[1]: T
\n
需要进行池化的感兴趣区域；形状为(num_rois, 5)的二维张量，内容为[[batch_index, x1, y1, x2, y2], ...]。rois的坐标为输入特征图的坐标系。
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
池化的输出；形状为(num_rois, C, output_height, output_width)的四维张量。每个输出特征feat[i]都与输入感兴趣区域rois[i]一一对应。
\n
\n#### 类型约束\n\n- T:tensor(float32, Linear)\n\n### ScatterND\n\n#### 描述\n\nScatterND接收三个输入，分别为秩为r >= 1的`data`，秩为q >= 1的`indices`以及秩为 q + r - indices.shape[-1] -1 的`update`。输出的计算方式为：首先创建一个`data`的拷贝，然后根据`indces`的值使用`update`对拷贝的`data`进行更新。注意`indices`中不应该存在相同的条目，也就是说对同一个位置进行一次以上的更新是不允许的。\n\n输出的计算方式可以参考如下代码：\n\n```python\n output = np.copy(data)\n update_indices = indices.shape[:-1]\n for idx in np.ndindex(update_indices):\n output[indices[idx]] = updates[idx]\n```\n\n#### 模型参数\n\n无\n\n#### 输入\n\n
\n
inputs[0]: T
\n
秩为r >= 1的输入`data`
\n\n
inputs[1]: tensor(int32, Linear)
\n
秩为q >= 1的输入`update`
\n\n
inputs[2]: T
\n
秩为 q + r - indices.shape[-1] -1 的输入`update`
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
秩为r >= 1的输出张量
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear), tensor(int32, Linear)\n\n### NonMaxSuppression\n\n#### 描述\n\n根据IoU阈值对候选框进行非极大值抑制。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ------- | ---------------------------- | ---------------------------------------------------------------------------------------- |\n| `int` | `center_point_box` | 0 - 候选框的格式为[y1, x1, y2, x2]， 1-候选框的格式为[x_center, y_center, width, height] |\n| `int` | `max_output_boxes_per_class` | 每一类最大的输出检测框个数。默认为0，输出检测框个数等于输入候选框数 |\n| `float` | `iou_threshold` | 用来判断候选框重合度的阈值，取值范围[0, 1]。默认值为0 |\n| `float` | `score_threshold` | 用来判断候选框是否合法的阈值 |\n| `int` | `offset` | 检测框长宽计算方式为(x2 - x1 + offset)，可选值0或1 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入候选框。形状为(num_batches, spatial_dimension, 4)的三维张量
\n
inputs[1]: T
\n
输入得分。形状为(num_batches, num_classes, spatial_dimension)的三维张量
\n
\n\n#### 输出\n\n
\n
outputs[0]: tensor(int32, Linear)
\n
被选中的候选框索引。形状为(num_selected_indices, 3)的二维张量。每一行内容为[batch_index, class_index, box_index]。
\n
其中 num_selected_indices=num_batches* num_classes* min(max_output_boxes_per_class, spatial_dimension)。
\n
所有未被选中的候选框索引都会被填充为-1
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n\n### MMCVDeformConv2d\n\n#### 描述\n\n在输入特征上计算Deformable Convolution，请阅读[Deformable Convolutional Network](https://arxiv.org/abs/1703.06211)了解更多细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| -------------- | ------------------ | --------------------------------------------------------------------------------------------- |\n| `list of ints` | `stride` | 卷积的步长 (sH, sW) |\n| `list of ints` | `padding` | 输入特征填充大小 (padH, padW) |\n| `list of ints` | `dilation` | 卷积核各元素间隔 (dH, dW) |\n| `int` | `deformable_group` | 可变偏移量的分组 |\n| `int` | `group` | 卷积分组数，`input_channel`会根据这个值被分为数个分组进行计算 |\n| `int` | `im2col_step` | 可变卷积使用im2col计算卷积。输入与偏移量会以im2col_step为步长分块计算，减少临时空间的使用量。 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入特征；形状为(N, C, inH, inW)的四维张量，其中N为batch大小，C为输入通道数，inH和inW为输入特征图的高和宽
\n
inputs[1]: T
\n
输入偏移量；形状为(N, deformable_group* 2* kH* kW, outH, outW)的四维张量，kH和kW为输入特征图的高和宽，outH和outW为输入特征图的高和宽
\n
inputs[2]: T
\n
输入权重；形状为(output_channel, input_channel, kH, kW)的四维张量
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
输出特征；形状为(N, output_channel, outH, outW)的四维张量
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n\n### grid_sampler\n\n#### 描述\n\n根据`grid`的像素位置对`input`进行网格采样。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ----- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `int` | `interpolation_mode` | 计算输出使用的插值模式。(0: `bilinear` , 1: `nearest`) |\n| `int` | `padding_mode` | 边缘填充模式。(0: `zeros`, 1: `border`, 2: `reflection`) |\n| `int` | `align_corners` | 如果`align_corners=1`，则极值(`-1`和`1`)会被当做输入边缘像素的中心点。如果`align_corners=0`，则它们会被看做是边缘像素的边缘点,减小分辨率对采样的影响 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入特征；形状为(N, C, inH, inW)的四维张量，其中N为batch大小，C为输入通道数，inH和inW为输入特征图的高和宽
\n
inputs[1]: T
\n
输入网格；形状为(N, outH, outW, 2)的四维张量，outH和outW为输出的高和宽
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
输出特征；形状为(N, C, outH, outW)的四维张量
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n\n### cummax\n\n#### 描述\n\n返回一个元组(`values`, `indices`)，其中`values`为`input`第`dim`维的累计最大值，`indices`为第`dim`维最大值位置。请阅读[torch.cummax](https://pytorch.org/docs/stable/generated/torch.cummax.html)了解更多细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ----- | ------ | ------------------ |\n| `int` | `dim` | 进行累计计算的维度 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入张量；可以使任意形状
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
`input`第`dim`维的累计最大值，形状与`input`相同。类型和`input`一致
\n
outputs[1]: (int32, Linear)
\n
第`dim`维最大值位置，形状与`input`相同
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n\n### cummin\n\n#### 描述\n\n返回一个元组(`values`, `indices`)，其中`values`为`input`第`dim`维的累计最小值，`indices`为第`dim`维最小值位置。请阅读[torch.cummin](https://pytorch.org/docs/stable/generated/torch.cummin.html)了解更多细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ----- | ------ | ------------------ |\n| `int` | `dim` | 进行累计计算的维度 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入张量；可以使任意形状
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
`input`第`dim`维的累计最小值，形状与`input`相同。类型和`input`一致
\n
outputs[1]: (int32, Linear)
\n
第`dim`维最小值位置，形状与`input`相同
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n\n### MMCVInstanceNormalization\n\n#### 描述\n\n对特征计算instance normalization，请阅读[Instance Normalization: The Missing Ingredient for Fast Stylization](https://arxiv.org/abs/1607.08022)了解更多详细信息。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| ------- | --------- | ---------------------------- |\n| `float` | `epsilon` | 用来避免除0错误。默认为1e-05 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入特征。形状为(N, C, H， W)的四维张量，其中N为batch大小，C为输入通道数，H和W为输入特征图的高和宽
\n
inputs[1]: T
\n
输入缩放系数。形状为(C，)的一维张量
\n
inputs[2]: T
\n
输入偏移量。形状为(C，)的一维张量
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
输出特征。形状为(N, C, H， W)的四维张量
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n\n### MMCVModulatedDeformConv2d\n\n#### 描述\n\n在输入特征上计算Modulated Deformable Convolution，请阅读[Deformable ConvNets v2: More Deformable, Better Results](https://arxiv.org/abs/1811.11168?from=timeline)了解更多细节。\n\n#### 模型参数\n\n| 类型 | 参数名 | 描述 |\n| -------------- | ------------------- | ------------------------------------------------------------- |\n| `list of ints` | `stride` | 卷积的步长 (sH, sW) |\n| `list of ints` | `padding` | 输入特征填充大小 (padH, padW) |\n| `list of ints` | `dilation` | 卷积核各元素间隔 (dH, dW) |\n| `int` | `deformable_groups` | 可变偏移量的分组，通常置位1即可 |\n| `int` | `groups` | 卷积分组数，`input_channel`会根据这个值被分为数个分组进行计算 |\n\n#### 输入\n\n
\n
inputs[0]: T
\n
输入特征；形状为(N, C, inH, inW)的四维张量，其中N为batch大小，C为输入通道数，inH和inW为输入特征图的高和宽
\n
inputs[1]: T
\n
输入偏移量；形状为(N, deformable_group* 2* kH* kW, outH, outW)的四维张量，kH和kW为输入特征图的高和宽，outH和outW为输入特征图的高和宽
\n
inputs[2]: T
\n
输入掩码；形状为(N, deformable_group* kH* kW, outH, outW)的四维张量
\n
inputs[3]: T
\n
输入权重；形状为(output_channel, input_channel, kH, kW)的四维张量
\n
inputs[4]: T, optional
\n
输入偏移量；形状为(output_channel)的一维张量
\n
\n\n#### 输出\n\n
\n
outputs[0]: T
\n
输出特征；形状为(N, output_channel, outH, outW)的四维张量
\n
\n\n#### 类型约束\n\n- T:tensor(float32, Linear)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/deployment/tensorrt_plugin.md", "content": "## MMCV中的TensorRT自定义算子 (实验性)\n\n\n\n- [MMCV中的TensorRT自定义算子 (实验性)](#mmcv中的tensorrt自定义算子-实验性)\n - [介绍](#介绍)\n - [MMCV中的TensorRT插件列表](#mmcv中的tensorrt插件列表)\n - [如何编译MMCV中的TensorRT插件](#如何编译mmcv中的tensorrt插件)\n - [准备](#准备)\n - [在Linux上编译](#在linux上编译)\n - [创建TensorRT推理引擎并在python下进行推理](#创建tensorrt推理引擎并在python下进行推理)\n - [如何在MMCV中添加新的TensorRT自定义算子](#如何在mmcv中添加新的tensorrt自定义算子)\n - [主要流程](#主要流程)\n - [注意](#注意)\n - [已知问题](#已知问题)\n - [引用](#引用)\n\n\n\n### 介绍\n\n**NVIDIA TensorRT**是一个为深度学习模型高性能推理准备的软件开发工具(SDK)。它包括深度学习推理优化器和运行时，可为深度学习推理应用提供低延迟和高吞吐量。请访问[developer's website](https://developer.nvidia.com/tensorrt)了解更多信息。\n为了简化TensorRT部署带有MMCV自定义算子的模型的流程，MMCV中添加了一系列TensorRT插件。\n\n### MMCV中的TensorRT插件列表\n\n| ONNX算子 | TensorRT插件 | MMCV版本 |\n| :-----------------------: | :-----------------------------------------------------------------------------: | :------: |\n| MMCVRoiAlign | [MMCVRoiAlign](./tensorrt_custom_ops.md#mmcvroialign) | 1.2.6 |\n| ScatterND | [ScatterND](./tensorrt_custom_ops.md#scatternd) | 1.2.6 |\n| NonMaxSuppression | [NonMaxSuppression](./tensorrt_custom_ops.md#nonmaxsuppression) | 1.3.0 |\n| MMCVDeformConv2d | [MMCVDeformConv2d](./tensorrt_custom_ops.md#mmcvdeformconv2d) | 1.3.0 |\n| grid_sampler | [grid_sampler](./tensorrt_custom_ops.md#grid-sampler) | 1.3.1 |\n| cummax | [cummax](./tensorrt_custom_ops.md#cummax) | 1.3.5 |\n| cummin | [cummin](./tensorrt_custom_ops.md#cummin) | 1.3.5 |\n| MMCVInstanceNormalization | [MMCVInstanceNormalization](./tensorrt_custom_ops.md#mmcvinstancenormalization) | 1.3.5 |\n| MMCVModulatedDeformConv2d | [MMCVModulatedDeformConv2d](./tensorrt_custom_ops.md#mmcvmodulateddeformconv2d) | master |\n\n注意\n\n- 以上所有算子均在 TensorRT-7.2.1.6.Ubuntu-16.04.x86_64-gnu.cuda-10.2.cudnn8.0 环境下开发。\n\n### 如何编译MMCV中的TensorRT插件\n\n#### 准备\n\n- 克隆代码仓库\n\n```bash\ngit clone https://github.com/open-mmlab/mmcv.git\n```\n\n- 安装TensorRT\n\n从 [NVIDIA Developer Zone](https://developer.nvidia.com/nvidia-tensorrt-download) 下载合适的TensorRT版本。\n\n比如，对安装了cuda-10.2的x86-64的Ubuntu 16.04，下载文件为`TensorRT-7.2.1.6.Ubuntu-16.04.x86_64-gnu.cuda-10.2.cudnn8.0.tar.gz`.\n\n然后使用下面方式安装并配置环境\n\n```bash\ncd ~/Downloads\ntar -xvzf TensorRT-7.2.1.6.Ubuntu-16.04.x86_64-gnu.cuda-10.2.cudnn8.0.tar.gz\nexport TENSORRT_DIR=`pwd`/TensorRT-7.2.1.6\nexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TENSORRT_DIR/lib\n```\n\n安装python依赖: tensorrt, graphsurgeon, onnx-graphsurgeon\n\n```bash\npip install $TENSORRT_DIR/python/tensorrt-7.2.1.6-cp37-none-linux_x86_64.whl\npip install $TENSORRT_DIR/onnx_graphsurgeon/onnx_graphsurgeon-0.2.6-py2.py3-none-any.whl\npip install $TENSORRT_DIR/graphsurgeon/graphsurgeon-0.4.5-py2.py3-none-any.whl\n```\n\n想了解更多通过tar包安装TensorRT，请访问[Nvidia' website](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-721/install-guide/index.html#installing-tar).\n\n#### 在Linux上编译\n\n```bash\ncd mmcv ## to MMCV root directory\nMMCV_WITH_OPS=1 MMCV_WITH_TRT=1 pip install -e .\n```\n\n### 创建TensorRT推理引擎并在python下进行推理\n\n范例如下：\n\n```python\nimport torch\nimport onnx\n\nfrom mmcv.tensorrt import (TRTWrapper, onnx2trt, save_trt_engine,\n is_tensorrt_plugin_loaded)\n\nassert is_tensorrt_plugin_loaded(), 'Requires to complie TensorRT plugins in mmcv'\n\nonnx_file = 'sample.onnx'\ntrt_file = 'sample.trt'\nonnx_model = onnx.load(onnx_file)\n\n## Model input\ninputs = torch.rand(1, 3, 224, 224).cuda()\n## Model input shape info\nopt_shape_dict = {\n 'input': [list(inputs.shape),\n list(inputs.shape),\n list(inputs.shape)]\n}\n\n## Create TensorRT engine\nmax_workspace_size = 1 << 30\ntrt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n max_workspace_size=max_workspace_size)\n\n## Save TensorRT engine\nsave_trt_engine(trt_engine, trt_file)\n\n## Run inference with TensorRT\ntrt_model = TRTWrapper(trt_file, ['input'], ['output'])\n\nwith torch.no_grad():\n trt_outputs = trt_model({'input': inputs})\n output = trt_outputs['output']\n\n```\n\n### 如何在MMCV中添加新的TensorRT自定义算子\n\n#### 主要流程\n\n下面是主要的步骤：\n\n1. 添加c++头文件\n2. 添加c++源文件\n3. 添加cuda kernel文件\n4. 在`trt_plugin.cpp`中注册插件\n5. 在`tests/test_ops/test_tensorrt.py`中添加单元测试\n\n**以RoIAlign算子插件`roi_align`举例。**\n\n1. 在TensorRT包含目录`mmcv/ops/csrc/tensorrt/`中添加头文件`trt_roi_align.hpp`\n2. 在TensorRT源码目录`mmcv/ops/csrc/tensorrt/plugins/`中添加头文件`trt_roi_align.cpp`\n3. 在TensorRT源码目录`mmcv/ops/csrc/tensorrt/plugins/`中添加cuda kernel文件`trt_roi_align_kernel.cu`\n4. 在[trt_plugin.cpp](https://github.com/open-mmlab/mmcv/blob/master/mmcv/ops/csrc/tensorrt/plugins/trt_plugin.cpp)中注册`roi_align`插件\n\n ```c++\n #include \"trt_plugin.hpp\"\n\n #include \"trt_roi_align.hpp\"\n\n REGISTER_TENSORRT_PLUGIN(RoIAlignPluginDynamicCreator);\n\n extern \"C\" {\n bool initLibMMCVInferPlugins() { return true; }\n } // extern \"C\"\n ```\n\n5. 在`tests/test_ops/test_tensorrt.py`中添加单元测试\n\n#### 注意\n\n- 部分MMCV中的自定义算子存在对应的cuda实现，在进行TensorRT插件开发的时候可以参考。\n\n### 已知问题\n\n- 无\n\n### 引用\n\n- [Developer guide of Nvidia TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html)\n- [TensorRT Open Source Software](https://github.com/NVIDIA/TensorRT)\n- [onnx-tensorrt](https://github.com/onnx/onnx-tensorrt)\n- [TensorRT python API](https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/index.html)\n- [TensorRT c++ plugin API](https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_plugin.html)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/faq.md", "content": "## 常见问题\n\n在这里我们列出了用户经常遇到的问题以及对应的解决方法。如果您遇到了其他常见的问题，并且知道可以帮到大家的解决办法，\n欢迎随时丰富这个列表。\n\n### 安装问题\n\n- KeyError: \"xxx: 'yyy is not in the zzz registry'\"\n\n 只有模块所在的文件被导入时，注册机制才会被触发，所以您需要在某处导入该文件，更多详情请查看 https://github.com/open-mmlab/mmdetection/issues/5974。\n\n- \"No module named 'mmcv.ops'\"; \"No module named 'mmcv._ext'\"\n\n 1. 使用 `pip uninstall mmcv` 卸载您环境中的 mmcv\n 2. 参考 [installation instruction](https://mmcv.readthedocs.io/en/latest/get_started/installation.html) 或者 [Build MMCV from source](https://mmcv.readthedocs.io/en/latest/get_started/build.html) 安装 mmcv-full\n\n- \"invalid device function\" 或者 \"no kernel image is available for execution\"\n\n 1. 检查 GPU 的 CUDA 计算能力\n 2. 运行 `python mmdet/utils/collect_env.py` 来检查 PyTorch、torchvision 和 MMCV 是否是针对正确的 GPU 架构构建的，您可能需要去设置 `TORCH_CUDA_ARCH_LIST` 来重新安装 MMCV。兼容性问题可能会出现在使用旧版的 GPUs，如：colab 上的 Tesla K80 (3.7)\n 3. 检查运行环境是否和 mmcv/mmdet 编译时的环境相同。例如，您可能使用 CUDA 10.0 编译 mmcv，但在 CUDA 9.0 的环境中运行它\n\n- \"undefined symbol\" 或者 \"cannot open xxx.so\"\n\n 1. 如果符号和 CUDA/C++ 相关（例如：libcudart.so 或者 GLIBCXX），请检查 CUDA/GCC 运行时的版本是否和编译 mmcv 的一致\n 2. 如果符号和 PyTorch 相关（例如：符号包含 caffe、aten 和 TH），请检查 PyTorch 运行时的版本是否和编译 mmcv 的一致\n 3. 运行 `python mmdet/utils/collect_env.py` 以检查 PyTorch、torchvision 和 MMCV 构建和运行的环境是否相同\n\n- \"RuntimeError: CUDA error: invalid configuration argument\"\n\n 这个错误可能是由于您的 GPU 性能不佳造成的。尝试降低[THREADS_PER_BLOCK](https://github.com/open-mmlab/mmcv/blob/cac22f8cf5a904477e3b5461b1cc36856c2793da/mmcv/ops/csrc/common_cuda_helper.hpp#L10)\n 的值并重新编译 mmcv。\n\n- \"RuntimeError: nms is not compiled with GPU support\"\n\n 这个错误是由于您的 CUDA 环境没有正确安装。\n 您可以尝试重新安装您的 CUDA 环境，然后删除 mmcv/build 文件夹并重新编译 mmcv。\n\n- \"Segmentation fault\"\n\n 1. 检查 GCC 的版本，通常是因为 PyTorch 版本与 GCC 版本不匹配 （例如 GCC < 4.9 )，我们推荐用户使用 GCC 5.4，我们也不推荐使用 GCC 5.5， 因为有反馈 GCC 5.5 会导致 \"segmentation fault\" 并且切换到 GCC 5.4 就可以解决问题\n 2. 检查是否正确安装 CUDA 版本的 PyTorc。输入以下命令并检查是否返回 True\n ```shell\n python -c 'import torch; print(torch.cuda.is_available())'\n ```\n 3. 如果 `torch` 安装成功，那么检查 MMCV 是否安装成功。输入以下命令，如果没有报错说明 mmcv-full 安装成。\n ```shell\n python -c 'import mmcv; import mmcv.ops'\n ```\n 4. 如果 MMCV 与 PyTorch 都安装成功了，则可以使用 `ipdb` 设置断点或者使用 `print` 函数，分析是哪一部分的代码导致了 `segmentation fault`\n\n- \"libtorch_cuda_cu.so: cannot open shared object file\"\n\n `mmcv-full` 依赖 `libtorch_cuda_cu.so` 文件，但程序运行时没能找到该文件。我们可以检查该文件是否存在 `~/miniconda3/envs/{environment-name}/lib/python3.7/site-packages/torch/lib` 也可以尝试重装 PyTorch。\n\n- \"fatal error C1189: #error: -- unsupported Microsoft Visual Studio version!\"\n\n 如果您在 Windows 上编译 mmcv-full 并且 CUDA 的版本是 9.2，您很可能会遇到这个问题 `\"C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v9.2\\include\\crt/host_config.h(133): fatal error C1189: #error: -- unsupported Microsoft Visual Studio version! Only the versions 2012, 2013, 2015 and 2017 are supported!\"`，您可以尝试使用低版本的 Microsoft Visual Studio，例如 vs2017。\n\n- \"error: member \"torch::jit::detail::ModulePolicy::all_slots\" may not be initialized\"\n\n 如果您在 Windows 上编译 mmcv-full 并且 PyTorch 的版本是 1.5.0，您很可能会遇到这个问题 `- torch/csrc/jit/api/module.h(474): error: member \"torch::jit::detail::ModulePolicy::all_slots\" may not be initialized`。解决这个问题的方法是将 `torch/csrc/jit/api/module.h` 文件中所有 `static constexpr bool all_slots = false;` 替换为 `static bool all_slots = false;`。更多细节可以查看 https://github.com/pytorch/pytorch/issues/39394。\n\n- \"error: a member with an in-class initializer must be const\"\n\n 如果您在 Windows 上编译 mmcv-full 并且 PyTorch 的版本是 1.6.0，您很可能会遇到这个问题 `\"- torch/include\\torch/csrc/jit/api/module.h(483): error: a member with an in-class initializer must be const\"`. 解决这个问题的方法是将 `torch/include\\torch/csrc/jit/api/module.h` 文件中的所有 `CONSTEXPR_EXCEPT_WIN_CUDA ` 替换为 `const`。更多细节可以查看 https://github.com/open-mmlab/mmcv/issues/575。\n\n- \"error: member \"torch::jit::ProfileOptionalOp::Kind\" may not be initialized\"\n\n 如果您在 Windows 上编译 mmcv-full 并且 PyTorch 的版本是 1.7.0，您很可能会遇到这个问题 `torch/include\\torch/csrc/jit/ir/ir.h(1347): error: member \"torch::jit::ProfileOptionalOp::Kind\" may not be initialized`. 解决这个问题的方法是修改 PyTorch 中的几个文件：\n\n - 删除 `torch/include\\torch/csrc/jit/ir/ir.h` 文件中的 `static constexpr Symbol Kind = ::c10::prim::profile;` 和 `tatic constexpr Symbol Kind = ::c10::prim::profile_optional;`\n - 将 `torch\\include\\pybind11\\cast.h` 文件中的 `explicit operator type&() { return *(this->value); }` 替换为 `explicit operator type&() { return *((type*)this->value); }`\n - 将 `torch/include\\torch/csrc/jit/api/module.h` 文件中的 所有 `CONSTEXPR_EXCEPT_WIN_CUDA` 替换为 `const`\n\n 更多细节可以查看 https://github.com/pytorch/pytorch/pull/45956。\n\n- MMCV 和 MMDetection 的兼容性问题；\"ConvWS is already registered in conv layer\"\n\n 请参考 [installation instruction](https://mmdetection.readthedocs.io/en/latest/get_started.html#installation) 为您的 MMDetection 版本安装正确版本的 MMCV。\n\n### 使用问题\n\n- \"RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one\"\n\n 1. 这个错误是因为有些参数没有参与 loss 的计算，可能是代码中存在多个分支，导致有些分支没有参与 loss 的计算。更多细节见 https://github.com/pytorch/pytorch/issues/55582。\n 2. 你可以设置 DDP 中的 `find_unused_parameters` 为 `True`，或者手动查找哪些参数没有用到。\n\n- \"RuntimeError: Trying to backward through the graph a second time\"\n\n 不能同时设置 `GradientCumulativeOptimizerHook` 和 `OptimizerHook`，这会导致 `loss.backward()` 被调用两次，于是程序抛出 `RuntimeError`。我们只需设置其中的一个。更多细节见 https://github.com/open-mmlab/mmcv/issues/1379。\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/get_started/build.md", "content": "## 从源码编译 MMCV\n\n### 在 Linux 或者 macOS 上编译 MMCV\n\n克隆算法库\n\n```bash\ngit clone https://github.com/open-mmlab/mmcv.git\ncd mmcv\n```\n\n建议安装 `ninja` 以加快编译速度\n\n```bash\npip install -r requirements/optional.txt\n```\n\n你可以安装 lite 版本\n\n```bash\npip install -e .\n```\n\n也可以安装 full 版本\n\n```bash\nMMCV_WITH_OPS=1 pip install -e .\n```\n\n如果是在 macOS 上编译，则需要在安装命令前添加一些环境变量\n\n```bash\nCC=clang CXX=clang++ CFLAGS='-stdlib=libc++'\n```\n\n例如\n\n```bash\nCC=clang CXX=clang++ CFLAGS='-stdlib=libc++' MMCV_WITH_OPS=1 pip install -e .\n```\n\n```{note}\n如果你打算使用 `opencv-python-headless` 而不是 `opencv-python`，例如在一个很小的容器环境或者没有图形用户界面的服务器中，你可以先安装 `opencv-python-headless`，这样在安装 mmcv 依赖的过程中会跳过 `opencv-python`\n```\n### 在 Windows 上编译 MMCV\n\n在 Windows 上编译 MMCV 比 Linux 复杂，本节将一步步介绍如何在 Windows 上编译 MMCV。\n\n#### 依赖项\n\n请首先安装以下的依赖项：\n\n- [Git](https://git-scm.com/download/win)：安装期间，请选择 **add git to Path**\n- [Visual Studio Community 2019](https://visualstudio.microsoft.com)：用于编译 C++ 和 CUDA 代码\n- [Miniconda](https://docs.conda.io/en/latest/miniconda.html)：包管理工具\n- [CUDA 10.2](https://developer.nvidia.com/cuda-10.2-download-archive)：如果只需要 CPU 版本可以不安装 CUDA，安装CUDA时，可根据需要进行自定义安装。如果已经安装新版本的显卡驱动，建议取消驱动程序的安装\n\n```{note}\n您需要知道如何在 Windows 上设置变量环境，尤其是 \"PATH\" 的设置，以下安装过程都会用到。\n```\n\n#### 设置 Python 环境\n\n1. 从 Windows 菜单启动 Anaconda 命令行\n\n```{note}\n如 Miniconda 安装程序建议，不要使用原始的 `cmd.exe` 或是 `powershell.exe`。命令行有两个版本，一个基于 PowerShell，一个基于传统的 `cmd.exe`。请注意以下说明都是使用的基于 PowerShell\n```\n\n2. 创建一个新的 Conda 环境\n\n ```shell\n conda create --name mmcv python=3.7 # 经测试，3.6, 3.7, 3.8 也能通过\n conda activate mmcv # 确保做任何操作前先激活环境\n ```\n\n3. 安装 PyTorch 时，可以根据需要安装支持 CUDA 或不支持 CUDA 的版本\n\n ```shell\n # CUDA version\n conda install pytorch torchvision cudatoolkit=10.2 -c pytorch\n # CPU version\n conda install pytorch torchvision cpuonly -c pytorch\n ```\n\n4. 准备 MMCV 源代码\n\n ```shell\n git clone https://github.com/open-mmlab/mmcv.git\n cd mmcv\n ```\n\n5. 安装所需 Python 依赖包\n\n ```shell\n pip3 install -r requirements/runtime.txt\n ```\n\n6. 建议安装 `ninja` 以加快编译速度\n\n ```bash\n pip install -r requirements/optional.txt\n ```\n\n#### 编译与安装 MMCV\n\nMMCV 有三种安装的模式：\n\n1. Lite 版本（不包含算子）\n\n 这种方式下，没有算子被编译，这种模式的 mmcv 是原生的 python 包\n\n2. Full 版本（只包含 CPU 算子）\n\n 编译 CPU 算子，但只有 x86 将会被编译，并且编译版本只能在 CPU only 情况下运行\n\n3. Full 版本（既包含 CPU 算子，又包含 CUDA 算子）\n\n 同时编译 CPU 和 CUDA 算子，`ops` 模块的 x86 与 CUDA 的代码都可以被编译。同时编译的版本可以在 CUDA 上调用 GPU\n\n##### 通用步骤\n\n1. 设置 MSVC 编译器\n\n 设置环境变量。添加 `C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\Hostx86\\x64` 到 `PATH`，则 `cl.exe` 可以在命令行中运行，如下所示。\n\n ```none\n (base) PS C:\\Users\\xxx> cl\n Microsoft (R) C/C++ Optimizing Compiler Version 19.27.29111 for x64\n Copyright (C) Microsoft Corporation. All rights reserved.\n\n usage: cl [ option... ] filename... [ / link linkoption... ]\n ```\n\n 为了兼容性，我们使用 x86-hosted 以及 x64-targeted 版本，即路径中的 `Hostx86\\x64` 。\n\n 因为 PyTorch 将解析 `cl.exe` 的输出以检查其版本，只有 utf-8 将会被识别，你可能需要将系统语言更改为英语。控制面板 -> 地区-> 管理-> 非 Unicode 来进行语言转换。\n\n##### 安装方式一：Lite version（不包含算子）\n\n在完成上述的公共步骤后，从菜单打开 Anaconda 命令框，输入以下命令\n\n```shell\n# 激活环境\nconda activate mmcv\n# 切换到 mmcv 根目录\ncd mmcv\n# 安装\npython setup.py develop\n# 检查是否安装成功\npip list\n```\n\n##### 安装方式二：Full version（只编译 CPU 算子）\n\n1. 完成上述的公共步骤\n\n2. 设置环境变量\n\n ```shell\n $env:MMCV_WITH_OPS = 1\n $env:MAX_JOBS = 8 # 根据你可用CPU以及内存量进行设置\n ```\n\n3. 编译安装\n\n ```shell\n conda activate mmcv # 激活环境\n cd mmcv # 改变路径\n python setup.py build_ext # 如果成功, cl 将被启动用于编译算子\n python setup.py develop # 安装\n pip list # 检查是否安装成功\n ```\n\n##### 安装方式三：Full version（既编译 CPU 算子又编译 CUDA 算子）\n\n1. 完成上述的公共步骤\n\n2. 设置环境变量\n\n ```shell\n $env:MMCV_WITH_OPS = 1\n $env:MAX_JOBS = 8 # 根据你可用CPU以及内存量进行设置\n ```\n\n3. 检查 `CUDA_PATH` 或者 `CUDA_HOME` 环境变量已经存在在 `envs` 之中\n\n ```none\n (base) PS C:\\Users\\WRH> ls env:\n\n Name Value\n ---- -----\n CUDA_PATH C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.2\n CUDA_PATH_V10_1 C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.1\n CUDA_PATH_V10_2 C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.2\n ```\n\n 如果没有，你可以按照下面的步骤设置\n\n ```shell\n $env:CUDA_HOME = \"C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.2\"\n # 或者\n $env:CUDA_HOME = $env:CUDA_PATH_V10_2 # CUDA_PATH_V10_2 已经在环境变量中\n ```\n\n4. 设置 CUDA 的目标架构\n\n ```shell\n $env:TORCH_CUDA_ARCH_LIST=\"6.1\" # 支持 GTX 1080\n # 或者用所有支持的版本，但可能会变得很慢\n $env:TORCH_CUDA_ARCH_LIST=\"3.5 3.7 5.0 5.2 6.0 6.1 7.0 7.5\"\n ```\n\n```{note}\n我们可以在 [here](https://developer.nvidia.com/cuda-gpus) 查看 GPU 的计算能力\n```\n\n5. 编译安装\n\n ```shell\n $env:MMCV_WITH_OPS = 1\n $env:MAX_JOBS = 8 # 根据你可用CPU以及内存量进行设置\n conda activate mmcv # 激活环境\n cd mmcv # 改变路径\n python setup.py build_ext # 如果成功, cl 将被启动用于编译算子\n python setup.py develop # 安装\n pip list # 检查是否安装成功\n ```\n\n```{note}\n如果你的 PyTorch 版本是 1.6.0，你可能会遇到一些这个 [issue](https://github.com/pytorch/pytorch/issues/42467) 提到的错误，则可以参考这个 [pull request](https://github.com/pytorch/pytorch/pull/43380/files) 修改 本地环境的 PyTorch 源代码\n```\n\n如果编译安装 mmcv 的过程中遇到了问题，你也许可以在 [Frequently Asked Question](../faq.html) 找到解决方法\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/get_started/installation.md", "content": "## 安装 MMCV\n\nMMCV 有两个版本：\n\n- **mmcv-full**: 完整版，包含所有的特性以及丰富的开箱即用的 CUDA 算子。注意完整版本可能需要更长时间来编译。\n- **mmcv**: 精简版，不包含 CUDA 算子但包含其余所有特性和功能，类似 MMCV 1.0 之前的版本。如果你不需要使用 CUDA 算子的话，精简版可以作为一个考虑选项。\n\n```{warning}\n请不要在同一个环境中安装两个版本，否则可能会遇到类似 `ModuleNotFound` 的错误。在安装一个版本之前，需要先卸载另一个。`如果CUDA可用，强烈推荐安装mmcv-full`。\n```\n\na. 安装完整版\n\n在安装 mmcv-full 之前，请确保 PyTorch 已经成功安装在环境中，可以参考 PyTorch 官方[文档](https://pytorch.org/)。\n\n我们提供了不同 PyTorch 和 CUDA 版本的 mmcv-full 预编译包，可以大大简化用户安装编译过程。强烈推荐通过预编译包来安装。另外，安装完成后可以运行 [check_installation.py](https://github.com/open-mmlab/mmcv/.dev_scripts/check_installation.py) 脚本检查 mmcv-full 是否安装成功。\n\ni. 安装最新版本\n\n如下是安装最新版 ``mmcv-full`` 的命令\n\n```shell\npip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html\n```\n\n请将链接中的 ``{cu_version}`` 和 ``{torch_version}`` 根据自身需求替换成实际的版本号，例如想安装和 ``CUDA 11.1``、``PyTorch 1.9.0`` 兼容的最新版 ``mmcv-full``，使用如下替换过的命令\n\n```shell\npip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html\n```\n\n```{note}\nPyTorch 在 1.x.0 和 1.x.1 之间通常是兼容的，故 mmcv-full 只提供 1.x.0 的编译包。如果你\n的 PyTorch 版本是 1.x.1，你可以放心地安装在 1.x.0 版本编译的 mmcv-full。例如，如果你的\nPyTorch 版本是 1.8.1、CUDA 版本是 11.1，你可以使用以下命令安装 mmcv-full。\n\n`pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.8.0/index.html`\n```\n\n如果想知道更多 CUDA 和 PyTorch 版本的命令，可以参考下面的表格，将链接中的 ``=={mmcv_version}`` 删去即可。\n\nii. 安装特定的版本\n\n如下是安装特定版本 ``mmcv-full`` 的命令\n\n```shell\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html\n```\n\n首先请参考版本发布信息找到想要安装的版本号，将 ``{mmcv_version}`` 替换成该版本号，例如 ``1.3.9``。\n然后将链接中的 ``{cu_version}`` 和 ``{torch_version}`` 根据自身需求替换成实际的版本号，例如想安装和 ``CUDA 11.1``、``PyTorch 1.9.0`` 兼容的 ``mmcv-full`` 1.3.9 版本，使用如下替换过的命令\n\n```shell\npip install mmcv-full==1.3.9 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html\n```\n\n对于更多的 PyTorch 和 CUDA 版本组合，请参考下表：\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
CUDA torch 1.10torch 1.9torch 1.8torch 1.7torch 1.6torch 1.5
11.3
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
11.1
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.10.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.8.0/index.html
11.0
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html
10.2
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.10.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.9.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.7.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.6.0/index.html
安装 
pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.5.0/index.html
10.1
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.8.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.7.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.6.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.5.0/index.html
9.2
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.7.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.6.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu92/torch1.5.0/index.html
cpu
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.10.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.9.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.8.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.7.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.6.0/index.html
安装 
 pip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.5.0/index.html
\n\n```{note}\n以上提供的预编译包并不囊括所有的 mmcv-full 版本，我们可以点击对应链接查看支持的版本。例如，点击 [cu102-torch1.8.0](https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html)，可以看到 `cu102-torch1.8.0` 只提供了 1.3.0 及以上的 mmcv-full 版本。另外，从 `mmcv v1.3.17` 开始，我们不再提供`PyTorch 1.3 & 1.4` 对应的 mmcv-full 预编译包。你可以在 [这](./previous_versions.md) 找到 `PyTorch 1.3 & 1.4` 对应的预编包。虽然我们不再提供 `PyTorch 1.3 & 1.4` 对应的预编译包，但是我们依然在 CI 中保证对它们的兼容持续到下一年。\n```\n\n除了使用预编译包之外，另一种方式是在本地进行编译，直接运行下述命令\n\n```python\npip install mmcv-full\n```\n\n但注意本地编译可能会耗时 10 分钟以上。\n\nb. 安装精简版\n\n```python\npip install mmcv\n```\n\nc. 安装完整版并且编译 onnxruntime 的自定义算子\n\n- 详细的指南请查看 [这里](https://mmcv.readthedocs.io/zh_CN/latest/deployment/onnxruntime_custom_ops.html)。\n\n如果想从源码编译 MMCV，请参考[该文档](https://mmcv.readthedocs.io/zh_CN/latest/get_started/build.html)。\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/get_started/introduction.md", "content": "## 介绍 MMCV\n\nMMCV 是一个面向计算机视觉的基础库，它支持了很多开源项目，例如：\n\n- [MIM](https://github.com/open-mmlab/mim): OpenMMLab 项目、算法、模型的统一入口\n- [MMClassification](https://github.com/open-mmlab/mmclassification): OpenMMLab 图像分类工具箱与测试基准\n- [MMDetection](https://github.com/open-mmlab/mmdetection): OpenMMLab 检测工具箱与测试基准\n- [MMDetection3D](https://github.com/open-mmlab/mmdetection3d): OpenMMLab 新一代通用3D目标检测平台\n- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation): OpenMMLab 语义分割工具箱与测试基准\n- [MMAction2](https://github.com/open-mmlab/mmaction2): OpenMMLab 新一代视频理解工具箱与测试基准\n- [MMTracking](https://github.com/open-mmlab/mmtracking): OpenMMLab 一体化视频目标感知平台\n- [MMPose](https://github.com/open-mmlab/mmpose): OpenMMLab 姿态估计工具箱与测试基准\n- [MMEditing](https://github.com/open-mmlab/mmediting): OpenMMLab 图像视频编辑工具箱\n- [MMOCR](https://github.com/open-mmlab/mmocr): OpenMMLab 全流程文字检测识别理解工具包\n- [MMGeneration](https://github.com/open-mmlab/mmgeneration): OpenMMLab 新一代生成模型工具箱\n- [MMFlow](https://github.com/open-mmlab/mmflow): OpenMMLab 光流估计工具箱与测试基准\n- [MMFewShot](https://github.com/open-mmlab/mmfewshot): OpenMMLab 少样本学习工具箱与测试基准\n- [MMHuman3D](https://github.com/open-mmlab/mmhuman3d): OpenMMLab 人体参数化模型工具箱与测试基准\n- [MMSelfSup](https://github.com/open-mmlab/mmselfsup): OpenMMLab 自监督学习工具箱与测试基准\n- [MMRazor](https://github.com/open-mmlab/mmrazor): OpenMMLab 模型压缩工具箱与测试基准\n- [MMDeploy](https://github.com/open-mmlab/mmdeploy): OpenMMLab 模型部署框架\n\nMMCV 提供了如下众多功能：\n\n- 通用的 IO 接口\n- 图像和视频处理\n- 图像和标注结果可视化\n- 常用小工具（进度条，计时器等）\n- 基于 PyTorch 的通用训练框架\n- 多种 CNN 网络结构\n- 高质量实现的常见 CUDA 算子\n\n如想了解更多特性和使用，请参考[文档](https://mmcv.readthedocs.io/zh_CN/latest)。\n\n```{note}\nMMCV 需要 Python 3.6 以上版本。\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/get_started/previous_versions.md", "content": "\n## 其他版本的 PyTorch\n\n我们不再提供在较低的 `PyTorch` 版本下编译的 `mmcv-full` 包，但为了您的方便，您可以在下面找到它们。\n\n### PyTorch 1.4\n\n| 1.0.0 <= mmcv_version <= 1.2.1\n\n#### CUDA 10.1\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.4.0/index.html\n```\n\n#### CUDA 9.2\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.4.0/index.html\n```\n\n#### CPU\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.4.0/index.html\n```\n\n### PyTorch v1.3\n\n| 1.0.0 <= mmcv_version <= 1.3.16\n\n#### CUDA 10.1\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.3.0/index.html\n```\n\n#### CUDA 9.2\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.3.0/index.html\n```\n\n#### CPU\n\n```bash\npip install mmcv-full=={mmcv_version} -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.3.0/index.html\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/index.rst", "content": "欢迎来到 MMCV 的中文文档！\n=============================\n\n您可以在页面左下角切换中英文文档。\n\n.. toctree::\n :maxdepth: 2\n :caption: 介绍与安装\n\n get_started/introduction.md\n get_started/installation.md\n get_started/build.md\n\n.. toctree::\n :maxdepth: 2\n :caption: 深入理解 MMCV\n\n understand_mmcv/config.md\n understand_mmcv/registry.md\n understand_mmcv/runner.md\n understand_mmcv/io.md\n understand_mmcv/data_process.md\n understand_mmcv/visualization.md\n understand_mmcv/cnn.md\n understand_mmcv/ops.md\n understand_mmcv/utils.md\n\n.. toctree::\n :maxdepth: 2\n :caption: 部署\n\n deployment/onnx.md\n deployment/onnxruntime_op.md\n deployment/onnxruntime_custom_ops.md\n deployment/tensorrt_plugin.md\n deployment/tensorrt_custom_ops.md\n\n.. toctree::\n :maxdepth: 2\n :caption: 兼容性\n\n compatibility.md\n\n.. toctree::\n :maxdepth: 2\n :caption: 常见问题\n\n faq.md\n\n.. toctree::\n :maxdepth: 2\n :caption: 社区\n\n community/contributing.md\n community/pr.md\n\n.. toctree::\n :maxdepth: 2\n :caption: API 文档\n\n api.rst\n\n\nIndices and tables\n==================\n\n* :ref:`genindex`\n* :ref:`search`\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/make.bat", "content": "@ECHO OFF\n\npushd %~dp0\n\nREM Command file for Sphinx documentation\n\nif \"%SPHINXBUILD%\" == \"\" (\n\tset SPHINXBUILD=sphinx-build\n)\nset SOURCEDIR=.\nset BUILDDIR=_build\n\nif \"%1\" == \"\" goto help\n\n%SPHINXBUILD% >NUL 2>NUL\nif errorlevel 9009 (\n\techo.\n\techo.The 'sphinx-build' command was not found. Make sure you have Sphinx\n\techo.installed, then set the SPHINXBUILD environment variable to point\n\techo.to the full path of the 'sphinx-build' executable. Alternatively you\n\techo.may add the Sphinx directory to PATH.\n\techo.\n\techo.If you don't have Sphinx installed, grab it from\n\techo.http://sphinx-doc.org/\n\texit /b 1\n)\n\n%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\ngoto end\n\n:help\n%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\n\n:end\npopd\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/cnn.md", "content": "## 卷积神经网络\n\n我们为卷积神经网络提供了一些构建模块，包括层构建、模块组件和权重初始化。\n\n### 网络层的构建\n\n在运行实验时，我们可能需要尝试同属一种类型但不同配置的层，但又不希望每次都修改代码。于是我们提供一些层构建方法，可以从字典构建层，字典可以在配置文件中配置，也可以通过命令行参数指定。\n\n#### 用法\n\n一个简单的例子：\n\n```python\ncfg = dict(type='Conv3d')\nlayer = build_conv_layer(cfg, in_channels=3, out_channels=8, kernel_size=3)\n```\n\n- `build_conv_layer`: 支持的类型包括 Conv1d、Conv2d、Conv3d、Conv (Conv是Conv2d的别名）\n- `build_norm_layer`: 支持的类型包括 BN1d、BN2d、BN3d、BN (alias for BN2d)、SyncBN、GN、LN、IN1d、IN2d、IN3d、IN（IN是IN2d的别名）\n- `build_activation_layer`：支持的类型包括 ReLU、LeakyReLU、PReLU、RReLU、ReLU6、ELU、Sigmoid、Tanh、GELU\n- `build_upsample_layer`: 支持的类型包括 nearest、bilinear、deconv、pixel_shuffle\n- `build_padding_layer`: 支持的类型包括 zero、reflect、replicate\n\n#### 拓展\n\n我们还允许自定义层和算子来扩展构建方法。\n\n1. 编写和注册自己的模块：\n\n ```python\n from mmcv.cnn import UPSAMPLE_LAYERS\n\n @UPSAMPLE_LAYERS.register_module()\n class MyUpsample:\n\n def __init__(self, scale_factor):\n pass\n\n def forward(self, x):\n pass\n ```\n\n2. 在某处导入 `MyUpsample` （例如 `__init__.py` ）然后使用它：\n\n ```python\n cfg = dict(type='MyUpsample', scale_factor=2)\n layer = build_upsample_layer(cfg)\n ```\n\n### 模块组件\n\n我们还提供了常用的模块组件，以方便网络构建。\n卷积组件 `ConvModule` 由 convolution、normalization以及activation layers 组成，更多细节请参考 [ConvModule api](api.html#mmcv.cnn.ConvModule)。\n\n```python\n# conv + bn + relu\nconv = ConvModule(3, 8, 2, norm_cfg=dict(type='BN'))\n# conv + gn + relu\nconv = ConvModule(3, 8, 2, norm_cfg=dict(type='GN', num_groups=2))\n# conv + relu\nconv = ConvModule(3, 8, 2)\n# conv\nconv = ConvModule(3, 8, 2, act_cfg=None)\n# conv + leaky relu\nconv = ConvModule(3, 8, 3, padding=1, act_cfg=dict(type='LeakyReLU'))\n# bn + conv + relu\nconv = ConvModule(\n 3, 8, 2, norm_cfg=dict(type='BN'), order=('norm', 'conv', 'act'))\n```\n\n### Weight initialization\n\n> 实现细节可以在 [mmcv/cnn/utils/weight_init.py](../../mmcv/cnn/utils/weight_init.py)中找到\n\n在训练过程中，适当的初始化策略有利于加快训练速度或者获得更高的性能。 在MMCV中，我们提供了一些常用的方法来初始化模块，比如 `nn.Conv2d` 模块。当然，我们也提供了一些高级API，可用于初始化包含一个或多个模块的模型。\n\n#### Initialization functions\n\n以函数的方式初始化 `nn.Module` ，例如 `nn.Conv2d` 、 `nn.Linear` 等。\n\n我们提供以下初始化方法，\n\n- constant_init\n\n 使用给定常量值初始化模型参数\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import constant_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # constant_init(module, val, bias=0)\n >>> constant_init(conv1, 1, 0)\n >>> conv1.weight\n ```\n\n- xavier_init\n\n 按照 [Understanding the difficulty of training deep feedforward neural networks - Glorot, X. & Bengio, Y. (2010)](http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf) 描述的方法初始化模型参数\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import xavier_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # xavier_init(module, gain=1, bias=0, distribution='normal')\n >>> xavier_init(conv1, distribution='normal')\n ```\n\n- normal_init\n\n 使用正态分布（高斯分布）初始化模型参数\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import normal_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # normal_init(module, mean=0, std=1, bias=0)\n >>> normal_init(conv1, std=0.01, bias=0)\n ```\n\n- uniform_init\n\n 使用均匀分布初始化模型参数\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import uniform_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # uniform_init(module, a=0, b=1, bias=0)\n >>> uniform_init(conv1, a=0, b=1)\n ```\n\n- kaiming_init\n\n 按照 [Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification - He, K. et al. (2015)](https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/He_Delving_Deep_into_ICCV_2015_paper.pdf) 描述的方法来初始化模型参数。\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import kaiming_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # kaiming_init(module, a=0, mode='fan_out', nonlinearity='relu', bias=0, distribution='normal')\n >>> kaiming_init(conv1)\n ```\n\n- caffe2_xavier_init\n\n caffe2中实现的 `xavier initialization`，对应于 PyTorch中的 `kaiming_uniform_`\n\n ```python\n >>> import torch.nn as nn\n >>> from mmcv.cnn import caffe2_xavier_init\n >>> conv1 = nn.Conv2d(3, 3, 1)\n >>> # caffe2_xavier_init(module, bias=0)\n >>> caffe2_xavier_init(conv1)\n ```\n\n- bias_init_with_prob\n\n 根据给定的概率初始化 `conv/fc`, 这在 [Focal Loss for Dense Object Detection](https://arxiv.org/pdf/1708.02002.pdf) 提出。\n\n ```python\n >>> from mmcv.cnn import bias_init_with_prob\n >>> # bias_init_with_prob is proposed in Focal Loss\n >>> bias = bias_init_with_prob(0.01)\n >>> bias\n -4.59511985013459\n ```\n\n#### Initializers and configs\n\n在初始化方法的基础上，我们定义了相应的初始化类，并将它们注册到 `INITIALIZERS` 中，这样我们就可以使用 `config` 配置来初始化模型了。\n\n我们提供以下初始化类：\n\n- ConstantInit\n- XavierInit\n- NormalInit\n- UniformInit\n- KaimingInit\n- Caffe2XavierInit\n- PretrainedInit\n\n接下来详细介绍 `initialize` 的使用方法\n\n1. 通过关键字 `layer` 来初始化模型\n\n 如果我们只定义了关键字 `layer` ，那么只初始化 `layer` 中包含的层。\n\n 注意: 关键字 `layer` 支持的模块是带有 weights 和 bias 属性的 PyTorch 模块，所以不支持 `MultiheadAttention layer`\n\n- 定义关键字 `layer` 列表并使用相同相同配置初始化模块\n\n ```python\n import torch.nn as nn\n from mmcv.cnn import initialize\n\n class FooNet(nn.Module):\n def __init__(self):\n super().__init__()\n self.feat = nn.Conv1d(3, 1, 3)\n self.reg = nn.Conv2d(3, 3, 3)\n self.cls = nn.Linear(1, 2)\n\n model = FooNet()\n init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d', 'Linear'], val=1)\n # 使用相同的配置初始化整个模块\n initialize(model, init_cfg)\n # model.feat.weight\n # Parameter containing:\n # tensor([[[1., 1., 1.],\n # [1., 1., 1.],\n # [1., 1., 1.]]], requires_grad=True)\n ```\n\n- 定义关键字 `layer` 用于初始化不同配置的层\n\n ```python\n import torch.nn as nn\n from mmcv.cnn.utils import initialize\n\n class FooNet(nn.Module):\n def __init__(self):\n super().__init__()\n self.feat = nn.Conv1d(3, 1, 3)\n self.reg = nn.Conv2d(3, 3, 3)\n self.cls = nn.Linear(1,2)\n\n model = FooNet()\n init_cfg = [dict(type='Constant', layer='Conv1d', val=1),\n dict(type='Constant', layer='Conv2d', val=2),\n dict(type='Constant', layer='Linear', val=3)]\n # nn.Conv1d 使用 dict(type='Constant', val=1) 初始化\n # nn.Conv2d 使用 dict(type='Constant', val=2) 初始化\n # nn.Linear 使用 dict(type='Constant', val=3) 初始化\n initialize(model, init_cfg)\n # model.reg.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n ```\n\n2. 定义关键字`override`初始化模型\n\n- 当用属性名初始化某个特定部分时, 我们可以使用关键字 `override`, 关键字 `override` 对应的Value会替代init_cfg中相应的值\n\n ```python\n import torch.nn as nn\n from mmcv.cnn import initialize\n\n class FooNet(nn.Module):\n def __init__(self):\n super().__init__()\n self.feat = nn.Conv1d(3, 1, 3)\n self.reg = nn.Conv2d(3, 3, 3)\n self.cls = nn.Sequential(nn.Conv1d(3, 1, 3), nn.Linear(1,2))\n\n # 如果我们想将模型的权重初始化为 1，将偏差初始化为 2\n # 但希望 `cls` 中的权重为 3，偏差为 4，则我们可以使用关键字override\n\n model = FooNet()\n init_cfg = dict(type='Constant', layer=['Conv1d','Conv2d'], val=1, bias=2,\n override=dict(type='Constant', name='reg', val=3, bias=4))\n # 使用 dict(type='Constant', val=1, bias=2)来初始化 self.feat and self.cls\n # 使用dict(type='Constant', val=3, bias=4)来初始化‘reg’模块。\n initialize(model, init_cfg)\n # model.reg.weight\n # Parameter containing:\n # tensor([[[[3., 3., 3.],\n # [3., 3., 3.],\n # [3., 3., 3.]],\n # ...,\n # [[3., 3., 3.],\n # [3., 3., 3.],\n # [3., 3., 3.]]]], requires_grad=True)\n ```\n\n- 如果 init_cfg 中的关键字`layer`为None，则只初始化在关键字override中的子模块，并且省略override中的 type 和其他参数\n\n ```python\n model = FooNet()\n init_cfg = dict(type='Constant', val=1, bias=2, override=dict(name='reg'))\n # self.feat 和 self.cls 使用pyTorch默认的初始化\n # 将使用 dict(type='Constant', val=1, bias=2) 初始化名为 'reg' 的模块\n initialize(model, init_cfg)\n # model.reg.weight\n # Parameter containing:\n # tensor([[[[1., 1., 1.],\n # [1., 1., 1.],\n # [1., 1., 1.]],\n # ...,\n # [[1., 1., 1.],\n # [1., 1., 1.],\n # [1., 1., 1.]]]], requires_grad=True)\n ```\n\n- 如果我们没有定义关键字`layer`或`override` , 将不会初始化任何东西\n\n- 关键字`override`的无效用法\n\n ```python\n # 没有重写任何子模块\n init_cfg = dict(type='Constant', layer=['Conv1d','Conv2d'],\n val=1, bias=2,\n override=dict(type='Constant', val=3, bias=4))\n\n # 没有指定type，即便有其他参数，也是无效的。\n init_cfg = dict(type='Constant', layer=['Conv1d','Conv2d'],\n val=1, bias=2,\n override=dict(name='reg', val=3, bias=4))\n ```\n\n3. 用预训练模型初始化\n\n ```python\n import torch.nn as nn\n import torchvision.models as models\n from mmcv.cnn import initialize\n\n # 使用预训练模型来初始化\n model = models.resnet50()\n # model.conv1.weight\n # Parameter containing:\n # tensor([[[[-6.7435e-03, -2.3531e-02, -9.0143e-03, ..., -2.1245e-03,\n # -1.8077e-03, 3.0338e-03],\n # [-1.2603e-02, -2.7831e-02, 2.3187e-02, ..., -1.5793e-02,\n # 1.1655e-02, 4.5889e-03],\n # [-3.7916e-02, 1.2014e-02, 1.3815e-02, ..., -4.2651e-03,\n # 1.7314e-02, -9.9998e-03],\n # ...,\n\n init_cfg = dict(type='Pretrained',\n checkpoint='torchvision://resnet50')\n initialize(model, init_cfg)\n # model.conv1.weight\n # Parameter containing:\n # tensor([[[[ 1.3335e-02, 1.4664e-02, -1.5351e-02, ..., -4.0896e-02,\n # -4.3034e-02, -7.0755e-02],\n # [ 4.1205e-03, 5.8477e-03, 1.4948e-02, ..., 2.2060e-03,\n # -2.0912e-02, -3.8517e-02],\n # [ 2.2331e-02, 2.3595e-02, 1.6120e-02, ..., 1.0281e-01,\n # 6.2641e-02, 5.1977e-02],\n # ...,\n\n # 使用关键字'prefix'用预训练模型的特定部分来初始化子模块权重\n model = models.resnet50()\n url = 'http://download.openmmlab.com/mmdetection/v2.0/retinanet/'\\\n 'retinanet_r50_fpn_1x_coco/'\\\n 'retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth'\n init_cfg = dict(type='Pretrained',\n checkpoint=url, prefix='backbone.')\n initialize(model, init_cfg)\n ```\n\n4. 初始化继承自BaseModule、Sequential、ModuleList、ModuleDict的模型\n\n `BaseModule` 继承自 `torch.nn.Module`, 它们之间唯一的不同是 `BaseModule` 实现了 `init_weight`\n\n `Sequential` 继承自 `BaseModule` 和 `torch.nn.Sequential`\n\n `ModuleList` 继承自 `BaseModule` 和 `torch.nn.ModuleList`\n\n `ModuleDict` 继承自 `BaseModule` 和 `torch.nn.ModuleDict`\n\n `````python\n import torch.nn as nn\n from mmcv.runner import BaseModule, Sequential, ModuleList, ModuleDict\n\n class FooConv1d(BaseModule):\n\n def __init__(self, init_cfg=None):\n super().__init__(init_cfg)\n self.conv1d = nn.Conv1d(4, 1, 4)\n\n def forward(self, x):\n return self.conv1d(x)\n\n class FooConv2d(BaseModule):\n\n def __init__(self, init_cfg=None):\n super().__init__(init_cfg)\n self.conv2d = nn.Conv2d(3, 1, 3)\n\n def forward(self, x):\n return self.conv2d(x)\n\n # BaseModule\n init_cfg = dict(type='Constant', layer='Conv1d', val=0., bias=1.)\n model = FooConv1d(init_cfg)\n model.init_weights()\n # model.conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n\n # Sequential\n init_cfg1 = dict(type='Constant', layer='Conv1d', val=0., bias=1.)\n init_cfg2 = dict(type='Constant', layer='Conv2d', val=2., bias=3.)\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n seq_model = Sequential(model1, model2)\n seq_model.init_weights()\n # seq_model[0].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # seq_model[1].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # inner init_cfg has higher priority\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.)\n seq_model = Sequential(model1, model2, init_cfg=init_cfg)\n seq_model.init_weights()\n # seq_model[0].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # seq_model[1].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # ModuleList\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n modellist = ModuleList([model1, model2])\n modellist.init_weights()\n # modellist[0].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # modellist[1].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # inner init_cfg has higher priority\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.)\n modellist = ModuleList([model1, model2], init_cfg=init_cfg)\n modellist.init_weights()\n # modellist[0].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # modellist[1].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # ModuleDict\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n modeldict = ModuleDict(dict(model1=model1, model2=model2))\n modeldict.init_weights()\n # modeldict['model1'].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # modeldict['model2'].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n\n # inner init_cfg has higher priority\n model1 = FooConv1d(init_cfg1)\n model2 = FooConv2d(init_cfg2)\n init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.)\n modeldict = ModuleDict(dict(model1=model1, model2=model2), init_cfg=init_cfg)\n modeldict.init_weights()\n # modeldict['model1'].conv1d.weight\n # Parameter containing:\n # tensor([[[0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.],\n # [0., 0., 0., 0.]]], requires_grad=True)\n # modeldict['model2'].conv2d.weight\n # Parameter containing:\n # tensor([[[[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]],\n # ...,\n # [[2., 2., 2.],\n # [2., 2., 2.],\n # [2., 2., 2.]]]], requires_grad=True)\n `````\n\n### Model Zoo\n\n除了`torchvision`的预训练模型，我们还提供以下 CNN 的预训练模型：\n\n- VGG Caffe\n- ResNet Caffe\n- ResNeXt\n- ResNet with Group Normalization\n- ResNet with Group Normalization and Weight Standardization\n- HRNetV2\n- Res2Net\n- RegNet\n\n#### Model URLs in JSON\n\nMMCV中的Model Zoo Link 由 JSON 文件管理。 json 文件由模型名称及其url或path的键值对组成,一个json文件可能类似于:\n\n```json\n{\n \"model_a\": \"https://example.com/models/model_a_9e5bac.pth\",\n \"model_b\": \"pretrain/model_b_ab3ef2c.pth\"\n}\n```\n\n可以在[此处](https://github.com/open-mmlab/mmcv/blob/master/mmcv/model_zoo/open_mmlab.json)找到托管在 OpenMMLab AWS 上的预训练模型的默认链接。\n\n你可以通过将 `open-mmlab.json` 放在 `MMCV_HOME`下来覆盖默认链接，如果在环境中找不到`MMCV_HOME`，则默认使用 `~/.cache/mmcv`。当然你也可以使用命令 `export MMCV_HOME=/your/path`来设置自己的路径。\n\n外部的json文件将被合并为默认文件，如果相同的键出现在外部`json`和默认`json`中，则将使用外部`json`。\n\n#### Load Checkpoint\n\n`mmcv.load_checkpoint()`的参数`filename`支持以下类型：\n\n- filepath: `checkpoint`路径\n- `http://xxx` and `https://xxx`: 下载checkpoint的链接，文件名中必需包含`SHA256`后缀\n- `torchvision://xxx`: `torchvision.models`中的模型链接，更多细节参考 [torchvision](https://pytorch.org/docs/stable/torchvision/models.html)\n- `open-mmlab://xxx`: 默认和其他 json 文件中提供的模型链接或文件路径\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/config.md", "content": "## 配置\n\n`Config` 类用于操作配置文件，它支持从多种文件格式中加载配置，包括 **python**, **json** 和 **yaml**。\n它提供了类似字典对象的接口来获取和设置值。\n\n以配置文件 `test.py` 为例\n\n```python\na = 1\nb = dict(b1=[0, 1, 2], b2=None)\nc = (1, 2)\nd = 'string'\n```\n\n加载与使用配置文件\n\n```python\n>>> cfg = Config.fromfile('test.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b1=[0, 1, 2], b2=None),\n... c=(1, 2),\n... d='string')\n```\n\n对于所有格式的配置文件，都支持一些预定义变量。它会将 `{{ var }}` 替换为实际值。\n\n目前支持以下四个预定义变量：\n\n`{{ fileDirname }}` - 当前打开文件的目录名，例如 /home/your-username/your-project/folder\n\n`{{ fileBasename }}` - 当前打开文件的文件名，例如 file.ext\n\n`{{ fileBasenameNoExtension }}` - 当前打开文件不包含扩展名的文件名，例如 file\n\n`{{ fileExtname }}` - 当前打开文件的扩展名，例如 .ext\n\n这些变量名引用自 [VS Code](https://code.visualstudio.com/docs/editor/variables-reference)。\n\n这里是一个带有预定义变量的配置文件的例子。\n\n`config_a.py`\n```python\na = 1\nb = './work_dir/{{ fileBasenameNoExtension }}'\nc = '{{ fileExtname }}'\n```\n\n```python\n>>> cfg = Config.fromfile('./config_a.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b='./work_dir/config_a',\n... c='.py')\n```\n\n对于所有格式的配置文件, 都支持继承。为了重用其他配置文件的字段，\n需要指定 `_base_='./config_a.py'` 或者一个包含配置文件的列表 `_base_=['./config_a.py', './config_b.py']`。\n\n这里有 4 个配置继承关系的例子。\n\n`config_a.py` 作为基类配置文件\n\n```python\na = 1\nb = dict(b1=[0, 1, 2], b2=None)\n```\n### 不含重复键值对从基类配置文件继承\n\n`config_b.py`\n\n```python\n_base_ = './config_a.py'\nc = (1, 2)\nd = 'string'\n```\n\n```python\n>>> cfg = Config.fromfile('./config_b.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b1=[0, 1, 2], b2=None),\n... c=(1, 2),\n... d='string')\n```\n在`config_b.py`里的新字段与在`config_a.py`里的旧字段拼接\n\n### 含重复键值对从基类配置文件继承\n\n`config_c.py`\n\n```python\n_base_ = './config_a.py'\nb = dict(b2=1)\nc = (1, 2)\n```\n\n```python\n>>> cfg = Config.fromfile('./config_c.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b1=[0, 1, 2], b2=1),\n... c=(1, 2))\n```\n\n在基类配置文件：`config_a` 里的 `b.b2=None`被配置文件：`config_c.py`里的 `b.b2=1`替代。\n\n### 从具有忽略字段的配置文件继承\n\n`config_d.py`\n\n```python\n_base_ = './config_a.py'\nb = dict(_delete_=True, b2=None, b3=0.1)\nc = (1, 2)\n```\n\n```python\n>>> cfg = Config.fromfile('./config_d.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b2=None, b3=0.1),\n... c=(1, 2))\n```\n\n您还可以设置 `_delete_=True`忽略基类配置文件中的某些字段。所有在`b`中的旧键 `b1, b2, b3` 将会被新键 `b2, b3` 所取代。\n\n### 从多个基类配置文件继承（基类配置文件不应包含相同的键）\n\n`config_e.py`\n\n```python\nc = (1, 2)\nd = 'string'\n```\n\n`config_f.py`\n\n```python\n_base_ = ['./config_a.py', './config_e.py']\n```\n\n```python\n>>> cfg = Config.fromfile('./config_f.py')\n>>> print(cfg)\n>>> dict(a=1,\n... b=dict(b1=[0, 1, 2], b2=None),\n... c=(1, 2),\n... d='string')\n```\n\n### 从基类引用变量\n\n您可以使用以下语法引用在基类中定义的变量。\n\n`base.py`\n\n```python\nitem1 = 'a'\nitem2 = dict(item3 = 'b')\n```\n\n`config_g.py`\n\n```python\n_base_ = ['./base.py']\nitem = dict(a = {{ _base_.item1 }}, b = {{ _base_.item2.item3 }})\n```\n\n```python\n>>> cfg = Config.fromfile('./config_g.py')\n>>> print(cfg.pretty_text)\nitem1 = 'a'\nitem2 = dict(item3='b')\nitem = dict(a='a', b='b')\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/data_process.md", "content": "## 数据处理\n\n### 图像\n\n图像模块提供了一些图像预处理的函数，该模块依赖 `opencv` 。\n\n#### 读取/保存/显示\n\n使用 `imread` 和 `imwrite` 函数可以读取和保存图像。\n\n```python\nimport mmcv\n\nimg = mmcv.imread('test.jpg')\nimg = mmcv.imread('test.jpg', flag='grayscale')\nimg_ = mmcv.imread(img) # 相当于什么也没做\nmmcv.imwrite(img, 'out.jpg')\n```\n\n从二进制中读取图像\n\n```python\nwith open('test.jpg', 'rb') as f:\n data = f.read()\nimg = mmcv.imfrombytes(data)\n```\n\n显示图像文件或已读取的图像\n\n```python\nmmcv.imshow('tests/data/color.jpg')\n\nfor i in range(10):\n img = np.random.randint(256, size=(100, 100, 3), dtype=np.uint8)\n mmcv.imshow(img, win_name='test image', wait_time=200)\n```\n\n#### 色彩空间转换\n\n支持的转换函数：\n\n- bgr2gray\n- gray2bgr\n- bgr2rgb\n- rgb2bgr\n- bgr2hsv\n- hsv2bgr\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\nimg1 = mmcv.bgr2rgb(img)\nimg2 = mmcv.rgb2gray(img1)\nimg3 = mmcv.bgr2hsv(img)\n```\n\n#### 缩放\n\n有三种缩放图像的方法。所有以 `imresize_*` 开头的函数都有一个 `return_scale` 参数，如果\n该参数为 `False` ，函数的返回值只有调整之后的图像，否则是一个元组 `(resized_img, scale)` 。\n\n```python\n# 缩放图像至给定的尺寸\nmmcv.imresize(img, (1000, 600), return_scale=True)\n\n# 缩放图像至与给定的图像同样的尺寸\nmmcv.imresize_like(img, dst_img, return_scale=False)\n\n# 以一定的比例缩放图像\nmmcv.imrescale(img, 0.5)\n\n# 缩放图像至最长的边不大于1000、最短的边不大于800并且没有改变图像的长宽比\nmmcv.imrescale(img, (1000, 800))\n```\n\n#### 旋转\n\n我们可以使用 `imrotate` 旋转图像一定的角度。旋转的中心需要指定，默认值是原始图像的中心。有\n两种旋转的模式，一种保持图像的尺寸不变，因此旋转后原始图像中的某些部分会被裁剪，另一种是扩大\n图像的尺寸进而保留完整的原始图像。\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# 顺时针旋转图像30度\nimg_ = mmcv.imrotate(img, 30)\n\n# 逆时针旋转图像90度\nimg_ = mmcv.imrotate(img, -90)\n\n# 顺时针旋转图像30度并且缩放图像为原始图像的1.5倍\nimg_ = mmcv.imrotate(img, 30, scale=1.5)\n\n# 以坐标(100, 100)为中心顺时针旋转图像30度\nimg_ = mmcv.imrotate(img, 30, center=(100, 100))\n\n# 顺时针旋转图像30度并扩大图像的尺寸\nimg_ = mmcv.imrotate(img, 30, auto_bound=True)\n```\n\n#### 翻转\n\n我们可以使用 `imflip` 翻转图像。\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# 水平翻转图像\nmmcv.imflip(img)\n\n# 垂直翻转图像\nmmcv.imflip(img, direction='vertical')\n```\n\n#### 裁剪\n\n`imcrop` 可以裁剪图像的一个或多个区域，每个区域用左上角和右下角坐标表示，形如(x1, y1, x2, y2)\n\n```python\nimport mmcv\nimport numpy as np\n\nimg = mmcv.imread('tests/data/color.jpg')\n\n# 裁剪区域 (10, 10, 100, 120)\nbboxes = np.array([10, 10, 100, 120])\npatch = mmcv.imcrop(img, bboxes)\n\n# 裁剪两个区域，分别是 (10, 10, 100, 120) 和 (0, 0, 50, 50)\nbboxes = np.array([[10, 10, 100, 120], [0, 0, 50, 50]])\npatches = mmcv.imcrop(img, bboxes)\n\n# 裁剪两个区域并且缩放区域1.2倍\npatches = mmcv.imcrop(img, bboxes, scale_ratio=1.2)\n```\n\n#### 填充\n\n`impad` and `impad_to_multiple` 可以用给定的值将图像填充至给定的尺寸。\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# 用给定值将图像填充至 (1000, 1200)\nimg_ = mmcv.impad(img, shape=(1000, 1200), pad_val=0)\n\n# 用给定值分别填充图像的3个通道至 (1000, 1200)\nimg_ = mmcv.impad(img, shape=(1000, 1200), pad_val=[100, 50, 200])\n\n# 用给定值填充图像的左、右、上、下四条边\nimg_ = mmcv.impad(img, padding=(10, 20, 30, 40), pad_val=0)\n\n# 用3个值分别填充图像的左、右、上、下四条边的3个通道\nimg_ = mmcv.impad(img, padding=(10, 20, 30, 40), pad_val=[100, 50, 200])\n\n# 将图像的四条边填充至能够被给定值整除\nimg_ = mmcv.impad_to_multiple(img, 32)\n```\n\n### 视频\n\n视频模块提供了以下的功能：\n\n- 一个 `VideoReader` 类，具有友好的 API 接口可以读取和转换视频\n- 一些编辑视频的方法，包括 `cut` ， `concat` ， `resize`\n- 光流的读取/保存/变换\n\n#### VideoReader\n\n`VideoReader` 类提供了和序列一样的接口去获取视频帧。该类会缓存所有被访问过的帧。\n\n```python\nvideo = mmcv.VideoReader('test.mp4')\n\n# 获取基本的信息\nprint(len(video))\nprint(video.width, video.height, video.resolution, video.fps)\n\n# 遍历所有的帧\nfor frame in video:\n print(frame.shape)\n\n# 读取下一帧\nimg = video.read()\n\n# 使用索引获取帧\nimg = video[100]\n\n# 获取指定范围的帧\nimg = video[5:10]\n```\n\n将视频切成帧并保存至给定目录或者从给定目录中生成视频。\n\n```python\n# 将视频切成帧并保存至目录\nvideo = mmcv.VideoReader('test.mp4')\nvideo.cvt2frames('out_dir')\n\n# 从给定目录中生成视频\nmmcv.frames2video('out_dir', 'test.avi')\n```\n\n#### 编辑函数\n\n有几个用于编辑视频的函数，这些函数是对 `ffmpeg` 的封装。\n\n```python\n# 裁剪视频\nmmcv.cut_video('test.mp4', 'clip1.mp4', start=3, end=10, vcodec='h264')\n\n# 将多个视频拼接成一个视频\nmmcv.concat_video(['clip1.mp4', 'clip2.mp4'], 'joined.mp4', log_level='quiet')\n\n# 将视频缩放至给定的尺寸\nmmcv.resize_video('test.mp4', 'resized1.mp4', (360, 240))\n\n# 将视频缩放至给定的倍率\nmmcv.resize_video('test.mp4', 'resized2.mp4', ratio=2)\n```\n\n#### 光流\n\n`mmcv` 提供了以下用于操作光流的函数：\n\n- 读取/保存\n- 可视化\n- 流变换\n\n我们提供了两种将光流dump到文件的方法，分别是非压缩和压缩的方法。非压缩的方法直接将浮点数值的光流\n保存至二进制文件，虽然光流无损但文件会比较大。而压缩的方法先量化光流至 0-255 整形数值再保存为\njpeg图像。光流的x维度和y维度会被拼接到图像中。\n\n1. 读取/保存\n\n```python\nflow = np.random.rand(800, 600, 2).astype(np.float32)\n# 保存光流到flo文件 (~3.7M)\nmmcv.flowwrite(flow, 'uncompressed.flo')\n# 保存光流为jpeg图像 (~230K)，图像的尺寸为 (800, 1200)\nmmcv.flowwrite(flow, 'compressed.jpg', quantize=True, concat_axis=1)\n\n# 读取光流文件，以下两种方式读取的光流尺寸均为 (800, 600, 2)\nflow = mmcv.flowread('uncompressed.flo')\nflow = mmcv.flowread('compressed.jpg', quantize=True, concat_axis=1)\n```\n\n2. 可视化\n\n使用 `mmcv.flowshow()` 可视化光流\n\n```python\nmmcv.flowshow(flow)\n```\n\n![progress](../../en/_static/flow_visualization.png)\n\n1. 流变换\n\n```python\nimg1 = mmcv.imread('img1.jpg')\nflow = mmcv.flowread('flow.flo')\nwarpped_img2 = mmcv.flow_warp(img1, flow)\n```\n\nimg1 (左) and img2 (右)\n\n![raw images](../../en/_static/flow_raw_images.png)\n\n光流 (img2 -> img1)\n\n![optical flow](../../en/_static/flow_img2toimg1.png)\n\n变换后的图像和真实图像的差异\n\n![warpped image](../../en/_static/flow_warp_diff.png)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/io.md", "content": "## 文件输入输出\n\n文件输入输出模块提供了两个通用的 API 接口用于读取和保存不同格式的文件。\n\n```{note}\n在 v1.3.16 及之后的版本中，IO 模块支持从不同后端读取数据并支持将数据至不同后端。更多细节请访问 PR [#1330](https://github.com/open-mmlab/mmcv/pull/1330)。\n```\n\n### 读取和保存数据\n\n`mmcv` 提供了一个通用的 api 用于读取和保存数据，目前支持的格式有 json、yaml 和 pickle。\n\n#### 从硬盘读取数据或者将数据保存至硬盘\n\n```python\nimport mmcv\n\n# 从文件中读取数据\ndata = mmcv.load('test.json')\ndata = mmcv.load('test.yaml')\ndata = mmcv.load('test.pkl')\n# 从文件对象中读取数据\nwith open('test.json', 'r') as f:\n data = mmcv.load(f, file_format='json')\n\n# 将数据序列化为字符串\njson_str = mmcv.dump(data, file_format='json')\n\n# 将数据保存至文件 (根据文件名后缀反推文件类型)\nmmcv.dump(data, 'out.pkl')\n\n# 将数据保存至文件对象\nwith open('test.yaml', 'w') as f:\n data = mmcv.dump(data, f, file_format='yaml')\n```\n\n#### 从其他后端加载或者保存至其他后端\n\n```python\nimport mmcv\n\n# 从 s3 文件读取数据\ndata = mmcv.load('s3://bucket-name/test.json')\ndata = mmcv.load('s3://bucket-name/test.yaml')\ndata = mmcv.load('s3://bucket-name/test.pkl')\n\n# 将数据保存至 s3 文件 (根据文件名后缀反推文件类型)\nmmcv.dump(data, 's3://bucket-name/out.pkl')\n```\n\n我们提供了易于拓展的方式以支持更多的文件格式。我们只需要创建一个继承自 `BaseFileHandler` 的\n文件句柄类并将其注册到 `mmcv` 中即可。句柄类至少需要重写三个方法。\n\n```python\nimport mmcv\n\n# 支持为文件句柄类注册多个文件格式\n# @mmcv.register_handler(['txt', 'log'])\n@mmcv.register_handler('txt')\nclass TxtHandler1(mmcv.BaseFileHandler):\n\n def load_from_fileobj(self, file):\n return file.read()\n\n def dump_to_fileobj(self, obj, file):\n file.write(str(obj))\n\n def dump_to_str(self, obj, **kwargs):\n return str(obj)\n```\n\n以 `PickleHandler` 为例\n\n```python\nimport pickle\n\nclass PickleHandler(mmcv.BaseFileHandler):\n\n def load_from_fileobj(self, file, **kwargs):\n return pickle.load(file, **kwargs)\n\n def load_from_path(self, filepath, **kwargs):\n return super(PickleHandler, self).load_from_path(\n filepath, mode='rb', **kwargs)\n\n def dump_to_str(self, obj, **kwargs):\n kwargs.setdefault('protocol', 2)\n return pickle.dumps(obj, **kwargs)\n\n def dump_to_fileobj(self, obj, file, **kwargs):\n kwargs.setdefault('protocol', 2)\n pickle.dump(obj, file, **kwargs)\n\n def dump_to_path(self, obj, filepath, **kwargs):\n super(PickleHandler, self).dump_to_path(\n obj, filepath, mode='wb', **kwargs)\n```\n\n### 读取文件并返回列表或字典\n\n例如， `a.txt` 是文本文件，一共有5行内容。\n\n```\na\nb\nc\nd\ne\n```\n#### 从硬盘读取\n\n使用 `list_from_file` 读取 `a.txt`\n\n```python\n>>> mmcv.list_from_file('a.txt')\n['a', 'b', 'c', 'd', 'e']\n>>> mmcv.list_from_file('a.txt', offset=2)\n['c', 'd', 'e']\n>>> mmcv.list_from_file('a.txt', max_num=2)\n['a', 'b']\n>>> mmcv.list_from_file('a.txt', prefix='/mnt/')\n['/mnt/a', '/mnt/b', '/mnt/c', '/mnt/d', '/mnt/e']\n```\n\n同样， `b.txt` 也是文本文件，一共有3行内容\n\n```\n1 cat\n2 dog cow\n3 panda\n```\n\n使用 `dict_from_file` 读取 `b.txt`\n\n```python\n>>> mmcv.dict_from_file('b.txt')\n{'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n>>> mmcv.dict_from_file('b.txt', key_type=int)\n{1: 'cat', 2: ['dog', 'cow'], 3: 'panda'}\n```\n\n#### 从其他后端读取\n\n使用 `list_from_file` 读取 `s3://bucket-name/a.txt`\n\n```python\n>>> mmcv.list_from_file('s3://bucket-name/a.txt')\n['a', 'b', 'c', 'd', 'e']\n>>> mmcv.list_from_file('s3://bucket-name/a.txt', offset=2)\n['c', 'd', 'e']\n>>> mmcv.list_from_file('s3://bucket-name/a.txt', max_num=2)\n['a', 'b']\n>>> mmcv.list_from_file('s3://bucket-name/a.txt', prefix='/mnt/')\n['/mnt/a', '/mnt/b', '/mnt/c', '/mnt/d', '/mnt/e']\n```\n\n使用 `dict_from_file` 读取 `b.txt`\n\n```python\n>>> mmcv.dict_from_file('s3://bucket-name/b.txt')\n{'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n>>> mmcv.dict_from_file('s3://bucket-name/b.txt', key_type=int)\n{1: 'cat', 2: ['dog', 'cow'], 3: 'panda'}\n```\n\n### 读取和保存权重文件\n\n#### 从硬盘读取权重文件或者将权重文件保存至硬盘\n\n我们可以通过下面的方式从磁盘读取权重文件或者将权重文件保存至磁盘\n\n```python\nimport torch\n\nfilepath1 = '/path/of/your/checkpoint1.pth'\nfilepath2 = '/path/of/your/checkpoint2.pth'\n# 从 filepath1 读取权重文件\ncheckpoint = torch.load(filepath1)\n# 将权重文件保存至 filepath2\ntorch.save(checkpoint, filepath2)\n```\n\nMMCV 提供了很多后端，`HardDiskBackend` 是其中一个，我们可以通过它来读取或者保存权重文件。\n\n```python\nimport io\nfrom mmcv.fileio.file_client import HardDiskBackend\n\ndisk_backend = HardDiskBackend()\nwith io.BytesIO(disk_backend.get(filepath1)) as buffer:\n checkpoint = torch.load(buffer)\nwith io.BytesIO() as buffer:\n torch.save(checkpoint, f)\n disk_backend.put(f.getvalue(), filepath2)\n```\n\n如果我们想在接口中实现根据文件路径自动选择对应的后端，我们可以使用 `FileClient`。\n例如，我们想实现两个方法，分别是读取权重以及保存权重，它们需支持不同类型的文件路径，可以是磁盘路径，也可以是网络路径或者其他路径。\n\n```python\nfrom mmcv.fileio.file_client import FileClient\n\ndef load_checkpoint(path):\n file_client = FileClient.infer(uri=path)\n with io.BytesIO(file_client.get(path)) as buffer:\n checkpoint = torch.load(buffer)\n return checkpoint\n\ndef save_checkpoint(checkpoint, path):\n with io.BytesIO() as buffer:\n torch.save(checkpoint, buffer)\n file_client.put(buffer.getvalue(), path)\n\nfile_client = FileClient.infer_client(uri=filepath1)\ncheckpoint = load_checkpoint(filepath1)\nsave_checkpoint(checkpoint, filepath2)\n```\n\n#### 从网络远端读取权重文件\n\n```{note}\n目前只支持从网络远端读取权重文件，暂不支持将权重文件写入网络远端\n```\n\n```python\nimport io\nimport torch\nfrom mmcv.fileio.file_client import HTTPBackend, FileClient\n\nfilepath = 'http://path/of/your/checkpoint.pth'\ncheckpoint = torch.utils.model_zoo.load_url(filepath)\n\nhttp_backend = HTTPBackend()\nwith io.BytesIO(http_backend.get(filepath)) as buffer:\n checkpoint = torch.load(buffer)\n\nfile_client = FileClient.infer_client(uri=filepath)\nwith io.BytesIO(file_client.get(filepath)) as buffer:\n checkpoint = torch.load(buffer)\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/ops.md", "content": "## CUDA 算子\n\nMMCV 提供了检测、分割等任务中常用的 CUDA 算子\n\n- ActiveRotatedFilter\n- AssignScoreWithK\n- BallQuery\n- BBoxOverlaps\n- CARAFE\n- CrissCrossAttention\n- ContextBlock\n- ConvexIoU\n- CornerPool\n- Deformable Convolution v1/v2\n- Deformable RoIPool\n- DynamicScatter\n- GatherPoints\n- FurthestPointSample\n- FurthestPointSampleWithDist\n- GeneralizedAttention\n- KNN\n- MaskedConv\n- MinAreaPolygon\n- NMS\n- PointsInPolygons\n- PSAMask\n- RotatedFeatureAlign\n- RoIPointPool3d\n- RoIPool\n- RiRoIAlignRotated\n- RoIAlign\n- RoIAwarePool3d\n- SimpleRoIAlign\n- SigmoidFocalLoss\n- SoftmaxFocalLoss\n- SoftNMS\n- Synchronized BatchNorm\n- Voxelization\n- ThreeInterpolate\n- ThreeNN\n- Weight standardization\n- Correlation\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/registry.md", "content": "## 注册器\nMMCV 使用 [注册器](https://github.com/open-mmlab/mmcv/blob/master/mmcv/utils/registry.py) 来管理具有相似功能的不同模块, 例如, 检测器中的主干网络、头部、和模型颈部。\n在 OpenMMLab 家族中的绝大部分开源项目使用注册器去管理数据集和模型的模块，例如 [MMDetection](https://github.com/open-mmlab/mmdetection), [MMDetection3D](https://github.com/open-mmlab/mmdetection3d), [MMClassification](https://github.com/open-mmlab/mmclassification), [MMEditing](https://github.com/open-mmlab/mmediting) 等。\n\n### 什么是注册器\n在MMCV中，注册器可以看作类到字符串的映射。\n一个注册器中的类通常有相似的接口，但是可以实现不同的算法或支持不同的数据集。\n借助注册器，用户可以通过使用相应的字符串查找并实例化该类，并根据他们的需要实例化对应模块。\n一个典型的案例是，OpenMMLab　中的大部分开源项目的配置系统，这些系统通过配置文件来使用注册器创建钩子、执行器、模型和数据集。\n可以在[这里](https://mmcv.readthedocs.io/en/latest/api.html?highlight=registry#mmcv.utils.Registry)找到注册器接口使用文档。\n\n使用 `registry`（注册器）管理代码库中的模型，需要以下三个步骤。\n\n1. 创建一个构建方法（可选，在大多数情况下您可以只使用默认方法）\n2. 创建注册器\n3. 使用此注册器来管理模块\n\n`Registry`（注册器）的参数 `build_func`（构建函数） 用来自定以如何实例化类的实例，默认使用 [这里](https://mmcv.readthedocs.io/en/latest/api.html?highlight=registry#mmcv.utils.build_from_cfg)实现的`build_from_cfg`。\n\n### 一个简单的例子\n\n这里是一个使用注册器管理包中模块的简单示例。您可以在 OpenMMLab 开源项目中找到更多实例。\n\n假设我们要实现一系列数据集转换器（Dataset Converter），用于将不同格式的数据转换为标准数据格式。我们先创建一个名为converters的目录作为包，在包中我们创建一个文件来实现构建器（builder），命名为converters/builder.py，如下\n\n```python\nfrom mmcv.utils import Registry\n# 创建转换器（converter）的注册器（registry）\nCONVERTERS = Registry('converter')\n```\n\n然后我们在包中可以实现不同的转换器（converter）。例如，在 `converters/converter1.py` 中实现 `Converter1`。\n\n```python\nfrom .builder import CONVERTERS\n\n# 使用注册器管理模块\n@CONVERTERS.register_module()\nclass Converter1(object):\n def __init__(self, a, b):\n self.a = a\n self.b = b\n```\n使用注册器管理模块的关键步骤是，将实现的模块注册到注册表 `CONVERTERS` 中。通过 `@CONVERTERS.register_module()` 装饰所实现的模块，字符串和类之间的映射就可以由 `CONVERTERS` 构建和维护，如下所示：\n\n通过这种方式，就可以通过 `CONVERTERS` 建立字符串与类之间的映射，如下所示：\n\n```python\n'Converter1' -> \n```\n```{note}\n只有模块所在的文件被导入时，注册机制才会被触发，所以您需要在某处导入该文件。更多详情请查看 https://github.com/open-mmlab/mmdetection/issues/5974。\n```\n如果模块被成功注册了，你可以通过配置文件使用这个转换器（converter），如下所示：\n\n```python\nconverter_cfg = dict(type='Converter1', a=a_value, b=b_value)\nconverter = CONVERTERS.build(converter_cfg)\n```\n\n### 自定义构建函数\n\n假设我们想自定义 `converters` 的构建流程，我们可以实现一个自定义的 `build_func` （构建函数）并将其传递到注册器中。\n\n```python\nfrom mmcv.utils import Registry\n\n# 创建一个构建函数\ndef build_converter(cfg, registry, *args, **kwargs):\n cfg_ = cfg.copy()\n converter_type = cfg_.pop('type')\n if converter_type not in registry:\n raise KeyError(f'Unrecognized converter type {converter_type}')\n else:\n converter_cls = registry.get(converter_type)\n\n converter = converter_cls(*args, **kwargs, **cfg_)\n return converter\n\n# 创建一个用于转换器（converters）的注册器，并传递（registry）``build_converter`` 函数\nCONVERTERS = Registry('converter', build_func=build_converter)\n```\n\n```{note}\n注：在这个例子中，我们演示了如何使用参数：`build_func` 自定义构建类的实例的方法。\n该功能类似于默认的`build_from_cfg`。在大多数情况下，默认就足够了。\n```\n\n`build_model_from_cfg`也实现了在`nn.Sequentail`中构建PyTorch模块，你可以直接使用它们。\n\n### 注册器层结构\n\n你也可以从多个 OpenMMLab 开源框架中构建模块，例如，你可以把所有 [MMClassification](https://github.com/open-mmlab/mmclassification) 中的主干网络（backbone）用到 [MMDetection](https://github.com/open-mmlab/mmdetection) 的目标检测中，你也可以融合 [MMDetection](https://github.com/open-mmlab/mmdetection) 中的目标检测模型 和 [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) 语义分割模型。\n\n下游代码库中所有 `MODELS` 注册器都是MMCV `MODELS` 注册器的子注册器。基本上，使用以下两种方法从子注册器或相邻兄弟注册器构建模块。\n\n1. 从子注册器中构建\n\n 例如：\n\n 我们在 MMDetection 中定义：\n\n ```python\n from mmcv.utils import Registry\n from mmcv.cnn import MODELS as MMCV_MODELS\n MODELS = Registry('model', parent=MMCV_MODELS)\n\n @MODELS.register_module()\n class NetA(nn.Module):\n def forward(self, x):\n return x\n ```\n\n 我们在 MMClassification 中定义：\n\n ```python\n from mmcv.utils import Registry\n from mmcv.cnn import MODELS as MMCV_MODELS\n MODELS = Registry('model', parent=MMCV_MODELS)\n\n @MODELS.register_module()\n class NetB(nn.Module):\n def forward(self, x):\n return x + 1\n ```\n\n 我们可以通过以下代码在 MMDetection 或 MMClassification 中构建两个网络：\n\n ```python\n from mmdet.models import MODELS\n net_a = MODELS.build(cfg=dict(type='NetA'))\n net_b = MODELS.build(cfg=dict(type='mmcls.NetB'))\n ```\n\n 或\n\n ```python\n from mmcls.models import MODELS\n net_a = MODELS.build(cfg=dict(type='mmdet.NetA'))\n net_b = MODELS.build(cfg=dict(type='NetB'))\n ```\n\n2. 从父注册器中构建\n\n MMCV中的共享`MODELS`注册器是所有下游代码库的父注册器（根注册器）：\n\n ```python\n from mmcv.cnn import MODELS as MMCV_MODELS\n net_a = MMCV_MODELS.build(cfg=dict(type='mmdet.NetA'))\n net_b = MMCV_MODELS.build(cfg=dict(type='mmcls.NetB'))\n ```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/runner.md", "content": "## 执行器\n\n执行器模块负责模型训练过程调度，主要目的是让用户使用更少的代码以及灵活可配置方式开启训练。其具备如下核心特性:\n\n- 支持以 `EpochBasedRunner` 和 `IterBasedRunner` 为单位的迭代模式以满足不同场景\n- 支持定制工作流以满足训练过程中各状态自由切换，目前支持训练和验证两个工作流。工作流可以简单理解为一个完成的训练和验证迭代过程。\n- 配合各类默认和自定义 Hook，对外提供了灵活扩展能力\n\n### EpochBasedRunner\n\n顾名思义，`EpochBasedRunner` 是指以 epoch 为周期的工作流，例如设置 workflow = [('train', 2), ('val', 1)] 表示循环迭代地训练 2 个 epoch，然后验证 1 个 epoch。MMDetection 目标检测框架默认采用的是 `EpochBasedRunner`。\n\n其抽象逻辑如下所示：\n\n```python\n# 训练终止条件\nwhile curr_epoch < max_epochs:\n # 遍历用户设置的工作流，例如 workflow = [('train', 2)，('val', 1)]\n for i, flow in enumerate(workflow):\n # mode 是工作流函数，例如 train, epochs 是迭代次数\n mode, epochs = flow\n # 要么调用 self.train()，要么调用 self.val()\n epoch_runner = getattr(self, mode)\n # 运行对应工作流函数\n for _ in range(epochs):\n epoch_runner(data_loaders[i], **kwargs)\n```\n目前支持训练和验证两个工作流，以训练函数为例，其抽象逻辑是：\n\n```python\n# epoch_runner 目前可以是 train 或者 val\ndef train(self, data_loader, **kwargs):\n # 遍历 dataset，共返回一个 epoch 的 batch 数据\n for i, data_batch in enumerate(data_loader):\n self.call_hook('before_train_iter')\n # 验证时候 train_mode=False\n self.run_iter(data_batch, train_mode=True, **kwargs)\n self.call_hook('after_train_iter')\n self.call_hook('after_train_epoch')\n```\n\n### IterBasedRunner\n不同于 `EpochBasedRunner`，`IterBasedRunner` 是指以 iter 为周期的工作流，例如设置 workflow = [('train', 2)， ('val', 1)] 表示循环迭代的训练 2 个 iter，然后验证 1 个 iter，MMSegmentation 语义分割框架默认采用的是 `IterBasedRunner`。\n\n其抽象逻辑如下所示：\n\n```python\n# 虽然是 iter 单位，但是某些场合需要 epoch 信息，由 IterLoader 提供\niter_loaders = [IterLoader(x) for x in data_loaders]\n# 训练终止条件\nwhile curr_iter < max_iters:\n # 遍历用户设置的工作流，例如 workflow = [('train', 2)， ('val', 1)]\n for i, flow in enumerate(workflow):\n # mode 是工作流函数，例如 train, iters 是迭代次数\n mode, iters = flow\n # 要么调用 self.train()，要么调用 self.val()\n iter_runner = getattr(self, mode)\n # 运行对应工作流函数\n for _ in range(iters):\n iter_runner(iter_loaders[i], **kwargs)\n```\n目前支持训练和验证两个工作流，以验证函数为例，其抽象逻辑是：\n\n```python\n# iter_runner 目前可以是 train 或者 val\ndef val(self, data_loader, **kwargs):\n # 获取 batch 数据，用于一次迭代\n data_batch = next(data_loader)\n self.call_hook('before_val_iter')\n outputs = self.model.val_step(data_batch, self.optimizer, **kwargs)\n self.outputs = outputs\n self.call_hook('after_val_iter')\n```\n\n除了上述基础功能外，`EpochBasedRunner` 和 `IterBasedRunner` 还提供了 resume 、 save_checkpoint 和注册 hook 功能。\n\n### 一个简单例子\n以最常用的分类任务为例详细说明 `runner` 的使用方法。 开启任何一个训练任务，都需要包括如下步骤：\n\n**(1) dataloader、model 和优化器等类初始化**\n\n```python\n# 模型类初始化\nmodel=...\n# 优化器类初始化，典型值 cfg.optimizer = dict(type='SGD', lr=0.1, momentum=0.9, weight_decay=0.0001)\noptimizer = build_optimizer(model, cfg.optimizer)\n# 工作流对应的 dataloader 初始化\ndata_loaders = [\n build_dataloader(\n ds,\n cfg.data.samples_per_gpu,\n cfg.data.workers_per_gpu,\n ...) for ds in dataset\n ]\n```\n\n**(2) runner 类初始化**\n\n```python\nrunner = build_runner(\n # cfg.runner 典型配置为\n # runner = dict(type='EpochBasedRunner', max_epochs=200)\n cfg.runner,\n default_args=dict(\n model=model,\n batch_processor=None,\n optimizer=optimizer,\n logger=logger))\n```\n\n**(3) 注册默认训练所必须的 hook，和用户自定义 hook**\n\n```python\n# 注册定制必需的 hook\nrunner.register_training_hooks(\n # lr相关配置，典型为\n # lr_config = dict(policy='step', step=[100, 150])\n cfg.lr_config,\n # 优化相关配置，例如 grad_clip 等\n optimizer_config,\n # 权重保存相关配置，典型为\n # checkpoint_config = dict(interval=1)，每个单位都保存权重\n cfg.checkpoint_config,\n # 日志相关配置\n cfg.log_config,\n ...)\n\n# 注册用户自定义 hook\n# 例如想使用 ema 功能，则可以设置 custom_hooks=[dict(type='EMAHook')]\nif cfg.get('custom_hooks', None):\n custom_hooks = cfg.custom_hooks\n for hook_cfg in cfg.custom_hooks:\n hook_cfg = hook_cfg.copy()\n priority = hook_cfg.pop('priority', 'NORMAL')\n hook = build_from_cfg(hook_cfg, HOOKS)\n runner.register_hook(hook, priority=priority)\n```\n\n然后可以进行 resume 或者 load_checkpoint 对权重进行加载。\n\n**(4) 开启训练流**\n\n```python\n# workflow 典型为 workflow = [('train', 1)]\n# 此时就真正开启了训练\nrunner.run(data_loaders, cfg.workflow)\n```\n\n关于 workflow 设置，以 `EpochBasedRunner` 为例，详情如下：\n\n- 假设只想运行训练工作流，则可以设置 workflow = [('train', 1)]，表示只进行迭代训练\n- 假设想运行训练和验证工作流，则可以设置 workflow = [('train', 3), ('val', 1)]，表示先训练 3 个 epoch ，然后切换到 val 工作流，运行 1 个 epoch，然后循环，直到训练 epoch 次数达到指定值\n- 工作流设置还自由定制，例如你可以先验证再训练 workflow = [('val', 1), ('train', 1)]\n\n上述代码都已经封装到了各个代码库的 train.py 中，用户只需要设置相应的配置即可，上述流程会自动运行。\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/utils.md", "content": "## 辅助函数\n\n### 进度条\n\n如果你想跟踪函数批处理任务的进度，可以使用 `track_progress` 。它能以进度条的形式展示任务的完成情况以及剩余任务所需的时间（内部实现为for循环）。\n\n```python\nimport mmcv\n\ndef func(item):\n # 执行相关操作\n pass\n\ntasks = [item_1, item_2, ..., item_n]\n\nmmcv.track_progress(func, tasks)\n```\n\n效果如下\n![progress](../../en/_static/progress.*)\n\n如果你想可视化多进程任务的进度，你可以使用 `track_parallel_progress` 。\n\n```python\nmmcv.track_parallel_progress(func, tasks, 8) # 8 workers\n```\n\n![progress](../../_static/parallel_progress.*)\n\n如果你想要迭代或枚举数据列表并可视化进度,你可以使用 `track_iter_progress` 。\n\n```python\nimport mmcv\n\ntasks = [item_1, item_2, ..., item_n]\n\nfor task in mmcv.track_iter_progress(tasks):\n # do something like print\n print(task)\n\nfor i, task in enumerate(mmcv.track_iter_progress(tasks)):\n # do something like print\n print(i)\n print(task)\n```\n\n### 计时器\n\nmmcv提供的 `Timer` 可以很方便地计算代码块的执行时间。\n\n```python\nimport time\n\nwith mmcv.Timer():\n # simulate some code block\n time.sleep(1)\n```\n\n你也可以使用 `since_start()` 和 `since_last_check()` 。前者返回计时器启动后的运行时长，后者返回最近一次查看计时器后的运行时长。\n\n\n```python\ntimer = mmcv.Timer()\n# code block 1 here\nprint(timer.since_start())\n# code block 2 here\nprint(timer.since_last_check())\nprint(timer.since_start())\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/docs/zh_cn/understand_mmcv/visualization.md", "content": "## 可视化\n\n`mmcv` 可以展示图像以及标注（目前只支持标注框）\n\n```python\n# 展示图像文件\nmmcv.imshow('a.jpg')\n\n# 展示已加载的图像\nimg = np.random.rand(100, 100, 3)\nmmcv.imshow(img)\n\n# 展示带有标注框的图像\nimg = np.random.rand(100, 100, 3)\nbboxes = np.array([[0, 0, 50, 50], [20, 20, 60, 60]])\nmmcv.imshow_bboxes(img, bboxes)\n```\n\n`mmcv` 也可以展示特殊的图像，例如光流\n\n```python\nflow = mmcv.flowread('test.flo')\nmmcv.flowshow(flow)\n```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/examples/train.py", "content": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nimport torchvision.transforms as transforms\nfrom torch.utils.data import DataLoader\nfrom torchvision.datasets import CIFAR10\n\nfrom mmcv.parallel import MMDataParallel\nfrom mmcv.runner import EpochBasedRunner\nfrom mmcv.utils import get_logger\n\n\nclass Model(nn.Module):\n\n def __init__(self):\n super(Model, self).__init__()\n self.conv1 = nn.Conv2d(3, 6, 5)\n self.pool = nn.MaxPool2d(2, 2)\n self.conv2 = nn.Conv2d(6, 16, 5)\n self.fc1 = nn.Linear(16 * 5 * 5, 120)\n self.fc2 = nn.Linear(120, 84)\n self.fc3 = nn.Linear(84, 10)\n self.loss_fn = nn.CrossEntropyLoss()\n\n def forward(self, x):\n x = self.pool(F.relu(self.conv1(x)))\n x = self.pool(F.relu(self.conv2(x)))\n x = x.view(-1, 16 * 5 * 5)\n x = F.relu(self.fc1(x))\n x = F.relu(self.fc2(x))\n x = self.fc3(x)\n return x\n\n def train_step(self, data, optimizer):\n images, labels = data\n predicts = self(images) # -> self.__call__() -> self.forward()\n loss = self.loss_fn(predicts, labels)\n return {'loss': loss}\n\n\nif __name__ == '__main__':\n model = Model()\n if torch.cuda.is_available():\n # only use gpu:0 to train\n # Solved issue https://github.com/open-mmlab/mmcv/issues/1470\n model = MMDataParallel(model.cuda(), device_ids=[0])\n\n # dataset and dataloader\n transform = transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n ])\n trainset = CIFAR10(\n root='data', train=True, download=True, transform=transform)\n trainloader = DataLoader(\n trainset, batch_size=128, shuffle=True, num_workers=2)\n\n optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)\n logger = get_logger('mmcv')\n # runner is a scheduler to manage the training\n runner = EpochBasedRunner(\n model,\n optimizer=optimizer,\n work_dir='./work_dir',\n logger=logger,\n max_epochs=4)\n\n # learning rate scheduler config\n lr_config = dict(policy='step', step=[2, 3])\n # configuration of optimizer\n optimizer_config = dict(grad_clip=None)\n # configuration of saving checkpoints periodically\n checkpoint_config = dict(interval=1)\n # save log periodically and multiple hooks can be used simultaneously\n log_config = dict(interval=100, hooks=[dict(type='TextLoggerHook')])\n # register hooks to runner and those hooks will be invoked automatically\n runner.register_training_hooks(\n lr_config=lr_config,\n optimizer_config=optimizer_config,\n checkpoint_config=checkpoint_config,\n log_config=log_config)\n\n runner.run([trainloader], [('train', 1)])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n# flake8: noqa\nfrom .arraymisc import *\nfrom .fileio import *\nfrom .image import *\nfrom .utils import *\nfrom .version import *\nfrom .video import *\nfrom .visualization import *\n\n# The following modules are not imported to this level, so mmcv may be used\n# without PyTorch.\n# - runner\n# - parallel\n# - op\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/arraymisc/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .quantization import dequantize, quantize\n\n__all__ = ['quantize', 'dequantize']\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/arraymisc/quantization.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numpy as np\n\n\ndef quantize(arr, min_val, max_val, levels, dtype=np.int64):\n \"\"\"Quantize an array of (-inf, inf) to [0, levels-1].\n\n Args:\n arr (ndarray): Input array.\n min_val (scalar): Minimum value to be clipped.\n max_val (scalar): Maximum value to be clipped.\n levels (int): Quantization levels.\n dtype (np.type): The type of the quantized array.\n\n Returns:\n tuple: Quantized array.\n \"\"\"\n if not (isinstance(levels, int) and levels > 1):\n raise ValueError(\n f'levels must be a positive integer, but got {levels}')\n if min_val >= max_val:\n raise ValueError(\n f'min_val ({min_val}) must be smaller than max_val ({max_val})')\n\n arr = np.clip(arr, min_val, max_val) - min_val\n quantized_arr = np.minimum(\n np.floor(levels * arr / (max_val - min_val)).astype(dtype), levels - 1)\n\n return quantized_arr\n\n\ndef dequantize(arr, min_val, max_val, levels, dtype=np.float64):\n \"\"\"Dequantize an array.\n\n Args:\n arr (ndarray): Input array.\n min_val (scalar): Minimum value to be clipped.\n max_val (scalar): Maximum value to be clipped.\n levels (int): Quantization levels.\n dtype (np.type): The type of the dequantized array.\n\n Returns:\n tuple: Dequantized array.\n \"\"\"\n if not (isinstance(levels, int) and levels > 1):\n raise ValueError(\n f'levels must be a positive integer, but got {levels}')\n if min_val >= max_val:\n raise ValueError(\n f'min_val ({min_val}) must be smaller than max_val ({max_val})')\n\n dequantized_arr = (arr + 0.5).astype(dtype) * (max_val -\n min_val) / levels + min_val\n\n return dequantized_arr\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .alexnet import AlexNet\n# yapf: disable\nfrom .bricks import (ACTIVATION_LAYERS, CONV_LAYERS, NORM_LAYERS,\n PADDING_LAYERS, PLUGIN_LAYERS, UPSAMPLE_LAYERS,\n ContextBlock, Conv2d, Conv3d, ConvAWS2d, ConvModule,\n ConvTranspose2d, ConvTranspose3d, ConvWS2d,\n DepthwiseSeparableConvModule, GeneralizedAttention,\n HSigmoid, HSwish, Linear, MaxPool2d, MaxPool3d,\n NonLocal1d, NonLocal2d, NonLocal3d, Scale, Swish,\n build_activation_layer, build_conv_layer,\n build_norm_layer, build_padding_layer, build_plugin_layer,\n build_upsample_layer, conv_ws_2d, is_norm)\nfrom .builder import MODELS, build_model_from_cfg\n# yapf: enable\nfrom .resnet import ResNet, make_res_layer\nfrom .utils import (INITIALIZERS, Caffe2XavierInit, ConstantInit, KaimingInit,\n NormalInit, PretrainedInit, TruncNormalInit, UniformInit,\n XavierInit, bias_init_with_prob, caffe2_xavier_init,\n constant_init, fuse_conv_bn, get_model_complexity_info,\n initialize, kaiming_init, normal_init, trunc_normal_init,\n uniform_init, xavier_init)\nfrom .vgg import VGG, make_vgg_layer\n\n__all__ = [\n 'AlexNet', 'VGG', 'make_vgg_layer', 'ResNet', 'make_res_layer',\n 'constant_init', 'xavier_init', 'normal_init', 'trunc_normal_init',\n 'uniform_init', 'kaiming_init', 'caffe2_xavier_init',\n 'bias_init_with_prob', 'ConvModule', 'build_activation_layer',\n 'build_conv_layer', 'build_norm_layer', 'build_padding_layer',\n 'build_upsample_layer', 'build_plugin_layer', 'is_norm', 'NonLocal1d',\n 'NonLocal2d', 'NonLocal3d', 'ContextBlock', 'HSigmoid', 'Swish', 'HSwish',\n 'GeneralizedAttention', 'ACTIVATION_LAYERS', 'CONV_LAYERS', 'NORM_LAYERS',\n 'PADDING_LAYERS', 'UPSAMPLE_LAYERS', 'PLUGIN_LAYERS', 'Scale',\n 'get_model_complexity_info', 'conv_ws_2d', 'ConvAWS2d', 'ConvWS2d',\n 'fuse_conv_bn', 'DepthwiseSeparableConvModule', 'Linear', 'Conv2d',\n 'ConvTranspose2d', 'MaxPool2d', 'ConvTranspose3d', 'MaxPool3d', 'Conv3d',\n 'initialize', 'INITIALIZERS', 'ConstantInit', 'XavierInit', 'NormalInit',\n 'TruncNormalInit', 'UniformInit', 'KaimingInit', 'PretrainedInit',\n 'Caffe2XavierInit', 'MODELS', 'build_model_from_cfg'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/alexnet.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport logging\n\nimport torch.nn as nn\n\n\nclass AlexNet(nn.Module):\n \"\"\"AlexNet backbone.\n\n Args:\n num_classes (int): number of classes for classification.\n \"\"\"\n\n def __init__(self, num_classes=-1):\n super(AlexNet, self).__init__()\n self.num_classes = num_classes\n self.features = nn.Sequential(\n nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=3, stride=2),\n nn.Conv2d(64, 192, kernel_size=5, padding=2),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=3, stride=2),\n nn.Conv2d(192, 384, kernel_size=3, padding=1),\n nn.ReLU(inplace=True),\n nn.Conv2d(384, 256, kernel_size=3, padding=1),\n nn.ReLU(inplace=True),\n nn.Conv2d(256, 256, kernel_size=3, padding=1),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=3, stride=2),\n )\n if self.num_classes > 0:\n self.classifier = nn.Sequential(\n nn.Dropout(),\n nn.Linear(256 * 6 * 6, 4096),\n nn.ReLU(inplace=True),\n nn.Dropout(),\n nn.Linear(4096, 4096),\n nn.ReLU(inplace=True),\n nn.Linear(4096, num_classes),\n )\n\n def init_weights(self, pretrained=None):\n if isinstance(pretrained, str):\n logger = logging.getLogger()\n from ..runner import load_checkpoint\n load_checkpoint(self, pretrained, strict=False, logger=logger)\n elif pretrained is None:\n # use default initializer\n pass\n else:\n raise TypeError('pretrained must be a str or None')\n\n def forward(self, x):\n\n x = self.features(x)\n if self.num_classes > 0:\n x = x.view(x.size(0), 256 * 6 * 6)\n x = self.classifier(x)\n\n return x\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .activation import build_activation_layer\nfrom .context_block import ContextBlock\nfrom .conv import build_conv_layer\nfrom .conv2d_adaptive_padding import Conv2dAdaptivePadding\nfrom .conv_module import ConvModule\nfrom .conv_ws import ConvAWS2d, ConvWS2d, conv_ws_2d\nfrom .depthwise_separable_conv_module import DepthwiseSeparableConvModule\nfrom .drop import Dropout, DropPath\nfrom .generalized_attention import GeneralizedAttention\nfrom .hsigmoid import HSigmoid\nfrom .hswish import HSwish\nfrom .non_local import NonLocal1d, NonLocal2d, NonLocal3d\nfrom .norm import build_norm_layer, is_norm\nfrom .padding import build_padding_layer\nfrom .plugin import build_plugin_layer\nfrom .registry import (ACTIVATION_LAYERS, CONV_LAYERS, NORM_LAYERS,\n PADDING_LAYERS, PLUGIN_LAYERS, UPSAMPLE_LAYERS)\nfrom .scale import Scale\nfrom .swish import Swish\nfrom .upsample import build_upsample_layer\nfrom .wrappers import (Conv2d, Conv3d, ConvTranspose2d, ConvTranspose3d,\n Linear, MaxPool2d, MaxPool3d)\n\n__all__ = [\n 'ConvModule', 'build_activation_layer', 'build_conv_layer',\n 'build_norm_layer', 'build_padding_layer', 'build_upsample_layer',\n 'build_plugin_layer', 'is_norm', 'HSigmoid', 'HSwish', 'NonLocal1d',\n 'NonLocal2d', 'NonLocal3d', 'ContextBlock', 'GeneralizedAttention',\n 'ACTIVATION_LAYERS', 'CONV_LAYERS', 'NORM_LAYERS', 'PADDING_LAYERS',\n 'UPSAMPLE_LAYERS', 'PLUGIN_LAYERS', 'Scale', 'ConvAWS2d', 'ConvWS2d',\n 'conv_ws_2d', 'DepthwiseSeparableConvModule', 'Swish', 'Linear',\n 'Conv2dAdaptivePadding', 'Conv2d', 'ConvTranspose2d', 'MaxPool2d',\n 'ConvTranspose3d', 'MaxPool3d', 'Conv3d', 'Dropout', 'DropPath'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/activation.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom mmcv.utils import TORCH_VERSION, build_from_cfg, digit_version\nfrom .registry import ACTIVATION_LAYERS\n\nfor module in [\n nn.ReLU, nn.LeakyReLU, nn.PReLU, nn.RReLU, nn.ReLU6, nn.ELU,\n nn.Sigmoid, nn.Tanh\n]:\n ACTIVATION_LAYERS.register_module(module=module)\n\n\n@ACTIVATION_LAYERS.register_module(name='Clip')\n@ACTIVATION_LAYERS.register_module()\nclass Clamp(nn.Module):\n \"\"\"Clamp activation layer.\n\n This activation function is to clamp the feature map value within\n :math:`[min, max]`. More details can be found in ``torch.clamp()``.\n\n Args:\n min (Number | optional): Lower-bound of the range to be clamped to.\n Default to -1.\n max (Number | optional): Upper-bound of the range to be clamped to.\n Default to 1.\n \"\"\"\n\n def __init__(self, min=-1., max=1.):\n super(Clamp, self).__init__()\n self.min = min\n self.max = max\n\n def forward(self, x):\n \"\"\"Forward function.\n\n Args:\n x (torch.Tensor): The input tensor.\n\n Returns:\n torch.Tensor: Clamped tensor.\n \"\"\"\n return torch.clamp(x, min=self.min, max=self.max)\n\n\nclass GELU(nn.Module):\n r\"\"\"Applies the Gaussian Error Linear Units function:\n\n .. math::\n \\text{GELU}(x) = x * \\Phi(x)\n where :math:`\\Phi(x)` is the Cumulative Distribution Function for\n Gaussian Distribution.\n\n Shape:\n - Input: :math:`(N, *)` where `*` means, any number of additional\n dimensions\n - Output: :math:`(N, *)`, same shape as the input\n\n .. image:: scripts/activation_images/GELU.png\n\n Examples::\n\n >>> m = nn.GELU()\n >>> input = torch.randn(2)\n >>> output = m(input)\n \"\"\"\n\n def forward(self, input):\n return F.gelu(input)\n\n\nif (TORCH_VERSION == 'parrots'\n or digit_version(TORCH_VERSION) < digit_version('1.4')):\n ACTIVATION_LAYERS.register_module(module=GELU)\nelse:\n ACTIVATION_LAYERS.register_module(module=nn.GELU)\n\n\ndef build_activation_layer(cfg):\n \"\"\"Build activation layer.\n\n Args:\n cfg (dict): The activation layer config, which should contain:\n\n - type (str): Layer type.\n - layer args: Args needed to instantiate an activation layer.\n\n Returns:\n nn.Module: Created activation layer.\n \"\"\"\n return build_from_cfg(cfg, ACTIVATION_LAYERS)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/context_block.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch import nn\n\nfrom ..utils import constant_init, kaiming_init\nfrom .registry import PLUGIN_LAYERS\n\n\ndef last_zero_init(m):\n if isinstance(m, nn.Sequential):\n constant_init(m[-1], val=0)\n else:\n constant_init(m, val=0)\n\n\n@PLUGIN_LAYERS.register_module()\nclass ContextBlock(nn.Module):\n \"\"\"ContextBlock module in GCNet.\n\n See 'GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond'\n (https://arxiv.org/abs/1904.11492) for details.\n\n Args:\n in_channels (int): Channels of the input feature map.\n ratio (float): Ratio of channels of transform bottleneck\n pooling_type (str): Pooling method for context modeling.\n Options are 'att' and 'avg', stand for attention pooling and\n average pooling respectively. Default: 'att'.\n fusion_types (Sequence[str]): Fusion method for feature fusion,\n Options are 'channels_add', 'channel_mul', stand for channelwise\n addition and multiplication respectively. Default: ('channel_add',)\n \"\"\"\n\n _abbr_ = 'context_block'\n\n def __init__(self,\n in_channels,\n ratio,\n pooling_type='att',\n fusion_types=('channel_add', )):\n super(ContextBlock, self).__init__()\n assert pooling_type in ['avg', 'att']\n assert isinstance(fusion_types, (list, tuple))\n valid_fusion_types = ['channel_add', 'channel_mul']\n assert all([f in valid_fusion_types for f in fusion_types])\n assert len(fusion_types) > 0, 'at least one fusion should be used'\n self.in_channels = in_channels\n self.ratio = ratio\n self.planes = int(in_channels * ratio)\n self.pooling_type = pooling_type\n self.fusion_types = fusion_types\n if pooling_type == 'att':\n self.conv_mask = nn.Conv2d(in_channels, 1, kernel_size=1)\n self.softmax = nn.Softmax(dim=2)\n else:\n self.avg_pool = nn.AdaptiveAvgPool2d(1)\n if 'channel_add' in fusion_types:\n self.channel_add_conv = nn.Sequential(\n nn.Conv2d(self.in_channels, self.planes, kernel_size=1),\n nn.LayerNorm([self.planes, 1, 1]),\n nn.ReLU(inplace=True), # yapf: disable\n nn.Conv2d(self.planes, self.in_channels, kernel_size=1))\n else:\n self.channel_add_conv = None\n if 'channel_mul' in fusion_types:\n self.channel_mul_conv = nn.Sequential(\n nn.Conv2d(self.in_channels, self.planes, kernel_size=1),\n nn.LayerNorm([self.planes, 1, 1]),\n nn.ReLU(inplace=True), # yapf: disable\n nn.Conv2d(self.planes, self.in_channels, kernel_size=1))\n else:\n self.channel_mul_conv = None\n self.reset_parameters()\n\n def reset_parameters(self):\n if self.pooling_type == 'att':\n kaiming_init(self.conv_mask, mode='fan_in')\n self.conv_mask.inited = True\n\n if self.channel_add_conv is not None:\n last_zero_init(self.channel_add_conv)\n if self.channel_mul_conv is not None:\n last_zero_init(self.channel_mul_conv)\n\n def spatial_pool(self, x):\n batch, channel, height, width = x.size()\n if self.pooling_type == 'att':\n input_x = x\n # [N, C, H * W]\n input_x = input_x.view(batch, channel, height * width)\n # [N, 1, C, H * W]\n input_x = input_x.unsqueeze(1)\n # [N, 1, H, W]\n context_mask = self.conv_mask(x)\n # [N, 1, H * W]\n context_mask = context_mask.view(batch, 1, height * width)\n # [N, 1, H * W]\n context_mask = self.softmax(context_mask)\n # [N, 1, H * W, 1]\n context_mask = context_mask.unsqueeze(-1)\n # [N, 1, C, 1]\n context = torch.matmul(input_x, context_mask)\n # [N, C, 1, 1]\n context = context.view(batch, channel, 1, 1)\n else:\n # [N, C, 1, 1]\n context = self.avg_pool(x)\n\n return context\n\n def forward(self, x):\n # [N, C, 1, 1]\n context = self.spatial_pool(x)\n\n out = x\n if self.channel_mul_conv is not None:\n # [N, C, 1, 1]\n channel_mul_term = torch.sigmoid(self.channel_mul_conv(context))\n out = out * channel_mul_term\n if self.channel_add_conv is not None:\n # [N, C, 1, 1]\n channel_add_term = self.channel_add_conv(context)\n out = out + channel_add_term\n\n return out\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/conv.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom torch import nn\n\nfrom .registry import CONV_LAYERS\n\nCONV_LAYERS.register_module('Conv1d', module=nn.Conv1d)\nCONV_LAYERS.register_module('Conv2d', module=nn.Conv2d)\nCONV_LAYERS.register_module('Conv3d', module=nn.Conv3d)\nCONV_LAYERS.register_module('Conv', module=nn.Conv2d)\n\n\ndef build_conv_layer(cfg, *args, **kwargs):\n \"\"\"Build convolution layer.\n\n Args:\n cfg (None or dict): The conv layer config, which should contain:\n - type (str): Layer type.\n - layer args: Args needed to instantiate an conv layer.\n args (argument list): Arguments passed to the `__init__`\n method of the corresponding conv layer.\n kwargs (keyword arguments): Keyword arguments passed to the `__init__`\n method of the corresponding conv layer.\n\n Returns:\n nn.Module: Created conv layer.\n \"\"\"\n if cfg is None:\n cfg_ = dict(type='Conv2d')\n else:\n if not isinstance(cfg, dict):\n raise TypeError('cfg must be a dict')\n if 'type' not in cfg:\n raise KeyError('the cfg dict must contain the key \"type\"')\n cfg_ = cfg.copy()\n\n layer_type = cfg_.pop('type')\n if layer_type not in CONV_LAYERS:\n raise KeyError(f'Unrecognized norm type {layer_type}')\n else:\n conv_layer = CONV_LAYERS.get(layer_type)\n\n layer = conv_layer(*args, **kwargs, **cfg_)\n\n return layer\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/conv2d_adaptive_padding.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport math\n\nfrom torch import nn\nfrom torch.nn import functional as F\n\nfrom .registry import CONV_LAYERS\n\n\n@CONV_LAYERS.register_module()\nclass Conv2dAdaptivePadding(nn.Conv2d):\n \"\"\"Implementation of 2D convolution in tensorflow with `padding` as \"same\",\n which applies padding to input (if needed) so that input image gets fully\n covered by filter and stride you specified. For stride 1, this will ensure\n that output image size is same as input. For stride of 2, output dimensions\n will be half, for example.\n\n Args:\n in_channels (int): Number of channels in the input image\n out_channels (int): Number of channels produced by the convolution\n kernel_size (int or tuple): Size of the convolving kernel\n stride (int or tuple, optional): Stride of the convolution. Default: 1\n padding (int or tuple, optional): Zero-padding added to both sides of\n the input. Default: 0\n dilation (int or tuple, optional): Spacing between kernel elements.\n Default: 1\n groups (int, optional): Number of blocked connections from input\n channels to output channels. Default: 1\n bias (bool, optional): If ``True``, adds a learnable bias to the\n output. Default: ``True``\n \"\"\"\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n bias=True):\n super().__init__(in_channels, out_channels, kernel_size, stride, 0,\n dilation, groups, bias)\n\n def forward(self, x):\n img_h, img_w = x.size()[-2:]\n kernel_h, kernel_w = self.weight.size()[-2:]\n stride_h, stride_w = self.stride\n output_h = math.ceil(img_h / stride_h)\n output_w = math.ceil(img_w / stride_w)\n pad_h = (\n max((output_h - 1) * self.stride[0] +\n (kernel_h - 1) * self.dilation[0] + 1 - img_h, 0))\n pad_w = (\n max((output_w - 1) * self.stride[1] +\n (kernel_w - 1) * self.dilation[1] + 1 - img_w, 0))\n if pad_h > 0 or pad_w > 0:\n x = F.pad(x, [\n pad_w // 2, pad_w - pad_w // 2, pad_h // 2, pad_h - pad_h // 2\n ])\n return F.conv2d(x, self.weight, self.bias, self.stride, self.padding,\n self.dilation, self.groups)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/conv_module.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport warnings\n\nimport torch.nn as nn\n\nfrom mmcv.utils import _BatchNorm, _InstanceNorm\nfrom ..utils import constant_init, kaiming_init\nfrom .activation import build_activation_layer\nfrom .conv import build_conv_layer\nfrom .norm import build_norm_layer\nfrom .padding import build_padding_layer\nfrom .registry import PLUGIN_LAYERS\n\n\n@PLUGIN_LAYERS.register_module()\nclass ConvModule(nn.Module):\n \"\"\"A conv block that bundles conv/norm/activation layers.\n\n This block simplifies the usage of convolution layers, which are commonly\n used with a norm layer (e.g., BatchNorm) and activation layer (e.g., ReLU).\n It is based upon three build methods: `build_conv_layer()`,\n `build_norm_layer()` and `build_activation_layer()`.\n\n Besides, we add some additional features in this module.\n 1. Automatically set `bias` of the conv layer.\n 2. Spectral norm is supported.\n 3. More padding modes are supported. Before PyTorch 1.5, nn.Conv2d only\n supports zero and circular padding, and we add \"reflect\" padding mode.\n\n Args:\n in_channels (int): Number of channels in the input feature map.\n Same as that in ``nn._ConvNd``.\n out_channels (int): Number of channels produced by the convolution.\n Same as that in ``nn._ConvNd``.\n kernel_size (int | tuple[int]): Size of the convolving kernel.\n Same as that in ``nn._ConvNd``.\n stride (int | tuple[int]): Stride of the convolution.\n Same as that in ``nn._ConvNd``.\n padding (int | tuple[int]): Zero-padding added to both sides of\n the input. Same as that in ``nn._ConvNd``.\n dilation (int | tuple[int]): Spacing between kernel elements.\n Same as that in ``nn._ConvNd``.\n groups (int): Number of blocked connections from input channels to\n output channels. Same as that in ``nn._ConvNd``.\n bias (bool | str): If specified as `auto`, it will be decided by the\n norm_cfg. Bias will be set as True if `norm_cfg` is None, otherwise\n False. Default: \"auto\".\n conv_cfg (dict): Config dict for convolution layer. Default: None,\n which means using conv2d.\n norm_cfg (dict): Config dict for normalization layer. Default: None.\n act_cfg (dict): Config dict for activation layer.\n Default: dict(type='ReLU').\n inplace (bool): Whether to use inplace mode for activation.\n Default: True.\n with_spectral_norm (bool): Whether use spectral norm in conv module.\n Default: False.\n padding_mode (str): If the `padding_mode` has not been supported by\n current `Conv2d` in PyTorch, we will use our own padding layer\n instead. Currently, we support ['zeros', 'circular'] with official\n implementation and ['reflect'] with our own implementation.\n Default: 'zeros'.\n order (tuple[str]): The order of conv/norm/activation layers. It is a\n sequence of \"conv\", \"norm\" and \"act\". Common examples are\n (\"conv\", \"norm\", \"act\") and (\"act\", \"conv\", \"norm\").\n Default: ('conv', 'norm', 'act').\n \"\"\"\n\n _abbr_ = 'conv_block'\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n bias='auto',\n conv_cfg=None,\n norm_cfg=None,\n act_cfg=dict(type='ReLU'),\n inplace=True,\n with_spectral_norm=False,\n padding_mode='zeros',\n order=('conv', 'norm', 'act')):\n super(ConvModule, self).__init__()\n assert conv_cfg is None or isinstance(conv_cfg, dict)\n assert norm_cfg is None or isinstance(norm_cfg, dict)\n assert act_cfg is None or isinstance(act_cfg, dict)\n official_padding_mode = ['zeros', 'circular']\n self.conv_cfg = conv_cfg\n self.norm_cfg = norm_cfg\n self.act_cfg = act_cfg\n self.inplace = inplace\n self.with_spectral_norm = with_spectral_norm\n self.with_explicit_padding = padding_mode not in official_padding_mode\n self.order = order\n assert isinstance(self.order, tuple) and len(self.order) == 3\n assert set(order) == set(['conv', 'norm', 'act'])\n\n self.with_norm = norm_cfg is not None\n self.with_activation = act_cfg is not None\n # if the conv layer is before a norm layer, bias is unnecessary.\n if bias == 'auto':\n bias = not self.with_norm\n self.with_bias = bias\n\n if self.with_explicit_padding:\n pad_cfg = dict(type=padding_mode)\n self.padding_layer = build_padding_layer(pad_cfg, padding)\n\n # reset padding to 0 for conv module\n conv_padding = 0 if self.with_explicit_padding else padding\n # build convolution layer\n self.conv = build_conv_layer(\n conv_cfg,\n in_channels,\n out_channels,\n kernel_size,\n stride=stride,\n padding=conv_padding,\n dilation=dilation,\n groups=groups,\n bias=bias)\n # export the attributes of self.conv to a higher level for convenience\n self.in_channels = self.conv.in_channels\n self.out_channels = self.conv.out_channels\n self.kernel_size = self.conv.kernel_size\n self.stride = self.conv.stride\n self.padding = padding\n self.dilation = self.conv.dilation\n self.transposed = self.conv.transposed\n self.output_padding = self.conv.output_padding\n self.groups = self.conv.groups\n\n if self.with_spectral_norm:\n self.conv = nn.utils.spectral_norm(self.conv)\n\n # build normalization layers\n if self.with_norm:\n # norm layer is after conv layer\n if order.index('norm') > order.index('conv'):\n norm_channels = out_channels\n else:\n norm_channels = in_channels\n self.norm_name, norm = build_norm_layer(norm_cfg, norm_channels)\n self.add_module(self.norm_name, norm)\n if self.with_bias:\n if isinstance(norm, (_BatchNorm, _InstanceNorm)):\n warnings.warn(\n 'Unnecessary conv bias before batch/instance norm')\n else:\n self.norm_name = None\n\n # build activation layer\n if self.with_activation:\n act_cfg_ = act_cfg.copy()\n # nn.Tanh has no 'inplace' argument\n if act_cfg_['type'] not in [\n 'Tanh', 'PReLU', 'Sigmoid', 'HSigmoid', 'Swish'\n ]:\n act_cfg_.setdefault('inplace', inplace)\n self.activate = build_activation_layer(act_cfg_)\n\n # Use msra init by default\n self.init_weights()\n\n @property\n def norm(self):\n if self.norm_name:\n return getattr(self, self.norm_name)\n else:\n return None\n\n def init_weights(self):\n # 1. It is mainly for customized conv layers with their own\n # initialization manners by calling their own ``init_weights()``,\n # and we do not want ConvModule to override the initialization.\n # 2. For customized conv layers without their own initialization\n # manners (that is, they don't have their own ``init_weights()``)\n # and PyTorch's conv layers, they will be initialized by\n # this method with default ``kaiming_init``.\n # Note: For PyTorch's conv layers, they will be overwritten by our\n # initialization implementation using default ``kaiming_init``.\n if not hasattr(self.conv, 'init_weights'):\n if self.with_activation and self.act_cfg['type'] == 'LeakyReLU':\n nonlinearity = 'leaky_relu'\n a = self.act_cfg.get('negative_slope', 0.01)\n else:\n nonlinearity = 'relu'\n a = 0\n kaiming_init(self.conv, a=a, nonlinearity=nonlinearity)\n if self.with_norm:\n constant_init(self.norm, 1, bias=0)\n\n def forward(self, x, activate=True, norm=True):\n for layer in self.order:\n if layer == 'conv':\n if self.with_explicit_padding:\n x = self.padding_layer(x)\n x = self.conv(x)\n elif layer == 'norm' and norm and self.with_norm:\n x = self.norm(x)\n elif layer == 'act' and activate and self.with_activation:\n x = self.activate(x)\n return x\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/conv_ws.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom .registry import CONV_LAYERS\n\n\ndef conv_ws_2d(input,\n weight,\n bias=None,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n eps=1e-5):\n c_in = weight.size(0)\n weight_flat = weight.view(c_in, -1)\n mean = weight_flat.mean(dim=1, keepdim=True).view(c_in, 1, 1, 1)\n std = weight_flat.std(dim=1, keepdim=True).view(c_in, 1, 1, 1)\n weight = (weight - mean) / (std + eps)\n return F.conv2d(input, weight, bias, stride, padding, dilation, groups)\n\n\n@CONV_LAYERS.register_module('ConvWS')\nclass ConvWS2d(nn.Conv2d):\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n bias=True,\n eps=1e-5):\n super(ConvWS2d, self).__init__(\n in_channels,\n out_channels,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n groups=groups,\n bias=bias)\n self.eps = eps\n\n def forward(self, x):\n return conv_ws_2d(x, self.weight, self.bias, self.stride, self.padding,\n self.dilation, self.groups, self.eps)\n\n\n@CONV_LAYERS.register_module(name='ConvAWS')\nclass ConvAWS2d(nn.Conv2d):\n \"\"\"AWS (Adaptive Weight Standardization)\n\n This is a variant of Weight Standardization\n (https://arxiv.org/pdf/1903.10520.pdf)\n It is used in DetectoRS to avoid NaN\n (https://arxiv.org/pdf/2006.02334.pdf)\n\n Args:\n in_channels (int): Number of channels in the input image\n out_channels (int): Number of channels produced by the convolution\n kernel_size (int or tuple): Size of the conv kernel\n stride (int or tuple, optional): Stride of the convolution. Default: 1\n padding (int or tuple, optional): Zero-padding added to both sides of\n the input. Default: 0\n dilation (int or tuple, optional): Spacing between kernel elements.\n Default: 1\n groups (int, optional): Number of blocked connections from input\n channels to output channels. Default: 1\n bias (bool, optional): If set True, adds a learnable bias to the\n output. Default: True\n \"\"\"\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n bias=True):\n super().__init__(\n in_channels,\n out_channels,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n groups=groups,\n bias=bias)\n self.register_buffer('weight_gamma',\n torch.ones(self.out_channels, 1, 1, 1))\n self.register_buffer('weight_beta',\n torch.zeros(self.out_channels, 1, 1, 1))\n\n def _get_weight(self, weight):\n weight_flat = weight.view(weight.size(0), -1)\n mean = weight_flat.mean(dim=1).view(-1, 1, 1, 1)\n std = torch.sqrt(weight_flat.var(dim=1) + 1e-5).view(-1, 1, 1, 1)\n weight = (weight - mean) / std\n weight = self.weight_gamma * weight + self.weight_beta\n return weight\n\n def forward(self, x):\n weight = self._get_weight(self.weight)\n return F.conv2d(x, weight, self.bias, self.stride, self.padding,\n self.dilation, self.groups)\n\n def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,\n missing_keys, unexpected_keys, error_msgs):\n \"\"\"Override default load function.\n\n AWS overrides the function _load_from_state_dict to recover\n weight_gamma and weight_beta if they are missing. If weight_gamma and\n weight_beta are found in the checkpoint, this function will return\n after super()._load_from_state_dict. Otherwise, it will compute the\n mean and std of the pretrained weights and store them in weight_beta\n and weight_gamma.\n \"\"\"\n\n self.weight_gamma.data.fill_(-1)\n local_missing_keys = []\n super()._load_from_state_dict(state_dict, prefix, local_metadata,\n strict, local_missing_keys,\n unexpected_keys, error_msgs)\n if self.weight_gamma.data.mean() > 0:\n for k in local_missing_keys:\n missing_keys.append(k)\n return\n weight = self.weight.data\n weight_flat = weight.view(weight.size(0), -1)\n mean = weight_flat.mean(dim=1).view(-1, 1, 1, 1)\n std = torch.sqrt(weight_flat.var(dim=1) + 1e-5).view(-1, 1, 1, 1)\n self.weight_beta.data.copy_(mean)\n self.weight_gamma.data.copy_(std)\n missing_gamma_beta = [\n k for k in local_missing_keys\n if k.endswith('weight_gamma') or k.endswith('weight_beta')\n ]\n for k in missing_gamma_beta:\n local_missing_keys.remove(k)\n for k in local_missing_keys:\n missing_keys.append(k)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/depthwise_separable_conv_module.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch.nn as nn\n\nfrom .conv_module import ConvModule\n\n\nclass DepthwiseSeparableConvModule(nn.Module):\n \"\"\"Depthwise separable convolution module.\n\n See https://arxiv.org/pdf/1704.04861.pdf for details.\n\n This module can replace a ConvModule with the conv block replaced by two\n conv block: depthwise conv block and pointwise conv block. The depthwise\n conv block contains depthwise-conv/norm/activation layers. The pointwise\n conv block contains pointwise-conv/norm/activation layers. It should be\n noted that there will be norm/activation layer in the depthwise conv block\n if `norm_cfg` and `act_cfg` are specified.\n\n Args:\n in_channels (int): Number of channels in the input feature map.\n Same as that in ``nn._ConvNd``.\n out_channels (int): Number of channels produced by the convolution.\n Same as that in ``nn._ConvNd``.\n kernel_size (int | tuple[int]): Size of the convolving kernel.\n Same as that in ``nn._ConvNd``.\n stride (int | tuple[int]): Stride of the convolution.\n Same as that in ``nn._ConvNd``. Default: 1.\n padding (int | tuple[int]): Zero-padding added to both sides of\n the input. Same as that in ``nn._ConvNd``. Default: 0.\n dilation (int | tuple[int]): Spacing between kernel elements.\n Same as that in ``nn._ConvNd``. Default: 1.\n norm_cfg (dict): Default norm config for both depthwise ConvModule and\n pointwise ConvModule. Default: None.\n act_cfg (dict): Default activation config for both depthwise ConvModule\n and pointwise ConvModule. Default: dict(type='ReLU').\n dw_norm_cfg (dict): Norm config of depthwise ConvModule. If it is\n 'default', it will be the same as `norm_cfg`. Default: 'default'.\n dw_act_cfg (dict): Activation config of depthwise ConvModule. If it is\n 'default', it will be the same as `act_cfg`. Default: 'default'.\n pw_norm_cfg (dict): Norm config of pointwise ConvModule. If it is\n 'default', it will be the same as `norm_cfg`. Default: 'default'.\n pw_act_cfg (dict): Activation config of pointwise ConvModule. If it is\n 'default', it will be the same as `act_cfg`. Default: 'default'.\n kwargs (optional): Other shared arguments for depthwise and pointwise\n ConvModule. See ConvModule for ref.\n \"\"\"\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n norm_cfg=None,\n act_cfg=dict(type='ReLU'),\n dw_norm_cfg='default',\n dw_act_cfg='default',\n pw_norm_cfg='default',\n pw_act_cfg='default',\n **kwargs):\n super(DepthwiseSeparableConvModule, self).__init__()\n assert 'groups' not in kwargs, 'groups should not be specified'\n\n # if norm/activation config of depthwise/pointwise ConvModule is not\n # specified, use default config.\n dw_norm_cfg = dw_norm_cfg if dw_norm_cfg != 'default' else norm_cfg\n dw_act_cfg = dw_act_cfg if dw_act_cfg != 'default' else act_cfg\n pw_norm_cfg = pw_norm_cfg if pw_norm_cfg != 'default' else norm_cfg\n pw_act_cfg = pw_act_cfg if pw_act_cfg != 'default' else act_cfg\n\n # depthwise convolution\n self.depthwise_conv = ConvModule(\n in_channels,\n in_channels,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n groups=in_channels,\n norm_cfg=dw_norm_cfg,\n act_cfg=dw_act_cfg,\n **kwargs)\n\n self.pointwise_conv = ConvModule(\n in_channels,\n out_channels,\n 1,\n norm_cfg=pw_norm_cfg,\n act_cfg=pw_act_cfg,\n **kwargs)\n\n def forward(self, x):\n x = self.depthwise_conv(x)\n x = self.pointwise_conv(x)\n return x\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/drop.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\n\nfrom mmcv import build_from_cfg\nfrom .registry import DROPOUT_LAYERS\n\n\ndef drop_path(x, drop_prob=0., training=False):\n \"\"\"Drop paths (Stochastic Depth) per sample (when applied in main path of\n residual blocks).\n\n We follow the implementation\n https://github.com/rwightman/pytorch-image-models/blob/a2727c1bf78ba0d7b5727f5f95e37fb7f8866b1f/timm/models/layers/drop.py # noqa: E501\n \"\"\"\n if drop_prob == 0. or not training:\n return x\n keep_prob = 1 - drop_prob\n # handle tensors with different dimensions, not just 4D tensors.\n shape = (x.shape[0], ) + (1, ) * (x.ndim - 1)\n random_tensor = keep_prob + torch.rand(\n shape, dtype=x.dtype, device=x.device)\n output = x.div(keep_prob) * random_tensor.floor()\n return output\n\n\n@DROPOUT_LAYERS.register_module()\nclass DropPath(nn.Module):\n \"\"\"Drop paths (Stochastic Depth) per sample (when applied in main path of\n residual blocks).\n\n We follow the implementation\n https://github.com/rwightman/pytorch-image-models/blob/a2727c1bf78ba0d7b5727f5f95e37fb7f8866b1f/timm/models/layers/drop.py # noqa: E501\n\n Args:\n drop_prob (float): Probability of the path to be zeroed. Default: 0.1\n \"\"\"\n\n def __init__(self, drop_prob=0.1):\n super(DropPath, self).__init__()\n self.drop_prob = drop_prob\n\n def forward(self, x):\n return drop_path(x, self.drop_prob, self.training)\n\n\n@DROPOUT_LAYERS.register_module()\nclass Dropout(nn.Dropout):\n \"\"\"A wrapper for ``torch.nn.Dropout``, We rename the ``p`` of\n ``torch.nn.Dropout`` to ``drop_prob`` so as to be consistent with\n ``DropPath``\n\n Args:\n drop_prob (float): Probability of the elements to be\n zeroed. Default: 0.5.\n inplace (bool): Do the operation inplace or not. Default: False.\n \"\"\"\n\n def __init__(self, drop_prob=0.5, inplace=False):\n super().__init__(p=drop_prob, inplace=inplace)\n\n\ndef build_dropout(cfg, default_args=None):\n \"\"\"Builder for drop out layers.\"\"\"\n return build_from_cfg(cfg, DROPOUT_LAYERS, default_args)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/generalized_attention.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport math\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom ..utils import kaiming_init\nfrom .registry import PLUGIN_LAYERS\n\n\n@PLUGIN_LAYERS.register_module()\nclass GeneralizedAttention(nn.Module):\n \"\"\"GeneralizedAttention module.\n\n See 'An Empirical Study of Spatial Attention Mechanisms in Deep Networks'\n (https://arxiv.org/abs/1711.07971) for details.\n\n Args:\n in_channels (int): Channels of the input feature map.\n spatial_range (int): The spatial range. -1 indicates no spatial range\n constraint. Default: -1.\n num_heads (int): The head number of empirical_attention module.\n Default: 9.\n position_embedding_dim (int): The position embedding dimension.\n Default: -1.\n position_magnitude (int): A multiplier acting on coord difference.\n Default: 1.\n kv_stride (int): The feature stride acting on key/value feature map.\n Default: 2.\n q_stride (int): The feature stride acting on query feature map.\n Default: 1.\n attention_type (str): A binary indicator string for indicating which\n items in generalized empirical_attention module are used.\n Default: '1111'.\n\n - '1000' indicates 'query and key content' (appr - appr) item,\n - '0100' indicates 'query content and relative position'\n (appr - position) item,\n - '0010' indicates 'key content only' (bias - appr) item,\n - '0001' indicates 'relative position only' (bias - position) item.\n \"\"\"\n\n _abbr_ = 'gen_attention_block'\n\n def __init__(self,\n in_channels,\n spatial_range=-1,\n num_heads=9,\n position_embedding_dim=-1,\n position_magnitude=1,\n kv_stride=2,\n q_stride=1,\n attention_type='1111'):\n\n super(GeneralizedAttention, self).__init__()\n\n # hard range means local range for non-local operation\n self.position_embedding_dim = (\n position_embedding_dim\n if position_embedding_dim > 0 else in_channels)\n\n self.position_magnitude = position_magnitude\n self.num_heads = num_heads\n self.in_channels = in_channels\n self.spatial_range = spatial_range\n self.kv_stride = kv_stride\n self.q_stride = q_stride\n self.attention_type = [bool(int(_)) for _ in attention_type]\n self.qk_embed_dim = in_channels // num_heads\n out_c = self.qk_embed_dim * num_heads\n\n if self.attention_type[0] or self.attention_type[1]:\n self.query_conv = nn.Conv2d(\n in_channels=in_channels,\n out_channels=out_c,\n kernel_size=1,\n bias=False)\n self.query_conv.kaiming_init = True\n\n if self.attention_type[0] or self.attention_type[2]:\n self.key_conv = nn.Conv2d(\n in_channels=in_channels,\n out_channels=out_c,\n kernel_size=1,\n bias=False)\n self.key_conv.kaiming_init = True\n\n self.v_dim = in_channels // num_heads\n self.value_conv = nn.Conv2d(\n in_channels=in_channels,\n out_channels=self.v_dim * num_heads,\n kernel_size=1,\n bias=False)\n self.value_conv.kaiming_init = True\n\n if self.attention_type[1] or self.attention_type[3]:\n self.appr_geom_fc_x = nn.Linear(\n self.position_embedding_dim // 2, out_c, bias=False)\n self.appr_geom_fc_x.kaiming_init = True\n\n self.appr_geom_fc_y = nn.Linear(\n self.position_embedding_dim // 2, out_c, bias=False)\n self.appr_geom_fc_y.kaiming_init = True\n\n if self.attention_type[2]:\n stdv = 1.0 / math.sqrt(self.qk_embed_dim * 2)\n appr_bias_value = -2 * stdv * torch.rand(out_c) + stdv\n self.appr_bias = nn.Parameter(appr_bias_value)\n\n if self.attention_type[3]:\n stdv = 1.0 / math.sqrt(self.qk_embed_dim * 2)\n geom_bias_value = -2 * stdv * torch.rand(out_c) + stdv\n self.geom_bias = nn.Parameter(geom_bias_value)\n\n self.proj_conv = nn.Conv2d(\n in_channels=self.v_dim * num_heads,\n out_channels=in_channels,\n kernel_size=1,\n bias=True)\n self.proj_conv.kaiming_init = True\n self.gamma = nn.Parameter(torch.zeros(1))\n\n if self.spatial_range >= 0:\n # only works when non local is after 3*3 conv\n if in_channels == 256:\n max_len = 84\n elif in_channels == 512:\n max_len = 42\n\n max_len_kv = int((max_len - 1.0) / self.kv_stride + 1)\n local_constraint_map = np.ones(\n (max_len, max_len, max_len_kv, max_len_kv), dtype=int)\n for iy in range(max_len):\n for ix in range(max_len):\n local_constraint_map[\n iy, ix,\n max((iy - self.spatial_range) //\n self.kv_stride, 0):min((iy + self.spatial_range +\n 1) // self.kv_stride +\n 1, max_len),\n max((ix - self.spatial_range) //\n self.kv_stride, 0):min((ix + self.spatial_range +\n 1) // self.kv_stride +\n 1, max_len)] = 0\n\n self.local_constraint_map = nn.Parameter(\n torch.from_numpy(local_constraint_map).byte(),\n requires_grad=False)\n\n if self.q_stride > 1:\n self.q_downsample = nn.AvgPool2d(\n kernel_size=1, stride=self.q_stride)\n else:\n self.q_downsample = None\n\n if self.kv_stride > 1:\n self.kv_downsample = nn.AvgPool2d(\n kernel_size=1, stride=self.kv_stride)\n else:\n self.kv_downsample = None\n\n self.init_weights()\n\n def get_position_embedding(self,\n h,\n w,\n h_kv,\n w_kv,\n q_stride,\n kv_stride,\n device,\n dtype,\n feat_dim,\n wave_length=1000):\n # the default type of Tensor is float32, leading to type mismatch\n # in fp16 mode. Cast it to support fp16 mode.\n h_idxs = torch.linspace(0, h - 1, h).to(device=device, dtype=dtype)\n h_idxs = h_idxs.view((h, 1)) * q_stride\n\n w_idxs = torch.linspace(0, w - 1, w).to(device=device, dtype=dtype)\n w_idxs = w_idxs.view((w, 1)) * q_stride\n\n h_kv_idxs = torch.linspace(0, h_kv - 1, h_kv).to(\n device=device, dtype=dtype)\n h_kv_idxs = h_kv_idxs.view((h_kv, 1)) * kv_stride\n\n w_kv_idxs = torch.linspace(0, w_kv - 1, w_kv).to(\n device=device, dtype=dtype)\n w_kv_idxs = w_kv_idxs.view((w_kv, 1)) * kv_stride\n\n # (h, h_kv, 1)\n h_diff = h_idxs.unsqueeze(1) - h_kv_idxs.unsqueeze(0)\n h_diff *= self.position_magnitude\n\n # (w, w_kv, 1)\n w_diff = w_idxs.unsqueeze(1) - w_kv_idxs.unsqueeze(0)\n w_diff *= self.position_magnitude\n\n feat_range = torch.arange(0, feat_dim / 4).to(\n device=device, dtype=dtype)\n\n dim_mat = torch.Tensor([wave_length]).to(device=device, dtype=dtype)\n dim_mat = dim_mat**((4. / feat_dim) * feat_range)\n dim_mat = dim_mat.view((1, 1, -1))\n\n embedding_x = torch.cat(\n ((w_diff / dim_mat).sin(), (w_diff / dim_mat).cos()), dim=2)\n\n embedding_y = torch.cat(\n ((h_diff / dim_mat).sin(), (h_diff / dim_mat).cos()), dim=2)\n\n return embedding_x, embedding_y\n\n def forward(self, x_input):\n num_heads = self.num_heads\n\n # use empirical_attention\n if self.q_downsample is not None:\n x_q = self.q_downsample(x_input)\n else:\n x_q = x_input\n n, _, h, w = x_q.shape\n\n if self.kv_downsample is not None:\n x_kv = self.kv_downsample(x_input)\n else:\n x_kv = x_input\n _, _, h_kv, w_kv = x_kv.shape\n\n if self.attention_type[0] or self.attention_type[1]:\n proj_query = self.query_conv(x_q).view(\n (n, num_heads, self.qk_embed_dim, h * w))\n proj_query = proj_query.permute(0, 1, 3, 2)\n\n if self.attention_type[0] or self.attention_type[2]:\n proj_key = self.key_conv(x_kv).view(\n (n, num_heads, self.qk_embed_dim, h_kv * w_kv))\n\n if self.attention_type[1] or self.attention_type[3]:\n position_embed_x, position_embed_y = self.get_position_embedding(\n h, w, h_kv, w_kv, self.q_stride, self.kv_stride,\n x_input.device, x_input.dtype, self.position_embedding_dim)\n # (n, num_heads, w, w_kv, dim)\n position_feat_x = self.appr_geom_fc_x(position_embed_x).\\\n view(1, w, w_kv, num_heads, self.qk_embed_dim).\\\n permute(0, 3, 1, 2, 4).\\\n repeat(n, 1, 1, 1, 1)\n\n # (n, num_heads, h, h_kv, dim)\n position_feat_y = self.appr_geom_fc_y(position_embed_y).\\\n view(1, h, h_kv, num_heads, self.qk_embed_dim).\\\n permute(0, 3, 1, 2, 4).\\\n repeat(n, 1, 1, 1, 1)\n\n position_feat_x /= math.sqrt(2)\n position_feat_y /= math.sqrt(2)\n\n # accelerate for saliency only\n if (np.sum(self.attention_type) == 1) and self.attention_type[2]:\n appr_bias = self.appr_bias.\\\n view(1, num_heads, 1, self.qk_embed_dim).\\\n repeat(n, 1, 1, 1)\n\n energy = torch.matmul(appr_bias, proj_key).\\\n view(n, num_heads, 1, h_kv * w_kv)\n\n h = 1\n w = 1\n else:\n # (n, num_heads, h*w, h_kv*w_kv), query before key, 540mb for\n if not self.attention_type[0]:\n energy = torch.zeros(\n n,\n num_heads,\n h,\n w,\n h_kv,\n w_kv,\n dtype=x_input.dtype,\n device=x_input.device)\n\n # attention_type[0]: appr - appr\n # attention_type[1]: appr - position\n # attention_type[2]: bias - appr\n # attention_type[3]: bias - position\n if self.attention_type[0] or self.attention_type[2]:\n if self.attention_type[0] and self.attention_type[2]:\n appr_bias = self.appr_bias.\\\n view(1, num_heads, 1, self.qk_embed_dim)\n energy = torch.matmul(proj_query + appr_bias, proj_key).\\\n view(n, num_heads, h, w, h_kv, w_kv)\n\n elif self.attention_type[0]:\n energy = torch.matmul(proj_query, proj_key).\\\n view(n, num_heads, h, w, h_kv, w_kv)\n\n elif self.attention_type[2]:\n appr_bias = self.appr_bias.\\\n view(1, num_heads, 1, self.qk_embed_dim).\\\n repeat(n, 1, 1, 1)\n\n energy += torch.matmul(appr_bias, proj_key).\\\n view(n, num_heads, 1, 1, h_kv, w_kv)\n\n if self.attention_type[1] or self.attention_type[3]:\n if self.attention_type[1] and self.attention_type[3]:\n geom_bias = self.geom_bias.\\\n view(1, num_heads, 1, self.qk_embed_dim)\n\n proj_query_reshape = (proj_query + geom_bias).\\\n view(n, num_heads, h, w, self.qk_embed_dim)\n\n energy_x = torch.matmul(\n proj_query_reshape.permute(0, 1, 3, 2, 4),\n position_feat_x.permute(0, 1, 2, 4, 3))\n energy_x = energy_x.\\\n permute(0, 1, 3, 2, 4).unsqueeze(4)\n\n energy_y = torch.matmul(\n proj_query_reshape,\n position_feat_y.permute(0, 1, 2, 4, 3))\n energy_y = energy_y.unsqueeze(5)\n\n energy += energy_x + energy_y\n\n elif self.attention_type[1]:\n proj_query_reshape = proj_query.\\\n view(n, num_heads, h, w, self.qk_embed_dim)\n proj_query_reshape = proj_query_reshape.\\\n permute(0, 1, 3, 2, 4)\n position_feat_x_reshape = position_feat_x.\\\n permute(0, 1, 2, 4, 3)\n position_feat_y_reshape = position_feat_y.\\\n permute(0, 1, 2, 4, 3)\n\n energy_x = torch.matmul(proj_query_reshape,\n position_feat_x_reshape)\n energy_x = energy_x.permute(0, 1, 3, 2, 4).unsqueeze(4)\n\n energy_y = torch.matmul(proj_query_reshape,\n position_feat_y_reshape)\n energy_y = energy_y.unsqueeze(5)\n\n energy += energy_x + energy_y\n\n elif self.attention_type[3]:\n geom_bias = self.geom_bias.\\\n view(1, num_heads, self.qk_embed_dim, 1).\\\n repeat(n, 1, 1, 1)\n\n position_feat_x_reshape = position_feat_x.\\\n view(n, num_heads, w*w_kv, self.qk_embed_dim)\n\n position_feat_y_reshape = position_feat_y.\\\n view(n, num_heads, h * h_kv, self.qk_embed_dim)\n\n energy_x = torch.matmul(position_feat_x_reshape, geom_bias)\n energy_x = energy_x.view(n, num_heads, 1, w, 1, w_kv)\n\n energy_y = torch.matmul(position_feat_y_reshape, geom_bias)\n energy_y = energy_y.view(n, num_heads, h, 1, h_kv, 1)\n\n energy += energy_x + energy_y\n\n energy = energy.view(n, num_heads, h * w, h_kv * w_kv)\n\n if self.spatial_range >= 0:\n cur_local_constraint_map = \\\n self.local_constraint_map[:h, :w, :h_kv, :w_kv].\\\n contiguous().\\\n view(1, 1, h*w, h_kv*w_kv)\n\n energy = energy.masked_fill_(cur_local_constraint_map,\n float('-inf'))\n\n attention = F.softmax(energy, 3)\n\n proj_value = self.value_conv(x_kv)\n proj_value_reshape = proj_value.\\\n view((n, num_heads, self.v_dim, h_kv * w_kv)).\\\n permute(0, 1, 3, 2)\n\n out = torch.matmul(attention, proj_value_reshape).\\\n permute(0, 1, 3, 2).\\\n contiguous().\\\n view(n, self.v_dim * self.num_heads, h, w)\n\n out = self.proj_conv(out)\n\n # output is downsampled, upsample back to input size\n if self.q_downsample is not None:\n out = F.interpolate(\n out,\n size=x_input.shape[2:],\n mode='bilinear',\n align_corners=False)\n\n out = self.gamma * out + x_input\n return out\n\n def init_weights(self):\n for m in self.modules():\n if hasattr(m, 'kaiming_init') and m.kaiming_init:\n kaiming_init(\n m,\n mode='fan_in',\n nonlinearity='leaky_relu',\n bias=0,\n distribution='uniform',\n a=1)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/hsigmoid.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport warnings\n\nimport torch.nn as nn\n\nfrom .registry import ACTIVATION_LAYERS\n\n\n@ACTIVATION_LAYERS.register_module()\nclass HSigmoid(nn.Module):\n \"\"\"Hard Sigmoid Module. Apply the hard sigmoid function:\n Hsigmoid(x) = min(max((x + bias) / divisor, min_value), max_value)\n Default: Hsigmoid(x) = min(max((x + 3) / 6, 0), 1)\n\n Note:\n In MMCV v1.4.4, we modified the default value of args to align with\n PyTorch official.\n\n Args:\n bias (float): Bias of the input feature map. Default: 3.0.\n divisor (float): Divisor of the input feature map. Default: 6.0.\n min_value (float): Lower bound value. Default: 0.0.\n max_value (float): Upper bound value. Default: 1.0.\n\n Returns:\n Tensor: The output tensor.\n \"\"\"\n\n def __init__(self, bias=3.0, divisor=6.0, min_value=0.0, max_value=1.0):\n super(HSigmoid, self).__init__()\n warnings.warn(\n 'In MMCV v1.4.4, we modified the default value of args to align '\n 'with PyTorch official. Previous Implementation: '\n 'Hsigmoid(x) = min(max((x + 1) / 2, 0), 1). '\n 'Current Implementation: '\n 'Hsigmoid(x) = min(max((x + 3) / 6, 0), 1).')\n self.bias = bias\n self.divisor = divisor\n assert self.divisor != 0\n self.min_value = min_value\n self.max_value = max_value\n\n def forward(self, x):\n x = (x + self.bias) / self.divisor\n\n return x.clamp_(self.min_value, self.max_value)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/hswish.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch.nn as nn\n\nfrom .registry import ACTIVATION_LAYERS\n\n\n@ACTIVATION_LAYERS.register_module()\nclass HSwish(nn.Module):\n \"\"\"Hard Swish Module.\n\n This module applies the hard swish function:\n\n .. math::\n Hswish(x) = x * ReLU6(x + 3) / 6\n\n Args:\n inplace (bool): can optionally do the operation in-place.\n Default: False.\n\n Returns:\n Tensor: The output tensor.\n \"\"\"\n\n def __init__(self, inplace=False):\n super(HSwish, self).__init__()\n self.act = nn.ReLU6(inplace)\n\n def forward(self, x):\n return x * self.act(x + 3) / 6\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/non_local.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom abc import ABCMeta\n\nimport torch\nimport torch.nn as nn\n\nfrom ..utils import constant_init, normal_init\nfrom .conv_module import ConvModule\nfrom .registry import PLUGIN_LAYERS\n\n\nclass _NonLocalNd(nn.Module, metaclass=ABCMeta):\n \"\"\"Basic Non-local module.\n\n This module is proposed in\n \"Non-local Neural Networks\"\n Paper reference: https://arxiv.org/abs/1711.07971\n Code reference: https://github.com/AlexHex7/Non-local_pytorch\n\n Args:\n in_channels (int): Channels of the input feature map.\n reduction (int): Channel reduction ratio. Default: 2.\n use_scale (bool): Whether to scale pairwise_weight by\n `1/sqrt(inter_channels)` when the mode is `embedded_gaussian`.\n Default: True.\n conv_cfg (None | dict): The config dict for convolution layers.\n If not specified, it will use `nn.Conv2d` for convolution layers.\n Default: None.\n norm_cfg (None | dict): The config dict for normalization layers.\n Default: None. (This parameter is only applicable to conv_out.)\n mode (str): Options are `gaussian`, `concatenation`,\n `embedded_gaussian` and `dot_product`. Default: embedded_gaussian.\n \"\"\"\n\n def __init__(self,\n in_channels,\n reduction=2,\n use_scale=True,\n conv_cfg=None,\n norm_cfg=None,\n mode='embedded_gaussian',\n **kwargs):\n super(_NonLocalNd, self).__init__()\n self.in_channels = in_channels\n self.reduction = reduction\n self.use_scale = use_scale\n self.inter_channels = max(in_channels // reduction, 1)\n self.mode = mode\n\n if mode not in [\n 'gaussian', 'embedded_gaussian', 'dot_product', 'concatenation'\n ]:\n raise ValueError(\"Mode should be in 'gaussian', 'concatenation', \"\n f\"'embedded_gaussian' or 'dot_product', but got \"\n f'{mode} instead.')\n\n # g, theta, phi are defaulted as `nn.ConvNd`.\n # Here we use ConvModule for potential usage.\n self.g = ConvModule(\n self.in_channels,\n self.inter_channels,\n kernel_size=1,\n conv_cfg=conv_cfg,\n act_cfg=None)\n self.conv_out = ConvModule(\n self.inter_channels,\n self.in_channels,\n kernel_size=1,\n conv_cfg=conv_cfg,\n norm_cfg=norm_cfg,\n act_cfg=None)\n\n if self.mode != 'gaussian':\n self.theta = ConvModule(\n self.in_channels,\n self.inter_channels,\n kernel_size=1,\n conv_cfg=conv_cfg,\n act_cfg=None)\n self.phi = ConvModule(\n self.in_channels,\n self.inter_channels,\n kernel_size=1,\n conv_cfg=conv_cfg,\n act_cfg=None)\n\n if self.mode == 'concatenation':\n self.concat_project = ConvModule(\n self.inter_channels * 2,\n 1,\n kernel_size=1,\n stride=1,\n padding=0,\n bias=False,\n act_cfg=dict(type='ReLU'))\n\n self.init_weights(**kwargs)\n\n def init_weights(self, std=0.01, zeros_init=True):\n if self.mode != 'gaussian':\n for m in [self.g, self.theta, self.phi]:\n normal_init(m.conv, std=std)\n else:\n normal_init(self.g.conv, std=std)\n if zeros_init:\n if self.conv_out.norm_cfg is None:\n constant_init(self.conv_out.conv, 0)\n else:\n constant_init(self.conv_out.norm, 0)\n else:\n if self.conv_out.norm_cfg is None:\n normal_init(self.conv_out.conv, std=std)\n else:\n normal_init(self.conv_out.norm, std=std)\n\n def gaussian(self, theta_x, phi_x):\n # NonLocal1d pairwise_weight: [N, H, H]\n # NonLocal2d pairwise_weight: [N, HxW, HxW]\n # NonLocal3d pairwise_weight: [N, TxHxW, TxHxW]\n pairwise_weight = torch.matmul(theta_x, phi_x)\n pairwise_weight = pairwise_weight.softmax(dim=-1)\n return pairwise_weight\n\n def embedded_gaussian(self, theta_x, phi_x):\n # NonLocal1d pairwise_weight: [N, H, H]\n # NonLocal2d pairwise_weight: [N, HxW, HxW]\n # NonLocal3d pairwise_weight: [N, TxHxW, TxHxW]\n pairwise_weight = torch.matmul(theta_x, phi_x)\n if self.use_scale:\n # theta_x.shape[-1] is `self.inter_channels`\n pairwise_weight /= theta_x.shape[-1]**0.5\n pairwise_weight = pairwise_weight.softmax(dim=-1)\n return pairwise_weight\n\n def dot_product(self, theta_x, phi_x):\n # NonLocal1d pairwise_weight: [N, H, H]\n # NonLocal2d pairwise_weight: [N, HxW, HxW]\n # NonLocal3d pairwise_weight: [N, TxHxW, TxHxW]\n pairwise_weight = torch.matmul(theta_x, phi_x)\n pairwise_weight /= pairwise_weight.shape[-1]\n return pairwise_weight\n\n def concatenation(self, theta_x, phi_x):\n # NonLocal1d pairwise_weight: [N, H, H]\n # NonLocal2d pairwise_weight: [N, HxW, HxW]\n # NonLocal3d pairwise_weight: [N, TxHxW, TxHxW]\n h = theta_x.size(2)\n w = phi_x.size(3)\n theta_x = theta_x.repeat(1, 1, 1, w)\n phi_x = phi_x.repeat(1, 1, h, 1)\n\n concat_feature = torch.cat([theta_x, phi_x], dim=1)\n pairwise_weight = self.concat_project(concat_feature)\n n, _, h, w = pairwise_weight.size()\n pairwise_weight = pairwise_weight.view(n, h, w)\n pairwise_weight /= pairwise_weight.shape[-1]\n\n return pairwise_weight\n\n def forward(self, x):\n # Assume `reduction = 1`, then `inter_channels = C`\n # or `inter_channels = C` when `mode=\"gaussian\"`\n\n # NonLocal1d x: [N, C, H]\n # NonLocal2d x: [N, C, H, W]\n # NonLocal3d x: [N, C, T, H, W]\n n = x.size(0)\n\n # NonLocal1d g_x: [N, H, C]\n # NonLocal2d g_x: [N, HxW, C]\n # NonLocal3d g_x: [N, TxHxW, C]\n g_x = self.g(x).view(n, self.inter_channels, -1)\n g_x = g_x.permute(0, 2, 1)\n\n # NonLocal1d theta_x: [N, H, C], phi_x: [N, C, H]\n # NonLocal2d theta_x: [N, HxW, C], phi_x: [N, C, HxW]\n # NonLocal3d theta_x: [N, TxHxW, C], phi_x: [N, C, TxHxW]\n if self.mode == 'gaussian':\n theta_x = x.view(n, self.in_channels, -1)\n theta_x = theta_x.permute(0, 2, 1)\n if self.sub_sample:\n phi_x = self.phi(x).view(n, self.in_channels, -1)\n else:\n phi_x = x.view(n, self.in_channels, -1)\n elif self.mode == 'concatenation':\n theta_x = self.theta(x).view(n, self.inter_channels, -1, 1)\n phi_x = self.phi(x).view(n, self.inter_channels, 1, -1)\n else:\n theta_x = self.theta(x).view(n, self.inter_channels, -1)\n theta_x = theta_x.permute(0, 2, 1)\n phi_x = self.phi(x).view(n, self.inter_channels, -1)\n\n pairwise_func = getattr(self, self.mode)\n # NonLocal1d pairwise_weight: [N, H, H]\n # NonLocal2d pairwise_weight: [N, HxW, HxW]\n # NonLocal3d pairwise_weight: [N, TxHxW, TxHxW]\n pairwise_weight = pairwise_func(theta_x, phi_x)\n\n # NonLocal1d y: [N, H, C]\n # NonLocal2d y: [N, HxW, C]\n # NonLocal3d y: [N, TxHxW, C]\n y = torch.matmul(pairwise_weight, g_x)\n # NonLocal1d y: [N, C, H]\n # NonLocal2d y: [N, C, H, W]\n # NonLocal3d y: [N, C, T, H, W]\n y = y.permute(0, 2, 1).contiguous().reshape(n, self.inter_channels,\n *x.size()[2:])\n\n output = x + self.conv_out(y)\n\n return output\n\n\nclass NonLocal1d(_NonLocalNd):\n \"\"\"1D Non-local module.\n\n Args:\n in_channels (int): Same as `NonLocalND`.\n sub_sample (bool): Whether to apply max pooling after pairwise\n function (Note that the `sub_sample` is applied on spatial only).\n Default: False.\n conv_cfg (None | dict): Same as `NonLocalND`.\n Default: dict(type='Conv1d').\n \"\"\"\n\n def __init__(self,\n in_channels,\n sub_sample=False,\n conv_cfg=dict(type='Conv1d'),\n **kwargs):\n super(NonLocal1d, self).__init__(\n in_channels, conv_cfg=conv_cfg, **kwargs)\n\n self.sub_sample = sub_sample\n\n if sub_sample:\n max_pool_layer = nn.MaxPool1d(kernel_size=2)\n self.g = nn.Sequential(self.g, max_pool_layer)\n if self.mode != 'gaussian':\n self.phi = nn.Sequential(self.phi, max_pool_layer)\n else:\n self.phi = max_pool_layer\n\n\n@PLUGIN_LAYERS.register_module()\nclass NonLocal2d(_NonLocalNd):\n \"\"\"2D Non-local module.\n\n Args:\n in_channels (int): Same as `NonLocalND`.\n sub_sample (bool): Whether to apply max pooling after pairwise\n function (Note that the `sub_sample` is applied on spatial only).\n Default: False.\n conv_cfg (None | dict): Same as `NonLocalND`.\n Default: dict(type='Conv2d').\n \"\"\"\n\n _abbr_ = 'nonlocal_block'\n\n def __init__(self,\n in_channels,\n sub_sample=False,\n conv_cfg=dict(type='Conv2d'),\n **kwargs):\n super(NonLocal2d, self).__init__(\n in_channels, conv_cfg=conv_cfg, **kwargs)\n\n self.sub_sample = sub_sample\n\n if sub_sample:\n max_pool_layer = nn.MaxPool2d(kernel_size=(2, 2))\n self.g = nn.Sequential(self.g, max_pool_layer)\n if self.mode != 'gaussian':\n self.phi = nn.Sequential(self.phi, max_pool_layer)\n else:\n self.phi = max_pool_layer\n\n\nclass NonLocal3d(_NonLocalNd):\n \"\"\"3D Non-local module.\n\n Args:\n in_channels (int): Same as `NonLocalND`.\n sub_sample (bool): Whether to apply max pooling after pairwise\n function (Note that the `sub_sample` is applied on spatial only).\n Default: False.\n conv_cfg (None | dict): Same as `NonLocalND`.\n Default: dict(type='Conv3d').\n \"\"\"\n\n def __init__(self,\n in_channels,\n sub_sample=False,\n conv_cfg=dict(type='Conv3d'),\n **kwargs):\n super(NonLocal3d, self).__init__(\n in_channels, conv_cfg=conv_cfg, **kwargs)\n self.sub_sample = sub_sample\n\n if sub_sample:\n max_pool_layer = nn.MaxPool3d(kernel_size=(1, 2, 2))\n self.g = nn.Sequential(self.g, max_pool_layer)\n if self.mode != 'gaussian':\n self.phi = nn.Sequential(self.phi, max_pool_layer)\n else:\n self.phi = max_pool_layer\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/norm.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport inspect\n\nimport torch.nn as nn\n\nfrom mmcv.utils import is_tuple_of\nfrom mmcv.utils.parrots_wrapper import SyncBatchNorm, _BatchNorm, _InstanceNorm\nfrom .registry import NORM_LAYERS\n\nNORM_LAYERS.register_module('BN', module=nn.BatchNorm2d)\nNORM_LAYERS.register_module('BN1d', module=nn.BatchNorm1d)\nNORM_LAYERS.register_module('BN2d', module=nn.BatchNorm2d)\nNORM_LAYERS.register_module('BN3d', module=nn.BatchNorm3d)\nNORM_LAYERS.register_module('SyncBN', module=SyncBatchNorm)\nNORM_LAYERS.register_module('GN', module=nn.GroupNorm)\nNORM_LAYERS.register_module('LN', module=nn.LayerNorm)\nNORM_LAYERS.register_module('IN', module=nn.InstanceNorm2d)\nNORM_LAYERS.register_module('IN1d', module=nn.InstanceNorm1d)\nNORM_LAYERS.register_module('IN2d', module=nn.InstanceNorm2d)\nNORM_LAYERS.register_module('IN3d', module=nn.InstanceNorm3d)\n\n\ndef infer_abbr(class_type):\n \"\"\"Infer abbreviation from the class name.\n\n When we build a norm layer with `build_norm_layer()`, we want to preserve\n the norm type in variable names, e.g, self.bn1, self.gn. This method will\n infer the abbreviation to map class types to abbreviations.\n\n Rule 1: If the class has the property \"_abbr_\", return the property.\n Rule 2: If the parent class is _BatchNorm, GroupNorm, LayerNorm or\n InstanceNorm, the abbreviation of this layer will be \"bn\", \"gn\", \"ln\" and\n \"in\" respectively.\n Rule 3: If the class name contains \"batch\", \"group\", \"layer\" or \"instance\",\n the abbreviation of this layer will be \"bn\", \"gn\", \"ln\" and \"in\"\n respectively.\n Rule 4: Otherwise, the abbreviation falls back to \"norm\".\n\n Args:\n class_type (type): The norm layer type.\n\n Returns:\n str: The inferred abbreviation.\n \"\"\"\n if not inspect.isclass(class_type):\n raise TypeError(\n f'class_type must be a type, but got {type(class_type)}')\n if hasattr(class_type, '_abbr_'):\n return class_type._abbr_\n if issubclass(class_type, _InstanceNorm): # IN is a subclass of BN\n return 'in'\n elif issubclass(class_type, _BatchNorm):\n return 'bn'\n elif issubclass(class_type, nn.GroupNorm):\n return 'gn'\n elif issubclass(class_type, nn.LayerNorm):\n return 'ln'\n else:\n class_name = class_type.__name__.lower()\n if 'batch' in class_name:\n return 'bn'\n elif 'group' in class_name:\n return 'gn'\n elif 'layer' in class_name:\n return 'ln'\n elif 'instance' in class_name:\n return 'in'\n else:\n return 'norm_layer'\n\n\ndef build_norm_layer(cfg, num_features, postfix=''):\n \"\"\"Build normalization layer.\n\n Args:\n cfg (dict): The norm layer config, which should contain:\n\n - type (str): Layer type.\n - layer args: Args needed to instantiate a norm layer.\n - requires_grad (bool, optional): Whether stop gradient updates.\n num_features (int): Number of input channels.\n postfix (int | str): The postfix to be appended into norm abbreviation\n to create named layer.\n\n Returns:\n tuple[str, nn.Module]: The first element is the layer name consisting\n of abbreviation and postfix, e.g., bn1, gn. The second element is the\n created norm layer.\n \"\"\"\n if not isinstance(cfg, dict):\n raise TypeError('cfg must be a dict')\n if 'type' not in cfg:\n raise KeyError('the cfg dict must contain the key \"type\"')\n cfg_ = cfg.copy()\n\n layer_type = cfg_.pop('type')\n if layer_type not in NORM_LAYERS:\n raise KeyError(f'Unrecognized norm type {layer_type}')\n\n norm_layer = NORM_LAYERS.get(layer_type)\n abbr = infer_abbr(norm_layer)\n\n assert isinstance(postfix, (int, str))\n name = abbr + str(postfix)\n\n requires_grad = cfg_.pop('requires_grad', True)\n cfg_.setdefault('eps', 1e-5)\n if layer_type != 'GN':\n layer = norm_layer(num_features, **cfg_)\n if layer_type == 'SyncBN' and hasattr(layer, '_specify_ddp_gpu_num'):\n layer._specify_ddp_gpu_num(1)\n else:\n assert 'num_groups' in cfg_\n layer = norm_layer(num_channels=num_features, **cfg_)\n\n for param in layer.parameters():\n param.requires_grad = requires_grad\n\n return name, layer\n\n\ndef is_norm(layer, exclude=None):\n \"\"\"Check if a layer is a normalization layer.\n\n Args:\n layer (nn.Module): The layer to be checked.\n exclude (type | tuple[type]): Types to be excluded.\n\n Returns:\n bool: Whether the layer is a norm layer.\n \"\"\"\n if exclude is not None:\n if not isinstance(exclude, tuple):\n exclude = (exclude, )\n if not is_tuple_of(exclude, type):\n raise TypeError(\n f'\"exclude\" must be either None or type or a tuple of types, '\n f'but got {type(exclude)}: {exclude}')\n\n if exclude and isinstance(layer, exclude):\n return False\n\n all_norm_bases = (_BatchNorm, _InstanceNorm, nn.GroupNorm, nn.LayerNorm)\n return isinstance(layer, all_norm_bases)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/padding.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch.nn as nn\n\nfrom .registry import PADDING_LAYERS\n\nPADDING_LAYERS.register_module('zero', module=nn.ZeroPad2d)\nPADDING_LAYERS.register_module('reflect', module=nn.ReflectionPad2d)\nPADDING_LAYERS.register_module('replicate', module=nn.ReplicationPad2d)\n\n\ndef build_padding_layer(cfg, *args, **kwargs):\n \"\"\"Build padding layer.\n\n Args:\n cfg (None or dict): The padding layer config, which should contain:\n - type (str): Layer type.\n - layer args: Args needed to instantiate a padding layer.\n\n Returns:\n nn.Module: Created padding layer.\n \"\"\"\n if not isinstance(cfg, dict):\n raise TypeError('cfg must be a dict')\n if 'type' not in cfg:\n raise KeyError('the cfg dict must contain the key \"type\"')\n\n cfg_ = cfg.copy()\n padding_type = cfg_.pop('type')\n if padding_type not in PADDING_LAYERS:\n raise KeyError(f'Unrecognized padding type {padding_type}.')\n else:\n padding_layer = PADDING_LAYERS.get(padding_type)\n\n layer = padding_layer(*args, **kwargs, **cfg_)\n\n return layer\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/plugin.py", "content": "import inspect\nimport platform\n\nfrom .registry import PLUGIN_LAYERS\n\nif platform.system() == 'Windows':\n import regex as re\nelse:\n import re\n\n\ndef infer_abbr(class_type):\n \"\"\"Infer abbreviation from the class name.\n\n This method will infer the abbreviation to map class types to\n abbreviations.\n\n Rule 1: If the class has the property \"abbr\", return the property.\n Rule 2: Otherwise, the abbreviation falls back to snake case of class\n name, e.g. the abbreviation of ``FancyBlock`` will be ``fancy_block``.\n\n Args:\n class_type (type): The norm layer type.\n\n Returns:\n str: The inferred abbreviation.\n \"\"\"\n\n def camel2snack(word):\n \"\"\"Convert camel case word into snack case.\n\n Modified from `inflection lib\n `_.\n\n Example::\n\n >>> camel2snack(\"FancyBlock\")\n 'fancy_block'\n \"\"\"\n\n word = re.sub(r'([A-Z]+)([A-Z][a-z])', r'\\1_\\2', word)\n word = re.sub(r'([a-z\\d])([A-Z])', r'\\1_\\2', word)\n word = word.replace('-', '_')\n return word.lower()\n\n if not inspect.isclass(class_type):\n raise TypeError(\n f'class_type must be a type, but got {type(class_type)}')\n if hasattr(class_type, '_abbr_'):\n return class_type._abbr_\n else:\n return camel2snack(class_type.__name__)\n\n\ndef build_plugin_layer(cfg, postfix='', **kwargs):\n \"\"\"Build plugin layer.\n\n Args:\n cfg (None or dict): cfg should contain:\n\n - type (str): identify plugin layer type.\n - layer args: args needed to instantiate a plugin layer.\n postfix (int, str): appended into norm abbreviation to\n create named layer. Default: ''.\n\n Returns:\n tuple[str, nn.Module]: The first one is the concatenation of\n abbreviation and postfix. The second is the created plugin layer.\n \"\"\"\n if not isinstance(cfg, dict):\n raise TypeError('cfg must be a dict')\n if 'type' not in cfg:\n raise KeyError('the cfg dict must contain the key \"type\"')\n cfg_ = cfg.copy()\n\n layer_type = cfg_.pop('type')\n if layer_type not in PLUGIN_LAYERS:\n raise KeyError(f'Unrecognized plugin type {layer_type}')\n\n plugin_layer = PLUGIN_LAYERS.get(layer_type)\n abbr = infer_abbr(plugin_layer)\n\n assert isinstance(postfix, (int, str))\n name = abbr + str(postfix)\n\n layer = plugin_layer(**kwargs, **cfg_)\n\n return name, layer\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/registry.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom mmcv.utils import Registry\n\nCONV_LAYERS = Registry('conv layer')\nNORM_LAYERS = Registry('norm layer')\nACTIVATION_LAYERS = Registry('activation layer')\nPADDING_LAYERS = Registry('padding layer')\nUPSAMPLE_LAYERS = Registry('upsample layer')\nPLUGIN_LAYERS = Registry('plugin layer')\n\nDROPOUT_LAYERS = Registry('drop out layers')\nPOSITIONAL_ENCODING = Registry('position encoding')\nATTENTION = Registry('attention')\nFEEDFORWARD_NETWORK = Registry('feed-forward Network')\nTRANSFORMER_LAYER = Registry('transformerLayer')\nTRANSFORMER_LAYER_SEQUENCE = Registry('transformer-layers sequence')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/scale.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\n\n\nclass Scale(nn.Module):\n \"\"\"A learnable scale parameter.\n\n This layer scales the input by a learnable factor. It multiplies a\n learnable scale parameter of shape (1,) with input of any shape.\n\n Args:\n scale (float): Initial value of scale factor. Default: 1.0\n \"\"\"\n\n def __init__(self, scale=1.0):\n super(Scale, self).__init__()\n self.scale = nn.Parameter(torch.tensor(scale, dtype=torch.float))\n\n def forward(self, x):\n return x * self.scale\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/swish.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\n\nfrom .registry import ACTIVATION_LAYERS\n\n\n@ACTIVATION_LAYERS.register_module()\nclass Swish(nn.Module):\n \"\"\"Swish Module.\n\n This module applies the swish function:\n\n .. math::\n Swish(x) = x * Sigmoid(x)\n\n Returns:\n Tensor: The output tensor.\n \"\"\"\n\n def __init__(self):\n super(Swish, self).__init__()\n\n def forward(self, x):\n return x * torch.sigmoid(x)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/transformer.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport copy\nimport math\nimport warnings\nfrom typing import Sequence\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom mmcv.cnn import (Linear, build_activation_layer, build_conv_layer,\n build_norm_layer)\nfrom mmcv.runner.base_module import BaseModule, ModuleList, Sequential\nfrom mmcv.utils import (ConfigDict, build_from_cfg, deprecated_api_warning,\n to_2tuple)\nfrom .drop import build_dropout\nfrom .registry import (ATTENTION, FEEDFORWARD_NETWORK, POSITIONAL_ENCODING,\n TRANSFORMER_LAYER, TRANSFORMER_LAYER_SEQUENCE)\n\n# Avoid BC-breaking of importing MultiScaleDeformableAttention from this file\ntry:\n from mmcv.ops.multi_scale_deform_attn import MultiScaleDeformableAttention # noqa F401\n warnings.warn(\n ImportWarning(\n '``MultiScaleDeformableAttention`` has been moved to '\n '``mmcv.ops.multi_scale_deform_attn``, please change original path ' # noqa E501\n '``from mmcv.cnn.bricks.transformer import MultiScaleDeformableAttention`` ' # noqa E501\n 'to ``from mmcv.ops.multi_scale_deform_attn import MultiScaleDeformableAttention`` ' # noqa E501\n ))\n\nexcept ImportError:\n warnings.warn('Fail to import ``MultiScaleDeformableAttention`` from '\n '``mmcv.ops.multi_scale_deform_attn``, '\n 'You should install ``mmcv-full`` if you need this module. ')\n\n\ndef build_positional_encoding(cfg, default_args=None):\n \"\"\"Builder for Position Encoding.\"\"\"\n return build_from_cfg(cfg, POSITIONAL_ENCODING, default_args)\n\n\ndef build_attention(cfg, default_args=None):\n \"\"\"Builder for attention.\"\"\"\n return build_from_cfg(cfg, ATTENTION, default_args)\n\n\ndef build_feedforward_network(cfg, default_args=None):\n \"\"\"Builder for feed-forward network (FFN).\"\"\"\n return build_from_cfg(cfg, FEEDFORWARD_NETWORK, default_args)\n\n\ndef build_transformer_layer(cfg, default_args=None):\n \"\"\"Builder for transformer layer.\"\"\"\n return build_from_cfg(cfg, TRANSFORMER_LAYER, default_args)\n\n\ndef build_transformer_layer_sequence(cfg, default_args=None):\n \"\"\"Builder for transformer encoder and transformer decoder.\"\"\"\n return build_from_cfg(cfg, TRANSFORMER_LAYER_SEQUENCE, default_args)\n\n\nclass AdaptivePadding(nn.Module):\n \"\"\"Applies padding adaptively to the input.\n\n This module can make input get fully covered by filter\n you specified. It support two modes \"same\" and \"corner\". The\n \"same\" mode is same with \"SAME\" padding mode in TensorFlow, pad\n zero around input. The \"corner\" mode would pad zero\n to bottom right.\n\n Args:\n kernel_size (int | tuple): Size of the kernel. Default: 1.\n stride (int | tuple): Stride of the filter. Default: 1.\n dilation (int | tuple): Spacing between kernel elements.\n Default: 1.\n padding (str): Support \"same\" and \"corner\", \"corner\" mode\n would pad zero to bottom right, and \"same\" mode would\n pad zero around input. Default: \"corner\".\n\n Example:\n >>> kernel_size = 16\n >>> stride = 16\n >>> dilation = 1\n >>> input = torch.rand(1, 1, 15, 17)\n >>> adap_pad = AdaptivePadding(\n >>> kernel_size=kernel_size,\n >>> stride=stride,\n >>> dilation=dilation,\n >>> padding=\"corner\")\n >>> out = adap_pad(input)\n >>> assert (out.shape[2], out.shape[3]) == (16, 32)\n >>> input = torch.rand(1, 1, 16, 17)\n >>> out = adap_pad(input)\n >>> assert (out.shape[2], out.shape[3]) == (16, 32)\n \"\"\"\n\n def __init__(self, kernel_size=1, stride=1, dilation=1, padding='corner'):\n super(AdaptivePadding, self).__init__()\n assert padding in ('same', 'corner')\n\n kernel_size = to_2tuple(kernel_size)\n stride = to_2tuple(stride)\n dilation = to_2tuple(dilation)\n\n self.padding = padding\n self.kernel_size = kernel_size\n self.stride = stride\n self.dilation = dilation\n\n def get_pad_shape(self, input_shape):\n \"\"\"Calculate the padding size of input.\n\n Args:\n input_shape (:obj:`torch.Size`): arrange as (H, W).\n\n Returns:\n Tuple[int]: The padding size along the\n original H and W directions\n \"\"\"\n input_h, input_w = input_shape\n kernel_h, kernel_w = self.kernel_size\n stride_h, stride_w = self.stride\n output_h = math.ceil(input_h / stride_h)\n output_w = math.ceil(input_w / stride_w)\n pad_h = max((output_h - 1) * stride_h +\n (kernel_h - 1) * self.dilation[0] + 1 - input_h, 0)\n pad_w = max((output_w - 1) * stride_w +\n (kernel_w - 1) * self.dilation[1] + 1 - input_w, 0)\n return pad_h, pad_w\n\n def forward(self, x):\n \"\"\"Add padding to `x`\n\n Args:\n x (Tensor): Input tensor has shape (B, C, H, W).\n\n Returns:\n Tensor: The tensor with adaptive padding\n \"\"\"\n pad_h, pad_w = self.get_pad_shape(x.size()[-2:])\n if pad_h > 0 or pad_w > 0:\n if self.padding == 'corner':\n x = F.pad(x, [0, pad_w, 0, pad_h])\n elif self.padding == 'same':\n x = F.pad(x, [\n pad_w // 2, pad_w - pad_w // 2, pad_h // 2,\n pad_h - pad_h // 2\n ])\n return x\n\n\nclass PatchEmbed(BaseModule):\n \"\"\"Image to Patch Embedding.\n\n We use a conv layer to implement PatchEmbed.\n\n Args:\n in_channels (int): The num of input channels. Default: 3\n embed_dims (int): The dimensions of embedding. Default: 768\n conv_type (str): The type of convolution\n to generate patch embedding. Default: \"Conv2d\".\n kernel_size (int): The kernel_size of embedding conv. Default: 16.\n stride (int): The slide stride of embedding conv.\n Default: 16.\n padding (int | tuple | string): The padding length of\n embedding conv. When it is a string, it means the mode\n of adaptive padding, support \"same\" and \"corner\" now.\n Default: \"corner\".\n dilation (int): The dilation rate of embedding conv. Default: 1.\n bias (bool): Bias of embed conv. Default: True.\n norm_cfg (dict, optional): Config dict for normalization layer.\n Default: None.\n input_size (int | tuple | None): The size of input, which will be\n used to calculate the out size. Only works when `dynamic_size`\n is False. Default: None.\n init_cfg (`mmcv.ConfigDict`, optional): The Config for initialization.\n Default: None.\n \"\"\"\n\n def __init__(self,\n in_channels=3,\n embed_dims=768,\n conv_type='Conv2d',\n kernel_size=16,\n stride=16,\n padding='corner',\n dilation=1,\n bias=True,\n norm_cfg=None,\n input_size=None,\n init_cfg=None):\n super(PatchEmbed, self).__init__(init_cfg=init_cfg)\n\n self.embed_dims = embed_dims\n if stride is None:\n stride = kernel_size\n\n kernel_size = to_2tuple(kernel_size)\n stride = to_2tuple(stride)\n dilation = to_2tuple(dilation)\n\n if isinstance(padding, str):\n self.adaptive_padding = AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=padding)\n # disable the padding of conv\n padding = 0\n else:\n self.adaptive_padding = None\n padding = to_2tuple(padding)\n\n self.projection = build_conv_layer(\n dict(type=conv_type),\n in_channels=in_channels,\n out_channels=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n if norm_cfg is not None:\n self.norm = build_norm_layer(norm_cfg, embed_dims)[1]\n else:\n self.norm = None\n\n if input_size:\n input_size = to_2tuple(input_size)\n # `init_out_size` would be used outside to\n # calculate the num_patches\n # e.g. when `use_abs_pos_embed` outside\n self.init_input_size = input_size\n if self.adaptive_padding:\n pad_h, pad_w = self.adaptive_padding.get_pad_shape(input_size)\n input_h, input_w = input_size\n input_h = input_h + pad_h\n input_w = input_w + pad_w\n input_size = (input_h, input_w)\n\n # https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html\n h_out = (input_size[0] + 2 * padding[0] - dilation[0] *\n (kernel_size[0] - 1) - 1) // stride[0] + 1\n w_out = (input_size[1] + 2 * padding[1] - dilation[1] *\n (kernel_size[1] - 1) - 1) // stride[1] + 1\n self.init_out_size = (h_out, w_out)\n else:\n self.init_input_size = None\n self.init_out_size = None\n\n def forward(self, x):\n \"\"\"\n Args:\n x (Tensor): Has shape (B, C, H, W). In most case, C is 3.\n\n Returns:\n tuple: Contains merged results and its spatial shape.\n\n - x (Tensor): Has shape (B, out_h * out_w, embed_dims)\n - out_size (tuple[int]): Spatial shape of x, arrange as\n (out_h, out_w).\n \"\"\"\n\n if self.adaptive_padding:\n x = self.adaptive_padding(x)\n\n x = self.projection(x)\n out_size = (x.shape[2], x.shape[3])\n x = x.flatten(2).transpose(1, 2)\n if self.norm is not None:\n x = self.norm(x)\n return x, out_size\n\n\nclass PatchMerging(BaseModule):\n \"\"\"Merge patch feature map.\n\n This layer groups feature map by kernel_size, and applies norm and linear\n layers to the grouped feature map ((used in Swin Transformer)).\n Our implementation uses `nn.Unfold` to\n merge patches, which is about 25% faster than the original\n implementation. However, we need to modify pretrained\n models for compatibility.\n\n Args:\n in_channels (int): The num of input channels.\n to gets fully covered by filter and stride you specified.\n out_channels (int): The num of output channels.\n kernel_size (int | tuple, optional): the kernel size in the unfold\n layer. Defaults to 2.\n stride (int | tuple, optional): the stride of the sliding blocks in the\n unfold layer. Default: None. (Would be set as `kernel_size`)\n padding (int | tuple | string ): The padding length of\n embedding conv. When it is a string, it means the mode\n of adaptive padding, support \"same\" and \"corner\" now.\n Default: \"corner\".\n dilation (int | tuple, optional): dilation parameter in the unfold\n layer. Default: 1.\n bias (bool, optional): Whether to add bias in linear layer or not.\n Defaults: False.\n norm_cfg (dict, optional): Config dict for normalization layer.\n Default: dict(type='LN').\n init_cfg (dict, optional): The extra config for initialization.\n Default: None.\n \"\"\"\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size=2,\n stride=None,\n padding='corner',\n dilation=1,\n bias=False,\n norm_cfg=dict(type='LN'),\n init_cfg=None):\n super().__init__(init_cfg=init_cfg)\n self.in_channels = in_channels\n self.out_channels = out_channels\n if stride:\n stride = stride\n else:\n stride = kernel_size\n\n kernel_size = to_2tuple(kernel_size)\n stride = to_2tuple(stride)\n dilation = to_2tuple(dilation)\n\n if isinstance(padding, str):\n self.adaptive_padding = AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=padding)\n # disable the padding of unfold\n padding = 0\n else:\n self.adaptive_padding = None\n\n padding = to_2tuple(padding)\n self.sampler = nn.Unfold(\n kernel_size=kernel_size,\n dilation=dilation,\n padding=padding,\n stride=stride)\n\n sample_dim = kernel_size[0] * kernel_size[1] * in_channels\n\n if norm_cfg is not None:\n self.norm = build_norm_layer(norm_cfg, sample_dim)[1]\n else:\n self.norm = None\n\n self.reduction = nn.Linear(sample_dim, out_channels, bias=bias)\n\n def forward(self, x, input_size):\n \"\"\"\n Args:\n x (Tensor): Has shape (B, H*W, C_in).\n input_size (tuple[int]): The spatial shape of x, arrange as (H, W).\n Default: None.\n\n Returns:\n tuple: Contains merged results and its spatial shape.\n\n - x (Tensor): Has shape (B, Merged_H * Merged_W, C_out)\n - out_size (tuple[int]): Spatial shape of x, arrange as\n (Merged_H, Merged_W).\n \"\"\"\n B, L, C = x.shape\n assert isinstance(input_size, Sequence), f'Expect ' \\\n f'input_size is ' \\\n f'`Sequence` ' \\\n f'but get {input_size}'\n\n H, W = input_size\n assert L == H * W, 'input feature has wrong size'\n\n x = x.view(B, H, W, C).permute([0, 3, 1, 2]) # B, C, H, W\n\n if self.adaptive_padding:\n x = self.adaptive_padding(x)\n H, W = x.shape[-2:]\n\n # Use nn.Unfold to merge patch. About 25% faster than original method,\n # but need to modify pretrained model for compatibility\n # if kernel_size=2 and stride=2, x should has shape (B, 4*C, H/2*W/2)\n x = self.sampler(x)\n\n out_h = (H + 2 * self.sampler.padding[0] - self.sampler.dilation[0] *\n (self.sampler.kernel_size[0] - 1) -\n 1) // self.sampler.stride[0] + 1\n out_w = (W + 2 * self.sampler.padding[1] - self.sampler.dilation[1] *\n (self.sampler.kernel_size[1] - 1) -\n 1) // self.sampler.stride[1] + 1\n\n output_size = (out_h, out_w)\n x = x.transpose(1, 2) # B, H/2*W/2, 4*C\n x = self.norm(x) if self.norm else x\n x = self.reduction(x)\n return x, output_size\n\n\n@ATTENTION.register_module()\nclass MultiheadAttention(BaseModule):\n \"\"\"A wrapper for ``torch.nn.MultiheadAttention``.\n\n This module implements MultiheadAttention with identity connection,\n and positional encoding is also passed as input.\n\n Args:\n embed_dims (int): The embedding dimension.\n num_heads (int): Parallel attention heads.\n attn_drop (float): A Dropout layer on attn_output_weights.\n Default: 0.0.\n proj_drop (float): A Dropout layer after `nn.MultiheadAttention`.\n Default: 0.0.\n dropout_layer (obj:`ConfigDict`): The dropout_layer used\n when adding the shortcut.\n init_cfg (obj:`mmcv.ConfigDict`): The Config for initialization.\n Default: None.\n batch_first (bool): When it is True, Key, Query and Value are shape of\n (batch, n, embed_dim), otherwise (n, batch, embed_dim).\n Default to False.\n \"\"\"\n\n def __init__(self,\n embed_dims,\n num_heads,\n attn_drop=0.,\n proj_drop=0.,\n dropout_layer=dict(type='Dropout', drop_prob=0.),\n init_cfg=None,\n batch_first=False,\n **kwargs):\n super(MultiheadAttention, self).__init__(init_cfg)\n if 'dropout' in kwargs:\n warnings.warn(\n 'The arguments `dropout` in MultiheadAttention '\n 'has been deprecated, now you can separately '\n 'set `attn_drop`(float), proj_drop(float), '\n 'and `dropout_layer`(dict) ', DeprecationWarning)\n attn_drop = kwargs['dropout']\n dropout_layer['drop_prob'] = kwargs.pop('dropout')\n\n self.embed_dims = embed_dims\n self.num_heads = num_heads\n self.batch_first = batch_first\n\n self.attn = nn.MultiheadAttention(embed_dims, num_heads, attn_drop,\n **kwargs)\n\n self.proj_drop = nn.Dropout(proj_drop)\n self.dropout_layer = build_dropout(\n dropout_layer) if dropout_layer else nn.Identity()\n\n @deprecated_api_warning({'residual': 'identity'},\n cls_name='MultiheadAttention')\n def forward(self,\n query,\n key=None,\n value=None,\n identity=None,\n query_pos=None,\n key_pos=None,\n attn_mask=None,\n key_padding_mask=None,\n **kwargs):\n \"\"\"Forward function for `MultiheadAttention`.\n\n **kwargs allow passing a more general data flow when combining\n with other operations in `transformerlayer`.\n\n Args:\n query (Tensor): The input query with shape [num_queries, bs,\n embed_dims] if self.batch_first is False, else\n [bs, num_queries embed_dims].\n key (Tensor): The key tensor with shape [num_keys, bs,\n embed_dims] if self.batch_first is False, else\n [bs, num_keys, embed_dims] .\n If None, the ``query`` will be used. Defaults to None.\n value (Tensor): The value tensor with same shape as `key`.\n Same in `nn.MultiheadAttention.forward`. Defaults to None.\n If None, the `key` will be used.\n identity (Tensor): This tensor, with the same shape as x,\n will be used for the identity link.\n If None, `x` will be used. Defaults to None.\n query_pos (Tensor): The positional encoding for query, with\n the same shape as `x`. If not None, it will\n be added to `x` before forward function. Defaults to None.\n key_pos (Tensor): The positional encoding for `key`, with the\n same shape as `key`. Defaults to None. If not None, it will\n be added to `key` before forward function. If None, and\n `query_pos` has the same shape as `key`, then `query_pos`\n will be used for `key_pos`. Defaults to None.\n attn_mask (Tensor): ByteTensor mask with shape [num_queries,\n num_keys]. Same in `nn.MultiheadAttention.forward`.\n Defaults to None.\n key_padding_mask (Tensor): ByteTensor with shape [bs, num_keys].\n Defaults to None.\n\n Returns:\n Tensor: forwarded results with shape\n [num_queries, bs, embed_dims]\n if self.batch_first is False, else\n [bs, num_queries embed_dims].\n \"\"\"\n\n if key is None:\n key = query\n if value is None:\n value = key\n if identity is None:\n identity = query\n if key_pos is None:\n if query_pos is not None:\n # use query_pos if key_pos is not available\n if query_pos.shape == key.shape:\n key_pos = query_pos\n else:\n warnings.warn(f'position encoding of key is'\n f'missing in {self.__class__.__name__}.')\n if query_pos is not None:\n query = query + query_pos\n if key_pos is not None:\n key = key + key_pos\n\n # Because the dataflow('key', 'query', 'value') of\n # ``torch.nn.MultiheadAttention`` is (num_query, batch,\n # embed_dims), We should adjust the shape of dataflow from\n # batch_first (batch, num_query, embed_dims) to num_query_first\n # (num_query ,batch, embed_dims), and recover ``attn_output``\n # from num_query_first to batch_first.\n if self.batch_first:\n query = query.transpose(0, 1)\n key = key.transpose(0, 1)\n value = value.transpose(0, 1)\n\n out = self.attn(\n query=query,\n key=key,\n value=value,\n attn_mask=attn_mask,\n key_padding_mask=key_padding_mask)[0]\n\n if self.batch_first:\n out = out.transpose(0, 1)\n\n return identity + self.dropout_layer(self.proj_drop(out))\n\n\n@FEEDFORWARD_NETWORK.register_module()\nclass FFN(BaseModule):\n \"\"\"Implements feed-forward networks (FFNs) with identity connection.\n\n Args:\n embed_dims (int): The feature dimension. Same as\n `MultiheadAttention`. Defaults: 256.\n feedforward_channels (int): The hidden dimension of FFNs.\n Defaults: 1024.\n num_fcs (int, optional): The number of fully-connected layers in\n FFNs. Default: 2.\n act_cfg (dict, optional): The activation config for FFNs.\n Default: dict(type='ReLU')\n ffn_drop (float, optional): Probability of an element to be\n zeroed in FFN. Default 0.0.\n add_identity (bool, optional): Whether to add the\n identity connection. Default: `True`.\n dropout_layer (obj:`ConfigDict`): The dropout_layer used\n when adding the shortcut.\n init_cfg (obj:`mmcv.ConfigDict`): The Config for initialization.\n Default: None.\n \"\"\"\n\n @deprecated_api_warning(\n {\n 'dropout': 'ffn_drop',\n 'add_residual': 'add_identity'\n },\n cls_name='FFN')\n def __init__(self,\n embed_dims=256,\n feedforward_channels=1024,\n num_fcs=2,\n act_cfg=dict(type='ReLU', inplace=True),\n ffn_drop=0.,\n dropout_layer=None,\n add_identity=True,\n init_cfg=None,\n **kwargs):\n super(FFN, self).__init__(init_cfg)\n assert num_fcs >= 2, 'num_fcs should be no less ' \\\n f'than 2. got {num_fcs}.'\n self.embed_dims = embed_dims\n self.feedforward_channels = feedforward_channels\n self.num_fcs = num_fcs\n self.act_cfg = act_cfg\n self.activate = build_activation_layer(act_cfg)\n\n layers = []\n in_channels = embed_dims\n for _ in range(num_fcs - 1):\n layers.append(\n Sequential(\n Linear(in_channels, feedforward_channels), self.activate,\n nn.Dropout(ffn_drop)))\n in_channels = feedforward_channels\n layers.append(Linear(feedforward_channels, embed_dims))\n layers.append(nn.Dropout(ffn_drop))\n self.layers = Sequential(*layers)\n self.dropout_layer = build_dropout(\n dropout_layer) if dropout_layer else torch.nn.Identity()\n self.add_identity = add_identity\n\n @deprecated_api_warning({'residual': 'identity'}, cls_name='FFN')\n def forward(self, x, identity=None):\n \"\"\"Forward function for `FFN`.\n\n The function would add x to the output tensor if residue is None.\n \"\"\"\n out = self.layers(x)\n if not self.add_identity:\n return self.dropout_layer(out)\n if identity is None:\n identity = x\n return identity + self.dropout_layer(out)\n\n\n@TRANSFORMER_LAYER.register_module()\nclass BaseTransformerLayer(BaseModule):\n \"\"\"Base `TransformerLayer` for vision transformer.\n\n It can be built from `mmcv.ConfigDict` and support more flexible\n customization, for example, using any number of `FFN or LN ` and\n use different kinds of `attention` by specifying a list of `ConfigDict`\n named `attn_cfgs`. It is worth mentioning that it supports `prenorm`\n when you specifying `norm` as the first element of `operation_order`.\n More details about the `prenorm`: `On Layer Normalization in the\n Transformer Architecture `_ .\n\n Args:\n attn_cfgs (list[`mmcv.ConfigDict`] | obj:`mmcv.ConfigDict` | None )):\n Configs for `self_attention` or `cross_attention` modules,\n The order of the configs in the list should be consistent with\n corresponding attentions in operation_order.\n If it is a dict, all of the attention modules in operation_order\n will be built with this config. Default: None.\n ffn_cfgs (list[`mmcv.ConfigDict`] | obj:`mmcv.ConfigDict` | None )):\n Configs for FFN, The order of the configs in the list should be\n consistent with corresponding ffn in operation_order.\n If it is a dict, all of the attention modules in operation_order\n will be built with this config.\n operation_order (tuple[str]): The execution order of operation\n in transformer. Such as ('self_attn', 'norm', 'ffn', 'norm').\n Support `prenorm` when you specifying first element as `norm`.\n Default：None.\n norm_cfg (dict): Config dict for normalization layer.\n Default: dict(type='LN').\n init_cfg (obj:`mmcv.ConfigDict`): The Config for initialization.\n Default: None.\n batch_first (bool): Key, Query and Value are shape\n of (batch, n, embed_dim)\n or (n, batch, embed_dim). Default to False.\n \"\"\"\n\n def __init__(self,\n attn_cfgs=None,\n ffn_cfgs=dict(\n type='FFN',\n embed_dims=256,\n feedforward_channels=1024,\n num_fcs=2,\n ffn_drop=0.,\n act_cfg=dict(type='ReLU', inplace=True),\n ),\n operation_order=None,\n norm_cfg=dict(type='LN'),\n init_cfg=None,\n batch_first=False,\n **kwargs):\n\n deprecated_args = dict(\n feedforward_channels='feedforward_channels',\n ffn_dropout='ffn_drop',\n ffn_num_fcs='num_fcs')\n for ori_name, new_name in deprecated_args.items():\n if ori_name in kwargs:\n warnings.warn(\n f'The arguments `{ori_name}` in BaseTransformerLayer '\n f'has been deprecated, now you should set `{new_name}` '\n f'and other FFN related arguments '\n f'to a dict named `ffn_cfgs`. ', DeprecationWarning)\n ffn_cfgs[new_name] = kwargs[ori_name]\n\n super(BaseTransformerLayer, self).__init__(init_cfg)\n\n self.batch_first = batch_first\n\n assert set(operation_order) & set(\n ['self_attn', 'norm', 'ffn', 'cross_attn']) == \\\n set(operation_order), f'The operation_order of' \\\n f' {self.__class__.__name__} should ' \\\n f'contains all four operation type ' \\\n f\"{['self_attn', 'norm', 'ffn', 'cross_attn']}\"\n\n num_attn = operation_order.count('self_attn') + operation_order.count(\n 'cross_attn')\n if isinstance(attn_cfgs, dict):\n attn_cfgs = [copy.deepcopy(attn_cfgs) for _ in range(num_attn)]\n else:\n assert num_attn == len(attn_cfgs), f'The length ' \\\n f'of attn_cfg {num_attn} is ' \\\n f'not consistent with the number of attention' \\\n f'in operation_order {operation_order}.'\n\n self.num_attn = num_attn\n self.operation_order = operation_order\n self.norm_cfg = norm_cfg\n self.pre_norm = operation_order[0] == 'norm'\n self.attentions = ModuleList()\n\n index = 0\n for operation_name in operation_order:\n if operation_name in ['self_attn', 'cross_attn']:\n if 'batch_first' in attn_cfgs[index]:\n assert self.batch_first == attn_cfgs[index]['batch_first']\n else:\n attn_cfgs[index]['batch_first'] = self.batch_first\n attention = build_attention(attn_cfgs[index])\n # Some custom attentions used as `self_attn`\n # or `cross_attn` can have different behavior.\n attention.operation_name = operation_name\n self.attentions.append(attention)\n index += 1\n\n self.embed_dims = self.attentions[0].embed_dims\n\n self.ffns = ModuleList()\n num_ffns = operation_order.count('ffn')\n if isinstance(ffn_cfgs, dict):\n ffn_cfgs = ConfigDict(ffn_cfgs)\n if isinstance(ffn_cfgs, dict):\n ffn_cfgs = [copy.deepcopy(ffn_cfgs) for _ in range(num_ffns)]\n assert len(ffn_cfgs) == num_ffns\n for ffn_index in range(num_ffns):\n if 'embed_dims' not in ffn_cfgs[ffn_index]:\n ffn_cfgs[ffn_index]['embed_dims'] = self.embed_dims\n else:\n assert ffn_cfgs[ffn_index]['embed_dims'] == self.embed_dims\n self.ffns.append(\n build_feedforward_network(ffn_cfgs[ffn_index],\n dict(type='FFN')))\n\n self.norms = ModuleList()\n num_norms = operation_order.count('norm')\n for _ in range(num_norms):\n self.norms.append(build_norm_layer(norm_cfg, self.embed_dims)[1])\n\n def forward(self,\n query,\n key=None,\n value=None,\n query_pos=None,\n key_pos=None,\n attn_masks=None,\n query_key_padding_mask=None,\n key_padding_mask=None,\n **kwargs):\n \"\"\"Forward function for `TransformerDecoderLayer`.\n\n **kwargs contains some specific arguments of attentions.\n\n Args:\n query (Tensor): The input query with shape\n [num_queries, bs, embed_dims] if\n self.batch_first is False, else\n [bs, num_queries embed_dims].\n key (Tensor): The key tensor with shape [num_keys, bs,\n embed_dims] if self.batch_first is False, else\n [bs, num_keys, embed_dims] .\n value (Tensor): The value tensor with same shape as `key`.\n query_pos (Tensor): The positional encoding for `query`.\n Default: None.\n key_pos (Tensor): The positional encoding for `key`.\n Default: None.\n attn_masks (List[Tensor] | None): 2D Tensor used in\n calculation of corresponding attention. The length of\n it should equal to the number of `attention` in\n `operation_order`. Default: None.\n query_key_padding_mask (Tensor): ByteTensor for `query`, with\n shape [bs, num_queries]. Only used in `self_attn` layer.\n Defaults to None.\n key_padding_mask (Tensor): ByteTensor for `query`, with\n shape [bs, num_keys]. Default: None.\n\n Returns:\n Tensor: forwarded results with shape [num_queries, bs, embed_dims].\n \"\"\"\n\n norm_index = 0\n attn_index = 0\n ffn_index = 0\n identity = query\n if attn_masks is None:\n attn_masks = [None for _ in range(self.num_attn)]\n elif isinstance(attn_masks, torch.Tensor):\n attn_masks = [\n copy.deepcopy(attn_masks) for _ in range(self.num_attn)\n ]\n warnings.warn(f'Use same attn_mask in all attentions in '\n f'{self.__class__.__name__} ')\n else:\n assert len(attn_masks) == self.num_attn, f'The length of ' \\\n f'attn_masks {len(attn_masks)} must be equal ' \\\n f'to the number of attention in ' \\\n f'operation_order {self.num_attn}'\n\n for layer in self.operation_order:\n if layer == 'self_attn':\n temp_key = temp_value = query\n query = self.attentions[attn_index](\n query,\n temp_key,\n temp_value,\n identity if self.pre_norm else None,\n query_pos=query_pos,\n key_pos=query_pos,\n attn_mask=attn_masks[attn_index],\n key_padding_mask=query_key_padding_mask,\n **kwargs)\n attn_index += 1\n identity = query\n\n elif layer == 'norm':\n query = self.norms[norm_index](query)\n norm_index += 1\n\n elif layer == 'cross_attn':\n query = self.attentions[attn_index](\n query,\n key,\n value,\n identity if self.pre_norm else None,\n query_pos=query_pos,\n key_pos=key_pos,\n attn_mask=attn_masks[attn_index],\n key_padding_mask=key_padding_mask,\n **kwargs)\n attn_index += 1\n identity = query\n\n elif layer == 'ffn':\n query = self.ffns[ffn_index](\n query, identity if self.pre_norm else None)\n ffn_index += 1\n\n return query\n\n\n@TRANSFORMER_LAYER_SEQUENCE.register_module()\nclass TransformerLayerSequence(BaseModule):\n \"\"\"Base class for TransformerEncoder and TransformerDecoder in vision\n transformer.\n\n As base-class of Encoder and Decoder in vision transformer.\n Support customization such as specifying different kind\n of `transformer_layer` in `transformer_coder`.\n\n Args:\n transformerlayer (list[obj:`mmcv.ConfigDict`] |\n obj:`mmcv.ConfigDict`): Config of transformerlayer\n in TransformerCoder. If it is obj:`mmcv.ConfigDict`,\n it would be repeated `num_layer` times to a\n list[`mmcv.ConfigDict`]. Default: None.\n num_layers (int): The number of `TransformerLayer`. Default: None.\n init_cfg (obj:`mmcv.ConfigDict`): The Config for initialization.\n Default: None.\n \"\"\"\n\n def __init__(self, transformerlayers=None, num_layers=None, init_cfg=None):\n super(TransformerLayerSequence, self).__init__(init_cfg)\n if isinstance(transformerlayers, dict):\n transformerlayers = [\n copy.deepcopy(transformerlayers) for _ in range(num_layers)\n ]\n else:\n assert isinstance(transformerlayers, list) and \\\n len(transformerlayers) == num_layers\n self.num_layers = num_layers\n self.layers = ModuleList()\n for i in range(num_layers):\n self.layers.append(build_transformer_layer(transformerlayers[i]))\n self.embed_dims = self.layers[0].embed_dims\n self.pre_norm = self.layers[0].pre_norm\n\n def forward(self,\n query,\n key,\n value,\n query_pos=None,\n key_pos=None,\n attn_masks=None,\n query_key_padding_mask=None,\n key_padding_mask=None,\n **kwargs):\n \"\"\"Forward function for `TransformerCoder`.\n\n Args:\n query (Tensor): Input query with shape\n `(num_queries, bs, embed_dims)`.\n key (Tensor): The key tensor with shape\n `(num_keys, bs, embed_dims)`.\n value (Tensor): The value tensor with shape\n `(num_keys, bs, embed_dims)`.\n query_pos (Tensor): The positional encoding for `query`.\n Default: None.\n key_pos (Tensor): The positional encoding for `key`.\n Default: None.\n attn_masks (List[Tensor], optional): Each element is 2D Tensor\n which is used in calculation of corresponding attention in\n operation_order. Default: None.\n query_key_padding_mask (Tensor): ByteTensor for `query`, with\n shape [bs, num_queries]. Only used in self-attention\n Default: None.\n key_padding_mask (Tensor): ByteTensor for `query`, with\n shape [bs, num_keys]. Default: None.\n\n Returns:\n Tensor: results with shape [num_queries, bs, embed_dims].\n \"\"\"\n for layer in self.layers:\n query = layer(\n query,\n key,\n value,\n query_pos=query_pos,\n key_pos=key_pos,\n attn_masks=attn_masks,\n query_key_padding_mask=query_key_padding_mask,\n key_padding_mask=key_padding_mask,\n **kwargs)\n return query\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/upsample.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom ..utils import xavier_init\nfrom .registry import UPSAMPLE_LAYERS\n\nUPSAMPLE_LAYERS.register_module('nearest', module=nn.Upsample)\nUPSAMPLE_LAYERS.register_module('bilinear', module=nn.Upsample)\n\n\n@UPSAMPLE_LAYERS.register_module(name='pixel_shuffle')\nclass PixelShufflePack(nn.Module):\n \"\"\"Pixel Shuffle upsample layer.\n\n This module packs `F.pixel_shuffle()` and a nn.Conv2d module together to\n achieve a simple upsampling with pixel shuffle.\n\n Args:\n in_channels (int): Number of input channels.\n out_channels (int): Number of output channels.\n scale_factor (int): Upsample ratio.\n upsample_kernel (int): Kernel size of the conv layer to expand the\n channels.\n \"\"\"\n\n def __init__(self, in_channels, out_channels, scale_factor,\n upsample_kernel):\n super(PixelShufflePack, self).__init__()\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.scale_factor = scale_factor\n self.upsample_kernel = upsample_kernel\n self.upsample_conv = nn.Conv2d(\n self.in_channels,\n self.out_channels * scale_factor * scale_factor,\n self.upsample_kernel,\n padding=(self.upsample_kernel - 1) // 2)\n self.init_weights()\n\n def init_weights(self):\n xavier_init(self.upsample_conv, distribution='uniform')\n\n def forward(self, x):\n x = self.upsample_conv(x)\n x = F.pixel_shuffle(x, self.scale_factor)\n return x\n\n\ndef build_upsample_layer(cfg, *args, **kwargs):\n \"\"\"Build upsample layer.\n\n Args:\n cfg (dict): The upsample layer config, which should contain:\n\n - type (str): Layer type.\n - scale_factor (int): Upsample ratio, which is not applicable to\n deconv.\n - layer args: Args needed to instantiate a upsample layer.\n args (argument list): Arguments passed to the ``__init__``\n method of the corresponding conv layer.\n kwargs (keyword arguments): Keyword arguments passed to the\n ``__init__`` method of the corresponding conv layer.\n\n Returns:\n nn.Module: Created upsample layer.\n \"\"\"\n if not isinstance(cfg, dict):\n raise TypeError(f'cfg must be a dict, but got {type(cfg)}')\n if 'type' not in cfg:\n raise KeyError(\n f'the cfg dict must contain the key \"type\", but got {cfg}')\n cfg_ = cfg.copy()\n\n layer_type = cfg_.pop('type')\n if layer_type not in UPSAMPLE_LAYERS:\n raise KeyError(f'Unrecognized upsample type {layer_type}')\n else:\n upsample = UPSAMPLE_LAYERS.get(layer_type)\n\n if upsample is nn.Upsample:\n cfg_['mode'] = layer_type\n layer = upsample(*args, **kwargs, **cfg_)\n return layer\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/bricks/wrappers.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nr\"\"\"Modified from https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/wrappers.py # noqa: E501\n\nWrap some nn modules to support empty tensor input. Currently, these wrappers\nare mainly used in mask heads like fcn_mask_head and maskiou_heads since mask\nheads are trained on only positive RoIs.\n\"\"\"\nimport math\n\nimport torch\nimport torch.nn as nn\nfrom torch.nn.modules.utils import _pair, _triple\n\nfrom .registry import CONV_LAYERS, UPSAMPLE_LAYERS\n\nif torch.__version__ == 'parrots':\n TORCH_VERSION = torch.__version__\nelse:\n # torch.__version__ could be 1.3.1+cu92, we only need the first two\n # for comparison\n TORCH_VERSION = tuple(int(x) for x in torch.__version__.split('.')[:2])\n\n\ndef obsolete_torch_version(torch_version, version_threshold):\n return torch_version == 'parrots' or torch_version <= version_threshold\n\n\nclass NewEmptyTensorOp(torch.autograd.Function):\n\n @staticmethod\n def forward(ctx, x, new_shape):\n ctx.shape = x.shape\n return x.new_empty(new_shape)\n\n @staticmethod\n def backward(ctx, grad):\n shape = ctx.shape\n return NewEmptyTensorOp.apply(grad, shape), None\n\n\n@CONV_LAYERS.register_module('Conv', force=True)\nclass Conv2d(nn.Conv2d):\n\n def forward(self, x):\n if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)):\n out_shape = [x.shape[0], self.out_channels]\n for i, k, p, s, d in zip(x.shape[-2:], self.kernel_size,\n self.padding, self.stride, self.dilation):\n o = (i + 2 * p - (d * (k - 1) + 1)) // s + 1\n out_shape.append(o)\n empty = NewEmptyTensorOp.apply(x, out_shape)\n if self.training:\n # produce dummy gradient to avoid DDP warning.\n dummy = sum(x.view(-1)[0] for x in self.parameters()) * 0.0\n return empty + dummy\n else:\n return empty\n\n return super().forward(x)\n\n\n@CONV_LAYERS.register_module('Conv3d', force=True)\nclass Conv3d(nn.Conv3d):\n\n def forward(self, x):\n if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)):\n out_shape = [x.shape[0], self.out_channels]\n for i, k, p, s, d in zip(x.shape[-3:], self.kernel_size,\n self.padding, self.stride, self.dilation):\n o = (i + 2 * p - (d * (k - 1) + 1)) // s + 1\n out_shape.append(o)\n empty = NewEmptyTensorOp.apply(x, out_shape)\n if self.training:\n # produce dummy gradient to avoid DDP warning.\n dummy = sum(x.view(-1)[0] for x in self.parameters()) * 0.0\n return empty + dummy\n else:\n return empty\n\n return super().forward(x)\n\n\n@CONV_LAYERS.register_module()\n@CONV_LAYERS.register_module('deconv')\n@UPSAMPLE_LAYERS.register_module('deconv', force=True)\nclass ConvTranspose2d(nn.ConvTranspose2d):\n\n def forward(self, x):\n if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)):\n out_shape = [x.shape[0], self.out_channels]\n for i, k, p, s, d, op in zip(x.shape[-2:], self.kernel_size,\n self.padding, self.stride,\n self.dilation, self.output_padding):\n out_shape.append((i - 1) * s - 2 * p + (d * (k - 1) + 1) + op)\n empty = NewEmptyTensorOp.apply(x, out_shape)\n if self.training:\n # produce dummy gradient to avoid DDP warning.\n dummy = sum(x.view(-1)[0] for x in self.parameters()) * 0.0\n return empty + dummy\n else:\n return empty\n\n return super().forward(x)\n\n\n@CONV_LAYERS.register_module()\n@CONV_LAYERS.register_module('deconv3d')\n@UPSAMPLE_LAYERS.register_module('deconv3d', force=True)\nclass ConvTranspose3d(nn.ConvTranspose3d):\n\n def forward(self, x):\n if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)):\n out_shape = [x.shape[0], self.out_channels]\n for i, k, p, s, d, op in zip(x.shape[-3:], self.kernel_size,\n self.padding, self.stride,\n self.dilation, self.output_padding):\n out_shape.append((i - 1) * s - 2 * p + (d * (k - 1) + 1) + op)\n empty = NewEmptyTensorOp.apply(x, out_shape)\n if self.training:\n # produce dummy gradient to avoid DDP warning.\n dummy = sum(x.view(-1)[0] for x in self.parameters()) * 0.0\n return empty + dummy\n else:\n return empty\n\n return super().forward(x)\n\n\nclass MaxPool2d(nn.MaxPool2d):\n\n def forward(self, x):\n # PyTorch 1.9 does not support empty tensor inference yet\n if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 9)):\n out_shape = list(x.shape[:2])\n for i, k, p, s, d in zip(x.shape[-2:], _pair(self.kernel_size),\n _pair(self.padding), _pair(self.stride),\n _pair(self.dilation)):\n o = (i + 2 * p - (d * (k - 1) + 1)) / s + 1\n o = math.ceil(o) if self.ceil_mode else math.floor(o)\n out_shape.append(o)\n empty = NewEmptyTensorOp.apply(x, out_shape)\n return empty\n\n return super().forward(x)\n\n\nclass MaxPool3d(nn.MaxPool3d):\n\n def forward(self, x):\n # PyTorch 1.9 does not support empty tensor inference yet\n if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 9)):\n out_shape = list(x.shape[:2])\n for i, k, p, s, d in zip(x.shape[-3:], _triple(self.kernel_size),\n _triple(self.padding),\n _triple(self.stride),\n _triple(self.dilation)):\n o = (i + 2 * p - (d * (k - 1) + 1)) / s + 1\n o = math.ceil(o) if self.ceil_mode else math.floor(o)\n out_shape.append(o)\n empty = NewEmptyTensorOp.apply(x, out_shape)\n return empty\n\n return super().forward(x)\n\n\nclass Linear(torch.nn.Linear):\n\n def forward(self, x):\n # empty tensor forward of Linear layer is supported in Pytorch 1.6\n if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 5)):\n out_shape = [x.shape[0], self.out_features]\n empty = NewEmptyTensorOp.apply(x, out_shape)\n if self.training:\n # produce dummy gradient to avoid DDP warning.\n dummy = sum(x.view(-1)[0] for x in self.parameters()) * 0.0\n return empty + dummy\n else:\n return empty\n\n return super().forward(x)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/builder.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom ..runner import Sequential\nfrom ..utils import Registry, build_from_cfg\n\n\ndef build_model_from_cfg(cfg, registry, default_args=None):\n \"\"\"Build a PyTorch model from config dict(s). Different from\n ``build_from_cfg``, if cfg is a list, a ``nn.Sequential`` will be built.\n\n Args:\n cfg (dict, list[dict]): The config of modules, is is either a config\n dict or a list of config dicts. If cfg is a list, a\n the built modules will be wrapped with ``nn.Sequential``.\n registry (:obj:`Registry`): A registry the module belongs to.\n default_args (dict, optional): Default arguments to build the module.\n Defaults to None.\n\n Returns:\n nn.Module: A built nn module.\n \"\"\"\n if isinstance(cfg, list):\n modules = [\n build_from_cfg(cfg_, registry, default_args) for cfg_ in cfg\n ]\n return Sequential(*modules)\n else:\n return build_from_cfg(cfg, registry, default_args)\n\n\nMODELS = Registry('model', build_func=build_model_from_cfg)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/resnet.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport logging\n\nimport torch.nn as nn\nimport torch.utils.checkpoint as cp\n\nfrom .utils import constant_init, kaiming_init\n\n\ndef conv3x3(in_planes, out_planes, stride=1, dilation=1):\n \"\"\"3x3 convolution with padding.\"\"\"\n return nn.Conv2d(\n in_planes,\n out_planes,\n kernel_size=3,\n stride=stride,\n padding=dilation,\n dilation=dilation,\n bias=False)\n\n\nclass BasicBlock(nn.Module):\n expansion = 1\n\n def __init__(self,\n inplanes,\n planes,\n stride=1,\n dilation=1,\n downsample=None,\n style='pytorch',\n with_cp=False):\n super(BasicBlock, self).__init__()\n assert style in ['pytorch', 'caffe']\n self.conv1 = conv3x3(inplanes, planes, stride, dilation)\n self.bn1 = nn.BatchNorm2d(planes)\n self.relu = nn.ReLU(inplace=True)\n self.conv2 = conv3x3(planes, planes)\n self.bn2 = nn.BatchNorm2d(planes)\n self.downsample = downsample\n self.stride = stride\n self.dilation = dilation\n assert not with_cp\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n out = self.relu(out)\n\n return out\n\n\nclass Bottleneck(nn.Module):\n expansion = 4\n\n def __init__(self,\n inplanes,\n planes,\n stride=1,\n dilation=1,\n downsample=None,\n style='pytorch',\n with_cp=False):\n \"\"\"Bottleneck block.\n\n If style is \"pytorch\", the stride-two layer is the 3x3 conv layer, if\n it is \"caffe\", the stride-two layer is the first 1x1 conv layer.\n \"\"\"\n super(Bottleneck, self).__init__()\n assert style in ['pytorch', 'caffe']\n if style == 'pytorch':\n conv1_stride = 1\n conv2_stride = stride\n else:\n conv1_stride = stride\n conv2_stride = 1\n self.conv1 = nn.Conv2d(\n inplanes, planes, kernel_size=1, stride=conv1_stride, bias=False)\n self.conv2 = nn.Conv2d(\n planes,\n planes,\n kernel_size=3,\n stride=conv2_stride,\n padding=dilation,\n dilation=dilation,\n bias=False)\n\n self.bn1 = nn.BatchNorm2d(planes)\n self.bn2 = nn.BatchNorm2d(planes)\n self.conv3 = nn.Conv2d(\n planes, planes * self.expansion, kernel_size=1, bias=False)\n self.bn3 = nn.BatchNorm2d(planes * self.expansion)\n self.relu = nn.ReLU(inplace=True)\n self.downsample = downsample\n self.stride = stride\n self.dilation = dilation\n self.with_cp = with_cp\n\n def forward(self, x):\n\n def _inner_forward(x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n out = self.relu(out)\n\n out = self.conv3(out)\n out = self.bn3(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n\n return out\n\n if self.with_cp and x.requires_grad:\n out = cp.checkpoint(_inner_forward, x)\n else:\n out = _inner_forward(x)\n\n out = self.relu(out)\n\n return out\n\n\ndef make_res_layer(block,\n inplanes,\n planes,\n blocks,\n stride=1,\n dilation=1,\n style='pytorch',\n with_cp=False):\n downsample = None\n if stride != 1 or inplanes != planes * block.expansion:\n downsample = nn.Sequential(\n nn.Conv2d(\n inplanes,\n planes * block.expansion,\n kernel_size=1,\n stride=stride,\n bias=False),\n nn.BatchNorm2d(planes * block.expansion),\n )\n\n layers = []\n layers.append(\n block(\n inplanes,\n planes,\n stride,\n dilation,\n downsample,\n style=style,\n with_cp=with_cp))\n inplanes = planes * block.expansion\n for _ in range(1, blocks):\n layers.append(\n block(inplanes, planes, 1, dilation, style=style, with_cp=with_cp))\n\n return nn.Sequential(*layers)\n\n\nclass ResNet(nn.Module):\n \"\"\"ResNet backbone.\n\n Args:\n depth (int): Depth of resnet, from {18, 34, 50, 101, 152}.\n num_stages (int): Resnet stages, normally 4.\n strides (Sequence[int]): Strides of the first block of each stage.\n dilations (Sequence[int]): Dilation of each stage.\n out_indices (Sequence[int]): Output from which stages.\n style (str): `pytorch` or `caffe`. If set to \"pytorch\", the stride-two\n layer is the 3x3 conv layer, otherwise the stride-two layer is\n the first 1x1 conv layer.\n frozen_stages (int): Stages to be frozen (all param fixed). -1 means\n not freezing any parameters.\n bn_eval (bool): Whether to set BN layers as eval mode, namely, freeze\n running stats (mean and var).\n bn_frozen (bool): Whether to freeze weight and bias of BN layers.\n with_cp (bool): Use checkpoint or not. Using checkpoint will save some\n memory while slowing down the training speed.\n \"\"\"\n\n arch_settings = {\n 18: (BasicBlock, (2, 2, 2, 2)),\n 34: (BasicBlock, (3, 4, 6, 3)),\n 50: (Bottleneck, (3, 4, 6, 3)),\n 101: (Bottleneck, (3, 4, 23, 3)),\n 152: (Bottleneck, (3, 8, 36, 3))\n }\n\n def __init__(self,\n depth,\n num_stages=4,\n strides=(1, 2, 2, 2),\n dilations=(1, 1, 1, 1),\n out_indices=(0, 1, 2, 3),\n style='pytorch',\n frozen_stages=-1,\n bn_eval=True,\n bn_frozen=False,\n with_cp=False):\n super(ResNet, self).__init__()\n if depth not in self.arch_settings:\n raise KeyError(f'invalid depth {depth} for resnet')\n assert num_stages >= 1 and num_stages <= 4\n block, stage_blocks = self.arch_settings[depth]\n stage_blocks = stage_blocks[:num_stages]\n assert len(strides) == len(dilations) == num_stages\n assert max(out_indices) < num_stages\n\n self.out_indices = out_indices\n self.style = style\n self.frozen_stages = frozen_stages\n self.bn_eval = bn_eval\n self.bn_frozen = bn_frozen\n self.with_cp = with_cp\n\n self.inplanes = 64\n self.conv1 = nn.Conv2d(\n 3, 64, kernel_size=7, stride=2, padding=3, bias=False)\n self.bn1 = nn.BatchNorm2d(64)\n self.relu = nn.ReLU(inplace=True)\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n\n self.res_layers = []\n for i, num_blocks in enumerate(stage_blocks):\n stride = strides[i]\n dilation = dilations[i]\n planes = 64 * 2**i\n res_layer = make_res_layer(\n block,\n self.inplanes,\n planes,\n num_blocks,\n stride=stride,\n dilation=dilation,\n style=self.style,\n with_cp=with_cp)\n self.inplanes = planes * block.expansion\n layer_name = f'layer{i + 1}'\n self.add_module(layer_name, res_layer)\n self.res_layers.append(layer_name)\n\n self.feat_dim = block.expansion * 64 * 2**(len(stage_blocks) - 1)\n\n def init_weights(self, pretrained=None):\n if isinstance(pretrained, str):\n logger = logging.getLogger()\n from ..runner import load_checkpoint\n load_checkpoint(self, pretrained, strict=False, logger=logger)\n elif pretrained is None:\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n kaiming_init(m)\n elif isinstance(m, nn.BatchNorm2d):\n constant_init(m, 1)\n else:\n raise TypeError('pretrained must be a str or None')\n\n def forward(self, x):\n x = self.conv1(x)\n x = self.bn1(x)\n x = self.relu(x)\n x = self.maxpool(x)\n outs = []\n for i, layer_name in enumerate(self.res_layers):\n res_layer = getattr(self, layer_name)\n x = res_layer(x)\n if i in self.out_indices:\n outs.append(x)\n if len(outs) == 1:\n return outs[0]\n else:\n return tuple(outs)\n\n def train(self, mode=True):\n super(ResNet, self).train(mode)\n if self.bn_eval:\n for m in self.modules():\n if isinstance(m, nn.BatchNorm2d):\n m.eval()\n if self.bn_frozen:\n for params in m.parameters():\n params.requires_grad = False\n if mode and self.frozen_stages >= 0:\n for param in self.conv1.parameters():\n param.requires_grad = False\n for param in self.bn1.parameters():\n param.requires_grad = False\n self.bn1.eval()\n self.bn1.weight.requires_grad = False\n self.bn1.bias.requires_grad = False\n for i in range(1, self.frozen_stages + 1):\n mod = getattr(self, f'layer{i}')\n mod.eval()\n for param in mod.parameters():\n param.requires_grad = False\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/utils/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .flops_counter import get_model_complexity_info\nfrom .fuse_conv_bn import fuse_conv_bn\nfrom .sync_bn import revert_sync_batchnorm\nfrom .weight_init import (INITIALIZERS, Caffe2XavierInit, ConstantInit,\n KaimingInit, NormalInit, PretrainedInit,\n TruncNormalInit, UniformInit, XavierInit,\n bias_init_with_prob, caffe2_xavier_init,\n constant_init, initialize, kaiming_init, normal_init,\n trunc_normal_init, uniform_init, xavier_init)\n\n__all__ = [\n 'get_model_complexity_info', 'bias_init_with_prob', 'caffe2_xavier_init',\n 'constant_init', 'kaiming_init', 'normal_init', 'trunc_normal_init',\n 'uniform_init', 'xavier_init', 'fuse_conv_bn', 'initialize',\n 'INITIALIZERS', 'ConstantInit', 'XavierInit', 'NormalInit',\n 'TruncNormalInit', 'UniformInit', 'KaimingInit', 'PretrainedInit',\n 'Caffe2XavierInit', 'revert_sync_batchnorm'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/utils/flops_counter.py", "content": "# Modified from flops-counter.pytorch by Vladislav Sovrasov\n# original repo: https://github.com/sovrasov/flops-counter.pytorch\n\n# MIT License\n\n# Copyright (c) 2018 Vladislav Sovrasov\n\n# Permission is hereby granted, free of charge, to any person obtaining a copy\n# of this software and associated documentation files (the \"Software\"), to deal\n# in the Software without restriction, including without limitation the rights\n# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n# copies of the Software, and to permit persons to whom the Software is\n# furnished to do so, subject to the following conditions:\n\n# The above copyright notice and this permission notice shall be included in\n# all copies or substantial portions of the Software.\n\n# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n# SOFTWARE.\n\nimport sys\nimport warnings\nfrom functools import partial\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\n\nimport mmcv\n\n\ndef get_model_complexity_info(model,\n input_shape,\n print_per_layer_stat=True,\n as_strings=True,\n input_constructor=None,\n flush=False,\n ost=sys.stdout):\n \"\"\"Get complexity information of a model.\n\n This method can calculate FLOPs and parameter counts of a model with\n corresponding input shape. It can also print complexity information for\n each layer in a model.\n\n Supported layers are listed as below:\n - Convolutions: ``nn.Conv1d``, ``nn.Conv2d``, ``nn.Conv3d``.\n - Activations: ``nn.ReLU``, ``nn.PReLU``, ``nn.ELU``,\n ``nn.LeakyReLU``, ``nn.ReLU6``.\n - Poolings: ``nn.MaxPool1d``, ``nn.MaxPool2d``, ``nn.MaxPool3d``,\n ``nn.AvgPool1d``, ``nn.AvgPool2d``, ``nn.AvgPool3d``,\n ``nn.AdaptiveMaxPool1d``, ``nn.AdaptiveMaxPool2d``,\n ``nn.AdaptiveMaxPool3d``, ``nn.AdaptiveAvgPool1d``,\n ``nn.AdaptiveAvgPool2d``, ``nn.AdaptiveAvgPool3d``.\n - BatchNorms: ``nn.BatchNorm1d``, ``nn.BatchNorm2d``,\n ``nn.BatchNorm3d``, ``nn.GroupNorm``, ``nn.InstanceNorm1d``,\n ``InstanceNorm2d``, ``InstanceNorm3d``, ``nn.LayerNorm``.\n - Linear: ``nn.Linear``.\n - Deconvolution: ``nn.ConvTranspose2d``.\n - Upsample: ``nn.Upsample``.\n\n Args:\n model (nn.Module): The model for complexity calculation.\n input_shape (tuple): Input shape used for calculation.\n print_per_layer_stat (bool): Whether to print complexity information\n for each layer in a model. Default: True.\n as_strings (bool): Output FLOPs and params counts in a string form.\n Default: True.\n input_constructor (None | callable): If specified, it takes a callable\n method that generates input. otherwise, it will generate a random\n tensor with input shape to calculate FLOPs. Default: None.\n flush (bool): same as that in :func:`print`. Default: False.\n ost (stream): same as ``file`` param in :func:`print`.\n Default: sys.stdout.\n\n Returns:\n tuple[float | str]: If ``as_strings`` is set to True, it will return\n FLOPs and parameter counts in a string format. otherwise, it will\n return those in a float number format.\n \"\"\"\n assert type(input_shape) is tuple\n assert len(input_shape) >= 1\n assert isinstance(model, nn.Module)\n flops_model = add_flops_counting_methods(model)\n flops_model.eval()\n flops_model.start_flops_count()\n if input_constructor:\n input = input_constructor(input_shape)\n _ = flops_model(**input)\n else:\n try:\n batch = torch.ones(()).new_empty(\n (1, *input_shape),\n dtype=next(flops_model.parameters()).dtype,\n device=next(flops_model.parameters()).device)\n except StopIteration:\n # Avoid StopIteration for models which have no parameters,\n # like `nn.Relu()`, `nn.AvgPool2d`, etc.\n batch = torch.ones(()).new_empty((1, *input_shape))\n\n _ = flops_model(batch)\n\n flops_count, params_count = flops_model.compute_average_flops_cost()\n if print_per_layer_stat:\n print_model_with_flops(\n flops_model, flops_count, params_count, ost=ost, flush=flush)\n flops_model.stop_flops_count()\n\n if as_strings:\n return flops_to_string(flops_count), params_to_string(params_count)\n\n return flops_count, params_count\n\n\ndef flops_to_string(flops, units='GFLOPs', precision=2):\n \"\"\"Convert FLOPs number into a string.\n\n Note that Here we take a multiply-add counts as one FLOP.\n\n Args:\n flops (float): FLOPs number to be converted.\n units (str | None): Converted FLOPs units. Options are None, 'GFLOPs',\n 'MFLOPs', 'KFLOPs', 'FLOPs'. If set to None, it will automatically\n choose the most suitable unit for FLOPs. Default: 'GFLOPs'.\n precision (int): Digit number after the decimal point. Default: 2.\n\n Returns:\n str: The converted FLOPs number with units.\n\n Examples:\n >>> flops_to_string(1e9)\n '1.0 GFLOPs'\n >>> flops_to_string(2e5, 'MFLOPs')\n '0.2 MFLOPs'\n >>> flops_to_string(3e-9, None)\n '3e-09 FLOPs'\n \"\"\"\n if units is None:\n if flops // 10**9 > 0:\n return str(round(flops / 10.**9, precision)) + ' GFLOPs'\n elif flops // 10**6 > 0:\n return str(round(flops / 10.**6, precision)) + ' MFLOPs'\n elif flops // 10**3 > 0:\n return str(round(flops / 10.**3, precision)) + ' KFLOPs'\n else:\n return str(flops) + ' FLOPs'\n else:\n if units == 'GFLOPs':\n return str(round(flops / 10.**9, precision)) + ' ' + units\n elif units == 'MFLOPs':\n return str(round(flops / 10.**6, precision)) + ' ' + units\n elif units == 'KFLOPs':\n return str(round(flops / 10.**3, precision)) + ' ' + units\n else:\n return str(flops) + ' FLOPs'\n\n\ndef params_to_string(num_params, units=None, precision=2):\n \"\"\"Convert parameter number into a string.\n\n Args:\n num_params (float): Parameter number to be converted.\n units (str | None): Converted FLOPs units. Options are None, 'M',\n 'K' and ''. If set to None, it will automatically choose the most\n suitable unit for Parameter number. Default: None.\n precision (int): Digit number after the decimal point. Default: 2.\n\n Returns:\n str: The converted parameter number with units.\n\n Examples:\n >>> params_to_string(1e9)\n '1000.0 M'\n >>> params_to_string(2e5)\n '200.0 k'\n >>> params_to_string(3e-9)\n '3e-09'\n \"\"\"\n if units is None:\n if num_params // 10**6 > 0:\n return str(round(num_params / 10**6, precision)) + ' M'\n elif num_params // 10**3:\n return str(round(num_params / 10**3, precision)) + ' k'\n else:\n return str(num_params)\n else:\n if units == 'M':\n return str(round(num_params / 10.**6, precision)) + ' ' + units\n elif units == 'K':\n return str(round(num_params / 10.**3, precision)) + ' ' + units\n else:\n return str(num_params)\n\n\ndef print_model_with_flops(model,\n total_flops,\n total_params,\n units='GFLOPs',\n precision=3,\n ost=sys.stdout,\n flush=False):\n \"\"\"Print a model with FLOPs for each layer.\n\n Args:\n model (nn.Module): The model to be printed.\n total_flops (float): Total FLOPs of the model.\n total_params (float): Total parameter counts of the model.\n units (str | None): Converted FLOPs units. Default: 'GFLOPs'.\n precision (int): Digit number after the decimal point. Default: 3.\n ost (stream): same as `file` param in :func:`print`.\n Default: sys.stdout.\n flush (bool): same as that in :func:`print`. Default: False.\n\n Example:\n >>> class ExampleModel(nn.Module):\n\n >>> def __init__(self):\n >>> super().__init__()\n >>> self.conv1 = nn.Conv2d(3, 8, 3)\n >>> self.conv2 = nn.Conv2d(8, 256, 3)\n >>> self.conv3 = nn.Conv2d(256, 8, 3)\n >>> self.avg_pool = nn.AdaptiveAvgPool2d((1, 1))\n >>> self.flatten = nn.Flatten()\n >>> self.fc = nn.Linear(8, 1)\n\n >>> def forward(self, x):\n >>> x = self.conv1(x)\n >>> x = self.conv2(x)\n >>> x = self.conv3(x)\n >>> x = self.avg_pool(x)\n >>> x = self.flatten(x)\n >>> x = self.fc(x)\n >>> return x\n\n >>> model = ExampleModel()\n >>> x = (3, 16, 16)\n to print the complexity information state for each layer, you can use\n >>> get_model_complexity_info(model, x)\n or directly use\n >>> print_model_with_flops(model, 4579784.0, 37361)\n ExampleModel(\n 0.037 M, 100.000% Params, 0.005 GFLOPs, 100.000% FLOPs,\n (conv1): Conv2d(0.0 M, 0.600% Params, 0.0 GFLOPs, 0.959% FLOPs, 3, 8, kernel_size=(3, 3), stride=(1, 1)) # noqa: E501\n (conv2): Conv2d(0.019 M, 50.020% Params, 0.003 GFLOPs, 58.760% FLOPs, 8, 256, kernel_size=(3, 3), stride=(1, 1))\n (conv3): Conv2d(0.018 M, 49.356% Params, 0.002 GFLOPs, 40.264% FLOPs, 256, 8, kernel_size=(3, 3), stride=(1, 1))\n (avg_pool): AdaptiveAvgPool2d(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.017% FLOPs, output_size=(1, 1))\n (flatten): Flatten(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )\n (fc): Linear(0.0 M, 0.024% Params, 0.0 GFLOPs, 0.000% FLOPs, in_features=8, out_features=1, bias=True)\n )\n \"\"\"\n\n def accumulate_params(self):\n if is_supported_instance(self):\n return self.__params__\n else:\n sum = 0\n for m in self.children():\n sum += m.accumulate_params()\n return sum\n\n def accumulate_flops(self):\n if is_supported_instance(self):\n return self.__flops__ / model.__batch_counter__\n else:\n sum = 0\n for m in self.children():\n sum += m.accumulate_flops()\n return sum\n\n def flops_repr(self):\n accumulated_num_params = self.accumulate_params()\n accumulated_flops_cost = self.accumulate_flops()\n return ', '.join([\n params_to_string(\n accumulated_num_params, units='M', precision=precision),\n '{:.3%} Params'.format(accumulated_num_params / total_params),\n flops_to_string(\n accumulated_flops_cost, units=units, precision=precision),\n '{:.3%} FLOPs'.format(accumulated_flops_cost / total_flops),\n self.original_extra_repr()\n ])\n\n def add_extra_repr(m):\n m.accumulate_flops = accumulate_flops.__get__(m)\n m.accumulate_params = accumulate_params.__get__(m)\n flops_extra_repr = flops_repr.__get__(m)\n if m.extra_repr != flops_extra_repr:\n m.original_extra_repr = m.extra_repr\n m.extra_repr = flops_extra_repr\n assert m.extra_repr != m.original_extra_repr\n\n def del_extra_repr(m):\n if hasattr(m, 'original_extra_repr'):\n m.extra_repr = m.original_extra_repr\n del m.original_extra_repr\n if hasattr(m, 'accumulate_flops'):\n del m.accumulate_flops\n\n model.apply(add_extra_repr)\n print(model, file=ost, flush=flush)\n model.apply(del_extra_repr)\n\n\ndef get_model_parameters_number(model):\n \"\"\"Calculate parameter number of a model.\n\n Args:\n model (nn.module): The model for parameter number calculation.\n\n Returns:\n float: Parameter number of the model.\n \"\"\"\n num_params = sum(p.numel() for p in model.parameters() if p.requires_grad)\n return num_params\n\n\ndef add_flops_counting_methods(net_main_module):\n # adding additional methods to the existing module object,\n # this is done this way so that each function has access to self object\n net_main_module.start_flops_count = start_flops_count.__get__(\n net_main_module)\n net_main_module.stop_flops_count = stop_flops_count.__get__(\n net_main_module)\n net_main_module.reset_flops_count = reset_flops_count.__get__(\n net_main_module)\n net_main_module.compute_average_flops_cost = compute_average_flops_cost.__get__( # noqa: E501\n net_main_module)\n\n net_main_module.reset_flops_count()\n\n return net_main_module\n\n\ndef compute_average_flops_cost(self):\n \"\"\"Compute average FLOPs cost.\n\n A method to compute average FLOPs cost, which will be available after\n `add_flops_counting_methods()` is called on a desired net object.\n\n Returns:\n float: Current mean flops consumption per image.\n \"\"\"\n batches_count = self.__batch_counter__\n flops_sum = 0\n for module in self.modules():\n if is_supported_instance(module):\n flops_sum += module.__flops__\n params_sum = get_model_parameters_number(self)\n return flops_sum / batches_count, params_sum\n\n\ndef start_flops_count(self):\n \"\"\"Activate the computation of mean flops consumption per image.\n\n A method to activate the computation of mean flops consumption per image.\n which will be available after ``add_flops_counting_methods()`` is called on\n a desired net object. It should be called before running the network.\n \"\"\"\n add_batch_counter_hook_function(self)\n\n def add_flops_counter_hook_function(module):\n if is_supported_instance(module):\n if hasattr(module, '__flops_handle__'):\n return\n\n else:\n handle = module.register_forward_hook(\n get_modules_mapping()[type(module)])\n\n module.__flops_handle__ = handle\n\n self.apply(partial(add_flops_counter_hook_function))\n\n\ndef stop_flops_count(self):\n \"\"\"Stop computing the mean flops consumption per image.\n\n A method to stop computing the mean flops consumption per image, which will\n be available after ``add_flops_counting_methods()`` is called on a desired\n net object. It can be called to pause the computation whenever.\n \"\"\"\n remove_batch_counter_hook_function(self)\n self.apply(remove_flops_counter_hook_function)\n\n\ndef reset_flops_count(self):\n \"\"\"Reset statistics computed so far.\n\n A method to Reset computed statistics, which will be available after\n `add_flops_counting_methods()` is called on a desired net object.\n \"\"\"\n add_batch_counter_variables_or_reset(self)\n self.apply(add_flops_counter_variable_or_reset)\n\n\n# ---- Internal functions\ndef empty_flops_counter_hook(module, input, output):\n module.__flops__ += 0\n\n\ndef upsample_flops_counter_hook(module, input, output):\n output_size = output[0]\n batch_size = output_size.shape[0]\n output_elements_count = batch_size\n for val in output_size.shape[1:]:\n output_elements_count *= val\n module.__flops__ += int(output_elements_count)\n\n\ndef relu_flops_counter_hook(module, input, output):\n active_elements_count = output.numel()\n module.__flops__ += int(active_elements_count)\n\n\ndef linear_flops_counter_hook(module, input, output):\n input = input[0]\n output_last_dim = output.shape[\n -1] # pytorch checks dimensions, so here we don't care much\n module.__flops__ += int(np.prod(input.shape) * output_last_dim)\n\n\ndef pool_flops_counter_hook(module, input, output):\n input = input[0]\n module.__flops__ += int(np.prod(input.shape))\n\n\ndef norm_flops_counter_hook(module, input, output):\n input = input[0]\n\n batch_flops = np.prod(input.shape)\n if (getattr(module, 'affine', False)\n or getattr(module, 'elementwise_affine', False)):\n batch_flops *= 2\n module.__flops__ += int(batch_flops)\n\n\ndef deconv_flops_counter_hook(conv_module, input, output):\n # Can have multiple inputs, getting the first one\n input = input[0]\n\n batch_size = input.shape[0]\n input_height, input_width = input.shape[2:]\n\n kernel_height, kernel_width = conv_module.kernel_size\n in_channels = conv_module.in_channels\n out_channels = conv_module.out_channels\n groups = conv_module.groups\n\n filters_per_channel = out_channels // groups\n conv_per_position_flops = (\n kernel_height * kernel_width * in_channels * filters_per_channel)\n\n active_elements_count = batch_size * input_height * input_width\n overall_conv_flops = conv_per_position_flops * active_elements_count\n bias_flops = 0\n if conv_module.bias is not None:\n output_height, output_width = output.shape[2:]\n bias_flops = out_channels * batch_size * output_height * output_height\n overall_flops = overall_conv_flops + bias_flops\n\n conv_module.__flops__ += int(overall_flops)\n\n\ndef conv_flops_counter_hook(conv_module, input, output):\n # Can have multiple inputs, getting the first one\n input = input[0]\n\n batch_size = input.shape[0]\n output_dims = list(output.shape[2:])\n\n kernel_dims = list(conv_module.kernel_size)\n in_channels = conv_module.in_channels\n out_channels = conv_module.out_channels\n groups = conv_module.groups\n\n filters_per_channel = out_channels // groups\n conv_per_position_flops = int(\n np.prod(kernel_dims)) * in_channels * filters_per_channel\n\n active_elements_count = batch_size * int(np.prod(output_dims))\n\n overall_conv_flops = conv_per_position_flops * active_elements_count\n\n bias_flops = 0\n\n if conv_module.bias is not None:\n\n bias_flops = out_channels * active_elements_count\n\n overall_flops = overall_conv_flops + bias_flops\n\n conv_module.__flops__ += int(overall_flops)\n\n\ndef batch_counter_hook(module, input, output):\n batch_size = 1\n if len(input) > 0:\n # Can have multiple inputs, getting the first one\n input = input[0]\n batch_size = len(input)\n else:\n warnings.warn('No positional inputs found for a module, '\n 'assuming batch size is 1.')\n module.__batch_counter__ += batch_size\n\n\ndef add_batch_counter_variables_or_reset(module):\n\n module.__batch_counter__ = 0\n\n\ndef add_batch_counter_hook_function(module):\n if hasattr(module, '__batch_counter_handle__'):\n return\n\n handle = module.register_forward_hook(batch_counter_hook)\n module.__batch_counter_handle__ = handle\n\n\ndef remove_batch_counter_hook_function(module):\n if hasattr(module, '__batch_counter_handle__'):\n module.__batch_counter_handle__.remove()\n del module.__batch_counter_handle__\n\n\ndef add_flops_counter_variable_or_reset(module):\n if is_supported_instance(module):\n if hasattr(module, '__flops__') or hasattr(module, '__params__'):\n warnings.warn('variables __flops__ or __params__ are already '\n 'defined for the module' + type(module).__name__ +\n ' ptflops can affect your code!')\n module.__flops__ = 0\n module.__params__ = get_model_parameters_number(module)\n\n\ndef is_supported_instance(module):\n if type(module) in get_modules_mapping():\n return True\n return False\n\n\ndef remove_flops_counter_hook_function(module):\n if is_supported_instance(module):\n if hasattr(module, '__flops_handle__'):\n module.__flops_handle__.remove()\n del module.__flops_handle__\n\n\ndef get_modules_mapping():\n return {\n # convolutions\n nn.Conv1d: conv_flops_counter_hook,\n nn.Conv2d: conv_flops_counter_hook,\n mmcv.cnn.bricks.Conv2d: conv_flops_counter_hook,\n nn.Conv3d: conv_flops_counter_hook,\n mmcv.cnn.bricks.Conv3d: conv_flops_counter_hook,\n # activations\n nn.ReLU: relu_flops_counter_hook,\n nn.PReLU: relu_flops_counter_hook,\n nn.ELU: relu_flops_counter_hook,\n nn.LeakyReLU: relu_flops_counter_hook,\n nn.ReLU6: relu_flops_counter_hook,\n # poolings\n nn.MaxPool1d: pool_flops_counter_hook,\n nn.AvgPool1d: pool_flops_counter_hook,\n nn.AvgPool2d: pool_flops_counter_hook,\n nn.MaxPool2d: pool_flops_counter_hook,\n mmcv.cnn.bricks.MaxPool2d: pool_flops_counter_hook,\n nn.MaxPool3d: pool_flops_counter_hook,\n mmcv.cnn.bricks.MaxPool3d: pool_flops_counter_hook,\n nn.AvgPool3d: pool_flops_counter_hook,\n nn.AdaptiveMaxPool1d: pool_flops_counter_hook,\n nn.AdaptiveAvgPool1d: pool_flops_counter_hook,\n nn.AdaptiveMaxPool2d: pool_flops_counter_hook,\n nn.AdaptiveAvgPool2d: pool_flops_counter_hook,\n nn.AdaptiveMaxPool3d: pool_flops_counter_hook,\n nn.AdaptiveAvgPool3d: pool_flops_counter_hook,\n # normalizations\n nn.BatchNorm1d: norm_flops_counter_hook,\n nn.BatchNorm2d: norm_flops_counter_hook,\n nn.BatchNorm3d: norm_flops_counter_hook,\n nn.GroupNorm: norm_flops_counter_hook,\n nn.InstanceNorm1d: norm_flops_counter_hook,\n nn.InstanceNorm2d: norm_flops_counter_hook,\n nn.InstanceNorm3d: norm_flops_counter_hook,\n nn.LayerNorm: norm_flops_counter_hook,\n # FC\n nn.Linear: linear_flops_counter_hook,\n mmcv.cnn.bricks.Linear: linear_flops_counter_hook,\n # Upscale\n nn.Upsample: upsample_flops_counter_hook,\n # Deconvolution\n nn.ConvTranspose2d: deconv_flops_counter_hook,\n mmcv.cnn.bricks.ConvTranspose2d: deconv_flops_counter_hook,\n }\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/utils/fuse_conv_bn.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\n\n\ndef _fuse_conv_bn(conv, bn):\n \"\"\"Fuse conv and bn into one module.\n\n Args:\n conv (nn.Module): Conv to be fused.\n bn (nn.Module): BN to be fused.\n\n Returns:\n nn.Module: Fused module.\n \"\"\"\n conv_w = conv.weight\n conv_b = conv.bias if conv.bias is not None else torch.zeros_like(\n bn.running_mean)\n\n factor = bn.weight / torch.sqrt(bn.running_var + bn.eps)\n conv.weight = nn.Parameter(conv_w *\n factor.reshape([conv.out_channels, 1, 1, 1]))\n conv.bias = nn.Parameter((conv_b - bn.running_mean) * factor + bn.bias)\n return conv\n\n\ndef fuse_conv_bn(module):\n \"\"\"Recursively fuse conv and bn in a module.\n\n During inference, the functionary of batch norm layers is turned off\n but only the mean and var alone channels are used, which exposes the\n chance to fuse it with the preceding conv layers to save computations and\n simplify network structures.\n\n Args:\n module (nn.Module): Module to be fused.\n\n Returns:\n nn.Module: Fused module.\n \"\"\"\n last_conv = None\n last_conv_name = None\n\n for name, child in module.named_children():\n if isinstance(child,\n (nn.modules.batchnorm._BatchNorm, nn.SyncBatchNorm)):\n if last_conv is None: # only fuse BN that is after Conv\n continue\n fused_conv = _fuse_conv_bn(last_conv, child)\n module._modules[last_conv_name] = fused_conv\n # To reduce changes, set BN as Identity instead of deleting it.\n module._modules[name] = nn.Identity()\n last_conv = None\n elif isinstance(child, nn.Conv2d):\n last_conv = child\n last_conv_name = name\n else:\n fuse_conv_bn(child)\n return module\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/utils/sync_bn.py", "content": "import torch\n\nimport mmcv\n\n\nclass _BatchNormXd(torch.nn.modules.batchnorm._BatchNorm):\n \"\"\"A general BatchNorm layer without input dimension check.\n\n Reproduced from @kapily's work:\n (https://github.com/pytorch/pytorch/issues/41081#issuecomment-783961547)\n The only difference between BatchNorm1d, BatchNorm2d, BatchNorm3d, etc\n is `_check_input_dim` that is designed for tensor sanity checks.\n The check has been bypassed in this class for the convenience of converting\n SyncBatchNorm.\n \"\"\"\n\n def _check_input_dim(self, input):\n return\n\n\ndef revert_sync_batchnorm(module):\n \"\"\"Helper function to convert all `SyncBatchNorm` (SyncBN) and\n `mmcv.ops.sync_bn.SyncBatchNorm`(MMSyncBN) layers in the model to\n `BatchNormXd` layers.\n\n Adapted from @kapily's work:\n (https://github.com/pytorch/pytorch/issues/41081#issuecomment-783961547)\n\n Args:\n module (nn.Module): The module containing `SyncBatchNorm` layers.\n\n Returns:\n module_output: The converted module with `BatchNormXd` layers.\n \"\"\"\n module_output = module\n module_checklist = [torch.nn.modules.batchnorm.SyncBatchNorm]\n if hasattr(mmcv, 'ops'):\n module_checklist.append(mmcv.ops.SyncBatchNorm)\n if isinstance(module, tuple(module_checklist)):\n module_output = _BatchNormXd(module.num_features, module.eps,\n module.momentum, module.affine,\n module.track_running_stats)\n if module.affine:\n # no_grad() may not be needed here but\n # just to be consistent with `convert_sync_batchnorm()`\n with torch.no_grad():\n module_output.weight = module.weight\n module_output.bias = module.bias\n module_output.running_mean = module.running_mean\n module_output.running_var = module.running_var\n module_output.num_batches_tracked = module.num_batches_tracked\n module_output.training = module.training\n # qconfig exists in quantized models\n if hasattr(module, 'qconfig'):\n module_output.qconfig = module.qconfig\n for name, child in module.named_children():\n module_output.add_module(name, revert_sync_batchnorm(child))\n del module\n return module_output\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/utils/weight_init.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport copy\nimport math\nimport warnings\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torch import Tensor\nfrom mmcv.utils import Registry, build_from_cfg, print_log, get_logger\n\nINITIALIZERS = Registry('initializer')\n\n\ndef update_init_info(module, init_info):\n \"\"\"Update the `_params_init_info` in the module if the value of parameters\n are changed.\n\n Args:\n module (obj:`nn.Module`): The module of PyTorch with a user-defined\n attribute `_params_init_info` which records the initialization\n information.\n init_info (str): The string that describes the initialization.\n \"\"\"\n assert hasattr(\n module,\n '_params_init_info'), f'Can not find `_params_init_info` in {module}'\n for name, param in module.named_parameters():\n\n assert param in module._params_init_info, (\n f'Find a new :obj:`Parameter` '\n f'named `{name}` during executing the '\n f'`init_weights` of '\n f'`{module.__class__.__name__}`. '\n f'Please do not add or '\n f'replace parameters during executing '\n f'the `init_weights`. ')\n\n # The parameter has been changed during executing the\n # `init_weights` of module\n mean_value = param.data.mean()\n if module._params_init_info[param]['tmp_mean_value'] != mean_value:\n module._params_init_info[param]['init_info'] = init_info\n module._params_init_info[param]['tmp_mean_value'] = mean_value\n\n\ndef constant_init(module, val, bias=0):\n if hasattr(module, 'weight') and module.weight is not None:\n nn.init.constant_(module.weight, val)\n if hasattr(module, 'bias') and module.bias is not None:\n nn.init.constant_(module.bias, bias)\n\n\ndef xavier_init(module, gain=1, bias=0, distribution='normal'):\n assert distribution in ['uniform', 'normal']\n if hasattr(module, 'weight') and module.weight is not None:\n if distribution == 'uniform':\n nn.init.xavier_uniform_(module.weight, gain=gain)\n else:\n nn.init.xavier_normal_(module.weight, gain=gain)\n if hasattr(module, 'bias') and module.bias is not None:\n nn.init.constant_(module.bias, bias)\n\n\ndef normal_init(module, mean=0, std=1, bias=0):\n if hasattr(module, 'weight') and module.weight is not None:\n nn.init.normal_(module.weight, mean, std)\n if hasattr(module, 'bias') and module.bias is not None:\n nn.init.constant_(module.bias, bias)\n\n\ndef trunc_normal_init(module: nn.Module,\n mean: float = 0,\n std: float = 1,\n a: float = -2,\n b: float = 2,\n bias: float = 0) -> None:\n if hasattr(module, 'weight') and module.weight is not None:\n trunc_normal_(module.weight, mean, std, a, b) # type: ignore\n if hasattr(module, 'bias') and module.bias is not None:\n nn.init.constant_(module.bias, bias) # type: ignore\n\n\ndef uniform_init(module, a=0, b=1, bias=0):\n if hasattr(module, 'weight') and module.weight is not None:\n nn.init.uniform_(module.weight, a, b)\n if hasattr(module, 'bias') and module.bias is not None:\n nn.init.constant_(module.bias, bias)\n\n\ndef kaiming_init(module,\n a=0,\n mode='fan_out',\n nonlinearity='relu',\n bias=0,\n distribution='normal'):\n assert distribution in ['uniform', 'normal']\n if hasattr(module, 'weight') and module.weight is not None:\n if distribution == 'uniform':\n nn.init.kaiming_uniform_(\n module.weight, a=a, mode=mode, nonlinearity=nonlinearity)\n else:\n nn.init.kaiming_normal_(\n module.weight, a=a, mode=mode, nonlinearity=nonlinearity)\n if hasattr(module, 'bias') and module.bias is not None:\n nn.init.constant_(module.bias, bias)\n\n\ndef caffe2_xavier_init(module, bias=0):\n # `XavierFill` in Caffe2 corresponds to `kaiming_uniform_` in PyTorch\n # Acknowledgment to FAIR's internal code\n kaiming_init(\n module,\n a=1,\n mode='fan_in',\n nonlinearity='leaky_relu',\n bias=bias,\n distribution='uniform')\n\n\ndef bias_init_with_prob(prior_prob):\n \"\"\"initialize conv/fc bias value according to a given probability value.\"\"\"\n bias_init = float(-np.log((1 - prior_prob) / prior_prob))\n return bias_init\n\n\ndef _get_bases_name(m):\n return [b.__name__ for b in m.__class__.__bases__]\n\n\nclass BaseInit(object):\n\n def __init__(self, *, bias=0, bias_prob=None, layer=None):\n self.wholemodule = False\n if not isinstance(bias, (int, float)):\n raise TypeError(f'bias must be a number, but got a {type(bias)}')\n\n if bias_prob is not None:\n if not isinstance(bias_prob, float):\n raise TypeError(f'bias_prob type must be float, \\\n but got {type(bias_prob)}')\n\n if layer is not None:\n if not isinstance(layer, (str, list)):\n raise TypeError(f'layer must be a str or a list of str, \\\n but got a {type(layer)}')\n else:\n layer = []\n\n if bias_prob is not None:\n self.bias = bias_init_with_prob(bias_prob)\n else:\n self.bias = bias\n self.layer = [layer] if isinstance(layer, str) else layer\n\n def _get_init_info(self):\n info = f'{self.__class__.__name__}, bias={self.bias}'\n return info\n\n\n@INITIALIZERS.register_module(name='Constant')\nclass ConstantInit(BaseInit):\n \"\"\"Initialize module parameters with constant values.\n\n Args:\n val (int | float): the value to fill the weights in the module with\n bias (int | float): the value to fill the bias. Defaults to 0.\n bias_prob (float, optional): the probability for bias initialization.\n Defaults to None.\n layer (str | list[str], optional): the layer will be initialized.\n Defaults to None.\n \"\"\"\n\n def __init__(self, val, **kwargs):\n super().__init__(**kwargs)\n self.val = val\n\n def __call__(self, module):\n\n def init(m):\n if self.wholemodule:\n constant_init(m, self.val, self.bias)\n else:\n layername = m.__class__.__name__\n basesname = _get_bases_name(m)\n if len(set(self.layer) & set([layername] + basesname)):\n constant_init(m, self.val, self.bias)\n\n module.apply(init)\n if hasattr(module, '_params_init_info'):\n update_init_info(module, init_info=self._get_init_info())\n\n def _get_init_info(self):\n info = f'{self.__class__.__name__}: val={self.val}, bias={self.bias}'\n return info\n\n\n@INITIALIZERS.register_module(name='Xavier')\nclass XavierInit(BaseInit):\n r\"\"\"Initialize module parameters with values according to the method\n described in `Understanding the difficulty of training deep feedforward\n neural networks - Glorot, X. & Bengio, Y. (2010).\n `_\n\n Args:\n gain (int | float): an optional scaling factor. Defaults to 1.\n bias (int | float): the value to fill the bias. Defaults to 0.\n bias_prob (float, optional): the probability for bias initialization.\n Defaults to None.\n distribution (str): distribution either be ``'normal'``\n or ``'uniform'``. Defaults to ``'normal'``.\n layer (str | list[str], optional): the layer will be initialized.\n Defaults to None.\n \"\"\"\n\n def __init__(self, gain=1, distribution='normal', **kwargs):\n super().__init__(**kwargs)\n self.gain = gain\n self.distribution = distribution\n\n def __call__(self, module):\n\n def init(m):\n if self.wholemodule:\n xavier_init(m, self.gain, self.bias, self.distribution)\n else:\n layername = m.__class__.__name__\n basesname = _get_bases_name(m)\n if len(set(self.layer) & set([layername] + basesname)):\n xavier_init(m, self.gain, self.bias, self.distribution)\n\n module.apply(init)\n if hasattr(module, '_params_init_info'):\n update_init_info(module, init_info=self._get_init_info())\n\n def _get_init_info(self):\n info = f'{self.__class__.__name__}: gain={self.gain}, ' \\\n f'distribution={self.distribution}, bias={self.bias}'\n return info\n\n\n@INITIALIZERS.register_module(name='Normal')\nclass NormalInit(BaseInit):\n r\"\"\"Initialize module parameters with the values drawn from the normal\n distribution :math:`\\mathcal{N}(\\text{mean}, \\text{std}^2)`.\n\n Args:\n mean (int | float):the mean of the normal distribution. Defaults to 0.\n std (int | float): the standard deviation of the normal distribution.\n Defaults to 1.\n bias (int | float): the value to fill the bias. Defaults to 0.\n bias_prob (float, optional): the probability for bias initialization.\n Defaults to None.\n layer (str | list[str], optional): the layer will be initialized.\n Defaults to None.\n\n \"\"\"\n\n def __init__(self, mean=0, std=1, **kwargs):\n super().__init__(**kwargs)\n self.mean = mean\n self.std = std\n\n def __call__(self, module):\n\n def init(m):\n if self.wholemodule:\n normal_init(m, self.mean, self.std, self.bias)\n else:\n layername = m.__class__.__name__\n basesname = _get_bases_name(m)\n if len(set(self.layer) & set([layername] + basesname)):\n normal_init(m, self.mean, self.std, self.bias)\n\n module.apply(init)\n if hasattr(module, '_params_init_info'):\n update_init_info(module, init_info=self._get_init_info())\n\n def _get_init_info(self):\n info = f'{self.__class__.__name__}: mean={self.mean},' \\\n f' std={self.std}, bias={self.bias}'\n return info\n\n\n@INITIALIZERS.register_module(name='TruncNormal')\nclass TruncNormalInit(BaseInit):\n r\"\"\"Initialize module parameters with the values drawn from the normal\n distribution :math:`\\mathcal{N}(\\text{mean}, \\text{std}^2)` with values\n outside :math:`[a, b]`.\n\n Args:\n mean (float): the mean of the normal distribution. Defaults to 0.\n std (float): the standard deviation of the normal distribution.\n Defaults to 1.\n a (float): The minimum cutoff value.\n b ( float): The maximum cutoff value.\n bias (float): the value to fill the bias. Defaults to 0.\n bias_prob (float, optional): the probability for bias initialization.\n Defaults to None.\n layer (str | list[str], optional): the layer will be initialized.\n Defaults to None.\n\n \"\"\"\n\n def __init__(self,\n mean: float = 0,\n std: float = 1,\n a: float = -2,\n b: float = 2,\n **kwargs) -> None:\n super().__init__(**kwargs)\n self.mean = mean\n self.std = std\n self.a = a\n self.b = b\n\n def __call__(self, module: nn.Module) -> None:\n\n def init(m):\n if self.wholemodule:\n trunc_normal_init(m, self.mean, self.std, self.a, self.b,\n self.bias)\n else:\n layername = m.__class__.__name__\n basesname = _get_bases_name(m)\n if len(set(self.layer) & set([layername] + basesname)):\n trunc_normal_init(m, self.mean, self.std, self.a, self.b,\n self.bias)\n\n module.apply(init)\n if hasattr(module, '_params_init_info'):\n update_init_info(module, init_info=self._get_init_info())\n\n def _get_init_info(self):\n info = f'{self.__class__.__name__}: a={self.a}, b={self.b},' \\\n f' mean={self.mean}, std={self.std}, bias={self.bias}'\n return info\n\n\n@INITIALIZERS.register_module(name='Uniform')\nclass UniformInit(BaseInit):\n r\"\"\"Initialize module parameters with values drawn from the uniform\n distribution :math:`\\mathcal{U}(a, b)`.\n\n Args:\n a (int | float): the lower bound of the uniform distribution.\n Defaults to 0.\n b (int | float): the upper bound of the uniform distribution.\n Defaults to 1.\n bias (int | float): the value to fill the bias. Defaults to 0.\n bias_prob (float, optional): the probability for bias initialization.\n Defaults to None.\n layer (str | list[str], optional): the layer will be initialized.\n Defaults to None.\n \"\"\"\n\n def __init__(self, a=0, b=1, **kwargs):\n super().__init__(**kwargs)\n self.a = a\n self.b = b\n\n def __call__(self, module):\n\n def init(m):\n if self.wholemodule:\n uniform_init(m, self.a, self.b, self.bias)\n else:\n layername = m.__class__.__name__\n basesname = _get_bases_name(m)\n if len(set(self.layer) & set([layername] + basesname)):\n uniform_init(m, self.a, self.b, self.bias)\n\n module.apply(init)\n if hasattr(module, '_params_init_info'):\n update_init_info(module, init_info=self._get_init_info())\n\n def _get_init_info(self):\n info = f'{self.__class__.__name__}: a={self.a},' \\\n f' b={self.b}, bias={self.bias}'\n return info\n\n\n@INITIALIZERS.register_module(name='Kaiming')\nclass KaimingInit(BaseInit):\n r\"\"\"Initialize module parameters with the values according to the method\n described in `Delving deep into rectifiers: Surpassing human-level\n performance on ImageNet classification - He, K. et al. (2015).\n `_\n\n Args:\n a (int | float): the negative slope of the rectifier used after this\n layer (only used with ``'leaky_relu'``). Defaults to 0.\n mode (str): either ``'fan_in'`` or ``'fan_out'``. Choosing\n ``'fan_in'`` preserves the magnitude of the variance of the weights\n in the forward pass. Choosing ``'fan_out'`` preserves the\n magnitudes in the backwards pass. Defaults to ``'fan_out'``.\n nonlinearity (str): the non-linear function (`nn.functional` name),\n recommended to use only with ``'relu'`` or ``'leaky_relu'`` .\n Defaults to 'relu'.\n bias (int | float): the value to fill the bias. Defaults to 0.\n bias_prob (float, optional): the probability for bias initialization.\n Defaults to None.\n distribution (str): distribution either be ``'normal'`` or\n ``'uniform'``. Defaults to ``'normal'``.\n layer (str | list[str], optional): the layer will be initialized.\n Defaults to None.\n \"\"\"\n\n def __init__(self,\n a=0,\n mode='fan_out',\n nonlinearity='relu',\n distribution='normal',\n **kwargs):\n super().__init__(**kwargs)\n self.a = a\n self.mode = mode\n self.nonlinearity = nonlinearity\n self.distribution = distribution\n\n def __call__(self, module):\n\n def init(m):\n if self.wholemodule:\n kaiming_init(m, self.a, self.mode, self.nonlinearity,\n self.bias, self.distribution)\n else:\n layername = m.__class__.__name__\n basesname = _get_bases_name(m)\n if len(set(self.layer) & set([layername] + basesname)):\n kaiming_init(m, self.a, self.mode, self.nonlinearity,\n self.bias, self.distribution)\n\n module.apply(init)\n if hasattr(module, '_params_init_info'):\n update_init_info(module, init_info=self._get_init_info())\n\n def _get_init_info(self):\n info = f'{self.__class__.__name__}: a={self.a}, mode={self.mode}, ' \\\n f'nonlinearity={self.nonlinearity}, ' \\\n f'distribution ={self.distribution}, bias={self.bias}'\n return info\n\n\n@INITIALIZERS.register_module(name='Caffe2Xavier')\nclass Caffe2XavierInit(KaimingInit):\n # `XavierFill` in Caffe2 corresponds to `kaiming_uniform_` in PyTorch\n # Acknowledgment to FAIR's internal code\n def __init__(self, **kwargs):\n super().__init__(\n a=1,\n mode='fan_in',\n nonlinearity='leaky_relu',\n distribution='uniform',\n **kwargs)\n\n def __call__(self, module):\n super().__call__(module)\n\n\n@INITIALIZERS.register_module(name='Pretrained')\nclass PretrainedInit(object):\n \"\"\"Initialize module by loading a pretrained model.\n\n Args:\n checkpoint (str): the checkpoint file of the pretrained model should\n be load.\n prefix (str, optional): the prefix of a sub-module in the pretrained\n model. it is for loading a part of the pretrained model to\n initialize. For example, if we would like to only load the\n backbone of a detector model, we can set ``prefix='backbone.'``.\n Defaults to None.\n map_location (str): map tensors into proper locations.\n \"\"\"\n\n def __init__(self, checkpoint, prefix=None, map_location=None):\n self.checkpoint = checkpoint\n self.prefix = prefix\n self.map_location = map_location\n\n def __call__(self, module):\n from mmcv.runner import (_load_checkpoint_with_prefix, load_checkpoint,\n load_state_dict)\n logger = get_logger()\n if self.prefix is None:\n print_log(f'load model from: {self.checkpoint}', logger=logger)\n load_checkpoint(\n module,\n self.checkpoint,\n map_location=self.map_location,\n strict=False,\n logger=logger)\n else:\n print_log(\n f'load {self.prefix} in model from: {self.checkpoint}',\n logger=logger)\n state_dict = _load_checkpoint_with_prefix(\n self.prefix, self.checkpoint, map_location=self.map_location)\n load_state_dict(module, state_dict, strict=False, logger=logger)\n\n if hasattr(module, '_params_init_info'):\n update_init_info(module, init_info=self._get_init_info())\n\n def _get_init_info(self):\n info = f'{self.__class__.__name__}: load from {self.checkpoint}'\n return info\n\n\ndef _initialize(module, cfg, wholemodule=False):\n func = build_from_cfg(cfg, INITIALIZERS)\n # wholemodule flag is for override mode, there is no layer key in override\n # and initializer will give init values for the whole module with the name\n # in override.\n func.wholemodule = wholemodule\n func(module)\n\n\ndef _initialize_override(module, override, cfg):\n if not isinstance(override, (dict, list)):\n raise TypeError(f'override must be a dict or a list of dict, \\\n but got {type(override)}')\n\n override = [override] if isinstance(override, dict) else override\n\n for override_ in override:\n\n cp_override = copy.deepcopy(override_)\n name = cp_override.pop('name', None)\n if name is None:\n raise ValueError('`override` must contain the key \"name\",'\n f'but got {cp_override}')\n # if override only has name key, it means use args in init_cfg\n if not cp_override:\n cp_override.update(cfg)\n # if override has name key and other args except type key, it will\n # raise error\n elif 'type' not in cp_override.keys():\n raise ValueError(\n f'`override` need \"type\" key, but got {cp_override}')\n\n if hasattr(module, name):\n _initialize(getattr(module, name), cp_override, wholemodule=True)\n else:\n raise RuntimeError(f'module did not have attribute {name}, '\n f'but init_cfg is {cp_override}.')\n\n\ndef initialize(module, init_cfg):\n r\"\"\"Initialize a module.\n\n Args:\n module (``torch.nn.Module``): the module will be initialized.\n init_cfg (dict | list[dict]): initialization configuration dict to\n define initializer. OpenMMLab has implemented 6 initializers\n including ``Constant``, ``Xavier``, ``Normal``, ``Uniform``,\n ``Kaiming``, and ``Pretrained``.\n\n Example:\n >>> module = nn.Linear(2, 3, bias=True)\n >>> init_cfg = dict(type='Constant', layer='Linear', val =1 , bias =2)\n >>> initialize(module, init_cfg)\n\n >>> module = nn.Sequential(nn.Conv1d(3, 1, 3), nn.Linear(1,2))\n >>> # define key ``'layer'`` for initializing layer with different\n >>> # configuration\n >>> init_cfg = [dict(type='Constant', layer='Conv1d', val=1),\n dict(type='Constant', layer='Linear', val=2)]\n >>> initialize(module, init_cfg)\n\n >>> # define key``'override'`` to initialize some specific part in\n >>> # module\n >>> class FooNet(nn.Module):\n >>> def __init__(self):\n >>> super().__init__()\n >>> self.feat = nn.Conv2d(3, 16, 3)\n >>> self.reg = nn.Conv2d(16, 10, 3)\n >>> self.cls = nn.Conv2d(16, 5, 3)\n >>> model = FooNet()\n >>> init_cfg = dict(type='Constant', val=1, bias=2, layer='Conv2d',\n >>> override=dict(type='Constant', name='reg', val=3, bias=4))\n >>> initialize(model, init_cfg)\n\n >>> model = ResNet(depth=50)\n >>> # Initialize weights with the pretrained model.\n >>> init_cfg = dict(type='Pretrained',\n checkpoint='torchvision://resnet50')\n >>> initialize(model, init_cfg)\n\n >>> # Initialize weights of a sub-module with the specific part of\n >>> # a pretrained model by using \"prefix\".\n >>> url = 'http://download.openmmlab.com/mmdetection/v2.0/retinanet/'\\\n >>> 'retinanet_r50_fpn_1x_coco/'\\\n >>> 'retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth'\n >>> init_cfg = dict(type='Pretrained',\n checkpoint=url, prefix='backbone.')\n \"\"\"\n if not isinstance(init_cfg, (dict, list)):\n raise TypeError(f'init_cfg must be a dict or a list of dict, \\\n but got {type(init_cfg)}')\n\n if isinstance(init_cfg, dict):\n init_cfg = [init_cfg]\n\n for cfg in init_cfg:\n # should deeply copy the original config because cfg may be used by\n # other modules, e.g., one init_cfg shared by multiple bottleneck\n # blocks, the expected cfg will be changed after pop and will change\n # the initialization behavior of other modules\n cp_cfg = copy.deepcopy(cfg)\n override = cp_cfg.pop('override', None)\n _initialize(module, cp_cfg)\n\n if override is not None:\n cp_cfg.pop('layer', None)\n _initialize_override(module, override, cp_cfg)\n else:\n # All attributes in module have same initialization.\n pass\n\n\ndef _no_grad_trunc_normal_(tensor: Tensor, mean: float, std: float, a: float,\n b: float) -> Tensor:\n # Method based on\n # https://people.sc.fsu.edu/~jburkardt/presentations/truncated_normal.pdf\n # Modified from\n # https://github.com/pytorch/pytorch/blob/master/torch/nn/init.py\n def norm_cdf(x):\n # Computes standard normal cumulative distribution function\n return (1. + math.erf(x / math.sqrt(2.))) / 2.\n\n if (mean < a - 2 * std) or (mean > b + 2 * std):\n warnings.warn(\n 'mean is more than 2 std from [a, b] in nn.init.trunc_normal_. '\n 'The distribution of values may be incorrect.',\n stacklevel=2)\n\n with torch.no_grad():\n # Values are generated by using a truncated uniform distribution and\n # then using the inverse CDF for the normal distribution.\n # Get upper and lower cdf values\n lower = norm_cdf((a - mean) / std)\n upper = norm_cdf((b - mean) / std)\n\n # Uniformly fill tensor with values from [lower, upper], then translate\n # to [2lower-1, 2upper-1].\n tensor.uniform_(2 * lower - 1, 2 * upper - 1)\n\n # Use inverse cdf transform for normal distribution to get truncated\n # standard normal\n tensor.erfinv_()\n\n # Transform to proper mean, std\n tensor.mul_(std * math.sqrt(2.))\n tensor.add_(mean)\n\n # Clamp to ensure it's in the proper range\n tensor.clamp_(min=a, max=b)\n return tensor\n\n\ndef trunc_normal_(tensor: Tensor,\n mean: float = 0.,\n std: float = 1.,\n a: float = -2.,\n b: float = 2.) -> Tensor:\n r\"\"\"Fills the input Tensor with values drawn from a truncated\n normal distribution. The values are effectively drawn from the\n normal distribution :math:`\\mathcal{N}(\\text{mean}, \\text{std}^2)`\n with values outside :math:`[a, b]` redrawn until they are within\n the bounds. The method used for generating the random values works\n best when :math:`a \\leq \\text{mean} \\leq b`.\n\n Modified from\n https://github.com/pytorch/pytorch/blob/master/torch/nn/init.py\n\n Args:\n tensor (``torch.Tensor``): an n-dimensional `torch.Tensor`.\n mean (float): the mean of the normal distribution.\n std (float): the standard deviation of the normal distribution.\n a (float): the minimum cutoff value.\n b (float): the maximum cutoff value.\n \"\"\"\n return _no_grad_trunc_normal_(tensor, mean, std, a, b)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/cnn/vgg.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport logging\n\nimport torch.nn as nn\n\nfrom .utils import constant_init, kaiming_init, normal_init\n\n\ndef conv3x3(in_planes, out_planes, dilation=1):\n \"\"\"3x3 convolution with padding.\"\"\"\n return nn.Conv2d(\n in_planes,\n out_planes,\n kernel_size=3,\n padding=dilation,\n dilation=dilation)\n\n\ndef make_vgg_layer(inplanes,\n planes,\n num_blocks,\n dilation=1,\n with_bn=False,\n ceil_mode=False):\n layers = []\n for _ in range(num_blocks):\n layers.append(conv3x3(inplanes, planes, dilation))\n if with_bn:\n layers.append(nn.BatchNorm2d(planes))\n layers.append(nn.ReLU(inplace=True))\n inplanes = planes\n layers.append(nn.MaxPool2d(kernel_size=2, stride=2, ceil_mode=ceil_mode))\n\n return layers\n\n\nclass VGG(nn.Module):\n \"\"\"VGG backbone.\n\n Args:\n depth (int): Depth of vgg, from {11, 13, 16, 19}.\n with_bn (bool): Use BatchNorm or not.\n num_classes (int): number of classes for classification.\n num_stages (int): VGG stages, normally 5.\n dilations (Sequence[int]): Dilation of each stage.\n out_indices (Sequence[int]): Output from which stages.\n frozen_stages (int): Stages to be frozen (all param fixed). -1 means\n not freezing any parameters.\n bn_eval (bool): Whether to set BN layers as eval mode, namely, freeze\n running stats (mean and var).\n bn_frozen (bool): Whether to freeze weight and bias of BN layers.\n \"\"\"\n\n arch_settings = {\n 11: (1, 1, 2, 2, 2),\n 13: (2, 2, 2, 2, 2),\n 16: (2, 2, 3, 3, 3),\n 19: (2, 2, 4, 4, 4)\n }\n\n def __init__(self,\n depth,\n with_bn=False,\n num_classes=-1,\n num_stages=5,\n dilations=(1, 1, 1, 1, 1),\n out_indices=(0, 1, 2, 3, 4),\n frozen_stages=-1,\n bn_eval=True,\n bn_frozen=False,\n ceil_mode=False,\n with_last_pool=True):\n super(VGG, self).__init__()\n if depth not in self.arch_settings:\n raise KeyError(f'invalid depth {depth} for vgg')\n assert num_stages >= 1 and num_stages <= 5\n stage_blocks = self.arch_settings[depth]\n self.stage_blocks = stage_blocks[:num_stages]\n assert len(dilations) == num_stages\n assert max(out_indices) <= num_stages\n\n self.num_classes = num_classes\n self.out_indices = out_indices\n self.frozen_stages = frozen_stages\n self.bn_eval = bn_eval\n self.bn_frozen = bn_frozen\n\n self.inplanes = 3\n start_idx = 0\n vgg_layers = []\n self.range_sub_modules = []\n for i, num_blocks in enumerate(self.stage_blocks):\n num_modules = num_blocks * (2 + with_bn) + 1\n end_idx = start_idx + num_modules\n dilation = dilations[i]\n planes = 64 * 2**i if i < 4 else 512\n vgg_layer = make_vgg_layer(\n self.inplanes,\n planes,\n num_blocks,\n dilation=dilation,\n with_bn=with_bn,\n ceil_mode=ceil_mode)\n vgg_layers.extend(vgg_layer)\n self.inplanes = planes\n self.range_sub_modules.append([start_idx, end_idx])\n start_idx = end_idx\n if not with_last_pool:\n vgg_layers.pop(-1)\n self.range_sub_modules[-1][1] -= 1\n self.module_name = 'features'\n self.add_module(self.module_name, nn.Sequential(*vgg_layers))\n\n if self.num_classes > 0:\n self.classifier = nn.Sequential(\n nn.Linear(512 * 7 * 7, 4096),\n nn.ReLU(True),\n nn.Dropout(),\n nn.Linear(4096, 4096),\n nn.ReLU(True),\n nn.Dropout(),\n nn.Linear(4096, num_classes),\n )\n\n def init_weights(self, pretrained=None):\n if isinstance(pretrained, str):\n logger = logging.getLogger()\n from ..runner import load_checkpoint\n load_checkpoint(self, pretrained, strict=False, logger=logger)\n elif pretrained is None:\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n kaiming_init(m)\n elif isinstance(m, nn.BatchNorm2d):\n constant_init(m, 1)\n elif isinstance(m, nn.Linear):\n normal_init(m, std=0.01)\n else:\n raise TypeError('pretrained must be a str or None')\n\n def forward(self, x):\n outs = []\n vgg_layers = getattr(self, self.module_name)\n for i in range(len(self.stage_blocks)):\n for j in range(*self.range_sub_modules[i]):\n vgg_layer = vgg_layers[j]\n x = vgg_layer(x)\n if i in self.out_indices:\n outs.append(x)\n if self.num_classes > 0:\n x = x.view(x.size(0), -1)\n x = self.classifier(x)\n outs.append(x)\n if len(outs) == 1:\n return outs[0]\n else:\n return tuple(outs)\n\n def train(self, mode=True):\n super(VGG, self).train(mode)\n if self.bn_eval:\n for m in self.modules():\n if isinstance(m, nn.BatchNorm2d):\n m.eval()\n if self.bn_frozen:\n for params in m.parameters():\n params.requires_grad = False\n vgg_layers = getattr(self, self.module_name)\n if mode and self.frozen_stages >= 0:\n for i in range(self.frozen_stages):\n for j in range(*self.range_sub_modules[i]):\n mod = vgg_layers[j]\n mod.eval()\n for param in mod.parameters():\n param.requires_grad = False\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/engine/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .test import (collect_results_cpu, collect_results_gpu, multi_gpu_test,\n single_gpu_test)\n\n__all__ = [\n 'collect_results_cpu', 'collect_results_gpu', 'multi_gpu_test',\n 'single_gpu_test'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/engine/test.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\nimport pickle\nimport shutil\nimport tempfile\nimport time\n\nimport torch\nimport torch.distributed as dist\n\nimport mmcv\nfrom mmcv.runner import get_dist_info\n\n\ndef single_gpu_test(model, data_loader):\n \"\"\"Test model with a single gpu.\n\n This method tests model with a single gpu and displays test progress bar.\n\n Args:\n model (nn.Module): Model to be tested.\n data_loader (nn.Dataloader): Pytorch data loader.\n\n Returns:\n list: The prediction results.\n \"\"\"\n model.eval()\n results = []\n dataset = data_loader.dataset\n prog_bar = mmcv.ProgressBar(len(dataset))\n for data in data_loader:\n with torch.no_grad():\n result = model(return_loss=False, **data)\n results.extend(result)\n\n # Assume result has the same length of batch_size\n # refer to https://github.com/open-mmlab/mmcv/issues/985\n batch_size = len(result)\n for _ in range(batch_size):\n prog_bar.update()\n return results\n\n\ndef multi_gpu_test(model, data_loader, tmpdir=None, gpu_collect=False):\n \"\"\"Test model with multiple gpus.\n\n This method tests model with multiple gpus and collects the results\n under two different modes: gpu and cpu modes. By setting\n ``gpu_collect=True``, it encodes results to gpu tensors and use gpu\n communication for results collection. On cpu mode it saves the results on\n different gpus to ``tmpdir`` and collects them by the rank 0 worker.\n\n Args:\n model (nn.Module): Model to be tested.\n data_loader (nn.Dataloader): Pytorch data loader.\n tmpdir (str): Path of directory to save the temporary results from\n different gpus under cpu mode.\n gpu_collect (bool): Option to use either gpu or cpu to collect results.\n\n Returns:\n list: The prediction results.\n \"\"\"\n model.eval()\n results = []\n dataset = data_loader.dataset\n rank, world_size = get_dist_info()\n if rank == 0:\n prog_bar = mmcv.ProgressBar(len(dataset))\n time.sleep(2) # This line can prevent deadlock problem in some cases.\n for i, data in enumerate(data_loader):\n with torch.no_grad():\n result = model(return_loss=False, **data)\n results.extend(result)\n\n if rank == 0:\n batch_size = len(result)\n batch_size_all = batch_size * world_size\n if batch_size_all + prog_bar.completed > len(dataset):\n batch_size_all = len(dataset) - prog_bar.completed\n for _ in range(batch_size_all):\n prog_bar.update()\n\n # collect results from all ranks\n if gpu_collect:\n results = collect_results_gpu(results, len(dataset))\n else:\n results = collect_results_cpu(results, len(dataset), tmpdir)\n return results\n\n\ndef collect_results_cpu(result_part, size, tmpdir=None):\n \"\"\"Collect results under cpu mode.\n\n On cpu mode, this function will save the results on different gpus to\n ``tmpdir`` and collect them by the rank 0 worker.\n\n Args:\n result_part (list): Result list containing result parts\n to be collected.\n size (int): Size of the results, commonly equal to length of\n the results.\n tmpdir (str | None): temporal directory for collected results to\n store. If set to None, it will create a random temporal directory\n for it.\n\n Returns:\n list: The collected results.\n \"\"\"\n rank, world_size = get_dist_info()\n # create a tmp dir if it is not specified\n if tmpdir is None:\n MAX_LEN = 512\n # 32 is whitespace\n dir_tensor = torch.full((MAX_LEN, ),\n 32,\n dtype=torch.uint8,\n device='cuda')\n if rank == 0:\n mmcv.mkdir_or_exist('.dist_test')\n tmpdir = tempfile.mkdtemp(dir='.dist_test')\n tmpdir = torch.tensor(\n bytearray(tmpdir.encode()), dtype=torch.uint8, device='cuda')\n dir_tensor[:len(tmpdir)] = tmpdir\n dist.broadcast(dir_tensor, 0)\n tmpdir = dir_tensor.cpu().numpy().tobytes().decode().rstrip()\n else:\n mmcv.mkdir_or_exist(tmpdir)\n # dump the part result to the dir\n mmcv.dump(result_part, osp.join(tmpdir, f'part_{rank}.pkl'))\n dist.barrier()\n # collect all parts\n if rank != 0:\n return None\n else:\n # load results of all parts from tmp dir\n part_list = []\n for i in range(world_size):\n part_file = osp.join(tmpdir, f'part_{i}.pkl')\n part_result = mmcv.load(part_file)\n # When data is severely insufficient, an empty part_result\n # on a certain gpu could makes the overall outputs empty.\n if part_result:\n part_list.append(part_result)\n # sort the results\n ordered_results = []\n for res in zip(*part_list):\n ordered_results.extend(list(res))\n # the dataloader may pad some samples\n ordered_results = ordered_results[:size]\n # remove tmp dir\n shutil.rmtree(tmpdir)\n return ordered_results\n\n\ndef collect_results_gpu(result_part, size):\n \"\"\"Collect results under gpu mode.\n\n On gpu mode, this function will encode results to gpu tensors and use gpu\n communication for results collection.\n\n Args:\n result_part (list): Result list containing result parts\n to be collected.\n size (int): Size of the results, commonly equal to length of\n the results.\n\n Returns:\n list: The collected results.\n \"\"\"\n rank, world_size = get_dist_info()\n # dump result part to tensor with pickle\n part_tensor = torch.tensor(\n bytearray(pickle.dumps(result_part)), dtype=torch.uint8, device='cuda')\n # gather all result part tensor shape\n shape_tensor = torch.tensor(part_tensor.shape, device='cuda')\n shape_list = [shape_tensor.clone() for _ in range(world_size)]\n dist.all_gather(shape_list, shape_tensor)\n # padding result part tensor to max length\n shape_max = torch.tensor(shape_list).max()\n part_send = torch.zeros(shape_max, dtype=torch.uint8, device='cuda')\n part_send[:shape_tensor[0]] = part_tensor\n part_recv_list = [\n part_tensor.new_zeros(shape_max) for _ in range(world_size)\n ]\n # gather all result part\n dist.all_gather(part_recv_list, part_send)\n\n if rank == 0:\n part_list = []\n for recv, shape in zip(part_recv_list, shape_list):\n part_result = pickle.loads(recv[:shape[0]].cpu().numpy().tobytes())\n # When data is severely insufficient, an empty part_result\n # on a certain gpu could makes the overall outputs empty.\n if part_result:\n part_list.append(part_result)\n # sort the results\n ordered_results = []\n for res in zip(*part_list):\n ordered_results.extend(list(res))\n # the dataloader may pad some samples\n ordered_results = ordered_results[:size]\n return ordered_results\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .file_client import BaseStorageBackend, FileClient\nfrom .handlers import BaseFileHandler, JsonHandler, PickleHandler, YamlHandler\nfrom .io import dump, load, register_handler\nfrom .parse import dict_from_file, list_from_file\n\n__all__ = [\n 'BaseStorageBackend', 'FileClient', 'load', 'dump', 'register_handler',\n 'BaseFileHandler', 'JsonHandler', 'PickleHandler', 'YamlHandler',\n 'list_from_file', 'dict_from_file'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/file_client.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport inspect\nimport os\nimport os.path as osp\nimport re\nimport tempfile\nimport warnings\nfrom abc import ABCMeta, abstractmethod\nfrom contextlib import contextmanager\nfrom pathlib import Path\nfrom typing import Iterable, Iterator, Optional, Tuple, Union\nfrom urllib.request import urlopen\n\nimport mmcv\nfrom mmcv.utils.misc import has_method\nfrom mmcv.utils.path import is_filepath\n\n\nclass BaseStorageBackend(metaclass=ABCMeta):\n \"\"\"Abstract class of storage backends.\n\n All backends need to implement two apis: ``get()`` and ``get_text()``.\n ``get()`` reads the file as a byte stream and ``get_text()`` reads the file\n as texts.\n \"\"\"\n\n # a flag to indicate whether the backend can create a symlink for a file\n _allow_symlink = False\n\n @property\n def name(self):\n return self.__class__.__name__\n\n @property\n def allow_symlink(self):\n return self._allow_symlink\n\n @abstractmethod\n def get(self, filepath):\n pass\n\n @abstractmethod\n def get_text(self, filepath):\n pass\n\n\nclass CephBackend(BaseStorageBackend):\n \"\"\"Ceph storage backend (for internal use).\n\n Args:\n path_mapping (dict|None): path mapping dict from local path to Petrel\n path. When ``path_mapping={'src': 'dst'}``, ``src`` in ``filepath``\n will be replaced by ``dst``. Default: None.\n\n .. warning::\n :class:`mmcv.fileio.file_client.CephBackend` will be deprecated,\n please use :class:`mmcv.fileio.file_client.PetrelBackend` instead.\n \"\"\"\n\n def __init__(self, path_mapping=None):\n try:\n import ceph\n except ImportError:\n raise ImportError('Please install ceph to enable CephBackend.')\n\n warnings.warn(\n 'CephBackend will be deprecated, please use PetrelBackend instead',\n DeprecationWarning)\n self._client = ceph.S3Client()\n assert isinstance(path_mapping, dict) or path_mapping is None\n self.path_mapping = path_mapping\n\n def get(self, filepath):\n filepath = str(filepath)\n if self.path_mapping is not None:\n for k, v in self.path_mapping.items():\n filepath = filepath.replace(k, v)\n value = self._client.Get(filepath)\n value_buf = memoryview(value)\n return value_buf\n\n def get_text(self, filepath, encoding=None):\n raise NotImplementedError\n\n\nclass PetrelBackend(BaseStorageBackend):\n \"\"\"Petrel storage backend (for internal use).\n\n PetrelBackend supports reading and writing data to multiple clusters.\n If the file path contains the cluster name, PetrelBackend will read data\n from specified cluster or write data to it. Otherwise, PetrelBackend will\n access the default cluster.\n\n Args:\n path_mapping (dict, optional): Path mapping dict from local path to\n Petrel path. When ``path_mapping={'src': 'dst'}``, ``src`` in\n ``filepath`` will be replaced by ``dst``. Default: None.\n enable_mc (bool, optional): Whether to enable memcached support.\n Default: True.\n\n Examples:\n >>> filepath1 = 's3://path/of/file'\n >>> filepath2 = 'cluster-name:s3://path/of/file'\n >>> client = PetrelBackend()\n >>> client.get(filepath1) # get data from default cluster\n >>> client.get(filepath2) # get data from 'cluster-name' cluster\n \"\"\"\n\n def __init__(self,\n path_mapping: Optional[dict] = None,\n enable_mc: bool = True):\n try:\n from petrel_client import client\n except ImportError:\n raise ImportError('Please install petrel_client to enable '\n 'PetrelBackend.')\n\n self._client = client.Client(enable_mc=enable_mc)\n assert isinstance(path_mapping, dict) or path_mapping is None\n self.path_mapping = path_mapping\n\n def _map_path(self, filepath: Union[str, Path]) -> str:\n \"\"\"Map ``filepath`` to a string path whose prefix will be replaced by\n :attr:`self.path_mapping`.\n\n Args:\n filepath (str): Path to be mapped.\n \"\"\"\n filepath = str(filepath)\n if self.path_mapping is not None:\n for k, v in self.path_mapping.items():\n filepath = filepath.replace(k, v)\n return filepath\n\n def _format_path(self, filepath: str) -> str:\n \"\"\"Convert a ``filepath`` to standard format of petrel oss.\n\n If the ``filepath`` is concatenated by ``os.path.join``, in a Windows\n environment, the ``filepath`` will be the format of\n 's3://bucket_name\\\\image.jpg'. By invoking :meth:`_format_path`, the\n above ``filepath`` will be converted to 's3://bucket_name/image.jpg'.\n\n Args:\n filepath (str): Path to be formatted.\n \"\"\"\n return re.sub(r'\\\\+', '/', filepath)\n\n def get(self, filepath: Union[str, Path]) -> memoryview:\n \"\"\"Read data from a given ``filepath`` with 'rb' mode.\n\n Args:\n filepath (str or Path): Path to read data.\n\n Returns:\n memoryview: A memory view of expected bytes object to avoid\n copying. The memoryview object can be converted to bytes by\n ``value_buf.tobytes()``.\n \"\"\"\n filepath = self._map_path(filepath)\n filepath = self._format_path(filepath)\n value = self._client.Get(filepath)\n value_buf = memoryview(value)\n return value_buf\n\n def get_text(self,\n filepath: Union[str, Path],\n encoding: str = 'utf-8') -> str:\n \"\"\"Read data from a given ``filepath`` with 'r' mode.\n\n Args:\n filepath (str or Path): Path to read data.\n encoding (str): The encoding format used to open the ``filepath``.\n Default: 'utf-8'.\n\n Returns:\n str: Expected text reading from ``filepath``.\n \"\"\"\n return str(self.get(filepath), encoding=encoding)\n\n def put(self, obj: bytes, filepath: Union[str, Path]) -> None:\n \"\"\"Save data to a given ``filepath``.\n\n Args:\n obj (bytes): Data to be saved.\n filepath (str or Path): Path to write data.\n \"\"\"\n filepath = self._map_path(filepath)\n filepath = self._format_path(filepath)\n self._client.put(filepath, obj)\n\n def put_text(self,\n obj: str,\n filepath: Union[str, Path],\n encoding: str = 'utf-8') -> None:\n \"\"\"Save data to a given ``filepath``.\n\n Args:\n obj (str): Data to be written.\n filepath (str or Path): Path to write data.\n encoding (str): The encoding format used to encode the ``obj``.\n Default: 'utf-8'.\n \"\"\"\n self.put(bytes(obj, encoding=encoding), filepath)\n\n def remove(self, filepath: Union[str, Path]) -> None:\n \"\"\"Remove a file.\n\n Args:\n filepath (str or Path): Path to be removed.\n \"\"\"\n if not has_method(self._client, 'delete'):\n raise NotImplementedError(\n ('Current version of Petrel Python SDK has not supported '\n 'the `delete` method, please use a higher version or dev'\n ' branch instead.'))\n\n filepath = self._map_path(filepath)\n filepath = self._format_path(filepath)\n self._client.delete(filepath)\n\n def exists(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path exists.\n\n Args:\n filepath (str or Path): Path to be checked whether exists.\n\n Returns:\n bool: Return ``True`` if ``filepath`` exists, ``False`` otherwise.\n \"\"\"\n if not (has_method(self._client, 'contains')\n and has_method(self._client, 'isdir')):\n raise NotImplementedError(\n ('Current version of Petrel Python SDK has not supported '\n 'the `contains` and `isdir` methods, please use a higher'\n 'version or dev branch instead.'))\n\n filepath = self._map_path(filepath)\n filepath = self._format_path(filepath)\n return self._client.contains(filepath) or self._client.isdir(filepath)\n\n def isdir(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path is a directory.\n\n Args:\n filepath (str or Path): Path to be checked whether it is a\n directory.\n\n Returns:\n bool: Return ``True`` if ``filepath`` points to a directory,\n ``False`` otherwise.\n \"\"\"\n if not has_method(self._client, 'isdir'):\n raise NotImplementedError(\n ('Current version of Petrel Python SDK has not supported '\n 'the `isdir` method, please use a higher version or dev'\n ' branch instead.'))\n\n filepath = self._map_path(filepath)\n filepath = self._format_path(filepath)\n return self._client.isdir(filepath)\n\n def isfile(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path is a file.\n\n Args:\n filepath (str or Path): Path to be checked whether it is a file.\n\n Returns:\n bool: Return ``True`` if ``filepath`` points to a file, ``False``\n otherwise.\n \"\"\"\n if not has_method(self._client, 'contains'):\n raise NotImplementedError(\n ('Current version of Petrel Python SDK has not supported '\n 'the `contains` method, please use a higher version or '\n 'dev branch instead.'))\n\n filepath = self._map_path(filepath)\n filepath = self._format_path(filepath)\n return self._client.contains(filepath)\n\n def join_path(self, filepath: Union[str, Path],\n *filepaths: Union[str, Path]) -> str:\n \"\"\"Concatenate all file paths.\n\n Args:\n filepath (str or Path): Path to be concatenated.\n\n Returns:\n str: The result after concatenation.\n \"\"\"\n filepath = self._format_path(self._map_path(filepath))\n if filepath.endswith('/'):\n filepath = filepath[:-1]\n formatted_paths = [filepath]\n for path in filepaths:\n formatted_paths.append(self._format_path(self._map_path(path)))\n return '/'.join(formatted_paths)\n\n @contextmanager\n def get_local_path(self, filepath: Union[str, Path]) -> Iterable[str]:\n \"\"\"Download a file from ``filepath`` and return a temporary path.\n\n ``get_local_path`` is decorated by :meth:`contxtlib.contextmanager`. It\n can be called with ``with`` statement, and when exists from the\n ``with`` statement, the temporary path will be released.\n\n Args:\n filepath (str | Path): Download a file from ``filepath``.\n\n Examples:\n >>> client = PetrelBackend()\n >>> # After existing from the ``with`` clause,\n >>> # the path will be removed\n >>> with client.get_local_path('s3://path/of/your/file') as path:\n ... # do something here\n\n Yields:\n Iterable[str]: Only yield one temporary path.\n \"\"\"\n filepath = self._map_path(filepath)\n filepath = self._format_path(filepath)\n assert self.isfile(filepath)\n try:\n f = tempfile.NamedTemporaryFile(delete=False)\n f.write(self.get(filepath))\n f.close()\n yield f.name\n finally:\n os.remove(f.name)\n\n def list_dir_or_file(self,\n dir_path: Union[str, Path],\n list_dir: bool = True,\n list_file: bool = True,\n suffix: Optional[Union[str, Tuple[str]]] = None,\n recursive: bool = False) -> Iterator[str]:\n \"\"\"Scan a directory to find the interested directories or files in\n arbitrary order.\n\n Note:\n Petrel has no concept of directories but it simulates the directory\n hierarchy in the filesystem through public prefixes. In addition,\n if the returned path ends with '/', it means the path is a public\n prefix which is a logical directory.\n\n Note:\n :meth:`list_dir_or_file` returns the path relative to ``dir_path``.\n In addition, the returned path of directory will not contains the\n suffix '/' which is consistent with other backends.\n\n Args:\n dir_path (str | Path): Path of the directory.\n list_dir (bool): List the directories. Default: True.\n list_file (bool): List the path of files. Default: True.\n suffix (str or tuple[str], optional): File suffix\n that we are interested in. Default: None.\n recursive (bool): If set to True, recursively scan the\n directory. Default: False.\n\n Yields:\n Iterable[str]: A relative path to ``dir_path``.\n \"\"\"\n if not has_method(self._client, 'list'):\n raise NotImplementedError(\n ('Current version of Petrel Python SDK has not supported '\n 'the `list` method, please use a higher version or dev'\n ' branch instead.'))\n\n dir_path = self._map_path(dir_path)\n dir_path = self._format_path(dir_path)\n if list_dir and suffix is not None:\n raise TypeError(\n '`list_dir` should be False when `suffix` is not None')\n\n if (suffix is not None) and not isinstance(suffix, (str, tuple)):\n raise TypeError('`suffix` must be a string or tuple of strings')\n\n # Petrel's simulated directory hierarchy assumes that directory paths\n # should end with `/`\n if not dir_path.endswith('/'):\n dir_path += '/'\n\n root = dir_path\n\n def _list_dir_or_file(dir_path, list_dir, list_file, suffix,\n recursive):\n for path in self._client.list(dir_path):\n # the `self.isdir` is not used here to determine whether path\n # is a directory, because `self.isdir` relies on\n # `self._client.list`\n if path.endswith('/'): # a directory path\n next_dir_path = self.join_path(dir_path, path)\n if list_dir:\n # get the relative path and exclude the last\n # character '/'\n rel_dir = next_dir_path[len(root):-1]\n yield rel_dir\n if recursive:\n yield from _list_dir_or_file(next_dir_path, list_dir,\n list_file, suffix,\n recursive)\n else: # a file path\n absolute_path = self.join_path(dir_path, path)\n rel_path = absolute_path[len(root):]\n if (suffix is None\n or rel_path.endswith(suffix)) and list_file:\n yield rel_path\n\n return _list_dir_or_file(dir_path, list_dir, list_file, suffix,\n recursive)\n\n\nclass MemcachedBackend(BaseStorageBackend):\n \"\"\"Memcached storage backend.\n\n Attributes:\n server_list_cfg (str): Config file for memcached server list.\n client_cfg (str): Config file for memcached client.\n sys_path (str | None): Additional path to be appended to `sys.path`.\n Default: None.\n \"\"\"\n\n def __init__(self, server_list_cfg, client_cfg, sys_path=None):\n if sys_path is not None:\n import sys\n sys.path.append(sys_path)\n try:\n import mc\n except ImportError:\n raise ImportError(\n 'Please install memcached to enable MemcachedBackend.')\n\n self.server_list_cfg = server_list_cfg\n self.client_cfg = client_cfg\n self._client = mc.MemcachedClient.GetInstance(self.server_list_cfg,\n self.client_cfg)\n # mc.pyvector servers as a point which points to a memory cache\n self._mc_buffer = mc.pyvector()\n\n def get(self, filepath):\n filepath = str(filepath)\n import mc\n self._client.Get(filepath, self._mc_buffer)\n value_buf = mc.ConvertBuffer(self._mc_buffer)\n return value_buf\n\n def get_text(self, filepath, encoding=None):\n raise NotImplementedError\n\n\nclass LmdbBackend(BaseStorageBackend):\n \"\"\"Lmdb storage backend.\n\n Args:\n db_path (str): Lmdb database path.\n readonly (bool, optional): Lmdb environment parameter. If True,\n disallow any write operations. Default: True.\n lock (bool, optional): Lmdb environment parameter. If False, when\n concurrent access occurs, do not lock the database. Default: False.\n readahead (bool, optional): Lmdb environment parameter. If False,\n disable the OS filesystem readahead mechanism, which may improve\n random read performance when a database is larger than RAM.\n Default: False.\n\n Attributes:\n db_path (str): Lmdb database path.\n \"\"\"\n\n def __init__(self,\n db_path,\n readonly=True,\n lock=False,\n readahead=False,\n **kwargs):\n try:\n import lmdb\n except ImportError:\n raise ImportError('Please install lmdb to enable LmdbBackend.')\n\n self.db_path = str(db_path)\n self._client = lmdb.open(\n self.db_path,\n readonly=readonly,\n lock=lock,\n readahead=readahead,\n **kwargs)\n\n def get(self, filepath):\n \"\"\"Get values according to the filepath.\n\n Args:\n filepath (str | obj:`Path`): Here, filepath is the lmdb key.\n \"\"\"\n filepath = str(filepath)\n with self._client.begin(write=False) as txn:\n value_buf = txn.get(filepath.encode('ascii'))\n return value_buf\n\n def get_text(self, filepath, encoding=None):\n raise NotImplementedError\n\n\nclass HardDiskBackend(BaseStorageBackend):\n \"\"\"Raw hard disks storage backend.\"\"\"\n\n _allow_symlink = True\n\n def get(self, filepath: Union[str, Path]) -> bytes:\n \"\"\"Read data from a given ``filepath`` with 'rb' mode.\n\n Args:\n filepath (str or Path): Path to read data.\n\n Returns:\n bytes: Expected bytes object.\n \"\"\"\n with open(filepath, 'rb') as f:\n value_buf = f.read()\n return value_buf\n\n def get_text(self,\n filepath: Union[str, Path],\n encoding: str = 'utf-8') -> str:\n \"\"\"Read data from a given ``filepath`` with 'r' mode.\n\n Args:\n filepath (str or Path): Path to read data.\n encoding (str): The encoding format used to open the ``filepath``.\n Default: 'utf-8'.\n\n Returns:\n str: Expected text reading from ``filepath``.\n \"\"\"\n with open(filepath, 'r', encoding=encoding) as f:\n value_buf = f.read()\n return value_buf\n\n def put(self, obj: bytes, filepath: Union[str, Path]) -> None:\n \"\"\"Write data to a given ``filepath`` with 'wb' mode.\n\n Note:\n ``put`` will create a directory if the directory of ``filepath``\n does not exist.\n\n Args:\n obj (bytes): Data to be written.\n filepath (str or Path): Path to write data.\n \"\"\"\n mmcv.mkdir_or_exist(osp.dirname(filepath))\n with open(filepath, 'wb') as f:\n f.write(obj)\n\n def put_text(self,\n obj: str,\n filepath: Union[str, Path],\n encoding: str = 'utf-8') -> None:\n \"\"\"Write data to a given ``filepath`` with 'w' mode.\n\n Note:\n ``put_text`` will create a directory if the directory of\n ``filepath`` does not exist.\n\n Args:\n obj (str): Data to be written.\n filepath (str or Path): Path to write data.\n encoding (str): The encoding format used to open the ``filepath``.\n Default: 'utf-8'.\n \"\"\"\n mmcv.mkdir_or_exist(osp.dirname(filepath))\n with open(filepath, 'w', encoding=encoding) as f:\n f.write(obj)\n\n def remove(self, filepath: Union[str, Path]) -> None:\n \"\"\"Remove a file.\n\n Args:\n filepath (str or Path): Path to be removed.\n \"\"\"\n os.remove(filepath)\n\n def exists(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path exists.\n\n Args:\n filepath (str or Path): Path to be checked whether exists.\n\n Returns:\n bool: Return ``True`` if ``filepath`` exists, ``False`` otherwise.\n \"\"\"\n return osp.exists(filepath)\n\n def isdir(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path is a directory.\n\n Args:\n filepath (str or Path): Path to be checked whether it is a\n directory.\n\n Returns:\n bool: Return ``True`` if ``filepath`` points to a directory,\n ``False`` otherwise.\n \"\"\"\n return osp.isdir(filepath)\n\n def isfile(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path is a file.\n\n Args:\n filepath (str or Path): Path to be checked whether it is a file.\n\n Returns:\n bool: Return ``True`` if ``filepath`` points to a file, ``False``\n otherwise.\n \"\"\"\n return osp.isfile(filepath)\n\n def join_path(self, filepath: Union[str, Path],\n *filepaths: Union[str, Path]) -> str:\n \"\"\"Concatenate all file paths.\n\n Join one or more filepath components intelligently. The return value\n is the concatenation of filepath and any members of *filepaths.\n\n Args:\n filepath (str or Path): Path to be concatenated.\n\n Returns:\n str: The result of concatenation.\n \"\"\"\n return osp.join(filepath, *filepaths)\n\n @contextmanager\n def get_local_path(\n self, filepath: Union[str, Path]) -> Iterable[Union[str, Path]]:\n \"\"\"Only for unified API and do nothing.\"\"\"\n yield filepath\n\n def list_dir_or_file(self,\n dir_path: Union[str, Path],\n list_dir: bool = True,\n list_file: bool = True,\n suffix: Optional[Union[str, Tuple[str]]] = None,\n recursive: bool = False) -> Iterator[str]:\n \"\"\"Scan a directory to find the interested directories or files in\n arbitrary order.\n\n Note:\n :meth:`list_dir_or_file` returns the path relative to ``dir_path``.\n\n Args:\n dir_path (str | Path): Path of the directory.\n list_dir (bool): List the directories. Default: True.\n list_file (bool): List the path of files. Default: True.\n suffix (str or tuple[str], optional): File suffix\n that we are interested in. Default: None.\n recursive (bool): If set to True, recursively scan the\n directory. Default: False.\n\n Yields:\n Iterable[str]: A relative path to ``dir_path``.\n \"\"\"\n if list_dir and suffix is not None:\n raise TypeError('`suffix` should be None when `list_dir` is True')\n\n if (suffix is not None) and not isinstance(suffix, (str, tuple)):\n raise TypeError('`suffix` must be a string or tuple of strings')\n\n root = dir_path\n\n def _list_dir_or_file(dir_path, list_dir, list_file, suffix,\n recursive):\n for entry in os.scandir(dir_path):\n if not entry.name.startswith('.') and entry.is_file():\n rel_path = osp.relpath(entry.path, root)\n if (suffix is None\n or rel_path.endswith(suffix)) and list_file:\n yield rel_path\n elif osp.isdir(entry.path):\n if list_dir:\n rel_dir = osp.relpath(entry.path, root)\n yield rel_dir\n if recursive:\n yield from _list_dir_or_file(entry.path, list_dir,\n list_file, suffix,\n recursive)\n\n return _list_dir_or_file(dir_path, list_dir, list_file, suffix,\n recursive)\n\n\nclass HTTPBackend(BaseStorageBackend):\n \"\"\"HTTP and HTTPS storage bachend.\"\"\"\n\n def get(self, filepath):\n value_buf = urlopen(filepath).read()\n return value_buf\n\n def get_text(self, filepath, encoding='utf-8'):\n value_buf = urlopen(filepath).read()\n return value_buf.decode(encoding)\n\n @contextmanager\n def get_local_path(self, filepath: str) -> Iterable[str]:\n \"\"\"Download a file from ``filepath``.\n\n ``get_local_path`` is decorated by :meth:`contxtlib.contextmanager`. It\n can be called with ``with`` statement, and when exists from the\n ``with`` statement, the temporary path will be released.\n\n Args:\n filepath (str): Download a file from ``filepath``.\n\n Examples:\n >>> client = HTTPBackend()\n >>> # After existing from the ``with`` clause,\n >>> # the path will be removed\n >>> with client.get_local_path('http://path/of/your/file') as path:\n ... # do something here\n \"\"\"\n try:\n f = tempfile.NamedTemporaryFile(delete=False)\n f.write(self.get(filepath))\n f.close()\n yield f.name\n finally:\n os.remove(f.name)\n\n\nclass FileClient:\n \"\"\"A general file client to access files in different backends.\n\n The client loads a file or text in a specified backend from its path\n and returns it as a binary or text file. There are two ways to choose a\n backend, the name of backend and the prefix of path. Although both of them\n can be used to choose a storage backend, ``backend`` has a higher priority\n that is if they are all set, the storage backend will be chosen by the\n backend argument. If they are all `None`, the disk backend will be chosen.\n Note that It can also register other backend accessor with a given name,\n prefixes, and backend class. In addition, We use the singleton pattern to\n avoid repeated object creation. If the arguments are the same, the same\n object will be returned.\n\n Args:\n backend (str, optional): The storage backend type. Options are \"disk\",\n \"ceph\", \"memcached\", \"lmdb\", \"http\" and \"petrel\". Default: None.\n prefix (str, optional): The prefix of the registered storage backend.\n Options are \"s3\", \"http\", \"https\". Default: None.\n\n Examples:\n >>> # only set backend\n >>> file_client = FileClient(backend='petrel')\n >>> # only set prefix\n >>> file_client = FileClient(prefix='s3')\n >>> # set both backend and prefix but use backend to choose client\n >>> file_client = FileClient(backend='petrel', prefix='s3')\n >>> # if the arguments are the same, the same object is returned\n >>> file_client1 = FileClient(backend='petrel')\n >>> file_client1 is file_client\n True\n\n Attributes:\n client (:obj:`BaseStorageBackend`): The backend object.\n \"\"\"\n\n _backends = {\n 'disk': HardDiskBackend,\n 'ceph': CephBackend,\n 'memcached': MemcachedBackend,\n 'lmdb': LmdbBackend,\n 'petrel': PetrelBackend,\n 'http': HTTPBackend,\n }\n # This collection is used to record the overridden backends, and when a\n # backend appears in the collection, the singleton pattern is disabled for\n # that backend, because if the singleton pattern is used, then the object\n # returned will be the backend before overwriting\n _overridden_backends = set()\n _prefix_to_backends = {\n 's3': PetrelBackend,\n 'http': HTTPBackend,\n 'https': HTTPBackend,\n }\n _overridden_prefixes = set()\n\n _instances = {}\n\n def __new__(cls, backend=None, prefix=None, **kwargs):\n if backend is None and prefix is None:\n backend = 'disk'\n if backend is not None and backend not in cls._backends:\n raise ValueError(\n f'Backend {backend} is not supported. Currently supported ones'\n f' are {list(cls._backends.keys())}')\n if prefix is not None and prefix not in cls._prefix_to_backends:\n raise ValueError(\n f'prefix {prefix} is not supported. Currently supported ones '\n f'are {list(cls._prefix_to_backends.keys())}')\n\n # concatenate the arguments to a unique key for determining whether\n # objects with the same arguments were created\n arg_key = f'{backend}:{prefix}'\n for key, value in kwargs.items():\n arg_key += f':{key}:{value}'\n\n # if a backend was overridden, it will create a new object\n if (arg_key in cls._instances\n and backend not in cls._overridden_backends\n and prefix not in cls._overridden_prefixes):\n _instance = cls._instances[arg_key]\n else:\n # create a new object and put it to _instance\n _instance = super().__new__(cls)\n if backend is not None:\n _instance.client = cls._backends[backend](**kwargs)\n else:\n _instance.client = cls._prefix_to_backends[prefix](**kwargs)\n\n cls._instances[arg_key] = _instance\n\n return _instance\n\n @property\n def name(self):\n return self.client.name\n\n @property\n def allow_symlink(self):\n return self.client.allow_symlink\n\n @staticmethod\n def parse_uri_prefix(uri: Union[str, Path]) -> Optional[str]:\n \"\"\"Parse the prefix of a uri.\n\n Args:\n uri (str | Path): Uri to be parsed that contains the file prefix.\n\n Examples:\n >>> FileClient.parse_uri_prefix('s3://path/of/your/file')\n 's3'\n\n Returns:\n str | None: Return the prefix of uri if the uri contains '://' else\n ``None``.\n \"\"\"\n assert is_filepath(uri)\n uri = str(uri)\n if '://' not in uri:\n return None\n else:\n prefix, _ = uri.split('://')\n # In the case of PetrelBackend, the prefix may contains the cluster\n # name like clusterName:s3\n if ':' in prefix:\n _, prefix = prefix.split(':')\n return prefix\n\n @classmethod\n def infer_client(cls,\n file_client_args: Optional[dict] = None,\n uri: Optional[Union[str, Path]] = None) -> 'FileClient':\n \"\"\"Infer a suitable file client based on the URI and arguments.\n\n Args:\n file_client_args (dict, optional): Arguments to instantiate a\n FileClient. Default: None.\n uri (str | Path, optional): Uri to be parsed that contains the file\n prefix. Default: None.\n\n Examples:\n >>> uri = 's3://path/of/your/file'\n >>> file_client = FileClient.infer_client(uri=uri)\n >>> file_client_args = {'backend': 'petrel'}\n >>> file_client = FileClient.infer_client(file_client_args)\n\n Returns:\n FileClient: Instantiated FileClient object.\n \"\"\"\n assert file_client_args is not None or uri is not None\n if file_client_args is None:\n file_prefix = cls.parse_uri_prefix(uri) # type: ignore\n return cls(prefix=file_prefix)\n else:\n return cls(**file_client_args)\n\n @classmethod\n def _register_backend(cls, name, backend, force=False, prefixes=None):\n if not isinstance(name, str):\n raise TypeError('the backend name should be a string, '\n f'but got {type(name)}')\n if not inspect.isclass(backend):\n raise TypeError(\n f'backend should be a class but got {type(backend)}')\n if not issubclass(backend, BaseStorageBackend):\n raise TypeError(\n f'backend {backend} is not a subclass of BaseStorageBackend')\n if not force and name in cls._backends:\n raise KeyError(\n f'{name} is already registered as a storage backend, '\n 'add \"force=True\" if you want to override it')\n\n if name in cls._backends and force:\n cls._overridden_backends.add(name)\n cls._backends[name] = backend\n\n if prefixes is not None:\n if isinstance(prefixes, str):\n prefixes = [prefixes]\n else:\n assert isinstance(prefixes, (list, tuple))\n for prefix in prefixes:\n if prefix not in cls._prefix_to_backends:\n cls._prefix_to_backends[prefix] = backend\n elif (prefix in cls._prefix_to_backends) and force:\n cls._overridden_prefixes.add(prefix)\n cls._prefix_to_backends[prefix] = backend\n else:\n raise KeyError(\n f'{prefix} is already registered as a storage backend,'\n ' add \"force=True\" if you want to override it')\n\n @classmethod\n def register_backend(cls, name, backend=None, force=False, prefixes=None):\n \"\"\"Register a backend to FileClient.\n\n This method can be used as a normal class method or a decorator.\n\n .. code-block:: python\n\n class NewBackend(BaseStorageBackend):\n\n def get(self, filepath):\n return filepath\n\n def get_text(self, filepath):\n return filepath\n\n FileClient.register_backend('new', NewBackend)\n\n or\n\n .. code-block:: python\n\n @FileClient.register_backend('new')\n class NewBackend(BaseStorageBackend):\n\n def get(self, filepath):\n return filepath\n\n def get_text(self, filepath):\n return filepath\n\n Args:\n name (str): The name of the registered backend.\n backend (class, optional): The backend class to be registered,\n which must be a subclass of :class:`BaseStorageBackend`.\n When this method is used as a decorator, backend is None.\n Defaults to None.\n force (bool, optional): Whether to override the backend if the name\n has already been registered. Defaults to False.\n prefixes (str or list[str] or tuple[str], optional): The prefixes\n of the registered storage backend. Default: None.\n `New in version 1.3.15.`\n \"\"\"\n if backend is not None:\n cls._register_backend(\n name, backend, force=force, prefixes=prefixes)\n return\n\n def _register(backend_cls):\n cls._register_backend(\n name, backend_cls, force=force, prefixes=prefixes)\n return backend_cls\n\n return _register\n\n def get(self, filepath: Union[str, Path]) -> Union[bytes, memoryview]:\n \"\"\"Read data from a given ``filepath`` with 'rb' mode.\n\n Note:\n There are two types of return values for ``get``, one is ``bytes``\n and the other is ``memoryview``. The advantage of using memoryview\n is that you can avoid copying, and if you want to convert it to\n ``bytes``, you can use ``.tobytes()``.\n\n Args:\n filepath (str or Path): Path to read data.\n\n Returns:\n bytes | memoryview: Expected bytes object or a memory view of the\n bytes object.\n \"\"\"\n return self.client.get(filepath)\n\n def get_text(self, filepath: Union[str, Path], encoding='utf-8') -> str:\n \"\"\"Read data from a given ``filepath`` with 'r' mode.\n\n Args:\n filepath (str or Path): Path to read data.\n encoding (str): The encoding format used to open the ``filepath``.\n Default: 'utf-8'.\n\n Returns:\n str: Expected text reading from ``filepath``.\n \"\"\"\n return self.client.get_text(filepath, encoding)\n\n def put(self, obj: bytes, filepath: Union[str, Path]) -> None:\n \"\"\"Write data to a given ``filepath`` with 'wb' mode.\n\n Note:\n ``put`` should create a directory if the directory of ``filepath``\n does not exist.\n\n Args:\n obj (bytes): Data to be written.\n filepath (str or Path): Path to write data.\n \"\"\"\n self.client.put(obj, filepath)\n\n def put_text(self, obj: str, filepath: Union[str, Path]) -> None:\n \"\"\"Write data to a given ``filepath`` with 'w' mode.\n\n Note:\n ``put_text`` should create a directory if the directory of\n ``filepath`` does not exist.\n\n Args:\n obj (str): Data to be written.\n filepath (str or Path): Path to write data.\n encoding (str, optional): The encoding format used to open the\n `filepath`. Default: 'utf-8'.\n \"\"\"\n self.client.put_text(obj, filepath)\n\n def remove(self, filepath: Union[str, Path]) -> None:\n \"\"\"Remove a file.\n\n Args:\n filepath (str, Path): Path to be removed.\n \"\"\"\n self.client.remove(filepath)\n\n def exists(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path exists.\n\n Args:\n filepath (str or Path): Path to be checked whether exists.\n\n Returns:\n bool: Return ``True`` if ``filepath`` exists, ``False`` otherwise.\n \"\"\"\n return self.client.exists(filepath)\n\n def isdir(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path is a directory.\n\n Args:\n filepath (str or Path): Path to be checked whether it is a\n directory.\n\n Returns:\n bool: Return ``True`` if ``filepath`` points to a directory,\n ``False`` otherwise.\n \"\"\"\n return self.client.isdir(filepath)\n\n def isfile(self, filepath: Union[str, Path]) -> bool:\n \"\"\"Check whether a file path is a file.\n\n Args:\n filepath (str or Path): Path to be checked whether it is a file.\n\n Returns:\n bool: Return ``True`` if ``filepath`` points to a file, ``False``\n otherwise.\n \"\"\"\n return self.client.isfile(filepath)\n\n def join_path(self, filepath: Union[str, Path],\n *filepaths: Union[str, Path]) -> str:\n \"\"\"Concatenate all file paths.\n\n Join one or more filepath components intelligently. The return value\n is the concatenation of filepath and any members of *filepaths.\n\n Args:\n filepath (str or Path): Path to be concatenated.\n\n Returns:\n str: The result of concatenation.\n \"\"\"\n return self.client.join_path(filepath, *filepaths)\n\n @contextmanager\n def get_local_path(self, filepath: Union[str, Path]) -> Iterable[str]:\n \"\"\"Download data from ``filepath`` and write the data to local path.\n\n ``get_local_path`` is decorated by :meth:`contxtlib.contextmanager`. It\n can be called with ``with`` statement, and when exists from the\n ``with`` statement, the temporary path will be released.\n\n Note:\n If the ``filepath`` is a local path, just return itself.\n\n .. warning::\n ``get_local_path`` is an experimental interface that may change in\n the future.\n\n Args:\n filepath (str or Path): Path to be read data.\n\n Examples:\n >>> file_client = FileClient(prefix='s3')\n >>> with file_client.get_local_path('s3://bucket/abc.jpg') as path:\n ... # do something here\n\n Yields:\n Iterable[str]: Only yield one path.\n \"\"\"\n with self.client.get_local_path(str(filepath)) as local_path:\n yield local_path\n\n def list_dir_or_file(self,\n dir_path: Union[str, Path],\n list_dir: bool = True,\n list_file: bool = True,\n suffix: Optional[Union[str, Tuple[str]]] = None,\n recursive: bool = False) -> Iterator[str]:\n \"\"\"Scan a directory to find the interested directories or files in\n arbitrary order.\n\n Note:\n :meth:`list_dir_or_file` returns the path relative to ``dir_path``.\n\n Args:\n dir_path (str | Path): Path of the directory.\n list_dir (bool): List the directories. Default: True.\n list_file (bool): List the path of files. Default: True.\n suffix (str or tuple[str], optional): File suffix\n that we are interested in. Default: None.\n recursive (bool): If set to True, recursively scan the\n directory. Default: False.\n\n Yields:\n Iterable[str]: A relative path to ``dir_path``.\n \"\"\"\n yield from self.client.list_dir_or_file(dir_path, list_dir, list_file,\n suffix, recursive)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/handlers/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .base import BaseFileHandler\nfrom .json_handler import JsonHandler\nfrom .pickle_handler import PickleHandler\nfrom .yaml_handler import YamlHandler\n\n__all__ = ['BaseFileHandler', 'JsonHandler', 'PickleHandler', 'YamlHandler']\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/handlers/base.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom abc import ABCMeta, abstractmethod\n\n\nclass BaseFileHandler(metaclass=ABCMeta):\n # `str_like` is a flag to indicate whether the type of file object is\n # str-like object or bytes-like object. Pickle only processes bytes-like\n # objects but json only processes str-like object. If it is str-like\n # object, `StringIO` will be used to process the buffer.\n str_like = True\n\n @abstractmethod\n def load_from_fileobj(self, file, **kwargs):\n pass\n\n @abstractmethod\n def dump_to_fileobj(self, obj, file, **kwargs):\n pass\n\n @abstractmethod\n def dump_to_str(self, obj, **kwargs):\n pass\n\n def load_from_path(self, filepath, mode='r', **kwargs):\n with open(filepath, mode) as f:\n return self.load_from_fileobj(f, **kwargs)\n\n def dump_to_path(self, obj, filepath, mode='w', **kwargs):\n with open(filepath, mode) as f:\n self.dump_to_fileobj(obj, f, **kwargs)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/handlers/json_handler.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport json\n\nimport numpy as np\n\nfrom .base import BaseFileHandler\n\n\ndef set_default(obj):\n \"\"\"Set default json values for non-serializable values.\n\n It helps convert ``set``, ``range`` and ``np.ndarray`` data types to list.\n It also converts ``np.generic`` (including ``np.int32``, ``np.float32``,\n etc.) into plain numbers of plain python built-in types.\n \"\"\"\n if isinstance(obj, (set, range)):\n return list(obj)\n elif isinstance(obj, np.ndarray):\n return obj.tolist()\n elif isinstance(obj, np.generic):\n return obj.item()\n raise TypeError(f'{type(obj)} is unsupported for json dump')\n\n\nclass JsonHandler(BaseFileHandler):\n\n def load_from_fileobj(self, file):\n return json.load(file)\n\n def dump_to_fileobj(self, obj, file, **kwargs):\n kwargs.setdefault('default', set_default)\n json.dump(obj, file, **kwargs)\n\n def dump_to_str(self, obj, **kwargs):\n kwargs.setdefault('default', set_default)\n return json.dumps(obj, **kwargs)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/handlers/pickle_handler.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport pickle\n\nfrom .base import BaseFileHandler\n\n\nclass PickleHandler(BaseFileHandler):\n\n str_like = False\n\n def load_from_fileobj(self, file, **kwargs):\n return pickle.load(file, **kwargs)\n\n def load_from_path(self, filepath, **kwargs):\n return super(PickleHandler, self).load_from_path(\n filepath, mode='rb', **kwargs)\n\n def dump_to_str(self, obj, **kwargs):\n kwargs.setdefault('protocol', 2)\n return pickle.dumps(obj, **kwargs)\n\n def dump_to_fileobj(self, obj, file, **kwargs):\n kwargs.setdefault('protocol', 2)\n pickle.dump(obj, file, **kwargs)\n\n def dump_to_path(self, obj, filepath, **kwargs):\n super(PickleHandler, self).dump_to_path(\n obj, filepath, mode='wb', **kwargs)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/handlers/yaml_handler.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport yaml\n\ntry:\n from yaml import CLoader as Loader, CDumper as Dumper\nexcept ImportError:\n from yaml import Loader, Dumper\n\nfrom .base import BaseFileHandler # isort:skip\n\n\nclass YamlHandler(BaseFileHandler):\n\n def load_from_fileobj(self, file, **kwargs):\n kwargs.setdefault('Loader', Loader)\n return yaml.load(file, **kwargs)\n\n def dump_to_fileobj(self, obj, file, **kwargs):\n kwargs.setdefault('Dumper', Dumper)\n yaml.dump(obj, file, **kwargs)\n\n def dump_to_str(self, obj, **kwargs):\n kwargs.setdefault('Dumper', Dumper)\n return yaml.dump(obj, **kwargs)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/io.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom io import BytesIO, StringIO\nfrom pathlib import Path\n\nfrom ..utils import is_list_of, is_str\nfrom .file_client import FileClient\nfrom .handlers import BaseFileHandler, JsonHandler, PickleHandler, YamlHandler\n\nfile_handlers = {\n 'json': JsonHandler(),\n 'yaml': YamlHandler(),\n 'yml': YamlHandler(),\n 'pickle': PickleHandler(),\n 'pkl': PickleHandler()\n}\n\n\ndef load(file, file_format=None, file_client_args=None, **kwargs):\n \"\"\"Load data from json/yaml/pickle files.\n\n This method provides a unified api for loading data from serialized files.\n\n Note:\n In v1.3.16 and later, ``load`` supports loading data from serialized\n files those can be storaged in different backends.\n\n Args:\n file (str or :obj:`Path` or file-like object): Filename or a file-like\n object.\n file_format (str, optional): If not specified, the file format will be\n inferred from the file extension, otherwise use the specified one.\n Currently supported formats include \"json\", \"yaml/yml\" and\n \"pickle/pkl\".\n file_client_args (dict, optional): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n\n Examples:\n >>> load('/path/of/your/file') # file is storaged in disk\n >>> load('https://path/of/your/file') # file is storaged in Internet\n >>> load('s3://path/of/your/file') # file is storaged in petrel\n\n Returns:\n The content from the file.\n \"\"\"\n if isinstance(file, Path):\n file = str(file)\n if file_format is None and is_str(file):\n file_format = file.split('.')[-1]\n if file_format not in file_handlers:\n raise TypeError(f'Unsupported format: {file_format}')\n\n handler = file_handlers[file_format]\n if is_str(file):\n file_client = FileClient.infer_client(file_client_args, file)\n if handler.str_like:\n with StringIO(file_client.get_text(file)) as f:\n obj = handler.load_from_fileobj(f, **kwargs)\n else:\n with BytesIO(file_client.get(file)) as f:\n obj = handler.load_from_fileobj(f, **kwargs)\n elif hasattr(file, 'read'):\n obj = handler.load_from_fileobj(file, **kwargs)\n else:\n raise TypeError('\"file\" must be a filepath str or a file-object')\n return obj\n\n\ndef dump(obj, file=None, file_format=None, file_client_args=None, **kwargs):\n \"\"\"Dump data to json/yaml/pickle strings or files.\n\n This method provides a unified api for dumping data as strings or to files,\n and also supports custom arguments for each file format.\n\n Note:\n In v1.3.16 and later, ``dump`` supports dumping data as strings or to\n files which is saved to different backends.\n\n Args:\n obj (any): The python object to be dumped.\n file (str or :obj:`Path` or file-like object, optional): If not\n specified, then the object is dumped to a str, otherwise to a file\n specified by the filename or file-like object.\n file_format (str, optional): Same as :func:`load`.\n file_client_args (dict, optional): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n\n Examples:\n >>> dump('hello world', '/path/of/your/file') # disk\n >>> dump('hello world', 's3://path/of/your/file') # ceph or petrel\n\n Returns:\n bool: True for success, False otherwise.\n \"\"\"\n if isinstance(file, Path):\n file = str(file)\n if file_format is None:\n if is_str(file):\n file_format = file.split('.')[-1]\n elif file is None:\n raise ValueError(\n 'file_format must be specified since file is None')\n if file_format not in file_handlers:\n raise TypeError(f'Unsupported format: {file_format}')\n\n handler = file_handlers[file_format]\n if file is None:\n return handler.dump_to_str(obj, **kwargs)\n elif is_str(file):\n file_client = FileClient.infer_client(file_client_args, file)\n if handler.str_like:\n with StringIO() as f:\n handler.dump_to_fileobj(obj, f, **kwargs)\n file_client.put_text(f.getvalue(), file)\n else:\n with BytesIO() as f:\n handler.dump_to_fileobj(obj, f, **kwargs)\n file_client.put(f.getvalue(), file)\n elif hasattr(file, 'write'):\n handler.dump_to_fileobj(obj, file, **kwargs)\n else:\n raise TypeError('\"file\" must be a filename str or a file-object')\n\n\ndef _register_handler(handler, file_formats):\n \"\"\"Register a handler for some file extensions.\n\n Args:\n handler (:obj:`BaseFileHandler`): Handler to be registered.\n file_formats (str or list[str]): File formats to be handled by this\n handler.\n \"\"\"\n if not isinstance(handler, BaseFileHandler):\n raise TypeError(\n f'handler must be a child of BaseFileHandler, not {type(handler)}')\n if isinstance(file_formats, str):\n file_formats = [file_formats]\n if not is_list_of(file_formats, str):\n raise TypeError('file_formats must be a str or a list of str')\n for ext in file_formats:\n file_handlers[ext] = handler\n\n\ndef register_handler(file_formats, **kwargs):\n\n def wrap(cls):\n _register_handler(cls(**kwargs), file_formats)\n return cls\n\n return wrap\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/fileio/parse.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n\nfrom io import StringIO\n\nfrom .file_client import FileClient\n\n\ndef list_from_file(filename,\n prefix='',\n offset=0,\n max_num=0,\n encoding='utf-8',\n file_client_args=None):\n \"\"\"Load a text file and parse the content as a list of strings.\n\n Note:\n In v1.3.16 and later, ``list_from_file`` supports loading a text file\n which can be storaged in different backends and parsing the content as\n a list for strings.\n\n Args:\n filename (str): Filename.\n prefix (str): The prefix to be inserted to the beginning of each item.\n offset (int): The offset of lines.\n max_num (int): The maximum number of lines to be read,\n zeros and negatives mean no limitation.\n encoding (str): Encoding used to open the file. Default utf-8.\n file_client_args (dict, optional): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n\n Examples:\n >>> list_from_file('/path/of/your/file') # disk\n ['hello', 'world']\n >>> list_from_file('s3://path/of/your/file') # ceph or petrel\n ['hello', 'world']\n\n Returns:\n list[str]: A list of strings.\n \"\"\"\n cnt = 0\n item_list = []\n file_client = FileClient.infer_client(file_client_args, filename)\n with StringIO(file_client.get_text(filename, encoding)) as f:\n for _ in range(offset):\n f.readline()\n for line in f:\n if 0 < max_num <= cnt:\n break\n item_list.append(prefix + line.rstrip('\\n\\r'))\n cnt += 1\n return item_list\n\n\ndef dict_from_file(filename,\n key_type=str,\n encoding='utf-8',\n file_client_args=None):\n \"\"\"Load a text file and parse the content as a dict.\n\n Each line of the text file will be two or more columns split by\n whitespaces or tabs. The first column will be parsed as dict keys, and\n the following columns will be parsed as dict values.\n\n Note:\n In v1.3.16 and later, ``dict_from_file`` supports loading a text file\n which can be storaged in different backends and parsing the content as\n a dict.\n\n Args:\n filename(str): Filename.\n key_type(type): Type of the dict keys. str is user by default and\n type conversion will be performed if specified.\n encoding (str): Encoding used to open the file. Default utf-8.\n file_client_args (dict, optional): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n\n Examples:\n >>> dict_from_file('/path/of/your/file') # disk\n {'key1': 'value1', 'key2': 'value2'}\n >>> dict_from_file('s3://path/of/your/file') # ceph or petrel\n {'key1': 'value1', 'key2': 'value2'}\n\n Returns:\n dict: The parsed contents.\n \"\"\"\n mapping = {}\n file_client = FileClient.infer_client(file_client_args, filename)\n with StringIO(file_client.get_text(filename, encoding)) as f:\n for line in f:\n items = line.rstrip('\\n').split()\n assert len(items) >= 2\n key = key_type(items[0])\n val = items[1:] if len(items) > 2 else items[1]\n mapping[key] = val\n return mapping\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/image/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .colorspace import (bgr2gray, bgr2hls, bgr2hsv, bgr2rgb, bgr2ycbcr,\n gray2bgr, gray2rgb, hls2bgr, hsv2bgr, imconvert,\n rgb2bgr, rgb2gray, rgb2ycbcr, ycbcr2bgr, ycbcr2rgb)\nfrom .geometric import (cutout, imcrop, imflip, imflip_, impad,\n impad_to_multiple, imrescale, imresize, imresize_like,\n imresize_to_multiple, imrotate, imshear, imtranslate,\n rescale_size)\nfrom .io import imfrombytes, imread, imwrite, supported_backends, use_backend\nfrom .misc import tensor2imgs\nfrom .photometric import (adjust_brightness, adjust_color, adjust_contrast,\n adjust_lighting, adjust_sharpness, auto_contrast,\n clahe, imdenormalize, imequalize, iminvert,\n imnormalize, imnormalize_, lut_transform, posterize,\n solarize)\n\n__all__ = [\n 'bgr2gray', 'bgr2hls', 'bgr2hsv', 'bgr2rgb', 'gray2bgr', 'gray2rgb',\n 'hls2bgr', 'hsv2bgr', 'imconvert', 'rgb2bgr', 'rgb2gray', 'imrescale',\n 'imresize', 'imresize_like', 'imresize_to_multiple', 'rescale_size',\n 'imcrop', 'imflip', 'imflip_', 'impad', 'impad_to_multiple', 'imrotate',\n 'imfrombytes', 'imread', 'imwrite', 'supported_backends', 'use_backend',\n 'imdenormalize', 'imnormalize', 'imnormalize_', 'iminvert', 'posterize',\n 'solarize', 'rgb2ycbcr', 'bgr2ycbcr', 'ycbcr2rgb', 'ycbcr2bgr',\n 'tensor2imgs', 'imshear', 'imtranslate', 'adjust_color', 'imequalize',\n 'adjust_brightness', 'adjust_contrast', 'lut_transform', 'clahe',\n 'adjust_sharpness', 'auto_contrast', 'cutout', 'adjust_lighting'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/image/colorspace.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport cv2\nimport numpy as np\n\n\ndef imconvert(img, src, dst):\n \"\"\"Convert an image from the src colorspace to dst colorspace.\n\n Args:\n img (ndarray): The input image.\n src (str): The source colorspace, e.g., 'rgb', 'hsv'.\n dst (str): The destination colorspace, e.g., 'rgb', 'hsv'.\n\n Returns:\n ndarray: The converted image.\n \"\"\"\n code = getattr(cv2, f'COLOR_{src.upper()}2{dst.upper()}')\n out_img = cv2.cvtColor(img, code)\n return out_img\n\n\ndef bgr2gray(img, keepdim=False):\n \"\"\"Convert a BGR image to grayscale image.\n\n Args:\n img (ndarray): The input image.\n keepdim (bool): If False (by default), then return the grayscale image\n with 2 dims, otherwise 3 dims.\n\n Returns:\n ndarray: The converted grayscale image.\n \"\"\"\n out_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n if keepdim:\n out_img = out_img[..., None]\n return out_img\n\n\ndef rgb2gray(img, keepdim=False):\n \"\"\"Convert a RGB image to grayscale image.\n\n Args:\n img (ndarray): The input image.\n keepdim (bool): If False (by default), then return the grayscale image\n with 2 dims, otherwise 3 dims.\n\n Returns:\n ndarray: The converted grayscale image.\n \"\"\"\n out_img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)\n if keepdim:\n out_img = out_img[..., None]\n return out_img\n\n\ndef gray2bgr(img):\n \"\"\"Convert a grayscale image to BGR image.\n\n Args:\n img (ndarray): The input image.\n\n Returns:\n ndarray: The converted BGR image.\n \"\"\"\n img = img[..., None] if img.ndim == 2 else img\n out_img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)\n return out_img\n\n\ndef gray2rgb(img):\n \"\"\"Convert a grayscale image to RGB image.\n\n Args:\n img (ndarray): The input image.\n\n Returns:\n ndarray: The converted RGB image.\n \"\"\"\n img = img[..., None] if img.ndim == 2 else img\n out_img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)\n return out_img\n\n\ndef _convert_input_type_range(img):\n \"\"\"Convert the type and range of the input image.\n\n It converts the input image to np.float32 type and range of [0, 1].\n It is mainly used for pre-processing the input image in colorspace\n conversion functions such as rgb2ycbcr and ycbcr2rgb.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n\n Returns:\n (ndarray): The converted image with type of np.float32 and range of\n [0, 1].\n \"\"\"\n img_type = img.dtype\n img = img.astype(np.float32)\n if img_type == np.float32:\n pass\n elif img_type == np.uint8:\n img /= 255.\n else:\n raise TypeError('The img type should be np.float32 or np.uint8, '\n f'but got {img_type}')\n return img\n\n\ndef _convert_output_type_range(img, dst_type):\n \"\"\"Convert the type and range of the image according to dst_type.\n\n It converts the image to desired type and range. If `dst_type` is np.uint8,\n images will be converted to np.uint8 type with range [0, 255]. If\n `dst_type` is np.float32, it converts the image to np.float32 type with\n range [0, 1].\n It is mainly used for post-processing images in colorspace conversion\n functions such as rgb2ycbcr and ycbcr2rgb.\n\n Args:\n img (ndarray): The image to be converted with np.float32 type and\n range [0, 255].\n dst_type (np.uint8 | np.float32): If dst_type is np.uint8, it\n converts the image to np.uint8 type with range [0, 255]. If\n dst_type is np.float32, it converts the image to np.float32 type\n with range [0, 1].\n\n Returns:\n (ndarray): The converted image with desired type and range.\n \"\"\"\n if dst_type not in (np.uint8, np.float32):\n raise TypeError('The dst_type should be np.float32 or np.uint8, '\n f'but got {dst_type}')\n if dst_type == np.uint8:\n img = img.round()\n else:\n img /= 255.\n return img.astype(dst_type)\n\n\ndef rgb2ycbcr(img, y_only=False):\n \"\"\"Convert a RGB image to YCbCr image.\n\n This function produces the same results as Matlab's `rgb2ycbcr` function.\n It implements the ITU-R BT.601 conversion for standard-definition\n television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n It differs from a similar function in cv2.cvtColor: `RGB <-> YCrCb`.\n In OpenCV, it implements a JPEG conversion. See more details in\n https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n y_only (bool): Whether to only return Y channel. Default: False.\n\n Returns:\n ndarray: The converted YCbCr image. The output image has the same type\n and range as input image.\n \"\"\"\n img_type = img.dtype\n img = _convert_input_type_range(img)\n if y_only:\n out_img = np.dot(img, [65.481, 128.553, 24.966]) + 16.0\n else:\n out_img = np.matmul(\n img, [[65.481, -37.797, 112.0], [128.553, -74.203, -93.786],\n [24.966, 112.0, -18.214]]) + [16, 128, 128]\n out_img = _convert_output_type_range(out_img, img_type)\n return out_img\n\n\ndef bgr2ycbcr(img, y_only=False):\n \"\"\"Convert a BGR image to YCbCr image.\n\n The bgr version of rgb2ycbcr.\n It implements the ITU-R BT.601 conversion for standard-definition\n television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n It differs from a similar function in cv2.cvtColor: `BGR <-> YCrCb`.\n In OpenCV, it implements a JPEG conversion. See more details in\n https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n y_only (bool): Whether to only return Y channel. Default: False.\n\n Returns:\n ndarray: The converted YCbCr image. The output image has the same type\n and range as input image.\n \"\"\"\n img_type = img.dtype\n img = _convert_input_type_range(img)\n if y_only:\n out_img = np.dot(img, [24.966, 128.553, 65.481]) + 16.0\n else:\n out_img = np.matmul(\n img, [[24.966, 112.0, -18.214], [128.553, -74.203, -93.786],\n [65.481, -37.797, 112.0]]) + [16, 128, 128]\n out_img = _convert_output_type_range(out_img, img_type)\n return out_img\n\n\ndef ycbcr2rgb(img):\n \"\"\"Convert a YCbCr image to RGB image.\n\n This function produces the same results as Matlab's ycbcr2rgb function.\n It implements the ITU-R BT.601 conversion for standard-definition\n television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n It differs from a similar function in cv2.cvtColor: `YCrCb <-> RGB`.\n In OpenCV, it implements a JPEG conversion. See more details in\n https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n\n Returns:\n ndarray: The converted RGB image. The output image has the same type\n and range as input image.\n \"\"\"\n img_type = img.dtype\n img = _convert_input_type_range(img) * 255\n out_img = np.matmul(img, [[0.00456621, 0.00456621, 0.00456621],\n [0, -0.00153632, 0.00791071],\n [0.00625893, -0.00318811, 0]]) * 255.0 + [\n -222.921, 135.576, -276.836\n ]\n out_img = _convert_output_type_range(out_img, img_type)\n return out_img\n\n\ndef ycbcr2bgr(img):\n \"\"\"Convert a YCbCr image to BGR image.\n\n The bgr version of ycbcr2rgb.\n It implements the ITU-R BT.601 conversion for standard-definition\n television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n It differs from a similar function in cv2.cvtColor: `YCrCb <-> BGR`.\n In OpenCV, it implements a JPEG conversion. See more details in\n https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n\n Returns:\n ndarray: The converted BGR image. The output image has the same type\n and range as input image.\n \"\"\"\n img_type = img.dtype\n img = _convert_input_type_range(img) * 255\n out_img = np.matmul(img, [[0.00456621, 0.00456621, 0.00456621],\n [0.00791071, -0.00153632, 0],\n [0, -0.00318811, 0.00625893]]) * 255.0 + [\n -276.836, 135.576, -222.921\n ]\n out_img = _convert_output_type_range(out_img, img_type)\n return out_img\n\n\ndef convert_color_factory(src, dst):\n\n code = getattr(cv2, f'COLOR_{src.upper()}2{dst.upper()}')\n\n def convert_color(img):\n out_img = cv2.cvtColor(img, code)\n return out_img\n\n convert_color.__doc__ = f\"\"\"Convert a {src.upper()} image to {dst.upper()}\n image.\n\n Args:\n img (ndarray or str): The input image.\n\n Returns:\n ndarray: The converted {dst.upper()} image.\n \"\"\"\n\n return convert_color\n\n\nbgr2rgb = convert_color_factory('bgr', 'rgb')\n\nrgb2bgr = convert_color_factory('rgb', 'bgr')\n\nbgr2hsv = convert_color_factory('bgr', 'hsv')\n\nhsv2bgr = convert_color_factory('hsv', 'bgr')\n\nbgr2hls = convert_color_factory('bgr', 'hls')\n\nhls2bgr = convert_color_factory('hls', 'bgr')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/image/geometric.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numbers\n\nimport cv2\nimport numpy as np\n\nfrom ..utils import to_2tuple\nfrom .io import imread_backend\n\ntry:\n from PIL import Image\nexcept ImportError:\n Image = None\n\n\ndef _scale_size(size, scale):\n \"\"\"Rescale a size by a ratio.\n\n Args:\n size (tuple[int]): (w, h).\n scale (float | tuple(float)): Scaling factor.\n\n Returns:\n tuple[int]: scaled size.\n \"\"\"\n if isinstance(scale, (float, int)):\n scale = (scale, scale)\n w, h = size\n return int(w * float(scale[0]) + 0.5), int(h * float(scale[1]) + 0.5)\n\n\ncv2_interp_codes = {\n 'nearest': cv2.INTER_NEAREST,\n 'bilinear': cv2.INTER_LINEAR,\n 'bicubic': cv2.INTER_CUBIC,\n 'area': cv2.INTER_AREA,\n 'lanczos': cv2.INTER_LANCZOS4\n}\n\nif Image is not None:\n pillow_interp_codes = {\n 'nearest': Image.NEAREST,\n 'bilinear': Image.BILINEAR,\n 'bicubic': Image.BICUBIC,\n 'box': Image.BOX,\n 'lanczos': Image.LANCZOS,\n 'hamming': Image.HAMMING\n }\n\n\ndef imresize(img,\n size,\n return_scale=False,\n interpolation='bilinear',\n out=None,\n backend=None):\n \"\"\"Resize image to a given size.\n\n Args:\n img (ndarray): The input image.\n size (tuple[int]): Target size (w, h).\n return_scale (bool): Whether to return `w_scale` and `h_scale`.\n interpolation (str): Interpolation method, accepted values are\n \"nearest\", \"bilinear\", \"bicubic\", \"area\", \"lanczos\" for 'cv2'\n backend, \"nearest\", \"bilinear\" for 'pillow' backend.\n out (ndarray): The output destination.\n backend (str | None): The image resize backend type. Options are `cv2`,\n `pillow`, `None`. If backend is None, the global imread_backend\n specified by ``mmcv.use_backend()`` will be used. Default: None.\n\n Returns:\n tuple | ndarray: (`resized_img`, `w_scale`, `h_scale`) or\n `resized_img`.\n \"\"\"\n h, w = img.shape[:2]\n if backend is None:\n backend = imread_backend\n if backend not in ['cv2', 'pillow']:\n raise ValueError(f'backend: {backend} is not supported for resize.'\n f\"Supported backends are 'cv2', 'pillow'\")\n\n if backend == 'pillow':\n assert img.dtype == np.uint8, 'Pillow backend only support uint8 type'\n pil_image = Image.fromarray(img)\n pil_image = pil_image.resize(size, pillow_interp_codes[interpolation])\n resized_img = np.array(pil_image)\n else:\n resized_img = cv2.resize(\n img, size, dst=out, interpolation=cv2_interp_codes[interpolation])\n if not return_scale:\n return resized_img\n else:\n w_scale = size[0] / w\n h_scale = size[1] / h\n return resized_img, w_scale, h_scale\n\n\ndef imresize_to_multiple(img,\n divisor,\n size=None,\n scale_factor=None,\n keep_ratio=False,\n return_scale=False,\n interpolation='bilinear',\n out=None,\n backend=None):\n \"\"\"Resize image according to a given size or scale factor and then rounds\n up the the resized or rescaled image size to the nearest value that can be\n divided by the divisor.\n\n Args:\n img (ndarray): The input image.\n divisor (int | tuple): Resized image size will be a multiple of\n divisor. If divisor is a tuple, divisor should be\n (w_divisor, h_divisor).\n size (None | int | tuple[int]): Target size (w, h). Default: None.\n scale_factor (None | float | tuple[float]): Multiplier for spatial\n size. Should match input size if it is a tuple and the 2D style is\n (w_scale_factor, h_scale_factor). Default: None.\n keep_ratio (bool): Whether to keep the aspect ratio when resizing the\n image. Default: False.\n return_scale (bool): Whether to return `w_scale` and `h_scale`.\n interpolation (str): Interpolation method, accepted values are\n \"nearest\", \"bilinear\", \"bicubic\", \"area\", \"lanczos\" for 'cv2'\n backend, \"nearest\", \"bilinear\" for 'pillow' backend.\n out (ndarray): The output destination.\n backend (str | None): The image resize backend type. Options are `cv2`,\n `pillow`, `None`. If backend is None, the global imread_backend\n specified by ``mmcv.use_backend()`` will be used. Default: None.\n\n Returns:\n tuple | ndarray: (`resized_img`, `w_scale`, `h_scale`) or\n `resized_img`.\n \"\"\"\n h, w = img.shape[:2]\n if size is not None and scale_factor is not None:\n raise ValueError('only one of size or scale_factor should be defined')\n elif size is None and scale_factor is None:\n raise ValueError('one of size or scale_factor should be defined')\n elif size is not None:\n size = to_2tuple(size)\n if keep_ratio:\n size = rescale_size((w, h), size, return_scale=False)\n else:\n size = _scale_size((w, h), scale_factor)\n\n divisor = to_2tuple(divisor)\n size = tuple([int(np.ceil(s / d)) * d for s, d in zip(size, divisor)])\n resized_img, w_scale, h_scale = imresize(\n img,\n size,\n return_scale=True,\n interpolation=interpolation,\n out=out,\n backend=backend)\n if return_scale:\n return resized_img, w_scale, h_scale\n else:\n return resized_img\n\n\ndef imresize_like(img,\n dst_img,\n return_scale=False,\n interpolation='bilinear',\n backend=None):\n \"\"\"Resize image to the same size of a given image.\n\n Args:\n img (ndarray): The input image.\n dst_img (ndarray): The target image.\n return_scale (bool): Whether to return `w_scale` and `h_scale`.\n interpolation (str): Same as :func:`resize`.\n backend (str | None): Same as :func:`resize`.\n\n Returns:\n tuple or ndarray: (`resized_img`, `w_scale`, `h_scale`) or\n `resized_img`.\n \"\"\"\n h, w = dst_img.shape[:2]\n return imresize(img, (w, h), return_scale, interpolation, backend=backend)\n\n\ndef rescale_size(old_size, scale, return_scale=False):\n \"\"\"Calculate the new size to be rescaled to.\n\n Args:\n old_size (tuple[int]): The old size (w, h) of image.\n scale (float | tuple[int]): The scaling factor or maximum size.\n If it is a float number, then the image will be rescaled by this\n factor, else if it is a tuple of 2 integers, then the image will\n be rescaled as large as possible within the scale.\n return_scale (bool): Whether to return the scaling factor besides the\n rescaled image size.\n\n Returns:\n tuple[int]: The new rescaled image size.\n \"\"\"\n w, h = old_size\n if isinstance(scale, (float, int)):\n if scale <= 0:\n raise ValueError(f'Invalid scale {scale}, must be positive.')\n scale_factor = scale\n elif isinstance(scale, tuple):\n max_long_edge = max(scale)\n max_short_edge = min(scale)\n scale_factor = min(max_long_edge / max(h, w),\n max_short_edge / min(h, w))\n else:\n raise TypeError(\n f'Scale must be a number or tuple of int, but got {type(scale)}')\n\n new_size = _scale_size((w, h), scale_factor)\n\n if return_scale:\n return new_size, scale_factor\n else:\n return new_size\n\n\ndef imrescale(img,\n scale,\n return_scale=False,\n interpolation='bilinear',\n backend=None):\n \"\"\"Resize image while keeping the aspect ratio.\n\n Args:\n img (ndarray): The input image.\n scale (float | tuple[int]): The scaling factor or maximum size.\n If it is a float number, then the image will be rescaled by this\n factor, else if it is a tuple of 2 integers, then the image will\n be rescaled as large as possible within the scale.\n return_scale (bool): Whether to return the scaling factor besides the\n rescaled image.\n interpolation (str): Same as :func:`resize`.\n backend (str | None): Same as :func:`resize`.\n\n Returns:\n ndarray: The rescaled image.\n \"\"\"\n h, w = img.shape[:2]\n new_size, scale_factor = rescale_size((w, h), scale, return_scale=True)\n rescaled_img = imresize(\n img, new_size, interpolation=interpolation, backend=backend)\n if return_scale:\n return rescaled_img, scale_factor\n else:\n return rescaled_img\n\n\ndef imflip(img, direction='horizontal'):\n \"\"\"Flip an image horizontally or vertically.\n\n Args:\n img (ndarray): Image to be flipped.\n direction (str): The flip direction, either \"horizontal\" or\n \"vertical\" or \"diagonal\".\n\n Returns:\n ndarray: The flipped image.\n \"\"\"\n assert direction in ['horizontal', 'vertical', 'diagonal']\n if direction == 'horizontal':\n return np.flip(img, axis=1)\n elif direction == 'vertical':\n return np.flip(img, axis=0)\n else:\n return np.flip(img, axis=(0, 1))\n\n\ndef imflip_(img, direction='horizontal'):\n \"\"\"Inplace flip an image horizontally or vertically.\n\n Args:\n img (ndarray): Image to be flipped.\n direction (str): The flip direction, either \"horizontal\" or\n \"vertical\" or \"diagonal\".\n\n Returns:\n ndarray: The flipped image (inplace).\n \"\"\"\n assert direction in ['horizontal', 'vertical', 'diagonal']\n if direction == 'horizontal':\n return cv2.flip(img, 1, img)\n elif direction == 'vertical':\n return cv2.flip(img, 0, img)\n else:\n return cv2.flip(img, -1, img)\n\n\ndef imrotate(img,\n angle,\n center=None,\n scale=1.0,\n border_value=0,\n interpolation='bilinear',\n auto_bound=False):\n \"\"\"Rotate an image.\n\n Args:\n img (ndarray): Image to be rotated.\n angle (float): Rotation angle in degrees, positive values mean\n clockwise rotation.\n center (tuple[float], optional): Center point (w, h) of the rotation in\n the source image. If not specified, the center of the image will be\n used.\n scale (float): Isotropic scale factor.\n border_value (int): Border value.\n interpolation (str): Same as :func:`resize`.\n auto_bound (bool): Whether to adjust the image size to cover the whole\n rotated image.\n\n Returns:\n ndarray: The rotated image.\n \"\"\"\n if center is not None and auto_bound:\n raise ValueError('`auto_bound` conflicts with `center`')\n h, w = img.shape[:2]\n if center is None:\n center = ((w - 1) * 0.5, (h - 1) * 0.5)\n assert isinstance(center, tuple)\n\n matrix = cv2.getRotationMatrix2D(center, -angle, scale)\n if auto_bound:\n cos = np.abs(matrix[0, 0])\n sin = np.abs(matrix[0, 1])\n new_w = h * sin + w * cos\n new_h = h * cos + w * sin\n matrix[0, 2] += (new_w - w) * 0.5\n matrix[1, 2] += (new_h - h) * 0.5\n w = int(np.round(new_w))\n h = int(np.round(new_h))\n rotated = cv2.warpAffine(\n img,\n matrix, (w, h),\n flags=cv2_interp_codes[interpolation],\n borderValue=border_value)\n return rotated\n\n\ndef bbox_clip(bboxes, img_shape):\n \"\"\"Clip bboxes to fit the image shape.\n\n Args:\n bboxes (ndarray): Shape (..., 4*k)\n img_shape (tuple[int]): (height, width) of the image.\n\n Returns:\n ndarray: Clipped bboxes.\n \"\"\"\n assert bboxes.shape[-1] % 4 == 0\n cmin = np.empty(bboxes.shape[-1], dtype=bboxes.dtype)\n cmin[0::2] = img_shape[1] - 1\n cmin[1::2] = img_shape[0] - 1\n clipped_bboxes = np.maximum(np.minimum(bboxes, cmin), 0)\n return clipped_bboxes\n\n\ndef bbox_scaling(bboxes, scale, clip_shape=None):\n \"\"\"Scaling bboxes w.r.t the box center.\n\n Args:\n bboxes (ndarray): Shape(..., 4).\n scale (float): Scaling factor.\n clip_shape (tuple[int], optional): If specified, bboxes that exceed the\n boundary will be clipped according to the given shape (h, w).\n\n Returns:\n ndarray: Scaled bboxes.\n \"\"\"\n if float(scale) == 1.0:\n scaled_bboxes = bboxes.copy()\n else:\n w = bboxes[..., 2] - bboxes[..., 0] + 1\n h = bboxes[..., 3] - bboxes[..., 1] + 1\n dw = (w * (scale - 1)) * 0.5\n dh = (h * (scale - 1)) * 0.5\n scaled_bboxes = bboxes + np.stack((-dw, -dh, dw, dh), axis=-1)\n if clip_shape is not None:\n return bbox_clip(scaled_bboxes, clip_shape)\n else:\n return scaled_bboxes\n\n\ndef imcrop(img, bboxes, scale=1.0, pad_fill=None):\n \"\"\"Crop image patches.\n\n 3 steps: scale the bboxes -> clip bboxes -> crop and pad.\n\n Args:\n img (ndarray): Image to be cropped.\n bboxes (ndarray): Shape (k, 4) or (4, ), location of cropped bboxes.\n scale (float, optional): Scale ratio of bboxes, the default value\n 1.0 means no padding.\n pad_fill (Number | list[Number]): Value to be filled for padding.\n Default: None, which means no padding.\n\n Returns:\n list[ndarray] | ndarray: The cropped image patches.\n \"\"\"\n chn = 1 if img.ndim == 2 else img.shape[2]\n if pad_fill is not None:\n if isinstance(pad_fill, (int, float)):\n pad_fill = [pad_fill for _ in range(chn)]\n assert len(pad_fill) == chn\n\n _bboxes = bboxes[None, ...] if bboxes.ndim == 1 else bboxes\n scaled_bboxes = bbox_scaling(_bboxes, scale).astype(np.int32)\n clipped_bbox = bbox_clip(scaled_bboxes, img.shape)\n\n patches = []\n for i in range(clipped_bbox.shape[0]):\n x1, y1, x2, y2 = tuple(clipped_bbox[i, :])\n if pad_fill is None:\n patch = img[y1:y2 + 1, x1:x2 + 1, ...]\n else:\n _x1, _y1, _x2, _y2 = tuple(scaled_bboxes[i, :])\n if chn == 1:\n patch_shape = (_y2 - _y1 + 1, _x2 - _x1 + 1)\n else:\n patch_shape = (_y2 - _y1 + 1, _x2 - _x1 + 1, chn)\n patch = np.array(\n pad_fill, dtype=img.dtype) * np.ones(\n patch_shape, dtype=img.dtype)\n x_start = 0 if _x1 >= 0 else -_x1\n y_start = 0 if _y1 >= 0 else -_y1\n w = x2 - x1 + 1\n h = y2 - y1 + 1\n patch[y_start:y_start + h, x_start:x_start + w,\n ...] = img[y1:y1 + h, x1:x1 + w, ...]\n patches.append(patch)\n\n if bboxes.ndim == 1:\n return patches[0]\n else:\n return patches\n\n\ndef impad(img,\n *,\n shape=None,\n padding=None,\n pad_val=0,\n padding_mode='constant'):\n \"\"\"Pad the given image to a certain shape or pad on all sides with\n specified padding mode and padding value.\n\n Args:\n img (ndarray): Image to be padded.\n shape (tuple[int]): Expected padding shape (h, w). Default: None.\n padding (int or tuple[int]): Padding on each border. If a single int is\n provided this is used to pad all borders. If tuple of length 2 is\n provided this is the padding on left/right and top/bottom\n respectively. If a tuple of length 4 is provided this is the\n padding for the left, top, right and bottom borders respectively.\n Default: None. Note that `shape` and `padding` can not be both\n set.\n pad_val (Number | Sequence[Number]): Values to be filled in padding\n areas when padding_mode is 'constant'. Default: 0.\n padding_mode (str): Type of padding. Should be: constant, edge,\n reflect or symmetric. Default: constant.\n\n - constant: pads with a constant value, this value is specified\n with pad_val.\n - edge: pads with the last value at the edge of the image.\n - reflect: pads with reflection of image without repeating the last\n value on the edge. For example, padding [1, 2, 3, 4] with 2\n elements on both sides in reflect mode will result in\n [3, 2, 1, 2, 3, 4, 3, 2].\n - symmetric: pads with reflection of image repeating the last value\n on the edge. For example, padding [1, 2, 3, 4] with 2 elements on\n both sides in symmetric mode will result in\n [2, 1, 1, 2, 3, 4, 4, 3]\n\n Returns:\n ndarray: The padded image.\n \"\"\"\n\n assert (shape is not None) ^ (padding is not None)\n if shape is not None:\n padding = (0, 0, shape[1] - img.shape[1], shape[0] - img.shape[0])\n\n # check pad_val\n if isinstance(pad_val, tuple):\n assert len(pad_val) == img.shape[-1]\n elif not isinstance(pad_val, numbers.Number):\n raise TypeError('pad_val must be a int or a tuple. '\n f'But received {type(pad_val)}')\n\n # check padding\n if isinstance(padding, tuple) and len(padding) in [2, 4]:\n if len(padding) == 2:\n padding = (padding[0], padding[1], padding[0], padding[1])\n elif isinstance(padding, numbers.Number):\n padding = (padding, padding, padding, padding)\n else:\n raise ValueError('Padding must be a int or a 2, or 4 element tuple.'\n f'But received {padding}')\n\n # check padding mode\n assert padding_mode in ['constant', 'edge', 'reflect', 'symmetric']\n\n border_type = {\n 'constant': cv2.BORDER_CONSTANT,\n 'edge': cv2.BORDER_REPLICATE,\n 'reflect': cv2.BORDER_REFLECT_101,\n 'symmetric': cv2.BORDER_REFLECT\n }\n img = cv2.copyMakeBorder(\n img,\n padding[1],\n padding[3],\n padding[0],\n padding[2],\n border_type[padding_mode],\n value=pad_val)\n\n return img\n\n\ndef impad_to_multiple(img, divisor, pad_val=0):\n \"\"\"Pad an image to ensure each edge to be multiple to some number.\n\n Args:\n img (ndarray): Image to be padded.\n divisor (int): Padded image edges will be multiple to divisor.\n pad_val (Number | Sequence[Number]): Same as :func:`impad`.\n\n Returns:\n ndarray: The padded image.\n \"\"\"\n pad_h = int(np.ceil(img.shape[0] / divisor)) * divisor\n pad_w = int(np.ceil(img.shape[1] / divisor)) * divisor\n return impad(img, shape=(pad_h, pad_w), pad_val=pad_val)\n\n\ndef cutout(img, shape, pad_val=0):\n \"\"\"Randomly cut out a rectangle from the original img.\n\n Args:\n img (ndarray): Image to be cutout.\n shape (int | tuple[int]): Expected cutout shape (h, w). If given as a\n int, the value will be used for both h and w.\n pad_val (int | float | tuple[int | float]): Values to be filled in the\n cut area. Defaults to 0.\n\n Returns:\n ndarray: The cutout image.\n \"\"\"\n\n channels = 1 if img.ndim == 2 else img.shape[2]\n if isinstance(shape, int):\n cut_h, cut_w = shape, shape\n else:\n assert isinstance(shape, tuple) and len(shape) == 2, \\\n f'shape must be a int or a tuple with length 2, but got type ' \\\n f'{type(shape)} instead.'\n cut_h, cut_w = shape\n if isinstance(pad_val, (int, float)):\n pad_val = tuple([pad_val] * channels)\n elif isinstance(pad_val, tuple):\n assert len(pad_val) == channels, \\\n 'Expected the num of elements in tuple equals the channels' \\\n 'of input image. Found {} vs {}'.format(\n len(pad_val), channels)\n else:\n raise TypeError(f'Invalid type {type(pad_val)} for `pad_val`')\n\n img_h, img_w = img.shape[:2]\n y0 = np.random.uniform(img_h)\n x0 = np.random.uniform(img_w)\n\n y1 = int(max(0, y0 - cut_h / 2.))\n x1 = int(max(0, x0 - cut_w / 2.))\n y2 = min(img_h, y1 + cut_h)\n x2 = min(img_w, x1 + cut_w)\n\n if img.ndim == 2:\n patch_shape = (y2 - y1, x2 - x1)\n else:\n patch_shape = (y2 - y1, x2 - x1, channels)\n\n img_cutout = img.copy()\n patch = np.array(\n pad_val, dtype=img.dtype) * np.ones(\n patch_shape, dtype=img.dtype)\n img_cutout[y1:y2, x1:x2, ...] = patch\n\n return img_cutout\n\n\ndef _get_shear_matrix(magnitude, direction='horizontal'):\n \"\"\"Generate the shear matrix for transformation.\n\n Args:\n magnitude (int | float): The magnitude used for shear.\n direction (str): The flip direction, either \"horizontal\"\n or \"vertical\".\n\n Returns:\n ndarray: The shear matrix with dtype float32.\n \"\"\"\n if direction == 'horizontal':\n shear_matrix = np.float32([[1, magnitude, 0], [0, 1, 0]])\n elif direction == 'vertical':\n shear_matrix = np.float32([[1, 0, 0], [magnitude, 1, 0]])\n return shear_matrix\n\n\ndef imshear(img,\n magnitude,\n direction='horizontal',\n border_value=0,\n interpolation='bilinear'):\n \"\"\"Shear an image.\n\n Args:\n img (ndarray): Image to be sheared with format (h, w)\n or (h, w, c).\n magnitude (int | float): The magnitude used for shear.\n direction (str): The flip direction, either \"horizontal\"\n or \"vertical\".\n border_value (int | tuple[int]): Value used in case of a\n constant border.\n interpolation (str): Same as :func:`resize`.\n\n Returns:\n ndarray: The sheared image.\n \"\"\"\n assert direction in ['horizontal',\n 'vertical'], f'Invalid direction: {direction}'\n height, width = img.shape[:2]\n if img.ndim == 2:\n channels = 1\n elif img.ndim == 3:\n channels = img.shape[-1]\n if isinstance(border_value, int):\n border_value = tuple([border_value] * channels)\n elif isinstance(border_value, tuple):\n assert len(border_value) == channels, \\\n 'Expected the num of elements in tuple equals the channels' \\\n 'of input image. Found {} vs {}'.format(\n len(border_value), channels)\n else:\n raise ValueError(\n f'Invalid type {type(border_value)} for `border_value`')\n shear_matrix = _get_shear_matrix(magnitude, direction)\n sheared = cv2.warpAffine(\n img,\n shear_matrix,\n (width, height),\n # Note case when the number elements in `border_value`\n # greater than 3 (e.g. shearing masks whose channels large\n # than 3) will raise TypeError in `cv2.warpAffine`.\n # Here simply slice the first 3 values in `border_value`.\n borderValue=border_value[:3],\n flags=cv2_interp_codes[interpolation])\n return sheared\n\n\ndef _get_translate_matrix(offset, direction='horizontal'):\n \"\"\"Generate the translate matrix.\n\n Args:\n offset (int | float): The offset used for translate.\n direction (str): The translate direction, either\n \"horizontal\" or \"vertical\".\n\n Returns:\n ndarray: The translate matrix with dtype float32.\n \"\"\"\n if direction == 'horizontal':\n translate_matrix = np.float32([[1, 0, offset], [0, 1, 0]])\n elif direction == 'vertical':\n translate_matrix = np.float32([[1, 0, 0], [0, 1, offset]])\n return translate_matrix\n\n\ndef imtranslate(img,\n offset,\n direction='horizontal',\n border_value=0,\n interpolation='bilinear'):\n \"\"\"Translate an image.\n\n Args:\n img (ndarray): Image to be translated with format\n (h, w) or (h, w, c).\n offset (int | float): The offset used for translate.\n direction (str): The translate direction, either \"horizontal\"\n or \"vertical\".\n border_value (int | tuple[int]): Value used in case of a\n constant border.\n interpolation (str): Same as :func:`resize`.\n\n Returns:\n ndarray: The translated image.\n \"\"\"\n assert direction in ['horizontal',\n 'vertical'], f'Invalid direction: {direction}'\n height, width = img.shape[:2]\n if img.ndim == 2:\n channels = 1\n elif img.ndim == 3:\n channels = img.shape[-1]\n if isinstance(border_value, int):\n border_value = tuple([border_value] * channels)\n elif isinstance(border_value, tuple):\n assert len(border_value) == channels, \\\n 'Expected the num of elements in tuple equals the channels' \\\n 'of input image. Found {} vs {}'.format(\n len(border_value), channels)\n else:\n raise ValueError(\n f'Invalid type {type(border_value)} for `border_value`.')\n translate_matrix = _get_translate_matrix(offset, direction)\n translated = cv2.warpAffine(\n img,\n translate_matrix,\n (width, height),\n # Note case when the number elements in `border_value`\n # greater than 3 (e.g. translating masks whose channels\n # large than 3) will raise TypeError in `cv2.warpAffine`.\n # Here simply slice the first 3 values in `border_value`.\n borderValue=border_value[:3],\n flags=cv2_interp_codes[interpolation])\n return translated\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/image/io.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport io\nimport os.path as osp\nimport warnings\nfrom pathlib import Path\n\nimport cv2\nimport numpy as np\nfrom cv2 import (IMREAD_COLOR, IMREAD_GRAYSCALE, IMREAD_IGNORE_ORIENTATION,\n IMREAD_UNCHANGED)\n\nfrom mmcv.fileio import FileClient\nfrom mmcv.utils import is_filepath, is_str\n\ntry:\n from turbojpeg import TJCS_RGB, TJPF_BGR, TJPF_GRAY, TurboJPEG\nexcept ImportError:\n TJCS_RGB = TJPF_GRAY = TJPF_BGR = TurboJPEG = None\n\ntry:\n from PIL import Image, ImageOps\nexcept ImportError:\n Image = None\n\ntry:\n import tifffile\nexcept ImportError:\n tifffile = None\n\njpeg = None\nsupported_backends = ['cv2', 'turbojpeg', 'pillow', 'tifffile']\n\nimread_flags = {\n 'color': IMREAD_COLOR,\n 'grayscale': IMREAD_GRAYSCALE,\n 'unchanged': IMREAD_UNCHANGED,\n 'color_ignore_orientation': IMREAD_IGNORE_ORIENTATION | IMREAD_COLOR,\n 'grayscale_ignore_orientation':\n IMREAD_IGNORE_ORIENTATION | IMREAD_GRAYSCALE\n}\n\nimread_backend = 'cv2'\n\n\ndef use_backend(backend):\n \"\"\"Select a backend for image decoding.\n\n Args:\n backend (str): The image decoding backend type. Options are `cv2`,\n `pillow`, `turbojpeg` (see https://github.com/lilohuang/PyTurboJPEG)\n and `tifffile`. `turbojpeg` is faster but it only supports `.jpeg`\n file format.\n \"\"\"\n assert backend in supported_backends\n global imread_backend\n imread_backend = backend\n if imread_backend == 'turbojpeg':\n if TurboJPEG is None:\n raise ImportError('`PyTurboJPEG` is not installed')\n global jpeg\n if jpeg is None:\n jpeg = TurboJPEG()\n elif imread_backend == 'pillow':\n if Image is None:\n raise ImportError('`Pillow` is not installed')\n elif imread_backend == 'tifffile':\n if tifffile is None:\n raise ImportError('`tifffile` is not installed')\n\n\ndef _jpegflag(flag='color', channel_order='bgr'):\n channel_order = channel_order.lower()\n if channel_order not in ['rgb', 'bgr']:\n raise ValueError('channel order must be either \"rgb\" or \"bgr\"')\n\n if flag == 'color':\n if channel_order == 'bgr':\n return TJPF_BGR\n elif channel_order == 'rgb':\n return TJCS_RGB\n elif flag == 'grayscale':\n return TJPF_GRAY\n else:\n raise ValueError('flag must be \"color\" or \"grayscale\"')\n\n\ndef _pillow2array(img, flag='color', channel_order='bgr'):\n \"\"\"Convert a pillow image to numpy array.\n\n Args:\n img (:obj:`PIL.Image.Image`): The image loaded using PIL\n flag (str): Flags specifying the color type of a loaded image,\n candidates are 'color', 'grayscale' and 'unchanged'.\n Default to 'color'.\n channel_order (str): The channel order of the output image array,\n candidates are 'bgr' and 'rgb'. Default to 'bgr'.\n\n Returns:\n np.ndarray: The converted numpy array\n \"\"\"\n channel_order = channel_order.lower()\n if channel_order not in ['rgb', 'bgr']:\n raise ValueError('channel order must be either \"rgb\" or \"bgr\"')\n\n if flag == 'unchanged':\n array = np.array(img)\n if array.ndim >= 3 and array.shape[2] >= 3: # color image\n array[:, :, :3] = array[:, :, (2, 1, 0)] # RGB to BGR\n else:\n # Handle exif orientation tag\n if flag in ['color', 'grayscale']:\n img = ImageOps.exif_transpose(img)\n # If the image mode is not 'RGB', convert it to 'RGB' first.\n if img.mode != 'RGB':\n if img.mode != 'LA':\n # Most formats except 'LA' can be directly converted to RGB\n img = img.convert('RGB')\n else:\n # When the mode is 'LA', the default conversion will fill in\n # the canvas with black, which sometimes shadows black objects\n # in the foreground.\n #\n # Therefore, a random color (124, 117, 104) is used for canvas\n img_rgba = img.convert('RGBA')\n img = Image.new('RGB', img_rgba.size, (124, 117, 104))\n img.paste(img_rgba, mask=img_rgba.split()[3]) # 3 is alpha\n if flag in ['color', 'color_ignore_orientation']:\n array = np.array(img)\n if channel_order != 'rgb':\n array = array[:, :, ::-1] # RGB to BGR\n elif flag in ['grayscale', 'grayscale_ignore_orientation']:\n img = img.convert('L')\n array = np.array(img)\n else:\n raise ValueError(\n 'flag must be \"color\", \"grayscale\", \"unchanged\", '\n f'\"color_ignore_orientation\" or \"grayscale_ignore_orientation\"'\n f' but got {flag}')\n return array\n\n\ndef imread(img_or_path,\n flag='color',\n channel_order='bgr',\n backend=None,\n file_client_args=None):\n \"\"\"Read an image.\n\n Note:\n In v1.4.1 and later, add `file_client_args` parameters.\n\n Args:\n img_or_path (ndarray or str or Path): Either a numpy array or str or\n pathlib.Path. If it is a numpy array (loaded image), then\n it will be returned as is.\n flag (str): Flags specifying the color type of a loaded image,\n candidates are `color`, `grayscale`, `unchanged`,\n `color_ignore_orientation` and `grayscale_ignore_orientation`.\n By default, `cv2` and `pillow` backend would rotate the image\n according to its EXIF info unless called with `unchanged` or\n `*_ignore_orientation` flags. `turbojpeg` and `tifffile` backend\n always ignore image's EXIF info regardless of the flag.\n The `turbojpeg` backend only supports `color` and `grayscale`.\n channel_order (str): Order of channel, candidates are `bgr` and `rgb`.\n backend (str | None): The image decoding backend type. Options are\n `cv2`, `pillow`, `turbojpeg`, `tifffile`, `None`.\n If backend is None, the global imread_backend specified by\n ``mmcv.use_backend()`` will be used. Default: None.\n file_client_args (dict | None): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n\n Returns:\n ndarray: Loaded image array.\n\n Examples:\n >>> import mmcv\n >>> img_path = '/path/to/img.jpg'\n >>> img = mmcv.imread(img_path)\n >>> img = mmcv.imread(img_path, flag='color', channel_order='rgb',\n ... backend='cv2')\n >>> img = mmcv.imread(img_path, flag='color', channel_order='bgr',\n ... backend='pillow')\n >>> s3_img_path = 's3://bucket/img.jpg'\n >>> # infer the file backend by the prefix s3\n >>> img = mmcv.imread(s3_img_path)\n >>> # manually set the file backend petrel\n >>> img = mmcv.imread(s3_img_path, file_client_args={\n ... 'backend': 'petrel'})\n >>> http_img_path = 'http://path/to/img.jpg'\n >>> img = mmcv.imread(http_img_path)\n >>> img = mmcv.imread(http_img_path, file_client_args={\n ... 'backend': 'http'})\n \"\"\"\n\n if isinstance(img_or_path, Path):\n img_or_path = str(img_or_path)\n\n if isinstance(img_or_path, np.ndarray):\n return img_or_path\n elif is_str(img_or_path):\n file_client = FileClient.infer_client(file_client_args, img_or_path)\n img_bytes = file_client.get(img_or_path)\n return imfrombytes(img_bytes, flag, channel_order, backend)\n else:\n raise TypeError('\"img\" must be a numpy array or a str or '\n 'a pathlib.Path object')\n\n\ndef imfrombytes(content, flag='color', channel_order='bgr', backend=None):\n \"\"\"Read an image from bytes.\n\n Args:\n content (bytes): Image bytes got from files or other streams.\n flag (str): Same as :func:`imread`.\n backend (str | None): The image decoding backend type. Options are\n `cv2`, `pillow`, `turbojpeg`, `tifffile`, `None`. If backend is\n None, the global imread_backend specified by ``mmcv.use_backend()``\n will be used. Default: None.\n\n Returns:\n ndarray: Loaded image array.\n\n Examples:\n >>> img_path = '/path/to/img.jpg'\n >>> with open(img_path, 'rb') as f:\n >>> img_buff = f.read()\n >>> img = mmcv.imfrombytes(img_buff)\n >>> img = mmcv.imfrombytes(img_buff, flag='color', channel_order='rgb')\n >>> img = mmcv.imfrombytes(img_buff, backend='pillow')\n >>> img = mmcv.imfrombytes(img_buff, backend='cv2')\n \"\"\"\n\n if backend is None:\n backend = imread_backend\n if backend not in supported_backends:\n raise ValueError(\n f'backend: {backend} is not supported. Supported '\n \"backends are 'cv2', 'turbojpeg', 'pillow', 'tifffile'\")\n if backend == 'turbojpeg':\n img = jpeg.decode(content, _jpegflag(flag, channel_order))\n if img.shape[-1] == 1:\n img = img[:, :, 0]\n return img\n elif backend == 'pillow':\n with io.BytesIO(content) as buff:\n img = Image.open(buff)\n img = _pillow2array(img, flag, channel_order)\n return img\n elif backend == 'tifffile':\n with io.BytesIO(content) as buff:\n img = tifffile.imread(buff)\n return img\n else:\n img_np = np.frombuffer(content, np.uint8)\n flag = imread_flags[flag] if is_str(flag) else flag\n img = cv2.imdecode(img_np, flag)\n if flag == IMREAD_COLOR and channel_order == 'rgb':\n cv2.cvtColor(img, cv2.COLOR_BGR2RGB, img)\n return img\n\n\ndef imwrite(img,\n file_path,\n params=None,\n auto_mkdir=None,\n file_client_args=None):\n \"\"\"Write image to file.\n\n Note:\n In v1.4.1 and later, add `file_client_args` parameters.\n\n Warning:\n The parameter `auto_mkdir` will be deprecated in the future and every\n file clients will make directory automatically.\n\n Args:\n img (ndarray): Image array to be written.\n file_path (str): Image file path.\n params (None or list): Same as opencv :func:`imwrite` interface.\n auto_mkdir (bool): If the parent folder of `file_path` does not exist,\n whether to create it automatically. It will be deprecated.\n file_client_args (dict | None): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n\n Returns:\n bool: Successful or not.\n\n Examples:\n >>> # write to hard disk client\n >>> ret = mmcv.imwrite(img, '/path/to/img.jpg')\n >>> # infer the file backend by the prefix s3\n >>> ret = mmcv.imwrite(img, 's3://bucket/img.jpg')\n >>> # manually set the file backend petrel\n >>> ret = mmcv.imwrite(img, 's3://bucket/img.jpg', file_client_args={\n ... 'backend': 'petrel'})\n \"\"\"\n assert is_filepath(file_path)\n file_path = str(file_path)\n if auto_mkdir is not None:\n warnings.warn(\n 'The parameter `auto_mkdir` will be deprecated in the future and '\n 'every file clients will make directory automatically.')\n file_client = FileClient.infer_client(file_client_args, file_path)\n img_ext = osp.splitext(file_path)[-1]\n # Encode image according to image suffix.\n # For example, if image path is '/path/your/img.jpg', the encode\n # format is '.jpg'.\n flag, img_buff = cv2.imencode(img_ext, img, params)\n file_client.put(img_buff.tobytes(), file_path)\n return flag\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/image/misc.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numpy as np\n\nimport mmcv\n\ntry:\n import torch\nexcept ImportError:\n torch = None\n\n\ndef tensor2imgs(tensor, mean=None, std=None, to_rgb=True):\n \"\"\"Convert tensor to 3-channel images or 1-channel gray images.\n\n Args:\n tensor (torch.Tensor): Tensor that contains multiple images, shape (\n N, C, H, W). :math:`C` can be either 3 or 1.\n mean (tuple[float], optional): Mean of images. If None,\n (0, 0, 0) will be used for tensor with 3-channel,\n while (0, ) for tensor with 1-channel. Defaults to None.\n std (tuple[float], optional): Standard deviation of images. If None,\n (1, 1, 1) will be used for tensor with 3-channel,\n while (1, ) for tensor with 1-channel. Defaults to None.\n to_rgb (bool, optional): Whether the tensor was converted to RGB\n format in the first place. If so, convert it back to BGR.\n For the tensor with 1 channel, it must be False. Defaults to True.\n\n Returns:\n list[np.ndarray]: A list that contains multiple images.\n \"\"\"\n\n if torch is None:\n raise RuntimeError('pytorch is not installed')\n assert torch.is_tensor(tensor) and tensor.ndim == 4\n channels = tensor.size(1)\n assert channels in [1, 3]\n if mean is None:\n mean = (0, ) * channels\n if std is None:\n std = (1, ) * channels\n assert (channels == len(mean) == len(std) == 3) or \\\n (channels == len(mean) == len(std) == 1 and not to_rgb)\n\n num_imgs = tensor.size(0)\n mean = np.array(mean, dtype=np.float32)\n std = np.array(std, dtype=np.float32)\n imgs = []\n for img_id in range(num_imgs):\n img = tensor[img_id, ...].cpu().numpy().transpose(1, 2, 0)\n img = mmcv.imdenormalize(\n img, mean, std, to_bgr=to_rgb).astype(np.uint8)\n imgs.append(np.ascontiguousarray(img))\n return imgs\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/image/photometric.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport cv2\nimport numpy as np\n\nfrom ..utils import is_tuple_of\nfrom .colorspace import bgr2gray, gray2bgr\n\n\ndef imnormalize(img, mean, std, to_rgb=True):\n \"\"\"Normalize an image with mean and std.\n\n Args:\n img (ndarray): Image to be normalized.\n mean (ndarray): The mean to be used for normalize.\n std (ndarray): The std to be used for normalize.\n to_rgb (bool): Whether to convert to rgb.\n\n Returns:\n ndarray: The normalized image.\n \"\"\"\n img = img.copy().astype(np.float32)\n return imnormalize_(img, mean, std, to_rgb)\n\n\ndef imnormalize_(img, mean, std, to_rgb=True):\n \"\"\"Inplace normalize an image with mean and std.\n\n Args:\n img (ndarray): Image to be normalized.\n mean (ndarray): The mean to be used for normalize.\n std (ndarray): The std to be used for normalize.\n to_rgb (bool): Whether to convert to rgb.\n\n Returns:\n ndarray: The normalized image.\n \"\"\"\n # cv2 inplace normalization does not accept uint8\n assert img.dtype != np.uint8\n mean = np.float64(mean.reshape(1, -1))\n stdinv = 1 / np.float64(std.reshape(1, -1))\n if to_rgb:\n cv2.cvtColor(img, cv2.COLOR_BGR2RGB, img) # inplace\n cv2.subtract(img, mean, img) # inplace\n cv2.multiply(img, stdinv, img) # inplace\n return img\n\n\ndef imdenormalize(img, mean, std, to_bgr=True):\n assert img.dtype != np.uint8\n mean = mean.reshape(1, -1).astype(np.float64)\n std = std.reshape(1, -1).astype(np.float64)\n img = cv2.multiply(img, std) # make a copy\n cv2.add(img, mean, img) # inplace\n if to_bgr:\n cv2.cvtColor(img, cv2.COLOR_RGB2BGR, img) # inplace\n return img\n\n\ndef iminvert(img):\n \"\"\"Invert (negate) an image.\n\n Args:\n img (ndarray): Image to be inverted.\n\n Returns:\n ndarray: The inverted image.\n \"\"\"\n return np.full_like(img, 255) - img\n\n\ndef solarize(img, thr=128):\n \"\"\"Solarize an image (invert all pixel values above a threshold)\n\n Args:\n img (ndarray): Image to be solarized.\n thr (int): Threshold for solarizing (0 - 255).\n\n Returns:\n ndarray: The solarized image.\n \"\"\"\n img = np.where(img < thr, img, 255 - img)\n return img\n\n\ndef posterize(img, bits):\n \"\"\"Posterize an image (reduce the number of bits for each color channel)\n\n Args:\n img (ndarray): Image to be posterized.\n bits (int): Number of bits (1 to 8) to use for posterizing.\n\n Returns:\n ndarray: The posterized image.\n \"\"\"\n shift = 8 - bits\n img = np.left_shift(np.right_shift(img, shift), shift)\n return img\n\n\ndef adjust_color(img, alpha=1, beta=None, gamma=0):\n r\"\"\"It blends the source image and its gray image:\n\n .. math::\n output = img * alpha + gray\\_img * beta + gamma\n\n Args:\n img (ndarray): The input source image.\n alpha (int | float): Weight for the source image. Default 1.\n beta (int | float): Weight for the converted gray image.\n If None, it's assigned the value (1 - `alpha`).\n gamma (int | float): Scalar added to each sum.\n Same as :func:`cv2.addWeighted`. Default 0.\n\n Returns:\n ndarray: Colored image which has the same size and dtype as input.\n \"\"\"\n gray_img = bgr2gray(img)\n gray_img = np.tile(gray_img[..., None], [1, 1, 3])\n if beta is None:\n beta = 1 - alpha\n colored_img = cv2.addWeighted(img, alpha, gray_img, beta, gamma)\n if not colored_img.dtype == np.uint8:\n # Note when the dtype of `img` is not the default `np.uint8`\n # (e.g. np.float32), the value in `colored_img` got from cv2\n # is not guaranteed to be in range [0, 255], so here clip\n # is needed.\n colored_img = np.clip(colored_img, 0, 255)\n return colored_img\n\n\ndef imequalize(img):\n \"\"\"Equalize the image histogram.\n\n This function applies a non-linear mapping to the input image,\n in order to create a uniform distribution of grayscale values\n in the output image.\n\n Args:\n img (ndarray): Image to be equalized.\n\n Returns:\n ndarray: The equalized image.\n \"\"\"\n\n def _scale_channel(im, c):\n \"\"\"Scale the data in the corresponding channel.\"\"\"\n im = im[:, :, c]\n # Compute the histogram of the image channel.\n histo = np.histogram(im, 256, (0, 255))[0]\n # For computing the step, filter out the nonzeros.\n nonzero_histo = histo[histo > 0]\n step = (np.sum(nonzero_histo) - nonzero_histo[-1]) // 255\n if not step:\n lut = np.array(range(256))\n else:\n # Compute the cumulative sum, shifted by step // 2\n # and then normalized by step.\n lut = (np.cumsum(histo) + (step // 2)) // step\n # Shift lut, prepending with 0.\n lut = np.concatenate([[0], lut[:-1]], 0)\n # handle potential integer overflow\n lut[lut > 255] = 255\n # If step is zero, return the original image.\n # Otherwise, index from lut.\n return np.where(np.equal(step, 0), im, lut[im])\n\n # Scales each channel independently and then stacks\n # the result.\n s1 = _scale_channel(img, 0)\n s2 = _scale_channel(img, 1)\n s3 = _scale_channel(img, 2)\n equalized_img = np.stack([s1, s2, s3], axis=-1)\n return equalized_img.astype(img.dtype)\n\n\ndef adjust_brightness(img, factor=1.):\n \"\"\"Adjust image brightness.\n\n This function controls the brightness of an image. An\n enhancement factor of 0.0 gives a black image.\n A factor of 1.0 gives the original image. This function\n blends the source image and the degenerated black image:\n\n .. math::\n output = img * factor + degenerated * (1 - factor)\n\n Args:\n img (ndarray): Image to be brightened.\n factor (float): A value controls the enhancement.\n Factor 1.0 returns the original image, lower\n factors mean less color (brightness, contrast,\n etc), and higher values more. Default 1.\n\n Returns:\n ndarray: The brightened image.\n \"\"\"\n degenerated = np.zeros_like(img)\n # Note manually convert the dtype to np.float32, to\n # achieve as close results as PIL.ImageEnhance.Brightness.\n # Set beta=1-factor, and gamma=0\n brightened_img = cv2.addWeighted(\n img.astype(np.float32), factor, degenerated.astype(np.float32),\n 1 - factor, 0)\n brightened_img = np.clip(brightened_img, 0, 255)\n return brightened_img.astype(img.dtype)\n\n\ndef adjust_contrast(img, factor=1.):\n \"\"\"Adjust image contrast.\n\n This function controls the contrast of an image. An\n enhancement factor of 0.0 gives a solid grey\n image. A factor of 1.0 gives the original image. It\n blends the source image and the degenerated mean image:\n\n .. math::\n output = img * factor + degenerated * (1 - factor)\n\n Args:\n img (ndarray): Image to be contrasted. BGR order.\n factor (float): Same as :func:`mmcv.adjust_brightness`.\n\n Returns:\n ndarray: The contrasted image.\n \"\"\"\n gray_img = bgr2gray(img)\n hist = np.histogram(gray_img, 256, (0, 255))[0]\n mean = round(np.sum(gray_img) / np.sum(hist))\n degenerated = (np.ones_like(img[..., 0]) * mean).astype(img.dtype)\n degenerated = gray2bgr(degenerated)\n contrasted_img = cv2.addWeighted(\n img.astype(np.float32), factor, degenerated.astype(np.float32),\n 1 - factor, 0)\n contrasted_img = np.clip(contrasted_img, 0, 255)\n return contrasted_img.astype(img.dtype)\n\n\ndef auto_contrast(img, cutoff=0):\n \"\"\"Auto adjust image contrast.\n\n This function maximize (normalize) image contrast by first removing cutoff\n percent of the lightest and darkest pixels from the histogram and remapping\n the image so that the darkest pixel becomes black (0), and the lightest\n becomes white (255).\n\n Args:\n img (ndarray): Image to be contrasted. BGR order.\n cutoff (int | float | tuple): The cutoff percent of the lightest and\n darkest pixels to be removed. If given as tuple, it shall be\n (low, high). Otherwise, the single value will be used for both.\n Defaults to 0.\n\n Returns:\n ndarray: The contrasted image.\n \"\"\"\n\n def _auto_contrast_channel(im, c, cutoff):\n im = im[:, :, c]\n # Compute the histogram of the image channel.\n histo = np.histogram(im, 256, (0, 255))[0]\n # Remove cut-off percent pixels from histo\n histo_sum = np.cumsum(histo)\n cut_low = histo_sum[-1] * cutoff[0] // 100\n cut_high = histo_sum[-1] - histo_sum[-1] * cutoff[1] // 100\n histo_sum = np.clip(histo_sum, cut_low, cut_high) - cut_low\n histo = np.concatenate([[histo_sum[0]], np.diff(histo_sum)], 0)\n\n # Compute mapping\n low, high = np.nonzero(histo)[0][0], np.nonzero(histo)[0][-1]\n # If all the values have been cut off, return the origin img\n if low >= high:\n return im\n scale = 255.0 / (high - low)\n offset = -low * scale\n lut = np.array(range(256))\n lut = lut * scale + offset\n lut = np.clip(lut, 0, 255)\n return lut[im]\n\n if isinstance(cutoff, (int, float)):\n cutoff = (cutoff, cutoff)\n else:\n assert isinstance(cutoff, tuple), 'cutoff must be of type int, ' \\\n f'float or tuple, but got {type(cutoff)} instead.'\n # Auto adjusts contrast for each channel independently and then stacks\n # the result.\n s1 = _auto_contrast_channel(img, 0, cutoff)\n s2 = _auto_contrast_channel(img, 1, cutoff)\n s3 = _auto_contrast_channel(img, 2, cutoff)\n contrasted_img = np.stack([s1, s2, s3], axis=-1)\n return contrasted_img.astype(img.dtype)\n\n\ndef adjust_sharpness(img, factor=1., kernel=None):\n \"\"\"Adjust image sharpness.\n\n This function controls the sharpness of an image. An\n enhancement factor of 0.0 gives a blurred image. A\n factor of 1.0 gives the original image. And a factor\n of 2.0 gives a sharpened image. It blends the source\n image and the degenerated mean image:\n\n .. math::\n output = img * factor + degenerated * (1 - factor)\n\n Args:\n img (ndarray): Image to be sharpened. BGR order.\n factor (float): Same as :func:`mmcv.adjust_brightness`.\n kernel (np.ndarray, optional): Filter kernel to be applied on the img\n to obtain the degenerated img. Defaults to None.\n\n Note:\n No value sanity check is enforced on the kernel set by users. So with\n an inappropriate kernel, the ``adjust_sharpness`` may fail to perform\n the function its name indicates but end up performing whatever\n transform determined by the kernel.\n\n Returns:\n ndarray: The sharpened image.\n \"\"\"\n\n if kernel is None:\n # adopted from PIL.ImageFilter.SMOOTH\n kernel = np.array([[1., 1., 1.], [1., 5., 1.], [1., 1., 1.]]) / 13\n assert isinstance(kernel, np.ndarray), \\\n f'kernel must be of type np.ndarray, but got {type(kernel)} instead.'\n assert kernel.ndim == 2, \\\n f'kernel must have a dimension of 2, but got {kernel.ndim} instead.'\n\n degenerated = cv2.filter2D(img, -1, kernel)\n sharpened_img = cv2.addWeighted(\n img.astype(np.float32), factor, degenerated.astype(np.float32),\n 1 - factor, 0)\n sharpened_img = np.clip(sharpened_img, 0, 255)\n return sharpened_img.astype(img.dtype)\n\n\ndef adjust_lighting(img, eigval, eigvec, alphastd=0.1, to_rgb=True):\n \"\"\"AlexNet-style PCA jitter.\n\n This data augmentation is proposed in `ImageNet Classification with Deep\n Convolutional Neural Networks\n `_.\n\n Args:\n img (ndarray): Image to be adjusted lighting. BGR order.\n eigval (ndarray): the eigenvalue of the convariance matrix of pixel\n values, respectively.\n eigvec (ndarray): the eigenvector of the convariance matrix of pixel\n values, respectively.\n alphastd (float): The standard deviation for distribution of alpha.\n Defaults to 0.1\n to_rgb (bool): Whether to convert img to rgb.\n\n Returns:\n ndarray: The adjusted image.\n \"\"\"\n assert isinstance(eigval, np.ndarray) and isinstance(eigvec, np.ndarray), \\\n f'eigval and eigvec should both be of type np.ndarray, got ' \\\n f'{type(eigval)} and {type(eigvec)} instead.'\n\n assert eigval.ndim == 1 and eigvec.ndim == 2\n assert eigvec.shape == (3, eigval.shape[0])\n n_eigval = eigval.shape[0]\n assert isinstance(alphastd, float), 'alphastd should be of type float, ' \\\n f'got {type(alphastd)} instead.'\n\n img = img.copy().astype(np.float32)\n if to_rgb:\n cv2.cvtColor(img, cv2.COLOR_BGR2RGB, img) # inplace\n\n alpha = np.random.normal(0, alphastd, n_eigval)\n alter = eigvec \\\n * np.broadcast_to(alpha.reshape(1, n_eigval), (3, n_eigval)) \\\n * np.broadcast_to(eigval.reshape(1, n_eigval), (3, n_eigval))\n alter = np.broadcast_to(alter.sum(axis=1).reshape(1, 1, 3), img.shape)\n img_adjusted = img + alter\n return img_adjusted\n\n\ndef lut_transform(img, lut_table):\n \"\"\"Transform array by look-up table.\n\n The function lut_transform fills the output array with values from the\n look-up table. Indices of the entries are taken from the input array.\n\n Args:\n img (ndarray): Image to be transformed.\n lut_table (ndarray): look-up table of 256 elements; in case of\n multi-channel input array, the table should either have a single\n channel (in this case the same table is used for all channels) or\n the same number of channels as in the input array.\n\n Returns:\n ndarray: The transformed image.\n \"\"\"\n assert isinstance(img, np.ndarray)\n assert 0 <= np.min(img) and np.max(img) <= 255\n assert isinstance(lut_table, np.ndarray)\n assert lut_table.shape == (256, )\n\n return cv2.LUT(np.array(img, dtype=np.uint8), lut_table)\n\n\ndef clahe(img, clip_limit=40.0, tile_grid_size=(8, 8)):\n \"\"\"Use CLAHE method to process the image.\n\n See `ZUIDERVELD,K. Contrast Limited Adaptive Histogram Equalization[J].\n Graphics Gems, 1994:474-485.` for more information.\n\n Args:\n img (ndarray): Image to be processed.\n clip_limit (float): Threshold for contrast limiting. Default: 40.0.\n tile_grid_size (tuple[int]): Size of grid for histogram equalization.\n Input image will be divided into equally sized rectangular tiles.\n It defines the number of tiles in row and column. Default: (8, 8).\n\n Returns:\n ndarray: The processed image.\n \"\"\"\n assert isinstance(img, np.ndarray)\n assert img.ndim == 2\n assert isinstance(clip_limit, (float, int))\n assert is_tuple_of(tile_grid_size, int)\n assert len(tile_grid_size) == 2\n\n clahe = cv2.createCLAHE(clip_limit, tile_grid_size)\n return clahe.apply(np.array(img, dtype=np.uint8))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/onnx/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .info import is_custom_op_loaded\nfrom .symbolic import register_extra_symbolics\n\n__all__ = ['register_extra_symbolics', 'is_custom_op_loaded']\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/onnx/info.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\n\nimport torch\n\n\ndef is_custom_op_loaded():\n flag = False\n try:\n from ..tensorrt import is_tensorrt_plugin_loaded\n flag = is_tensorrt_plugin_loaded()\n except (ImportError, ModuleNotFoundError):\n pass\n if not flag:\n try:\n from ..ops import get_onnxruntime_op_path\n ort_lib_path = get_onnxruntime_op_path()\n flag = os.path.exists(ort_lib_path)\n except (ImportError, ModuleNotFoundError):\n pass\n return flag or torch.__version__ == 'parrots'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/onnx/onnx_utils/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/onnx/onnx_utils/symbolic_helper.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n\"\"\"Modified from https://github.com/pytorch/pytorch.\"\"\"\nimport warnings\nfrom functools import wraps\nfrom sys import maxsize\n\nimport torch\nimport torch.onnx\n# This import monkey-patches graph manipulation methods on Graph, used for the\n# ONNX symbolics\nimport torch.onnx.utils\nfrom torch._C import ListType\n\n# ---------------------------------------------------------------------------------\n# Helper functions\n# ---------------------------------------------------------------------------------\n\n# Save some builtins as locals, because we'll shadown them below\n_sum = sum\n\n\ndef _parse_arg(value, desc):\n if desc == 'none':\n return value\n if desc == 'v' or not _is_value(value):\n return value\n if value.node().mustBeNone():\n return None\n if value.node().kind() == 'onnx::Constant':\n tval = value.node()['value']\n if desc == 'i':\n return int(tval)\n elif desc == 'f':\n return float(tval)\n elif desc == 'b':\n return bool(tval)\n elif desc == 's':\n return str(tval)\n elif desc == 't':\n return tval\n elif desc == 'is':\n return [int(v) for v in tval]\n elif desc == 'fs':\n return [float(v) for v in tval]\n else:\n raise RuntimeError(\n \"ONNX symbolic doesn't know to interpret Constant node\")\n elif value.node().kind() == 'prim::ListConstruct':\n if desc == 'is':\n for v in value.node().inputs():\n if v.node().kind() != 'onnx::Constant':\n raise RuntimeError(\n \"Failed to export an ONNX attribute '\" +\n v.node().kind() +\n \"', since it's not constant, please try to make \"\n 'things (e.g., kernel size) static if possible')\n return [int(v.node()['value']) for v in value.node().inputs()]\n else:\n raise RuntimeError(\n \"ONNX symbolic doesn't know to interpret ListConstruct node\")\n\n raise RuntimeError('Unexpected node type: {}'.format(value.node().kind()))\n\n\ndef _maybe_get_const(value, desc):\n if _is_value(value) and value.node().kind() == 'onnx::Constant':\n return _parse_arg(value, desc)\n return value\n\n\ndef _maybe_get_scalar(value):\n value_t = _maybe_get_const(value, 't')\n if isinstance(value_t, torch.Tensor) and value_t.shape == ():\n return value_t\n return value\n\n\ndef _get_const(value, desc, arg_name):\n if _is_value(value) and value.node().kind() not in ('onnx::Constant',\n 'prim::Constant'):\n raise RuntimeError('ONNX symbolic expected a constant'\n ' value of the {} argument, got `{}`'.format(\n arg_name, value))\n return _parse_arg(value, desc)\n\n\ndef _unpack_list(list_value):\n list_node = list_value.node()\n assert list_node.kind() == 'prim::ListConstruct'\n return list(list_node.inputs())\n\n\n# Check if list_value is output from prim::ListConstruct\n# This is usually called before _unpack_list to ensure the list can be\n# unpacked.\ndef _is_packed_list(list_value):\n return _is_value(\n list_value) and list_value.node().kind() == 'prim::ListConstruct'\n\n\ndef parse_args(*arg_descriptors):\n\n def decorator(fn):\n fn._arg_descriptors = arg_descriptors\n\n def wrapper(g, *args):\n # some args may be optional, so the length may be smaller\n assert len(arg_descriptors) >= len(args)\n args = [\n _parse_arg(arg, arg_desc)\n for arg, arg_desc in zip(args, arg_descriptors)\n ]\n return fn(g, *args)\n\n # In Python 2 functools.wraps chokes on partially applied functions, so\n # we need this as a workaround\n try:\n wrapper = wraps(fn)(wrapper)\n except Exception:\n pass\n return wrapper\n\n return decorator\n\n\ndef _scalar(x):\n \"\"\"Convert a scalar tensor into a Python value.\"\"\"\n assert x.numel() == 1\n return x.item()\n\n\ndef _if_scalar_type_as(g, self, tensor):\n \"\"\"Convert self into the same type of tensor, as necessary.\"\"\"\n if isinstance(self, torch._C.Value):\n return self\n\n scalar_type = tensor.type().scalarType()\n if scalar_type:\n ty = scalar_type.lower()\n return getattr(self, ty)()\n\n return self\n\n\ndef _is_none(x):\n return x.node().mustBeNone()\n\n\ndef _is_value(x):\n return isinstance(x, torch._C.Value)\n\n\ndef _is_tensor_list(x):\n return x.type().isSubtypeOf(ListType.ofTensors())\n\n\ndef _unimplemented(op, msg):\n warnings.warn('ONNX export failed on ' + op + ' because ' + msg +\n ' not supported')\n\n\ndef _try_get_scalar_type(*args):\n for arg in args:\n try:\n return arg.type().scalarType()\n except RuntimeError:\n pass\n return None\n\n\ndef _topk_helper(g, input, k, dim, largest=True, sorted=False, out=None):\n if out is not None:\n _unimplemented('TopK', 'Out parameter is not supported')\n if not _is_value(k):\n k = g.op('Constant', value_t=torch.tensor([k], dtype=torch.int64))\n else:\n k = g.op('Reshape', k, g.op('Constant', value_t=torch.tensor([1])))\n return g.op(\n 'TopK',\n input,\n k,\n axis_i=dim,\n largest_i=largest,\n sorted_i=sorted,\n outputs=2)\n\n\ndef _slice_helper(g,\n input,\n axes,\n starts,\n ends,\n steps=None,\n dynamic_slice=False):\n # TODO(ruobing): add support for opset<10\n from torch.onnx.symbolic_opset10 import _slice\n return _slice(g, input, axes, starts, ends, steps, dynamic_slice)\n\n\ndef _unsqueeze_helper(g, input, dim):\n from torch.onnx.symbolic_opset9 import unsqueeze\n return unsqueeze(g, input, dim)\n\n\ndef _interpolate_size_to_scales(g, input, output_size, dim):\n output_size = _maybe_get_const(output_size, 'is')\n if _is_value(output_size):\n offset = 2\n offsets = g.op(\n 'Constant', value_t=torch.ones(offset, dtype=torch.float32))\n dividend = g.op(\n 'Cast', output_size, to_i=cast_pytorch_to_onnx['Float'])\n divisor = _slice_helper(\n g, g.op('Shape', input), axes=[0], ends=[maxsize], starts=[offset])\n divisor = g.op('Cast', divisor, to_i=cast_pytorch_to_onnx['Float'])\n scale_dims = g.op('Div', dividend, divisor)\n scales = g.op('Concat', offsets, scale_dims, axis_i=0)\n else:\n scales_constant = [\n 1. if i < 2 else float(output_size[-(dim - i)]) /\n float(input.type().sizes()[-(dim - i)]) for i in range(0, dim)\n ]\n scales = g.op(\n 'Constant',\n value_t=torch.tensor(scales_constant, dtype=torch.float32))\n return scales\n\n\ndef _interpolate_get_scales_if_available(g, scales):\n if len(scales) == 0:\n return None\n # scales[0] is NoneType in Pytorch == 1.5.1\n # scales[0] is TensorType with sizes = [] in Pytorch == 1.6.0\n # scales[0] is ListType in Pytorch == 1.7.0\n # scales[0] is TensorType with sizes = [2] in Pytorch == 1.8.0\n scale_desc = 'fs' if scales[0].type().kind() == 'ListType' or (\n scales[0].type().kind() == 'TensorType' and\n (sum(scales[0].type().sizes()) > 1)) else 'f'\n available_scales = _maybe_get_const(\n scales[0], scale_desc) != -1 and not _is_none(scales[0])\n\n if not available_scales:\n return None\n\n offsets = g.op('Constant', value_t=torch.ones(2, dtype=torch.float32))\n if scale_desc == 'fs':\n scales_list = g.op(\n 'Constant',\n value_t=torch.tensor(_maybe_get_const(scales[0], scale_desc)))\n # modify to support PyTorch==1.7.0\n # https://github.com/pytorch/pytorch/blob/75ee5756715e7161314ce037474843b68f69fc04/torch/onnx/symbolic_helper.py#L375 # noqa: E501\n scales = g.op('Concat', offsets, scales_list, axis_i=0)\n else:\n # for PyTorch < 1.7.0\n scales_list = []\n for scale in scales:\n unsqueezed_scale = _unsqueeze_helper(g, scale, 0)\n # ONNX only supports float for the scales. double -> float.\n unsqueezed_scale = g.op(\n 'Cast', unsqueezed_scale, to_i=cast_pytorch_to_onnx['Float'])\n scales_list.append(unsqueezed_scale)\n scales = g.op('Concat', offsets, *scales_list, axis_i=0)\n return scales\n\n\ndef _get_interpolate_attributes(g, mode, args):\n if mode == 'nearest':\n align_corners = None\n scales = args[0:]\n else:\n align_corners = args[0]\n scales = args[1:]\n scales = _interpolate_get_scales_if_available(g, scales)\n return scales, align_corners\n\n\ndef _interpolate_get_scales(g, scale_factor, dim):\n offsets = g.op('Constant', value_t=torch.ones(2, dtype=torch.float32))\n if isinstance(scale_factor.type(), torch._C.ListType):\n return g.op('Concat', offsets, scale_factor, axis_i=0)\n else:\n scale_factor = _unsqueeze_helper(g, scale_factor, 0)\n scale_factor = g.op(\n 'Cast', scale_factor, to_i=cast_pytorch_to_onnx['Float'])\n scales = [scale_factor for i in range(dim - 2)]\n scale_factor = g.op('Concat', offsets, *scales, axis_i=0)\n return scale_factor\n\n\ndef _size_helper(g, self, dim):\n full_shape = g.op('Shape', self)\n from torch.onnx.symbolic_opset9 import select\n return select(g, full_shape, g.op('Constant', value_t=torch.tensor([0])),\n dim)\n\n\ndef _avgpool_helper(tuple_fn, padding, kernel_size, stride, divisor_override,\n name):\n if divisor_override and divisor_override.node().kind() != 'prim::Constant':\n return _unimplemented(name, 'divisor_override')\n if not stride:\n stride = kernel_size\n padding = tuple(tuple_fn(padding))\n return padding\n\n\n# Metaprogram symbolics for each ATen native specialized cast operator.\n# For e.g. we specify a function named `_cast_uint8_t` that instantiates an\n# ONNX cast node with `to` attribute 'UINT8'\n#\n# TODO: remove these once we support Type's in the JIT IR and we can once again\n# use the unified toType operator\ncast_pytorch_to_onnx = {\n 'Byte': torch.onnx.TensorProtoDataType.UINT8,\n 'Char': torch.onnx.TensorProtoDataType.INT8,\n 'Double': torch.onnx.TensorProtoDataType.DOUBLE,\n 'Float': torch.onnx.TensorProtoDataType.FLOAT,\n 'Half': torch.onnx.TensorProtoDataType.FLOAT16,\n 'Int': torch.onnx.TensorProtoDataType.INT32,\n 'Long': torch.onnx.TensorProtoDataType.INT64,\n 'Short': torch.onnx.TensorProtoDataType.INT16,\n 'Bool': torch.onnx.TensorProtoDataType.BOOL,\n 'ComplexFloat': torch.onnx.TensorProtoDataType.COMPLEX64,\n 'ComplexDouble': torch.onnx.TensorProtoDataType.COMPLEX128,\n 'Undefined': torch.onnx.TensorProtoDataType.UNDEFINED,\n}\n\n# Global set to store the list of quantized operators in the network.\n# This is currently only used in the conversion of quantized ops from PT\n# -> C2 via ONNX.\n_quantized_ops = set()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/onnx/symbolic.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n\"\"\"Modified from https://github.com/pytorch/pytorch.\"\"\"\nimport os\n\nimport numpy as np\nimport torch\nfrom torch.nn.modules.utils import _pair, _single, _triple\nfrom torch.onnx.symbolic_helper import parse_args\nfrom torch.onnx.symbolic_registry import register_op\n\nfrom .onnx_utils import symbolic_helper as sym_help\n\n\ndef _interpolate(name, dim, interpolate_mode):\n\n def symbolic_fn(g, input, output_size, *args):\n scales, align_corners = sym_help._get_interpolate_attributes(\n g, interpolate_mode, args)\n align_corners = sym_help._maybe_get_scalar(align_corners)\n transformation_mode = 'asymmetric' \\\n if interpolate_mode == 'nearest' \\\n else 'align_corners' if align_corners else 'pytorch_half_pixel'\n empty_tensor = g.op(\n 'Constant', value_t=torch.tensor([], dtype=torch.float32))\n\n if scales is None:\n if 'ONNX_BACKEND' in os.environ and os.environ[\n 'ONNX_BACKEND'] == 'TensorRT':\n input_size = input.type().sizes()\n # slice the first two dim\n input_size = input_size[:2]\n # convert output_size to int type\n output_size = sym_help._maybe_get_const(output_size, 'is')\n input_size.extend(output_size)\n output_size = g.op(\n 'Constant',\n value_t=torch.tensor(input_size, dtype=torch.int64))\n else:\n input_size = g.op('Shape', input)\n input_size_beg = sym_help._slice_helper(\n g, input_size, axes=[0], ends=[2], starts=[0])\n output_size = g.op(\n 'Cast',\n output_size,\n to_i=sym_help.cast_pytorch_to_onnx['Long'])\n output_size = g.op(\n 'Concat', input_size_beg, output_size, axis_i=0)\n scales = g.op(\n 'Constant', value_t=torch.tensor([], dtype=torch.float32))\n return g.op(\n 'Resize',\n input,\n empty_tensor,\n # roi only takes effect with\n # coordinate_transformation_mode=\"tf_crop_and_resize\"\n scales, # scales is not needed since we are sending out_size\n output_size,\n coordinate_transformation_mode_s=transformation_mode,\n cubic_coeff_a_f=-0.75, # only valid when mode=\"cubic\"\n mode_s=interpolate_mode, # nearest, linear, or cubic\n nearest_mode_s='floor') # only valid when mode=\"nearest\"\n else:\n return g.op(\n 'Resize',\n input,\n empty_tensor,\n # roi only takes effect with\n # coordinate_transformation_mode=\"tf_crop_and_resize\"\n scales, # scales is not needed since we are sending out_size\n coordinate_transformation_mode_s=transformation_mode,\n cubic_coeff_a_f=-0.75, # only valid when mode=\"cubic\"\n mode_s=interpolate_mode, # nearest, linear, or cubic\n nearest_mode_s='floor') # only valid when mode=\"nearest\"\n\n return symbolic_fn\n\n\nupsample_nearest1d = _interpolate('upsample_nearest1d', 3, 'nearest')\nupsample_nearest2d = _interpolate('upsample_nearest2d', 4, 'nearest')\nupsample_nearest3d = _interpolate('upsample_nearest3d', 5, 'nearest')\nupsample_linear1d = _interpolate('upsample_linear1d', 3, 'linear')\nupsample_bilinear2d = _interpolate('upsample_bilinear2d', 4, 'linear')\nupsample_trilinear3d = _interpolate('upsample_trilinear3d', 5, 'linear')\nupsample_bicubic2d = _interpolate('upsample_bicubic2d', 4, 'cubic')\n\n\n@parse_args('v', 'v', 'i', 'i', 'i', 'none')\ndef topk(g, self, k, dim, largest, sorted, out=None):\n return sym_help._topk_helper(\n g, self, k, dim, largest=largest, sorted=sorted, out=out)\n\n\ndef masked_select(g, self, mask):\n from torch.onnx.symbolic_opset9 import expand_as, nonzero\n index = nonzero(g, expand_as(g, mask, self))\n return g.op('GatherND', self, index)\n\n\ndef _prepare_onnx_paddings(g, dim, pad):\n pad_len = torch.onnx.symbolic_opset9.size(\n g, pad, g.op('Constant', value_t=torch.tensor([0])))\n # Set extension = [0] * (dim * 2 - len(pad))\n extension = g.op(\n 'Sub',\n g.op('Mul',\n g.op('Constant', value_t=torch.tensor(dim, dtype=torch.int64)),\n g.op('Constant', value_t=torch.tensor(2, dtype=torch.int64))),\n pad_len)\n pad = g.op('Cast', pad, to_i=sym_help.cast_pytorch_to_onnx['Long'])\n paddings = g.op(\n 'Concat',\n pad,\n g.op(\n 'ConstantOfShape',\n extension,\n value_t=torch.tensor([0], dtype=torch.int64)),\n axis_i=0)\n paddings = g.op('Reshape', paddings,\n g.op('Constant', value_t=torch.tensor([-1, 2])))\n paddings = g.op(\n 'Transpose',\n torch.onnx.symbolic_opset10.flip(g, paddings, [0]),\n perm_i=[1, 0])\n paddings = g.op('Reshape', paddings,\n g.op('Constant', value_t=torch.tensor([-1])))\n padding_c = g.op(\n 'Cast', paddings, to_i=sym_help.cast_pytorch_to_onnx['Long'])\n return padding_c\n\n\ndef constant_pad_nd(g, input, padding, value=None):\n mode = 'constant'\n value = sym_help._maybe_get_scalar(value)\n value = sym_help._if_scalar_type_as(g, value, input)\n pad = _prepare_onnx_paddings(g, input.type().dim(), padding)\n return g.op('Pad', input, pad, value, mode_s=mode)\n\n\ndef reflection_pad(g, input, padding):\n mode = 'reflect'\n paddings = _prepare_onnx_paddings(g, input.type().dim(), padding)\n return g.op('Pad', input, paddings, mode_s=mode)\n\n\nreflection_pad1d = reflection_pad\nreflection_pad2d = reflection_pad\nreflection_pad3d = reflection_pad\n\n\ndef _avg_pool(name, tuple_fn):\n\n @parse_args('v', 'is', 'is', 'is', 'i', 'i', 'none')\n def symbolic_fn(g,\n input,\n kernel_size,\n stride,\n padding,\n ceil_mode,\n count_include_pad,\n divisor_override=None):\n padding = sym_help._avgpool_helper(tuple_fn, padding, kernel_size,\n stride, divisor_override, name)\n if not stride:\n stride = kernel_size\n if count_include_pad:\n input = g.op(\n 'Pad',\n input,\n g.op(\n 'Constant',\n value_t=torch.tensor(((0, ) * 2 + padding) * 2)),\n mode_s='constant')\n padding = (0, ) * len(padding)\n output = g.op(\n 'AveragePool',\n input,\n kernel_shape_i=tuple_fn(kernel_size),\n strides_i=tuple_fn(stride),\n pads_i=padding * 2,\n ceil_mode_i=ceil_mode)\n return output\n\n return symbolic_fn\n\n\navg_pool1d = _avg_pool('avg_pool1d', _single)\navg_pool2d = _avg_pool('avg_pool2d', _pair)\navg_pool3d = _avg_pool('avg_pool3d', _triple)\n\n\ndef _get_im2col_indices_along_dim(g, input_d, kernel_size_d, dilation_d,\n padding_d, stride_d):\n # Input is always 4-D (N, C, H, W)\n # Calculate indices of sliding blocks along spatial dimension\n # Slide kernel over input each dim d:\n # each dimension d ranges from 0 to\n # input[d]+2xpadding[d]-dilation[d]x(kernel_size[d]-1)\n # with steps = stride\n\n blocks_d = g.op('Add', input_d,\n g.op('Constant', value_t=torch.tensor(padding_d * 2)))\n blocks_d = g.op(\n 'Sub', blocks_d,\n g.op(\n 'Constant',\n value_t=torch.tensor(dilation_d * (kernel_size_d - 1))))\n\n # Stride kernel over input and find starting indices along dim d\n blocks_d_indices = g.op('Range', g.op('Constant', value_t=torch.tensor(0)),\n blocks_d,\n g.op('Constant', value_t=torch.tensor(stride_d)))\n\n # Apply dilation on kernel and find its indices along dim d\n kernel_grid = np.arange(0, kernel_size_d * dilation_d, dilation_d)\n kernel_grid = g.op('Constant', value_t=torch.tensor([kernel_grid]))\n\n # Broadcast and add kernel staring positions (indices) with\n # kernel_grid along dim d, to get block indices along dim d\n blocks_d_indices = g.op(\n 'Unsqueeze', blocks_d_indices, axes_i=[0]) # Reshape to [1, -1]\n kernel_mask = g.op('Reshape', kernel_grid,\n g.op('Constant', value_t=torch.tensor([-1, 1])))\n block_mask = g.op('Add', blocks_d_indices, kernel_mask)\n\n return block_mask\n\n\ndef _get_im2col_padded_input(g, input, padding_h, padding_w):\n # Input is always 4-D tensor (N, C, H, W)\n # Padding tensor has the following format: (padding_h, padding_w)\n # Reshape the padding to follow ONNX format:\n # (dim1_begin, dim2_begin,...,dim1_end, dim2_end,...)\n pad = g.op(\n 'Constant', value_t=torch.LongTensor([0, 0, padding_h, padding_w] * 2))\n return g.op('Pad', input, pad)\n\n\ndef _get_im2col_output_shape(g, input, kernel_h, kernel_w):\n batch_dim = size(g, input, g.op('Constant', value_t=torch.tensor(0)))\n channel_dim = size(g, input, g.op('Constant', value_t=torch.tensor(1)))\n channel_unfolded = g.op(\n 'Mul', channel_dim,\n g.op('Constant', value_t=torch.tensor(kernel_h * kernel_w)))\n\n return g.op(\n 'Concat',\n g.op('Unsqueeze', batch_dim, axes_i=[0]),\n g.op('Unsqueeze', channel_unfolded, axes_i=[0]),\n g.op('Constant', value_t=torch.tensor([-1])),\n axis_i=0)\n\n\ndef size(g, self, dim=None):\n if dim is None:\n return g.op('Shape', self)\n return sym_help._size_helper(g, self, dim)\n\n\n@parse_args('v', 'is', 'is', 'is', 'is')\ndef im2col(g, input, kernel_size, dilation, padding, stride):\n # Input is always 4-D tensor (N, C, H, W)\n # All other args are int[2]\n\n input_h = size(g, input, g.op('Constant', value_t=torch.tensor(2)))\n input_w = size(g, input, g.op('Constant', value_t=torch.tensor(3)))\n\n stride_h, stride_w = stride[0], stride[1]\n padding_h, padding_w = padding[0], padding[1]\n dilation_h, dilation_w = dilation[0], dilation[1]\n kernel_h, kernel_w = kernel_size[0], kernel_size[1]\n\n blocks_row_indices = _get_im2col_indices_along_dim(g, input_h, kernel_h,\n dilation_h, padding_h,\n stride_h)\n blocks_col_indices = _get_im2col_indices_along_dim(g, input_w, kernel_w,\n dilation_w, padding_w,\n stride_w)\n\n output_shape = _get_im2col_output_shape(g, input, kernel_h, kernel_w)\n padded_input = _get_im2col_padded_input(g, input, padding_h, padding_w)\n\n output = g.op('Gather', padded_input, blocks_row_indices, axis_i=2)\n output = g.op('Gather', output, blocks_col_indices, axis_i=4)\n output = g.op('Transpose', output, perm_i=[0, 1, 2, 4, 3, 5])\n return g.op('Reshape', output, output_shape)\n\n\n@parse_args('v', 'i')\ndef one_hot(g, self, num_classes):\n values = g.op('Constant', value_t=torch.LongTensor([0, 1]))\n depth = g.op('Constant', value_t=torch.LongTensor([num_classes]))\n return g.op('OneHot', self, depth, values, axis_i=-1)\n\n\n@parse_args('v', 'i', 'none')\ndef softmax(g, input, dim, dtype=None):\n input_dim = input.type().dim()\n if input_dim:\n # TODO: remove this as onnx opset 11 spec allows negative axes\n if dim < 0:\n dim = input_dim + dim\n if input_dim == dim + 1:\n softmax = g.op('Softmax', input, axis_i=dim)\n if dtype and dtype.node().kind() != 'prim::Constant':\n parsed_dtype = sym_help._get_const(dtype, 'i', 'dtype')\n softmax = g.op(\n 'Cast',\n softmax,\n to_i=sym_help.scalar_type_to_onnx[parsed_dtype])\n return softmax\n\n max_value = g.op('ReduceMax', input, axes_i=[dim], keepdims_i=1)\n input = g.op('Sub', input, max_value)\n exp = g.op('Exp', input)\n sum = g.op('ReduceSum', exp, axes_i=[dim])\n softmax = g.op('Div', exp, sum)\n if dtype and dtype.node().kind() != 'prim::Constant':\n parsed_dtype = sym_help._get_const(dtype, 'i', 'dtype')\n softmax = g.op(\n 'Cast', softmax, to_i=sym_help.scalar_type_to_onnx[parsed_dtype])\n return softmax\n\n\ndef _adaptive_pool(name, type, tuple_fn, fn=None):\n\n @parse_args('v', 'is')\n def symbolic_fn(g, input, output_size):\n if output_size == [1] * len(output_size) and type == 'AveragePool':\n return g.op('GlobalAveragePool', input)\n if not input.isCompleteTensor():\n if output_size == [1] * len(output_size):\n return g.op('GlobalMaxPool', input), None\n raise NotImplementedError(\n '[Adaptive pool]:input size not accessible')\n dim = input.type().sizes()[2:]\n if output_size == [1] * len(output_size) and type == 'MaxPool':\n return g.op('GlobalMaxPool', input), None\n\n # compute stride = floor(input_size / output_size)\n s = [int(dim[i] / output_size[i]) for i in range(0, len(dim))]\n\n # compute kernel_size = input_size - (output_size - 1) * stride\n k = [dim[i] - (output_size[i] - 1) * s[i] for i in range(0, len(dim))]\n\n # call max_poolxd_with_indices to get indices in the output\n if type == 'MaxPool':\n return fn(g, input, k, k, (0, ) * len(dim), (1, ) * len(dim),\n False)\n output = g.op(\n type,\n input,\n kernel_shape_i=tuple_fn(k),\n strides_i=tuple_fn(s),\n ceil_mode_i=False)\n return output\n\n return symbolic_fn\n\n\nadaptive_avg_pool1d = _adaptive_pool('adaptive_avg_pool1d', 'AveragePool',\n _single)\nadaptive_avg_pool2d = _adaptive_pool('adaptive_avg_pool2d', 'AveragePool',\n _pair)\nadaptive_avg_pool3d = _adaptive_pool('adaptive_avg_pool3d', 'AveragePool',\n _triple)\n\n\ndef new_full(g,\n self,\n size,\n fill_value,\n dtype,\n layout,\n device,\n pin_memory=False):\n from torch.onnx.symbolic_opset9 import full\n if dtype is None and self.isCompleteTensor():\n dtype = self.type().scalarType()\n dtype = sym_help.scalar_type_to_onnx.index(\n sym_help.cast_pytorch_to_onnx[dtype])\n return full(g, size, fill_value, dtype, layout, device, pin_memory)\n\n\n@parse_args('v', 'v', 'i', 'i', 'i')\ndef grid_sampler(g,\n input,\n grid,\n interpolation_mode,\n padding_mode,\n align_corners=False):\n return g.op(\n 'mmcv::grid_sampler',\n input,\n grid,\n interpolation_mode_i=interpolation_mode,\n padding_mode_i=padding_mode,\n align_corners_i=align_corners)\n\n\n@parse_args('v', 'i')\ndef cummax(g, input, dim):\n return g.op('mmcv::cummax', input, dim_i=dim, outputs=2)\n\n\n@parse_args('v', 'i')\ndef cummin(g, input, dim):\n return g.op('mmcv::cummin', input, dim_i=dim, outputs=2)\n\n\n@parse_args('v', 'v', 'is')\ndef roll(g, input, shifts, dims):\n from torch.onnx.symbolic_opset9 import squeeze\n from packaging import version\n input_shape = g.op('Shape', input)\n\n need_flatten = len(dims) == 0\n # If dims is not specified, the tensor will be flattened before\n # rolling and then restored to the original shape.\n if need_flatten:\n resize_shape = input_shape\n input = g.op('Reshape', input,\n g.op('Constant', value_t=torch.LongTensor([1, -1])))\n input_shape = g.op('Shape', input)\n dims = [1]\n\n for index, dim in enumerate(dims):\n end_size = sym_help._slice_helper(\n g, input_shape, axes=[0], ends=[dim + 1], starts=[dim])\n shift_size = sym_help._slice_helper(\n g, shifts, axes=[0], ends=[index + 1], starts=[index])\n slice_size = g.op('Sub', end_size, shift_size)\n\n # Can not use Mod because tensorrt does not support\n div_size = g.op('Div', slice_size, end_size)\n slice_size = g.op('Sub', slice_size, g.op('Mul', end_size, div_size))\n\n if version.parse(torch.__version__) >= version.parse('1.7.0'):\n # add dim=0 for pytorch 1.9.0\n end_size = squeeze(g, end_size, 0)\n slice_size = squeeze(g, slice_size, 0)\n else:\n end_size = g.op('Squeeze', end_size)\n slice_size = g.op('Squeeze', slice_size)\n dim = torch.LongTensor([dim])\n\n input_slice0 = sym_help._slice_helper(\n g,\n input,\n axes=dim,\n starts=torch.LongTensor([0]),\n ends=slice_size,\n dynamic_slice=True)\n input_slice1 = sym_help._slice_helper(\n g,\n input,\n axes=dim,\n ends=end_size,\n starts=slice_size,\n dynamic_slice=True)\n\n input = g.op('Concat', input_slice1, input_slice0, axis_i=dim)\n\n if need_flatten:\n input = g.op('Reshape', input, resize_shape)\n\n return input\n\n\ndef register_extra_symbolics(opset=11):\n register_op('one_hot', one_hot, '', opset)\n register_op('im2col', im2col, '', opset)\n register_op('topk', topk, '', opset)\n register_op('softmax', softmax, '', opset)\n register_op('constant_pad_nd', constant_pad_nd, '', opset)\n register_op('reflection_pad1d', reflection_pad1d, '', opset)\n register_op('reflection_pad2d', reflection_pad2d, '', opset)\n register_op('reflection_pad3d', reflection_pad3d, '', opset)\n register_op('avg_pool1d', avg_pool1d, '', opset)\n register_op('avg_pool2d', avg_pool2d, '', opset)\n register_op('avg_pool3d', avg_pool3d, '', opset)\n register_op('adaptive_avg_pool1d', adaptive_avg_pool1d, '', opset)\n register_op('adaptive_avg_pool2d', adaptive_avg_pool2d, '', opset)\n register_op('adaptive_avg_pool3d', adaptive_avg_pool3d, '', opset)\n register_op('masked_select', masked_select, '', opset)\n register_op('upsample_nearest1d', upsample_nearest1d, '', opset)\n register_op('upsample_nearest2d', upsample_nearest2d, '', opset)\n register_op('upsample_nearest3d', upsample_nearest3d, '', opset)\n register_op('upsample_linear1d', upsample_linear1d, '', opset)\n register_op('upsample_bilinear2d', upsample_bilinear2d, '', opset)\n register_op('upsample_trilinear3d', upsample_trilinear3d, '', opset)\n register_op('upsample_bicubic2d', upsample_bicubic2d, '', opset)\n register_op('new_full', new_full, '', opset)\n register_op('grid_sampler', grid_sampler, '', opset)\n register_op('cummax', cummax, '', opset)\n register_op('cummin', cummin, '', opset)\n register_op('roll', roll, '', opset)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .active_rotated_filter import active_rotated_filter\nfrom .assign_score_withk import assign_score_withk\nfrom .ball_query import ball_query\nfrom .bbox import bbox_overlaps\nfrom .border_align import BorderAlign, border_align\nfrom .box_iou_rotated import box_iou_rotated\nfrom .carafe import CARAFE, CARAFENaive, CARAFEPack, carafe, carafe_naive\nfrom .cc_attention import CrissCrossAttention\nfrom .contour_expand import contour_expand\nfrom .convex_iou import convex_giou, convex_iou\nfrom .corner_pool import CornerPool\nfrom .correlation import Correlation\nfrom .deform_conv import DeformConv2d, DeformConv2dPack, deform_conv2d\nfrom .deform_roi_pool import (DeformRoIPool, DeformRoIPoolPack,\n ModulatedDeformRoIPoolPack, deform_roi_pool)\nfrom .deprecated_wrappers import Conv2d_deprecated as Conv2d\nfrom .deprecated_wrappers import ConvTranspose2d_deprecated as ConvTranspose2d\nfrom .deprecated_wrappers import Linear_deprecated as Linear\nfrom .deprecated_wrappers import MaxPool2d_deprecated as MaxPool2d\nfrom .focal_loss import (SigmoidFocalLoss, SoftmaxFocalLoss,\n sigmoid_focal_loss, softmax_focal_loss)\nfrom .furthest_point_sample import (furthest_point_sample,\n furthest_point_sample_with_dist)\nfrom .fused_bias_leakyrelu import FusedBiasLeakyReLU, fused_bias_leakyrelu\nfrom .gather_points import gather_points\nfrom .group_points import GroupAll, QueryAndGroup, grouping_operation\nfrom .info import (get_compiler_version, get_compiling_cuda_version,\n get_onnxruntime_op_path)\nfrom .iou3d import boxes_iou_bev, nms_bev, nms_normal_bev\nfrom .knn import knn\nfrom .masked_conv import MaskedConv2d, masked_conv2d\nfrom .min_area_polygons import min_area_polygons\nfrom .modulated_deform_conv import (ModulatedDeformConv2d,\n ModulatedDeformConv2dPack,\n modulated_deform_conv2d)\nfrom .multi_scale_deform_attn import MultiScaleDeformableAttention\nfrom .nms import batched_nms, nms, nms_match, nms_rotated, soft_nms\nfrom .pixel_group import pixel_group\nfrom .point_sample import (SimpleRoIAlign, point_sample,\n rel_roi_point_to_rel_img_point)\nfrom .points_in_boxes import (points_in_boxes_all, points_in_boxes_cpu,\n points_in_boxes_part)\nfrom .points_in_polygons import points_in_polygons\nfrom .points_sampler import PointsSampler\nfrom .psa_mask import PSAMask\nfrom .riroi_align_rotated import RiRoIAlignRotated, riroi_align_rotated\nfrom .roi_align import RoIAlign, roi_align\nfrom .roi_align_rotated import RoIAlignRotated, roi_align_rotated\nfrom .roi_pool import RoIPool, roi_pool\nfrom .roiaware_pool3d import RoIAwarePool3d\nfrom .roipoint_pool3d import RoIPointPool3d\nfrom .rotated_feature_align import rotated_feature_align\nfrom .saconv import SAConv2d\nfrom .scatter_points import DynamicScatter, dynamic_scatter\nfrom .sync_bn import SyncBatchNorm\nfrom .three_interpolate import three_interpolate\nfrom .three_nn import three_nn\nfrom .tin_shift import TINShift, tin_shift\nfrom .upfirdn2d import upfirdn2d\nfrom .voxelize import Voxelization, voxelization\n\n__all__ = [\n 'bbox_overlaps', 'CARAFE', 'CARAFENaive', 'CARAFEPack', 'carafe',\n 'carafe_naive', 'CornerPool', 'DeformConv2d', 'DeformConv2dPack',\n 'deform_conv2d', 'DeformRoIPool', 'DeformRoIPoolPack',\n 'ModulatedDeformRoIPoolPack', 'deform_roi_pool', 'SigmoidFocalLoss',\n 'SoftmaxFocalLoss', 'sigmoid_focal_loss', 'softmax_focal_loss',\n 'get_compiler_version', 'get_compiling_cuda_version',\n 'get_onnxruntime_op_path', 'MaskedConv2d', 'masked_conv2d',\n 'ModulatedDeformConv2d', 'ModulatedDeformConv2dPack',\n 'modulated_deform_conv2d', 'batched_nms', 'nms', 'soft_nms', 'nms_match',\n 'RoIAlign', 'roi_align', 'RoIPool', 'roi_pool', 'SyncBatchNorm', 'Conv2d',\n 'ConvTranspose2d', 'Linear', 'MaxPool2d', 'CrissCrossAttention', 'PSAMask',\n 'point_sample', 'rel_roi_point_to_rel_img_point', 'SimpleRoIAlign',\n 'SAConv2d', 'TINShift', 'tin_shift', 'assign_score_withk',\n 'box_iou_rotated', 'RoIPointPool3d', 'nms_rotated', 'knn', 'ball_query',\n 'upfirdn2d', 'FusedBiasLeakyReLU', 'fused_bias_leakyrelu',\n 'rotated_feature_align', 'RiRoIAlignRotated', 'riroi_align_rotated',\n 'RoIAlignRotated', 'roi_align_rotated', 'pixel_group', 'QueryAndGroup',\n 'GroupAll', 'grouping_operation', 'contour_expand', 'three_nn',\n 'three_interpolate', 'MultiScaleDeformableAttention', 'BorderAlign',\n 'border_align', 'gather_points', 'furthest_point_sample',\n 'furthest_point_sample_with_dist', 'PointsSampler', 'Correlation',\n 'boxes_iou_bev', 'nms_bev', 'nms_normal_bev', 'Voxelization',\n 'voxelization', 'dynamic_scatter', 'DynamicScatter', 'RoIAwarePool3d',\n 'points_in_boxes_part', 'points_in_boxes_cpu', 'points_in_boxes_all',\n 'points_in_polygons', 'min_area_polygons', 'active_rotated_filter',\n 'convex_iou', 'convex_giou'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/active_rotated_filter.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext',\n ['active_rotated_filter_forward', 'active_rotated_filter_backward'])\n\n\nclass ActiveRotatedFilterFunction(Function):\n \"\"\"Encoding the orientation information and generating orientation-\n sensitive features.\n\n The details are described in the paper `Align Deep Features for Oriented\n Object Detection _`.\n \"\"\"\n\n @staticmethod\n def forward(ctx, input, indices):\n \"\"\"\n Args:\n input (torch.Tensor): Input features with shape\n [num_output_planes, num_input_planes, num_orientations, H, W].\n indices (torch.Tensor): Indices with shape\n [num_orientations, H, W, num_rotations].\n\n Returns:\n torch.Tensor: Refined features with shape [num_output_planes *\n num_rotations, num_input_planes * num_orientations, H, W].\n \"\"\"\n ctx.save_for_backward(input, indices)\n op, ip, o, h, w = input.size()\n o, h, w, r = indices.size()\n output = input.new_zeros((op * r, ip * o, h, w))\n ext_module.active_rotated_filter_forward(input, indices, output)\n\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_out):\n \"\"\"\n Args:\n grad_output (torch.Tensor): The gradiant of output features\n with shape [num_output_planes * num_rotations,\n num_input_planes * num_orientations, H, W].\n\n Returns:\n torch.Tensor: The gradiant of input features with shape\n [num_output_planes, num_input_planes, num_orientations, H, W].\n \"\"\"\n input, indices = ctx.saved_tensors\n grad_in = torch.zeros_like(input)\n ext_module.active_rotated_filter_backward(grad_out, indices, grad_in)\n return grad_in, None\n\n\nactive_rotated_filter = ActiveRotatedFilterFunction.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/assign_score_withk.py", "content": "from torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['assign_score_withk_forward', 'assign_score_withk_backward'])\n\n\nclass AssignScoreWithK(Function):\n r\"\"\"Perform weighted sum to generate output features according to scores.\n Modified from `PAConv `_.\n\n This is a memory-efficient CUDA implementation of assign_scores operation,\n which first transform all point features with weight bank, then assemble\n neighbor features with ``knn_idx`` and perform weighted sum of ``scores``.\n\n See the `paper `_ appendix Sec. D for\n more detailed descriptions.\n\n Note:\n This implementation assumes using ``neighbor`` kernel input, which is\n (point_features - center_features, point_features).\n See https://github.com/CVMI-Lab/PAConv/blob/main/scene_seg/model/\n pointnet2/paconv.py#L128 for more details.\n \"\"\"\n\n @staticmethod\n def forward(ctx,\n scores,\n point_features,\n center_features,\n knn_idx,\n aggregate='sum'):\n \"\"\"\n Args:\n scores (torch.Tensor): (B, npoint, K, M), predicted scores to\n aggregate weight matrices in the weight bank.\n ``npoint`` is the number of sampled centers.\n ``K`` is the number of queried neighbors.\n ``M`` is the number of weight matrices in the weight bank.\n point_features (torch.Tensor): (B, N, M, out_dim)\n Pre-computed point features to be aggregated.\n center_features (torch.Tensor): (B, N, M, out_dim)\n Pre-computed center features to be aggregated.\n knn_idx (torch.Tensor): (B, npoint, K), index of sampled kNN.\n We assume the first idx in each row is the idx of the center.\n aggregate (str, optional): Aggregation method.\n Can be 'sum', 'avg' or 'max'. Defaults: 'sum'.\n\n Returns:\n torch.Tensor: (B, out_dim, npoint, K), the aggregated features.\n \"\"\"\n agg = {'sum': 0, 'avg': 1, 'max': 2}\n\n B, N, M, out_dim = point_features.size()\n _, npoint, K, _ = scores.size()\n\n output = point_features.new_zeros((B, out_dim, npoint, K))\n ext_module.assign_score_withk_forward(\n point_features.contiguous(),\n center_features.contiguous(),\n scores.contiguous(),\n knn_idx.contiguous(),\n output,\n B=B,\n N0=N,\n N1=npoint,\n M=M,\n K=K,\n O=out_dim,\n aggregate=agg[aggregate])\n\n ctx.save_for_backward(output, point_features, center_features, scores,\n knn_idx)\n ctx.agg = agg[aggregate]\n\n return output\n\n @staticmethod\n def backward(ctx, grad_out):\n \"\"\"\n Args:\n grad_out (torch.Tensor): (B, out_dim, npoint, K)\n\n Returns:\n tuple[torch.Tensor]: A tuple contains five elements. The first one\n is the gradient of ``scores`` whose shape is (B, npoint, K, M). The\n second is the gradient of ``point_features`` whose shape is\n (B, N, M, out_dim). The third is the gradient of\n ``center_features`` with the shape of (B, N, M, out_dim). The last\n two are ``None``.\n \"\"\"\n _, point_features, center_features, scores, knn_idx = ctx.saved_tensors\n\n agg = ctx.agg\n\n B, N, M, out_dim = point_features.size()\n _, npoint, K, _ = scores.size()\n\n grad_point_features = point_features.new_zeros(point_features.shape)\n grad_center_features = center_features.new_zeros(center_features.shape)\n grad_scores = scores.new_zeros(scores.shape)\n\n ext_module.assign_score_withk_backward(\n grad_out.contiguous(),\n point_features.contiguous(),\n center_features.contiguous(),\n scores.contiguous(),\n knn_idx.contiguous(),\n grad_point_features,\n grad_center_features,\n grad_scores,\n B=B,\n N0=N,\n N1=npoint,\n M=M,\n K=K,\n O=out_dim,\n aggregate=agg)\n\n return grad_scores, grad_point_features, \\\n grad_center_features, None, None\n\n\nassign_score_withk = AssignScoreWithK.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/ball_query.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['ball_query_forward'])\n\n\nclass BallQuery(Function):\n \"\"\"Find nearby points in spherical space.\"\"\"\n\n @staticmethod\n def forward(ctx, min_radius: float, max_radius: float, sample_num: int,\n xyz: torch.Tensor, center_xyz: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Args:\n min_radius (float): minimum radius of the balls.\n max_radius (float): maximum radius of the balls.\n sample_num (int): maximum number of features in the balls.\n xyz (Tensor): (B, N, 3) xyz coordinates of the features.\n center_xyz (torch.Tensor): (B, npoint, 3) centers of the ball\n query.\n\n Returns:\n torch.Tensor: (B, npoint, nsample) tensor with the indices of the\n features that form the query balls.\n \"\"\"\n assert center_xyz.is_contiguous()\n assert xyz.is_contiguous()\n assert min_radius < max_radius\n\n B, N, _ = xyz.size()\n npoint = center_xyz.size(1)\n idx = xyz.new_zeros(B, npoint, sample_num, dtype=torch.int)\n\n ext_module.ball_query_forward(\n center_xyz,\n xyz,\n idx,\n b=B,\n n=N,\n m=npoint,\n min_radius=min_radius,\n max_radius=max_radius,\n nsample=sample_num)\n if torch.__version__ != 'parrots':\n ctx.mark_non_differentiable(idx)\n return idx\n\n @staticmethod\n def backward(ctx, a=None):\n return None, None, None, None\n\n\nball_query = BallQuery.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/bbox.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['bbox_overlaps'])\n\n\ndef bbox_overlaps(bboxes1, bboxes2, mode='iou', aligned=False, offset=0):\n \"\"\"Calculate overlap between two set of bboxes.\n\n If ``aligned`` is ``False``, then calculate the ious between each bbox\n of bboxes1 and bboxes2, otherwise the ious between each aligned pair of\n bboxes1 and bboxes2.\n\n Args:\n bboxes1 (torch.Tensor): shape (m, 4) in format or\n empty.\n bboxes2 (torch.Tensor): shape (n, 4) in format or\n empty. If aligned is ``True``, then m and n must be equal.\n mode (str): \"iou\" (intersection over union) or iof (intersection over\n foreground).\n\n Returns:\n torch.Tensor: Return the ious betweens boxes. If ``aligned`` is\n ``False``, the shape of ious is (m, n) else (m, 1).\n\n Example:\n >>> bboxes1 = torch.FloatTensor([\n >>> [0, 0, 10, 10],\n >>> [10, 10, 20, 20],\n >>> [32, 32, 38, 42],\n >>> ])\n >>> bboxes2 = torch.FloatTensor([\n >>> [0, 0, 10, 20],\n >>> [0, 10, 10, 19],\n >>> [10, 10, 20, 20],\n >>> ])\n >>> bbox_overlaps(bboxes1, bboxes2)\n tensor([[0.5000, 0.0000, 0.0000],\n [0.0000, 0.0000, 1.0000],\n [0.0000, 0.0000, 0.0000]])\n\n Example:\n >>> empty = torch.FloatTensor([])\n >>> nonempty = torch.FloatTensor([\n >>> [0, 0, 10, 9],\n >>> ])\n >>> assert tuple(bbox_overlaps(empty, nonempty).shape) == (0, 1)\n >>> assert tuple(bbox_overlaps(nonempty, empty).shape) == (1, 0)\n >>> assert tuple(bbox_overlaps(empty, empty).shape) == (0, 0)\n \"\"\"\n\n mode_dict = {'iou': 0, 'iof': 1}\n assert mode in mode_dict.keys()\n mode_flag = mode_dict[mode]\n # Either the boxes are empty or the length of boxes' last dimension is 4\n assert (bboxes1.size(-1) == 4 or bboxes1.size(0) == 0)\n assert (bboxes2.size(-1) == 4 or bboxes2.size(0) == 0)\n assert offset == 1 or offset == 0\n\n rows = bboxes1.size(0)\n cols = bboxes2.size(0)\n if aligned:\n assert rows == cols\n\n if rows * cols == 0:\n return bboxes1.new(rows, 1) if aligned else bboxes1.new(rows, cols)\n\n if aligned:\n ious = bboxes1.new_zeros(rows)\n else:\n ious = bboxes1.new_zeros((rows, cols))\n ext_module.bbox_overlaps(\n bboxes1, bboxes2, ious, mode=mode_flag, aligned=aligned, offset=offset)\n return ious\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/border_align.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n# modified from\n# https://github.com/Megvii-BaseDetection/cvpods/blob/master/cvpods/layers/border_align.py\n\nimport torch\nimport torch.nn as nn\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['border_align_forward', 'border_align_backward'])\n\n\nclass BorderAlignFunction(Function):\n\n @staticmethod\n def symbolic(g, input, boxes, pool_size):\n return g.op(\n 'mmcv::MMCVBorderAlign', input, boxes, pool_size_i=pool_size)\n\n @staticmethod\n def forward(ctx, input, boxes, pool_size):\n ctx.pool_size = pool_size\n ctx.input_shape = input.size()\n\n assert boxes.ndim == 3, 'boxes must be with shape [B, H*W, 4]'\n assert boxes.size(2) == 4, \\\n 'the last dimension of boxes must be (x1, y1, x2, y2)'\n assert input.size(1) % 4 == 0, \\\n 'the channel for input feature must be divisible by factor 4'\n\n # [B, C//4, H*W, 4]\n output_shape = (input.size(0), input.size(1) // 4, boxes.size(1), 4)\n output = input.new_zeros(output_shape)\n # `argmax_idx` only used for backward\n argmax_idx = input.new_zeros(output_shape).to(torch.int)\n\n ext_module.border_align_forward(\n input, boxes, output, argmax_idx, pool_size=ctx.pool_size)\n\n ctx.save_for_backward(boxes, argmax_idx)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n boxes, argmax_idx = ctx.saved_tensors\n grad_input = grad_output.new_zeros(ctx.input_shape)\n # complex head architecture may cause grad_output uncontiguous\n grad_output = grad_output.contiguous()\n ext_module.border_align_backward(\n grad_output,\n boxes,\n argmax_idx,\n grad_input,\n pool_size=ctx.pool_size)\n return grad_input, None, None\n\n\nborder_align = BorderAlignFunction.apply\n\n\nclass BorderAlign(nn.Module):\n r\"\"\"Border align pooling layer.\n\n Applies border_align over the input feature based on predicted bboxes.\n The details were described in the paper\n `BorderDet: Border Feature for Dense Object Detection\n `_.\n\n For each border line (e.g. top, left, bottom or right) of each box,\n border_align does the following:\n\n 1. uniformly samples ``pool_size`` +1 positions on this line, involving\n the start and end points.\n 2. the corresponding features on these points are computed by bilinear\n interpolation.\n 3. max pooling over all the ``pool_size`` +1 positions are used for\n computing pooled feature.\n\n Args:\n pool_size (int): number of positions sampled over the boxes' borders\n (e.g. top, bottom, left, right).\n \"\"\"\n\n def __init__(self, pool_size):\n super(BorderAlign, self).__init__()\n self.pool_size = pool_size\n\n def forward(self, input, boxes):\n \"\"\"\n Args:\n input: Features with shape [N,4C,H,W]. Channels ranged in [0,C),\n [C,2C), [2C,3C), [3C,4C) represent the top, left, bottom,\n right features respectively.\n boxes: Boxes with shape [N,H*W,4]. Coordinate format (x1,y1,x2,y2).\n\n Returns:\n torch.Tensor: Pooled features with shape [N,C,H*W,4]. The order is\n (top,left,bottom,right) for the last dimension.\n \"\"\"\n return border_align(input, boxes, self.pool_size)\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'(pool_size={self.pool_size})'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/box_iou_rotated.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['box_iou_rotated'])\n\n\ndef box_iou_rotated(bboxes1,\n bboxes2,\n mode='iou',\n aligned=False,\n clockwise=True):\n \"\"\"Return intersection-over-union (Jaccard index) of boxes.\n\n Both sets of boxes are expected to be in\n (x_center, y_center, width, height, angle) format.\n\n If ``aligned`` is ``False``, then calculate the ious between each bbox\n of bboxes1 and bboxes2, otherwise the ious between each aligned pair of\n bboxes1 and bboxes2.\n\n .. note::\n The operator assumes:\n\n 1) The positive direction along x axis is left -> right.\n\n 2) The positive direction along y axis is top -> down.\n\n 3) The w border is in parallel with x axis when angle = 0.\n\n However, there are 2 opposite definitions of the positive angular\n direction, clockwise (CW) and counter-clockwise (CCW). MMCV supports\n both definitions and uses CW by default.\n\n Please set ``clockwise=False`` if you are using the CCW definition.\n\n The coordinate system when ``clockwise`` is ``True`` (default)\n\n .. code-block:: none\n\n 0-------------------> x (0 rad)\n | A-------------B\n | | |\n | | box h\n | | angle=0 |\n | D------w------C\n v\n y (pi/2 rad)\n\n In such coordination system the rotation matrix is\n\n .. math::\n \\\\begin{pmatrix}\n \\\\cos\\\\alpha & -\\\\sin\\\\alpha \\\\\\\\\n \\\\sin\\\\alpha & \\\\cos\\\\alpha\n \\\\end{pmatrix}\n\n The coordinates of the corner point A can be calculated as:\n\n .. math::\n P_A=\n \\\\begin{pmatrix} x_A \\\\\\\\ y_A\\\\end{pmatrix}\n =\n \\\\begin{pmatrix} x_{center} \\\\\\\\ y_{center}\\\\end{pmatrix} +\n \\\\begin{pmatrix}\\\\cos\\\\alpha & -\\\\sin\\\\alpha \\\\\\\\\n \\\\sin\\\\alpha & \\\\cos\\\\alpha\\\\end{pmatrix}\n \\\\begin{pmatrix} -0.5w \\\\\\\\ -0.5h\\\\end{pmatrix} \\\\\\\\\n =\n \\\\begin{pmatrix} x_{center}-0.5w\\\\cos\\\\alpha+0.5h\\\\sin\\\\alpha\n \\\\\\\\\n y_{center}-0.5w\\\\sin\\\\alpha-0.5h\\\\cos\\\\alpha\\\\end{pmatrix}\n\n\n The coordinate system when ``clockwise`` is ``False``\n\n .. code-block:: none\n\n 0-------------------> x (0 rad)\n | A-------------B\n | | |\n | | box h\n | | angle=0 |\n | D------w------C\n v\n y (-pi/2 rad)\n\n In such coordination system the rotation matrix is\n\n .. math::\n \\\\begin{pmatrix}\n \\\\cos\\\\alpha & \\\\sin\\\\alpha \\\\\\\\\n -\\\\sin\\\\alpha & \\\\cos\\\\alpha\n \\\\end{pmatrix}\n\n The coordinates of the corner point A can be calculated as:\n\n .. math::\n P_A=\n \\\\begin{pmatrix} x_A \\\\\\\\ y_A\\\\end{pmatrix}\n =\n \\\\begin{pmatrix} x_{center} \\\\\\\\ y_{center}\\\\end{pmatrix} +\n \\\\begin{pmatrix}\\\\cos\\\\alpha & \\\\sin\\\\alpha \\\\\\\\\n -\\\\sin\\\\alpha & \\\\cos\\\\alpha\\\\end{pmatrix}\n \\\\begin{pmatrix} -0.5w \\\\\\\\ -0.5h\\\\end{pmatrix} \\\\\\\\\n =\n \\\\begin{pmatrix} x_{center}-0.5w\\\\cos\\\\alpha-0.5h\\\\sin\\\\alpha\n \\\\\\\\\n y_{center}+0.5w\\\\sin\\\\alpha-0.5h\\\\cos\\\\alpha\\\\end{pmatrix}\n\n Args:\n boxes1 (torch.Tensor): rotated bboxes 1. It has shape (N, 5),\n indicating (x, y, w, h, theta) for each row. Note that theta is in\n radian.\n boxes2 (torch.Tensor): rotated bboxes 2. It has shape (M, 5),\n indicating (x, y, w, h, theta) for each row. Note that theta is in\n radian.\n mode (str): \"iou\" (intersection over union) or iof (intersection over\n foreground).\n clockwise (bool): flag indicating whether the positive angular\n orientation is clockwise. default True.\n `New in version 1.4.3.`\n\n Returns:\n torch.Tensor: Return the ious betweens boxes. If ``aligned`` is\n ``False``, the shape of ious is (N, M) else (N,).\n \"\"\"\n assert mode in ['iou', 'iof']\n mode_dict = {'iou': 0, 'iof': 1}\n mode_flag = mode_dict[mode]\n rows = bboxes1.size(0)\n cols = bboxes2.size(0)\n if aligned:\n ious = bboxes1.new_zeros(rows)\n else:\n ious = bboxes1.new_zeros((rows * cols))\n if not clockwise:\n flip_mat = bboxes1.new_ones(bboxes1.shape[-1])\n flip_mat[-1] = -1\n bboxes1 = bboxes1 * flip_mat\n bboxes2 = bboxes2 * flip_mat\n bboxes1 = bboxes1.contiguous()\n bboxes2 = bboxes2.contiguous()\n ext_module.box_iou_rotated(\n bboxes1, bboxes2, ious, mode_flag=mode_flag, aligned=aligned)\n if not aligned:\n ious = ious.view(rows, cols)\n return ious\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/carafe.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.autograd import Function\nfrom torch.nn.modules.module import Module\n\nfrom ..cnn import UPSAMPLE_LAYERS, normal_init, xavier_init\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', [\n 'carafe_naive_forward', 'carafe_naive_backward', 'carafe_forward',\n 'carafe_backward'\n])\n\n\nclass CARAFENaiveFunction(Function):\n\n @staticmethod\n def symbolic(g, features, masks, kernel_size, group_size, scale_factor):\n return g.op(\n 'mmcv::MMCVCARAFENaive',\n features,\n masks,\n kernel_size_i=kernel_size,\n group_size_i=group_size,\n scale_factor_f=scale_factor)\n\n @staticmethod\n def forward(ctx, features, masks, kernel_size, group_size, scale_factor):\n assert scale_factor >= 1\n assert masks.size(1) == kernel_size * kernel_size * group_size\n assert masks.size(-1) == features.size(-1) * scale_factor\n assert masks.size(-2) == features.size(-2) * scale_factor\n assert features.size(1) % group_size == 0\n assert (kernel_size - 1) % 2 == 0 and kernel_size >= 1\n ctx.kernel_size = kernel_size\n ctx.group_size = group_size\n ctx.scale_factor = scale_factor\n ctx.feature_size = features.size()\n ctx.mask_size = masks.size()\n\n n, c, h, w = features.size()\n output = features.new_zeros((n, c, h * scale_factor, w * scale_factor))\n ext_module.carafe_naive_forward(\n features,\n masks,\n output,\n kernel_size=kernel_size,\n group_size=group_size,\n scale_factor=scale_factor)\n\n if features.requires_grad or masks.requires_grad:\n ctx.save_for_backward(features, masks)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n assert grad_output.is_cuda\n\n features, masks = ctx.saved_tensors\n kernel_size = ctx.kernel_size\n group_size = ctx.group_size\n scale_factor = ctx.scale_factor\n\n grad_input = torch.zeros_like(features)\n grad_masks = torch.zeros_like(masks)\n ext_module.carafe_naive_backward(\n grad_output.contiguous(),\n features,\n masks,\n grad_input,\n grad_masks,\n kernel_size=kernel_size,\n group_size=group_size,\n scale_factor=scale_factor)\n\n return grad_input, grad_masks, None, None, None\n\n\ncarafe_naive = CARAFENaiveFunction.apply\n\n\nclass CARAFENaive(Module):\n\n def __init__(self, kernel_size, group_size, scale_factor):\n super(CARAFENaive, self).__init__()\n\n assert isinstance(kernel_size, int) and isinstance(\n group_size, int) and isinstance(scale_factor, int)\n self.kernel_size = kernel_size\n self.group_size = group_size\n self.scale_factor = scale_factor\n\n def forward(self, features, masks):\n return carafe_naive(features, masks, self.kernel_size, self.group_size,\n self.scale_factor)\n\n\nclass CARAFEFunction(Function):\n\n @staticmethod\n def symbolic(g, features, masks, kernel_size, group_size, scale_factor):\n return g.op(\n 'mmcv::MMCVCARAFE',\n features,\n masks,\n kernel_size_i=kernel_size,\n group_size_i=group_size,\n scale_factor_f=scale_factor)\n\n @staticmethod\n def forward(ctx, features, masks, kernel_size, group_size, scale_factor):\n assert scale_factor >= 1\n assert masks.size(1) == kernel_size * kernel_size * group_size\n assert masks.size(-1) == features.size(-1) * scale_factor\n assert masks.size(-2) == features.size(-2) * scale_factor\n assert features.size(1) % group_size == 0\n assert (kernel_size - 1) % 2 == 0 and kernel_size >= 1\n ctx.kernel_size = kernel_size\n ctx.group_size = group_size\n ctx.scale_factor = scale_factor\n ctx.feature_size = features.size()\n ctx.mask_size = masks.size()\n\n n, c, h, w = features.size()\n output = features.new_zeros((n, c, h * scale_factor, w * scale_factor))\n routput = features.new_zeros(output.size(), requires_grad=False)\n rfeatures = features.new_zeros(features.size(), requires_grad=False)\n rmasks = masks.new_zeros(masks.size(), requires_grad=False)\n ext_module.carafe_forward(\n features,\n masks,\n rfeatures,\n routput,\n rmasks,\n output,\n kernel_size=kernel_size,\n group_size=group_size,\n scale_factor=scale_factor)\n\n if features.requires_grad or masks.requires_grad:\n ctx.save_for_backward(features, masks, rfeatures)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n assert grad_output.is_cuda\n\n features, masks, rfeatures = ctx.saved_tensors\n kernel_size = ctx.kernel_size\n group_size = ctx.group_size\n scale_factor = ctx.scale_factor\n\n rgrad_output = torch.zeros_like(grad_output, requires_grad=False)\n rgrad_input_hs = torch.zeros_like(grad_output, requires_grad=False)\n rgrad_input = torch.zeros_like(features, requires_grad=False)\n rgrad_masks = torch.zeros_like(masks, requires_grad=False)\n grad_input = torch.zeros_like(features, requires_grad=False)\n grad_masks = torch.zeros_like(masks, requires_grad=False)\n ext_module.carafe_backward(\n grad_output.contiguous(),\n rfeatures,\n masks,\n rgrad_output,\n rgrad_input_hs,\n rgrad_input,\n rgrad_masks,\n grad_input,\n grad_masks,\n kernel_size=kernel_size,\n group_size=group_size,\n scale_factor=scale_factor)\n return grad_input, grad_masks, None, None, None\n\n\ncarafe = CARAFEFunction.apply\n\n\nclass CARAFE(Module):\n \"\"\" CARAFE: Content-Aware ReAssembly of FEatures\n\n Please refer to `CARAFE: Content-Aware ReAssembly of FEatures\n `_ for more details.\n\n Args:\n kernel_size (int): reassemble kernel size\n group_size (int): reassemble group size\n scale_factor (int): upsample ratio\n\n Returns:\n upsampled feature map\n \"\"\"\n\n def __init__(self, kernel_size, group_size, scale_factor):\n super(CARAFE, self).__init__()\n\n assert isinstance(kernel_size, int) and isinstance(\n group_size, int) and isinstance(scale_factor, int)\n self.kernel_size = kernel_size\n self.group_size = group_size\n self.scale_factor = scale_factor\n\n def forward(self, features, masks):\n return carafe(features, masks, self.kernel_size, self.group_size,\n self.scale_factor)\n\n\n@UPSAMPLE_LAYERS.register_module(name='carafe')\nclass CARAFEPack(nn.Module):\n \"\"\"A unified package of CARAFE upsampler that contains: 1) channel\n compressor 2) content encoder 3) CARAFE op.\n\n Official implementation of ICCV 2019 paper\n `CARAFE: Content-Aware ReAssembly of FEatures\n `_.\n\n Args:\n channels (int): input feature channels\n scale_factor (int): upsample ratio\n up_kernel (int): kernel size of CARAFE op\n up_group (int): group size of CARAFE op\n encoder_kernel (int): kernel size of content encoder\n encoder_dilation (int): dilation of content encoder\n compressed_channels (int): output channels of channels compressor\n\n Returns:\n upsampled feature map\n \"\"\"\n\n def __init__(self,\n channels,\n scale_factor,\n up_kernel=5,\n up_group=1,\n encoder_kernel=3,\n encoder_dilation=1,\n compressed_channels=64):\n super(CARAFEPack, self).__init__()\n self.channels = channels\n self.scale_factor = scale_factor\n self.up_kernel = up_kernel\n self.up_group = up_group\n self.encoder_kernel = encoder_kernel\n self.encoder_dilation = encoder_dilation\n self.compressed_channels = compressed_channels\n self.channel_compressor = nn.Conv2d(channels, self.compressed_channels,\n 1)\n self.content_encoder = nn.Conv2d(\n self.compressed_channels,\n self.up_kernel * self.up_kernel * self.up_group *\n self.scale_factor * self.scale_factor,\n self.encoder_kernel,\n padding=int((self.encoder_kernel - 1) * self.encoder_dilation / 2),\n dilation=self.encoder_dilation,\n groups=1)\n self.init_weights()\n\n def init_weights(self):\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n xavier_init(m, distribution='uniform')\n normal_init(self.content_encoder, std=0.001)\n\n def kernel_normalizer(self, mask):\n mask = F.pixel_shuffle(mask, self.scale_factor)\n n, mask_c, h, w = mask.size()\n # use float division explicitly,\n # to void inconsistency while exporting to onnx\n mask_channel = int(mask_c / float(self.up_kernel**2))\n mask = mask.view(n, mask_channel, -1, h, w)\n\n mask = F.softmax(mask, dim=2, dtype=mask.dtype)\n mask = mask.view(n, mask_c, h, w).contiguous()\n\n return mask\n\n def feature_reassemble(self, x, mask):\n x = carafe(x, mask, self.up_kernel, self.up_group, self.scale_factor)\n return x\n\n def forward(self, x):\n compressed_x = self.channel_compressor(x)\n mask = self.content_encoder(compressed_x)\n mask = self.kernel_normalizer(mask)\n\n x = self.feature_reassemble(x, mask)\n return x\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/cc_attention.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom mmcv.cnn import PLUGIN_LAYERS, Scale\n\n\ndef NEG_INF_DIAG(n, device):\n \"\"\"Returns a diagonal matrix of size [n, n].\n\n The diagonal are all \"-inf\". This is for avoiding calculating the\n overlapped element in the Criss-Cross twice.\n \"\"\"\n return torch.diag(torch.tensor(float('-inf')).to(device).repeat(n), 0)\n\n\n@PLUGIN_LAYERS.register_module()\nclass CrissCrossAttention(nn.Module):\n \"\"\"Criss-Cross Attention Module.\n\n .. note::\n Before v1.3.13, we use a CUDA op. Since v1.3.13, we switch\n to a pure PyTorch and equivalent implementation. For more\n details, please refer to https://github.com/open-mmlab/mmcv/pull/1201.\n\n Speed comparison for one forward pass\n\n - Input size: [2,512,97,97]\n - Device: 1 NVIDIA GeForce RTX 2080 Ti\n\n +-----------------------+---------------+------------+---------------+\n | |PyTorch version|CUDA version|Relative speed |\n +=======================+===============+============+===============+\n |with torch.no_grad() |0.00554402 s |0.0299619 s |5.4x |\n +-----------------------+---------------+------------+---------------+\n |no with torch.no_grad()|0.00562803 s |0.0301349 s |5.4x |\n +-----------------------+---------------+------------+---------------+\n\n Args:\n in_channels (int): Channels of the input feature map.\n \"\"\"\n\n def __init__(self, in_channels):\n super().__init__()\n self.query_conv = nn.Conv2d(in_channels, in_channels // 8, 1)\n self.key_conv = nn.Conv2d(in_channels, in_channels // 8, 1)\n self.value_conv = nn.Conv2d(in_channels, in_channels, 1)\n self.gamma = Scale(0.)\n self.in_channels = in_channels\n\n def forward(self, x):\n \"\"\"forward function of Criss-Cross Attention.\n\n Args:\n x (torch.Tensor): Input feature with the shape of\n (batch_size, in_channels, height, width).\n\n Returns:\n torch.Tensor: Output of the layer, with the shape of\n (batch_size, in_channels, height, width)\n \"\"\"\n B, C, H, W = x.size()\n query = self.query_conv(x)\n key = self.key_conv(x)\n value = self.value_conv(x)\n energy_H = torch.einsum('bchw,bciw->bwhi', query, key) + NEG_INF_DIAG(\n H, query.device)\n energy_H = energy_H.transpose(1, 2)\n energy_W = torch.einsum('bchw,bchj->bhwj', query, key)\n attn = F.softmax(\n torch.cat([energy_H, energy_W], dim=-1), dim=-1) # [B,H,W,(H+W)]\n out = torch.einsum('bciw,bhwi->bchw', value, attn[..., :H])\n out += torch.einsum('bchj,bhwj->bchw', value, attn[..., H:])\n\n out = self.gamma(out) + x\n out = out.contiguous()\n\n return out\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'(in_channels={self.in_channels})'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/contour_expand.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numpy as np\nimport torch\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['contour_expand'])\n\n\ndef contour_expand(kernel_mask, internal_kernel_label, min_kernel_area,\n kernel_num):\n \"\"\"Expand kernel contours so that foreground pixels are assigned into\n instances.\n\n Args:\n kernel_mask (np.array or torch.Tensor): The instance kernel mask with\n size hxw.\n internal_kernel_label (np.array or torch.Tensor): The instance internal\n kernel label with size hxw.\n min_kernel_area (int): The minimum kernel area.\n kernel_num (int): The instance kernel number.\n\n Returns:\n list: The instance index map with size hxw.\n \"\"\"\n assert isinstance(kernel_mask, (torch.Tensor, np.ndarray))\n assert isinstance(internal_kernel_label, (torch.Tensor, np.ndarray))\n assert isinstance(min_kernel_area, int)\n assert isinstance(kernel_num, int)\n\n if isinstance(kernel_mask, np.ndarray):\n kernel_mask = torch.from_numpy(kernel_mask)\n if isinstance(internal_kernel_label, np.ndarray):\n internal_kernel_label = torch.from_numpy(internal_kernel_label)\n\n if torch.__version__ == 'parrots':\n if kernel_mask.shape[0] == 0 or internal_kernel_label.shape[0] == 0:\n label = []\n else:\n label = ext_module.contour_expand(\n kernel_mask,\n internal_kernel_label,\n min_kernel_area=min_kernel_area,\n kernel_num=kernel_num)\n label = label.tolist()\n else:\n label = ext_module.contour_expand(kernel_mask, internal_kernel_label,\n min_kernel_area, kernel_num)\n return label\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/convex_iou.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['convex_iou', 'convex_giou'])\n\n\ndef convex_giou(pointsets, polygons):\n \"\"\"Return generalized intersection-over-union (Jaccard index) between point\n sets and polygons.\n\n Args:\n pointsets (torch.Tensor): It has shape (N, 18),\n indicating (x1, y1, x2, y2, ..., x9, y9) for each row.\n polygons (torch.Tensor): It has shape (N, 8),\n indicating (x1, y1, x2, y2, x3, y3, x4, y4) for each row.\n\n Returns:\n tuple[torch.Tensor, torch.Tensor]: The first element is the gious\n between point sets and polygons with the shape (N,). The second\n element is the gradient of point sets with the shape (N, 18).\n \"\"\"\n output = pointsets.new_zeros((pointsets.size(0), 19))\n ext_module.convex_giou(pointsets, polygons, output)\n convex_giou = output[:, -1]\n points_grad = output[:, 0:-1]\n return convex_giou, points_grad\n\n\ndef convex_iou(pointsets, polygons):\n \"\"\"Return intersection-over-union (Jaccard index) between point sets and\n polygons.\n\n Args:\n pointsets (torch.Tensor): It has shape (N, 18),\n indicating (x1, y1, x2, y2, ..., x9, y9) for each row.\n polygons (torch.Tensor): It has shape (K, 8),\n indicating (x1, y1, x2, y2, x3, y3, x4, y4) for each row.\n\n Returns:\n torch.Tensor: Return the ious between point sets and polygons with the\n shape (N, K).\n \"\"\"\n N, K = pointsets.size(0), polygons.size(0)\n ious = pointsets.new_zeros((N, K))\n ext_module.convex_iou(pointsets, polygons, ious)\n return ious\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/corner_pool.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch import nn\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', [\n 'top_pool_forward', 'top_pool_backward', 'bottom_pool_forward',\n 'bottom_pool_backward', 'left_pool_forward', 'left_pool_backward',\n 'right_pool_forward', 'right_pool_backward'\n])\n\n_mode_dict = {'top': 0, 'bottom': 1, 'left': 2, 'right': 3}\n\n\nclass TopPoolFunction(Function):\n\n @staticmethod\n def symbolic(g, input):\n output = g.op(\n 'mmcv::MMCVCornerPool', input, mode_i=int(_mode_dict['top']))\n return output\n\n @staticmethod\n def forward(ctx, input):\n output = ext_module.top_pool_forward(input)\n ctx.save_for_backward(input)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n input, = ctx.saved_tensors\n output = ext_module.top_pool_backward(input, grad_output)\n return output\n\n\nclass BottomPoolFunction(Function):\n\n @staticmethod\n def symbolic(g, input):\n output = g.op(\n 'mmcv::MMCVCornerPool', input, mode_i=int(_mode_dict['bottom']))\n return output\n\n @staticmethod\n def forward(ctx, input):\n output = ext_module.bottom_pool_forward(input)\n ctx.save_for_backward(input)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n input, = ctx.saved_tensors\n output = ext_module.bottom_pool_backward(input, grad_output)\n return output\n\n\nclass LeftPoolFunction(Function):\n\n @staticmethod\n def symbolic(g, input):\n output = g.op(\n 'mmcv::MMCVCornerPool', input, mode_i=int(_mode_dict['left']))\n return output\n\n @staticmethod\n def forward(ctx, input):\n output = ext_module.left_pool_forward(input)\n ctx.save_for_backward(input)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n input, = ctx.saved_tensors\n output = ext_module.left_pool_backward(input, grad_output)\n return output\n\n\nclass RightPoolFunction(Function):\n\n @staticmethod\n def symbolic(g, input):\n output = g.op(\n 'mmcv::MMCVCornerPool', input, mode_i=int(_mode_dict['right']))\n return output\n\n @staticmethod\n def forward(ctx, input):\n output = ext_module.right_pool_forward(input)\n ctx.save_for_backward(input)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n input, = ctx.saved_tensors\n output = ext_module.right_pool_backward(input, grad_output)\n return output\n\n\nclass CornerPool(nn.Module):\n \"\"\"Corner Pooling.\n\n Corner Pooling is a new type of pooling layer that helps a\n convolutional network better localize corners of bounding boxes.\n\n Please refer to `CornerNet: Detecting Objects as Paired Keypoints\n `_ for more details.\n\n Code is modified from https://github.com/princeton-vl/CornerNet-Lite.\n\n Args:\n mode (str): Pooling orientation for the pooling layer\n\n - 'bottom': Bottom Pooling\n - 'left': Left Pooling\n - 'right': Right Pooling\n - 'top': Top Pooling\n\n Returns:\n Feature map after pooling.\n \"\"\"\n\n pool_functions = {\n 'bottom': BottomPoolFunction,\n 'left': LeftPoolFunction,\n 'right': RightPoolFunction,\n 'top': TopPoolFunction,\n }\n\n cummax_dim_flip = {\n 'bottom': (2, False),\n 'left': (3, True),\n 'right': (3, False),\n 'top': (2, True),\n }\n\n def __init__(self, mode):\n super(CornerPool, self).__init__()\n assert mode in self.pool_functions\n self.mode = mode\n self.corner_pool = self.pool_functions[mode]\n\n def forward(self, x):\n if torch.__version__ != 'parrots' and torch.__version__ >= '1.5.0':\n if torch.onnx.is_in_onnx_export():\n assert torch.__version__ >= '1.7.0', \\\n 'When `cummax` serves as an intermediate component whose '\\\n 'outputs is used as inputs for another modules, it\\'s '\\\n 'expected that pytorch version must be >= 1.7.0, '\\\n 'otherwise Error appears like: `RuntimeError: tuple '\\\n 'appears in op that does not forward tuples, unsupported '\\\n 'kind: prim::PythonOp`.'\n\n dim, flip = self.cummax_dim_flip[self.mode]\n if flip:\n x = x.flip(dim)\n pool_tensor, _ = torch.cummax(x, dim=dim)\n if flip:\n pool_tensor = pool_tensor.flip(dim)\n return pool_tensor\n else:\n return self.corner_pool.apply(x)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/correlation.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch import Tensor, nn\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\nfrom torch.nn.modules.utils import _pair\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['correlation_forward', 'correlation_backward'])\n\n\nclass CorrelationFunction(Function):\n\n @staticmethod\n def forward(ctx,\n input1,\n input2,\n kernel_size=1,\n max_displacement=1,\n stride=1,\n padding=1,\n dilation=1,\n dilation_patch=1):\n\n ctx.save_for_backward(input1, input2)\n\n kH, kW = ctx.kernel_size = _pair(kernel_size)\n patch_size = max_displacement * 2 + 1\n ctx.patch_size = patch_size\n dH, dW = ctx.stride = _pair(stride)\n padH, padW = ctx.padding = _pair(padding)\n dilationH, dilationW = ctx.dilation = _pair(dilation)\n dilation_patchH, dilation_patchW = ctx.dilation_patch = _pair(\n dilation_patch)\n\n output_size = CorrelationFunction._output_size(ctx, input1)\n\n output = input1.new_zeros(output_size)\n\n ext_module.correlation_forward(\n input1,\n input2,\n output,\n kH=kH,\n kW=kW,\n patchH=patch_size,\n patchW=patch_size,\n padH=padH,\n padW=padW,\n dilationH=dilationH,\n dilationW=dilationW,\n dilation_patchH=dilation_patchH,\n dilation_patchW=dilation_patchW,\n dH=dH,\n dW=dW)\n\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n input1, input2 = ctx.saved_tensors\n\n kH, kW = ctx.kernel_size\n patch_size = ctx.patch_size\n padH, padW = ctx.padding\n dilationH, dilationW = ctx.dilation\n dilation_patchH, dilation_patchW = ctx.dilation_patch\n dH, dW = ctx.stride\n grad_input1 = torch.zeros_like(input1)\n grad_input2 = torch.zeros_like(input2)\n\n ext_module.correlation_backward(\n grad_output,\n input1,\n input2,\n grad_input1,\n grad_input2,\n kH=kH,\n kW=kW,\n patchH=patch_size,\n patchW=patch_size,\n padH=padH,\n padW=padW,\n dilationH=dilationH,\n dilationW=dilationW,\n dilation_patchH=dilation_patchH,\n dilation_patchW=dilation_patchW,\n dH=dH,\n dW=dW)\n return grad_input1, grad_input2, None, None, None, None, None, None\n\n @staticmethod\n def _output_size(ctx, input1):\n iH, iW = input1.size(2), input1.size(3)\n batch_size = input1.size(0)\n kH, kW = ctx.kernel_size\n patch_size = ctx.patch_size\n dH, dW = ctx.stride\n padH, padW = ctx.padding\n dilationH, dilationW = ctx.dilation\n dilatedKH = (kH - 1) * dilationH + 1\n dilatedKW = (kW - 1) * dilationW + 1\n\n oH = int((iH + 2 * padH - dilatedKH) / dH + 1)\n oW = int((iW + 2 * padW - dilatedKW) / dW + 1)\n\n output_size = (batch_size, patch_size, patch_size, oH, oW)\n return output_size\n\n\nclass Correlation(nn.Module):\n r\"\"\"Correlation operator\n\n This correlation operator works for optical flow correlation computation.\n\n There are two batched tensors with shape :math:`(N, C, H, W)`,\n and the correlation output's shape is :math:`(N, max\\_displacement \\times\n 2 + 1, max\\_displacement * 2 + 1, H_{out}, W_{out})`\n\n where\n\n .. math::\n H_{out} = \\left\\lfloor\\frac{H_{in} + 2 \\times padding -\n dilation \\times (kernel\\_size - 1) - 1}\n {stride} + 1\\right\\rfloor\n\n .. math::\n W_{out} = \\left\\lfloor\\frac{W_{in} + 2 \\times padding - dilation\n \\times (kernel\\_size - 1) - 1}\n {stride} + 1\\right\\rfloor\n\n the correlation item :math:`(N_i, dy, dx)` is formed by taking the sliding\n window convolution between input1 and shifted input2,\n\n .. math::\n Corr(N_i, dx, dy) =\n \\sum_{c=0}^{C-1}\n input1(N_i, c) \\star\n \\mathcal{S}(input2(N_i, c), dy, dx)\n\n where :math:`\\star` is the valid 2d sliding window convolution operator,\n and :math:`\\mathcal{S}` means shifting the input features (auto-complete\n zero marginal), and :math:`dx, dy` are shifting distance, :math:`dx, dy \\in\n [-max\\_displacement \\times dilation\\_patch, max\\_displacement \\times\n dilation\\_patch]`.\n\n Args:\n kernel_size (int): The size of sliding window i.e. local neighborhood\n representing the center points and involved in correlation\n computation. Defaults to 1.\n max_displacement (int): The radius for computing correlation volume,\n but the actual working space can be dilated by dilation_patch.\n Defaults to 1.\n stride (int): The stride of the sliding blocks in the input spatial\n dimensions. Defaults to 1.\n padding (int): Zero padding added to all four sides of the input1.\n Defaults to 0.\n dilation (int): The spacing of local neighborhood that will involved\n in correlation. Defaults to 1.\n dilation_patch (int): The spacing between position need to compute\n correlation. Defaults to 1.\n \"\"\"\n\n def __init__(self,\n kernel_size: int = 1,\n max_displacement: int = 1,\n stride: int = 1,\n padding: int = 0,\n dilation: int = 1,\n dilation_patch: int = 1) -> None:\n super().__init__()\n self.kernel_size = kernel_size\n self.max_displacement = max_displacement\n self.stride = stride\n self.padding = padding\n self.dilation = dilation\n self.dilation_patch = dilation_patch\n\n def forward(self, input1: Tensor, input2: Tensor) -> Tensor:\n return CorrelationFunction.apply(input1, input2, self.kernel_size,\n self.max_displacement, self.stride,\n self.padding, self.dilation,\n self.dilation_patch)\n\n def __repr__(self) -> str:\n s = self.__class__.__name__\n s += f'(kernel_size={self.kernel_size}, '\n s += f'max_displacement={self.max_displacement}, '\n s += f'stride={self.stride}, '\n s += f'padding={self.padding}, '\n s += f'dilation={self.dilation}, '\n s += f'dilation_patch={self.dilation_patch})'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/README.md", "content": "# Code Structure of CUDA operators\n\nThis folder contains all non-python code for MMCV custom ops. Please follow the same architecture if you want to add new ops.\n\n## Directories Tree\n\n```folder\n.\n├── common\n│ ├── box_iou_rotated_utils.hpp\n│ ├── parrots_cpp_helper.hpp\n│ ├── parrots_cuda_helper.hpp\n│ ├── pytorch_cpp_helper.hpp\n│ ├── pytorch_cuda_helper.hpp\n│ ├── pytorch_device_registry.hpp\n│   └── cuda\n│   ├── common_cuda_helper.hpp\n│   ├── parrots_cudawarpfunction.cuh\n│   ├── ...\n│   └── ops_cuda_kernel.cuh\n├── onnxruntime\n│   ├── onnxruntime_register.h\n│   ├── onnxruntime_session_options_config_keys.h\n│   ├── ort_mmcv_utils.h\n│   ├── ...\n│   ├── onnx_ops.h\n│   └── cpu\n│ ├── onnxruntime_register.cpp\n│      ├── ...\n│      └── onnx_ops_impl.cpp\n├── parrots\n│   ├── ...\n│   ├── ops.cpp\n│   ├── ops_parrots.cpp\n│   └── ops_pytorch.h\n├── pytorch\n│   ├── info.cpp\n│   ├── pybind.cpp\n│   ├── ...\n│   ├── ops.cpp\n│   ├── cuda\n│   │   ├── ...\n│   │   └── ops_cuda.cu\n│   └── cpu\n│      ├── ...\n│      └── ops.cpp\n└── tensorrt\n ├── trt_cuda_helper.cuh\n ├── trt_plugin_helper.hpp\n ├── trt_plugin.hpp\n ├── trt_serialize.hpp\n ├── ...\n ├── trt_ops.hpp\n └── plugins\n    ├── trt_cuda_helper.cu\n    ├── trt_plugin.cpp\n    ├── ...\n    ├── trt_ops.cpp\n    └── trt_ops_kernel.cu\n```\n\n## Components\n\n- `common`: This directory contains all tools and shared codes.\n - `cuda`: The cuda kernels which can be shared by all backends. **HIP** kernel is also here since they have similar syntax.\n- `onnxruntime`: **ONNX Runtime** support for custom ops.\n - `cpu`: CPU implementation of supported ops.\n- `parrots`: **Parrots** is a deep learning frame for model training and inference. Parrots custom ops are placed in this directory.\n- `pytorch`: **PyTorch** custom ops are supported by binding C++ to Python with **pybind11**. The ops implementation and binding codes are placed in this directory.\n - `cuda`: This directory contains cuda kernel launchers, which feed memory pointers of tensor to the cuda kernel in `common/cuda`. The launchers provide c++ interface of cuda implementation of corresponding custom ops.\n - `cpu`: This directory contain cpu implementations of corresponding custom ops.\n- `tensorrt`: **TensorRT** support for custom ops.\n - `plugins`: This directory contains the implementation of the supported custom ops. Some ops might also use shared cuda kernel in `common/cuda`.\n\n## How to add new PyTorch ops?\n\n1. (Optional) Add shared kernel in `common` to support special hardware platform.\n\n ```c++\n // src/common/cuda/new_ops_cuda_kernel.cuh\n\n template \n __global__ void new_ops_forward_cuda_kernel(const T* input, T* output, ...) {\n // forward here\n }\n\n ```\n\n Add cuda kernel launcher in `pytorch/cuda`.\n\n ```c++\n // src/pytorch/cuda\n #include \n\n void NewOpsForwardCUDAKernelLauncher(Tensor input, Tensor output, ...){\n // initialize\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n ...\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"new_ops_forward_cuda_kernel\", ([&] {\n new_ops_forward_cuda_kernel\n <<>>(\n input.data_ptr(), output.data_ptr(),...);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n }\n ```\n\n2. Register implementation for different devices.\n\n ```c++\n // src/pytorch/cuda/cudabind.cpp\n ...\n\n Tensor new_ops_forward_cuda(Tensor input, Tensor output, ...){\n // implement cuda forward here\n // use `NewOpsForwardCUDAKernelLauncher` here\n }\n // declare interface here.\n Tensor new_ops_forward_impl(Tensor input, Tensor output, ...);\n // register the implementation for given device (CUDA here).\n REGISTER_DEVICE_IMPL(new_ops_forward_impl, CUDA, new_ops_forward_cuda);\n ```\n\n3. Add ops implementation in `pytorch` directory. Select different implementations according to device type.\n\n ```c++\n // src/pytorch/new_ops.cpp\n Tensor new_ops_forward_impl(Tensor input, Tensor output, ...){\n // dispatch the implementation according to the device type of input.\n DISPATCH_DEVICE_IMPL(new_ops_forward_impl, input, output, ...);\n }\n ...\n\n Tensor new_ops_forward(Tensor input, Tensor output, ...){\n return new_ops_forward_impl(input, output, ...);\n }\n ```\n\n4. Binding the implementation in `pytorch/pybind.cpp`\n\n ```c++\n // src/pytorch/pybind.cpp\n\n ...\n\n Tensor new_ops_forward(Tensor input, Tensor output, ...);\n\n ...\n\n // bind with pybind11\n m.def(\"new_ops_forward\", &new_ops_forward, \"new_ops_forward\",\n py::arg(\"input\"), py::arg(\"output\"), ...);\n\n ...\n\n ```\n\n5. Build MMCV again. Enjoy new ops in python\n\n ```python\n from ..utils import ext_loader\n ext_module = ext_loader.load_ext('_ext', ['new_ops_forward'])\n\n ...\n\n ext_module.new_ops_forward(input, output, ...)\n\n ```\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/box_iou_rotated_utils.hpp", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated_utils.h\n#pragma once\n#include \n#include \n\n#ifdef __CUDACC__\n// Designates functions callable from the host (CPU) and the device (GPU)\n#define HOST_DEVICE __host__ __device__\n#define HOST_DEVICE_INLINE HOST_DEVICE __forceinline__\n#else\n#include \n#define HOST_DEVICE\n#define HOST_DEVICE_INLINE HOST_DEVICE inline\n#endif\n\nnamespace {\n\ntemplate \nstruct RotatedBox {\n T x_ctr, y_ctr, w, h, a;\n};\n\ntemplate \nstruct Point {\n T x, y;\n HOST_DEVICE_INLINE Point(const T& px = 0, const T& py = 0) : x(px), y(py) {}\n HOST_DEVICE_INLINE Point operator+(const Point& p) const {\n return Point(x + p.x, y + p.y);\n }\n HOST_DEVICE_INLINE Point& operator+=(const Point& p) {\n x += p.x;\n y += p.y;\n return *this;\n }\n HOST_DEVICE_INLINE Point operator-(const Point& p) const {\n return Point(x - p.x, y - p.y);\n }\n HOST_DEVICE_INLINE Point operator*(const T coeff) const {\n return Point(x * coeff, y * coeff);\n }\n};\n\ntemplate \nHOST_DEVICE_INLINE T dot_2d(const Point& A, const Point& B) {\n return A.x * B.x + A.y * B.y;\n}\n\ntemplate \nHOST_DEVICE_INLINE T cross_2d(const Point& A, const Point& B) {\n return A.x * B.y - B.x * A.y;\n}\n\ntemplate \nHOST_DEVICE_INLINE void get_rotated_vertices(const RotatedBox& box,\n Point (&pts)[4]) {\n // M_PI / 180. == 0.01745329251\n // double theta = box.a * 0.01745329251;\n // MODIFIED\n double theta = box.a;\n T cosTheta2 = (T)cos(theta) * 0.5f;\n T sinTheta2 = (T)sin(theta) * 0.5f;\n\n // y: top --> down; x: left --> right\n pts[0].x = box.x_ctr - sinTheta2 * box.h - cosTheta2 * box.w;\n pts[0].y = box.y_ctr + cosTheta2 * box.h - sinTheta2 * box.w;\n pts[1].x = box.x_ctr + sinTheta2 * box.h - cosTheta2 * box.w;\n pts[1].y = box.y_ctr - cosTheta2 * box.h - sinTheta2 * box.w;\n pts[2].x = 2 * box.x_ctr - pts[0].x;\n pts[2].y = 2 * box.y_ctr - pts[0].y;\n pts[3].x = 2 * box.x_ctr - pts[1].x;\n pts[3].y = 2 * box.y_ctr - pts[1].y;\n}\n\ntemplate \nHOST_DEVICE_INLINE int get_intersection_points(const Point (&pts1)[4],\n const Point (&pts2)[4],\n Point (&intersections)[24]) {\n // Line vector\n // A line from p1 to p2 is: p1 + (p2-p1)*t, t=[0,1]\n Point vec1[4], vec2[4];\n for (int i = 0; i < 4; i++) {\n vec1[i] = pts1[(i + 1) % 4] - pts1[i];\n vec2[i] = pts2[(i + 1) % 4] - pts2[i];\n }\n\n // Line test - test all line combos for intersection\n int num = 0; // number of intersections\n for (int i = 0; i < 4; i++) {\n for (int j = 0; j < 4; j++) {\n // Solve for 2x2 Ax=b\n T det = cross_2d(vec2[j], vec1[i]);\n\n // This takes care of parallel lines\n if (fabs(det) <= 1e-14) {\n continue;\n }\n\n auto vec12 = pts2[j] - pts1[i];\n\n T t1 = cross_2d(vec2[j], vec12) / det;\n T t2 = cross_2d(vec1[i], vec12) / det;\n\n if (t1 >= 0.0f && t1 <= 1.0f && t2 >= 0.0f && t2 <= 1.0f) {\n intersections[num++] = pts1[i] + vec1[i] * t1;\n }\n }\n }\n\n // Check for vertices of rect1 inside rect2\n {\n const auto& AB = vec2[0];\n const auto& DA = vec2[3];\n auto ABdotAB = dot_2d(AB, AB);\n auto ADdotAD = dot_2d(DA, DA);\n for (int i = 0; i < 4; i++) {\n // assume ABCD is the rectangle, and P is the point to be judged\n // P is inside ABCD iff. P's projection on AB lies within AB\n // and P's projection on AD lies within AD\n\n auto AP = pts1[i] - pts2[0];\n\n auto APdotAB = dot_2d(AP, AB);\n auto APdotAD = -dot_2d(AP, DA);\n\n if ((APdotAB >= 0) && (APdotAD >= 0) && (APdotAB <= ABdotAB) &&\n (APdotAD <= ADdotAD)) {\n intersections[num++] = pts1[i];\n }\n }\n }\n\n // Reverse the check - check for vertices of rect2 inside rect1\n {\n const auto& AB = vec1[0];\n const auto& DA = vec1[3];\n auto ABdotAB = dot_2d(AB, AB);\n auto ADdotAD = dot_2d(DA, DA);\n for (int i = 0; i < 4; i++) {\n auto AP = pts2[i] - pts1[0];\n\n auto APdotAB = dot_2d(AP, AB);\n auto APdotAD = -dot_2d(AP, DA);\n\n if ((APdotAB >= 0) && (APdotAD >= 0) && (APdotAB <= ABdotAB) &&\n (APdotAD <= ADdotAD)) {\n intersections[num++] = pts2[i];\n }\n }\n }\n\n return num;\n}\n\ntemplate \nHOST_DEVICE_INLINE int convex_hull_graham(const Point (&p)[24],\n const int& num_in, Point (&q)[24],\n bool shift_to_zero = false) {\n assert(num_in >= 2);\n\n // Step 1:\n // Find point with minimum y\n // if more than 1 points have the same minimum y,\n // pick the one with the minimum x.\n int t = 0;\n for (int i = 1; i < num_in; i++) {\n if (p[i].y < p[t].y || (p[i].y == p[t].y && p[i].x < p[t].x)) {\n t = i;\n }\n }\n auto& start = p[t]; // starting point\n\n // Step 2:\n // Subtract starting point from every points (for sorting in the next step)\n for (int i = 0; i < num_in; i++) {\n q[i] = p[i] - start;\n }\n\n // Swap the starting point to position 0\n auto tmp = q[0];\n q[0] = q[t];\n q[t] = tmp;\n\n // Step 3:\n // Sort point 1 ~ num_in according to their relative cross-product values\n // (essentially sorting according to angles)\n // If the angles are the same, sort according to their distance to origin\n T dist[24];\n for (int i = 0; i < num_in; i++) {\n dist[i] = dot_2d(q[i], q[i]);\n }\n\n#ifdef __CUDACC__\n // CUDA version\n // In the future, we can potentially use thrust\n // for sorting here to improve speed (though not guaranteed)\n for (int i = 1; i < num_in - 1; i++) {\n for (int j = i + 1; j < num_in; j++) {\n T crossProduct = cross_2d(q[i], q[j]);\n if ((crossProduct < -1e-6) ||\n (fabs(crossProduct) < 1e-6 && dist[i] > dist[j])) {\n auto q_tmp = q[i];\n q[i] = q[j];\n q[j] = q_tmp;\n auto dist_tmp = dist[i];\n dist[i] = dist[j];\n dist[j] = dist_tmp;\n }\n }\n }\n#else\n // CPU version\n std::sort(q + 1, q + num_in,\n [](const Point& A, const Point& B) -> bool {\n T temp = cross_2d(A, B);\n if (fabs(temp) < 1e-6) {\n return dot_2d(A, A) < dot_2d(B, B);\n } else {\n return temp > 0;\n }\n });\n // compute distance to origin after sort, since the points are now different.\n for (int i = 0; i < num_in; i++) {\n dist[i] = dot_2d(q[i], q[i]);\n }\n#endif\n\n // Step 4:\n // Make sure there are at least 2 points (that don't overlap with each other)\n // in the stack\n int k; // index of the non-overlapped second point\n for (k = 1; k < num_in; k++) {\n if (dist[k] > 1e-8) {\n break;\n }\n }\n if (k == num_in) {\n // We reach the end, which means the convex hull is just one point\n q[0] = p[t];\n return 1;\n }\n q[1] = q[k];\n int m = 2; // 2 points in the stack\n // Step 5:\n // Finally we can start the scanning process.\n // When a non-convex relationship between the 3 points is found\n // (either concave shape or duplicated points),\n // we pop the previous point from the stack\n // until the 3-point relationship is convex again, or\n // until the stack only contains two points\n for (int i = k + 1; i < num_in; i++) {\n while (m > 1 && cross_2d(q[i] - q[m - 2], q[m - 1] - q[m - 2]) >= 0) {\n m--;\n }\n q[m++] = q[i];\n }\n\n // Step 6 (Optional):\n // In general sense we need the original coordinates, so we\n // need to shift the points back (reverting Step 2)\n // But if we're only interested in getting the area/perimeter of the shape\n // We can simply return.\n if (!shift_to_zero) {\n for (int i = 0; i < m; i++) {\n q[i] += start;\n }\n }\n\n return m;\n}\n\ntemplate \nHOST_DEVICE_INLINE T polygon_area(const Point (&q)[24], const int& m) {\n if (m <= 2) {\n return 0;\n }\n\n T area = 0;\n for (int i = 1; i < m - 1; i++) {\n area += fabs(cross_2d(q[i] - q[0], q[i + 1] - q[0]));\n }\n\n return area / 2.0;\n}\n\ntemplate \nHOST_DEVICE_INLINE T rotated_boxes_intersection(const RotatedBox& box1,\n const RotatedBox& box2) {\n // There are up to 4 x 4 + 4 + 4 = 24 intersections (including dups) returned\n // from rotated_rect_intersection_pts\n Point intersectPts[24], orderedPts[24];\n\n Point pts1[4];\n Point pts2[4];\n get_rotated_vertices(box1, pts1);\n get_rotated_vertices(box2, pts2);\n\n int num = get_intersection_points(pts1, pts2, intersectPts);\n\n if (num <= 2) {\n return 0.0;\n }\n\n // Convex Hull to order the intersection points in clockwise order and find\n // the contour area.\n int num_convex = convex_hull_graham(intersectPts, num, orderedPts, true);\n return polygon_area(orderedPts, num_convex);\n}\n\n} // namespace\n\ntemplate \nHOST_DEVICE_INLINE T single_box_iou_rotated(T const* const box1_raw,\n T const* const box2_raw,\n const int mode_flag) {\n // shift center to the middle point to achieve higher precision in result\n RotatedBox box1, box2;\n auto center_shift_x = (box1_raw[0] + box2_raw[0]) / 2.0;\n auto center_shift_y = (box1_raw[1] + box2_raw[1]) / 2.0;\n box1.x_ctr = box1_raw[0] - center_shift_x;\n box1.y_ctr = box1_raw[1] - center_shift_y;\n box1.w = box1_raw[2];\n box1.h = box1_raw[3];\n box1.a = box1_raw[4];\n box2.x_ctr = box2_raw[0] - center_shift_x;\n box2.y_ctr = box2_raw[1] - center_shift_y;\n box2.w = box2_raw[2];\n box2.h = box2_raw[3];\n box2.a = box2_raw[4];\n\n const T area1 = box1.w * box1.h;\n const T area2 = box2.w * box2.h;\n if (area1 < 1e-14 || area2 < 1e-14) {\n return 0.f;\n }\n\n const T intersection = rotated_boxes_intersection(box1, box2);\n T baseS = 1.0;\n if (mode_flag == 0) {\n baseS = (area1 + area2 - intersection);\n } else if (mode_flag == 1) {\n baseS = area1;\n }\n const T iou = intersection / baseS;\n return iou;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/active_rotated_filter_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/csuhan/s2anet/blob/master/mmdet/ops/orn/src/cuda/ActiveRotatingFilter_cuda.cu\n#ifndef ACTIVE_ROTATED_FILTER_CUDA_KERNEL_CUH\n#define ACTIVE_ROTATED_FILTER_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void active_rotated_filter_forward_cuda_kernel(\n const int nthreads, const scalar_t* weight_data, const int* indices_data,\n const int num_input_planes, const int num_output_planes,\n const int num_orientations, const int num_rotations, const int nEntry,\n scalar_t* output_data) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int l = index % nEntry;\n int j = (index / nEntry) % num_input_planes;\n int i = index / nEntry / num_input_planes;\n int k;\n scalar_t val = *(weight_data + index);\n for (k = 0; k < num_rotations; k++) {\n int idx = (int)(*(indices_data + l * num_rotations + k)) - 1;\n scalar_t* target = output_data +\n i * (num_rotations * num_input_planes * nEntry) +\n k * (num_input_planes * nEntry) + j * (nEntry) + idx;\n *target = val;\n }\n }\n}\n\ntemplate \n__global__ void active_rotated_filter_backward_cuda_kernel(\n const int nthreads, const scalar_t* gradWeight_data,\n const int* indices_data, const int num_input_planes,\n const int num_output_planes, const int num_orientations,\n const int num_rotations, const int nEntry, scalar_t* weight_data) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int l = index % nEntry;\n int j = (index / nEntry) % num_input_planes;\n int i = index / nEntry / num_input_planes;\n int k;\n scalar_t* val = weight_data + index;\n *val = 0;\n scalar_t tmp = 0;\n for (k = 0; k < num_rotations; k++) {\n int idx = (int)(*(indices_data + l * num_rotations + k)) - 1;\n scalar_t target =\n *(gradWeight_data + i * (num_rotations * num_input_planes * nEntry) +\n k * (num_input_planes * nEntry) + j * (nEntry) + idx);\n tmp = tmp + target;\n }\n *val = tmp;\n }\n}\n#endif // ACTIVE_ROTATED_FILTER_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/assign_score_withk_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ASSIGN_SCORE_WITHK_CUDA_KERNEL_CUH\n#define ASSIGN_SCORE_WITHK_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n// input: points(B,N0,M,O), centers(B,N0,M,O), scores(B,N1,K,M), knn_idx(B,N1,K)\n// output: fout(B,O,N)\n// algo: fout(b,i,k,j) = s(b,i,k,m)*p(b,c(i),k,m,j) = s(b,i,k,m)*p(b,i(k),m,j)\n// i(k) = idx(b,i,k)\n// sum: fout(b,i,j) = fout(b,i,j) + s(b,i,k,m)*p(b,i,k,m,j)\n// avg: fout(b,i,j) = sum(fout(b,i,k,j)) / k\n// max: fout(b,i,j) = max(fout(b,i,k,j), sum(s(b,i,k,m)*p(b,i,k,m,j)))\n\ntemplate \n__global__ void assign_score_withk_forward_cuda_kernel(\n const int B, const int N0, const int N1, const int M, const int K,\n const int O, const int aggregate, const T* points, const T* centers,\n const T* scores, const int64_t* knn_idx, T* output) {\n // ----- parallel loop for B, N1, K and O ---------\n CUDA_1D_KERNEL_LOOP(i, B * O * N1 * K) {\n // ------- loop for M ----------\n const int b = (int)(i / (O * N1 * K));\n const int o = (int)(i % (O * N1 * K) / (N1 * K));\n const int n = (int)(i % (N1 * K) / K);\n const int k = (int)(i % K);\n const int cn = (int)knn_idx[b * K * N1 + n * K +\n 0]; // The first neighbor is the center point\n const int kn = (int)knn_idx[b * K * N1 + n * K + k];\n if (kn >= N0 ||\n kn < 0) { // if index overflows, it is out of the neighborhood range\n return;\n }\n assert(b < B);\n assert(kn < N0);\n assert(cn < N0);\n assert(o < O);\n assert(n < N1);\n const int out_idx = b * N1 * O * K + o * N1 * K + n * K + k;\n T val = output[out_idx];\n for (int m = 0; m < M; m++) {\n val += points[b * N0 * M * O + kn * M * O + m * O + o] *\n scores[b * N1 * K * M + n * K * M + k * M + m] -\n centers[b * N0 * M * O + cn * M * O + m * O + o] *\n scores[b * N1 * K * M + n * K * M + k * M + m];\n }\n output[out_idx] = val;\n }\n}\n\ntemplate \n__global__ void assign_score_withk_points_backward_cuda_kernel(\n const int B, const int N0, const int N, const int M, const int K,\n const int O, const int aggregate, const T* grad_out, const T* scores,\n const int64_t* knn_idx, T* grad_points, T* grad_centers) {\n // ----- parallel loop for B, M, O ---------\n CUDA_1D_KERNEL_LOOP(i, B * M * O) {\n int b = (int)(i / (M * O));\n int m = (int)(i % (M * O) / O);\n int o = (int)(i % O);\n\n // ----- loop for N,K ---------\n for (int n = 0; n < N; n++) {\n for (int k = 0; k < K; k++) {\n int kn = knn_idx[b * N * K + n * K + k];\n int cn = knn_idx[b * N * K + n * K + 0];\n if (kn >= N0 || kn < 0) { // if index overflows, it is out of the\n // neighborhood range\n continue;\n }\n atomicAdd(grad_points + b * N0 * M * O + kn * M * O + m * O + o,\n scores[b * N * K * M + n * K * M + k * M + m] *\n grad_out[b * O * N * K + o * N * K + n * K + k]);\n atomicAdd(grad_centers + b * N0 * M * O + cn * M * O + m * O + o,\n -scores[b * N * K * M + n * K * M + k * M + m] *\n grad_out[b * O * N * K + o * N * K + n * K + k]);\n }\n }\n }\n}\n\ntemplate \n__global__ void assign_score_withk_scores_backward_cuda_kernel(\n const int B, const int N0, const int N, const int M, const int K,\n const int O, const int aggregate, const T* grad_out, const T* points,\n const T* centers, const int64_t* knn_idx, T* grad_scores) {\n // ----- parallel loop for B, N, K, M ---------\n CUDA_1D_KERNEL_LOOP(i, B * N * K * M) {\n const int b = (int)(i / (N * M * K));\n const int n = (int)(i % (N * M * K) / M / K);\n const int k = (int)(i % (M * K) / M);\n const int m = (int)(i % M);\n const int cn = knn_idx[b * N * K + n * K + 0];\n const int kn = knn_idx[b * N * K + n * K + k];\n if (kn >= N0 ||\n kn < 0) { // if index overflows, it is out of the neighborhood range\n return;\n }\n\n // -------------- loop for O ------------------------\n const int out_idx = b * N * K * M + n * K * M + k * M + m;\n T val = grad_scores[out_idx];\n for (int o = 0; o < O; o++) {\n val += (points[b * N0 * M * O + kn * M * O + m * O + o] -\n centers[b * N0 * M * O + cn * M * O + m * O + o]) *\n grad_out[b * O * N * K + o * N * K + n * K + k];\n }\n grad_scores[out_idx] = val;\n }\n}\n\n#endif // ASSIGN_SCORE_WITHK_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/ball_query_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/ball_query_gpu.cu\n#ifndef BALL_QUERY_CUDA_KERNEL_CUH\n#define BALL_QUERY_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void ball_query_forward_cuda_kernel(int b, int n, int m,\n float min_radius,\n float max_radius, int nsample,\n const T* new_xyz, const T* xyz,\n int* idx) {\n // new_xyz: (B, M, 3)\n // xyz: (B, N, 3)\n // output:\n // idx: (B, M, nsample)\n int bs_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, m) {\n if (bs_idx >= b) return;\n\n new_xyz += bs_idx * m * 3 + pt_idx * 3;\n xyz += bs_idx * n * 3;\n idx += bs_idx * m * nsample + pt_idx * nsample;\n\n float max_radius2 = max_radius * max_radius;\n float min_radius2 = min_radius * min_radius;\n T new_x = new_xyz[0];\n T new_y = new_xyz[1];\n T new_z = new_xyz[2];\n\n int cnt = 0;\n for (int k = 0; k < n; ++k) {\n T x = xyz[k * 3 + 0];\n T y = xyz[k * 3 + 1];\n T z = xyz[k * 3 + 2];\n T d2 = (new_x - x) * (new_x - x) + (new_y - y) * (new_y - y) +\n (new_z - z) * (new_z - z);\n if (d2 == 0 || (d2 >= min_radius2 && d2 < max_radius2)) {\n if (cnt == 0) {\n for (int l = 0; l < nsample; ++l) {\n idx[l] = k;\n }\n }\n idx[cnt] = k;\n ++cnt;\n if (cnt >= nsample) break;\n }\n }\n }\n}\n\n#endif // BALL_QUERY_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/bbox_overlaps_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef BBOX_OVERLAPS_CUDA_KERNEL_CUH\n#define BBOX_OVERLAPS_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void bbox_overlaps_cuda_kernel(const T* bbox1, const T* bbox2,\n T* ious, const int num_bbox1,\n const int num_bbox2, const int mode,\n const bool aligned,\n const int offset) {\n if (aligned) {\n CUDA_1D_KERNEL_LOOP(index, num_bbox1) {\n int b1 = index;\n int b2 = index;\n\n int base1 = b1 * 4;\n T b1_x1 = bbox1[base1];\n T b1_y1 = bbox1[base1 + 1];\n T b1_x2 = bbox1[base1 + 2];\n T b1_y2 = bbox1[base1 + 3];\n T b1_area = (b1_x2 - b1_x1 + offset) * (b1_y2 - b1_y1 + offset);\n\n int base2 = b2 * 4;\n T b2_x1 = bbox2[base2];\n T b2_y1 = bbox2[base2 + 1];\n T b2_x2 = bbox2[base2 + 2];\n T b2_y2 = bbox2[base2 + 3];\n T b2_area = (b2_x2 - b2_x1 + offset) * (b2_y2 - b2_y1 + offset);\n\n T left = fmaxf(b1_x1, b2_x1), right = fminf(b1_x2, b2_x2);\n T top = fmaxf(b1_y1, b2_y1), bottom = fminf(b1_y2, b2_y2);\n T width = fmaxf(right - left + offset, 0.f);\n T height = fmaxf(bottom - top + offset, 0.f);\n T interS = width * height;\n T baseS = 1.0;\n if (mode == 0) {\n baseS = fmaxf(b1_area + b2_area - interS, T(offset));\n } else if (mode == 1) {\n baseS = fmaxf(b1_area, T(offset));\n }\n ious[index] = interS / baseS;\n }\n } else {\n CUDA_1D_KERNEL_LOOP(index, num_bbox1 * num_bbox2) {\n int b1 = index / num_bbox2;\n int b2 = index % num_bbox2;\n\n int base1 = b1 * 4;\n T b1_x1 = bbox1[base1];\n T b1_y1 = bbox1[base1 + 1];\n T b1_x2 = bbox1[base1 + 2];\n T b1_y2 = bbox1[base1 + 3];\n T b1_area = (b1_x2 - b1_x1 + offset) * (b1_y2 - b1_y1 + offset);\n\n int base2 = b2 * 4;\n T b2_x1 = bbox2[base2];\n T b2_y1 = bbox2[base2 + 1];\n T b2_x2 = bbox2[base2 + 2];\n T b2_y2 = bbox2[base2 + 3];\n T b2_area = (b2_x2 - b2_x1 + offset) * (b2_y2 - b2_y1 + offset);\n\n T left = fmaxf(b1_x1, b2_x1), right = fminf(b1_x2, b2_x2);\n T top = fmaxf(b1_y1, b2_y1), bottom = fminf(b1_y2, b2_y2);\n T width = fmaxf(right - left + offset, 0.f);\n T height = fmaxf(bottom - top + offset, 0.f);\n T interS = width * height;\n T baseS = 1.0;\n if (mode == 0) {\n baseS = fmaxf(b1_area + b2_area - interS, T(offset));\n } else if (mode == 1) {\n baseS = fmaxf(b1_area, T(offset));\n }\n ious[index] = interS / baseS;\n }\n }\n}\n\n#endif // BBOX_OVERLAPS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/border_align_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// modified from\n// https://github.com/Megvii-BaseDetection/cvpods/blob/master/cvpods/layers/csrc/border_align/border_align_kernel.cu.\n// the main difference: (1) use `argmax_idx` for fast computing of gradient\n// during the backward. (2) `wh` is directly computed by `boxes`, rather than\n// passing it as argument to forward or backward functions.\n\n#ifndef BORDER_ALIGN_CUDA_KERNEL_CUH\n#define BORDER_ALIGN_CUDA_KERNEL_CUH\n\n#include \n#ifdef MMCV_WITH_TRT\n#include \"common_cuda_helper.hpp\"\n#else // MMCV_WITH_TRT\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else // MMCV_USE_PARROTS\n#include \"pytorch_cuda_helper.hpp\"\n#endif // MMCV_USE_PARROTS\n#endif // MMCV_WITH_TRT\n\nenum BorderMode { Top = 0, Left = 1, Bottom = 2, Right = 3 };\n\n/*** Forward ***/\ntemplate \n__global__ void border_align_forward_cuda_kernel(\n const int nthreads, const T* input, const T* boxes, T* output,\n int* argmax_idx, const int channels, const int box_size, const int height,\n const int width, const int pool_size) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (batch_idx, c_idx, box_idx) is an element paralleled for computing\n // output, and `extreme_idx` is in range [0,3]\n int batch_idx, c_idx, box_idx, extreme_idx, maxidx, *offset_argmax_idx;\n const T *offset_box, *offset_input, *offset_box_x;\n T *offset_output, box_width, box_height, stride, x_stride, y_stride, x, y,\n val, maxval;\n\n extreme_idx = threadIdx.y;\n // shape (N, C, box_size, 4) for output\n batch_idx = index / channels / box_size;\n // shape (N, box_size, 4) for boxes\n box_idx = index % box_size + batch_idx * box_size;\n c_idx = (index / box_size) % channels;\n\n offset_box = boxes + box_idx * 4;\n box_width = *(offset_box + 2) - *offset_box;\n box_height = *(offset_box + 3) - *(offset_box + 1);\n offset_output = output + index * 4 + extreme_idx;\n offset_argmax_idx = argmax_idx + index * 4 + extreme_idx;\n // shape (N, 4C, h, w) for input.\n // [0,C) for top feature, [C,2C) for left feature,\n // [2C,3C) for bottom feature, [3C,4C) for right feature\n offset_input =\n input + (batch_idx * channels * 4 + extreme_idx * channels + c_idx) *\n height * width;\n\n // extreme_idx in [0,1] -> offset_box_x indexed at x1\n // extreme_idx in [2,3] -> offset_box_x indexed at x2\n offset_box_x = offset_box + extreme_idx / 2 * 2;\n\n // (x1,y1) or (x2,y2) for (x,y)\n x = *offset_box_x;\n y = *(offset_box_x + 1);\n\n switch (extreme_idx) {\n // top\n case BorderMode::Top:\n stride = box_width / pool_size;\n x_stride = stride;\n y_stride = 0;\n break;\n // left\n case BorderMode::Left:\n stride = box_height / pool_size;\n x_stride = 0;\n y_stride = stride;\n break;\n // bottom\n case BorderMode::Bottom:\n stride = box_width / pool_size;\n x_stride = -stride;\n y_stride = 0;\n break;\n // right\n case BorderMode::Right:\n stride = box_height / pool_size;\n x_stride = 0;\n y_stride = -stride;\n break;\n }\n\n // initialize maxval and maxidx with the start position (e.g. (x1,y1) or\n // (x2,y2))\n maxval = bilinear_interpolate(offset_input, height, width, y, x, index);\n maxidx = 0;\n\n // do max_pool along the border\n for (int i = 1; i <= pool_size; i++) {\n x += x_stride;\n y += y_stride;\n val = bilinear_interpolate(offset_input, height, width, y, x, index);\n if (val > maxval) {\n maxval = val;\n maxidx = i;\n }\n }\n\n // update output and argmax_idx\n *offset_output = maxval;\n *offset_argmax_idx = maxidx;\n }\n}\n\n/*** Backward ***/\ntemplate \n__global__ void border_align_backward_cuda_kernel(\n const int nthreads, const T* grad_output, const T* boxes,\n const int* argmax_idx, T* grad_input, const int channels,\n const int box_size, const int height, const int width,\n const int pool_size) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (batch_idx, c_idx, box_idx) is an element paralleled for computing\n // output, and `extreme_idx` is in range [0,3]\n int batch_idx, c_idx, box_idx, extreme_idx;\n const int* offset_argmax_idx;\n const T *offset_grad_output, *offset_box, *offset_box_x;\n T *offset_grad_input, box_width, box_height, stride, x_stride, y_stride, x,\n y;\n\n extreme_idx = threadIdx.y;\n batch_idx = index / channels / box_size;\n box_idx = index % box_size + batch_idx * box_size;\n c_idx = (index / box_size) % channels;\n\n offset_box = boxes + box_idx * 4;\n box_width = *(offset_box + 2) - *offset_box;\n box_height = *(offset_box + 3) - *(offset_box + 1);\n offset_grad_output = grad_output + index * 4 + extreme_idx;\n offset_argmax_idx = argmax_idx + index * 4 + extreme_idx;\n // [0,C) for top feature grad, [C,2C) for left feature grad,\n // [2C,3C) for bottom feature grad, [3C,4C) for right feature grad\n offset_grad_input = grad_input + (batch_idx * channels * 4 +\n extreme_idx * channels + c_idx) *\n height * width;\n\n // extreme_idx in [0,1] -> offset_box_x indexed at x1\n // extreme_idx in [2,3] -> offset_box_x indexed at x2\n offset_box_x = offset_box + extreme_idx / 2 * 2;\n\n switch (extreme_idx) {\n // top\n case BorderMode::Top:\n stride = box_width / pool_size;\n x_stride = stride;\n y_stride = 0;\n break;\n // left\n case BorderMode::Left:\n stride = box_height / pool_size;\n x_stride = 0;\n y_stride = stride;\n break;\n // bottom\n case BorderMode::Bottom:\n stride = box_width / pool_size;\n x_stride = -stride;\n y_stride = 0;\n break;\n // right\n case BorderMode::Right:\n stride = box_height / pool_size;\n x_stride = 0;\n y_stride = -stride;\n break;\n }\n\n // get position (x,y) which has maximum value during forward\n x = *offset_box_x;\n y = *(offset_box_x + 1);\n x += x_stride * (T)(*offset_argmax_idx);\n y += y_stride * (T)(*offset_argmax_idx);\n\n T w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3, w4, x_low,\n x_high, y_low, y_high, index);\n\n // update grad_output\n atomicAdd(offset_grad_input + y_low * width + x_low,\n *offset_grad_output * w1);\n atomicAdd(offset_grad_input + y_low * width + x_high,\n *offset_grad_output * w2);\n atomicAdd(offset_grad_input + y_high * width + x_low,\n *offset_grad_output * w3);\n atomicAdd(offset_grad_input + y_high * width + x_high,\n *offset_grad_output * w4);\n }\n}\n\n#endif // BORDER_ALIGN_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/box_iou_rotated_cuda.cuh", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated_cuda.cu\n#ifndef BOX_IOU_ROTATED_CUDA_CUH\n#define BOX_IOU_ROTATED_CUDA_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n#include \"box_iou_rotated_utils.hpp\"\n\n// 2D block with 32 * 16 = 512 threads per block\nconst int BLOCK_DIM_X = 32;\nconst int BLOCK_DIM_Y = 16;\n\ninline int divideUP(const int x, const int y) { return (((x) + (y)-1) / (y)); }\n\ntemplate \n__global__ void box_iou_rotated_cuda_kernel(\n const int n_boxes1, const int n_boxes2, const T* dev_boxes1,\n const T* dev_boxes2, T* dev_ious, const int mode_flag, const bool aligned) {\n if (aligned) {\n CUDA_1D_KERNEL_LOOP(index, n_boxes1) {\n int b1 = index;\n int b2 = index;\n\n int base1 = b1 * 5;\n\n float block_boxes1[5];\n float block_boxes2[5];\n\n block_boxes1[0] = dev_boxes1[base1 + 0];\n block_boxes1[1] = dev_boxes1[base1 + 1];\n block_boxes1[2] = dev_boxes1[base1 + 2];\n block_boxes1[3] = dev_boxes1[base1 + 3];\n block_boxes1[4] = dev_boxes1[base1 + 4];\n\n int base2 = b2 * 5;\n\n block_boxes2[0] = dev_boxes2[base2 + 0];\n block_boxes2[1] = dev_boxes2[base2 + 1];\n block_boxes2[2] = dev_boxes2[base2 + 2];\n block_boxes2[3] = dev_boxes2[base2 + 3];\n block_boxes2[4] = dev_boxes2[base2 + 4];\n\n dev_ious[index] =\n single_box_iou_rotated(block_boxes1, block_boxes2, mode_flag);\n }\n } else {\n CUDA_1D_KERNEL_LOOP(index, n_boxes1 * n_boxes2) {\n int b1 = index / n_boxes2;\n int b2 = index % n_boxes2;\n\n int base1 = b1 * 5;\n\n float block_boxes1[5];\n float block_boxes2[5];\n\n block_boxes1[0] = dev_boxes1[base1 + 0];\n block_boxes1[1] = dev_boxes1[base1 + 1];\n block_boxes1[2] = dev_boxes1[base1 + 2];\n block_boxes1[3] = dev_boxes1[base1 + 3];\n block_boxes1[4] = dev_boxes1[base1 + 4];\n\n int base2 = b2 * 5;\n\n block_boxes2[0] = dev_boxes2[base2 + 0];\n block_boxes2[1] = dev_boxes2[base2 + 1];\n block_boxes2[2] = dev_boxes2[base2 + 2];\n block_boxes2[3] = dev_boxes2[base2 + 3];\n block_boxes2[4] = dev_boxes2[base2 + 4];\n\n dev_ious[index] =\n single_box_iou_rotated(block_boxes1, block_boxes2, mode_flag);\n }\n }\n}\n\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/carafe_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CARAFE_CUDA_KERNEL_CUH\n#define CARAFE_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n#ifdef HIP_DIFF\n#define WARP_SIZE 64\n#else\n#define WARP_SIZE 32\n#endif\n#define THREADS_PER_PIXEL 32\n#define MAX_SHARED_MEMORY 49152\n#define MAX_SHARED_SCALAR_T 6144 // 49152 / 8 = 6144\n#define MAXIMIZE_KERNEL_SIZE true\n#define kTileDim 32\n#define kBlockRows 8\n#define FULL_MASK 0xffffffff\n\ninline int divideUP(const int x, const int y) { return (((x) + (y)-1) / (y)); }\n\n__device__ inline int Loc2Index(const int n, const int c, const int h,\n const int w, const int channel_num,\n const int height, const int width) {\n int index = w + (h + (c + n * channel_num) * height) * width;\n return index;\n}\n#ifndef HIP_DIFF\n/* TODO: move this to a common place */\ntemplate \n__device__ inline scalar_t min(scalar_t a, scalar_t b) {\n return a < b ? a : b;\n}\n\ntemplate \n__device__ inline scalar_t max(scalar_t a, scalar_t b) {\n return a > b ? a : b;\n}\n#endif\ntemplate \n__device__ __forceinline__ scalar_t warpReduceSum(scalar_t val) {\n for (int offset = WARP_SIZE / 2; offset > 0; offset /= 2)\n#ifdef HIP_DIFF\n val += __shfl_down(val, offset);\n#else\n val += __shfl_down_sync(FULL_MASK, val, offset);\n#endif\n return val;\n}\n\ntemplate <>\n__device__ __forceinline__ phalf warpReduceSum(phalf val) {\n for (int offset = WARP_SIZE / 2; offset > 0; offset /= 2)\n#ifdef HIP_DIFF\n __PHALF(val) += __shfl_down(FULL_MASK, val, offset);\n#else\n __PHALF(val) +=\n __shfl_down_sync(FULL_MASK, static_cast<__half>(__PHALF(val)), offset);\n#endif\n return val;\n}\n\n// Splits the original matrix into submatrices with size 32 * 32.\n// Each block transposes one submatrix by loading it into shared memory.\n// Reference https://devblogs.nvidia.com/efficient-matrix-transpose-cuda-cc/\ntemplate \n__global__ void BatchTranspose2DCUDAKernel(const int N, const int H,\n const int W, const int dh,\n const int dw,\n const scalar_t *__restrict__ X,\n scalar_t *__restrict__ Y) {\n __shared__ scalar_t tile[kTileDim][kTileDim + 1];\n const int n = blockIdx.x / (dh * dw);\n const int k = blockIdx.x % (dh * dw);\n const int r = k / dw;\n const int c = k % dw;\n const int offset = n * H * W;\n int x = c * kTileDim + threadIdx.x;\n int y = r * kTileDim + threadIdx.y;\n if (x < W) {\n for (int i = 0; threadIdx.y + i < kTileDim && y + i < H; i += kBlockRows) {\n tile[threadIdx.y + i][threadIdx.x] = X[offset + (y + i) * W + x];\n }\n }\n __syncthreads();\n x = r * kTileDim + threadIdx.x;\n y = c * kTileDim + threadIdx.y;\n if (x < H) {\n for (int i = 0; threadIdx.y + i < kTileDim && y + i < W; i += kBlockRows) {\n Y[offset + (y + i) * H + x] = tile[threadIdx.x][threadIdx.y + i];\n }\n }\n}\ntemplate \n__global__ void CARAFEForward(\n const int num_kernels, const scalar_t *__restrict__ bottom_data,\n const scalar_t *__restrict__ bottom_masks, const int kernel_size,\n const int group_size, const int scale_factor, const int channels,\n const int down_height, const int down_width, const int height,\n const int width, const int mask_channels, scalar_t *__restrict__ top_data) {\n#if MAXIMIZE_KERNEL_SIZE\n __shared__ float shared_mask[MAX_SHARED_SCALAR_T * 2];\n#else\n __shared__ scalar_t shared_mask[MAX_SHARED_SCALAR_T];\n#endif\n\n int index = threadIdx.x + blockIdx.x * blockDim.x;\n if (index > num_kernels - 1) {\n return;\n }\n const int pixel_id = threadIdx.x / THREADS_PER_PIXEL;\n const int split_id = threadIdx.x % THREADS_PER_PIXEL;\n index = index / THREADS_PER_PIXEL;\n const int pw = index % width;\n const int ph = (index / width) % height;\n const int n = index / width / height;\n\n const int down_pw = pw / scale_factor;\n const int down_ph = ph / scale_factor;\n\n const int start_w = down_pw - (kernel_size - 1) / 2;\n const int end_w = down_pw + (kernel_size - 1) / 2 + 1;\n const int start_h = down_ph - (kernel_size - 1) / 2;\n const int end_h = down_ph + (kernel_size - 1) / 2 + 1;\n for (int c = split_id; c < mask_channels; c += THREADS_PER_PIXEL) {\n int mask_index = Loc2Index(n, ph, pw, c, height, width, mask_channels);\n shared_mask[c * WARP_SIZE + pixel_id] = bottom_masks[mask_index];\n }\n __syncthreads();\n\n const int channels_per_group = ceilf(channels / (float)group_size);\n#pragma unroll\n for (int c = split_id; c < channels; c += THREADS_PER_PIXEL) {\n int mask_group = c / channels_per_group;\n scalar_t output_val = 0;\n#pragma unroll\n for (int iy = start_h; iy < end_h; iy++) {\n#pragma unroll\n for (int ix = start_w; ix < end_w; ix++) {\n if (iy < 0 || iy > down_height - 1 || ix < 0 || ix > down_width - 1) {\n continue;\n }\n int mask_iy = iy - down_ph + (kernel_size - 1) / 2;\n int mask_ix = ix - down_pw + (kernel_size - 1) / 2;\n int mask_c =\n (mask_group * kernel_size + mask_iy) * kernel_size + mask_ix;\n int feat_index =\n Loc2Index(n, iy, ix, c, down_height, down_width, channels);\n\n output_val += bottom_data[feat_index] *\n shared_mask[mask_c * WARP_SIZE + pixel_id];\n }\n }\n\n int top_index = Loc2Index(n, ph, pw, c, height, width, channels);\n top_data[top_index] = output_val;\n }\n}\n\ntemplate \n__global__ void CARAFEBackward_Feature(\n const int num_kernels, const scalar_t *__restrict__ top_diff,\n const scalar_t *__restrict__ bottom_masks, const int kernel_size,\n const int group_size, const int scale_factor, const int channels,\n const int down_height, const int down_width, const int height,\n const int width, const int mask_channels,\n scalar_t *__restrict__ bottom_diff) {\n#if MAXIMIZE_KERNEL_SIZE\n __shared__ float shared_mask[MAX_SHARED_SCALAR_T * 2];\n#else\n __shared__ scalar_t shared_mask[MAX_SHARED_SCALAR_T];\n#endif\n\n int index = threadIdx.x + blockIdx.x * blockDim.x;\n if (index > num_kernels - 1) {\n return;\n }\n\n const int pixel_id = threadIdx.x / THREADS_PER_PIXEL;\n const int split_id = threadIdx.x % THREADS_PER_PIXEL;\n // (n, c, ph, pw) is an element in the bottom_data\n index = index / THREADS_PER_PIXEL;\n const int pw = index % width;\n const int ph = (index / width) % height;\n const int n = index / width / height;\n\n const int start_w = pw - (kernel_size - 1) * scale_factor / 2;\n const int end_w = pw + (kernel_size - 1) * scale_factor / 2 + 1;\n const int start_h = ph - (kernel_size - 1) * scale_factor / 2;\n const int end_h = ph + (kernel_size - 1) * scale_factor / 2 + 1;\n for (int c = split_id; c < mask_channels; c += THREADS_PER_PIXEL) {\n const int mask_w = (c % kernel_size) * scale_factor;\n const int mask_h = (c / kernel_size % kernel_size) * scale_factor;\n const int mask_x = start_w + mask_w;\n const int mask_y = start_h + mask_h;\n if (mask_y < 0 || mask_y > height - 1 || mask_x < 0 || mask_x > width - 1) {\n shared_mask[c * WARP_SIZE + pixel_id] = 0;\n continue;\n }\n const int mask_group = c / (kernel_size * kernel_size);\n const int mask_c = (2 * mask_group + 1) * kernel_size * kernel_size - c - 1;\n int mask_index =\n Loc2Index(n, mask_c, mask_y, mask_x, mask_channels, height, width);\n shared_mask[c * WARP_SIZE + pixel_id] = bottom_masks[mask_index];\n }\n __syncthreads();\n const int channels_per_group = ceilf(channels / (float)group_size);\n#pragma unroll\n for (int c = split_id; c < channels; c += THREADS_PER_PIXEL) {\n int mask_group = c / channels_per_group;\n int top_index = Loc2Index(n, ph, pw, c, height, width, channels);\n scalar_t output_val = 0;\n#pragma unroll\n for (int iy = start_h; iy < end_h; iy += scale_factor) {\n#pragma unroll\n for (int ix = start_w; ix < end_w; ix += scale_factor) {\n if (iy < 0 || iy > height - 1 || ix < 0 || ix > width - 1) {\n continue;\n }\n int mask_iy =\n (iy - ph + (kernel_size - 1) * scale_factor / 2) / scale_factor;\n int mask_ix =\n (ix - pw + (kernel_size - 1) * scale_factor / 2) / scale_factor;\n int mask_c =\n (mask_group * kernel_size + mask_iy) * kernel_size + mask_ix;\n int feat_index = Loc2Index(n, iy, ix, c, height, width, channels);\n output_val +=\n shared_mask[mask_c * WARP_SIZE + pixel_id] * top_diff[feat_index];\n }\n }\n bottom_diff[top_index] = output_val;\n }\n}\n\ntemplate \n__global__ void FeatureSum(const int num_kernels,\n const scalar_t *__restrict__ input_data,\n const int scale_factor, const int channels,\n const int height, const int width,\n scalar_t *__restrict__ output_data) {\n int index = threadIdx.x + blockIdx.x * blockDim.x;\n if (index > num_kernels - 1) {\n return;\n }\n const int split_id = threadIdx.x % THREADS_PER_PIXEL;\n index = index / THREADS_PER_PIXEL;\n const int pw = index % width;\n const int ph = (index / width) % height;\n const int n = index / width / height;\n for (int c = split_id; c < channels; c += THREADS_PER_PIXEL) {\n scalar_t output_val = 0;\n for (int iy = ph * scale_factor; iy < (ph + 1) * scale_factor; iy++) {\n for (int ix = pw * scale_factor; ix < (pw + 1) * scale_factor; ix++) {\n int input_id = Loc2Index(n, iy, ix, c, height * scale_factor,\n width * scale_factor, channels);\n output_val += input_data[input_id];\n }\n }\n const int output_id = Loc2Index(n, ph, pw, c, height, width, channels);\n output_data[output_id] = output_val;\n }\n}\n\ntemplate \n__global__ void CARAFEBackward_Mask(const int num_kernels,\n const scalar_t *__restrict__ top_diff,\n const scalar_t *__restrict__ bottom_data,\n const int kernel_size, const int group_size,\n const int scale_factor, const int channels,\n const int down_height, const int down_width,\n const int height, const int width,\n const int mask_channels,\n scalar_t *__restrict__ mask_diff) {\n int index = threadIdx.x + blockIdx.x * blockDim.x;\n if (index > num_kernels - 1) {\n return;\n }\n\n const int lane_id = index % WARP_SIZE;\n index = index / WARP_SIZE;\n const int mask_c = index % mask_channels;\n // (n, c, ph, pw) is an element in the bottom_data\n index = index / mask_channels;\n const int pw = index % width;\n const int ph = (index / width) % height;\n const int n = index / width / height;\n\n const int down_pw = pw / scale_factor;\n const int down_ph = ph / scale_factor;\n\n const int mask_group = mask_c / (kernel_size * kernel_size);\n const int mask_loc = mask_c % (kernel_size * kernel_size);\n\n const int offset_x = mask_loc % kernel_size - (kernel_size - 1) / 2;\n const int offset_y =\n mask_loc / kernel_size % kernel_size - (kernel_size - 1) / 2;\n\n const int down_x = down_pw + offset_x;\n const int down_y = down_ph + offset_y;\n\n scalar_t output_val = 0;\n\n if (down_y >= 0 && down_y <= down_height - 1 && down_x >= 0 &&\n down_x <= down_width - 1) {\n const int channels_per_mask = ceilf(channels / (float)group_size);\n const int start = channels_per_mask * mask_group;\n const int end = min(channels_per_mask * (mask_group + 1), channels);\n for (int c = start + lane_id; c < end; c += WARP_SIZE) {\n int bottom_id =\n Loc2Index(n, down_y, down_x, c, down_height, down_width, channels);\n int top_id = Loc2Index(n, ph, pw, c, height, width, channels);\n output_val += top_diff[top_id] * bottom_data[bottom_id];\n }\n }\n#ifdef HIP_DIFF\n __syncthreads();\n#else\n __syncwarp();\n#endif\n output_val = warpReduceSum(output_val);\n if (lane_id == 0) {\n const int mask_id =\n Loc2Index(n, ph, pw, mask_c, height, width, mask_channels);\n mask_diff[mask_id] = output_val;\n }\n}\n\n#endif // CARAFE_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/carafe_naive_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CARAFE_NAIVE_CUDA_KERNEL_CUH\n#define CARAFE_NAIVE_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n__device__ inline int Loc2Index(const int n, const int c, const int h,\n const int w, const int channel_num,\n const int height, const int width) {\n int index = w + (h + (c + n * channel_num) * height) * width;\n return index;\n}\n\ntemplate \n__global__ void carafe_naive_forward_cuda_kernel(\n const int nthreads, const scalar_t *bottom_data,\n const scalar_t *bottom_masks, scalar_t *top_data, const int kernel_size,\n const int group_size, const int scale_factor, const int channels,\n const int height, const int width) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the bottom_data\n int pw = index % width;\n int ph = (index / width) % height;\n int c = (index / width / height) % channels;\n int n = index / width / height / channels;\n\n int mask_channels = kernel_size * kernel_size * group_size;\n int mask_group = c / (channels / group_size);\n\n int down_pw = pw / scale_factor;\n int down_ph = ph / scale_factor;\n int down_width = width / scale_factor;\n int down_height = height / scale_factor;\n int start_w = down_pw - (kernel_size - 1) / 2;\n int end_w = down_pw + (kernel_size - 1) / 2 + 1;\n int start_h = down_ph - (kernel_size - 1) / 2;\n int end_h = down_ph + (kernel_size - 1) / 2 + 1;\n\n scalar_t output_val = 0;\n for (int iy = start_h; iy < end_h; iy++) {\n for (int ix = start_w; ix < end_w; ix++) {\n if (iy < 0 || iy > down_height - 1 || ix < 0 || ix > down_width - 1) {\n continue;\n }\n int mask_iy = iy - down_ph + (kernel_size - 1) / 2;\n int mask_ix = ix - down_pw + (kernel_size - 1) / 2;\n int mask_c =\n (mask_group * kernel_size + mask_iy) * kernel_size + mask_ix;\n int feat_index =\n Loc2Index(n, c, iy, ix, channels, down_height, down_width);\n int mask_index =\n Loc2Index(n, mask_c, ph, pw, mask_channels, height, width);\n output_val += bottom_data[feat_index] * bottom_masks[mask_index];\n }\n }\n top_data[index] = output_val;\n }\n}\n\ntemplate \n__global__ void carafe_naive_backward_cuda_kernel(\n const int nthreads, const scalar_t *top_diff, const scalar_t *bottom_data,\n const scalar_t *bottom_masks, scalar_t *bottom_diff, scalar_t *mask_diff,\n const int kernel_size, const int group_size, const int scale_factor,\n const int channels, const int height, const int width) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the bottom_data\n int pw = index % width;\n int ph = (index / width) % height;\n int c = (index / width / height) % channels;\n int n = index / width / height / channels;\n\n int mask_channels = kernel_size * kernel_size * group_size;\n int mask_group = c / (channels / group_size);\n\n int down_pw = pw / scale_factor;\n int down_ph = ph / scale_factor;\n int down_width = width / scale_factor;\n int down_height = height / scale_factor;\n int start_w = down_pw - (kernel_size - 1) / 2;\n int end_w = down_pw + (kernel_size - 1) / 2 + 1;\n int start_h = down_ph - (kernel_size - 1) / 2;\n int end_h = down_ph + (kernel_size - 1) / 2 + 1;\n\n for (int iy = start_h; iy < end_h; iy++) {\n for (int ix = start_w; ix < end_w; ix++) {\n if (iy < 0 || iy > down_height - 1 || ix < 0 || ix > down_width - 1) {\n continue;\n }\n int mask_iy = iy - down_ph + (kernel_size - 1) / 2;\n int mask_ix = ix - down_pw + (kernel_size - 1) / 2;\n int mask_c =\n (mask_group * kernel_size + mask_iy) * kernel_size + mask_ix;\n int feat_index =\n Loc2Index(n, c, iy, ix, channels, down_height, down_width);\n int mask_index =\n Loc2Index(n, mask_c, ph, pw, mask_channels, height, width);\n atomicAdd(bottom_diff + feat_index,\n bottom_masks[mask_index] * top_diff[index]);\n atomicAdd(mask_diff + mask_index,\n bottom_data[feat_index] * top_diff[index]);\n }\n }\n }\n}\n\n#endif // CARAFE_NAIVE_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/common_cuda_helper.hpp", "content": "#ifndef COMMON_CUDA_HELPER\n#define COMMON_CUDA_HELPER\n\n#include \n\n#define CUDA_1D_KERNEL_LOOP(i, n) \\\n for (int i = blockIdx.x * blockDim.x + threadIdx.x; i < (n); \\\n i += blockDim.x * gridDim.x)\n\n#define CUDA_2D_KERNEL_LOOP(i, n, j, m) \\\n for (size_t i = blockIdx.x * blockDim.x + threadIdx.x; i < (n); \\\n i += blockDim.x * gridDim.x) \\\n for (size_t j = blockIdx.y * blockDim.y + threadIdx.y; j < (m); \\\n j += blockDim.y * gridDim.y)\n\n#define CUDA_2D_KERNEL_BLOCK_LOOP(i, n, j, m) \\\n for (size_t i = blockIdx.x; i < (n); i += gridDim.x) \\\n for (size_t j = blockIdx.y; j < (m); j += gridDim.y)\n\n#define THREADS_PER_BLOCK 512\n\ninline int GET_BLOCKS(const int N, const int num_threads = THREADS_PER_BLOCK) {\n int optimal_block_num = (N + num_threads - 1) / num_threads;\n int max_block_num = 4096;\n return min(optimal_block_num, max_block_num);\n}\n\ntemplate \n__device__ T bilinear_interpolate(const T* input, const int height,\n const int width, T y, T x,\n const int index /* index for debug only*/) {\n // deal with cases that inverse elements are out of feature map boundary\n if (y < -1.0 || y > height || x < -1.0 || x > width) return 0;\n\n if (y <= 0) y = 0;\n if (x <= 0) x = 0;\n\n int y_low = (int)y;\n int x_low = (int)x;\n int y_high;\n int x_high;\n\n if (y_low >= height - 1) {\n y_high = y_low = height - 1;\n y = (T)y_low;\n } else {\n y_high = y_low + 1;\n }\n\n if (x_low >= width - 1) {\n x_high = x_low = width - 1;\n x = (T)x_low;\n } else {\n x_high = x_low + 1;\n }\n\n T ly = y - y_low;\n T lx = x - x_low;\n T hy = 1. - ly, hx = 1. - lx;\n // do bilinear interpolation\n T v1 = input[y_low * width + x_low];\n T v2 = input[y_low * width + x_high];\n T v3 = input[y_high * width + x_low];\n T v4 = input[y_high * width + x_high];\n T w1 = hy * hx, w2 = hy * lx, w3 = ly * hx, w4 = ly * lx;\n\n T val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n\n return val;\n}\n\ntemplate \n__device__ void bilinear_interpolate_gradient(\n const int height, const int width, T y, T x, T& w1, T& w2, T& w3, T& w4,\n int& x_low, int& x_high, int& y_low, int& y_high,\n const int index /* index for debug only*/) {\n // deal with cases that inverse elements are out of feature map boundary\n if (y < -1.0 || y > height || x < -1.0 || x > width) {\n // empty\n w1 = w2 = w3 = w4 = 0.;\n x_low = x_high = y_low = y_high = -1;\n return;\n }\n\n if (y <= 0) y = 0;\n if (x <= 0) x = 0;\n\n y_low = (int)y;\n x_low = (int)x;\n\n if (y_low >= height - 1) {\n y_high = y_low = height - 1;\n y = (T)y_low;\n } else {\n y_high = y_low + 1;\n }\n\n if (x_low >= width - 1) {\n x_high = x_low = width - 1;\n x = (T)x_low;\n } else {\n x_high = x_low + 1;\n }\n\n T ly = y - y_low;\n T lx = x - x_low;\n T hy = 1. - ly, hx = 1. - lx;\n\n // reference in forward\n // T v1 = input[y_low * width + x_low];\n // T v2 = input[y_low * width + x_high];\n // T v3 = input[y_high * width + x_low];\n // T v4 = input[y_high * width + x_high];\n // T val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n\n w1 = hy * hx, w2 = hy * lx, w3 = ly * hx, w4 = ly * lx;\n\n return;\n}\n#endif // COMMON_CUDA_HELPER\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/convex_iou_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CONVEX_IOU_CUDA_KERNEL_CUH\n#define CONVEX_IOU_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n#define MAXN 100\n#define NMAX 512\n__device__ const double EPS = 1E-8;\n\n__device__ inline int sig(double d) { return (d > EPS) - (d < -EPS); }\n\nstruct Point {\n double x, y;\n __device__ Point() {}\n __device__ Point(double x, double y) : x(x), y(y) {}\n};\n\n__device__ inline bool point_same(Point& a, Point& b) {\n return sig(a.x - b.x) == 0 && sig(a.y - b.y) == 0;\n}\n\n__device__ inline void swap1(Point* a, Point* b) {\n Point temp;\n temp.x = a->x;\n temp.y = a->y;\n\n a->x = b->x;\n a->y = b->y;\n\n b->x = temp.x;\n b->y = temp.y;\n}\n\n__device__ inline void reverse1(Point* a, const int n) {\n for (int i = 0; i < (n - 1) / 2.0; i++) {\n Point* j = &(a[i]);\n Point* k = &(a[n - 1 - i]);\n swap1(j, k);\n }\n}\n\n__device__ inline double cross(Point o, Point a, Point b) {\n return (a.x - o.x) * (b.y - o.y) - (b.x - o.x) * (a.y - o.y);\n}\n\n__device__ inline double dis(Point a, Point b) {\n return (a.x - b.x) * (a.x - b.x) + (a.y - b.y) * (a.y - b.y);\n}\n__device__ inline double area(Point* ps, int n) {\n ps[n] = ps[0];\n double res = 0;\n for (int i = 0; i < n; i++) {\n res += ps[i].x * ps[i + 1].y - ps[i].y * ps[i + 1].x;\n }\n return res / 2.0;\n}\n__device__ inline double polygon_area_grad(Point* ps, int n,\n int* polygon_to_pred_index,\n int n_pred, double* grad_C) {\n ps[n] = ps[0];\n double partion_grad[4 * 30 + 2];\n double res = 0;\n for (int i = 0; i < n; i++) {\n res += ps[i].x * ps[i + 1].y - ps[i].y * ps[i + 1].x;\n partion_grad[i * 4 + 2] = ps[i + 1].y;\n partion_grad[i * 4 + 3] = -ps[i + 1].x;\n if (i != n - 1) {\n partion_grad[i * 4 + 4] = -ps[i].y;\n partion_grad[i * 4 + 5] = ps[i].x;\n } else {\n partion_grad[0] = -ps[i].y;\n partion_grad[1] = ps[i].x;\n }\n }\n for (int i = 0; i < n; i++) {\n for (int j = 0; j < n_pred; j++) {\n if (i == polygon_to_pred_index[j]) {\n grad_C[2 * polygon_to_pred_index[j + n_pred]] =\n (partion_grad[i * 4] + partion_grad[i * 4 + 2]) / 2;\n break;\n }\n }\n for (int j = 0; j < n_pred; j++) {\n if (i == polygon_to_pred_index[j]) {\n grad_C[2 * polygon_to_pred_index[j + n_pred] + 1] =\n (partion_grad[i * 4 + 1] + partion_grad[i * 4 + 1 + 2]) / 2;\n break;\n }\n }\n }\n\n return res / 2.0;\n}\n\n__device__ inline int lineCross(Point a, Point b, Point c, Point d, Point& p,\n double* cut_grad, int m, int n, int i) {\n double s1, s2;\n double s2_s1_2;\n double ds1_dxc, ds1_dyc, ds2_dxd, ds2_dyd;\n double dxp_dxc, dxp_dyc, dxp_dxd, dxp_dyd, dyp_dxc, dyp_dyc, dyp_dxd, dyp_dyd;\n s1 = cross(a, b, c);\n s2 = cross(a, b, d);\n\n ds1_dxc = -(b.y - a.y);\n ds1_dyc = b.x - a.x;\n ds2_dxd = ds1_dxc;\n ds2_dyd = ds1_dyc;\n s2_s1_2 = (s2 - s1) * (s2 - s1);\n\n if (sig(s1) == 0 && sig(s2) == 0) return 2;\n if (sig(s2 - s1) == 0) return 0;\n\n dxp_dxc =\n ((s2 - d.x * ds1_dxc) * (s2 - s1) - (c.x * s2 - d.x * s1) * (-ds1_dxc)) /\n (s2_s1_2);\n dxp_dyc =\n ((0 - d.x * ds1_dyc) * (s2 - s1) - (c.x * s2 - d.x * s1) * (-ds1_dyc)) /\n (s2_s1_2);\n dxp_dxd =\n ((c.x * ds2_dxd - s1) * (s2 - s1) - (c.x * s2 - d.x * s1) * (ds2_dxd)) /\n (s2_s1_2);\n dxp_dyd =\n ((c.x * ds2_dyd - 0) * (s2 - s1) - (c.x * s2 - d.x * s1) * (ds2_dyd)) /\n (s2_s1_2);\n\n dyp_dxc =\n ((0 - d.y * ds1_dxc) * (s2 - s1) - (c.y * s2 - d.y * s1) * (-ds1_dxc)) /\n (s2_s1_2);\n dyp_dyc =\n ((s2 - d.y * ds1_dyc) * (s2 - s1) - (c.y * s2 - d.y * s1) * (-ds1_dyc)) /\n (s2_s1_2);\n dyp_dxd =\n ((c.y * ds2_dxd - 0) * (s2 - s1) - (c.y * s2 - d.y * s1) * (ds2_dxd)) /\n (s2_s1_2);\n dyp_dyd =\n ((c.y * ds2_dyd - s1) * (s2 - s1) - (c.y * s2 - d.y * s1) * (ds2_dyd)) /\n (s2_s1_2);\n\n p.x = (c.x * s2 - d.x * s1) / (s2 - s1);\n p.y = (c.y * s2 - d.y * s1) / (s2 - s1);\n if (i == n - 1) {\n cut_grad[4 * n * m + 4 * i] = dxp_dxc; // + dyp_dxc;\n cut_grad[4 * n * m + 4 * i + 1] = dyp_dxc;\n cut_grad[4 * n * m + 4 * i + 2] = dxp_dyc; // + dyp_dyc;\n cut_grad[4 * n * m + 4 * i + 3] = dyp_dyc;\n cut_grad[4 * n * m + 0] = dxp_dxd; // + dyp_dxd;\n cut_grad[4 * n * m + 1] = dyp_dxd;\n cut_grad[4 * n * m + 2] = dxp_dyd; // + dyp_dyd;\n cut_grad[4 * n * m + 3] = dyp_dyd;\n } else {\n cut_grad[4 * n * m + 4 * i] = dxp_dxc; // + dyp_dxc;\n cut_grad[4 * n * m + 4 * i + 1] = dyp_dxc;\n cut_grad[4 * n * m + 4 * i + 2] = dxp_dyc; // + dyp_dyc;\n cut_grad[4 * n * m + 4 * i + 3] = dyp_dyc;\n cut_grad[4 * n * m + 4 * (i + 1)] = dxp_dxd; // + dyp_dxd;\n cut_grad[4 * n * m + 4 * (i + 1) + 1] = dyp_dxd;\n cut_grad[4 * n * m + 4 * (i + 1) + 2] = dxp_dyd; // + dyp_dyd;\n cut_grad[4 * n * m + 4 * (i + 1) + 3] = dyp_dyd;\n }\n\n return 1;\n}\n__device__ inline void polygon_cut(Point* p, int& n, Point a, Point b,\n double* cut_grad) {\n Point pp[MAXN];\n double ccur_grad[MAXN] = {};\n int m = 0;\n p[n] = p[0];\n int k = n;\n for (int i = 0; i < n; i++) {\n if (sig(cross(a, b, p[i])) > 0) {\n pp[m] = p[i];\n ccur_grad[4 * n * m + 4 * i] = 1.0;\n ccur_grad[4 * n * m + 4 * i + 3] = 1.0;\n m++;\n }\n if (sig(cross(a, b, p[i])) != sig(cross(a, b, p[i + 1]))) {\n lineCross(a, b, p[i], p[i + 1], pp[m], ccur_grad, m, n, i);\n m++;\n }\n }\n\n n = 0;\n for (int i = 0; i < m; i++) {\n if (!i || !(point_same(pp[i], pp[i - 1]))) {\n p[n] = pp[i];\n for (int j = 0; j < 4 * k; j++) {\n cut_grad[4 * k * n + j] = ccur_grad[4 * k * i + j];\n }\n n++;\n }\n }\n\n while (n > 1 && point_same(p[n - 1], p[0])) n--;\n}\n\n__device__ inline double intersectArea(Point a, Point b, Point c, Point d,\n double* grad_AB, int order,\n int convex_n) {\n Point o(0, 0);\n int res_flag = 0;\n int s1 = sig(cross(o, a, b));\n int s2 = sig(cross(o, c, d));\n if (s1 == 0 || s2 == 0) return 0.0;\n if (s1 == -1) {\n Point* i = &a;\n Point* j = &b;\n swap1(i, j);\n res_flag = 1;\n }\n if (s2 == -1) {\n Point* i = &c;\n Point* j = &d;\n swap1(i, j);\n }\n Point p[10] = {o, a, b};\n int n = 3, n0 = 3, n1, n2, n3;\n double cut_grad1[MAXN] = {};\n double cut_grad2[MAXN] = {};\n double cut_grad3[MAXN] = {};\n double p1_p_grad[10][10] = {};\n double p2_p1_grad[10][10] = {};\n double p3_p2_grad[10][10] = {};\n\n double p3_p1_grad[10][10] = {};\n double p3_p_grad[10][10] = {};\n\n // 1\n polygon_cut(p, n, o, c, cut_grad1);\n n1 = n;\n for (int i = 0; i < n; i++) {\n for (int j = 0; j < 4 * n0; j++) {\n if (!(j % 2)) {\n p1_p_grad[2 * i][j / 2] = cut_grad1[4 * n0 * i + j];\n } else {\n p1_p_grad[2 * i + 1][j / 2] = cut_grad1[4 * n0 * i + j];\n }\n }\n }\n\n // 2\n polygon_cut(p, n, c, d, cut_grad2);\n n2 = n;\n for (int i = 0; i < n; i++) {\n for (int j = 0; j < 4 * n1; j++) {\n if (!(j % 2)) {\n p2_p1_grad[2 * i][j / 2] = cut_grad2[4 * n1 * i + j];\n } else {\n p2_p1_grad[2 * i + 1][j / 2] = cut_grad2[4 * n1 * i + j];\n }\n }\n }\n // 3\n polygon_cut(p, n, d, o, cut_grad3);\n n3 = n;\n for (int i = 0; i < n; i++) {\n for (int j = 0; j < 4 * n2; j++) {\n if (!(j % 2)) {\n p3_p2_grad[2 * i][j / 2] = cut_grad3[4 * n2 * i + j];\n } else {\n p3_p2_grad[2 * i + 1][j / 2] = cut_grad3[4 * n2 * i + j];\n }\n }\n }\n\n // mul\n // p3_p2(n3 * n2) * p2_p1(n2 * n1) = p3_p1 (n3 * n1)\n for (int i = 0; i < 2 * n3; i++) {\n for (int j = 0; j < 2 * n1; j++) {\n double sum = 0.0;\n for (int m = 0; m < 2 * n2; m++) {\n sum = sum + p3_p2_grad[i][m] * p2_p1_grad[m][j];\n }\n p3_p1_grad[i][j] = sum;\n }\n }\n\n // p3_p1 (n3 * n1) * p1_p (n1 * n0) = p3_p (n3 * n0)\n for (int i = 0; i < 2 * n3; i++) {\n for (int j = 0; j < 2 * n0; j++) {\n double sum = 0.0;\n for (int m = 0; m < 2 * n1; m++) {\n sum = sum + p3_p1_grad[i][m] * p1_p_grad[m][j];\n }\n p3_p_grad[i][j] = sum;\n }\n }\n\n // calculate S_grad\n int polygon_index_box_index[20];\n double grad_polygon[20];\n double S_grad[6];\n\n for (int i = 0; i < n3; i++) {\n polygon_index_box_index[i] = i;\n polygon_index_box_index[i + n3] = i;\n }\n\n double res =\n polygon_area_grad(p, n3, polygon_index_box_index, n3, grad_polygon);\n\n if (s1 * s2 == -1) {\n for (int j = 0; j < 2 * 3; j++) {\n double sum = 0.0;\n for (int m = 0; m < 2 * n3; m++) {\n sum = sum - grad_polygon[m] * p3_p_grad[m][j];\n }\n S_grad[j] = sum;\n }\n\n if (order != convex_n - 1) {\n if (res_flag) {\n grad_AB[2 * order] += S_grad[4];\n grad_AB[2 * order + 1] += S_grad[5];\n grad_AB[2 * order + 2] += S_grad[2];\n grad_AB[2 * order + 3] += S_grad[3];\n\n } else {\n grad_AB[2 * order] += S_grad[2];\n grad_AB[2 * order + 1] += S_grad[3];\n grad_AB[2 * order + 2] += S_grad[4];\n grad_AB[2 * order + 3] += S_grad[5];\n }\n } else {\n if (res_flag) {\n grad_AB[2 * order] += S_grad[4];\n grad_AB[2 * order + 1] += S_grad[5];\n grad_AB[0] += S_grad[2];\n grad_AB[1] += S_grad[3];\n\n } else {\n grad_AB[2 * order] += S_grad[2];\n grad_AB[2 * order + 1] += S_grad[3];\n grad_AB[0] += S_grad[4];\n grad_AB[1] += S_grad[5];\n }\n }\n res = -res;\n } else {\n for (int j = 0; j < 2 * 3; j++) {\n double sum = 0.0;\n for (int m = 0; m < 2 * n3; m++) {\n sum = sum + grad_polygon[m] * p3_p_grad[m][j];\n }\n S_grad[j] = sum;\n }\n\n if (order != convex_n - 1) {\n if (res_flag) {\n grad_AB[2 * order] += S_grad[4];\n grad_AB[2 * order + 1] += S_grad[5];\n grad_AB[2 * order + 2] += S_grad[2];\n grad_AB[2 * order + 3] += S_grad[3];\n } else {\n grad_AB[2 * order] += S_grad[2];\n grad_AB[2 * order + 1] += S_grad[3];\n grad_AB[2 * order + 2] += S_grad[4];\n grad_AB[2 * order + 3] += S_grad[5];\n }\n } else {\n if (res_flag) {\n grad_AB[2 * order] += S_grad[4];\n grad_AB[2 * order + 1] += S_grad[5];\n grad_AB[0] += S_grad[2];\n grad_AB[1] += S_grad[3];\n } else {\n grad_AB[2 * order] += S_grad[2];\n grad_AB[2 * order + 1] += S_grad[3];\n grad_AB[0] += S_grad[4];\n grad_AB[1] += S_grad[5];\n }\n }\n }\n return res;\n}\n\n__device__ inline double intersectAreaO(Point* ps1, int n1, Point* ps2, int n2,\n double* grad_AB) {\n if (area(ps1, n1) < 0) reverse1(ps1, n1);\n if (area(ps2, n2) < 0) reverse1(ps2, n2);\n ps1[n1] = ps1[0];\n ps2[n2] = ps2[0];\n double res = 0;\n for (int i = 0; i < n1; i++) {\n for (int j = 0; j < n2; j++) {\n res +=\n intersectArea(ps1[i], ps1[i + 1], ps2[j], ps2[j + 1], grad_AB, i, n1);\n }\n }\n return res;\n}\n\n__device__ inline void Jarvis(Point* in_poly, int& n_poly) {\n Point p_max, p_k;\n int max_index, k_index;\n int Stack[NMAX] = {}, top1, top2;\n double sign;\n Point right_point[10], left_point[10];\n\n for (int i = 0; i < n_poly; i++) {\n if (in_poly[i].y < in_poly[0].y ||\n in_poly[i].y == in_poly[0].y && in_poly[i].x < in_poly[0].x) {\n Point* j = &(in_poly[0]);\n Point* k = &(in_poly[i]);\n swap1(j, k);\n }\n if (i == 0) {\n p_max = in_poly[0];\n max_index = 0;\n }\n if (in_poly[i].y > p_max.y ||\n in_poly[i].y == p_max.y && in_poly[i].x > p_max.x) {\n p_max = in_poly[i];\n max_index = i;\n }\n }\n\n if (max_index == 0) {\n max_index = 1;\n p_max = in_poly[max_index];\n }\n\n k_index = 0, Stack[0] = 0, top1 = 0;\n while (k_index != max_index) {\n p_k = p_max;\n k_index = max_index;\n for (int i = 1; i < n_poly; i++) {\n sign = cross(in_poly[Stack[top1]], in_poly[i], p_k);\n if ((sign > 0) || ((sign == 0) && (dis(in_poly[Stack[top1]], in_poly[i]) >\n dis(in_poly[Stack[top1]], p_k)))) {\n p_k = in_poly[i];\n k_index = i;\n }\n }\n top1++;\n Stack[top1] = k_index;\n }\n for (int i = 0; i <= top1; i++) right_point[i] = in_poly[Stack[i]];\n\n k_index = 0, Stack[0] = 0, top2 = 0;\n\n while (k_index != max_index) {\n p_k = p_max;\n k_index = max_index;\n for (int i = 1; i < n_poly; i++) {\n sign = cross(in_poly[Stack[top2]], in_poly[i], p_k);\n if ((sign < 0) || (sign == 0) && (dis(in_poly[Stack[top2]], in_poly[i]) >\n dis(in_poly[Stack[top2]], p_k))) {\n p_k = in_poly[i];\n k_index = i;\n }\n }\n top2++;\n Stack[top2] = k_index;\n }\n for (int i = top2 - 1; i >= 0; i--) left_point[i] = in_poly[Stack[i]];\n\n for (int i = 0; i < top1 + top2; i++) {\n if (i <= top1) {\n in_poly[i] = right_point[i];\n } else {\n in_poly[i] = left_point[top2 - (i - top1)];\n }\n }\n n_poly = top1 + top2;\n}\n\n__device__ inline double intersectAreaPoly(Point* ps1, int n1, Point* ps2,\n int n2, double* grad_C) {\n Point polygon[MAXN];\n int n = n1 + n2, n_poly = 0;\n for (int i = 0; i < n1; i++) {\n for (int j = 0; j < n - n1; j++) {\n if (point_same(ps1[i], ps2[j])) {\n for (int k = j; k < n - n1 - 1; k++) {\n ps2[k] = ps2[k + 1];\n }\n n2--;\n break;\n }\n }\n }\n n_poly = n1 + n2;\n for (int i = 0; i < n_poly; i++) {\n if (i < n1) {\n polygon[i] = ps1[i];\n } else {\n polygon[i] = ps2[i - n1];\n }\n }\n\n Jarvis(polygon, n_poly);\n\n int polygon_to_pred_index[18] = {-1, -1, -1, -1, -1, -1, -1, -1, -1,\n -1, -1, -1, -1, -1, -1, -1, -1, -1};\n int n_pred = 0;\n for (int i = 0; i < n_poly; i++) {\n for (int j = 0; j < n1; j++) {\n if (polygon[i].x == ps1[j].x && polygon[i].y == ps1[j].y) {\n polygon_to_pred_index[n_pred] = i;\n polygon_to_pred_index[n_pred + n1] = j;\n n_pred += 1;\n break;\n }\n }\n }\n if (n_pred == 0) {\n double polygon_area = fabs(area(polygon, n_poly));\n for (int i = 0; i < 18; i++) {\n grad_C[i] = 0.0;\n }\n return polygon_area;\n } else {\n double polygon_area =\n polygon_area_grad(polygon, n_poly, polygon_to_pred_index, n1, grad_C);\n if (polygon_area < 0) {\n for (int i = 0; i < 18; i++) {\n grad_C[i] = -grad_C[i];\n }\n }\n return fabs(polygon_area);\n }\n}\n\n// convex_find and get the polygon_index_box_index\n__device__ inline void Jarvis_and_index(Point* in_poly, int& n_poly,\n int* points_to_convex_ind) {\n int n_input = n_poly;\n Point input_poly[20];\n for (int i = 0; i < n_input; i++) {\n input_poly[i].x = in_poly[i].x;\n input_poly[i].y = in_poly[i].y;\n }\n Point p_max, p_k;\n int max_index, k_index;\n int Stack[20], top1, top2;\n double sign;\n Point right_point[10], left_point[10];\n\n for (int i = 0; i < n_poly; i++) {\n if (in_poly[i].y < in_poly[0].y ||\n in_poly[i].y == in_poly[0].y && in_poly[i].x < in_poly[0].x) {\n Point* j = &(in_poly[0]);\n Point* k = &(in_poly[i]);\n swap1(j, k);\n }\n if (i == 0) {\n p_max = in_poly[0];\n max_index = 0;\n }\n if (in_poly[i].y > p_max.y ||\n in_poly[i].y == p_max.y && in_poly[i].x > p_max.x) {\n p_max = in_poly[i];\n max_index = i;\n }\n }\n if (max_index == 0) {\n max_index = 1;\n p_max = in_poly[max_index];\n }\n\n k_index = 0, Stack[0] = 0, top1 = 0;\n while (k_index != max_index) {\n p_k = p_max;\n k_index = max_index;\n for (int i = 1; i < n_poly; i++) {\n sign = cross(in_poly[Stack[top1]], in_poly[i], p_k);\n if ((sign > 0) || ((sign == 0) && (dis(in_poly[Stack[top1]], in_poly[i]) >\n dis(in_poly[Stack[top1]], p_k)))) {\n p_k = in_poly[i];\n k_index = i;\n }\n }\n top1++;\n Stack[top1] = k_index;\n }\n for (int i = 0; i <= top1; i++) {\n right_point[i] = in_poly[Stack[i]];\n }\n\n k_index = 0, Stack[0] = 0, top2 = 0;\n\n while (k_index != max_index) {\n p_k = p_max;\n k_index = max_index;\n for (int i = 1; i < n_poly; i++) {\n sign = cross(in_poly[Stack[top2]], in_poly[i], p_k);\n if ((sign < 0) || (sign == 0) && (dis(in_poly[Stack[top2]], in_poly[i]) >\n dis(in_poly[Stack[top2]], p_k))) {\n p_k = in_poly[i];\n k_index = i;\n }\n }\n top2++;\n Stack[top2] = k_index;\n }\n\n for (int i = top2 - 1; i >= 0; i--) {\n left_point[i] = in_poly[Stack[i]];\n }\n\n for (int i = 0; i < top1 + top2; i++) {\n if (i <= top1) {\n in_poly[i] = right_point[i];\n } else {\n in_poly[i] = left_point[top2 - (i - top1)];\n }\n }\n n_poly = top1 + top2;\n for (int i = 0; i < n_poly; i++) {\n for (int j = 0; j < n_input; j++) {\n if (point_same(in_poly[i], input_poly[j])) {\n points_to_convex_ind[i] = j;\n break;\n }\n }\n }\n}\n\ntemplate \n__device__ inline float devrIoU(T const* const p, T const* const q,\n T* point_grad, const int idx) {\n Point ps1[MAXN], ps2[MAXN];\n\n Point convex[MAXN];\n for (int i = 0; i < 9; i++) {\n convex[i].x = (double)p[i * 2];\n convex[i].y = (double)p[i * 2 + 1];\n }\n int n_convex = 9;\n int points_to_convex_ind[9] = {-1, -1, -1, -1, -1, -1, -1, -1, -1};\n Jarvis_and_index(convex, n_convex, points_to_convex_ind);\n\n int n1 = n_convex;\n int n2 = 4;\n\n for (int i = 0; i < n1; i++) {\n ps1[i].x = (double)convex[i].x;\n ps1[i].y = (double)convex[i].y;\n }\n\n for (int i = 0; i < n2; i++) {\n ps2[i].x = (double)q[i * 2];\n ps2[i].y = (double)q[i * 2 + 1];\n }\n\n int polygon_index_box_index[18];\n for (int i = 0; i < n1; i++) {\n polygon_index_box_index[i] = i;\n polygon_index_box_index[i + n1] = i;\n }\n\n double grad_A[18] = {};\n double grad_AB[18] = {};\n double grad_C[18] = {};\n\n double inter_area = intersectAreaO(ps1, n1, ps2, n2, grad_AB);\n double S_pred =\n polygon_area_grad(ps1, n1, polygon_index_box_index, n1, grad_A);\n if (S_pred < 0) {\n for (int i = 0; i < n_convex * 2; i++) {\n grad_A[i] = -grad_A[i];\n }\n }\n double union_area = fabs(S_pred) + fabs(area(ps2, n2)) - inter_area;\n\n double iou = inter_area / union_area;\n double polygon_area = intersectAreaPoly(ps1, n1, ps2, n2, grad_C);\n\n // printf(\"%d:live\\n\", idx);\n double rot_giou = iou - (polygon_area - union_area) / polygon_area;\n\n float grad_point_temp[18] = {};\n\n for (int i = 0; i < n_convex; i++) {\n int grad_point = points_to_convex_ind[i];\n grad_point_temp[2 * grad_point] =\n (float)((union_area + inter_area) / (union_area * union_area) *\n grad_AB[2 * i] -\n iou / union_area * grad_A[2 * i] -\n 1 / polygon_area * (grad_AB[2 * i] - grad_A[2 * i]) -\n (union_area) / polygon_area / polygon_area * grad_C[2 * i]);\n grad_point_temp[2 * grad_point + 1] =\n (float)((union_area + inter_area) / (union_area * union_area) *\n grad_AB[2 * i + 1] -\n iou / union_area * grad_A[2 * i + 1] -\n 1 / polygon_area * (grad_AB[2 * i + 1] - grad_A[2 * i + 1]) -\n (union_area) / polygon_area / polygon_area * grad_C[2 * i + 1]);\n }\n\n for (int i = 0; i < 9; i++) {\n point_grad[2 * i] = grad_point_temp[2 * i];\n point_grad[2 * i + 1] = grad_point_temp[2 * i + 1];\n }\n return (float)rot_giou;\n}\n\ntemplate \n__global__ void convex_giou_cuda_kernel(const int ex_n_boxes,\n const int gt_n_boxes, const T* ex_boxes,\n const T* gt_boxes, T* point_grad) {\n CUDA_1D_KERNEL_LOOP(index, ex_n_boxes) {\n const T* cur_box = ex_boxes + index * 18;\n const T* cur_gt_box = gt_boxes + index * 8;\n T* cur_grad = point_grad + index * 19;\n T giou = devrIoU(cur_box, cur_gt_box, cur_grad, threadIdx.x);\n cur_grad[18] = giou;\n }\n}\n\n__device__ inline int lineCross(Point a, Point b, Point c, Point d, Point& p) {\n double s1, s2;\n s1 = cross(a, b, c);\n s2 = cross(a, b, d);\n if (sig(s1) == 0 && sig(s2) == 0) return 2;\n if (sig(s2 - s1) == 0) return 0;\n p.x = (c.x * s2 - d.x * s1) / (s2 - s1);\n p.y = (c.y * s2 - d.y * s1) / (s2 - s1);\n return 1;\n}\n\n__device__ inline void polygon_cut(Point* p, int& n, Point a, Point b) {\n Point pp[MAXN];\n int m = 0;\n p[n] = p[0];\n for (int i = 0; i < n; i++) {\n if (sig(cross(a, b, p[i])) > 0) {\n pp[m] = p[i];\n m++;\n }\n if (sig(cross(a, b, p[i])) != sig(cross(a, b, p[i + 1]))) {\n lineCross(a, b, p[i], p[i + 1], pp[m]);\n m++;\n }\n }\n n = 0;\n for (int i = 0; i < m; i++) {\n if (!i || !(point_same(pp[i], pp[i - 1]))) {\n p[n] = pp[i];\n n++;\n }\n }\n\n while (n > 1 && point_same(p[n - 1], p[0])) n--;\n}\n\n__device__ inline double intersectArea(Point a, Point b, Point c, Point d) {\n Point o(0, 0);\n int s1 = sig(cross(o, a, b));\n int s2 = sig(cross(o, c, d));\n if (s1 == 0 || s2 == 0) return 0.0;\n if (s1 == -1) {\n Point* i = &a;\n Point* j = &b;\n swap1(i, j);\n }\n if (s2 == -1) {\n Point* i = &c;\n Point* j = &d;\n swap1(i, j);\n }\n Point p[10] = {o, a, b};\n int n = 3;\n\n polygon_cut(p, n, o, c);\n polygon_cut(p, n, c, d);\n polygon_cut(p, n, d, o);\n double res = area(p, n);\n if (s1 * s2 == -1) res = -res;\n return res;\n}\n__device__ inline double intersectAreaO(Point* ps1, int n1, Point* ps2,\n int n2) {\n if (area(ps1, n1) < 0) reverse1(ps1, n1);\n if (area(ps2, n2) < 0) reverse1(ps2, n2);\n ps1[n1] = ps1[0];\n ps2[n2] = ps2[0];\n double res = 0;\n for (int i = 0; i < n1; i++) {\n for (int j = 0; j < n2; j++) {\n res += intersectArea(ps1[i], ps1[i + 1], ps2[j], ps2[j + 1]);\n }\n }\n return res;\n}\n\ntemplate \n__device__ inline float devrIoU(T const* const p, T const* const q) {\n Point ps1[MAXN], ps2[MAXN];\n Point convex[MAXN];\n for (int i = 0; i < 9; i++) {\n convex[i].x = (double)p[i * 2];\n convex[i].y = (double)p[i * 2 + 1];\n }\n int n_convex = 9;\n int points_to_convex_ind[9] = {-1, -1, -1, -1, -1, -1, -1, -1, -1};\n Jarvis_and_index(convex, n_convex, points_to_convex_ind);\n int n1 = n_convex;\n for (int i = 0; i < n1; i++) {\n ps1[i].x = (double)convex[i].x;\n ps1[i].y = (double)convex[i].y;\n }\n int n2 = 4;\n for (int i = 0; i < n2; i++) {\n ps2[i].x = (double)q[i * 2];\n ps2[i].y = (double)q[i * 2 + 1];\n }\n double inter_area = intersectAreaO(ps1, n1, ps2, n2);\n double S_pred = area(ps1, n1);\n double union_area = fabs(S_pred) + fabs(area(ps2, n2)) - inter_area;\n double iou = inter_area / union_area;\n return (float)iou;\n}\n\ntemplate \n__global__ void convex_iou_cuda_kernel(const int ex_n_boxes,\n const int gt_n_boxes, const T* ex_boxes,\n const T* gt_boxes, T* iou) {\n CUDA_1D_KERNEL_LOOP(index, ex_n_boxes) {\n const T* cur_box = ex_boxes + index * 18;\n for (int i = 0; i < gt_n_boxes; i++) {\n iou[index * gt_n_boxes + i] = devrIoU(cur_box, gt_boxes + i * 8);\n }\n }\n}\n#endif // CONVEX_IOU_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/correlation_cuda.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/ClementPinard/Pytorch-Correlation-extension/blob/master/Correlation_Module/correlation_cuda_kernel.cu\n// Original licence: Under MIT License\n\n#ifndef CORRELATION_CUDA\n#define CORRELATION_CUDA\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n#include \n#include \n// Using is recommended in the official documentation in\n// https://pytorch.org/tutorials/advanced/cpp_extension.html#writing-the-c-op.\n// However, we use for compatibility with CUDA 9.0\n// Read https://github.com/pytorch/extension-cpp/issues/35 for more details.\n#include \n\n#include \n#include \n\nusing namespace torch;\n\n#define TensorAcc4R PackedTensorAccessor32\n#define TensorAcc5R PackedTensorAccessor32\n#define WITHIN_BOUNDS(x, y, H, W) (x >= 0 && x < H && y >= 0 && y < W)\n\n#define THREADS_FORWARD 32\n#define THREADS_BACKWARD 16\n\ntemplate \n__global__ void correlation_forward_cuda_kernel(\n const TensorAcc4R rInput1, const TensorAcc4R rInput2, TensorAcc5R output,\n int kH, int kW, int patchH, int patchW, int padH, int padW, int dilationH,\n int dilationW, int dilation_patchH, int dilation_patchW, int dH, int dW) {\n const int iH = rInput1.size(1);\n const int iW = rInput1.size(2);\n const int C = rInput1.size(3);\n\n const int n = blockIdx.x;\n const int h = blockIdx.y;\n const int w = blockIdx.z;\n const int thread = threadIdx.x;\n\n const int start_i = -padH + h * dH;\n const int start_j = -padW + w * dW;\n\n const int patchRadH = dilation_patchH * (patchH - 1) / 2;\n const int patchRadW = dilation_patchW * (patchW - 1) / 2;\n\n __shared__ scalar_t prod_sum[THREADS_FORWARD];\n\n for (int ph = 0; ph < patchH; ++ph) {\n int ph_dilated = ph * dilation_patchH - patchRadH;\n for (int pw = 0; pw < patchW; ++pw) {\n int pw_dilated = pw * dilation_patchW - patchRadW;\n prod_sum[thread] = 0;\n for (int i = 0; i < kH; ++i) {\n int i1 = start_i + i * dilationH;\n int i2 = i1 + ph_dilated;\n if\n WITHIN_BOUNDS(i1, i2, iH, iH) {\n for (int j = 0; j < kW; ++j) {\n int j1 = start_j + j * dilationW;\n int j2 = j1 + pw_dilated;\n if\n WITHIN_BOUNDS(j1, j2, iW, iW) {\n for (int c = thread; c < C; c += THREADS_FORWARD) {\n scalar_t v1 = rInput1[n][i1][j1][c];\n scalar_t v2 = rInput2[n][i2][j2][c];\n prod_sum[thread] += v1 * v2;\n }\n }\n }\n }\n }\n // accumulate\n __syncthreads();\n if (thread == 0) {\n scalar_t reduce_sum = 0;\n for (int index = 0; index < THREADS_FORWARD; ++index) {\n reduce_sum += prod_sum[index];\n }\n output[n][ph][pw][h][w] = reduce_sum;\n }\n }\n }\n}\n\ntemplate \n__global__ void correlation_backward_cuda_kernel_input1(\n const TensorAcc5R grad_output, const TensorAcc4R input2,\n TensorAcc4R grad_input1, const int kH, const int kW, const int patchH,\n const int patchW, const int padH, const int padW, const int dilationH,\n const int dilationW, const int dilation_patchH, const int dilation_patchW,\n const int dH, const int dW, const int batch) {\n const int iH = input2.size(2);\n const int iW = input2.size(3);\n\n const int H = grad_output.size(3);\n const int W = grad_output.size(4);\n\n const int patchRadH = (patchH - 1) / 2;\n const int patchRadW = (patchW - 1) / 2;\n\n const int n = batch;\n const int c = blockIdx.x;\n const int h = blockIdx.y;\n const int w = blockIdx.z;\n const int ph_off = threadIdx.x;\n const int pw_off = threadIdx.y;\n\n const int h_2 = h + padH;\n const int w_2 = w + padW;\n const int min_h = h_2 - kH * dilationH;\n const int min_w = w_2 - kW * dilationW;\n\n __shared__ scalar_t prod_sum[THREADS_BACKWARD][THREADS_BACKWARD];\n prod_sum[ph_off][pw_off] = 0;\n\n for (int ph = ph_off; ph < patchH; ph += THREADS_BACKWARD) {\n int i1 = h + dilation_patchH * (ph - patchRadH);\n for (int pw = pw_off; pw < patchW; pw += THREADS_BACKWARD) {\n int j1 = w + dilation_patchW * (pw - patchRadW);\n if (WITHIN_BOUNDS(i1, j1, iH, iW)) {\n scalar_t val = input2[n][c][i1][j1];\n for (int h_3 = h_2; h_3 > min_h; h_3 -= dilationH) {\n int i2 = (h_3) / dH;\n if (i2 * dH != h_3) continue;\n for (int w_3 = w_2; w_3 > min_w; w_3 -= dilationW) {\n int j2 = (w_3) / dW;\n if (j2 * dW != w_3) continue;\n if\n WITHIN_BOUNDS(i2, j2, H, W) {\n prod_sum[ph_off][pw_off] +=\n grad_output[n][ph][pw][i2][j2] * val;\n }\n }\n }\n }\n }\n }\n\n __syncthreads();\n\n if (ph_off == 0 && pw_off == 0) {\n scalar_t reduce_sum = 0;\n for (int ph = 0; ph < THREADS_BACKWARD; ++ph) {\n for (int pw = 0; pw < THREADS_BACKWARD; ++pw) {\n reduce_sum += prod_sum[ph][pw];\n }\n }\n grad_input1[n][c][h][w] = reduce_sum;\n }\n}\n\ntemplate \n__global__ void correlation_backward_cuda_kernel_input2(\n const TensorAcc5R grad_output, const TensorAcc4R input1,\n TensorAcc4R grad_input2, int kH, int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW, int batch) {\n const int iH = input1.size(2);\n const int iW = input1.size(3);\n\n const int patchRadH = (patchH - 1) / 2;\n const int patchRadW = (patchW - 1) / 2;\n\n const int H = grad_output.size(3);\n const int W = grad_output.size(4);\n\n const int dilatedKH = kH * dilationH;\n const int dilatedKW = kW * dilationW;\n\n const int n = batch;\n const int c = blockIdx.x;\n const int h = blockIdx.y;\n const int w = blockIdx.z;\n const int ph_off = threadIdx.x;\n const int pw_off = threadIdx.y;\n\n __shared__ scalar_t prod_sum[THREADS_BACKWARD][THREADS_BACKWARD];\n prod_sum[ph_off][pw_off] = 0;\n\n for (int ph = ph_off; ph < patchH; ph += THREADS_BACKWARD) {\n int i1 = h - dilation_patchH * (ph - patchRadH);\n for (int pw = pw_off; pw < patchW; pw += THREADS_BACKWARD) {\n int j1 = w - dilation_patchW * (pw - patchRadW);\n if\n WITHIN_BOUNDS(i1, j1, iH, iW) {\n scalar_t val = input1[n][c][i1][j1];\n\n const int h_2 = i1 + padH;\n const int w_2 = j1 + padW;\n const int min_h = h_2 - dilatedKH;\n const int min_w = w_2 - dilatedKW;\n\n for (int h_3 = h_2; h_3 > min_h; h_3 -= dilationH) {\n int i2 = (h_3) / dH;\n if (i2 * dH != h_3) continue;\n for (int w_3 = w_2; w_3 > min_w; w_3 -= dilationW) {\n int j2 = (w_3) / dW;\n if (j2 * dW != w_3) continue;\n if\n WITHIN_BOUNDS(i2, j2, H, W) {\n prod_sum[ph_off][pw_off] +=\n grad_output[n][ph][pw][i2][j2] * val;\n }\n }\n }\n }\n }\n }\n\n __syncthreads();\n\n if (ph_off == 0 && pw_off == 0) {\n scalar_t reduce_sum = 0;\n for (int ph = 0; ph < THREADS_BACKWARD; ++ph) {\n for (int pw = 0; pw < THREADS_BACKWARD; ++pw) {\n reduce_sum += prod_sum[ph][pw];\n }\n }\n grad_input2[n][c][h][w] = reduce_sum;\n }\n}\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/deform_conv_cuda_kernel.cuh", "content": "/*!\n ******************* BEGIN Caffe Copyright Notice and Disclaimer\n *****************\n *\n * COPYRIGHT\n *\n * All contributions by the University of California:\n * Copyright (c) 2014-2017 The Regents of the University of California (Regents)\n * All rights reserved.\n *\n * All other contributions:\n * Copyright (c) 2014-2017, the respective contributors\n * All rights reserved.\n *\n * Caffe uses a shared copyright model: each contributor holds copyright over\n * their contributions to Caffe. The project versioning records all such\n * contribution and copyright details. If a contributor wants to further mark\n * their specific copyright on a particular contribution, they should indicate\n * their copyright solely in the commit message of the change when it is\n * committed.\n *\n * LICENSE\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions are met:\n *\n * 1. Redistributions of source code must retain the above copyright notice,\n *this list of conditions and the following disclaimer.\n * 2. Redistributions in binary form must reproduce the above copyright notice,\n * this list of conditions and the following disclaimer in the documentation\n * and/or other materials provided with the distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\n *AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n *IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\n * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE\n *FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n *DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n *SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n *CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\n *OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n *OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n *\n * CONTRIBUTION AGREEMENT\n *\n * By contributing to the BVLC/caffe repository through pull-request, comment,\n * or otherwise, the contributor releases their content to the\n * license and copyright terms herein.\n *\n ***************** END Caffe Copyright Notice and Disclaimer\n *********************\n *\n * Copyright (c) 2018 Microsoft\n * Licensed under The MIT License [see LICENSE for details]\n * \\file modulated_deformable_im2col.cuh\n * \\brief Function definitions of converting an image to\n * column matrix based on kernel, padding, dilation, and offset.\n * These functions are mainly used in deformable convolution operators.\n * \\ref: https://arxiv.org/abs/1703.06211\n * \\author Yuwen Xiong, Haozhi Qi, Jifeng Dai, Xizhou Zhu, Han Hu, Dazhi Cheng\n */\n\n// modified from\n// https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/blob/mmdetection/mmdet/ops/dcn/src/deform_conv_cuda_kernel.cu\n\n#ifndef DEFORM_CONV_CUDA_KERNEL_CUH\n#define DEFORM_CONV_CUDA_KERNEL_CUH\n\n#include \n#ifdef MMCV_WITH_TRT\n#include \"common_cuda_helper.hpp\"\n#else // MMCV_WITH_TRT\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else // MMCV_USE_PARROTS\n#include \"pytorch_cuda_helper.hpp\"\n#endif // MMCV_USE_PARROTS\n#endif // MMCV_WITH_TRT\n\ntemplate \n__device__ T deformable_im2col_bilinear(const T *input, const int data_width,\n const int height, const int width, T h,\n T w) {\n if (h <= -1 || height <= h || w <= -1 || width <= w) {\n return 0;\n }\n\n int h_low = floorf(h);\n int w_low = floorf(w);\n int h_high = h_low + 1;\n int w_high = w_low + 1;\n\n T lh = h - h_low;\n T lw = w - w_low;\n T hh = 1 - lh, hw = 1 - lw;\n\n T v1 = 0;\n if (h_low >= 0 && w_low >= 0) v1 = input[h_low * data_width + w_low];\n T v2 = 0;\n if (h_low >= 0 && w_high <= width - 1)\n v2 = input[h_low * data_width + w_high];\n T v3 = 0;\n if (h_high <= height - 1 && w_low >= 0)\n v3 = input[h_high * data_width + w_low];\n T v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1)\n v4 = input[h_high * data_width + w_high];\n\n T w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n\n T val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n return val;\n}\n\ntemplate \n__device__ T get_gradient_weight(T argmax_h, T argmax_w, const int h,\n const int w, const int height,\n const int width) {\n if (argmax_h <= -1 || argmax_h >= height || argmax_w <= -1 ||\n argmax_w >= width) {\n // empty\n return 0;\n }\n\n int argmax_h_low = floorf(argmax_h);\n int argmax_w_low = floorf(argmax_w);\n int argmax_h_high = argmax_h_low + 1;\n int argmax_w_high = argmax_w_low + 1;\n\n T weight = 0;\n if (h == argmax_h_low && w == argmax_w_low)\n weight = (h + 1 - argmax_h) * (w + 1 - argmax_w);\n if (h == argmax_h_low && w == argmax_w_high)\n weight = (h + 1 - argmax_h) * (argmax_w + 1 - w);\n if (h == argmax_h_high && w == argmax_w_low)\n weight = (argmax_h + 1 - h) * (w + 1 - argmax_w);\n if (h == argmax_h_high && w == argmax_w_high)\n weight = (argmax_h + 1 - h) * (argmax_w + 1 - w);\n return weight;\n}\n\ntemplate \n__device__ T get_coordinate_weight(T argmax_h, T argmax_w, const int height,\n const int width, const T *im_data,\n const int data_width, const int bp_dir) {\n if (argmax_h <= -1 || argmax_h >= height || argmax_w <= -1 ||\n argmax_w >= width) {\n // empty\n return 0;\n }\n\n int argmax_h_low = floorf(argmax_h);\n int argmax_w_low = floorf(argmax_w);\n int argmax_h_high = argmax_h_low + 1;\n int argmax_w_high = argmax_w_low + 1;\n\n T weight = 0;\n\n if (bp_dir == 0) {\n if (argmax_h_low >= 0 && argmax_w_low >= 0)\n weight += -1 * (argmax_w_low + 1 - argmax_w) *\n im_data[argmax_h_low * data_width + argmax_w_low];\n if (argmax_h_low >= 0 && argmax_w_high <= width - 1)\n weight += -1 * (argmax_w - argmax_w_low) *\n im_data[argmax_h_low * data_width + argmax_w_high];\n if (argmax_h_high <= height - 1 && argmax_w_low >= 0)\n weight += (argmax_w_low + 1 - argmax_w) *\n im_data[argmax_h_high * data_width + argmax_w_low];\n if (argmax_h_high <= height - 1 && argmax_w_high <= width - 1)\n weight += (argmax_w - argmax_w_low) *\n im_data[argmax_h_high * data_width + argmax_w_high];\n } else if (bp_dir == 1) {\n if (argmax_h_low >= 0 && argmax_w_low >= 0)\n weight += -1 * (argmax_h_low + 1 - argmax_h) *\n im_data[argmax_h_low * data_width + argmax_w_low];\n if (argmax_h_low >= 0 && argmax_w_high <= width - 1)\n weight += (argmax_h_low + 1 - argmax_h) *\n im_data[argmax_h_low * data_width + argmax_w_high];\n if (argmax_h_high <= height - 1 && argmax_w_low >= 0)\n weight += -1 * (argmax_h - argmax_h_low) *\n im_data[argmax_h_high * data_width + argmax_w_low];\n if (argmax_h_high <= height - 1 && argmax_w_high <= width - 1)\n weight += (argmax_h - argmax_h_low) *\n im_data[argmax_h_high * data_width + argmax_w_high];\n }\n\n return weight;\n}\n\ntemplate \n__global__ void deformable_im2col_gpu_kernel(\n const int n, const T *data_im, const T *data_offset, const int height,\n const int width, const int kernel_h, const int kernel_w, const int pad_h,\n const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int num_channels, const int deformable_group, const int height_col,\n const int width_col, T *data_col) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n // index index of output matrix\n const int w_col = index % width_col;\n const int h_col = (index / width_col) % height_col;\n const int b_col = (index / width_col / height_col) % batch_size;\n const int c_im = (index / width_col / height_col) / batch_size;\n const int c_col = c_im * kernel_h * kernel_w;\n\n // compute deformable group index\n const int deformable_group_index = c_im / channel_per_deformable_group;\n\n const int h_in = h_col * stride_h - pad_h;\n const int w_in = w_col * stride_w - pad_w;\n T *data_col_ptr =\n data_col +\n ((c_col * batch_size + b_col) * height_col + h_col) * width_col + w_col;\n const T *data_im_ptr =\n data_im + (b_col * num_channels + c_im) * height * width;\n const T *data_offset_ptr =\n data_offset + (b_col * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n\n for (int i = 0; i < kernel_h; ++i) {\n for (int j = 0; j < kernel_w; ++j) {\n const int data_offset_h_ptr =\n ((2 * (i * kernel_w + j)) * height_col + h_col) * width_col + w_col;\n const int data_offset_w_ptr =\n ((2 * (i * kernel_w + j) + 1) * height_col + h_col) * width_col +\n w_col;\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n T val = static_cast(0);\n const T h_im = h_in + i * dilation_h + offset_h;\n const T w_im = w_in + j * dilation_w + offset_w;\n if (h_im > -1 && w_im > -1 && h_im < height && w_im < width)\n val = deformable_im2col_bilinear(data_im_ptr, width, height, width,\n h_im, w_im);\n *data_col_ptr = val;\n data_col_ptr += batch_size * height_col * width_col;\n }\n }\n }\n}\n\ntemplate \n__global__ void deformable_col2im_gpu_kernel(\n const int n, const T *data_col, const T *data_offset, const int channels,\n const int height, const int width, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int deformable_group, const int height_col, const int width_col,\n T *grad_im) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n const int j = (index / width_col / height_col / batch_size) % kernel_w;\n const int i =\n (index / width_col / height_col / batch_size / kernel_w) % kernel_h;\n const int c =\n index / width_col / height_col / batch_size / kernel_w / kernel_h;\n // compute the start and end of the output\n\n const int deformable_group_index = c / channel_per_deformable_group;\n\n int w_out = index % width_col;\n int h_out = (index / width_col) % height_col;\n int b = (index / width_col / height_col) % batch_size;\n int w_in = w_out * stride_w - pad_w;\n int h_in = h_out * stride_h - pad_h;\n\n const T *data_offset_ptr =\n data_offset + (b * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n const int data_offset_h_ptr =\n ((2 * (i * kernel_w + j)) * height_col + h_out) * width_col + w_out;\n const int data_offset_w_ptr =\n ((2 * (i * kernel_w + j) + 1) * height_col + h_out) * width_col + w_out;\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n const T cur_inv_h_data = h_in + i * dilation_h + offset_h;\n const T cur_inv_w_data = w_in + j * dilation_w + offset_w;\n\n const T cur_top_grad = data_col[index];\n const int cur_h = (int)cur_inv_h_data;\n const int cur_w = (int)cur_inv_w_data;\n for (int dy = -2; dy <= 2; dy++) {\n for (int dx = -2; dx <= 2; dx++) {\n if (cur_h + dy >= 0 && cur_h + dy < height && cur_w + dx >= 0 &&\n cur_w + dx < width && abs(cur_inv_h_data - (cur_h + dy)) < 1 &&\n abs(cur_inv_w_data - (cur_w + dx)) < 1) {\n int cur_bottom_grad_pos =\n ((b * channels + c) * height + cur_h + dy) * width + cur_w + dx;\n T weight = get_gradient_weight(cur_inv_h_data, cur_inv_w_data,\n cur_h + dy, cur_w + dx, height, width);\n atomicAdd(grad_im + cur_bottom_grad_pos, weight * cur_top_grad);\n }\n }\n }\n }\n}\n\ntemplate \n__global__ void deformable_col2im_coord_gpu_kernel(\n const int n, const T *data_col, const T *data_im, const T *data_offset,\n const int channels, const int height, const int width, const int kernel_h,\n const int kernel_w, const int pad_h, const int pad_w, const int stride_h,\n const int stride_w, const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int offset_channels, const int deformable_group, const int height_col,\n const int width_col, T *grad_offset) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n T val = 0;\n int w = index % width_col;\n int h = (index / width_col) % height_col;\n int c = (index / width_col / height_col) % offset_channels;\n int b = (index / width_col / height_col) / offset_channels;\n // compute the start and end of the output\n\n const int deformable_group_index = c / (2 * kernel_h * kernel_w);\n const int col_step = kernel_h * kernel_w;\n int cnt = 0;\n const T *data_col_ptr = data_col + deformable_group_index *\n channel_per_deformable_group *\n batch_size * width_col * height_col;\n const T *data_im_ptr =\n data_im + (b * deformable_group + deformable_group_index) *\n channel_per_deformable_group / kernel_h / kernel_w *\n height * width;\n const T *data_offset_ptr =\n data_offset + (b * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n\n const int offset_c = c - deformable_group_index * 2 * kernel_h * kernel_w;\n\n for (int col_c = (offset_c / 2); col_c < channel_per_deformable_group;\n col_c += col_step) {\n const int col_pos =\n (((col_c * batch_size + b) * height_col) + h) * width_col + w;\n const int bp_dir = offset_c % 2;\n\n int j = (col_pos / width_col / height_col / batch_size) % kernel_w;\n int i =\n (col_pos / width_col / height_col / batch_size / kernel_w) % kernel_h;\n int w_out = col_pos % width_col;\n int h_out = (col_pos / width_col) % height_col;\n int w_in = w_out * stride_w - pad_w;\n int h_in = h_out * stride_h - pad_h;\n const int data_offset_h_ptr =\n (((2 * (i * kernel_w + j)) * height_col + h_out) * width_col + w_out);\n const int data_offset_w_ptr =\n (((2 * (i * kernel_w + j) + 1) * height_col + h_out) * width_col +\n w_out);\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n T inv_h = h_in + i * dilation_h + offset_h;\n T inv_w = w_in + j * dilation_w + offset_w;\n if (inv_h <= -1 || inv_w <= -1 || inv_h >= height || inv_w >= width)\n inv_h = inv_w = -2;\n const T weight = get_coordinate_weight(inv_h, inv_w, height, width,\n data_im_ptr + cnt * height * width,\n width, bp_dir);\n val += weight * data_col_ptr[col_pos];\n cnt += 1;\n }\n\n grad_offset[index] = val;\n }\n}\n\n#endif // DEFORM_CONV_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/deform_roi_pool_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef DEFORM_ROI_POOL_CUDA_KERNEL_CUH\n#define DEFORM_ROI_POOL_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void deform_roi_pool_forward_cuda_kernel(\n const int nthreads, const T* input, const T* rois, const T* offset,\n T* output, const int pooled_height, const int pooled_width,\n const T spatial_scale, const int sampling_ratio, const T gamma,\n const int channels, const int height, const int width) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const T* offset_rois = rois + n * 5;\n int roi_batch_ind = offset_rois[0];\n\n // Do not using rounding; this implementation detail is critical\n T roi_start_w = offset_rois[1] * spatial_scale - 0.5;\n T roi_start_h = offset_rois[2] * spatial_scale - 0.5;\n T roi_end_w = offset_rois[3] * spatial_scale - 0.5;\n T roi_end_h = offset_rois[4] * spatial_scale - 0.5;\n\n T roi_width = roi_end_w - roi_start_w;\n T roi_height = roi_end_h - roi_start_h;\n\n T bin_size_h = static_cast(roi_height) / static_cast(pooled_height);\n T bin_size_w = static_cast(roi_width) / static_cast(pooled_width);\n\n const T* offset_input =\n input + (roi_batch_ind * channels + c) * height * width;\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h =\n (sampling_ratio > 0)\n ? sampling_ratio\n : static_cast(ceilf(roi_height / pooled_height));\n int roi_bin_grid_w =\n (sampling_ratio > 0)\n ? sampling_ratio\n : static_cast(ceilf(roi_width / pooled_width));\n\n // Compute roi offset\n if (offset != NULL) {\n const T* offset_cur_w = offset + n * pooled_width * pooled_height * 2 +\n ph * pooled_width + pw;\n T offset_roi_w = gamma * roi_width * offset_cur_w[0];\n T offset_roi_h =\n gamma * roi_height * offset_cur_w[pooled_width * pooled_height];\n roi_start_w += offset_roi_w;\n roi_start_h += offset_roi_h;\n }\n\n // We do average pooling inside a bin\n const T count = max(roi_bin_grid_h * roi_bin_grid_w, 1);\n T output_val = 0.;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const T y = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h);\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const T x = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n T val = bilinear_interpolate(offset_input, height, width, y, x, index);\n output_val += val;\n }\n }\n output[index] = output_val / count;\n }\n}\n\ntemplate \n__global__ void deform_roi_pool_backward_cuda_kernel(\n const int nthreads, const T* grad_output, const T* input, const T* rois,\n const T* offset, T* grad_input, T* grad_offset, const int pooled_height,\n const int pooled_width, const T spatial_scale, const int sampling_ratio,\n const T gamma, const int channels, const int height, const int width) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const T* offset_rois = rois + n * 5;\n int roi_batch_ind = offset_rois[0];\n const T* offset_input =\n input + ((roi_batch_ind * channels + c) * height * width);\n T* offset_grad_input =\n grad_input + ((roi_batch_ind * channels + c) * height * width);\n\n // Do not using rounding; this implementation detail is critical\n T roi_start_w = offset_rois[1] * spatial_scale - 0.5;\n T roi_start_h = offset_rois[2] * spatial_scale - 0.5;\n T roi_end_w = offset_rois[3] * spatial_scale - 0.5;\n T roi_end_h = offset_rois[4] * spatial_scale - 0.5;\n\n T roi_width = roi_end_w - roi_start_w;\n T roi_height = roi_end_h - roi_start_h;\n\n T bin_size_h = static_cast(roi_height) / static_cast(pooled_height);\n T bin_size_w = static_cast(roi_width) / static_cast(pooled_width);\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h =\n (sampling_ratio > 0)\n ? sampling_ratio\n : static_cast(ceilf(roi_height / pooled_height));\n int roi_bin_grid_w =\n (sampling_ratio > 0)\n ? sampling_ratio\n : static_cast(ceilf(roi_width / pooled_width));\n\n // Compute roi offset\n if (offset != NULL) {\n const T* offset_cur_w = offset + n * pooled_width * pooled_height * 2 +\n ph * pooled_width + pw;\n T offset_roi_w = gamma * roi_width * offset_cur_w[0];\n T offset_roi_h =\n gamma * roi_height * offset_cur_w[pooled_width * pooled_height];\n roi_start_w += offset_roi_w;\n roi_start_h += offset_roi_h;\n }\n\n // We do average (integral) pooling inside a bin\n const T count = roi_bin_grid_h * roi_bin_grid_w; // e.g. = 4\n const T grad_output_this_bin = grad_output[index] / count;\n\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const T y = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h);\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const T x = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n T w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3, w4,\n x_low, x_high, y_low, y_high, index);\n\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n atomicAdd(offset_grad_input + y_low * width + x_low,\n grad_output_this_bin * w1);\n atomicAdd(offset_grad_input + y_low * width + x_high,\n grad_output_this_bin * w2);\n atomicAdd(offset_grad_input + y_high * width + x_low,\n grad_output_this_bin * w3);\n atomicAdd(offset_grad_input + y_high * width + x_high,\n grad_output_this_bin * w4);\n if (offset != NULL) {\n T input_00 = offset_input[y_low * width + x_low];\n T input_10 = offset_input[y_low * width + x_high];\n T input_01 = offset_input[y_high * width + x_low];\n T input_11 = offset_input[y_high * width + x_high];\n T ogx = gamma * roi_width * grad_output_this_bin *\n (input_11 * (y - y_low) + input_10 * (y_high - y) +\n input_01 * (y_low - y) + input_00 * (y - y_high));\n T ogy = gamma * roi_height * grad_output_this_bin *\n (input_11 * (x - x_low) + input_01 * (x_high - x) +\n input_10 * (x_low - x) + input_00 * (x - x_high));\n atomicAdd(grad_offset + n * pooled_width * pooled_height * 2 +\n ph * pooled_width + pw,\n ogx);\n atomicAdd(grad_offset + n * pooled_width * pooled_height * 2 +\n pooled_width * pooled_height + ph * pooled_width + pw,\n ogy);\n }\n }\n }\n }\n }\n}\n\n#endif // DEFORM_ROI_POOL_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/furthest_point_sample_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef FURTHEST_POINT_SAMPLE_CUDA_KERNEL_CUH\n#define FURTHEST_POINT_SAMPLE_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n__device__ void __update(float *__restrict__ dists, int *__restrict__ dists_i,\n int idx1, int idx2) {\n const float v1 = dists[idx1], v2 = dists[idx2];\n const int i1 = dists_i[idx1], i2 = dists_i[idx2];\n dists[idx1] = max(v1, v2);\n dists_i[idx1] = v2 > v1 ? i2 : i1;\n}\n\ntemplate \n__global__ void furthest_point_sampling_forward_cuda_kernel(\n int b, int n, int m, const float *__restrict__ dataset,\n float *__restrict__ temp, int *__restrict__ idxs) {\n // dataset: (B, N, 3)\n // tmp: (B, N)\n // output:\n // idx: (B, M)\n\n if (m <= 0) return;\n __shared__ float dists[block_size];\n __shared__ int dists_i[block_size];\n\n int batch_index = blockIdx.x;\n dataset += batch_index * n * 3;\n temp += batch_index * n;\n idxs += batch_index * m;\n\n int tid = threadIdx.x;\n const int stride = block_size;\n\n int old = 0;\n if (threadIdx.x == 0) idxs[0] = old;\n\n __syncthreads();\n for (int j = 1; j < m; j++) {\n int besti = 0;\n float best = -1;\n float x1 = dataset[old * 3 + 0];\n float y1 = dataset[old * 3 + 1];\n float z1 = dataset[old * 3 + 2];\n for (int k = tid; k < n; k += stride) {\n float x2, y2, z2;\n x2 = dataset[k * 3 + 0];\n y2 = dataset[k * 3 + 1];\n z2 = dataset[k * 3 + 2];\n // float mag = (x2 * x2) + (y2 * y2) + (z2 * z2);\n // if (mag <= 1e-3)\n // continue;\n\n float d =\n (x2 - x1) * (x2 - x1) + (y2 - y1) * (y2 - y1) + (z2 - z1) * (z2 - z1);\n float d2 = min(d, temp[k]);\n temp[k] = d2;\n besti = d2 > best ? k : besti;\n best = d2 > best ? d2 : best;\n }\n dists[tid] = best;\n dists_i[tid] = besti;\n __syncthreads();\n\n#pragma unroll\n for (int block_size_thres = 1024; block_size_thres >= 2;\n block_size_thres >>= 1) {\n const int tid_thres = block_size_thres / 2;\n if (block_size >= block_size_thres && tid < tid_thres) {\n __update(dists, dists_i, tid, tid + tid_thres);\n }\n __syncthreads();\n }\n\n old = dists_i[0];\n if (tid == 0) idxs[j] = old;\n }\n}\n\n// Modified from\n// https://github.com/qiqihaer/3DSSD-pytorch/blob/master/lib/pointnet2/src/sampling_gpu.cu\ntemplate \n__global__ void furthest_point_sampling_with_dist_forward_cuda_kernel(\n int b, int n, int m, const float *__restrict__ dataset,\n float *__restrict__ temp, int *__restrict__ idxs) {\n // dataset: (B, N, N)\n // tmp: (B, N)\n // output:\n // idx: (B, M)\n\n if (m <= 0) return;\n __shared__ float dists[block_size];\n __shared__ int dists_i[block_size];\n\n int batch_index = blockIdx.x;\n dataset += batch_index * n * n;\n temp += batch_index * n;\n idxs += batch_index * m;\n\n int tid = threadIdx.x;\n const int stride = block_size;\n\n int old = 0;\n if (threadIdx.x == 0) idxs[0] = old;\n\n __syncthreads();\n for (int j = 1; j < m; j++) {\n int besti = 0;\n float best = -1;\n // float x1 = dataset[old * 3 + 0];\n // float y1 = dataset[old * 3 + 1];\n // float z1 = dataset[old * 3 + 2];\n for (int k = tid; k < n; k += stride) {\n // float x2, y2, z2;\n // x2 = dataset[k * 3 + 0];\n // y2 = dataset[k * 3 + 1];\n // z2 = dataset[k * 3 + 2];\n\n // float d = (x2 - x1) * (x2 - x1) + (y2 - y1) * (y2 - y1) + (z2 - z1) *\n // (z2 - z1);\n float d = dataset[old * n + k];\n\n float d2 = min(d, temp[k]);\n temp[k] = d2;\n besti = d2 > best ? k : besti;\n best = d2 > best ? d2 : best;\n }\n dists[tid] = best;\n dists_i[tid] = besti;\n __syncthreads();\n\n#pragma unroll\n for (int block_size_thres = 1024; block_size_thres >= 2;\n block_size_thres >>= 1) {\n const int tid_thres = block_size_thres / 2;\n if (block_size >= block_size_thres && tid < tid_thres) {\n __update(dists, dists_i, tid, tid + tid_thres);\n }\n __syncthreads();\n }\n\n old = dists_i[0];\n if (tid == 0) idxs[j] = old;\n }\n}\n\n#endif // FURTHEST_POINT_SAMPLE_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/gather_points_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef GATHER_POINTS_CUDA_KERNEL_CUH\n#define GATHER_POINTS_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n#define TOTAL_THREADS 1024\n\ntemplate \n__global__ void gather_points_forward_cuda_kernel(int b, int c, int n, int m,\n const T *points,\n const int *__restrict__ idx,\n T *out) {\n // points: (B, C, N)\n // idx: (B, M)\n // output:\n // out: (B, C, M)\n\n int bs_idx = blockIdx.z;\n int c_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, m) {\n if (bs_idx >= b || c_idx >= c) return;\n\n out += bs_idx * c * m + c_idx * m + pt_idx;\n idx += bs_idx * m + pt_idx;\n points += bs_idx * c * n + c_idx * n;\n out[0] = points[idx[0]];\n }\n}\n\ntemplate \n__global__ void gather_points_backward_cuda_kernel(int b, int c, int n, int m,\n const T *grad_out,\n const int *__restrict__ idx,\n T *grad_points) {\n // grad_out: (B, C, M)\n // idx: (B, M)\n // output:\n // grad_points: (B, C, N)\n\n int bs_idx = blockIdx.z;\n int c_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, m) {\n if (bs_idx >= b || c_idx >= c) return;\n\n grad_out += bs_idx * c * m + c_idx * m + pt_idx;\n idx += bs_idx * m + pt_idx;\n grad_points += bs_idx * c * n + c_idx * n;\n\n atomicAdd(grad_points + idx[0], grad_out[0]);\n }\n}\n\n#endif // GATHER_POINTS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/group_points_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/group_points_gpu.cu\n#ifndef GROUP_POINTS_CUDA_KERNEL_CUH\n#define GROUP_POINTS_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void group_points_forward_cuda_kernel(int b, int c, int n,\n int npoints, int nsample,\n const T *points,\n const int *__restrict__ idx,\n T *out) {\n // points: (B, C, N)\n // idx: (B, npoints, nsample)\n // output:\n // out: (B, C, npoints, nsample)\n int bs_idx = blockIdx.z;\n int c_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(index, npoints * nsample) {\n if (bs_idx >= b || c_idx >= c) return;\n\n int pt_idx = index / nsample;\n int sample_idx = index % nsample;\n\n idx += bs_idx * npoints * nsample + pt_idx * nsample + sample_idx;\n int in_idx = bs_idx * c * n + c_idx * n + idx[0];\n int out_idx = bs_idx * c * npoints * nsample + c_idx * npoints * nsample +\n pt_idx * nsample + sample_idx;\n\n out[out_idx] = points[in_idx];\n }\n}\n\ntemplate \n__global__ void group_points_backward_cuda_kernel(int b, int c, int n,\n int npoints, int nsample,\n const T *grad_out,\n const int *__restrict__ idx,\n T *grad_points) {\n // grad_out: (B, C, npoints, nsample)\n // idx: (B, npoints, nsample)\n // output:\n // grad_points: (B, C, N)\n int bs_idx = blockIdx.z;\n int c_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(index, npoints * nsample) {\n int pt_idx = index / nsample;\n if (bs_idx >= b || c_idx >= c) return;\n\n int sample_idx = index % nsample;\n grad_out += bs_idx * c * npoints * nsample + c_idx * npoints * nsample +\n pt_idx * nsample + sample_idx;\n idx += bs_idx * npoints * nsample + pt_idx * nsample + sample_idx;\n\n atomicAdd(grad_points + bs_idx * c * n + c_idx * n + idx[0], grad_out[0]);\n }\n}\n\n#endif // GROUP_POINTS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/iou3d_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef IOU3D_CUDA_KERNEL_CUH\n#define IOU3D_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\nconst int THREADS_PER_BLOCK_IOU3D = 16;\nconst int THREADS_PER_BLOCK_NMS = sizeof(unsigned long long) * 8;\n__device__ const float EPS = 1e-8;\n\nstruct Point {\n float x, y;\n __device__ Point() {}\n __device__ Point(double _x, double _y) { x = _x, y = _y; }\n\n __device__ void set(float _x, float _y) {\n x = _x;\n y = _y;\n }\n\n __device__ Point operator+(const Point &b) const {\n return Point(x + b.x, y + b.y);\n }\n\n __device__ Point operator-(const Point &b) const {\n return Point(x - b.x, y - b.y);\n }\n};\n\n__device__ inline float cross(const Point &a, const Point &b) {\n return a.x * b.y - a.y * b.x;\n}\n\n__device__ inline float cross(const Point &p1, const Point &p2,\n const Point &p0) {\n return (p1.x - p0.x) * (p2.y - p0.y) - (p2.x - p0.x) * (p1.y - p0.y);\n}\n\n__device__ int check_rect_cross(const Point &p1, const Point &p2,\n const Point &q1, const Point &q2) {\n int ret = min(p1.x, p2.x) <= max(q1.x, q2.x) &&\n min(q1.x, q2.x) <= max(p1.x, p2.x) &&\n min(p1.y, p2.y) <= max(q1.y, q2.y) &&\n min(q1.y, q2.y) <= max(p1.y, p2.y);\n return ret;\n}\n\n__device__ inline int check_in_box2d(const float *box, const Point &p) {\n // params: box (5) [x1, y1, x2, y2, angle]\n const float MARGIN = 1e-5;\n\n float center_x = (box[0] + box[2]) / 2;\n float center_y = (box[1] + box[3]) / 2;\n float angle_cos = cos(-box[4]),\n angle_sin =\n sin(-box[4]); // rotate the point in the opposite direction of box\n float rot_x =\n (p.x - center_x) * angle_cos - (p.y - center_y) * angle_sin + center_x;\n float rot_y =\n (p.x - center_x) * angle_sin + (p.y - center_y) * angle_cos + center_y;\n\n return (rot_x > box[0] - MARGIN && rot_x < box[2] + MARGIN &&\n rot_y > box[1] - MARGIN && rot_y < box[3] + MARGIN);\n}\n\n__device__ inline int intersection(const Point &p1, const Point &p0,\n const Point &q1, const Point &q0,\n Point &ans_point) {\n // fast exclusion\n if (check_rect_cross(p0, p1, q0, q1) == 0) return 0;\n\n // check cross standing\n float s1 = cross(q0, p1, p0);\n float s2 = cross(p1, q1, p0);\n float s3 = cross(p0, q1, q0);\n float s4 = cross(q1, p1, q0);\n\n if (!(s1 * s2 > 0 && s3 * s4 > 0)) return 0;\n\n // calculate intersection of two lines\n float s5 = cross(q1, p1, p0);\n if (fabs(s5 - s1) > EPS) {\n ans_point.x = (s5 * q0.x - s1 * q1.x) / (s5 - s1);\n ans_point.y = (s5 * q0.y - s1 * q1.y) / (s5 - s1);\n\n } else {\n float a0 = p0.y - p1.y, b0 = p1.x - p0.x, c0 = p0.x * p1.y - p1.x * p0.y;\n float a1 = q0.y - q1.y, b1 = q1.x - q0.x, c1 = q0.x * q1.y - q1.x * q0.y;\n float D = a0 * b1 - a1 * b0;\n\n ans_point.x = (b0 * c1 - b1 * c0) / D;\n ans_point.y = (a1 * c0 - a0 * c1) / D;\n }\n\n return 1;\n}\n\n__device__ inline void rotate_around_center(const Point ¢er,\n const float angle_cos,\n const float angle_sin, Point &p) {\n float new_x =\n (p.x - center.x) * angle_cos - (p.y - center.y) * angle_sin + center.x;\n float new_y =\n (p.x - center.x) * angle_sin + (p.y - center.y) * angle_cos + center.y;\n p.set(new_x, new_y);\n}\n\n__device__ inline int point_cmp(const Point &a, const Point &b,\n const Point ¢er) {\n return atan2(a.y - center.y, a.x - center.x) >\n atan2(b.y - center.y, b.x - center.x);\n}\n\n__device__ inline float box_overlap(const float *box_a, const float *box_b) {\n // params: box_a (5) [x1, y1, x2, y2, angle]\n // params: box_b (5) [x1, y1, x2, y2, angle]\n\n float a_x1 = box_a[0], a_y1 = box_a[1], a_x2 = box_a[2], a_y2 = box_a[3],\n a_angle = box_a[4];\n float b_x1 = box_b[0], b_y1 = box_b[1], b_x2 = box_b[2], b_y2 = box_b[3],\n b_angle = box_b[4];\n\n Point center_a((a_x1 + a_x2) / 2, (a_y1 + a_y2) / 2);\n Point center_b((b_x1 + b_x2) / 2, (b_y1 + b_y2) / 2);\n\n Point box_a_corners[5];\n box_a_corners[0].set(a_x1, a_y1);\n box_a_corners[1].set(a_x2, a_y1);\n box_a_corners[2].set(a_x2, a_y2);\n box_a_corners[3].set(a_x1, a_y2);\n\n Point box_b_corners[5];\n box_b_corners[0].set(b_x1, b_y1);\n box_b_corners[1].set(b_x2, b_y1);\n box_b_corners[2].set(b_x2, b_y2);\n box_b_corners[3].set(b_x1, b_y2);\n\n // get oriented corners\n float a_angle_cos = cos(a_angle), a_angle_sin = sin(a_angle);\n float b_angle_cos = cos(b_angle), b_angle_sin = sin(b_angle);\n\n for (int k = 0; k < 4; k++) {\n rotate_around_center(center_a, a_angle_cos, a_angle_sin, box_a_corners[k]);\n rotate_around_center(center_b, b_angle_cos, b_angle_sin, box_b_corners[k]);\n }\n\n box_a_corners[4] = box_a_corners[0];\n box_b_corners[4] = box_b_corners[0];\n\n // get intersection of lines\n Point cross_points[16];\n Point poly_center;\n int cnt = 0, flag = 0;\n\n poly_center.set(0, 0);\n for (int i = 0; i < 4; i++) {\n for (int j = 0; j < 4; j++) {\n flag = intersection(box_a_corners[i + 1], box_a_corners[i],\n box_b_corners[j + 1], box_b_corners[j],\n cross_points[cnt]);\n if (flag) {\n poly_center = poly_center + cross_points[cnt];\n cnt++;\n }\n }\n }\n\n // check corners\n for (int k = 0; k < 4; k++) {\n if (check_in_box2d(box_a, box_b_corners[k])) {\n poly_center = poly_center + box_b_corners[k];\n cross_points[cnt] = box_b_corners[k];\n cnt++;\n }\n if (check_in_box2d(box_b, box_a_corners[k])) {\n poly_center = poly_center + box_a_corners[k];\n cross_points[cnt] = box_a_corners[k];\n cnt++;\n }\n }\n\n poly_center.x /= cnt;\n poly_center.y /= cnt;\n\n // sort the points of polygon\n Point temp;\n for (int j = 0; j < cnt - 1; j++) {\n for (int i = 0; i < cnt - j - 1; i++) {\n if (point_cmp(cross_points[i], cross_points[i + 1], poly_center)) {\n temp = cross_points[i];\n cross_points[i] = cross_points[i + 1];\n cross_points[i + 1] = temp;\n }\n }\n }\n\n // get the overlap areas\n float area = 0;\n for (int k = 0; k < cnt - 1; k++) {\n area += cross(cross_points[k] - cross_points[0],\n cross_points[k + 1] - cross_points[0]);\n }\n\n return fabs(area) / 2.0;\n}\n\n__device__ inline float iou_bev(const float *box_a, const float *box_b) {\n // params: box_a (5) [x1, y1, x2, y2, angle]\n // params: box_b (5) [x1, y1, x2, y2, angle]\n float sa = (box_a[2] - box_a[0]) * (box_a[3] - box_a[1]);\n float sb = (box_b[2] - box_b[0]) * (box_b[3] - box_b[1]);\n float s_overlap = box_overlap(box_a, box_b);\n return s_overlap / fmaxf(sa + sb - s_overlap, EPS);\n}\n\n__global__ void iou3d_boxes_overlap_bev_forward_cuda_kernel(\n const int num_a, const float *boxes_a, const int num_b,\n const float *boxes_b, float *ans_overlap) {\n CUDA_2D_KERNEL_LOOP(b_idx, num_b, a_idx, num_a) {\n if (a_idx >= num_a || b_idx >= num_b) {\n return;\n }\n const float *cur_box_a = boxes_a + a_idx * 5;\n const float *cur_box_b = boxes_b + b_idx * 5;\n float s_overlap = box_overlap(cur_box_a, cur_box_b);\n ans_overlap[a_idx * num_b + b_idx] = s_overlap;\n }\n}\n\n__global__ void iou3d_boxes_iou_bev_forward_cuda_kernel(const int num_a,\n const float *boxes_a,\n const int num_b,\n const float *boxes_b,\n float *ans_iou) {\n CUDA_2D_KERNEL_LOOP(b_idx, num_b, a_idx, num_a) {\n if (a_idx >= num_a || b_idx >= num_b) {\n return;\n }\n\n const float *cur_box_a = boxes_a + a_idx * 5;\n const float *cur_box_b = boxes_b + b_idx * 5;\n float cur_iou_bev = iou_bev(cur_box_a, cur_box_b);\n ans_iou[a_idx * num_b + b_idx] = cur_iou_bev;\n }\n}\n\n__global__ void nms_forward_cuda_kernel(const int boxes_num,\n const float nms_overlap_thresh,\n const float *boxes,\n unsigned long long *mask) {\n // params: boxes (N, 5) [x1, y1, x2, y2, ry]\n // params: mask (N, N/THREADS_PER_BLOCK_NMS)\n const int blocks =\n (boxes_num + THREADS_PER_BLOCK_NMS - 1) / THREADS_PER_BLOCK_NMS;\n CUDA_2D_KERNEL_BLOCK_LOOP(col_start, blocks, row_start, blocks) {\n // if (row_start > col_start) return;\n\n const int row_size = fminf(boxes_num - row_start * THREADS_PER_BLOCK_NMS,\n THREADS_PER_BLOCK_NMS);\n const int col_size = fminf(boxes_num - col_start * THREADS_PER_BLOCK_NMS,\n THREADS_PER_BLOCK_NMS);\n\n __shared__ float block_boxes[THREADS_PER_BLOCK_NMS * 5];\n\n if (threadIdx.x < col_size) {\n block_boxes[threadIdx.x * 5 + 0] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 0];\n block_boxes[threadIdx.x * 5 + 1] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 1];\n block_boxes[threadIdx.x * 5 + 2] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 2];\n block_boxes[threadIdx.x * 5 + 3] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 3];\n block_boxes[threadIdx.x * 5 + 4] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 4];\n }\n __syncthreads();\n\n if (threadIdx.x < row_size) {\n const int cur_box_idx = THREADS_PER_BLOCK_NMS * row_start + threadIdx.x;\n const float *cur_box = boxes + cur_box_idx * 5;\n\n int i = 0;\n unsigned long long t = 0;\n int start = 0;\n if (row_start == col_start) {\n start = threadIdx.x + 1;\n }\n for (i = start; i < col_size; i++) {\n if (iou_bev(cur_box, block_boxes + i * 5) > nms_overlap_thresh) {\n t |= 1ULL << i;\n }\n }\n const int col_blocks =\n (boxes_num + THREADS_PER_BLOCK_NMS - 1) / THREADS_PER_BLOCK_NMS;\n mask[cur_box_idx * col_blocks + col_start] = t;\n }\n }\n}\n\n__device__ inline float iou_normal(float const *const a, float const *const b) {\n float left = fmaxf(a[0], b[0]), right = fminf(a[2], b[2]);\n float top = fmaxf(a[1], b[1]), bottom = fminf(a[3], b[3]);\n float width = fmaxf(right - left, 0.f), height = fmaxf(bottom - top, 0.f);\n float interS = width * height;\n float Sa = (a[2] - a[0]) * (a[3] - a[1]);\n float Sb = (b[2] - b[0]) * (b[3] - b[1]);\n return interS / fmaxf(Sa + Sb - interS, EPS);\n}\n\n__global__ void nms_normal_forward_cuda_kernel(const int boxes_num,\n const float nms_overlap_thresh,\n const float *boxes,\n unsigned long long *mask) {\n // params: boxes (N, 5) [x1, y1, x2, y2, ry]\n // params: mask (N, N/THREADS_PER_BLOCK_NMS)\n\n const int blocks =\n (boxes_num + THREADS_PER_BLOCK_NMS - 1) / THREADS_PER_BLOCK_NMS;\n CUDA_2D_KERNEL_BLOCK_LOOP(col_start, blocks, row_start, blocks) {\n // if (row_start > col_start) return;\n\n const int row_size = fminf(boxes_num - row_start * THREADS_PER_BLOCK_NMS,\n THREADS_PER_BLOCK_NMS);\n const int col_size = fminf(boxes_num - col_start * THREADS_PER_BLOCK_NMS,\n THREADS_PER_BLOCK_NMS);\n\n __shared__ float block_boxes[THREADS_PER_BLOCK_NMS * 5];\n\n if (threadIdx.x < col_size) {\n block_boxes[threadIdx.x * 5 + 0] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 0];\n block_boxes[threadIdx.x * 5 + 1] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 1];\n block_boxes[threadIdx.x * 5 + 2] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 2];\n block_boxes[threadIdx.x * 5 + 3] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 3];\n block_boxes[threadIdx.x * 5 + 4] =\n boxes[(THREADS_PER_BLOCK_NMS * col_start + threadIdx.x) * 5 + 4];\n }\n __syncthreads();\n\n if (threadIdx.x < row_size) {\n const int cur_box_idx = THREADS_PER_BLOCK_NMS * row_start + threadIdx.x;\n const float *cur_box = boxes + cur_box_idx * 5;\n\n int i = 0;\n unsigned long long t = 0;\n int start = 0;\n if (row_start == col_start) {\n start = threadIdx.x + 1;\n }\n for (i = start; i < col_size; i++) {\n if (iou_normal(cur_box, block_boxes + i * 5) > nms_overlap_thresh) {\n t |= 1ULL << i;\n }\n }\n const int col_blocks =\n (boxes_num + THREADS_PER_BLOCK_NMS - 1) / THREADS_PER_BLOCK_NMS;\n mask[cur_box_idx * col_blocks + col_start] = t;\n }\n }\n}\n\n#endif // IOU3D_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/knn_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/CVMI-Lab/PAConv/tree/main/scene_seg/lib/pointops/src/knnquery_heap\n#ifndef KNN_CUDA_KERNEL_CUH\n#define KNN_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ninline __device__ void swap_float(float *x, float *y) {\n float tmp = *x;\n *x = *y;\n *y = tmp;\n}\n\ninline __device__ void swap_int(int *x, int *y) {\n int tmp = *x;\n *x = *y;\n *y = tmp;\n}\n\n__device__ void reheap(float *dist, int *idx, int k) {\n int root = 0;\n int child = root * 2 + 1;\n while (child < k) {\n if (child + 1 < k && dist[child + 1] > dist[child]) child++;\n if (dist[root] > dist[child]) return;\n swap_float(&dist[root], &dist[child]);\n swap_int(&idx[root], &idx[child]);\n root = child;\n child = root * 2 + 1;\n }\n}\n\n__device__ void heap_sort(float *dist, int *idx, int k) {\n int i;\n for (i = k - 1; i > 0; i--) {\n swap_float(&dist[0], &dist[i]);\n swap_int(&idx[0], &idx[i]);\n reheap(dist, idx, i);\n }\n}\n\n// input: xyz (b, n, 3) new_xyz (b, m, 3)\n// output: idx (b, m, nsample) dist2 (b, m, nsample)\ntemplate \n__global__ void knn_forward_cuda_kernel(int b, int n, int m, int nsample,\n const T *xyz, const T *new_xyz,\n int *__restrict__ idx, T *dist2) {\n int bs_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, m) {\n if (bs_idx >= b) return;\n\n new_xyz += bs_idx * m * 3 + pt_idx * 3;\n xyz += bs_idx * n * 3;\n idx += bs_idx * m * nsample + pt_idx * nsample;\n dist2 += bs_idx * m * nsample + pt_idx * nsample;\n\n T new_x = new_xyz[0];\n T new_y = new_xyz[1];\n T new_z = new_xyz[2];\n\n float best_dist[100];\n int best_idx[100];\n for (int i = 0; i < nsample; i++) {\n best_dist[i] = 1e10;\n best_idx[i] = 0;\n }\n for (int i = 0; i < n; i++) {\n T x = xyz[i * 3 + 0];\n T y = xyz[i * 3 + 1];\n T z = xyz[i * 3 + 2];\n T d2 = (new_x - x) * (new_x - x) + (new_y - y) * (new_y - y) +\n (new_z - z) * (new_z - z);\n if (d2 < best_dist[0]) {\n best_dist[0] = d2;\n best_idx[0] = i;\n reheap(best_dist, best_idx, nsample);\n }\n }\n heap_sort(best_dist, best_idx, nsample);\n for (int i = 0; i < nsample; i++) {\n idx[i] = best_idx[i];\n dist2[i] = best_dist[i];\n }\n }\n}\n\n#endif // KNN_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/masked_conv2d_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef MASKED_CONV2D_CUDA_KERNEL_CUH\n#define MASKED_CONV2D_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void MaskedIm2colForward(const int n, const scalar_t *data_im,\n const int height, const int width,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w,\n const int64_t *mask_h_idx,\n const int64_t *mask_w_idx,\n const int mask_cnt, scalar_t *data_col) {\n // mask_cnt * channels\n CUDA_1D_KERNEL_LOOP(index, n) {\n const int m_index = index % mask_cnt;\n const int h_col = mask_h_idx[m_index];\n const int w_col = mask_w_idx[m_index];\n const int c_im = index / mask_cnt;\n const int c_col = c_im * kernel_h * kernel_w;\n const int h_offset = h_col - pad_h;\n const int w_offset = w_col - pad_w;\n scalar_t *data_col_ptr = data_col + c_col * mask_cnt + m_index;\n for (int i = 0; i < kernel_h; ++i) {\n int h_im = h_offset + i;\n for (int j = 0; j < kernel_w; ++j) {\n int w_im = w_offset + j;\n if (h_im >= 0 && w_im >= 0 && h_im < height && w_im < width) {\n *data_col_ptr =\n (scalar_t)data_im[(c_im * height + h_im) * width + w_im];\n } else {\n *data_col_ptr = 0.0;\n }\n data_col_ptr += mask_cnt;\n }\n }\n }\n}\n\ntemplate \n__global__ void MaskedCol2imForward(const int n, const scalar_t *data_col,\n const int height, const int width,\n const int channels,\n const int64_t *mask_h_idx,\n const int64_t *mask_w_idx,\n const int mask_cnt, scalar_t *data_im) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n const int m_index = index % mask_cnt;\n const int h_im = mask_h_idx[m_index];\n const int w_im = mask_w_idx[m_index];\n const int c_im = index / mask_cnt;\n // compute the start and end of the output\n data_im[(c_im * height + h_im) * width + w_im] = data_col[index];\n }\n}\n\n#endif // MASKED_CONV2D_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/min_area_polygons_cuda.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef MIN_AREA_POLYGONS_CUDA_KERNEL_CUH\n#define MIN_AREA_POLYGONS_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n#define MAXN 20\n__device__ const float PI = 3.1415926;\n\nstruct Point {\n float x, y;\n __device__ Point() {}\n __device__ Point(float x, float y) : x(x), y(y) {}\n};\n\n__device__ inline void swap1(Point *a, Point *b) {\n Point temp;\n temp.x = a->x;\n temp.y = a->y;\n\n a->x = b->x;\n a->y = b->y;\n\n b->x = temp.x;\n b->y = temp.y;\n}\n__device__ inline float cross(Point o, Point a, Point b) {\n return (a.x - o.x) * (b.y - o.y) - (b.x - o.x) * (a.y - o.y);\n}\n\n__device__ inline float dis(Point a, Point b) {\n return (a.x - b.x) * (a.x - b.x) + (a.y - b.y) * (a.y - b.y);\n}\n__device__ inline void minBoundingRect(Point *ps, int n_points, float *minbox) {\n float convex_points[2][MAXN];\n for (int j = 0; j < n_points; j++) {\n convex_points[0][j] = ps[j].x;\n }\n for (int j = 0; j < n_points; j++) {\n convex_points[1][j] = ps[j].y;\n }\n\n Point edges[MAXN];\n float edges_angles[MAXN];\n float unique_angles[MAXN];\n int n_edges = n_points - 1;\n int n_unique = 0;\n int unique_flag = 0;\n\n for (int i = 0; i < n_edges; i++) {\n edges[i].x = ps[i + 1].x - ps[i].x;\n edges[i].y = ps[i + 1].y - ps[i].y;\n }\n for (int i = 0; i < n_edges; i++) {\n edges_angles[i] = atan2((double)edges[i].y, (double)edges[i].x);\n if (edges_angles[i] >= 0) {\n edges_angles[i] = fmod((double)edges_angles[i], (double)PI / 2);\n } else {\n edges_angles[i] =\n edges_angles[i] - (int)(edges_angles[i] / (PI / 2) - 1) * (PI / 2);\n }\n }\n unique_angles[0] = edges_angles[0];\n n_unique += 1;\n for (int i = 1; i < n_edges; i++) {\n for (int j = 0; j < n_unique; j++) {\n if (edges_angles[i] == unique_angles[j]) {\n unique_flag += 1;\n }\n }\n if (unique_flag == 0) {\n unique_angles[n_unique] = edges_angles[i];\n n_unique += 1;\n unique_flag = 0;\n } else {\n unique_flag = 0;\n }\n }\n\n float minarea = 1e12;\n for (int i = 0; i < n_unique; i++) {\n float R[2][2];\n float rot_points[2][MAXN];\n R[0][0] = cos(unique_angles[i]);\n R[0][1] = sin(unique_angles[i]);\n R[1][0] = -sin(unique_angles[i]);\n R[1][1] = cos(unique_angles[i]);\n // R x Points\n for (int m = 0; m < 2; m++) {\n for (int n = 0; n < n_points; n++) {\n float sum = 0.0;\n for (int k = 0; k < 2; k++) {\n sum = sum + R[m][k] * convex_points[k][n];\n }\n rot_points[m][n] = sum;\n }\n }\n\n // xmin;\n float xmin, ymin, xmax, ymax;\n xmin = 1e12;\n for (int j = 0; j < n_points; j++) {\n if (isinf(rot_points[0][j]) || isnan(rot_points[0][j])) {\n continue;\n } else {\n if (rot_points[0][j] < xmin) {\n xmin = rot_points[0][j];\n }\n }\n }\n // ymin\n ymin = 1e12;\n for (int j = 0; j < n_points; j++) {\n if (isinf(rot_points[1][j]) || isnan(rot_points[1][j])) {\n continue;\n } else {\n if (rot_points[1][j] < ymin) {\n ymin = rot_points[1][j];\n }\n }\n }\n // xmax\n xmax = -1e12;\n for (int j = 0; j < n_points; j++) {\n if (isinf(rot_points[0][j]) || isnan(rot_points[0][j])) {\n continue;\n } else {\n if (rot_points[0][j] > xmax) {\n xmax = rot_points[0][j];\n }\n }\n }\n // ymax\n ymax = -1e12;\n for (int j = 0; j < n_points; j++) {\n if (isinf(rot_points[1][j]) || isnan(rot_points[1][j])) {\n continue;\n } else {\n if (rot_points[1][j] > ymax) {\n ymax = rot_points[1][j];\n }\n }\n }\n float area = (xmax - xmin) * (ymax - ymin);\n if (area < minarea) {\n minarea = area;\n minbox[0] = unique_angles[i];\n minbox[1] = xmin;\n minbox[2] = ymin;\n minbox[3] = xmax;\n minbox[4] = ymax;\n }\n }\n}\n\n// convex_find\n__device__ inline void Jarvis(Point *in_poly, int &n_poly) {\n int n_input = n_poly;\n Point input_poly[20];\n for (int i = 0; i < n_input; i++) {\n input_poly[i].x = in_poly[i].x;\n input_poly[i].y = in_poly[i].y;\n }\n Point p_max, p_k;\n int max_index, k_index;\n int Stack[20], top1, top2;\n // float sign;\n double sign;\n Point right_point[10], left_point[10];\n\n for (int i = 0; i < n_poly; i++) {\n if (in_poly[i].y < in_poly[0].y ||\n in_poly[i].y == in_poly[0].y && in_poly[i].x < in_poly[0].x) {\n Point *j = &(in_poly[0]);\n Point *k = &(in_poly[i]);\n swap1(j, k);\n }\n if (i == 0) {\n p_max = in_poly[0];\n max_index = 0;\n }\n if (in_poly[i].y > p_max.y ||\n in_poly[i].y == p_max.y && in_poly[i].x > p_max.x) {\n p_max = in_poly[i];\n max_index = i;\n }\n }\n if (max_index == 0) {\n max_index = 1;\n p_max = in_poly[max_index];\n }\n\n k_index = 0, Stack[0] = 0, top1 = 0;\n while (k_index != max_index) {\n p_k = p_max;\n k_index = max_index;\n for (int i = 1; i < n_poly; i++) {\n sign = cross(in_poly[Stack[top1]], in_poly[i], p_k);\n if ((sign > 0) || ((sign == 0) && (dis(in_poly[Stack[top1]], in_poly[i]) >\n dis(in_poly[Stack[top1]], p_k)))) {\n p_k = in_poly[i];\n k_index = i;\n }\n }\n top1++;\n Stack[top1] = k_index;\n }\n\n for (int i = 0; i <= top1; i++) {\n right_point[i] = in_poly[Stack[i]];\n }\n\n k_index = 0, Stack[0] = 0, top2 = 0;\n\n while (k_index != max_index) {\n p_k = p_max;\n k_index = max_index;\n for (int i = 1; i < n_poly; i++) {\n sign = cross(in_poly[Stack[top2]], in_poly[i], p_k);\n if ((sign < 0) || (sign == 0) && (dis(in_poly[Stack[top2]], in_poly[i]) >\n dis(in_poly[Stack[top2]], p_k))) {\n p_k = in_poly[i];\n k_index = i;\n }\n }\n top2++;\n Stack[top2] = k_index;\n }\n\n for (int i = top2 - 1; i >= 0; i--) {\n left_point[i] = in_poly[Stack[i]];\n }\n\n for (int i = 0; i < top1 + top2; i++) {\n if (i <= top1) {\n in_poly[i] = right_point[i];\n } else {\n in_poly[i] = left_point[top2 - (i - top1)];\n }\n }\n n_poly = top1 + top2;\n}\n\ntemplate \n__device__ inline void Findminbox(T const *const p, T *minpoints) {\n Point ps1[MAXN];\n Point convex[MAXN];\n for (int i = 0; i < 9; i++) {\n convex[i].x = p[i * 2];\n convex[i].y = p[i * 2 + 1];\n }\n int n_convex = 9;\n Jarvis(convex, n_convex);\n int n1 = n_convex;\n for (int i = 0; i < n1; i++) {\n ps1[i].x = convex[i].x;\n ps1[i].y = convex[i].y;\n }\n ps1[n1].x = convex[0].x;\n ps1[n1].y = convex[0].y;\n\n float minbbox[5] = {0};\n minBoundingRect(ps1, n1 + 1, minbbox);\n float angle = minbbox[0];\n float xmin = minbbox[1];\n float ymin = minbbox[2];\n float xmax = minbbox[3];\n float ymax = minbbox[4];\n float R[2][2];\n\n R[0][0] = cos(angle);\n R[0][1] = sin(angle);\n R[1][0] = -sin(angle);\n R[1][1] = cos(angle);\n\n minpoints[0] = xmax * R[0][0] + ymin * R[1][0];\n minpoints[1] = xmax * R[0][1] + ymin * R[1][1];\n minpoints[2] = xmin * R[0][0] + ymin * R[1][0];\n minpoints[3] = xmin * R[0][1] + ymin * R[1][1];\n minpoints[4] = xmin * R[0][0] + ymax * R[1][0];\n minpoints[5] = xmin * R[0][1] + ymax * R[1][1];\n minpoints[6] = xmax * R[0][0] + ymax * R[1][0];\n minpoints[7] = xmax * R[0][1] + ymax * R[1][1];\n}\n\ntemplate \n__global__ void min_area_polygons_cuda_kernel(const int ex_n_boxes,\n const T *ex_boxes, T *minbox) {\n CUDA_1D_KERNEL_LOOP(index, ex_n_boxes) {\n const T *cur_box = ex_boxes + index * 18;\n T *cur_min_box = minbox + index * 8;\n Findminbox(cur_box, cur_min_box);\n }\n}\n\n#endif // MIN_AREA_POLYGONS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/modulated_deform_conv_cuda_kernel.cuh", "content": "/*!\n ******************* BEGIN Caffe Copyright Notice and Disclaimer\n *****************\n *\n * COPYRIGHT\n *\n * All contributions by the University of California:\n * Copyright (c) 2014-2017 The Regents of the University of California (Regents)\n * All rights reserved.\n *\n * All other contributions:\n * Copyright (c) 2014-2017, the respective contributors\n * All rights reserved.\n *\n * Caffe uses a shared copyright model: each contributor holds copyright over\n * their contributions to Caffe. The project versioning records all such\n * contribution and copyright details. If a contributor wants to further mark\n * their specific copyright on a particular contribution, they should indicate\n * their copyright solely in the commit message of the change when it is\n * committed.\n *\n * LICENSE\n *\n * Redistribution and use in source and binary forms, with or without\n * modification, are permitted provided that the following conditions are met:\n *\n * 1. Redistributions of source code must retain the above copyright notice,\n *this list of conditions and the following disclaimer.\n * 2. Redistributions in binary form must reproduce the above copyright notice,\n * this list of conditions and the following disclaimer in the documentation\n * and/or other materials provided with the distribution.\n *\n * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\n *AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n *IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\n * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE\n *FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n *DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n *SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n *CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\n *OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n *OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n *\n * CONTRIBUTION AGREEMENT\n *\n * By contributing to the BVLC/caffe repository through pull-request, comment,\n * or otherwise, the contributor releases their content to the\n * license and copyright terms herein.\n *\n ***************** END Caffe Copyright Notice and Disclaimer\n *********************\n *\n * Copyright (c) 2018 Microsoft\n * Licensed under The MIT License [see LICENSE for details]\n * \\file modulated_deformable_im2col.cuh\n * \\brief Function definitions of converting an image to\n * column matrix based on kernel, padding, dilation, and offset.\n * These functions are mainly used in deformable convolution operators.\n * \\ref: https://arxiv.org/abs/1703.06211\n * \\author Yuwen Xiong, Haozhi Qi, Jifeng Dai, Xizhou Zhu, Han Hu, Dazhi Cheng\n */\n\n// modified from\n// https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/blob/mmdetection/mmdet/ops/dcn/src/deform_conv_cuda_kernel.cu\n\n#ifndef MODULATED_DEFORM_CONV_CUDA_KERNEL_CUH\n#define MODULATED_DEFORM_CONV_CUDA_KERNEL_CUH\n\n#include \n#ifdef MMCV_WITH_TRT\n#include \"common_cuda_helper.hpp\"\n#else // MMCV_WITH_TRT\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else // MMCV_USE_PARROTS\n#include \"pytorch_cuda_helper.hpp\"\n#endif // MMCV_USE_PARROTS\n#endif // MMCV_WITH_TRT\n\ntemplate \n__device__ T dmcn_im2col_bilinear(const T *input, const int data_width,\n const int height, const int width, T h, T w) {\n int h_low = floorf(h);\n int w_low = floorf(w);\n int h_high = h_low + 1;\n int w_high = w_low + 1;\n\n T lh = h - h_low;\n T lw = w - w_low;\n T hh = 1 - lh, hw = 1 - lw;\n\n T v1 = 0;\n if (h_low >= 0 && w_low >= 0) v1 = input[h_low * data_width + w_low];\n T v2 = 0;\n if (h_low >= 0 && w_high <= width - 1)\n v2 = input[h_low * data_width + w_high];\n T v3 = 0;\n if (h_high <= height - 1 && w_low >= 0)\n v3 = input[h_high * data_width + w_low];\n T v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1)\n v4 = input[h_high * data_width + w_high];\n\n T w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n\n T val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n return val;\n}\n\ntemplate \n__device__ T dmcn_get_gradient_weight(T argmax_h, T argmax_w, const int h,\n const int w, const int height,\n const int width) {\n if (argmax_h <= -1 || argmax_h >= height || argmax_w <= -1 ||\n argmax_w >= width) {\n // empty\n return 0;\n }\n\n int argmax_h_low = floorf(argmax_h);\n int argmax_w_low = floorf(argmax_w);\n int argmax_h_high = argmax_h_low + 1;\n int argmax_w_high = argmax_w_low + 1;\n\n T weight = 0;\n if (h == argmax_h_low && w == argmax_w_low)\n weight = (h + 1 - argmax_h) * (w + 1 - argmax_w);\n if (h == argmax_h_low && w == argmax_w_high)\n weight = (h + 1 - argmax_h) * (argmax_w + 1 - w);\n if (h == argmax_h_high && w == argmax_w_low)\n weight = (argmax_h + 1 - h) * (w + 1 - argmax_w);\n if (h == argmax_h_high && w == argmax_w_high)\n weight = (argmax_h + 1 - h) * (argmax_w + 1 - w);\n return weight;\n}\n\ntemplate \n__device__ T dmcn_get_coordinate_weight(T argmax_h, T argmax_w,\n const int height, const int width,\n const T *im_data, const int data_width,\n const int bp_dir) {\n if (argmax_h <= -1 || argmax_h >= height || argmax_w <= -1 ||\n argmax_w >= width) {\n // empty\n return 0;\n }\n\n int argmax_h_low = floorf(argmax_h);\n int argmax_w_low = floorf(argmax_w);\n int argmax_h_high = argmax_h_low + 1;\n int argmax_w_high = argmax_w_low + 1;\n\n T weight = 0;\n\n if (bp_dir == 0) {\n if (argmax_h_low >= 0 && argmax_w_low >= 0)\n weight += -1 * (argmax_w_low + 1 - argmax_w) *\n im_data[argmax_h_low * data_width + argmax_w_low];\n if (argmax_h_low >= 0 && argmax_w_high <= width - 1)\n weight += -1 * (argmax_w - argmax_w_low) *\n im_data[argmax_h_low * data_width + argmax_w_high];\n if (argmax_h_high <= height - 1 && argmax_w_low >= 0)\n weight += (argmax_w_low + 1 - argmax_w) *\n im_data[argmax_h_high * data_width + argmax_w_low];\n if (argmax_h_high <= height - 1 && argmax_w_high <= width - 1)\n weight += (argmax_w - argmax_w_low) *\n im_data[argmax_h_high * data_width + argmax_w_high];\n } else if (bp_dir == 1) {\n if (argmax_h_low >= 0 && argmax_w_low >= 0)\n weight += -1 * (argmax_h_low + 1 - argmax_h) *\n im_data[argmax_h_low * data_width + argmax_w_low];\n if (argmax_h_low >= 0 && argmax_w_high <= width - 1)\n weight += (argmax_h_low + 1 - argmax_h) *\n im_data[argmax_h_low * data_width + argmax_w_high];\n if (argmax_h_high <= height - 1 && argmax_w_low >= 0)\n weight += -1 * (argmax_h - argmax_h_low) *\n im_data[argmax_h_high * data_width + argmax_w_low];\n if (argmax_h_high <= height - 1 && argmax_w_high <= width - 1)\n weight += (argmax_h - argmax_h_low) *\n im_data[argmax_h_high * data_width + argmax_w_high];\n }\n\n return weight;\n}\n\ntemplate \n__global__ void modulated_deformable_im2col_gpu_kernel(\n const int n, const T *data_im, const T *data_offset, const T *data_mask,\n const int height, const int width, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int num_channels, const int deformable_group, const int height_col,\n const int width_col, T *data_col) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n // index index of output matrix\n const int w_col = index % width_col;\n const int h_col = (index / width_col) % height_col;\n const int b_col = (index / width_col / height_col) % batch_size;\n const int c_im = (index / width_col / height_col) / batch_size;\n const int c_col = c_im * kernel_h * kernel_w;\n\n // compute deformable group index\n const int deformable_group_index = c_im / channel_per_deformable_group;\n\n const int h_in = h_col * stride_h - pad_h;\n const int w_in = w_col * stride_w - pad_w;\n\n T *data_col_ptr =\n data_col +\n ((c_col * batch_size + b_col) * height_col + h_col) * width_col + w_col;\n const T *data_im_ptr =\n data_im + (b_col * num_channels + c_im) * height * width;\n const T *data_offset_ptr =\n data_offset + (b_col * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n\n const T *data_mask_ptr =\n data_mask + (b_col * deformable_group + deformable_group_index) *\n kernel_h * kernel_w * height_col * width_col;\n\n for (int i = 0; i < kernel_h; ++i) {\n for (int j = 0; j < kernel_w; ++j) {\n const int data_offset_h_ptr =\n ((2 * (i * kernel_w + j)) * height_col + h_col) * width_col + w_col;\n const int data_offset_w_ptr =\n ((2 * (i * kernel_w + j) + 1) * height_col + h_col) * width_col +\n w_col;\n const int data_mask_hw_ptr =\n ((i * kernel_w + j) * height_col + h_col) * width_col + w_col;\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n const T mask = data_mask_ptr[data_mask_hw_ptr];\n T val = static_cast(0);\n const T h_im = h_in + i * dilation_h + offset_h;\n const T w_im = w_in + j * dilation_w + offset_w;\n if (h_im > -1 && w_im > -1 && h_im < height && w_im < width)\n val = dmcn_im2col_bilinear(data_im_ptr, width, height, width, h_im,\n w_im);\n *data_col_ptr = val * mask;\n data_col_ptr += batch_size * height_col * width_col;\n }\n }\n }\n}\n\ntemplate \n__global__ void modulated_deformable_col2im_gpu_kernel(\n const int n, const T *data_col, const T *data_offset, const T *data_mask,\n const int channels, const int height, const int width, const int kernel_h,\n const int kernel_w, const int pad_h, const int pad_w, const int stride_h,\n const int stride_w, const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int deformable_group, const int height_col, const int width_col,\n T *grad_im) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n const int j = (index / width_col / height_col / batch_size) % kernel_w;\n const int i =\n (index / width_col / height_col / batch_size / kernel_w) % kernel_h;\n const int c =\n index / width_col / height_col / batch_size / kernel_w / kernel_h;\n // compute the start and end of the output\n\n const int deformable_group_index = c / channel_per_deformable_group;\n\n int w_out = index % width_col;\n int h_out = (index / width_col) % height_col;\n int b = (index / width_col / height_col) % batch_size;\n int w_in = w_out * stride_w - pad_w;\n int h_in = h_out * stride_h - pad_h;\n\n const T *data_offset_ptr =\n data_offset + (b * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n const T *data_mask_ptr =\n data_mask + (b * deformable_group + deformable_group_index) * kernel_h *\n kernel_w * height_col * width_col;\n const int data_offset_h_ptr =\n ((2 * (i * kernel_w + j)) * height_col + h_out) * width_col + w_out;\n const int data_offset_w_ptr =\n ((2 * (i * kernel_w + j) + 1) * height_col + h_out) * width_col + w_out;\n const int data_mask_hw_ptr =\n ((i * kernel_w + j) * height_col + h_out) * width_col + w_out;\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n const T mask = data_mask_ptr[data_mask_hw_ptr];\n const T cur_inv_h_data = h_in + i * dilation_h + offset_h;\n const T cur_inv_w_data = w_in + j * dilation_w + offset_w;\n\n const T cur_top_grad = data_col[index] * mask;\n const int cur_h = (int)cur_inv_h_data;\n const int cur_w = (int)cur_inv_w_data;\n for (int dy = -2; dy <= 2; dy++) {\n for (int dx = -2; dx <= 2; dx++) {\n if (cur_h + dy >= 0 && cur_h + dy < height && cur_w + dx >= 0 &&\n cur_w + dx < width && abs(cur_inv_h_data - (cur_h + dy)) < 1 &&\n abs(cur_inv_w_data - (cur_w + dx)) < 1) {\n int cur_bottom_grad_pos =\n ((b * channels + c) * height + cur_h + dy) * width + cur_w + dx;\n T weight =\n dmcn_get_gradient_weight(cur_inv_h_data, cur_inv_w_data,\n cur_h + dy, cur_w + dx, height, width);\n atomicAdd(grad_im + cur_bottom_grad_pos, weight * cur_top_grad);\n }\n }\n }\n }\n}\n\ntemplate \n__global__ void modulated_deformable_col2im_coord_gpu_kernel(\n const int n, const T *data_col, const T *data_im, const T *data_offset,\n const T *data_mask, const int channels, const int height, const int width,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int channel_per_deformable_group,\n const int batch_size, const int offset_channels, const int deformable_group,\n const int height_col, const int width_col, T *grad_offset, T *grad_mask) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n T val = 0, mval = 0;\n int w = index % width_col;\n int h = (index / width_col) % height_col;\n int c = (index / width_col / height_col) % offset_channels;\n int b = (index / width_col / height_col) / offset_channels;\n // compute the start and end of the output\n\n const int deformable_group_index = c / (2 * kernel_h * kernel_w);\n const int col_step = kernel_h * kernel_w;\n int cnt = 0;\n const T *data_col_ptr = data_col + deformable_group_index *\n channel_per_deformable_group *\n batch_size * width_col * height_col;\n const T *data_im_ptr =\n data_im + (b * deformable_group + deformable_group_index) *\n channel_per_deformable_group / kernel_h / kernel_w *\n height * width;\n const T *data_offset_ptr =\n data_offset + (b * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n const T *data_mask_ptr =\n data_mask + (b * deformable_group + deformable_group_index) * kernel_h *\n kernel_w * height_col * width_col;\n\n const int offset_c = c - deformable_group_index * 2 * kernel_h * kernel_w;\n\n for (int col_c = (offset_c / 2); col_c < channel_per_deformable_group;\n col_c += col_step) {\n const int col_pos =\n (((col_c * batch_size + b) * height_col) + h) * width_col + w;\n const int bp_dir = offset_c % 2;\n\n int j = (col_pos / width_col / height_col / batch_size) % kernel_w;\n int i =\n (col_pos / width_col / height_col / batch_size / kernel_w) % kernel_h;\n int w_out = col_pos % width_col;\n int h_out = (col_pos / width_col) % height_col;\n int w_in = w_out * stride_w - pad_w;\n int h_in = h_out * stride_h - pad_h;\n const int data_offset_h_ptr =\n (((2 * (i * kernel_w + j)) * height_col + h_out) * width_col + w_out);\n const int data_offset_w_ptr =\n (((2 * (i * kernel_w + j) + 1) * height_col + h_out) * width_col +\n w_out);\n const int data_mask_hw_ptr =\n (((i * kernel_w + j) * height_col + h_out) * width_col + w_out);\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n const T mask = data_mask_ptr[data_mask_hw_ptr];\n T inv_h = h_in + i * dilation_h + offset_h;\n T inv_w = w_in + j * dilation_w + offset_w;\n if (inv_h <= -1 || inv_w <= -1 || inv_h >= height || inv_w >= width)\n inv_h = inv_w = -2;\n else\n mval += data_col_ptr[col_pos] *\n dmcn_im2col_bilinear(data_im_ptr + cnt * height * width, width,\n height, width, inv_h, inv_w);\n const T weight = dmcn_get_coordinate_weight(\n inv_h, inv_w, height, width, data_im_ptr + cnt * height * width,\n width, bp_dir);\n val += weight * data_col_ptr[col_pos] * mask;\n cnt += 1;\n }\n // KERNEL_ASSIGN(grad_offset[index], offset_req, val);\n grad_offset[index] = val;\n if (offset_c % 2 == 0)\n // KERNEL_ASSIGN(grad_mask[(((b * deformable_group +\n // deformable_group_index) * kernel_h * kernel_w + offset_c / 2) *\n // height_col + h) * width_col + w], mask_req, mval);\n grad_mask[(((b * deformable_group + deformable_group_index) * kernel_h *\n kernel_w +\n offset_c / 2) *\n height_col +\n h) *\n width_col +\n w] = mval;\n }\n}\n\n#endif // MODULATED_DEFORM_CONV_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/ms_deform_attn_cuda_kernel.cuh", "content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from\n*https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n#ifndef DEFORM_ATTN_CUDA_KERNEL\n#define DEFORM_ATTN_CUDA_KERNEL\n\n#include \"common_cuda_helper.hpp\"\n#include \"pytorch_cuda_helper.hpp\"\n\nconst int CUDA_NUM_THREADS = 1024;\n\ntemplate \n__device__ scalar_t ms_deform_attn_im2col_bilinear(\n const scalar_t *&bottom_data, const int &height, const int &width,\n const int &nheads, const int &channels, const scalar_t &h,\n const scalar_t &w, const int &m, const int &c) {\n const int h_low = floorf(h);\n const int w_low = floorf(w);\n const int h_high = h_low + 1;\n const int w_high = w_low + 1;\n\n const scalar_t lh = h - h_low;\n const scalar_t lw = w - w_low;\n const scalar_t hh = 1 - lh, hw = 1 - lw;\n\n const int w_stride = nheads * channels;\n const int h_stride = width * w_stride;\n const int h_low_ptr_offset = h_low * h_stride;\n const int h_high_ptr_offset = h_low_ptr_offset + h_stride;\n const int w_low_ptr_offset = w_low * w_stride;\n const int w_high_ptr_offset = w_low_ptr_offset + w_stride;\n const int base_ptr = m * channels + c;\n\n scalar_t v1 = 0;\n if (h_low >= 0 && w_low >= 0) {\n const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;\n v1 = bottom_data[ptr1];\n }\n scalar_t v2 = 0;\n if (h_low >= 0 && w_high <= width - 1) {\n const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;\n v2 = bottom_data[ptr2];\n }\n scalar_t v3 = 0;\n if (h_high <= height - 1 && w_low >= 0) {\n const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;\n v3 = bottom_data[ptr3];\n }\n scalar_t v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1) {\n const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;\n v4 = bottom_data[ptr4];\n }\n\n const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n\n const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n return val;\n}\n\ntemplate \n__device__ void ms_deform_attn_col2im_bilinear(\n const scalar_t *&bottom_data, const int &height, const int &width,\n const int &nheads, const int &channels, const scalar_t &h,\n const scalar_t &w, const int &m, const int &c, const scalar_t &top_grad,\n const scalar_t &attn_weight, scalar_t *&grad_value,\n scalar_t *grad_sampling_loc, scalar_t *grad_attn_weight) {\n const int h_low = floorf(h);\n const int w_low = floorf(w);\n const int h_high = h_low + 1;\n const int w_high = w_low + 1;\n\n const scalar_t lh = h - h_low;\n const scalar_t lw = w - w_low;\n const scalar_t hh = 1 - lh, hw = 1 - lw;\n\n const int w_stride = nheads * channels;\n const int h_stride = width * w_stride;\n const int h_low_ptr_offset = h_low * h_stride;\n const int h_high_ptr_offset = h_low_ptr_offset + h_stride;\n const int w_low_ptr_offset = w_low * w_stride;\n const int w_high_ptr_offset = w_low_ptr_offset + w_stride;\n const int base_ptr = m * channels + c;\n\n const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n const scalar_t top_grad_value = top_grad * attn_weight;\n scalar_t grad_h_weight = 0, grad_w_weight = 0;\n\n scalar_t v1 = 0;\n if (h_low >= 0 && w_low >= 0) {\n const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;\n v1 = bottom_data[ptr1];\n grad_h_weight -= hw * v1;\n grad_w_weight -= hh * v1;\n atomicAdd(grad_value + ptr1, w1 * top_grad_value);\n }\n scalar_t v2 = 0;\n if (h_low >= 0 && w_high <= width - 1) {\n const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;\n v2 = bottom_data[ptr2];\n grad_h_weight -= lw * v2;\n grad_w_weight += hh * v2;\n atomicAdd(grad_value + ptr2, w2 * top_grad_value);\n }\n scalar_t v3 = 0;\n if (h_high <= height - 1 && w_low >= 0) {\n const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;\n v3 = bottom_data[ptr3];\n grad_h_weight += hw * v3;\n grad_w_weight -= lh * v3;\n atomicAdd(grad_value + ptr3, w3 * top_grad_value);\n }\n scalar_t v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1) {\n const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;\n v4 = bottom_data[ptr4];\n grad_h_weight += lw * v4;\n grad_w_weight += lh * v4;\n atomicAdd(grad_value + ptr4, w4 * top_grad_value);\n }\n\n const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n *grad_attn_weight = top_grad * val;\n *grad_sampling_loc = width * grad_w_weight * top_grad_value;\n *(grad_sampling_loc + 1) = height * grad_h_weight * top_grad_value;\n}\n\ntemplate \n__device__ void ms_deform_attn_col2im_bilinear_gm(\n const scalar_t *&bottom_data, const int &height, const int &width,\n const int &nheads, const int &channels, const scalar_t &h,\n const scalar_t &w, const int &m, const int &c, const scalar_t &top_grad,\n const scalar_t &attn_weight, scalar_t *&grad_value,\n scalar_t *grad_sampling_loc, scalar_t *grad_attn_weight) {\n const int h_low = floorf(h);\n const int w_low = floorf(w);\n const int h_high = h_low + 1;\n const int w_high = w_low + 1;\n\n const scalar_t lh = h - h_low;\n const scalar_t lw = w - w_low;\n const scalar_t hh = 1 - lh, hw = 1 - lw;\n\n const int w_stride = nheads * channels;\n const int h_stride = width * w_stride;\n const int h_low_ptr_offset = h_low * h_stride;\n const int h_high_ptr_offset = h_low_ptr_offset + h_stride;\n const int w_low_ptr_offset = w_low * w_stride;\n const int w_high_ptr_offset = w_low_ptr_offset + w_stride;\n const int base_ptr = m * channels + c;\n\n const scalar_t w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n const scalar_t top_grad_value = top_grad * attn_weight;\n scalar_t grad_h_weight = 0, grad_w_weight = 0;\n\n scalar_t v1 = 0;\n if (h_low >= 0 && w_low >= 0) {\n const int ptr1 = h_low_ptr_offset + w_low_ptr_offset + base_ptr;\n v1 = bottom_data[ptr1];\n grad_h_weight -= hw * v1;\n grad_w_weight -= hh * v1;\n atomicAdd(grad_value + ptr1, w1 * top_grad_value);\n }\n scalar_t v2 = 0;\n if (h_low >= 0 && w_high <= width - 1) {\n const int ptr2 = h_low_ptr_offset + w_high_ptr_offset + base_ptr;\n v2 = bottom_data[ptr2];\n grad_h_weight -= lw * v2;\n grad_w_weight += hh * v2;\n atomicAdd(grad_value + ptr2, w2 * top_grad_value);\n }\n scalar_t v3 = 0;\n if (h_high <= height - 1 && w_low >= 0) {\n const int ptr3 = h_high_ptr_offset + w_low_ptr_offset + base_ptr;\n v3 = bottom_data[ptr3];\n grad_h_weight += hw * v3;\n grad_w_weight -= lh * v3;\n atomicAdd(grad_value + ptr3, w3 * top_grad_value);\n }\n scalar_t v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1) {\n const int ptr4 = h_high_ptr_offset + w_high_ptr_offset + base_ptr;\n v4 = bottom_data[ptr4];\n grad_h_weight += lw * v4;\n grad_w_weight += lh * v4;\n atomicAdd(grad_value + ptr4, w4 * top_grad_value);\n }\n\n const scalar_t val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n atomicAdd(grad_attn_weight, top_grad * val);\n atomicAdd(grad_sampling_loc, width * grad_w_weight * top_grad_value);\n atomicAdd(grad_sampling_loc + 1, height * grad_h_weight * top_grad_value);\n}\n\ntemplate \n__global__ void ms_deformable_im2col_gpu_kernel(\n const int n, const scalar_t *data_value, const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index, const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight, const int batch_size,\n const int spatial_size, const int num_heads, const int channels,\n const int num_levels, const int num_query, const int num_point,\n scalar_t *data_col) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp;\n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n _temp /= num_query;\n const int b_col = _temp;\n\n scalar_t *data_col_ptr = data_col + index;\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n scalar_t col = 0;\n\n for (int l_col = 0; l_col < num_levels; ++l_col) {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const scalar_t *data_value_ptr =\n data_value +\n (data_value_ptr_init_offset + level_start_id * qid_stride);\n for (int p_col = 0; p_col < num_point; ++p_col) {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w) {\n col += ms_deform_attn_im2col_bilinear(data_value_ptr, spatial_h,\n spatial_w, num_heads, channels,\n h_im, w_im, m_col, c_col) *\n weight;\n }\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n }\n }\n *data_col_ptr = col;\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1(\n const int n, const scalar_t *grad_col, const scalar_t *data_value,\n const int64_t *data_spatial_shapes, const int64_t *data_level_start_index,\n const scalar_t *data_sampling_loc, const scalar_t *data_attn_weight,\n const int batch_size, const int spatial_size, const int num_heads,\n const int channels, const int num_levels, const int num_query,\n const int num_point, scalar_t *grad_value, scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n __shared__ scalar_t cache_grad_sampling_loc[blockSize * 2];\n __shared__ scalar_t cache_grad_attn_weight[blockSize];\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp;\n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col = 0; l_col < num_levels; ++l_col) {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset =\n data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col = 0; p_col < num_point; ++p_col) {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc + (threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc + ((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight + threadIdx.x) = 0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w) {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im,\n w_im, m_col, c_col, top_grad, weight, grad_value_ptr,\n cache_grad_sampling_loc + (threadIdx.x << 1),\n cache_grad_attn_weight + threadIdx.x);\n }\n\n __syncthreads();\n if (tid == 0) {\n scalar_t _grad_w = cache_grad_sampling_loc[0],\n _grad_h = cache_grad_sampling_loc[1],\n _grad_a = cache_grad_attn_weight[0];\n int sid = 2;\n for (unsigned int tid = 1; tid < blockSize; ++tid) {\n _grad_w += cache_grad_sampling_loc[sid];\n _grad_h += cache_grad_sampling_loc[sid + 1];\n _grad_a += cache_grad_attn_weight[tid];\n sid += 2;\n }\n\n *grad_sampling_loc = _grad_w;\n *(grad_sampling_loc + 1) = _grad_h;\n *grad_attn_weight = _grad_a;\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2(\n const int n, const scalar_t *grad_col, const scalar_t *data_value,\n const int64_t *data_spatial_shapes, const int64_t *data_level_start_index,\n const scalar_t *data_sampling_loc, const scalar_t *data_attn_weight,\n const int batch_size, const int spatial_size, const int num_heads,\n const int channels, const int num_levels, const int num_query,\n const int num_point, scalar_t *grad_value, scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n __shared__ scalar_t cache_grad_sampling_loc[blockSize * 2];\n __shared__ scalar_t cache_grad_attn_weight[blockSize];\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp;\n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col = 0; l_col < num_levels; ++l_col) {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset =\n data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col = 0; p_col < num_point; ++p_col) {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc + (threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc + ((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight + threadIdx.x) = 0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w) {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im,\n w_im, m_col, c_col, top_grad, weight, grad_value_ptr,\n cache_grad_sampling_loc + (threadIdx.x << 1),\n cache_grad_attn_weight + threadIdx.x);\n }\n\n __syncthreads();\n\n for (unsigned int s = blockSize / 2; s > 0; s >>= 1) {\n if (tid < s) {\n const unsigned int xid1 = tid << 1;\n const unsigned int xid2 = (tid + s) << 1;\n cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];\n cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];\n cache_grad_sampling_loc[xid1 + 1] +=\n cache_grad_sampling_loc[xid2 + 1];\n }\n __syncthreads();\n }\n\n if (tid == 0) {\n *grad_sampling_loc = cache_grad_sampling_loc[0];\n *(grad_sampling_loc + 1) = cache_grad_sampling_loc[1];\n *grad_attn_weight = cache_grad_attn_weight[0];\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v1(\n const int n, const scalar_t *grad_col, const scalar_t *data_value,\n const int64_t *data_spatial_shapes, const int64_t *data_level_start_index,\n const scalar_t *data_sampling_loc, const scalar_t *data_attn_weight,\n const int batch_size, const int spatial_size, const int num_heads,\n const int channels, const int num_levels, const int num_query,\n const int num_point, scalar_t *grad_value, scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n extern __shared__ int _s[];\n scalar_t *cache_grad_sampling_loc = reinterpret_cast(_s);\n scalar_t *cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp;\n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col = 0; l_col < num_levels; ++l_col) {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset =\n data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col = 0; p_col < num_point; ++p_col) {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc + (threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc + ((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight + threadIdx.x) = 0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w) {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im,\n w_im, m_col, c_col, top_grad, weight, grad_value_ptr,\n cache_grad_sampling_loc + (threadIdx.x << 1),\n cache_grad_attn_weight + threadIdx.x);\n }\n\n __syncthreads();\n if (tid == 0) {\n scalar_t _grad_w = cache_grad_sampling_loc[0],\n _grad_h = cache_grad_sampling_loc[1],\n _grad_a = cache_grad_attn_weight[0];\n int sid = 2;\n for (unsigned int tid = 1; tid < blockDim.x; ++tid) {\n _grad_w += cache_grad_sampling_loc[sid];\n _grad_h += cache_grad_sampling_loc[sid + 1];\n _grad_a += cache_grad_attn_weight[tid];\n sid += 2;\n }\n\n *grad_sampling_loc = _grad_w;\n *(grad_sampling_loc + 1) = _grad_h;\n *grad_attn_weight = _grad_a;\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v2(\n const int n, const scalar_t *grad_col, const scalar_t *data_value,\n const int64_t *data_spatial_shapes, const int64_t *data_level_start_index,\n const scalar_t *data_sampling_loc, const scalar_t *data_attn_weight,\n const int batch_size, const int spatial_size, const int num_heads,\n const int channels, const int num_levels, const int num_query,\n const int num_point, scalar_t *grad_value, scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n extern __shared__ int _s[];\n scalar_t *cache_grad_sampling_loc = reinterpret_cast(_s);\n scalar_t *cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp;\n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col = 0; l_col < num_levels; ++l_col) {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset =\n data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col = 0; p_col < num_point; ++p_col) {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc + (threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc + ((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight + threadIdx.x) = 0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w) {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im,\n w_im, m_col, c_col, top_grad, weight, grad_value_ptr,\n cache_grad_sampling_loc + (threadIdx.x << 1),\n cache_grad_attn_weight + threadIdx.x);\n }\n\n __syncthreads();\n\n for (unsigned int s = blockDim.x / 2, spre = blockDim.x; s > 0;\n s >>= 1, spre >>= 1) {\n if (tid < s) {\n const unsigned int xid1 = tid << 1;\n const unsigned int xid2 = (tid + s) << 1;\n cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];\n cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];\n cache_grad_sampling_loc[xid1 + 1] +=\n cache_grad_sampling_loc[xid2 + 1];\n if (tid + (s << 1) < spre) {\n cache_grad_attn_weight[tid] +=\n cache_grad_attn_weight[tid + (s << 1)];\n cache_grad_sampling_loc[xid1] +=\n cache_grad_sampling_loc[xid2 + (s << 1)];\n cache_grad_sampling_loc[xid1 + 1] +=\n cache_grad_sampling_loc[xid2 + 1 + (s << 1)];\n }\n }\n __syncthreads();\n }\n\n if (tid == 0) {\n *grad_sampling_loc = cache_grad_sampling_loc[0];\n *(grad_sampling_loc + 1) = cache_grad_sampling_loc[1];\n *grad_attn_weight = cache_grad_attn_weight[0];\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_shm_reduce_v2_multi_blocks(\n const int n, const scalar_t *grad_col, const scalar_t *data_value,\n const int64_t *data_spatial_shapes, const int64_t *data_level_start_index,\n const scalar_t *data_sampling_loc, const scalar_t *data_attn_weight,\n const int batch_size, const int spatial_size, const int num_heads,\n const int channels, const int num_levels, const int num_query,\n const int num_point, scalar_t *grad_value, scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n extern __shared__ int _s[];\n scalar_t *cache_grad_sampling_loc = reinterpret_cast(_s);\n scalar_t *cache_grad_attn_weight = cache_grad_sampling_loc + 2 * blockDim.x;\n unsigned int tid = threadIdx.x;\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp;\n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col = 0; l_col < num_levels; ++l_col) {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset =\n data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col = 0; p_col < num_point; ++p_col) {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n *(cache_grad_sampling_loc + (threadIdx.x << 1)) = 0;\n *(cache_grad_sampling_loc + ((threadIdx.x << 1) + 1)) = 0;\n *(cache_grad_attn_weight + threadIdx.x) = 0;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w) {\n ms_deform_attn_col2im_bilinear(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im,\n w_im, m_col, c_col, top_grad, weight, grad_value_ptr,\n cache_grad_sampling_loc + (threadIdx.x << 1),\n cache_grad_attn_weight + threadIdx.x);\n }\n\n __syncthreads();\n\n for (unsigned int s = blockDim.x / 2, spre = blockDim.x; s > 0;\n s >>= 1, spre >>= 1) {\n if (tid < s) {\n const unsigned int xid1 = tid << 1;\n const unsigned int xid2 = (tid + s) << 1;\n cache_grad_attn_weight[tid] += cache_grad_attn_weight[tid + s];\n cache_grad_sampling_loc[xid1] += cache_grad_sampling_loc[xid2];\n cache_grad_sampling_loc[xid1 + 1] +=\n cache_grad_sampling_loc[xid2 + 1];\n if (tid + (s << 1) < spre) {\n cache_grad_attn_weight[tid] +=\n cache_grad_attn_weight[tid + (s << 1)];\n cache_grad_sampling_loc[xid1] +=\n cache_grad_sampling_loc[xid2 + (s << 1)];\n cache_grad_sampling_loc[xid1 + 1] +=\n cache_grad_sampling_loc[xid2 + 1 + (s << 1)];\n }\n }\n __syncthreads();\n }\n\n if (tid == 0) {\n atomicAdd(grad_sampling_loc, cache_grad_sampling_loc[0]);\n atomicAdd(grad_sampling_loc + 1, cache_grad_sampling_loc[1]);\n atomicAdd(grad_attn_weight, cache_grad_attn_weight[0]);\n }\n __syncthreads();\n\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n\ntemplate \n__global__ void ms_deformable_col2im_gpu_kernel_gm(\n const int n, const scalar_t *grad_col, const scalar_t *data_value,\n const int64_t *data_spatial_shapes, const int64_t *data_level_start_index,\n const scalar_t *data_sampling_loc, const scalar_t *data_attn_weight,\n const int batch_size, const int spatial_size, const int num_heads,\n const int channels, const int num_levels, const int num_query,\n const int num_point, scalar_t *grad_value, scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n int _temp = index;\n const int c_col = _temp % channels;\n _temp /= channels;\n const int sampling_index = _temp;\n const int m_col = _temp % num_heads;\n _temp /= num_heads;\n _temp /= num_query;\n const int b_col = _temp;\n\n const scalar_t top_grad = grad_col[index];\n\n int data_weight_ptr = sampling_index * num_levels * num_point;\n int data_loc_w_ptr = data_weight_ptr << 1;\n const int grad_sampling_ptr = data_weight_ptr;\n grad_sampling_loc += grad_sampling_ptr << 1;\n grad_attn_weight += grad_sampling_ptr;\n const int grad_weight_stride = 1;\n const int grad_loc_stride = 2;\n const int qid_stride = num_heads * channels;\n const int data_value_ptr_init_offset = b_col * spatial_size * qid_stride;\n\n for (int l_col = 0; l_col < num_levels; ++l_col) {\n const int level_start_id = data_level_start_index[l_col];\n const int spatial_h_ptr = l_col << 1;\n const int spatial_h = data_spatial_shapes[spatial_h_ptr];\n const int spatial_w = data_spatial_shapes[spatial_h_ptr + 1];\n const int value_ptr_offset =\n data_value_ptr_init_offset + level_start_id * qid_stride;\n const scalar_t *data_value_ptr = data_value + value_ptr_offset;\n scalar_t *grad_value_ptr = grad_value + value_ptr_offset;\n\n for (int p_col = 0; p_col < num_point; ++p_col) {\n const scalar_t loc_w = data_sampling_loc[data_loc_w_ptr];\n const scalar_t loc_h = data_sampling_loc[data_loc_w_ptr + 1];\n const scalar_t weight = data_attn_weight[data_weight_ptr];\n\n const scalar_t h_im = loc_h * spatial_h - 0.5;\n const scalar_t w_im = loc_w * spatial_w - 0.5;\n if (h_im > -1 && w_im > -1 && h_im < spatial_h && w_im < spatial_w) {\n ms_deform_attn_col2im_bilinear_gm(\n data_value_ptr, spatial_h, spatial_w, num_heads, channels, h_im,\n w_im, m_col, c_col, top_grad, weight, grad_value_ptr,\n grad_sampling_loc, grad_attn_weight);\n }\n data_weight_ptr += 1;\n data_loc_w_ptr += 2;\n grad_attn_weight += grad_weight_stride;\n grad_sampling_loc += grad_loc_stride;\n }\n }\n }\n}\n#endif // DEFORM_ATTN_CUDA_KERNEL\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/nms_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef NMS_CUDA_KERNEL_CUH\n#define NMS_CUDA_KERNEL_CUH\n\n#include \n#ifdef MMCV_WITH_TRT\n#include \"common_cuda_helper.hpp\"\n#else // MMCV_WITH_TRT\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else // MMCV_USE_PARROTS\n#include \"pytorch_cuda_helper.hpp\"\n#endif // MMCV_USE_PARROTS\n#endif // MMCV_WITH_TRT\n\nint const threadsPerBlock = sizeof(unsigned long long int) * 8;\n\n__device__ inline bool devIoU(float const *const a, float const *const b,\n const int offset, const float threshold) {\n float left = fmaxf(a[0], b[0]), right = fminf(a[2], b[2]);\n float top = fmaxf(a[1], b[1]), bottom = fminf(a[3], b[3]);\n float width = fmaxf(right - left + offset, 0.f),\n height = fmaxf(bottom - top + offset, 0.f);\n float interS = width * height;\n float Sa = (a[2] - a[0] + offset) * (a[3] - a[1] + offset);\n float Sb = (b[2] - b[0] + offset) * (b[3] - b[1] + offset);\n return interS > threshold * (Sa + Sb - interS);\n}\n\n__global__ void nms_cuda(const int n_boxes, const float iou_threshold,\n const int offset, const float *dev_boxes,\n unsigned long long *dev_mask) {\n int blocks = (n_boxes + threadsPerBlock - 1) / threadsPerBlock;\n CUDA_2D_KERNEL_BLOCK_LOOP(col_start, blocks, row_start, blocks) {\n const int tid = threadIdx.x;\n\n if (row_start > col_start) return;\n\n const int row_size =\n fminf(n_boxes - row_start * threadsPerBlock, threadsPerBlock);\n const int col_size =\n fminf(n_boxes - col_start * threadsPerBlock, threadsPerBlock);\n\n __shared__ float block_boxes[threadsPerBlock * 4];\n if (tid < col_size) {\n block_boxes[tid * 4 + 0] =\n dev_boxes[(threadsPerBlock * col_start + tid) * 4 + 0];\n block_boxes[tid * 4 + 1] =\n dev_boxes[(threadsPerBlock * col_start + tid) * 4 + 1];\n block_boxes[tid * 4 + 2] =\n dev_boxes[(threadsPerBlock * col_start + tid) * 4 + 2];\n block_boxes[tid * 4 + 3] =\n dev_boxes[(threadsPerBlock * col_start + tid) * 4 + 3];\n }\n __syncthreads();\n\n if (tid < row_size) {\n const int cur_box_idx = threadsPerBlock * row_start + tid;\n const float *cur_box = dev_boxes + cur_box_idx * 4;\n int i = 0;\n unsigned long long int t = 0;\n int start = 0;\n if (row_start == col_start) {\n start = tid + 1;\n }\n for (i = start; i < col_size; i++) {\n if (devIoU(cur_box, block_boxes + i * 4, offset, iou_threshold)) {\n t |= 1ULL << i;\n }\n }\n dev_mask[cur_box_idx * gridDim.y + col_start] = t;\n }\n }\n}\n#endif // NMS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/nms_rotated_cuda.cuh", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu\n#ifndef NMS_ROTATED_CUDA_CUH\n#define NMS_ROTATED_CUDA_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n#include \"box_iou_rotated_utils.hpp\"\n\n__host__ __device__ inline int divideUP(const int x, const int y) {\n return (((x) + (y)-1) / (y));\n}\n\nnamespace {\nint const threadsPerBlock = sizeof(unsigned long long) * 8;\n}\n\ntemplate \n__global__ void nms_rotated_cuda_kernel(const int n_boxes,\n const float iou_threshold,\n const T* dev_boxes,\n unsigned long long* dev_mask,\n const int multi_label) {\n // nms_rotated_cuda_kernel is modified from torchvision's nms_cuda_kernel\n\n if (multi_label == 1) {\n const int row_start = blockIdx.y;\n const int col_start = blockIdx.x;\n\n // if (row_start > col_start) return;\n\n const int row_size =\n min(n_boxes - row_start * threadsPerBlock, threadsPerBlock);\n const int col_size =\n min(n_boxes - col_start * threadsPerBlock, threadsPerBlock);\n\n // Compared to nms_cuda_kernel, where each box is represented with 4 values\n // (x1, y1, x2, y2), each rotated box is represented with 5 values\n // (x_center, y_center, width, height, angle_degrees) here.\n __shared__ T block_boxes[threadsPerBlock * 5];\n if (threadIdx.x < col_size) {\n block_boxes[threadIdx.x * 6 + 0] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 6 + 0];\n block_boxes[threadIdx.x * 6 + 1] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 6 + 1];\n block_boxes[threadIdx.x * 6 + 2] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 6 + 2];\n block_boxes[threadIdx.x * 6 + 3] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 6 + 3];\n block_boxes[threadIdx.x * 6 + 4] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 6 + 4];\n block_boxes[threadIdx.x * 6 + 5] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 6 + 5];\n }\n __syncthreads();\n\n if (threadIdx.x < row_size) {\n const int cur_box_idx = threadsPerBlock * row_start + threadIdx.x;\n const T* cur_box = dev_boxes + cur_box_idx * 6;\n int i = 0;\n unsigned long long t = 0;\n int start = 0;\n if (row_start == col_start) {\n start = threadIdx.x + 1;\n }\n for (i = start; i < col_size; i++) {\n // Instead of devIoU used by original horizontal nms, here\n // we use the single_box_iou_rotated function from\n // box_iou_rotated_utils.h\n if (single_box_iou_rotated(cur_box, block_boxes + i * 6, 0) >\n iou_threshold) {\n t |= 1ULL << i;\n }\n }\n const int col_blocks = divideUP(n_boxes, threadsPerBlock);\n dev_mask[cur_box_idx * col_blocks + col_start] = t;\n }\n } else {\n const int row_start = blockIdx.y;\n const int col_start = blockIdx.x;\n\n // if (row_start > col_start) return;\n\n const int row_size =\n min(n_boxes - row_start * threadsPerBlock, threadsPerBlock);\n const int col_size =\n min(n_boxes - col_start * threadsPerBlock, threadsPerBlock);\n\n // Compared to nms_cuda_kernel, where each box is represented with 4 values\n // (x1, y1, x2, y2), each rotated box is represented with 5 values\n // (x_center, y_center, width, height, angle_degrees) here.\n __shared__ T block_boxes[threadsPerBlock * 5];\n if (threadIdx.x < col_size) {\n block_boxes[threadIdx.x * 5 + 0] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 0];\n block_boxes[threadIdx.x * 5 + 1] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 1];\n block_boxes[threadIdx.x * 5 + 2] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 2];\n block_boxes[threadIdx.x * 5 + 3] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 3];\n block_boxes[threadIdx.x * 5 + 4] =\n dev_boxes[(threadsPerBlock * col_start + threadIdx.x) * 5 + 4];\n }\n __syncthreads();\n\n if (threadIdx.x < row_size) {\n const int cur_box_idx = threadsPerBlock * row_start + threadIdx.x;\n const T* cur_box = dev_boxes + cur_box_idx * 5;\n int i = 0;\n unsigned long long t = 0;\n int start = 0;\n if (row_start == col_start) {\n start = threadIdx.x + 1;\n }\n for (i = start; i < col_size; i++) {\n // Instead of devIoU used by original horizontal nms, here\n // we use the single_box_iou_rotated function from\n // box_iou_rotated_utils.h\n if (single_box_iou_rotated(cur_box, block_boxes + i * 5, 0) >\n iou_threshold) {\n t |= 1ULL << i;\n }\n }\n const int col_blocks = divideUP(n_boxes, threadsPerBlock);\n dev_mask[cur_box_idx * col_blocks + col_start] = t;\n }\n }\n}\n\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/parrots_cudawarpfunction.cuh", "content": "/*\n * Copyright (c) 2019, SenseTime.\n */\n\n#ifndef INCLUDE_PARROTS_DARRAY_CUDAWARPFUNCTION_CUH_\n#define INCLUDE_PARROTS_DARRAY_CUDAWARPFUNCTION_CUH_\n\n#ifndef __CUDACC__\n#error cudawarpfunction.cuh should only be included by .cu files\n#endif\n#include \n\n#include \n\n#ifdef PARROTS_USE_HALF\n#include \n#endif\n#ifdef __CUDA_ARCH__\n#define CUDA_INTRINSIC_FUNC(Expr) Expr\n#else\n#define CUDA_INTRINSIC_FUNC(Expr)\n#endif\n\n#if !defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 300\n\n#ifdef PARROTS_USE_HALF\n\n#if CUDA_VERSION < 9000\n\n__device__ inline float16 __shfl(float16 var, int srcLane, int width) {\n CUDA_INTRINSIC_FUNC(return __shfl(var.y, srcLane, width););\n}\n\n__device__ inline float16 __shfl_up(float16 var, unsigned delta, int width) {\n CUDA_INTRINSIC_FUNC(return __shfl_up(var.y, delta, width););\n}\n\n__device__ inline float16 __shfl_down(float16 var, unsigned delta, int width) {\n CUDA_INTRINSIC_FUNC(return __shfl_down(var.y, delta, width););\n}\n\n__device__ inline float16 __shfl_xor(float16 var, int laneMask, int width) {\n CUDA_INTRINSIC_FUNC(return __shfl_xor(var.y, laneMask, width););\n}\n\n#else // CUDA_VERSION >= 9000\n\n__device__ inline float16 __shfl_sync(unsigned mask, float16 var, int srcLane,\n int width = warpSize) {\n CUDA_INTRINSIC_FUNC(float16 r; r.y = __shfl_sync(mask, var.y, srcLane, width);\n return r;);\n}\n\n__device__ inline float16 __shfl_up_sync(unsigned mask, float16 var,\n unsigned delta, int width = warpSize) {\n CUDA_INTRINSIC_FUNC(\n float16 r; r.y = __shfl_up_sync(mask, var.y, delta, width); return r;);\n}\n\n__device__ inline float16 __shfl_down_sync(unsigned mask, float16 var,\n unsigned delta,\n int width = warpSize) {\n CUDA_INTRINSIC_FUNC(\n float16 r; r.y = __shfl_down_sync(mask, var.y, delta, width); return r;);\n}\n\n__device__ inline float16 __shfl_xor_sync(unsigned mask, float16 var,\n int laneMask, int width) {\n CUDA_INTRINSIC_FUNC(float16 r;\n r.y = __shfl_xor_sync(mask, var.y, laneMask, width);\n return r;);\n}\n\n#endif // CUDA_VERSION < 9000\n\n#endif // PARROTS_USE_HALF\n\n// warp shuffle interface with a dummy mask\n#if CUDA_VERSION < 9000\n\ntemplate \n__device__ inline T __shfl_sync(unsigned mask, T var, int srcLane,\n int width = warpSize) {\n CUDA_INTRINSIC_FUNC(return __shfl(var, srcLane, width););\n}\n\ntemplate \n__device__ inline T __shfl_up_sync(unsigned mask, T var, unsigned delta,\n int width = warpSize) {\n CUDA_INTRINSIC_FUNC(return __shfl_up(var, delta, width););\n}\n\ntemplate \n__device__ inline T __shfl_down_sync(unsigned mask, T var, unsigned delta,\n int width = warpSize) {\n CUDA_INTRINSIC_FUNC(return __shfl_down(var, delta, width););\n}\n\ntemplate \n__device__ inline T __shfl_xor_sync(unsigned mask, T var, int laneMask,\n int width = warpSize) {\n CUDA_INTRINSIC_FUNC(return __shfl_xor(var, laneMask, width););\n}\n\n#endif // CUDA_VERSION < 9000\n\n#endif // !defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 300\n\n#endif // INCLUDE_PARROTS_DARRAY_CUDAWARPFUNCTION_CUH_\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/points_in_boxes_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef POINT_IN_BOXES_CUDA_KERNEL_CUH\n#define POINT_IN_BOXES_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__device__ inline void lidar_to_local_coords(T shift_x, T shift_y, T rz,\n T &local_x, T &local_y) {\n T cosa = cos(-rz), sina = sin(-rz);\n local_x = shift_x * cosa + shift_y * (-sina);\n local_y = shift_x * sina + shift_y * cosa;\n}\n\ntemplate \n__device__ inline int check_pt_in_box3d(const T *pt, const T *box3d, T &local_x,\n T &local_y) {\n // param pt: (x, y, z)\n // param box3d: (cx, cy, cz, x_size, y_size, z_size, rz) in LiDAR coordinate,\n // cz in the bottom center\n T x = pt[0], y = pt[1], z = pt[2];\n T cx = box3d[0], cy = box3d[1], cz = box3d[2];\n T x_size = box3d[3], y_size = box3d[4], z_size = box3d[5], rz = box3d[6];\n cz += z_size /\n 2.0; // shift to the center since cz in box3d is the bottom center\n\n if (fabsf(z - cz) > z_size / 2.0) return 0;\n lidar_to_local_coords(x - cx, y - cy, rz, local_x, local_y);\n float in_flag = (local_x > -x_size / 2.0) & (local_x < x_size / 2.0) &\n (local_y > -y_size / 2.0) & (local_y < y_size / 2.0);\n return in_flag;\n}\n\ntemplate \n__global__ void points_in_boxes_part_forward_cuda_kernel(\n int batch_size, int boxes_num, int pts_num, const T *boxes, const T *pts,\n int *box_idx_of_points) {\n // params boxes: (B, N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is the bottom center, each box DO NOT overlaps params pts:\n // (B, npoints, 3) [x, y, z] in LiDAR coordinate params boxes_idx_of_points:\n // (B, npoints), default -1\n\n int bs_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, pts_num) {\n if (bs_idx >= batch_size) return;\n\n boxes += bs_idx * boxes_num * 7;\n pts += bs_idx * pts_num * 3 + pt_idx * 3;\n box_idx_of_points += bs_idx * pts_num + pt_idx;\n\n T local_x = 0, local_y = 0;\n int cur_in_flag = 0;\n for (int k = 0; k < boxes_num; k++) {\n cur_in_flag = check_pt_in_box3d(pts, boxes + k * 7, local_x, local_y);\n if (cur_in_flag) {\n box_idx_of_points[0] = k;\n break;\n }\n }\n }\n}\n\ntemplate \n__global__ void points_in_boxes_all_forward_cuda_kernel(\n int batch_size, int boxes_num, int pts_num, const T *boxes, const T *pts,\n int *box_idx_of_points) {\n // params boxes: (B, N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is the bottom center, each box DO NOT overlaps params pts:\n // (B, npoints, 3) [x, y, z] in LiDAR coordinate params boxes_idx_of_points:\n // (B, npoints), default -1\n\n int bs_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, pts_num) {\n if (bs_idx >= batch_size) return;\n\n boxes += bs_idx * boxes_num * 7;\n pts += bs_idx * pts_num * 3 + pt_idx * 3;\n box_idx_of_points += bs_idx * pts_num * boxes_num + pt_idx * boxes_num;\n\n T local_x = 0, local_y = 0;\n for (int k = 0; k < boxes_num; k++) {\n const int cur_in_flag =\n check_pt_in_box3d(pts, boxes + k * 7, local_x, local_y);\n if (cur_in_flag) {\n box_idx_of_points[k] = 1;\n }\n }\n }\n}\n\n#endif // POINT_IN_BOXES_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/points_in_polygons_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef POINTS_IN_POLYGONS_CUDA_KERNEL_CUH\n#define POINTS_IN_POLYGONS_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\nstruct point {\n float x, y;\n};\n\ntemplate \n__global__ void points_in_polygons_forward_cuda_kernel(\n const int nthreads, const scalar_t *vertex1, const scalar_t *vertex2,\n const int rows, const int cols, scalar_t *inside_flag) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int row = index / cols;\n int col = index % cols;\n\n const scalar_t *offset_vertex1 = vertex1 + row * 2;\n const scalar_t *offset_vertex2 = vertex2 + col * 8;\n\n point point_[1];\n point polygon[4];\n\n point_[0].x = offset_vertex1[0];\n point_[0].y = offset_vertex1[1];\n\n polygon[0].x = offset_vertex2[0];\n polygon[0].y = offset_vertex2[1];\n polygon[1].x = offset_vertex2[2];\n polygon[1].y = offset_vertex2[3];\n polygon[2].x = offset_vertex2[4];\n polygon[2].y = offset_vertex2[5];\n polygon[3].x = offset_vertex2[6];\n polygon[3].y = offset_vertex2[7];\n\n int nCross = 0;\n int i, j;\n float sx, sy, tx, ty, px, py, x;\n for (i = 0, j = 3; i < 4; j = i, i++) {\n sx = polygon[i].x;\n sy = polygon[i].y;\n tx = polygon[j].x;\n ty = polygon[j].y;\n\n px = point_[0].x;\n py = point_[0].y;\n\n if (py < min(sy, ty)) continue;\n if (py > max(sy, ty)) continue;\n\n if ((sx == px && sy == py) || (tx == px && ty == py)) {\n break;\n } else {\n if ((sy < py && ty >= py) || (sy >= py && ty < py)) {\n x = sx + (py - sy) * (tx - sx) / (ty - sy);\n if (x == px) {\n break;\n }\n if (x > px) {\n nCross++;\n }\n }\n }\n }\n if (nCross % 2 == 1) {\n inside_flag[index] = 1.0;\n } else {\n inside_flag[index] = 0.0;\n }\n return;\n }\n}\n\n#endif // POINTS_IN_POLYGONS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/psamask_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef PSAMASK_CUDA_KERNEL_CUH\n#define PSAMASK_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\n// CUDA: grid stride looping\n#ifndef CUDA_KERNEL_LOOP\n#define CUDA_KERNEL_LOOP(i, n) \\\n for (int i = blockIdx.x * blockDim.x + threadIdx.x; i < (n); \\\n i += blockDim.x * gridDim.x)\n#endif\n\ntemplate \n__global__ void psamask_collect_forward_cuda(\n const int nthreads, const int h_feature, const int w_feature,\n const int h_mask, const int w_mask, const int half_h_mask,\n const int half_w_mask, const T* mask_data, T* buffer_data) {\n CUDA_KERNEL_LOOP(index, nthreads) {\n const int w = index % w_feature;\n const int h = (index / w_feature) % h_feature;\n const int n = index / w_feature / h_feature;\n // effective mask region : [hstart, hend) x [wstart, wend) with mask-indexed\n const int hstart = max(0, half_h_mask - h);\n const int hend = min(h_mask, h_feature + half_h_mask - h);\n const int wstart = max(0, half_w_mask - w);\n const int wend = min(w_mask, w_feature + half_w_mask - w);\n // (hidx, widx ) with mask-indexed\n // (hidx + h - half_h_mask, widx + w - half_w_mask) with feature-indexed\n for (int hidx = hstart; hidx < hend; hidx++) {\n for (int widx = wstart; widx < wend; widx++) {\n buffer_data[(n * h_feature * w_feature +\n (hidx + h - half_h_mask) * w_feature +\n (widx + w - half_w_mask)) *\n h_feature * w_feature +\n h * w_feature + w] = mask_data\n [((n * h_mask * w_mask + hidx * w_mask + widx) * h_feature + h) *\n w_feature +\n w];\n }\n }\n }\n}\n\ntemplate \n__global__ void psamask_distribute_forward_cuda(\n const int nthreads, const int h_feature, const int w_feature,\n const int h_mask, const int w_mask, const int half_h_mask,\n const int half_w_mask, const T* mask_data, T* buffer_data) {\n CUDA_KERNEL_LOOP(index, nthreads) {\n const int w = index % w_feature;\n const int h = (index / w_feature) % h_feature;\n const int n = index / w_feature / h_feature;\n // effective mask region : [hstart, hend) x [wstart, wend) with mask-indexed\n const int hstart = max(0, half_h_mask - h);\n const int hend = min(h_mask, h_feature + half_h_mask - h);\n const int wstart = max(0, half_w_mask - w);\n const int wend = min(w_mask, w_feature + half_w_mask - w);\n // (hidx, widx ) with mask-indexed\n // (hidx + h - half_h_mask, widx + w - half_w_mask) with feature-indexed\n for (int hidx = hstart; hidx < hend; hidx++) {\n for (int widx = wstart; widx < wend; widx++) {\n buffer_data[(n * h_feature * w_feature + h * w_feature + w) *\n h_feature * w_feature +\n (hidx + h - half_h_mask) * w_feature +\n (widx + w - half_w_mask)] = mask_data\n [((n * h_mask * w_mask + hidx * w_mask + widx) * h_feature + h) *\n w_feature +\n w];\n }\n }\n }\n}\n\ntemplate \n__global__ void psamask_collect_backward_cuda(\n const int nthreads, const int h_feature, const int w_feature,\n const int h_mask, const int w_mask, const int half_h_mask,\n const int half_w_mask, const T* buffer_diff, T* mask_diff) {\n CUDA_KERNEL_LOOP(index, nthreads) {\n const int w = index % w_feature;\n const int h = (index / w_feature) % h_feature;\n const int n = index / w_feature / h_feature;\n // effective mask region : [hstart, hend) x [wstart, wend) with mask-indexed\n const int hstart = max(0, half_h_mask - h);\n const int hend = min(h_mask, h_feature + half_h_mask - h);\n const int wstart = max(0, half_w_mask - w);\n const int wend = min(w_mask, w_feature + half_w_mask - w);\n // (hidx, widx ) with mask-indexed\n // (hidx + h - half_h_mask, widx + w - half_w_mask) with feature-indexed\n for (int hidx = hstart; hidx < hend; hidx++) {\n for (int widx = wstart; widx < wend; widx++) {\n mask_diff[((n * h_mask * w_mask + hidx * w_mask + widx) * h_feature +\n h) *\n w_feature +\n w] = buffer_diff[(n * h_feature * w_feature +\n (hidx + h - half_h_mask) * w_feature +\n (widx + w - half_w_mask)) *\n h_feature * w_feature +\n h * w_feature + w];\n }\n }\n }\n}\n\ntemplate \n__global__ void psamask_distribute_backward_cuda(\n const int nthreads, const int h_feature, const int w_feature,\n const int h_mask, const int w_mask, const int half_h_mask,\n const int half_w_mask, const T* buffer_diff, T* mask_diff) {\n CUDA_KERNEL_LOOP(index, nthreads) {\n const int w = index % w_feature;\n const int h = (index / w_feature) % h_feature;\n const int n = index / w_feature / h_feature;\n // effective mask region : [hstart, hend) x [wstart, wend) with mask-indexed\n const int hstart = max(0, half_h_mask - h);\n const int hend = min(h_mask, h_feature + half_h_mask - h);\n const int wstart = max(0, half_w_mask - w);\n const int wend = min(w_mask, w_feature + half_w_mask - w);\n // (hidx, widx ) with mask-indexed\n // (hidx + h - half_h_mask, widx + w - half_w_mask) with feature-indexed\n for (int hidx = hstart; hidx < hend; hidx++) {\n for (int widx = wstart; widx < wend; widx++) {\n mask_diff[((n * h_mask * w_mask + hidx * w_mask + widx) * h_feature +\n h) *\n w_feature +\n w] =\n buffer_diff[(n * h_feature * w_feature + h * w_feature + w) *\n h_feature * w_feature +\n (hidx + h - half_h_mask) * w_feature +\n (widx + w - half_w_mask)];\n }\n }\n }\n}\n\n#endif // PSAMASK_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/riroi_align_rotated_cuda_kernel.cuh", "content": "// Modified from\n// https://github.com/csuhan/ReDet/blob/master/mmdet/ops/riroi_align/src/riroi_align_kernel.cu\n#ifndef RIROI_ALIGN_ROTATED_CUDA_KERNEL_CUH\n#define RIROI_ALIGN_ROTATED_CUDA_KERNEL_CUH\n\n#include \n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else // MMCV_USE_PARROTS\n#include \"pytorch_cuda_helper.hpp\"\n#endif // MMCV_USE_PARROTS\n\n/*** Forward ***/\ntemplate \n__global__ void riroi_align_rotated_forward_cuda_kernel(\n const int nthreads, const scalar_t *bottom_data,\n const scalar_t *bottom_rois, const scalar_t spatial_scale,\n const int num_samples, const bool clockwise, const int channels,\n const int height, const int width, const int pooled_height,\n const int pooled_width, const int num_orientations, scalar_t *top_data) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int o = (index / pooled_width / pooled_height) % num_orientations;\n int c =\n (index / pooled_width / pooled_height / num_orientations) % channels;\n int n = index / pooled_width / pooled_height / num_orientations / channels;\n\n const scalar_t *offset_bottom_rois = bottom_rois + n * 6;\n int roi_batch_ind = offset_bottom_rois[0];\n\n // Do not using rounding; this implementation detail is critical\n scalar_t roi_center_w = offset_bottom_rois[1] * spatial_scale;\n scalar_t roi_center_h = offset_bottom_rois[2] * spatial_scale;\n scalar_t roi_width = offset_bottom_rois[3] * spatial_scale;\n scalar_t roi_height = offset_bottom_rois[4] * spatial_scale;\n // scalar_t theta = offset_bottom_rois[5] * M_PI / 180.0;\n scalar_t theta = offset_bottom_rois[5];\n // Force malformed ROIs to be 1x1\n roi_width = max(roi_width, (scalar_t)1.);\n roi_height = max(roi_height, (scalar_t)1.);\n scalar_t bin_size_h = static_cast(roi_height) /\n static_cast(pooled_height);\n scalar_t bin_size_w =\n static_cast(roi_width) / static_cast(pooled_width);\n\n // find aligned index\n scalar_t ind_float = theta * num_orientations / (2 * M_PI);\n int ind = floorf(ind_float);\n scalar_t l_var = ind_float - (scalar_t)ind;\n scalar_t r_var = 1.0 - l_var;\n // correct start channel\n ind = (ind + num_orientations) % num_orientations;\n // rotated channel\n int ind_rot = (o - ind + num_orientations) % num_orientations;\n int ind_rot_plus = (ind_rot + 1 + num_orientations) % num_orientations;\n const scalar_t *offset_bottom_data =\n bottom_data + (roi_batch_ind * channels * num_orientations +\n c * num_orientations + ind_rot) *\n height * width;\n\n const scalar_t *offset_bottom_data_plus =\n bottom_data + (roi_batch_ind * channels * num_orientations +\n c * num_orientations + ind_rot_plus) *\n height * width;\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (num_samples > 0)\n ? num_samples\n : ceilf(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (num_samples > 0) ? num_samples : ceilf(roi_width / pooled_width);\n\n // roi_start_h and roi_start_w are computed wrt the center of RoI (x, y).\n // Appropriate translation needs to be applied after.\n if (clockwise) {\n theta = -theta; // If clockwise, the angle needs to be reversed.\n }\n scalar_t roi_start_h = -roi_height / 2.0;\n scalar_t roi_start_w = -roi_width / 2.0;\n scalar_t cosscalar_theta = cos(theta);\n scalar_t sinscalar_theta = sin(theta);\n\n // We do average (integral) pooling inside a bin\n const scalar_t count = max(roi_bin_grid_h * roi_bin_grid_w, 1); // e.g. = 4\n\n scalar_t output_val = 0.;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) { // e.g., iy = 0, 1\n const scalar_t yy =\n roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const scalar_t xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n // Rotate by theta (counterclockwise) around the center and translate\n scalar_t y = yy * cosscalar_theta - xx * sinscalar_theta + roi_center_h;\n scalar_t x = yy * sinscalar_theta + xx * cosscalar_theta + roi_center_w;\n\n scalar_t val = bilinear_interpolate(\n offset_bottom_data, height, width, y, x, index);\n scalar_t val_plus = bilinear_interpolate(\n offset_bottom_data_plus, height, width, y, x, index);\n output_val += r_var * val + l_var * val_plus;\n }\n }\n output_val /= count;\n\n top_data[index] = output_val;\n }\n}\n\n/*** Backward ***/\ntemplate \n__global__ void riroi_align_rotated_backward_cuda_kernel(\n const int nthreads, const scalar_t *top_diff, const scalar_t *bottom_rois,\n const scalar_t spatial_scale, const int num_samples, const bool clockwise,\n const int channels, const int height, const int width,\n const int pooled_height, const int pooled_width, const int num_orientations,\n scalar_t *bottom_diff) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int o = (index / pooled_width / pooled_height) % num_orientations;\n int c =\n (index / pooled_width / pooled_height / num_orientations) % channels;\n int n = index / pooled_width / pooled_height / num_orientations / channels;\n\n const scalar_t *offset_bottom_rois = bottom_rois + n * 6;\n int roi_batch_ind = offset_bottom_rois[0];\n\n // Do not round\n scalar_t roi_center_w = offset_bottom_rois[1] * spatial_scale;\n scalar_t roi_center_h = offset_bottom_rois[2] * spatial_scale;\n scalar_t roi_width = offset_bottom_rois[3] * spatial_scale;\n scalar_t roi_height = offset_bottom_rois[4] * spatial_scale;\n // scalar_t theta = offset_bottom_rois[5] * M_PI / 180.0;\n scalar_t theta = offset_bottom_rois[5];\n // Force malformed ROIs to be 1x1\n roi_width = max(roi_width, (scalar_t)1.);\n roi_height = max(roi_height, (scalar_t)1.);\n\n scalar_t bin_size_h = static_cast(roi_height) /\n static_cast(pooled_height);\n scalar_t bin_size_w =\n static_cast(roi_width) / static_cast(pooled_width);\n\n // find aligned index\n scalar_t ind_float = theta * num_orientations / (2 * M_PI);\n int ind = floorf(ind_float);\n scalar_t l_var = ind_float - (scalar_t)ind;\n scalar_t r_var = 1.0 - l_var;\n // correct start channel\n ind = (ind + num_orientations) % num_orientations;\n // rotated channel\n int ind_rot = (o - ind + num_orientations) % num_orientations;\n int ind_rot_plus = (ind_rot + 1 + num_orientations) % num_orientations;\n scalar_t *offset_bottom_diff =\n bottom_diff + (roi_batch_ind * channels * num_orientations +\n c * num_orientations + ind_rot) *\n height * width;\n scalar_t *offset_bottom_diff_plus =\n bottom_diff + (roi_batch_ind * channels * num_orientations +\n c * num_orientations + ind_rot_plus) *\n height * width;\n int top_offset =\n (n * channels * num_orientations + c * num_orientations + o) *\n pooled_height * pooled_width;\n const scalar_t *offset_top_diff = top_diff + top_offset;\n const scalar_t top_diff_this_bin = offset_top_diff[ph * pooled_width + pw];\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (num_samples > 0)\n ? num_samples\n : ceilf(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (num_samples > 0) ? num_samples : ceilf(roi_width / pooled_width);\n\n // roi_start_h and roi_start_w are computed wrt the center of RoI (x, y).\n // Appropriate translation needs to be applied after.\n if (clockwise) {\n theta = -theta; // If clockwise, the angle needs to be reversed.\n }\n scalar_t roi_start_h = -roi_height / 2.0;\n scalar_t roi_start_w = -roi_width / 2.0;\n scalar_t cosTheta = cos(theta);\n scalar_t sinTheta = sin(theta);\n\n // We do average (integral) pooling inside a bin\n const scalar_t count = roi_bin_grid_h * roi_bin_grid_w; // e.g. = 4\n\n for (int iy = 0; iy < roi_bin_grid_h; iy++) { // e.g., iy = 0, 1\n const scalar_t yy =\n roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const scalar_t xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n // Rotate by theta around the center and translate\n scalar_t y = yy * cosTheta - xx * sinTheta + roi_center_h;\n scalar_t x = yy * sinTheta + xx * cosTheta + roi_center_w;\n\n scalar_t w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3,\n w4, x_low, x_high, y_low,\n y_high, index);\n\n scalar_t g1 = top_diff_this_bin * w1 / count;\n scalar_t g2 = top_diff_this_bin * w2 / count;\n scalar_t g3 = top_diff_this_bin * w3 / count;\n scalar_t g4 = top_diff_this_bin * w4 / count;\n\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n atomicAdd(offset_bottom_diff + y_low * width + x_low, g1 * r_var);\n atomicAdd(offset_bottom_diff + y_low * width + x_high, g2 * r_var);\n atomicAdd(offset_bottom_diff + y_high * width + x_low, g3 * r_var);\n atomicAdd(offset_bottom_diff + y_high * width + x_high, g4 * r_var);\n\n atomicAdd(offset_bottom_diff_plus + y_low * width + x_low,\n g1 * l_var);\n atomicAdd(offset_bottom_diff_plus + y_low * width + x_high,\n g2 * l_var);\n atomicAdd(offset_bottom_diff_plus + y_high * width + x_low,\n g3 * l_var);\n atomicAdd(offset_bottom_diff_plus + y_high * width + x_high,\n g4 * l_var);\n\n } // if\n } // ix\n } // iy\n } // CUDA_1D_KERNEL_LOOP\n} // RiRoIAlignBackward\n\n#endif // RIROI_ALIGN_ROTATED_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/roi_align_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROI_ALIGN_CUDA_KERNEL_CUH\n#define ROI_ALIGN_CUDA_KERNEL_CUH\n\n#include \n#ifdef MMCV_WITH_TRT\n#include \"common_cuda_helper.hpp\"\n#else // MMCV_WITH_TRT\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else // MMCV_USE_PARROTS\n#include \"pytorch_cuda_helper.hpp\"\n#endif // MMCV_USE_PARROTS\n#endif // MMCV_WITH_TRT\n\n/*** Forward ***/\ntemplate \n__global__ void roi_align_forward_cuda_kernel(\n const int nthreads, const T* input, const T* rois, T* output, T* argmax_y,\n T* argmax_x, const int pooled_height, const int pooled_width,\n const T spatial_scale, const int sampling_ratio,\n const int pool_mode, // 0 - max pool, 1 - avg pool\n const bool aligned, const int channels, const int height, const int width) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const T* offset_rois = rois + n * 5;\n int roi_batch_ind = offset_rois[0];\n\n // Do not using rounding; this implementation detail is critical\n T offset = aligned ? (T)0.5 : (T)0.0;\n T roi_start_w = offset_rois[1] * spatial_scale - offset;\n T roi_start_h = offset_rois[2] * spatial_scale - offset;\n T roi_end_w = offset_rois[3] * spatial_scale - offset;\n T roi_end_h = offset_rois[4] * spatial_scale - offset;\n\n T roi_width = roi_end_w - roi_start_w;\n T roi_height = roi_end_h - roi_start_h;\n if (!aligned) { // for backward-compatibility only\n roi_width = max(roi_width, (T)1.);\n roi_height = max(roi_height, (T)1.);\n }\n\n T bin_size_h = static_cast(roi_height) / static_cast(pooled_height);\n T bin_size_w = static_cast(roi_width) / static_cast(pooled_width);\n\n const T* offset_input =\n input + (roi_batch_ind * channels + c) * height * width;\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h =\n (sampling_ratio > 0)\n ? sampling_ratio\n : static_cast(ceilf(roi_height / pooled_height));\n int roi_bin_grid_w =\n (sampling_ratio > 0)\n ? sampling_ratio\n : static_cast(ceilf(roi_width / pooled_width));\n\n if (pool_mode == 0) {\n // We do max pooling inside a bin\n T maxval = -FLT_MAX;\n T maxidx_y = -1.f, maxidx_x = -1.f;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const T y = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h);\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const T x = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n T val =\n bilinear_interpolate(offset_input, height, width, y, x, index);\n if (val > maxval) {\n maxval = val;\n maxidx_y = y;\n maxidx_x = x;\n }\n }\n }\n output[index] = maxval;\n argmax_y[index] = maxidx_y;\n argmax_x[index] = maxidx_x;\n } else if (pool_mode == 1) {\n // We do average pooling inside a bin\n const T count = max(roi_bin_grid_h * roi_bin_grid_w, 1);\n T output_val = 0.;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const T y = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h);\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const T x = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n T val =\n bilinear_interpolate(offset_input, height, width, y, x, index);\n output_val += val;\n }\n }\n output[index] = output_val / count;\n }\n }\n}\n\n/*** Backward ***/\ntemplate \n__global__ void roi_align_backward_cuda_kernel(\n const int nthreads, const T* grad_output, const T* rois, const T* argmax_y,\n const T* argmax_x, T* grad_input, const int pooled_height,\n const int pooled_width, const T spatial_scale, const int sampling_ratio,\n const int pool_mode, // 0 - max pool, 1 - avg pool\n const bool aligned, const int channels, const int height, const int width) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const T grad_output_this_bin = grad_output[index];\n\n const T* offset_rois = rois + n * 5;\n int roi_batch_ind = offset_rois[0];\n T* offset_grad_input =\n grad_input + ((roi_batch_ind * channels + c) * height * width);\n\n if (pool_mode == 0) {\n T y = argmax_y[index], x = argmax_x[index];\n if (y != -1.f) {\n T w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3, w4,\n x_low, x_high, y_low, y_high, index);\n\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n atomicAdd(offset_grad_input + y_low * width + x_low,\n grad_output_this_bin * w1);\n atomicAdd(offset_grad_input + y_low * width + x_high,\n grad_output_this_bin * w2);\n atomicAdd(offset_grad_input + y_high * width + x_low,\n grad_output_this_bin * w3);\n atomicAdd(offset_grad_input + y_high * width + x_high,\n grad_output_this_bin * w4);\n }\n }\n } else if (pool_mode == 1) {\n // Do not using rounding; this implementation detail is critical\n T offset = aligned ? (T)0.5 : (T)0.0;\n T roi_start_w = offset_rois[1] * spatial_scale - offset;\n T roi_start_h = offset_rois[2] * spatial_scale - offset;\n T roi_end_w = offset_rois[3] * spatial_scale - offset;\n T roi_end_h = offset_rois[4] * spatial_scale - offset;\n\n T roi_width = roi_end_w - roi_start_w;\n T roi_height = roi_end_h - roi_start_h;\n if (!aligned) { // for backward-compatibility only\n roi_width = max(roi_width, (T)1.);\n roi_height = max(roi_height, (T)1.);\n }\n\n T bin_size_h = static_cast(roi_height) / static_cast(pooled_height);\n T bin_size_w = static_cast(roi_width) / static_cast(pooled_width);\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h =\n (sampling_ratio > 0)\n ? sampling_ratio\n : static_cast(ceilf(roi_height / pooled_height));\n int roi_bin_grid_w =\n (sampling_ratio > 0)\n ? sampling_ratio\n : static_cast(ceilf(roi_width / pooled_width));\n\n // We do average (integral) pooling inside a bin\n const T count = roi_bin_grid_h * roi_bin_grid_w; // e.g. = 4\n\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const T y = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h);\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const T x = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n T w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3, w4,\n x_low, x_high, y_low, y_high, index);\n\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n atomicAdd(offset_grad_input + y_low * width + x_low,\n grad_output_this_bin * w1 / count);\n atomicAdd(offset_grad_input + y_low * width + x_high,\n grad_output_this_bin * w2 / count);\n atomicAdd(offset_grad_input + y_high * width + x_low,\n grad_output_this_bin * w3 / count);\n atomicAdd(offset_grad_input + y_high * width + x_high,\n grad_output_this_bin * w4 / count);\n }\n }\n }\n }\n }\n}\n\n#endif // ROI_ALIGN_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/roi_align_rotated_cuda_kernel.cuh", "content": "// Modified from\n// https://github.com/facebookresearch/detectron2/tree/master/detectron2/layers/csrc/ROIAlignRotated\n// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n#ifndef ROI_ALIGN_ROTATED_CUDA_KERNEL_CUH\n#define ROI_ALIGN_ROTATED_CUDA_KERNEL_CUH\n\n#include \n#ifdef MMCV_WITH_TRT\n#include \"common_cuda_helper.hpp\"\n#else // MMCV_WITH_TRT\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else // MMCV_USE_PARROTS\n#include \"pytorch_cuda_helper.hpp\"\n#endif // MMCV_USE_PARROTS\n#endif // MMCV_WITH_TRT\n\n/*** Forward ***/\ntemplate \n__global__ void roi_align_rotated_forward_cuda_kernel(\n const int nthreads, const scalar_t *bottom_data,\n const scalar_t *bottom_rois, const scalar_t spatial_scale,\n const int sample_num, const bool aligned, const bool clockwise,\n const int channels, const int height, const int width,\n const int pooled_height, const int pooled_width, scalar_t *top_data) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const scalar_t *offset_bottom_rois = bottom_rois + n * 6;\n int roi_batch_ind = offset_bottom_rois[0];\n\n // Do not using rounding; this implementation detail is critical\n scalar_t offset = aligned ? (scalar_t)0.5 : (scalar_t)0.0;\n scalar_t roi_center_w = offset_bottom_rois[1] * spatial_scale - offset;\n scalar_t roi_center_h = offset_bottom_rois[2] * spatial_scale - offset;\n scalar_t roi_width = offset_bottom_rois[3] * spatial_scale;\n scalar_t roi_height = offset_bottom_rois[4] * spatial_scale;\n // scalar_t theta = offset_bottom_rois[5] * M_PI / 180.0;\n scalar_t theta = offset_bottom_rois[5];\n if (clockwise) {\n theta = -theta; // If clockwise, the angle needs to be reversed.\n }\n if (!aligned) { // for backward-compatibility only\n // Force malformed ROIs to be 1x1\n roi_width = max(roi_width, (scalar_t)1.);\n roi_height = max(roi_height, (scalar_t)1.);\n }\n scalar_t bin_size_h = static_cast(roi_height) /\n static_cast(pooled_height);\n scalar_t bin_size_w =\n static_cast(roi_width) / static_cast(pooled_width);\n\n const scalar_t *offset_bottom_data =\n bottom_data + (roi_batch_ind * channels + c) * height * width;\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (sample_num > 0)\n ? sample_num\n : ceilf(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (sample_num > 0) ? sample_num : ceilf(roi_width / pooled_width);\n\n // roi_start_h and roi_start_w are computed wrt the center of RoI (x, y).\n // Appropriate translation needs to be applied after.\n scalar_t roi_start_h = -roi_height / 2.0;\n scalar_t roi_start_w = -roi_width / 2.0;\n scalar_t cosscalar_theta = cos(theta);\n scalar_t sinscalar_theta = sin(theta);\n\n // We do average (integral) pooling inside a bin\n const scalar_t count = max(roi_bin_grid_h * roi_bin_grid_w, 1); // e.g. = 4\n\n scalar_t output_val = 0.;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) { // e.g., iy = 0, 1\n const scalar_t yy =\n roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const scalar_t xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n // Rotate by theta (counterclockwise) around the center and translate\n scalar_t y = yy * cosscalar_theta - xx * sinscalar_theta + roi_center_h;\n scalar_t x = yy * sinscalar_theta + xx * cosscalar_theta + roi_center_w;\n\n scalar_t val = bilinear_interpolate(\n offset_bottom_data, height, width, y, x, index);\n output_val += val;\n }\n }\n output_val /= count;\n\n top_data[index] = output_val;\n }\n}\n\n/*** Backward ***/\ntemplate \n__global__ void roi_align_rotated_backward_cuda_kernel(\n const int nthreads, const scalar_t *top_diff, const scalar_t *bottom_rois,\n const scalar_t spatial_scale, const int sample_num, const bool aligned,\n const bool clockwise, const int channels, const int height, const int width,\n const int pooled_height, const int pooled_width, scalar_t *bottom_diff) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const scalar_t *offset_bottom_rois = bottom_rois + n * 6;\n int roi_batch_ind = offset_bottom_rois[0];\n\n // Do not round\n scalar_t offset = aligned ? (scalar_t)0.5 : (scalar_t)0.0;\n scalar_t roi_center_w = offset_bottom_rois[1] * spatial_scale - offset;\n scalar_t roi_center_h = offset_bottom_rois[2] * spatial_scale - offset;\n scalar_t roi_width = offset_bottom_rois[3] * spatial_scale;\n scalar_t roi_height = offset_bottom_rois[4] * spatial_scale;\n // scalar_t theta = offset_bottom_rois[5] * M_PI / 180.0;\n scalar_t theta = offset_bottom_rois[5];\n if (clockwise) {\n theta = -theta; // If clockwise, the angle needs to be reversed.\n }\n if (!aligned) { // for backward-compatibility only\n // Force malformed ROIs to be 1x1\n roi_width = max(roi_width, (scalar_t)1.);\n roi_height = max(roi_height, (scalar_t)1.);\n }\n scalar_t bin_size_h = static_cast(roi_height) /\n static_cast(pooled_height);\n scalar_t bin_size_w =\n static_cast(roi_width) / static_cast(pooled_width);\n\n scalar_t *offset_bottom_diff =\n bottom_diff + (roi_batch_ind * channels + c) * height * width;\n\n int top_offset = (n * channels + c) * pooled_height * pooled_width;\n const scalar_t *offset_top_diff = top_diff + top_offset;\n const scalar_t top_diff_this_bin = offset_top_diff[ph * pooled_width + pw];\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (sample_num > 0)\n ? sample_num\n : ceilf(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (sample_num > 0) ? sample_num : ceilf(roi_width / pooled_width);\n\n // roi_start_h and roi_start_w are computed wrt the center of RoI (x, y).\n // Appropriate translation needs to be applied after.\n scalar_t roi_start_h = -roi_height / 2.0;\n scalar_t roi_start_w = -roi_width / 2.0;\n scalar_t cosTheta = cos(theta);\n scalar_t sinTheta = sin(theta);\n\n // We do average (integral) pooling inside a bin\n const scalar_t count = roi_bin_grid_h * roi_bin_grid_w; // e.g. = 4\n\n for (int iy = 0; iy < roi_bin_grid_h; iy++) { // e.g., iy = 0, 1\n const scalar_t yy =\n roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const scalar_t xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n // Rotate by theta around the center and translate\n scalar_t y = yy * cosTheta - xx * sinTheta + roi_center_h;\n scalar_t x = yy * sinTheta + xx * cosTheta + roi_center_w;\n\n scalar_t w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3,\n w4, x_low, x_high, y_low,\n y_high, index);\n\n scalar_t g1 = top_diff_this_bin * w1 / count;\n scalar_t g2 = top_diff_this_bin * w2 / count;\n scalar_t g3 = top_diff_this_bin * w3 / count;\n scalar_t g4 = top_diff_this_bin * w4 / count;\n\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n atomicAdd(offset_bottom_diff + y_low * width + x_low, g1);\n atomicAdd(offset_bottom_diff + y_low * width + x_high, g2);\n atomicAdd(offset_bottom_diff + y_high * width + x_low, g3);\n atomicAdd(offset_bottom_diff + y_high * width + x_high, g4);\n } // if\n } // ix\n } // iy\n } // CUDA_1D_KERNEL_LOOP\n} // RoIAlignBackward\n\n#endif // ROI_ALIGN_ROTATED_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/roi_pool_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROI_POOL_CUDA_KERNEL_CUH\n#define ROI_POOL_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void roi_pool_forward_cuda_kernel(\n const int nthreads, const T* input, const T* rois, T* output, int* argmax,\n const int pooled_height, const int pooled_width, const T spatial_scale,\n const int channels, const int height, const int width) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const T* offset_rois = rois + n * 5;\n int roi_batch_ind = offset_rois[0];\n // calculate the roi region on feature maps\n T roi_x1 = offset_rois[1] * spatial_scale;\n T roi_y1 = offset_rois[2] * spatial_scale;\n T roi_x2 = (offset_rois[3] + 1) * spatial_scale;\n T roi_y2 = (offset_rois[4] + 1) * spatial_scale;\n\n // force malformed rois to be 1x1\n T roi_w = roi_x2 - roi_x1;\n T roi_h = roi_y2 - roi_y1;\n if (roi_w <= 0 || roi_h <= 0) continue;\n\n T bin_size_w = roi_w / static_cast(pooled_width);\n T bin_size_h = roi_h / static_cast(pooled_height);\n\n // the corresponding bin region\n int bin_x1 = floorf(static_cast(pw) * bin_size_w + roi_x1);\n int bin_y1 = floorf(static_cast(ph) * bin_size_h + roi_y1);\n int bin_x2 = ceilf(static_cast(pw + 1) * bin_size_w + roi_x1);\n int bin_y2 = ceilf(static_cast(ph + 1) * bin_size_h + roi_y1);\n\n // add roi offsets and clip to input boundaries\n bin_x1 = min(max(bin_x1, 0), width);\n bin_y1 = min(max(bin_y1, 0), height);\n bin_x2 = min(max(bin_x2, 0), width);\n bin_y2 = min(max(bin_y2, 0), height);\n bool is_empty = (bin_y2 <= bin_y1) || (bin_x2 <= bin_x1);\n\n const T* offset_input =\n input + (roi_batch_ind * channels + c) * height * width;\n // Define an empty pooling region to be zero\n // If nothing is pooled, argmax = -1 causes nothing to be backprop'd\n T max_val = is_empty ? 0 : -FLT_MAX;\n int max_idx = -1;\n for (int h = bin_y1; h < bin_y2; ++h) {\n for (int w = bin_x1; w < bin_x2; ++w) {\n int offset = h * width + w;\n if (offset_input[offset] > max_val) {\n max_val = offset_input[offset];\n max_idx = offset;\n }\n }\n }\n output[index] = max_val;\n if (argmax != NULL) argmax[index] = max_idx;\n }\n}\n\ntemplate \n__global__ void roi_pool_backward_cuda_kernel(\n const int nthreads, const T* grad_output, const T* rois, const int* argmax,\n T* grad_input, const int pooled_height, const int pooled_width,\n const int channels, const int height, const int width) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n // (n, c) is an element in the pooled output\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n int roi_batch_ind = rois[n * 5];\n T* grad_input_offset =\n grad_input + ((roi_batch_ind * channels + c) * height * width);\n int argmax_index = argmax[index];\n\n if (argmax_index != -1) {\n atomicAdd(grad_input_offset + argmax_index, grad_output[index]);\n }\n }\n}\n\n#endif // ROI_POOL_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/roiaware_pool3d_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROIAWARE_POOL3D_CUDA_KERNEL_CUH\n#define ROIAWARE_POOL3D_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__device__ inline void lidar_to_local_coords(T shift_x, T shift_y, T rz,\n T &local_x, T &local_y) {\n T cosa = cos(-rz), sina = sin(-rz);\n local_x = shift_x * cosa + shift_y * (-sina);\n local_y = shift_x * sina + shift_y * cosa;\n}\n\ntemplate \n__device__ inline int check_pt_in_box3d(const T *pt, const T *box3d, T &local_x,\n T &local_y) {\n // param pt: (x, y, z)\n // param box3d: (cx, cy, cz, x_size, y_size, z_size, rz) in LiDAR coordinate,\n // cz in the bottom center\n T x = pt[0], y = pt[1], z = pt[2];\n T cx = box3d[0], cy = box3d[1], cz = box3d[2];\n T x_size = box3d[3], y_size = box3d[4], z_size = box3d[5], rz = box3d[6];\n cz += z_size /\n 2.0; // shift to the center since cz in box3d is the bottom center\n\n if (fabsf(z - cz) > z_size / 2.0) return 0;\n lidar_to_local_coords(x - cx, y - cy, rz, local_x, local_y);\n float in_flag = (local_x > -x_size / 2.0) & (local_x < x_size / 2.0) &\n (local_y > -y_size / 2.0) & (local_y < y_size / 2.0);\n return in_flag;\n}\n\ntemplate \n__global__ void generate_pts_mask_for_box3d(int boxes_num, int pts_num,\n int out_x, int out_y, int out_z,\n const T *rois, const T *pts,\n int *pts_mask) {\n // params rois: (N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate params pts: (npoints, 3) [x, y, z] params pts_mask: (N,\n // npoints): -1 means point does not in this box, otherwise: encode (x_idxs,\n // y_idxs, z_idxs) by binary bit\n int box_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, pts_num) {\n if (box_idx >= boxes_num) return;\n\n pts += pt_idx * 3;\n rois += box_idx * 7;\n pts_mask += box_idx * pts_num + pt_idx;\n\n T local_x = 0, local_y = 0;\n int cur_in_flag = check_pt_in_box3d(pts, rois, local_x, local_y);\n\n pts_mask[0] = -1;\n if (cur_in_flag > 0) {\n T local_z = pts[2] - rois[2];\n T x_size = rois[3], y_size = rois[4], z_size = rois[5];\n\n T x_res = x_size / out_x;\n T y_res = y_size / out_y;\n T z_res = z_size / out_z;\n\n unsigned int x_idx = int((local_x + x_size / 2) / x_res);\n unsigned int y_idx = int((local_y + y_size / 2) / y_res);\n unsigned int z_idx = int(local_z / z_res);\n\n x_idx = min(max(x_idx, 0), out_x - 1);\n y_idx = min(max(y_idx, 0), out_y - 1);\n z_idx = min(max(z_idx, 0), out_z - 1);\n\n unsigned int idx_encoding = (x_idx << 16) + (y_idx << 8) + z_idx;\n\n pts_mask[0] = idx_encoding;\n }\n }\n}\n\ntemplate \n__global__ void collect_inside_pts_for_box3d(int boxes_num, int pts_num,\n int max_pts_each_voxel, int out_x,\n int out_y, int out_z,\n const int *pts_mask,\n T *pts_idx_of_voxels) {\n // params pts_mask: (N, npoints) 0 or 1\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel)\n CUDA_1D_KERNEL_LOOP(box_idx, boxes_num) {\n int max_num_pts = max_pts_each_voxel - 1; // index 0 is the counter\n pts_idx_of_voxels += box_idx * out_x * out_y * out_z * max_pts_each_voxel;\n\n for (int k = 0; k < pts_num; k++) {\n if (pts_mask[box_idx * pts_num + k] != -1) {\n unsigned int idx_encoding = pts_mask[box_idx * pts_num + k];\n unsigned int x_idx = (idx_encoding >> 16) & 0xFF;\n unsigned int y_idx = (idx_encoding >> 8) & 0xFF;\n unsigned int z_idx = idx_encoding & 0xFF;\n unsigned int base_offset = x_idx * out_y * out_z * max_pts_each_voxel +\n y_idx * out_z * max_pts_each_voxel +\n z_idx * max_pts_each_voxel;\n unsigned int cnt = pts_idx_of_voxels[base_offset];\n if (cnt < max_num_pts) {\n pts_idx_of_voxels[base_offset + cnt + 1] = k;\n pts_idx_of_voxels[base_offset]++;\n }\n }\n }\n }\n}\n\ntemplate \n__global__ void roiaware_maxpool3d(int boxes_num, int pts_num, int channels,\n int max_pts_each_voxel, int out_x, int out_y,\n int out_z, const T *pts_feature,\n const int *pts_idx_of_voxels,\n T *pooled_features, int *argmax) {\n // params pts_feature: (npoints, C)\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel),\n // index 0 is the counter params pooled_features: (N, out_x, out_y, out_z, C)\n // params argmax: (N, out_x, out_y, out_z, C)\n\n int box_idx = blockIdx.z;\n int channel_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(voxel_idx_flat, out_x * out_y * out_z) {\n int x_idx = voxel_idx_flat / (out_y * out_z);\n int y_idx = (voxel_idx_flat - x_idx * (out_y * out_z)) / out_z;\n int z_idx = voxel_idx_flat % out_z;\n if (box_idx >= boxes_num || channel_idx >= channels) return;\n\n int offset_base = x_idx * out_y * out_z + y_idx * out_z + z_idx;\n pts_idx_of_voxels += box_idx * out_x * out_y * out_z * max_pts_each_voxel +\n offset_base * max_pts_each_voxel;\n pooled_features += box_idx * out_x * out_y * out_z * channels +\n offset_base * channels + channel_idx;\n argmax += box_idx * out_x * out_y * out_z * channels +\n offset_base * channels + channel_idx;\n\n int argmax_idx = -1;\n float max_val = -1e50;\n\n int total_pts = pts_idx_of_voxels[0];\n\n for (int k = 1; k <= total_pts; k++) {\n if (pts_feature[pts_idx_of_voxels[k] * channels + channel_idx] >\n max_val) {\n max_val = pts_feature[pts_idx_of_voxels[k] * channels + channel_idx];\n argmax_idx = pts_idx_of_voxels[k];\n }\n }\n\n if (argmax_idx != -1) {\n pooled_features[0] = max_val;\n }\n argmax[0] = argmax_idx;\n }\n}\n\ntemplate \n__global__ void roiaware_avgpool3d(int boxes_num, int pts_num, int channels,\n int max_pts_each_voxel, int out_x, int out_y,\n int out_z, const T *pts_feature,\n const int *pts_idx_of_voxels,\n T *pooled_features) {\n // params pts_feature: (npoints, C)\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel),\n // index 0 is the counter params pooled_features: (N, out_x, out_y, out_z, C)\n // params argmax: (N, out_x, out_y, out_z, C)\n\n int box_idx = blockIdx.z;\n int channel_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(voxel_idx_flat, out_x * out_y * out_z) {\n int x_idx = voxel_idx_flat / (out_y * out_z);\n int y_idx = (voxel_idx_flat - x_idx * (out_y * out_z)) / out_z;\n int z_idx = voxel_idx_flat % out_z;\n if (box_idx >= boxes_num || channel_idx >= channels) return;\n\n int offset_base = x_idx * out_y * out_z + y_idx * out_z + z_idx;\n pts_idx_of_voxels += box_idx * out_x * out_y * out_z * max_pts_each_voxel +\n offset_base * max_pts_each_voxel;\n pooled_features += box_idx * out_x * out_y * out_z * channels +\n offset_base * channels + channel_idx;\n\n float sum_val = 0;\n int total_pts = pts_idx_of_voxels[0];\n\n for (int k = 1; k <= total_pts; k++) {\n sum_val += pts_feature[pts_idx_of_voxels[k] * channels + channel_idx];\n }\n\n if (total_pts > 0) {\n pooled_features[0] = sum_val / total_pts;\n }\n }\n}\n\ntemplate \n__global__ void roiaware_maxpool3d_backward(int boxes_num, int channels,\n int out_x, int out_y, int out_z,\n const int *argmax,\n const T *grad_out, T *grad_in) {\n // params argmax: (N, out_x, out_y, out_z, C)\n // params grad_out: (N, out_x, out_y, out_z, C)\n // params grad_in: (npoints, C), return value\n\n int box_idx = blockIdx.z;\n int channel_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(voxel_idx_flat, out_x * out_y * out_z) {\n int x_idx = voxel_idx_flat / (out_y * out_z);\n int y_idx = (voxel_idx_flat - x_idx * (out_y * out_z)) / out_z;\n int z_idx = voxel_idx_flat % out_z;\n if (box_idx >= boxes_num || channel_idx >= channels) return;\n\n int offset_base = x_idx * out_y * out_z + y_idx * out_z + z_idx;\n argmax += box_idx * out_x * out_y * out_z * channels +\n offset_base * channels + channel_idx;\n grad_out += box_idx * out_x * out_y * out_z * channels +\n offset_base * channels + channel_idx;\n\n if (argmax[0] == -1) return;\n\n atomicAdd(grad_in + argmax[0] * channels + channel_idx, grad_out[0] * 1);\n }\n}\n\ntemplate \n__global__ void roiaware_avgpool3d_backward(int boxes_num, int channels,\n int out_x, int out_y, int out_z,\n int max_pts_each_voxel,\n const int *pts_idx_of_voxels,\n const T *grad_out, T *grad_in) {\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel)\n // params grad_out: (N, out_x, out_y, out_z, C)\n // params grad_in: (npoints, C), return value\n\n int box_idx = blockIdx.z;\n int channel_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(voxel_idx_flat, out_x * out_y * out_z) {\n int x_idx = voxel_idx_flat / (out_y * out_z);\n int y_idx = (voxel_idx_flat - x_idx * (out_y * out_z)) / out_z;\n int z_idx = voxel_idx_flat % out_z;\n if (box_idx >= boxes_num || channel_idx >= channels) return;\n\n int offset_base = x_idx * out_y * out_z + y_idx * out_z + z_idx;\n pts_idx_of_voxels += box_idx * out_x * out_y * out_z * max_pts_each_voxel +\n offset_base * max_pts_each_voxel;\n grad_out += box_idx * out_x * out_y * out_z * channels +\n offset_base * channels + channel_idx;\n\n int total_pts = pts_idx_of_voxels[0];\n float cur_grad = 1 / fmaxf(float(total_pts), 1.0);\n for (int k = 1; k <= total_pts; k++) {\n atomicAdd(grad_in + pts_idx_of_voxels[k] * channels + channel_idx,\n grad_out[0] * cur_grad);\n }\n }\n}\n\n#endif // ROIAWARE_POOL3D_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/roipoint_pool3d_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROIPOINT_POOL3D_CUDA_KERNEL_CUH\n#define ROIPOINT_POOL3D_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__device__ inline void lidar_to_local_coords(T shift_x, T shift_y, T rz,\n T &local_x, T &local_y) {\n T cosa = cos(-rz), sina = sin(-rz);\n local_x = shift_x * cosa + shift_y * (-sina);\n local_y = shift_x * sina + shift_y * cosa;\n}\n\ntemplate \n__device__ inline int check_pt_in_box3d(const T *pt, const T *box3d, T &local_x,\n T &local_y) {\n // param pt: (x, y, z)\n // param box3d: (cx, cy, cz, dx, dy, dz, rz) in LiDAR coordinate, cz in the\n // bottom center\n T x = pt[0], y = pt[1], z = pt[2];\n T cx = box3d[0], cy = box3d[1], cz = box3d[2];\n T dx = box3d[3], dy = box3d[4], dz = box3d[5], rz = box3d[6];\n cz += dz / 2.0; // shift to the center since cz in box3d is the bottom center\n\n if (fabsf(z - cz) > dz / 2.0) return 0;\n lidar_to_local_coords(x - cx, y - cy, rz, local_x, local_y);\n T in_flag = (local_x > -dx / 2.0) & (local_x < dx / 2.0) &\n (local_y > -dy / 2.0) & (local_y < dy / 2.0);\n return in_flag;\n}\n\ntemplate \n__global__ void assign_pts_to_box3d(int batch_size, int pts_num, int boxes_num,\n const T *xyz, const T *boxes3d,\n int *pts_assign) {\n // params xyz: (B, N, 3)\n // params boxes3d: (B, M, 7)\n // params pts_assign: (B, N, M): idx of the corresponding box3d, -1 means\n // background points\n int box_idx = blockIdx.y;\n int bs_idx = blockIdx.z;\n CUDA_1D_KERNEL_LOOP(pt_idx, pts_num) {\n if (box_idx >= boxes_num || bs_idx >= batch_size) return;\n\n int assign_idx =\n bs_idx * pts_num * boxes_num + pt_idx * boxes_num + box_idx;\n pts_assign[assign_idx] = 0;\n\n int box_offset = bs_idx * boxes_num * 7 + box_idx * 7;\n int pt_offset = bs_idx * pts_num * 3 + pt_idx * 3;\n\n T local_x = 0, local_y = 0;\n int cur_in_flag = check_pt_in_box3d(xyz + pt_offset, boxes3d + box_offset,\n local_x, local_y);\n pts_assign[assign_idx] = cur_in_flag;\n }\n}\n\n__global__ void get_pooled_idx(int batch_size, int pts_num, int boxes_num,\n int sampled_pts_num, const int *pts_assign,\n int *pts_idx, int *pooled_empty_flag) {\n // params xyz: (B, N, 3)\n // params pts_feature: (B, N, C)\n // params pts_assign: (B, N)\n // params pts_idx: (B, M, 512)\n // params pooled_empty_flag: (B, M)\n CUDA_1D_KERNEL_LOOP(boxes_idx, boxes_num) {\n int bs_idx = blockIdx.y;\n\n int cnt = 0;\n for (int k = 0; k < pts_num; k++) {\n if (pts_assign[bs_idx * pts_num * boxes_num + k * boxes_num +\n boxes_idx]) {\n if (cnt < sampled_pts_num) {\n pts_idx[bs_idx * boxes_num * sampled_pts_num +\n boxes_idx * sampled_pts_num + cnt] = k;\n cnt++;\n } else\n break;\n }\n }\n\n if (cnt == 0) {\n pooled_empty_flag[bs_idx * boxes_num + boxes_idx] = 1;\n } else if (cnt < sampled_pts_num) {\n // duplicate same points for sampling\n for (int k = cnt; k < sampled_pts_num; k++) {\n int duplicate_idx = k % cnt;\n int base_offset =\n bs_idx * boxes_num * sampled_pts_num + boxes_idx * sampled_pts_num;\n pts_idx[base_offset + k] = pts_idx[base_offset + duplicate_idx];\n }\n }\n }\n}\n\ntemplate \n__global__ void roipoint_pool3d_forward(\n int batch_size, int pts_num, int boxes_num, int feature_in_len,\n int sampled_pts_num, const T *xyz, const int *pts_idx, const T *pts_feature,\n T *pooled_features, int *pooled_empty_flag) {\n // params xyz: (B, N, 3)\n // params pts_idx: (B, M, 512)\n // params pts_feature: (B, N, C)\n // params pooled_features: (B, M, 512, 3+C)\n // params pooled_empty_flag: (B, M)\n int box_idx = blockIdx.y;\n int bs_idx = blockIdx.z;\n CUDA_1D_KERNEL_LOOP(sample_pt_idx, sampled_pts_num) {\n if (box_idx >= boxes_num || bs_idx >= batch_size) return;\n if (pooled_empty_flag[bs_idx * boxes_num + box_idx]) return;\n\n int temp_idx = bs_idx * boxes_num * sampled_pts_num +\n box_idx * sampled_pts_num + sample_pt_idx;\n int src_pt_idx = pts_idx[temp_idx];\n int dst_feature_offset = temp_idx * (3 + feature_in_len);\n\n for (int j = 0; j < 3; j++)\n pooled_features[dst_feature_offset + j] =\n xyz[bs_idx * pts_num * 3 + src_pt_idx * 3 + j];\n\n int src_feature_offset =\n bs_idx * pts_num * feature_in_len + src_pt_idx * feature_in_len;\n memcpy(pooled_features + dst_feature_offset + 3,\n pts_feature + src_feature_offset, feature_in_len * sizeof(T));\n }\n}\n\n#endif // ROIPOINT_POOL3D_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/rotated_feature_align_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/SJTU-Thinklab-Det/r3det-on-mmdetection/blob/master/mmdet/ops/fr/src/feature_refine_kernel.cu\n#ifndef ROTATED_FEATURE_ALIGN_CUDA_KERNEL_CUH\n#define ROTATED_FEATURE_ALIGN_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void rotated_feature_align_forward_kernel(\n const int nthreads, const int points, const scalar_t* bottom_data,\n const scalar_t* best_bboxes, const scalar_t spatial_scale,\n const int channels, const int height, const int width, scalar_t* top_data) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int w = index % width;\n int h = (index / width) % height;\n int c = (index / width / height) % channels;\n int n = index / width / height / channels;\n\n const scalar_t* bbox_offset =\n best_bboxes + ((n * height + h) * width + w) * 5;\n scalar_t roi_y = bbox_offset[0] * spatial_scale;\n scalar_t roi_x = bbox_offset[1] * spatial_scale;\n\n scalar_t px[5] = {roi_x, 0, 0, 0, 0};\n scalar_t py[5] = {roi_y, 0, 0, 0, 0};\n\n if (points > 1) {\n scalar_t roi_w = bbox_offset[2] * spatial_scale;\n scalar_t roi_h = bbox_offset[3] * spatial_scale;\n scalar_t roi_a = bbox_offset[4];\n\n scalar_t w_2 = roi_w / 2, h_2 = roi_h / 2;\n scalar_t cosa = cosf(roi_a), sina = sinf(roi_a);\n scalar_t wx = cosa * w_2, wy = sina * w_2;\n scalar_t hx = -sina * h_2, hy = cosa * h_2;\n\n px[1] = roi_x + wx + hx;\n py[1] = roi_y + wy + hy;\n px[2] = roi_x - wx + hx;\n py[2] = roi_y - wy + hy;\n px[3] = roi_x - wx - hx;\n py[3] = roi_y - wy - hy;\n px[4] = roi_x + wx - hx;\n py[4] = roi_y + wy - hy;\n }\n\n const scalar_t* offset_bottom_data =\n bottom_data + (n * channels + c) * height * width;\n\n scalar_t output_val = bottom_data[index];\n for (int i = 0; i < points; i++) {\n output_val += bilinear_interpolate(offset_bottom_data, height,\n width, py[i], px[i], i);\n }\n top_data[index] = output_val;\n }\n}\n\ntemplate \n__global__ void rotated_feature_align_backward_kernel(\n const int nthreads, const int points, const scalar_t* top_diff,\n const scalar_t* best_bboxes, const scalar_t spatial_scale,\n const int channels, const int height, const int width,\n scalar_t* bottom_diff) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int w = index % width;\n int h = (index / width) % height;\n int c = (index / width / height) % channels;\n int n = index / width / height / channels;\n\n const scalar_t* bbox_offset =\n best_bboxes + ((n * height + h) * width + w) * 5;\n scalar_t roi_y = bbox_offset[0] * spatial_scale;\n scalar_t roi_x = bbox_offset[1] * spatial_scale;\n\n scalar_t px[5] = {roi_x, 0, 0, 0, 0};\n scalar_t py[5] = {roi_y, 0, 0, 0, 0};\n\n if (points > 1) {\n scalar_t roi_w = bbox_offset[2] * spatial_scale;\n scalar_t roi_h = bbox_offset[3] * spatial_scale;\n scalar_t roi_a = bbox_offset[4];\n\n scalar_t w_2 = roi_w / 2, h_2 = roi_h / 2;\n scalar_t cosa = cosf(roi_a), sina = sinf(roi_a);\n scalar_t wx = cosa * w_2, wy = sina * w_2;\n scalar_t hx = -sina * h_2, hy = cosa * h_2;\n\n px[1] = roi_x + wx + hx;\n py[1] = roi_y + wy + hy;\n px[2] = roi_x - wx + hx;\n py[2] = roi_y - wy + hy;\n px[3] = roi_x - wx - hx;\n py[3] = roi_y - wy - hy;\n px[4] = roi_x + wx - hx;\n py[4] = roi_y + wy - hy;\n }\n\n scalar_t* offset_bottom_diff =\n bottom_diff + (n * channels + c) * height * width;\n scalar_t value_top_diff = top_diff[index];\n\n atomicAdd(bottom_diff + index, value_top_diff);\n for (int i = 0; i < points; i++) {\n scalar_t w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n\n bilinear_interpolate_gradient(height, width, py[i], px[i], w1,\n w2, w3, w4, x_low, x_high, y_low,\n y_high, i);\n scalar_t g1 = value_top_diff * w1;\n scalar_t g2 = value_top_diff * w2;\n scalar_t g3 = value_top_diff * w3;\n scalar_t g4 = value_top_diff * w4;\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n atomicAdd(offset_bottom_diff + y_low * width + x_low, g1);\n atomicAdd(offset_bottom_diff + y_low * width + x_high, g2);\n atomicAdd(offset_bottom_diff + y_high * width + x_low, g3);\n atomicAdd(offset_bottom_diff + y_high * width + x_high, g4);\n }\n }\n }\n}\n#endif // ROTATED_FEATURE_ALIGN_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/scatter_points_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef SCATTER_POINTS_CUDA_KERNEL_CUH\n#define SCATTER_POINTS_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntypedef enum { SUM = 0, MEAN = 1, MAX = 2 } reduce_t;\nint const maxGridDim = 50000;\n\n__device__ __forceinline__ static void reduceMax(float *address, float val) {\n int *address_as_i = reinterpret_cast(address);\n int old = *address_as_i, assumed;\n do {\n assumed = old;\n old = atomicCAS(address_as_i, assumed,\n __float_as_int(fmaxf(val, __int_as_float(assumed))));\n } while (assumed != old || __int_as_float(old) < val);\n}\n\n__device__ __forceinline__ static void reduceMax(double *address, double val) {\n unsigned long long *address_as_ull =\n reinterpret_cast(address);\n unsigned long long old = *address_as_ull, assumed;\n do {\n assumed = old;\n old = atomicCAS(\n address_as_ull, assumed,\n __double_as_longlong(fmax(val, __longlong_as_double(assumed))));\n } while (assumed != old || __longlong_as_double(old) < val);\n}\n\n// get rid of meaningless warnings when compiling host code\n#ifdef HIP_DIFF\n__device__ __forceinline__ static void reduceAdd(float *address, float val) {\n atomicAdd(address, val);\n}\n__device__ __forceinline__ static void reduceAdd(double *address, double val) {\n atomicAdd(address, val);\n}\n#else\n#ifdef __CUDA_ARCH__\n__device__ __forceinline__ static void reduceAdd(float *address, float val) {\n#if (__CUDA_ARCH__ < 200)\n#ifdef _MSC_VER\n#pragma message( \\\n \"compute capability lower than 2.x. fall back to use CAS version of atomicAdd for float32\")\n#else\n#warning \\\n \"compute capability lower than 2.x. fall back to use CAS version of atomicAdd for float32\"\n#endif\n int *address_as_i = reinterpret_cast(address);\n int old = *address_as_i, assumed;\n do {\n assumed = old;\n old = atomicCAS(address_as_i, assumed,\n __float_as_int(val + __int_as_float(assumed)));\n } while (assumed != old);\n#else\n atomicAdd(address, val);\n#endif\n}\n\n__device__ __forceinline__ static void reduceAdd(double *address, double val) {\n#if (__CUDA_ARCH__ < 600)\n#ifdef _MSC_VER\n#pragma message( \\\n \"compute capability lower than 6.x. fall back to use CAS version of atomicAdd for float64\")\n#else\n#warning \\\n \"compute capability lower than 6.x. fall back to use CAS version of atomicAdd for float64\"\n#endif\n unsigned long long *address_as_ull =\n reinterpret_cast(address);\n unsigned long long old = *address_as_ull, assumed;\n do {\n assumed = old;\n old = atomicCAS(address_as_ull, assumed,\n __double_as_longlong(val + __longlong_as_double(assumed)));\n } while (assumed != old);\n#else\n atomicAdd(address, val);\n#endif\n}\n#endif // __CUDA_ARCH__\n#endif // HIP_DIFF\n\ntemplate \n__global__ void feats_reduce_kernel(\n const T *feats, const int32_t *coors_map,\n T *reduced_feats, // shall be 0 at initialization\n const int num_input, const int num_feats, const reduce_t reduce_type) {\n CUDA_1D_KERNEL_LOOP(x, num_input) {\n int32_t reduce_to = coors_map[x];\n if (reduce_to == -1) continue;\n\n const T *feats_offset = feats + x * num_feats;\n T *reduced_feats_offset = reduced_feats + reduce_to * num_feats;\n if (reduce_type == reduce_t::MAX) {\n for (int i = 0; i < num_feats; i++) {\n reduceMax(&reduced_feats_offset[i], feats_offset[i]);\n }\n } else {\n for (int i = 0; i < num_feats; i++) {\n reduceAdd(&reduced_feats_offset[i], feats_offset[i]);\n }\n }\n }\n}\n\ntemplate \n__global__ void add_reduce_traceback_grad_kernel(\n T *grad_feats, const T *grad_reduced_feats, const int32_t *coors_map,\n const int32_t *reduce_count, const int num_input, const int num_feats,\n const reduce_t reduce_type) {\n CUDA_1D_KERNEL_LOOP(x, num_input) {\n int32_t reduce_to = coors_map[x];\n if (reduce_to == -1) {\n continue;\n }\n\n const int input_offset = x * num_feats;\n T *grad_feats_offset = grad_feats + input_offset;\n const int reduced_offset = reduce_to * num_feats;\n const T *grad_reduced_feats_offset = grad_reduced_feats + reduced_offset;\n\n if (reduce_type == reduce_t::SUM) {\n for (int i = 0; i < num_feats; i++) {\n grad_feats_offset[i] = grad_reduced_feats_offset[i];\n }\n } else if (reduce_type == reduce_t::MEAN) {\n for (int i = 0; i < num_feats; i++) {\n grad_feats_offset[i] = grad_reduced_feats_offset[i] /\n static_cast(reduce_count[reduce_to]);\n }\n }\n }\n}\n\ntemplate \n__global__ void max_reduce_traceback_scatter_idx_kernel(\n const T *feats, const T *reduced_feats, int32_t *reduce_from,\n const int32_t *coors_map, const int num_input, const int num_feats) {\n CUDA_1D_KERNEL_LOOP(x, num_input) {\n int32_t reduce_to = coors_map[x];\n\n const int input_offset = x * num_feats;\n const T *feats_offset = feats + input_offset;\n\n if (reduce_to == -1) {\n continue;\n }\n\n const int reduced_offset = reduce_to * num_feats;\n const T *reduced_feats_offset = reduced_feats + reduced_offset;\n int32_t *reduce_from_offset = reduce_from + reduced_offset;\n\n for (int i = 0; i < num_feats; i++) {\n if (feats_offset[i] == reduced_feats_offset[i]) {\n atomicMin(&reduce_from_offset[i], static_cast(x));\n }\n }\n }\n}\n\ntemplate \n__global__ void max_reduce_scatter_grad_kernel(T *grad_feats,\n const T *grad_reduced_feats,\n const int32_t *reduce_from,\n const int num_reduced,\n const int num_feats) {\n CUDA_1D_KERNEL_LOOP(x, num_reduced) {\n const int reduced_offset = x * num_feats;\n const int32_t *scatter_to_offset = reduce_from + reduced_offset;\n const T *grad_reduced_feats_offset = grad_reduced_feats + reduced_offset;\n\n for (int i = 0; i < num_feats; i++) {\n grad_feats[scatter_to_offset[i] * num_feats + i] =\n grad_reduced_feats_offset[i];\n }\n }\n}\n\n#endif // SCATTER_POINTS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/sigmoid_focal_loss_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef SIGMOID_FOCAL_LOSS_CUDA_KERNEL_CUH\n#define SIGMOID_FOCAL_LOSS_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void sigmoid_focal_loss_forward_cuda_kernel(\n const int nthreads, const T* input, const int64_t* target, const T* weight,\n T* output, const T gamma, const T alpha, const int num_classes) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int n = index / num_classes;\n int c = index % num_classes;\n\n int64_t t = target[n];\n T flag_p = (t == c);\n T flag_n = (t != c);\n\n // p = sigmoid(x) = 1. / 1. + expf(-x)\n T p = (T)1. / ((T)1. + expf(-input[index]));\n\n // (1 - p)**gamma * log(p)\n T term_p = pow(((T)1. - p), gamma) * log(max(p, (T)FLT_MIN));\n // p**gamma * log(1 - p)\n T term_n = pow(p, gamma) * log(max((T)1. - p, (T)FLT_MIN));\n\n output[index] = (T)0.;\n output[index] += -flag_p * alpha * term_p;\n output[index] += -flag_n * ((T)1. - alpha) * term_n;\n if (weight != NULL) {\n output[index] *= weight[t];\n }\n }\n}\n\ntemplate \n__global__ void sigmoid_focal_loss_backward_cuda_kernel(\n const int nthreads, const T* input, const int64_t* target, const T* weight,\n T* grad_input, const T gamma, const T alpha, const int num_classes) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int n = index / num_classes;\n int c = index % num_classes;\n\n int64_t t = target[n];\n T flag_p = (t == c);\n T flag_n = (t != c);\n\n // p = sigmoid(x) = 1. / 1. + expf(-x)\n T p = (T)1. / ((T)1. + exp(-input[index]));\n\n // (1 - p)**gamma * (1 - p - gamma*p*log(p))\n T term_p = pow(((T)1. - p), gamma) *\n ((T)1. - p - (gamma * p * log(max(p, (T)FLT_MIN))));\n // p**gamma * (gamma * (1 - p) * log(1 - p) - p)\n T term_n = pow(p, gamma) *\n (gamma * ((T)1. - p) * log(max((T)1. - p, (T)FLT_MIN)) - p);\n\n grad_input[index] = (T)0.;\n grad_input[index] += -flag_p * alpha * term_p;\n grad_input[index] += -flag_n * ((T)1. - alpha) * term_n;\n if (weight != NULL) {\n grad_input[index] *= weight[t];\n }\n }\n}\n\n#endif // SIGMOID_FOCAL_LOSS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/softmax_focal_loss_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef SOFTMAX_FOCAL_LOSS_CUDA_KERNEL_CUH\n#define SOFTMAX_FOCAL_LOSS_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void softmax_focal_loss_forward_cuda_kernel(\n const int nthreads, const T* softmax, const int64_t* target,\n const T* weight, T* output, const T gamma, const T alpha,\n const int num_classes) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int64_t label = target[index];\n T pred = softmax[index * num_classes + label];\n\n if (label >= 0) {\n output[index] =\n -alpha * pow((T)1. - pred, gamma) * log(max(pred, (T)FLT_MIN));\n } else {\n output[index] = 0;\n }\n if (weight != NULL) {\n output[index] *= weight[label];\n }\n }\n}\n\ntemplate \n__global__ void softmax_focal_loss_backward_cuda1_kernel(\n const int nthreads, const T* softmax, const int64_t* target,\n const T* weight, T* buff, const T gamma, const T alpha,\n const int num_classes) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int64_t label = target[index];\n T pred = softmax[index * num_classes + label];\n\n if (label >= 0) {\n buff[index] = alpha * (-pow((T)1. - pred, gamma) +\n gamma * pow((T)1. - pred, gamma - 1) * pred *\n log(max(pred, (T)FLT_MIN)));\n } else {\n buff[index] = 0;\n }\n if (weight != NULL) {\n buff[index] *= weight[label];\n }\n }\n}\n\ntemplate \n__global__ void softmax_focal_loss_backward_cuda2_kernel(\n const int nthreads, const T* softmax, const int64_t* target, const T* buff,\n T* grad_input, const int num_classes) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int n = index / num_classes;\n int c = index % num_classes;\n int64_t label = target[n];\n\n if (label >= 0) {\n T flag = (label == c ? (T)1. : (T)0.);\n grad_input[index] = buff[n] * (flag - softmax[index]);\n } else {\n grad_input[index] = 0;\n }\n }\n}\n\n#endif // SOFTMAX_FOCAL_LOSS_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/sync_bn_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef SYNCBN_CUDA_KERNEL_CUH\n#define SYNCBN_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void sync_bn_forward_mean_cuda_kernel(const T *input, float *mean,\n int num, int channels,\n int spatial) {\n __shared__ float buffer[THREADS_PER_BLOCK];\n int tid = threadIdx.x;\n int c = blockIdx.x;\n buffer[tid] = 0;\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index = (i / spatial) * channels * spatial + c * spatial + i % spatial;\n buffer[tid] += input[index];\n }\n __syncthreads();\n\n for (int s = blockDim.x / 2; s > 0; s >>= 1) {\n if (tid < s) {\n buffer[tid] += buffer[tid + s];\n }\n __syncthreads();\n }\n int total = num * spatial;\n if (tid == 0) {\n mean[c] = buffer[0] / total;\n }\n}\n\ntemplate <>\n__global__ void sync_bn_forward_mean_cuda_kernel(const phalf *input,\n float *mean, int num,\n int channels, int spatial) {\n __shared__ float buffer[THREADS_PER_BLOCK];\n int tid = threadIdx.x;\n int c = blockIdx.x;\n buffer[tid] = 0;\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index = (i / spatial) * channels * spatial + c * spatial + i % spatial;\n buffer[tid] += static_cast(input[index]);\n }\n __syncthreads();\n\n for (int s = blockDim.x / 2; s > 0; s >>= 1) {\n if (tid < s) {\n buffer[tid] += buffer[tid + s];\n }\n __syncthreads();\n }\n int total = num * spatial;\n if (tid == 0) {\n mean[c] = buffer[0] / total;\n }\n}\n\ntemplate \n__global__ void sync_bn_forward_var_cuda_kernel(const T *input,\n const float *mean, float *var,\n int num, int channels,\n int spatial) {\n __shared__ float buffer[THREADS_PER_BLOCK];\n int tid = threadIdx.x;\n int c = blockIdx.x;\n buffer[tid] = 0;\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index = (i / spatial) * channels * spatial + c * spatial + i % spatial;\n float td = input[index] - mean[c];\n buffer[tid] += td * td;\n }\n __syncthreads();\n for (int s = blockDim.x / 2; s > 0; s >>= 1) {\n if (tid < s) {\n buffer[tid] += buffer[tid + s];\n }\n __syncthreads();\n }\n int total = num * spatial;\n if (tid == 0) {\n var[c] = buffer[0] / total;\n }\n}\n\ntemplate <>\n__global__ void sync_bn_forward_var_cuda_kernel(const phalf *input,\n const float *mean, float *var,\n int num, int channels,\n int spatial) {\n __shared__ float buffer[THREADS_PER_BLOCK];\n int tid = threadIdx.x;\n int c = blockIdx.x;\n buffer[tid] = 0;\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index = (i / spatial) * channels * spatial + c * spatial + i % spatial;\n float td = static_cast(input[index]) - mean[c];\n buffer[tid] += td * td;\n }\n __syncthreads();\n for (int s = blockDim.x / 2; s > 0; s >>= 1) {\n if (tid < s) {\n buffer[tid] += buffer[tid + s];\n }\n __syncthreads();\n }\n int total = num * spatial;\n if (tid == 0) {\n var[c] = buffer[0] / total;\n }\n}\n\ntemplate \n__global__ void sync_bn_forward_output_cuda_kernel(\n const T *input, const float *mean, const float *var, float *running_mean,\n float *running_var, const float *weight, const float *bias, float *norm,\n float *std, T *output, int num, int channels, int spatial, float eps,\n float momentum, int group_size) {\n int tid = threadIdx.x;\n int c = blockIdx.x;\n float mean_value = mean[c];\n float std_value = sqrt(var[c] + eps);\n\n if (weight != nullptr) {\n float weight_value = weight[c];\n float bias_value = bias[c];\n if (norm != nullptr) {\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index =\n (i / spatial) * channels * spatial + c * spatial + i % spatial;\n norm[index] = (input[index] - mean_value) / std_value;\n output[index] = norm[index] * weight_value + bias_value;\n }\n } else {\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index =\n (i / spatial) * channels * spatial + c * spatial + i % spatial;\n output[index] =\n (input[index] - mean_value) / std_value * weight_value + bias_value;\n }\n }\n } else {\n if (norm != nullptr) {\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index =\n (i / spatial) * channels * spatial + c * spatial + i % spatial;\n output[index] = norm[index] = (input[index] - mean_value) / std_value;\n }\n } else {\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index =\n (i / spatial) * channels * spatial + c * spatial + i % spatial;\n output[index] = (input[index] - mean_value) / std_value;\n }\n }\n }\n if (tid == 0) {\n if (std != nullptr) std[c] = std_value;\n if (running_mean != nullptr) {\n running_mean[c] =\n momentum * mean_value + (1 - momentum) * running_mean[c];\n int count = num * spatial * group_size;\n float var_unbias = count > 1 ? var[c] * count / (count - 1) : var[c];\n running_var[c] = momentum * var_unbias + (1 - momentum) * running_var[c];\n }\n }\n}\n\ntemplate <>\n__global__ void sync_bn_forward_output_cuda_kernel(\n const phalf *input, const float *mean, const float *var,\n float *running_mean, float *running_var, const float *weight,\n const float *bias, float *norm, float *std, phalf *output, int num,\n int channels, int spatial, float eps, float momentum, int group_size) {\n int tid = threadIdx.x;\n int c = blockIdx.x;\n float mean_value = mean[c];\n float std_value = sqrt(var[c] + eps);\n if (weight != nullptr) {\n float weight_value = weight[c];\n float bias_value = bias[c];\n if (norm != nullptr) {\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index =\n (i / spatial) * channels * spatial + c * spatial + i % spatial;\n norm[index] =\n (static_cast(input[index]) - mean_value) / std_value;\n output[index] =\n static_cast(norm[index] * weight_value + bias_value);\n }\n } else {\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index =\n (i / spatial) * channels * spatial + c * spatial + i % spatial;\n output[index] =\n static_cast((static_cast(input[index]) - mean_value) /\n std_value * weight_value +\n bias_value);\n }\n }\n } else {\n if (norm != nullptr) {\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index =\n (i / spatial) * channels * spatial + c * spatial + i % spatial;\n norm[index] =\n (static_cast(input[index]) - mean_value) / std_value;\n output[index] = static_cast(norm[index]);\n }\n } else {\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index =\n (i / spatial) * channels * spatial + c * spatial + i % spatial;\n output[index] = static_cast(\n (static_cast(input[index]) - mean_value) / std_value);\n }\n }\n }\n if (tid == 0) {\n if (std != nullptr) std[c] = std_value;\n if (running_mean != nullptr) {\n running_mean[c] =\n momentum * mean_value + (1 - momentum) * running_mean[c];\n int count = num * spatial * group_size;\n float var_unbias = count > 1 ? var[c] * count / (count - 1) : var[c];\n running_var[c] = momentum * var_unbias + (1 - momentum) * running_var[c];\n }\n }\n}\n\ntemplate \n__global__ void sync_bn_backward_param_cuda_kernel(const T *grad_output,\n const float *norm,\n float *grad_weight,\n float *grad_bias, int num,\n int channels, int spatial) {\n __shared__ float buffer1[THREADS_PER_BLOCK];\n __shared__ float buffer2[THREADS_PER_BLOCK];\n\n int tid = threadIdx.x;\n int c = blockIdx.x;\n buffer1[tid] = buffer2[tid] = 0;\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index = (i / spatial) * channels * spatial + c * spatial + i % spatial;\n buffer1[tid] += grad_output[index] * norm[index];\n buffer2[tid] += grad_output[index];\n }\n __syncthreads();\n\n for (int s = blockDim.x / 2; s > 0; s >>= 1) {\n if (tid < s) {\n buffer1[tid] += buffer1[tid + s];\n buffer2[tid] += buffer2[tid + s];\n }\n __syncthreads();\n }\n if (tid == 0) {\n grad_weight[c] = buffer1[0];\n grad_bias[c] = buffer2[0];\n }\n}\n\ntemplate <>\n__global__ void sync_bn_backward_param_cuda_kernel(const phalf *grad_output,\n const float *norm,\n float *grad_weight,\n float *grad_bias, int num,\n int channels, int spatial) {\n __shared__ float buffer1[THREADS_PER_BLOCK];\n __shared__ float buffer2[THREADS_PER_BLOCK];\n\n int tid = threadIdx.x;\n int c = blockIdx.x;\n buffer1[tid] = buffer2[tid] = 0;\n for (int i = tid; i < num * spatial; i += blockDim.x) {\n int index = (i / spatial) * channels * spatial + c * spatial + i % spatial;\n buffer1[tid] += static_cast(grad_output[index]) * norm[index];\n buffer2[tid] += static_cast(grad_output[index]);\n }\n __syncthreads();\n\n for (int s = blockDim.x / 2; s > 0; s >>= 1) {\n if (tid < s) {\n buffer1[tid] += buffer1[tid + s];\n buffer2[tid] += buffer2[tid + s];\n }\n __syncthreads();\n }\n if (tid == 0) {\n grad_weight[c] = buffer1[0];\n grad_bias[c] = buffer2[0];\n }\n}\n\ntemplate \n__global__ void sync_bn_backward_data_cuda_kernel(\n int output_size, const T *grad_output, const float *weight,\n const float *grad_weight, const float *grad_bias, const float *norm,\n const float *std, T *grad_input, int num, int channels, int spatial) {\n int factor = num * spatial;\n CUDA_1D_KERNEL_LOOP(index, output_size) {\n int c = (index / spatial) % channels;\n grad_input[index] =\n weight[c] *\n (grad_output[index] -\n (grad_weight[c] * norm[index] + grad_bias[c]) / factor) /\n std[c];\n }\n}\n\ntemplate <>\n__global__ void sync_bn_backward_data_cuda_kernel(\n int output_size, const phalf *grad_output, const float *weight,\n const float *grad_weight, const float *grad_bias, const float *norm,\n const float *std, phalf *grad_input, int num, int channels, int spatial) {\n int factor = num * spatial;\n CUDA_1D_KERNEL_LOOP(index, output_size) {\n int c = (index / spatial) % channels;\n grad_input[index] = static_cast(\n weight[c] *\n (static_cast(grad_output[index]) -\n (grad_weight[c] * norm[index] + grad_bias[c]) / factor) /\n std[c]);\n }\n}\n\n#endif // SYNCBN_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/three_interpolate_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef THREE_INTERPOLATE_CUDA_KERNEL_CUH\n#define THREE_INTERPOLATE_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void three_interpolate_forward_cuda_kernel(\n int b, int c, int m, int n, const T *points, const int *__restrict__ idx,\n const T *weight, T *out) {\n // points: (B, C, M)\n // idx: (B, N, 3)\n // weight: (B, N, 3)\n // output:\n // out: (B, C, N)\n\n int bs_idx = blockIdx.z;\n int c_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, n) {\n if (bs_idx >= b || c_idx >= c) return;\n\n weight += bs_idx * n * 3 + pt_idx * 3;\n points += bs_idx * c * m + c_idx * m;\n idx += bs_idx * n * 3 + pt_idx * 3;\n out += bs_idx * c * n + c_idx * n;\n\n out[pt_idx] = weight[0] * points[idx[0]] + weight[1] * points[idx[1]] +\n weight[2] * points[idx[2]];\n }\n}\n\ntemplate \n__global__ void three_interpolate_backward_cuda_kernel(\n int b, int c, int n, int m, const T *grad_out, const int *__restrict__ idx,\n const T *weight, T *grad_points) {\n // grad_out: (B, C, N)\n // weight: (B, N, 3)\n // output:\n // grad_points: (B, C, M)\n\n int bs_idx = blockIdx.z;\n int c_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, n) {\n if (bs_idx >= b || c_idx >= c) return;\n\n grad_out += bs_idx * c * n + c_idx * n + pt_idx;\n weight += bs_idx * n * 3 + pt_idx * 3;\n grad_points += bs_idx * c * m + c_idx * m;\n idx += bs_idx * n * 3 + pt_idx * 3;\n\n atomicAdd(grad_points + idx[0], grad_out[0] * weight[0]);\n atomicAdd(grad_points + idx[1], grad_out[0] * weight[1]);\n atomicAdd(grad_points + idx[2], grad_out[0] * weight[2]);\n }\n}\n\n#endif // THREE_INTERPOLATE_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/three_nn_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef THREE_NN_CUDA_KERNEL_CUH\n#define THREE_NN_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void three_nn_forward_cuda_kernel(int b, int n, int m,\n const T *unknown, const T *known,\n T *dist2, int *__restrict__ idx) {\n // unknown: (B, N, 3)\n // known: (B, M, 3)\n // output:\n // dist2: (B, N, 3)\n // idx: (B, N, 3)\n\n int bs_idx = blockIdx.y;\n CUDA_1D_KERNEL_LOOP(pt_idx, n) {\n if (bs_idx >= b) return;\n\n unknown += bs_idx * n * 3 + pt_idx * 3;\n known += bs_idx * m * 3;\n dist2 += bs_idx * n * 3 + pt_idx * 3;\n idx += bs_idx * n * 3 + pt_idx * 3;\n\n T ux = unknown[0];\n T uy = unknown[1];\n T uz = unknown[2];\n\n double best1 = 1e40, best2 = 1e40, best3 = 1e40;\n int besti1 = 0, besti2 = 0, besti3 = 0;\n for (int k = 0; k < m; ++k) {\n T x = known[k * 3 + 0];\n T y = known[k * 3 + 1];\n T z = known[k * 3 + 2];\n T d = (ux - x) * (ux - x) + (uy - y) * (uy - y) + (uz - z) * (uz - z);\n if (d < best1) {\n best3 = best2;\n besti3 = besti2;\n best2 = best1;\n besti2 = besti1;\n best1 = d;\n besti1 = k;\n } else if (d < best2) {\n best3 = best2;\n besti3 = besti2;\n best2 = d;\n besti2 = k;\n } else if (d < best3) {\n best3 = d;\n besti3 = k;\n }\n }\n dist2[0] = best1;\n dist2[1] = best2;\n dist2[2] = best3;\n idx[0] = besti1;\n idx[1] = besti2;\n idx[2] = besti3;\n }\n}\n\n#endif // THREE_NN_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/tin_shift_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef TIN_SHIFT_CUDA_KERNEL_CUH\n#define TIN_SHIFT_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntemplate \n__global__ void tin_shift_forward_cuda_kernel(\n const int nthreads, const T* input, const int* shift, T* output,\n const int batch_size, const int channels, const int t_size,\n const int hw_size, const int group_size, const int group_channel) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n const int hw_index = index % hw_size;\n const int j = (index / hw_size) % channels;\n\n const int n_index = (index / hw_size / channels) % batch_size;\n int group_id = j / group_channel;\n int t_shift = shift[n_index * group_size + group_id];\n int offset = n_index * t_size * hw_size * channels + hw_size * j + hw_index;\n for (int i = 0; i < t_size; i++) {\n int now_t = i + t_shift;\n int data_id = i * hw_size * channels + offset;\n if (now_t < 0 || now_t >= t_size) {\n continue;\n }\n int out_id = now_t * hw_size * channels + offset;\n output[out_id] = input[data_id];\n }\n }\n}\n\ntemplate \n__global__ void tin_shift_backward_cuda_kernel(\n const int nthreads, const T* input, const int* shift, T* output,\n const int batch_size, const int channels, const int t_size,\n const int hw_size, const int group_size, const int group_channel) {\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n const int hw_index = index % hw_size;\n const int j = (index / hw_size) % channels;\n\n const int n_index = (index / hw_size / channels) % batch_size;\n int group_id = j / group_channel;\n int t_shift = shift[n_index * group_size + group_id];\n int offset = n_index * t_size * hw_size * channels + hw_size * j + hw_index;\n for (int i = 0; i < t_size; i++) {\n int now_t = i + t_shift;\n int data_id = i * hw_size * channels + offset;\n if (now_t < 0 || now_t >= t_size) {\n continue;\n }\n int out_id = now_t * hw_size * channels + offset;\n output[out_id] = input[data_id];\n }\n }\n}\n\n#endif // TIN_SHIFT_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/cuda/voxelization_cuda_kernel.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#ifndef VOXELIZATION_CUDA_KERNEL_CUH\n#define VOXELIZATION_CUDA_KERNEL_CUH\n\n#ifdef MMCV_USE_PARROTS\n#include \"parrots_cuda_helper.hpp\"\n#else\n#include \"pytorch_cuda_helper.hpp\"\n#endif\n\ntypedef enum { SUM = 0, MEAN = 1, MAX = 2 } reduce_t;\n\ntemplate \n__global__ void dynamic_voxelize_kernel(\n const T* points, T_int* coors, const float voxel_x, const float voxel_y,\n const float voxel_z, const float coors_x_min, const float coors_y_min,\n const float coors_z_min, const float coors_x_max, const float coors_y_max,\n const float coors_z_max, const int grid_x, const int grid_y,\n const int grid_z, const int num_points, const int num_features,\n const int NDim) {\n // const int index = blockIdx.x * threadsPerBlock + threadIdx.x;\n CUDA_1D_KERNEL_LOOP(index, num_points) {\n // To save some computation\n auto points_offset = points + index * num_features;\n auto coors_offset = coors + index * NDim;\n int c_x = floorf((points_offset[0] - coors_x_min) / voxel_x);\n if (c_x < 0 || c_x >= grid_x) {\n coors_offset[0] = -1;\n continue;\n }\n\n int c_y = floorf((points_offset[1] - coors_y_min) / voxel_y);\n if (c_y < 0 || c_y >= grid_y) {\n coors_offset[0] = -1;\n coors_offset[1] = -1;\n continue;\n }\n\n int c_z = floorf((points_offset[2] - coors_z_min) / voxel_z);\n if (c_z < 0 || c_z >= grid_z) {\n coors_offset[0] = -1;\n coors_offset[1] = -1;\n coors_offset[2] = -1;\n } else {\n coors_offset[0] = c_z;\n coors_offset[1] = c_y;\n coors_offset[2] = c_x;\n }\n }\n}\n\ntemplate \n__global__ void assign_point_to_voxel(const int nthreads, const T* points,\n T_int* point_to_voxelidx,\n T_int* coor_to_voxelidx, T* voxels,\n const int max_points,\n const int num_features,\n const int num_points, const int NDim) {\n CUDA_1D_KERNEL_LOOP(thread_idx, nthreads) {\n // const int index = blockIdx.x * threadsPerBlock + threadIdx.x;\n int index = thread_idx / num_features;\n\n int num = point_to_voxelidx[index];\n int voxelidx = coor_to_voxelidx[index];\n if (num > -1 && voxelidx > -1) {\n auto voxels_offset =\n voxels + voxelidx * max_points * num_features + num * num_features;\n\n int k = thread_idx % num_features;\n voxels_offset[k] = points[thread_idx];\n }\n }\n}\n\ntemplate \n__global__ void assign_voxel_coors(const int nthreads, T_int* coor,\n T_int* point_to_voxelidx,\n T_int* coor_to_voxelidx, T_int* voxel_coors,\n const int num_points, const int NDim) {\n CUDA_1D_KERNEL_LOOP(thread_idx, nthreads) {\n // const int index = blockIdx.x * threadsPerBlock + threadIdx.x;\n // if (index >= num_points) return;\n int index = thread_idx / NDim;\n int num = point_to_voxelidx[index];\n int voxelidx = coor_to_voxelidx[index];\n if (num == 0 && voxelidx > -1) {\n auto coors_offset = voxel_coors + voxelidx * NDim;\n int k = thread_idx % NDim;\n coors_offset[k] = coor[thread_idx];\n }\n }\n}\n\ntemplate \n__global__ void point_to_voxelidx_kernel(const T_int* coor,\n T_int* point_to_voxelidx,\n T_int* point_to_pointidx,\n const int max_points,\n const int max_voxels,\n const int num_points, const int NDim) {\n CUDA_1D_KERNEL_LOOP(index, num_points) {\n auto coor_offset = coor + index * NDim;\n // skip invalid points\n if (coor_offset[0] == -1) return;\n\n int num = 0;\n int coor_x = coor_offset[0];\n int coor_y = coor_offset[1];\n int coor_z = coor_offset[2];\n // only calculate the coors before this coor[index]\n for (int i = 0; i < index; ++i) {\n auto prev_coor = coor + i * NDim;\n if (prev_coor[0] == -1) continue;\n\n // Find all previous points that have the same coors\n // if find the same coor, record it\n if ((prev_coor[0] == coor_x) && (prev_coor[1] == coor_y) &&\n (prev_coor[2] == coor_z)) {\n num++;\n if (num == 1) {\n // point to the same coor that first show up\n point_to_pointidx[index] = i;\n } else if (num >= max_points) {\n // out of boundary\n return;\n }\n }\n }\n if (num == 0) {\n point_to_pointidx[index] = index;\n }\n if (num < max_points) {\n point_to_voxelidx[index] = num;\n }\n }\n}\n\ntemplate \n__global__ void determin_voxel_num(\n // const T_int* coor,\n T_int* num_points_per_voxel, T_int* point_to_voxelidx,\n T_int* point_to_pointidx, T_int* coor_to_voxelidx, T_int* voxel_num,\n const int max_points, const int max_voxels, const int num_points) {\n // only calculate the coors before this coor[index]\n for (int i = 0; i < num_points; ++i) {\n int point_pos_in_voxel = point_to_voxelidx[i];\n // record voxel\n if (point_pos_in_voxel == -1) {\n // out of max_points or invalid point\n continue;\n } else if (point_pos_in_voxel == 0) {\n // record new voxel\n int voxelidx = voxel_num[0];\n if (voxel_num[0] >= max_voxels) continue;\n voxel_num[0] += 1;\n coor_to_voxelidx[i] = voxelidx;\n num_points_per_voxel[voxelidx] = 1;\n } else {\n int point_idx = point_to_pointidx[i];\n int voxelidx = coor_to_voxelidx[point_idx];\n if (voxelidx != -1) {\n coor_to_voxelidx[i] = voxelidx;\n num_points_per_voxel[voxelidx] += 1;\n }\n }\n }\n}\n\n#endif // VOXELIZATION_CUDA_KERNEL_CUH\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/parrots_cpp_helper.hpp", "content": "#ifndef PARROTS_CPP_HELPER\n#define PARROTS_CPP_HELPER\n#include \n#include \n#include \n#include \n#include \n\nusing namespace parrots;\n\n#define PARROTS_PRIVATE_CASE_TYPE(prim_type, type, ...) \\\n case prim_type: { \\\n using scalar_t = type; \\\n return __VA_ARGS__(); \\\n }\n\n#define PARROTS_DISPATCH_FLOATING_TYPES(TYPE, ...) \\\n [&] { \\\n const auto& the_type = TYPE; \\\n switch (the_type) { \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float64, double, __VA_ARGS__) \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float32, float, __VA_ARGS__) \\\n default: \\\n PARROTS_NOTSUPPORTED; \\\n } \\\n }()\n\n#define PARROTS_DISPATCH_FLOATING_TYPES_AND_HALF(TYPE, ...) \\\n [&] { \\\n const auto& the_type = TYPE; \\\n switch (the_type) { \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float64, double, __VA_ARGS__) \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float32, float, __VA_ARGS__) \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float16, float16, __VA_ARGS__) \\\n default: \\\n PARROTS_NOTSUPPORTED; \\\n } \\\n }()\n\n#endif // PARROTS_CPP_HELPER\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/parrots_cuda_helper.hpp", "content": "#ifndef PARROTS_CUDA_HELPER\n#define PARROTS_CUDA_HELPER\n\n#include \n#include \n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"common_cuda_helper.hpp\"\n#include \"parrots_cudawarpfunction.cuh\"\n\nusing namespace parrots;\nusing phalf = float16;\n\n#define __PHALF(x) (x.y)\n\n#define PARROTS_CUDA_CHECK(exp) \\\n do { \\\n cudaError_t err = exp; \\\n if (err != cudaSuccess) { \\\n fprintf(stderr, \"cudaCheckError() failed : %s\\n\", \\\n cudaGetErrorString(err)); \\\n exit(-1); \\\n } \\\n } while (0)\n\n#define PARROTS_PRIVATE_CASE_TYPE(prim_type, type, ...) \\\n case prim_type: { \\\n using scalar_t = type; \\\n return __VA_ARGS__(); \\\n }\n\n#define PARROTS_DISPATCH_FLOATING_TYPES(TYPE, ...) \\\n [&] { \\\n const auto& the_type = TYPE; \\\n switch (the_type) { \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float64, double, __VA_ARGS__) \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float32, float, __VA_ARGS__) \\\n default: \\\n PARROTS_NOTSUPPORTED; \\\n } \\\n }()\n\n#define PARROTS_DISPATCH_FLOATING_TYPES_AND_HALF(TYPE, ...) \\\n [&] { \\\n const auto& the_type = TYPE; \\\n switch (the_type) { \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float64, double, __VA_ARGS__) \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float32, float, __VA_ARGS__) \\\n PARROTS_PRIVATE_CASE_TYPE(Prim::Float16, float16, __VA_ARGS__) \\\n default: \\\n PARROTS_NOTSUPPORTED; \\\n } \\\n }()\n\n/** atomicAdd **/\n#if defined(__CUDA_ARCH__) && __CUDA_ARCH__ < 600\n\nstatic __inline__ __device__ double atomicAdd(double* address, double val) {\n unsigned long long int* address_as_ull = (unsigned long long int*)address;\n unsigned long long int old = *address_as_ull, assumed;\n if (val == 0.0) return __longlong_as_double(old);\n do {\n assumed = old;\n old = atomicCAS(address_as_ull, assumed,\n __double_as_longlong(val + __longlong_as_double(assumed)));\n } while (assumed != old);\n return __longlong_as_double(old);\n}\n\n#endif\n\nstatic __inline__ __device__ float16 atomicAdd(float16* address, float16 val) {\n unsigned int* aligned =\n (unsigned int*)((size_t)address - ((size_t)address & 2));\n unsigned int old = *aligned;\n unsigned int assumed;\n unsigned short old_as_us;\n do {\n assumed = old;\n old_as_us =\n (unsigned short)((size_t)address & 2 ? old >> 16 : old & 0xffff);\n\n#if __CUDACC_VER_MAJOR__ >= 9\n float16 tmp;\n tmp.x = old_as_us;\n float16 sum = tmp + val;\n unsigned short sum_as_us = sum.x;\n// half sum = __float2half_rn(__half2float(__ushort_as_half(old_as_us))\n// + (float)(val)); unsigned short sum_as_us = __half_as_ushort(sum);\n#else\n unsigned short sum_as_us =\n __float2half_rn(__half2float(old_as_us) + (float)(val));\n#endif\n\n unsigned int sum_as_ui = (size_t)address & 2\n ? (sum_as_us << 16) | (old & 0xffff)\n : (old & 0xffff0000) | sum_as_us;\n old = atomicCAS(aligned, assumed, sum_as_ui);\n } while (assumed != old);\n //__half_raw raw = {old_as_us};\n // return float16(raw);\n return *reinterpret_cast(&old_as_us);\n}\n#endif // PARROTS_CUDA_HELPER\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/pytorch_cpp_helper.hpp", "content": "#ifndef PYTORCH_CPP_HELPER\n#define PYTORCH_CPP_HELPER\n#include \n\n#include \n\nusing namespace at;\n\n#define CHECK_CUDA(x) \\\n TORCH_CHECK(x.device().is_cuda(), #x \" must be a CUDA tensor\")\n#define CHECK_CPU(x) \\\n TORCH_CHECK(!x.device().is_cuda(), #x \" must be a CPU tensor\")\n#define CHECK_CONTIGUOUS(x) \\\n TORCH_CHECK(x.is_contiguous(), #x \" must be contiguous\")\n#define CHECK_CUDA_INPUT(x) \\\n CHECK_CUDA(x); \\\n CHECK_CONTIGUOUS(x)\n#define CHECK_CPU_INPUT(x) \\\n CHECK_CPU(x); \\\n CHECK_CONTIGUOUS(x)\n\n#endif // PYTORCH_CPP_HELPER\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/pytorch_cuda_helper.hpp", "content": "#ifndef PYTORCH_CUDA_HELPER\n#define PYTORCH_CUDA_HELPER\n\n#include \n#include \n#include \n\n#include \n#include \n\n#include \"common_cuda_helper.hpp\"\n\nusing at::Half;\nusing at::Tensor;\nusing phalf = at::Half;\n\n#define __PHALF(x) (x)\n\n#endif // PYTORCH_CUDA_HELPER\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/common/pytorch_device_registry.hpp", "content": "#ifndef PYTORCH_DEVICE_REGISTRY_H\n#define PYTORCH_DEVICE_REGISTRY_H\n\n// Using is recommended in the official documentation in\n// https://pytorch.org/tutorials/advanced/cpp_extension.html#writing-the-c-op.\n// However, we use for compatibility with CUDA 9.0\n// Read https://github.com/pytorch/extension-cpp/issues/35 for more details.\n#include \n\n#include \n#include \n#include \n#include \n\ninline std::string GetDeviceStr(const at::Device& device) {\n std::string str = DeviceTypeName(device.type(), true);\n if (device.has_index()) {\n str.push_back(':');\n str.append(std::to_string(device.index()));\n }\n return str;\n}\n\n// Registry\ntemplate \nclass DeviceRegistry;\n\ntemplate \nclass DeviceRegistry {\n public:\n using FunctionType = Ret (*)(Args...);\n static const int MAX_DEVICE_TYPES =\n int8_t(at::DeviceType::COMPILE_TIME_MAX_DEVICE_TYPES);\n\n void Register(at::DeviceType device, FunctionType function) {\n funcs_[int8_t(device)] = function;\n }\n\n FunctionType Find(at::DeviceType device) const {\n return funcs_[int8_t(device)];\n }\n\n static DeviceRegistry& instance() {\n static DeviceRegistry inst;\n return inst;\n }\n\n private:\n DeviceRegistry() {\n for (size_t i = 0; i < MAX_DEVICE_TYPES; ++i) {\n funcs_[i] = nullptr;\n }\n };\n FunctionType funcs_[MAX_DEVICE_TYPES];\n};\n\n// get device of first tensor param\n\ntemplate , at::Tensor>::value,\n bool> = true>\nat::Device GetFirstTensorDevice(T&& t, Args&&... args) {\n return std::forward(t).device();\n}\ntemplate , at::Tensor>::value,\n bool> = true>\nat::Device GetFirstTensorDevice(T&& t, Args&&... args) {\n return GetFirstTensorDevice(std::forward(args)...);\n}\n\n// check device consistency\n\ninline std::pair CheckDeviceConsistency(\n const at::Device& device, int index) {\n return {index, device};\n}\n\ntemplate , at::Tensor>::value,\n bool> = true>\nstd::pair CheckDeviceConsistency(const at::Device& device,\n int index, T&& t,\n Args&&... args);\n\ntemplate , at::Tensor>::value,\n bool> = true>\nstd::pair CheckDeviceConsistency(const at::Device& device,\n int index, T&& t,\n Args&&... args) {\n auto new_device = std::forward(t).device();\n if (new_device.type() != device.type() ||\n new_device.index() != device.index()) {\n return {index, new_device};\n }\n return CheckDeviceConsistency(device, index + 1, std::forward(args)...);\n}\n\ntemplate <\n typename T, typename... Args,\n std::enable_if_t, at::Tensor>::value, bool>>\nstd::pair CheckDeviceConsistency(const at::Device& device,\n int index, T&& t,\n Args&&... args) {\n return CheckDeviceConsistency(device, index + 1, std::forward(args)...);\n}\n\n// dispatch\n\ntemplate \nauto Dispatch(const R& registry, const char* name, Args&&... args) {\n auto device = GetFirstTensorDevice(std::forward(args)...);\n auto inconsist =\n CheckDeviceConsistency(device, 0, std::forward(args)...);\n TORCH_CHECK(inconsist.first >= int(sizeof...(Args)), name, \": at param \",\n inconsist.first,\n \", inconsistent device: \", GetDeviceStr(inconsist.second).c_str(),\n \" vs \", GetDeviceStr(device).c_str(), \"\\n\")\n auto f_ptr = registry.Find(device.type());\n TORCH_CHECK(f_ptr != nullptr, name, \": implementation for device \",\n GetDeviceStr(device).c_str(), \" not found.\\n\")\n return f_ptr(std::forward(args)...);\n}\n\n// helper macro\n\n#define DEVICE_REGISTRY(key) DeviceRegistry::instance()\n\n#define REGISTER_DEVICE_IMPL(key, device, value) \\\n struct key##_##device##_registerer { \\\n key##_##device##_registerer() { \\\n DEVICE_REGISTRY(key).Register(at::k##device, value); \\\n } \\\n }; \\\n static key##_##device##_registerer _##key##_##device##_registerer;\n\n#define DISPATCH_DEVICE_IMPL(key, ...) \\\n Dispatch(DEVICE_REGISTRY(key), #key, __VA_ARGS__)\n\n#endif // PYTORCH_DEVICE_REGISTRY\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/corner_pool.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_CORNER_POOL_H\n#define ONNXRUNTIME_CORNER_POOL_H\n\n#include \n#include \n\nstruct MMCVCornerPoolKernel {\n public:\n MMCVCornerPoolKernel(Ort::CustomOpApi ort, const OrtKernelInfo* info)\n : ort_(ort) {\n mode_ = ort_.KernelInfoGetAttribute(info, \"mode\");\n }\n\n void Compute(OrtKernelContext* context);\n\n private:\n Ort::CustomOpApi ort_;\n\n int64_t mode_;\n};\n\nstruct MMCVCornerPoolCustomOp\n : Ort::CustomOpBase {\n void* CreateKernel(Ort::CustomOpApi api, const OrtKernelInfo* info) const {\n return new MMCVCornerPoolKernel(api, info);\n }\n\n const char* GetName() const { return \"MMCVCornerPool\"; }\n\n size_t GetInputTypeCount() const { return 1; }\n ONNXTensorElementDataType GetInputType(size_t) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n }\n\n size_t GetOutputTypeCount() const { return 1; }\n ONNXTensorElementDataType GetOutputType(size_t) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n }\n\n // force cpu\n const char* GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n }\n};\n#endif // ONNXRUNTIME_CORNER_POOL_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/corner_pool.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"corner_pool.h\"\n\n#include \"../ort_mmcv_utils.h\"\n\nvoid TopPoolForwardCPU(const float *input, float *output, const int batch_size,\n const int channels, const int height, const int width) {\n for (int n = 0; n < batch_size; n++) {\n int index_n = n * channels * width * height;\n for (int c = 0; c < channels; c++) {\n int index_n_c = index_n + c * width * height;\n for (int w = 0; w < width; w++) {\n // directly copy the most bottom value from input to output\n output[index_n_c + (height - 1) * width + w] =\n input[index_n_c + (height - 1) * width + w];\n // do top_pool\n for (int h = height - 2; h >= 0; h--) {\n output[index_n_c + h * width + w] =\n std::max(output[index_n_c + (h + 1) * width + w],\n input[index_n_c + h * width + w]);\n } // for h\n } // for w\n } // for c\n } // for n\n}\n\nvoid BottomPoolForwardCPU(const float *input, float *output,\n const int batch_size, const int channels,\n const int height, const int width) {\n for (int n = 0; n < batch_size; n++) {\n int index_n = n * channels * width * height;\n for (int c = 0; c < channels; c++) {\n int index_n_c = index_n + c * width * height;\n for (int w = 0; w < width; w++) {\n // directly copy the most top value from input to output\n output[index_n_c + w] = input[index_n_c + w];\n // do top_pool\n for (int h = 1; h < height; h++) {\n output[index_n_c + h * width + w] =\n std::max(output[index_n_c + (h - 1) * width + w],\n input[index_n_c + h * width + w]);\n } // for h\n } // for w\n } // for c\n } // for n\n}\n\nvoid LeftPoolForwardCPU(const float *input, float *output, const int batch_size,\n const int channels, const int height, const int width) {\n for (int n = 0; n < batch_size; n++) {\n int index_n = n * channels * width * height;\n for (int c = 0; c < channels; c++) {\n int index_n_c = index_n + c * width * height;\n for (int h = 0; h < height; h++) {\n // directly copy the most right value from input to output\n output[index_n_c + h * width + width - 1] =\n input[index_n_c + h * width + width - 1];\n // do left_pool\n for (int w = width - 2; w >= 0; w--) {\n output[index_n_c + h * width + w] =\n std::max(output[index_n_c + h * width + w + 1],\n input[index_n_c + h * width + w]);\n } // for w\n } // for h\n } // for c\n } // for n\n}\n\nvoid RightPoolForwardCPU(const float *input, float *output,\n const int batch_size, const int channels,\n const int height, const int width) {\n for (int n = 0; n < batch_size; n++) {\n int index_n = n * channels * width * height;\n for (int c = 0; c < channels; c++) {\n int index_n_c = index_n + c * width * height;\n for (int h = 0; h < height; h++) {\n // directly copy the most left value from input to output\n output[index_n_c + h * width] = input[index_n_c + h * width];\n // do right_pool\n for (int w = 1; w < width; w++) {\n output[index_n_c + h * width + w] =\n std::max(output[index_n_c + h * width + w - 1],\n input[index_n_c + h * width + w]);\n } // for w\n } // for h\n } // for c\n } // for n\n}\n\nvoid MMCVCornerPoolKernel::Compute(OrtKernelContext *context) {\n const int mode = int(mode_);\n typedef float T;\n const OrtValue *input = ort_.KernelContext_GetInput(context, 0);\n const T *input_data =\n reinterpret_cast(ort_.GetTensorData(input));\n\n // get output memory\n OrtTensorDimensions out_dimensions(ort_, input);\n OrtValue *output = ort_.KernelContext_GetOutput(\n context, 0, out_dimensions.data(), out_dimensions.size());\n T *output_data = ort_.GetTensorMutableData(output);\n\n // 'top': 0, 'bottom': 1, 'left': 2, 'right':3\n assert(mode == 0 || mode == 1 || mode == 2 || mode == 3);\n\n // do corner_pool\n int batch_size = out_dimensions.data()[0];\n int input_channels = out_dimensions.data()[1];\n int input_height = out_dimensions.data()[2];\n int input_width = out_dimensions.data()[3];\n if (mode == 0)\n TopPoolForwardCPU(input_data, output_data, batch_size, input_channels,\n input_height, input_width);\n else if (mode == 1)\n BottomPoolForwardCPU(input_data, output_data, batch_size, input_channels,\n input_height, input_width);\n else if (mode == 2)\n LeftPoolForwardCPU(input_data, output_data, batch_size, input_channels,\n input_height, input_width);\n else\n RightPoolForwardCPU(input_data, output_data, batch_size, input_channels,\n input_height, input_width);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"deform_conv.h\"\n\n#include \n#include \n\n#include \"../ort_mmcv_utils.h\"\n\nvoid gemm_ref_fp32_deform(const float *A, const float *B, const float *V,\n const float *H, const int32_t trans_A,\n const int32_t trans_B, const int32_t M,\n const int32_t N, const int32_t K, const float alpha,\n const float beta, float *Y) {\n if (!trans_A && !trans_B) { // MK, KN; NN\n for (int64_t m = 0; m < M; ++m) {\n for (int64_t n = 0; n < N; ++n) {\n float y = 0.0f;\n for (int64_t k = 0; k < K; ++k) {\n y += A[m * K + k] * B[k * N + n];\n }\n y *= alpha;\n if (V) y += beta * V[n];\n if (H) y += beta * H[m * N + n];\n Y[m * N + n] = y;\n }\n }\n }\n if (trans_A && !trans_B) { // KM, KN; TN\n for (int64_t m = 0; m < M; ++m) {\n for (int64_t n = 0; n < N; ++n) {\n float y = 0.0f;\n for (int64_t k = 0; k < K; ++k) {\n y += A[k * M + m] * B[k * N + n];\n }\n y *= alpha;\n if (V) y += beta * V[n];\n if (H) y += beta * H[m * N + n];\n Y[m * N + n] = y;\n }\n }\n }\n if (trans_A && trans_B) { // KM, NK; TT\n for (int64_t m = 0; m < M; ++m) {\n for (int64_t n = 0; n < N; ++n) {\n float y = 0.0f;\n for (int64_t k = 0; k < K; ++k) {\n y += A[k * M + m] * B[n * K + k];\n }\n y *= alpha;\n if (V) y += beta * V[n];\n if (H) y += beta * H[m * N + n];\n Y[m * N + n] = y;\n }\n }\n }\n if (!trans_A && trans_B) { // MK, NK; NT\n for (int64_t m = 0; m < M; ++m) {\n for (int64_t n = 0; n < N; ++n) {\n float y = 0.0f;\n for (int64_t k = 0; k < K; ++k) {\n y += A[m * K + k] * B[n * K + k];\n }\n y *= alpha;\n if (V) y += beta * V[n];\n if (H) y += beta * H[m * N + n];\n Y[m * N + n] = y;\n }\n }\n }\n}\n\nfloat bilinear_interpolate(const float *src, const int64_t src_h,\n const int64_t src_w, const float h, const float w) {\n if (h <= -1 || src_h <= h || w <= -1 || src_w <= w) {\n return 0;\n }\n\n int64_t h_low = floor(h);\n int64_t w_low = floor(w);\n int64_t h_high = h_low + 1;\n int64_t w_high = w_low + 1;\n\n float lh = h - h_low;\n float lw = w - w_low;\n float hh = 1 - lh;\n float hw = 1 - lw;\n\n float v1 = 0;\n if (h_low >= 0 && w_low >= 0) v1 = src[h_low * src_w + w_low];\n float v2 = 0;\n if (h_low >= 0 && w_high <= src_w - 1) v2 = src[h_low * src_w + w_high];\n float v3 = 0;\n if (h_high <= src_h - 1 && w_low >= 0) v3 = src[h_high * src_w + w_low];\n float v4 = 0;\n if (h_high <= src_h - 1 && w_high <= src_w - 1)\n v4 = src[h_high * src_w + w_high];\n\n float w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n\n float val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n return val;\n}\n\nvoid deformable_im2col(const float *input, const float *offset,\n const int64_t src_h, const int64_t src_w,\n const int64_t kernel_h, const int64_t kernel_w,\n const int64_t pad_h, const int64_t pad_w,\n const int64_t stride_h, const int64_t stride_w,\n const int64_t dilation_h, const int64_t dilation_w,\n const int64_t channels, const int64_t offset_groups,\n const int64_t dst_h, const int64_t dst_w,\n float *columns) {\n const int64_t indices = channels * dst_h * dst_w;\n for (int64_t index = 0; index != indices; ++index) {\n const int64_t w_col = index % dst_w;\n const int64_t h_col = (index / dst_w) % dst_h;\n const int64_t c_im = index / (dst_w * dst_h);\n const int64_t c_col = c_im * kernel_h * kernel_w;\n\n int64_t c_per_offset_grp = channels / offset_groups;\n const int64_t grp_idx = c_im / c_per_offset_grp;\n auto columns_ptr =\n columns + (c_col * (dst_h * dst_w) + h_col * dst_w + w_col);\n auto input_ptr = input + c_im * (src_h * src_w);\n auto offset_ptr =\n offset + grp_idx * 2 * kernel_h * kernel_w * dst_h * dst_w;\n\n for (int64_t kh = 0; kh < kernel_h; ++kh) {\n for (int64_t kw = 0; kw < kernel_w; ++kw) {\n const int data_offset_h_ptr =\n ((2 * (kh * kernel_w + kw)) * dst_h + h_col) * dst_w + w_col;\n const int data_offset_w_ptr =\n ((2 * (kh * kernel_w + kw) + 1) * dst_h + h_col) * dst_w + w_col;\n\n const float offset_h = offset_ptr[data_offset_h_ptr];\n const float offset_w = offset_ptr[data_offset_w_ptr];\n const float ih =\n (h_col * stride_h - pad_h) + kh * dilation_h + offset_h;\n const float iw =\n (w_col * stride_w - pad_w) + kw * dilation_w + offset_w;\n *columns_ptr = bilinear_interpolate(input_ptr, src_h, src_w, ih, iw);\n columns_ptr += dst_h * dst_w;\n }\n }\n }\n}\n\nvoid deformable_conv_forward(\n const float *src, const float *offset, const float *filter,\n const int64_t batch, const int64_t src_c, const int64_t src_h,\n const int64_t src_w, const int64_t dst_c, const int64_t dst_h,\n const int64_t dst_w, const int64_t group, const int64_t offset_group,\n const int64_t channels, const int64_t num_output, const int64_t kernel_h,\n const int64_t kernel_w, const int64_t stride_h, const int64_t stride_w,\n const int64_t pad_h, const int64_t pad_w, const int64_t dilation_h,\n const int64_t dilation_w, float *columns, float *dst) {\n const int64_t ic_per_gp = channels / group;\n const int64_t oc_per_gp = num_output / group;\n for (int64_t b = 0; b < batch; ++b) {\n for (int64_t g = 0; g < group; ++g) {\n deformable_im2col(\n src + b * src_c * src_h * src_w + g * ic_per_gp * src_h * src_w,\n offset + b * offset_group * 2 * kernel_h * kernel_w * dst_h * dst_w,\n src_h, src_w, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, ic_per_gp, offset_group, dst_h, dst_w,\n columns);\n float *dst_ptr =\n dst + b * dst_c * dst_h * dst_w + g * oc_per_gp * dst_h * dst_w;\n\n memset(dst_ptr, 0.0f, sizeof(float) * oc_per_gp * dst_h * dst_w);\n\n gemm_ref_fp32_deform(\n filter + g * oc_per_gp * ic_per_gp * kernel_h * kernel_w, columns,\n nullptr, dst_ptr, 0, 0, oc_per_gp, dst_h * dst_w,\n ic_per_gp * kernel_h * kernel_w, 1.0f, 1.0f, dst_ptr);\n }\n }\n}\n\nMMCVDeformConvKernel::MMCVDeformConvKernel(OrtApi api,\n const OrtKernelInfo *info)\n : api_(api), ort_(api_), info_(info) {\n std::vector stride =\n ort_.KernelInfoGetAttribute>(info, \"stride\");\n stride_height_ = stride[0];\n stride_width_ = stride[1];\n std::vector padding =\n ort_.KernelInfoGetAttribute>(info, \"padding\");\n padding_height_ = padding[0];\n padding_width_ = padding[1];\n std::vector dilation =\n ort_.KernelInfoGetAttribute>(info, \"dilation\");\n dilation_height_ = dilation[0];\n dilation_width_ = dilation[1];\n deformable_group_ =\n ort_.KernelInfoGetAttribute(info, \"deform_groups\");\n group_ = ort_.KernelInfoGetAttribute(info, \"groups\");\n\n // create allocator\n allocator_ = Ort::AllocatorWithDefaultOptions();\n}\n\nvoid MMCVDeformConvKernel::Compute(OrtKernelContext *context) {\n const int64_t stride_height = stride_height_;\n const int64_t stride_width = stride_width_;\n const int64_t padding_height = padding_height_;\n const int64_t padding_width = padding_width_;\n const int64_t dilation_height = dilation_height_;\n const int64_t dilation_width = dilation_width_;\n const int64_t deformable_group = deformable_group_;\n const int64_t group = group_;\n\n const OrtValue *input = ort_.KernelContext_GetInput(context, 0);\n const float *input_data =\n reinterpret_cast(ort_.GetTensorData(input));\n\n const OrtValue *offset = ort_.KernelContext_GetInput(context, 1);\n const float *offset_data =\n reinterpret_cast(ort_.GetTensorData(offset));\n\n const OrtValue *filter = ort_.KernelContext_GetInput(context, 2);\n const float *filter_data =\n reinterpret_cast(ort_.GetTensorData(filter));\n\n OrtTensorDimensions input_dims(ort_, input);\n OrtTensorDimensions filter_dims(ort_, filter);\n\n int64_t batch_size = input_dims[0];\n int64_t in_channels = input_dims[1];\n int64_t in_height = input_dims[2];\n int64_t in_width = input_dims[3];\n int64_t out_channels = filter_dims[0];\n int64_t kernel_height = filter_dims[2];\n int64_t kernel_width = filter_dims[3];\n\n // get output memory\n int64_t out_height = floor((in_height + 2 * padding_height -\n dilation_height * (kernel_height - 1) - 1) /\n stride_height +\n 1);\n int64_t out_width = floor(\n (in_width + 2 * padding_width - dilation_width * (kernel_width - 1) - 1) /\n stride_width +\n 1);\n\n std::vector output_dims = {batch_size, out_channels, out_height,\n out_width};\n\n OrtValue *output = ort_.KernelContext_GetOutput(\n context, 0, output_dims.data(), output_dims.size());\n float *out_ptr = ort_.GetTensorMutableData(output);\n\n // allocate tmp memory\n int64_t column_len = (in_channels / group) * kernel_height * kernel_width *\n out_height * out_width;\n float *columns = (float *)allocator_.Alloc(sizeof(float) * column_len);\n deformable_conv_forward(\n input_data, offset_data, filter_data, batch_size, in_channels, in_height,\n in_width, out_channels, out_height, out_width, group, deformable_group,\n in_channels, out_channels, kernel_height, kernel_width, stride_height,\n stride_width, padding_height, padding_width, dilation_height,\n dilation_width, columns, out_ptr);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/gridSample.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n\n#include \"../ort_mmcv_utils.h\"\n#include \"grid_sample.h\"\n\n#define MIN(a, b) (((a) < (b)) ? (a) : (b))\n#define MAX(a, b) (((a) < (b)) ? (b) : (a))\n#define CLIP_COORDINATES(in, out, clip_limit) \\\n out = MIN((clip_limit - 1), MAX(in, 0))\n\n// modified from\n// https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/GridSampler.cpp\n\nGridSampleKernel::GridSampleKernel(OrtApi api, const OrtKernelInfo *info)\n : api_(api), ort_(api_), info_(info) {\n align_corners_ = ort_.KernelInfoGetAttribute(info, \"align_corners\");\n interpolation_mode_ =\n ort_.KernelInfoGetAttribute(info, \"interpolation_mode\");\n padding_mode_ = ort_.KernelInfoGetAttribute(info, \"padding_mode\");\n\n allocator_ = Ort::AllocatorWithDefaultOptions();\n}\n\nenum GridSamplerInterpolation { Bilinear = 0, Nearest = 1, Bicubic = 2 };\nenum GridSamplerPadding { Zeros = 0, Border = 1, Reflection = 2 };\n\ntemplate \nstatic inline scalar_t grid_sampler_unnormalize(scalar_t coord, int64_t size,\n bool align_corners) {\n if (align_corners) {\n return ((coord + 1) / 2) * (size - 1);\n } else {\n return ((coord + 1) * size - 1) / 2;\n }\n}\n\n// Clips coordinates to between 0 and clip_limit - 1\ntemplate \nstatic inline scalar_t clip_coordinates(scalar_t in, int64_t clip_limit) {\n return std::min(static_cast(clip_limit - 1),\n std::max(in, static_cast(0)));\n}\n\n// Reflects coordinates until they fall between low and high (inclusive).\n// The bounds are passed as twice their value so that half-integer values\n// can be represented as ints.\ntemplate \nstatic inline scalar_t reflect_coordinates(scalar_t in, int64_t twice_low,\n int64_t twice_high) {\n if (twice_low == twice_high) {\n return static_cast(0);\n }\n scalar_t min = static_cast(twice_low) / 2;\n scalar_t span = static_cast(twice_high - twice_low) / 2;\n in = std::fabs(in - min);\n // `fmod` returns same sign as `in`, which is positive after the `fabs` above.\n scalar_t extra = std::fmod(in, span);\n int flips = static_cast(std::floor(in / span));\n if (flips % 2 == 0) {\n return extra + min;\n } else {\n return span - extra + min;\n }\n}\n\ntemplate \nstatic inline scalar_t compute_coordinates(scalar_t coord, int64_t size,\n int64_t padding_mode,\n bool align_corners) {\n if (padding_mode == GridSamplerPadding::Border) {\n coord = clip_coordinates(coord, size);\n } else if (padding_mode == GridSamplerPadding::Reflection) {\n if (align_corners) {\n coord = reflect_coordinates(coord, 0, 2 * (size - 1));\n } else {\n coord = reflect_coordinates(coord, -1, 2 * size - 1);\n }\n coord = clip_coordinates(coord, size);\n }\n return coord;\n}\n\n// Computes the pixel source index value for a grid coordinate\ntemplate \nstatic inline scalar_t grid_sampler_compute_source_index(scalar_t coord,\n int64_t size,\n int64_t padding_mode,\n bool align_corners) {\n coord = grid_sampler_unnormalize(coord, size, align_corners);\n coord = compute_coordinates(coord, size, padding_mode, align_corners);\n return coord;\n}\n\nstatic inline bool within_bounds_2d(int64_t h, int64_t w, int64_t H,\n int64_t W) {\n return h >= 0 && h < H && w >= 0 && w < W;\n}\n\ntemplate \nstatic inline scalar_t get_value_bounded(const scalar_t *data, scalar_t x,\n scalar_t y, int64_t W, int64_t H,\n int64_t sW, int64_t sH,\n int64_t padding_mode,\n bool align_corners) {\n x = compute_coordinates(x, W, padding_mode, align_corners);\n y = compute_coordinates(y, H, padding_mode, align_corners);\n\n int64_t ix = static_cast(x);\n int64_t iy = static_cast(y);\n\n if (within_bounds_2d(iy, ix, H, W)) {\n return data[iy * sH + ix * sW];\n }\n return static_cast(0);\n}\n\ntemplate \nstatic inline scalar_t cubic_convolution1(scalar_t x, scalar_t A) {\n return ((A + 2) * x - (A + 3)) * x * x + 1;\n}\n\ntemplate \nstatic inline scalar_t cubic_convolution2(scalar_t x, scalar_t A) {\n return ((A * x - 5 * A) * x + 8 * A) * x - 4 * A;\n}\n\ntemplate \nstatic inline void get_cubic_upsample_coefficients(scalar_t coeffs[4],\n scalar_t t) {\n scalar_t A = -0.75;\n\n scalar_t x1 = t;\n coeffs[0] = cubic_convolution2(x1 + 1.0, A);\n coeffs[1] = cubic_convolution1(x1, A);\n\n // opposite coefficients\n scalar_t x2 = 1.0 - t;\n coeffs[2] = cubic_convolution1(x2, A);\n coeffs[3] = cubic_convolution2(x2 + 1.0, A);\n}\n\ntemplate \nstatic inline scalar_t cubic_interp1d(scalar_t x0, scalar_t x1, scalar_t x2,\n scalar_t x3, scalar_t t) {\n scalar_t coeffs[4];\n get_cubic_upsample_coefficients(coeffs, t);\n\n return x0 * coeffs[0] + x1 * coeffs[1] + x2 * coeffs[2] + x3 * coeffs[3];\n}\n\nvoid GridSampleKernel::Compute(OrtKernelContext *context) {\n const bool align_corners = align_corners_;\n const int64_t padding_mode = padding_mode_;\n const int64_t interpolation_mode = interpolation_mode_;\n\n const OrtValue *input = ort_.KernelContext_GetInput(context, 0);\n const float *input_data =\n reinterpret_cast(ort_.GetTensorData(input));\n\n const OrtValue *grid = ort_.KernelContext_GetInput(context, 1);\n const float *grid_data =\n reinterpret_cast(ort_.GetTensorData(grid));\n\n OrtTensorDimensions input_dims(ort_, input);\n OrtTensorDimensions grid_dims(ort_, grid);\n int64_t N = input_dims[0];\n int64_t C = input_dims[1];\n int64_t inp_H = input_dims[2];\n int64_t inp_W = input_dims[3];\n int64_t out_H = grid_dims[1];\n int64_t out_W = grid_dims[2];\n\n std::vector output_dims = {N, C, out_H, out_W};\n OrtValue *output = ort_.KernelContext_GetOutput(\n context, 0, output_dims.data(), output_dims.size());\n float *out_ptr = ort_.GetTensorMutableData(output);\n\n int64_t inp_sN = input_dims[1] * input_dims[2] * input_dims[3];\n int64_t inp_sC = input_dims[2] * input_dims[3];\n int64_t inp_sH = input_dims[3];\n int64_t inp_sW = 1;\n int64_t grid_sN = grid_dims[1] * grid_dims[2] * grid_dims[3];\n int64_t grid_sH = grid_dims[2] * grid_dims[3];\n int64_t grid_sW = grid_dims[3];\n int64_t grid_sCoor = 1;\n int64_t out_sN = output_dims[1] * output_dims[2] * output_dims[3];\n int64_t out_sC = output_dims[2] * output_dims[3];\n int64_t out_sH = output_dims[3];\n int64_t out_sW = 1;\n\n // loop over each output pixel\n for (int64_t n = 0; n < N; ++n) {\n const float *grid_ptr_N = grid_data + n * grid_sN;\n const float *inp_ptr_N = input_data + n * inp_sN;\n for (int64_t h = 0; h < out_H; ++h) {\n for (int64_t w = 0; w < out_W; ++w) {\n const float *grid_ptr_NHW = grid_ptr_N + h * grid_sH + w * grid_sW;\n float x = *grid_ptr_NHW;\n float y = grid_ptr_NHW[grid_sCoor];\n\n float ix = grid_sampler_compute_source_index(x, inp_W, padding_mode,\n align_corners);\n float iy = grid_sampler_compute_source_index(y, inp_H, padding_mode,\n align_corners);\n\n if (interpolation_mode == GridSamplerInterpolation::Bilinear) {\n // get corner pixel values from (x, y)\n // for 4d, we use north-east-south-west\n int64_t ix_nw = static_cast(std::floor(ix));\n int64_t iy_nw = static_cast(std::floor(iy));\n\n int64_t ix_ne = ix_nw + 1;\n int64_t iy_ne = iy_nw;\n\n int64_t ix_sw = ix_nw;\n int64_t iy_sw = iy_nw + 1;\n\n int64_t ix_se = ix_nw + 1;\n int64_t iy_se = iy_nw + 1;\n\n // get surfaces to each neighbor:\n float nw = (ix_se - ix) * (iy_se - iy);\n float ne = (ix - ix_sw) * (iy_sw - iy);\n float sw = (ix_ne - ix) * (iy - iy_ne);\n float se = (ix - ix_nw) * (iy - iy_nw);\n\n // calculate bilinear weighted pixel value and set output pixel\n const float *inp_ptr_NC = inp_ptr_N;\n float *out_ptr_NCHW = out_ptr + n * out_sN + h * out_sH + w * out_sW;\n for (int64_t c = 0; c < C;\n ++c, out_ptr_NCHW += out_sC, inp_ptr_NC += inp_sC) {\n auto res = static_cast(0);\n if (within_bounds_2d(iy_nw, ix_nw, inp_H, inp_W)) {\n res += inp_ptr_NC[iy_nw * inp_sH + ix_nw * inp_sW] * nw;\n }\n if (within_bounds_2d(iy_ne, ix_ne, inp_H, inp_W)) {\n res += inp_ptr_NC[iy_ne * inp_sH + ix_ne * inp_sW] * ne;\n }\n if (within_bounds_2d(iy_sw, ix_sw, inp_H, inp_W)) {\n res += inp_ptr_NC[iy_sw * inp_sH + ix_sw * inp_sW] * sw;\n }\n if (within_bounds_2d(iy_se, ix_se, inp_H, inp_W)) {\n res += inp_ptr_NC[iy_se * inp_sH + ix_se * inp_sW] * se;\n }\n *out_ptr_NCHW = res;\n }\n } else if (interpolation_mode == GridSamplerInterpolation::Nearest) {\n int64_t ix_nearest = static_cast(std::nearbyint(ix));\n int64_t iy_nearest = static_cast(std::nearbyint(iy));\n\n // assign nearest neighbor pixel value to output pixel\n float *out_ptr_NCHW = out_ptr + n * out_sN + h * out_sH + w * out_sW;\n const float *inp_ptr_NC = inp_ptr_N;\n for (int64_t c = 0; c < C;\n ++c, out_ptr_NCHW += out_sC, inp_ptr_NC += inp_sC) {\n if (within_bounds_2d(iy_nearest, ix_nearest, inp_H, inp_W)) {\n *out_ptr_NCHW =\n inp_ptr_NC[iy_nearest * inp_sH + ix_nearest * inp_sW];\n } else {\n *out_ptr_NCHW = static_cast(0);\n }\n }\n } else if (interpolation_mode == GridSamplerInterpolation::Bicubic) {\n // grid_sampler_compute_source_index will \"clip the value\" of idx\n // depends on the padding,\n // which would cause calculation to be wrong,\n // for example x = -0.1 -> ix = 0 for zero padding, but in bicubic ix\n // = floor(x) = -1\n // There would be more problem in reflection padding, since the -1 and\n // +1 direction is not fixed in boundary condition\n ix = grid_sampler_unnormalize(x, inp_W, align_corners);\n iy = grid_sampler_unnormalize(y, inp_H, align_corners);\n\n float ix_nw = std::floor(ix);\n float iy_nw = std::floor(iy);\n\n const float tx = ix - ix_nw;\n const float ty = iy - iy_nw;\n\n const float *inp_ptr_NC = inp_ptr_N;\n float *out_ptr_NCHW = out_ptr + n * out_sN + h * out_sH + w * out_sW;\n for (int64_t c = 0; c < C;\n ++c, out_ptr_NCHW += out_sC, inp_ptr_NC += inp_sC) {\n float coefficients[4];\n\n // Interpolate 4 values in the x direction\n for (int64_t i = 0; i < 4; ++i) {\n coefficients[i] = cubic_interp1d(\n get_value_bounded(inp_ptr_NC, ix_nw - 1, iy_nw - 1 + i,\n inp_W, inp_H, inp_sW, inp_sH,\n padding_mode, align_corners),\n get_value_bounded(inp_ptr_NC, ix_nw + 0, iy_nw - 1 + i,\n inp_W, inp_H, inp_sW, inp_sH,\n padding_mode, align_corners),\n get_value_bounded(inp_ptr_NC, ix_nw + 1, iy_nw - 1 + i,\n inp_W, inp_H, inp_sW, inp_sH,\n padding_mode, align_corners),\n get_value_bounded(inp_ptr_NC, ix_nw + 2, iy_nw - 1 + i,\n inp_W, inp_H, inp_sW, inp_sH,\n padding_mode, align_corners),\n tx);\n }\n\n // Interpolate in the y direction\n *out_ptr_NCHW =\n cubic_interp1d(coefficients[0], coefficients[1],\n coefficients[2], coefficients[3], ty);\n }\n }\n }\n }\n }\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/modulated_deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"modulated_deform_conv.h\"\n\n#include \n#include \n\n#include \"../ort_mmcv_utils.h\"\n\nfloat bilinear_interpolate_2d(const float *src, const int64_t src_h,\n const int64_t src_w, const float h,\n const float w) {\n if (h <= -1 || src_h <= h || w <= -1 || src_w <= w) {\n return 0;\n }\n\n int64_t h_low = floor(h);\n int64_t w_low = floor(w);\n int64_t h_high = h_low + 1;\n int64_t w_high = w_low + 1;\n\n float lh = h - h_low;\n float lw = w - w_low;\n float hh = 1 - lh;\n float hw = 1 - lw;\n\n float v1 = 0;\n if (h_low >= 0 && w_low >= 0) v1 = src[h_low * src_w + w_low];\n float v2 = 0;\n if (h_low >= 0 && w_high <= src_w - 1) v2 = src[h_low * src_w + w_high];\n float v3 = 0;\n if (h_high <= src_h - 1 && w_low >= 0) v3 = src[h_high * src_w + w_low];\n float v4 = 0;\n if (h_high <= src_h - 1 && w_high <= src_w - 1)\n v4 = src[h_high * src_w + w_high];\n\n float w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n\n float val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n return val;\n}\n\n// output: (channels * kernel_h * kernel_w, dst_h * dst_w)\nvoid deformable_im2col_2d(const float *input, const float *offset,\n const float *mask, const int64_t src_h,\n const int64_t src_w, const int64_t kernel_h,\n const int64_t kernel_w, const int64_t pad_h,\n const int64_t pad_w, const int64_t stride_h,\n const int64_t stride_w, const int64_t dilation_h,\n const int64_t dilation_w, const int64_t channels,\n const int64_t offset_groups, const int64_t dst_h,\n const int64_t dst_w, const bool use_mask,\n float *columns) {\n const int64_t workload = channels * dst_h * dst_w;\n for (int64_t index = 0; index != workload; ++index) {\n const int64_t ow = index % dst_w;\n const int64_t oh = (index / dst_w) % dst_h;\n const int64_t ic = index / (dst_w * dst_h);\n const int64_t oc = ic * kernel_h * kernel_w;\n\n int64_t c_per_offset_grp = channels / offset_groups;\n const int64_t grp_idx = ic / c_per_offset_grp;\n\n auto columns_ptr = columns + (oc * (dst_h * dst_w) + oh * dst_w + ow);\n auto input_ptr = input + ic * (src_h * src_w);\n auto offset_ptr =\n offset + grp_idx * 2 * kernel_h * kernel_w * dst_h * dst_w;\n auto mask_ptr = mask;\n if (use_mask) {\n mask_ptr += grp_idx * kernel_h * kernel_w * dst_h * dst_w;\n }\n\n for (int64_t kh = 0; kh < kernel_h; ++kh) {\n for (int64_t kw = 0; kw < kernel_w; ++kw) {\n const int64_t mask_idx = kh * kernel_w + kw;\n const int64_t offset_idx = 2 * mask_idx;\n\n float mask_value = 1;\n if (use_mask) {\n mask_value = mask_ptr[mask_idx * (dst_h * dst_w) + oh * dst_w + ow];\n }\n\n const float offset_h =\n offset_ptr[offset_idx * (dst_h * dst_w) + oh * dst_w + ow];\n const float offset_w =\n offset_ptr[(offset_idx + 1) * (dst_h * dst_w) + oh * dst_w + ow];\n const float ih = (oh * stride_h - pad_h) + kh * dilation_h + offset_h;\n const float iw = (ow * stride_w - pad_w) + kw * dilation_w + offset_w;\n *columns_ptr = mask_value *\n bilinear_interpolate_2d(input_ptr, src_h, src_w, ih, iw);\n columns_ptr += dst_h * dst_w;\n }\n }\n }\n}\n\nvoid gemm_ref_fp32(const float *A, const float *B, const float *V,\n const float *H, const int32_t trans_A, const int32_t trans_B,\n const int32_t M, const int32_t N, const int32_t K,\n const float alpha, const float beta, float *Y) {\n if (!trans_A && !trans_B) { // MK, KN; NN\n for (int64_t m = 0; m < M; ++m) {\n for (int64_t n = 0; n < N; ++n) {\n float y = 0.0f;\n for (int64_t k = 0; k < K; ++k) {\n y += A[m * K + k] * B[k * N + n];\n }\n y *= alpha;\n if (V) y += beta * V[n];\n if (H) y += beta * H[m * N + n];\n Y[m * N + n] = y;\n }\n }\n }\n if (trans_A && !trans_B) { // KM, KN; TN\n for (int64_t m = 0; m < M; ++m) {\n for (int64_t n = 0; n < N; ++n) {\n float y = 0.0f;\n for (int64_t k = 0; k < K; ++k) {\n y += A[k * M + m] * B[k * N + n];\n }\n y *= alpha;\n if (V) y += beta * V[n];\n if (H) y += beta * H[m * N + n];\n Y[m * N + n] = y;\n }\n }\n }\n if (trans_A && trans_B) { // KM, NK; TT\n for (int64_t m = 0; m < M; ++m) {\n for (int64_t n = 0; n < N; ++n) {\n float y = 0.0f;\n for (int64_t k = 0; k < K; ++k) {\n y += A[k * M + m] * B[n * K + k];\n }\n y *= alpha;\n if (V) y += beta * V[n];\n if (H) y += beta * H[m * N + n];\n Y[m * N + n] = y;\n }\n }\n }\n if (!trans_A && trans_B) { // MK, NK; NT\n for (int64_t m = 0; m < M; ++m) {\n for (int64_t n = 0; n < N; ++n) {\n float y = 0.0f;\n for (int64_t k = 0; k < K; ++k) {\n y += A[m * K + k] * B[n * K + k];\n }\n y *= alpha;\n if (V) y += beta * V[n];\n if (H) y += beta * H[m * N + n];\n Y[m * N + n] = y;\n }\n }\n }\n}\n\nvoid deformable_conv2d_ref_fp32(\n const float *src, const float *offset, const float *mask,\n const float *filter, const float *bias, const int64_t batch,\n const int64_t src_c, const int64_t src_h, const int64_t src_w,\n const int64_t dst_c, const int64_t dst_h, const int64_t dst_w,\n const int64_t group, const int64_t offset_group, const int64_t channels,\n const int64_t num_output, const int64_t kernel_h, const int64_t kernel_w,\n const int64_t stride_h, const int64_t stride_w, const int64_t pad_h,\n const int64_t pad_w, const int64_t dilation_h, const int64_t dilation_w,\n float *columns, float *dst) {\n const int64_t ic_per_gp = channels / group;\n const int64_t oc_per_gp = num_output / group;\n\n for (int64_t b = 0; b < batch; ++b) {\n for (int64_t g = 0; g < group; ++g) {\n deformable_im2col_2d(\n src + b * src_c * src_h * src_w + g * ic_per_gp * src_h * src_w,\n offset + b * offset_group * 2 * kernel_h * kernel_w * dst_h * dst_w,\n mask + b * offset_group * kernel_h * kernel_w * dst_h * dst_w, src_h,\n src_w, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, ic_per_gp, offset_group, dst_h, dst_w,\n mask != nullptr, columns);\n float *dst_ptr =\n dst + b * dst_c * dst_h * dst_w + g * oc_per_gp * dst_h * dst_w;\n if (bias != nullptr) {\n const float *bias_ptr = bias + g * oc_per_gp;\n for (int64_t oc = 0; oc < oc_per_gp; ++oc) {\n for (int64_t hw = 0; hw < dst_h * dst_w; ++hw) {\n dst_ptr[oc * dst_h * dst_w + hw] = bias_ptr[oc];\n }\n }\n } else {\n memset(dst_ptr, 0.0f, sizeof(float) * oc_per_gp * dst_h * dst_w);\n }\n gemm_ref_fp32(filter + g * oc_per_gp * ic_per_gp * kernel_h * kernel_w,\n columns, nullptr, dst_ptr, 0, 0, oc_per_gp, dst_h * dst_w,\n ic_per_gp * kernel_h * kernel_w, 1.0f, 1.0f, dst_ptr);\n }\n }\n}\n\nMMCVModulatedDeformConvKernel::MMCVModulatedDeformConvKernel(\n OrtApi api, const OrtKernelInfo *info)\n : api_(api), ort_(api_), info_(info) {\n std::vector stride =\n ort_.KernelInfoGetAttribute>(info, \"stride\");\n stride_height_ = stride[0];\n stride_width_ = stride[1];\n std::vector padding =\n ort_.KernelInfoGetAttribute>(info, \"padding\");\n padding_height_ = padding[0];\n padding_width_ = padding[1];\n std::vector dilation =\n ort_.KernelInfoGetAttribute>(info, \"dilation\");\n dilation_height_ = dilation[0];\n dilation_width_ = dilation[1];\n deformable_group_ =\n ort_.KernelInfoGetAttribute(info, \"deform_groups\");\n group_ = ort_.KernelInfoGetAttribute(info, \"groups\");\n\n // create allocator\n allocator_ = Ort::AllocatorWithDefaultOptions();\n}\n\nvoid MMCVModulatedDeformConvKernel::Compute(OrtKernelContext *context) {\n const int64_t stride_height = stride_height_;\n const int64_t stride_width = stride_width_;\n const int64_t padding_height = padding_height_;\n const int64_t padding_width = padding_width_;\n const int64_t dilation_height = dilation_height_;\n const int64_t dilation_width = dilation_width_;\n const int64_t deformable_group = deformable_group_;\n const int64_t group = group_;\n\n const OrtValue *input = ort_.KernelContext_GetInput(context, 0);\n const float *input_data =\n reinterpret_cast(ort_.GetTensorData(input));\n\n const OrtValue *offset = ort_.KernelContext_GetInput(context, 1);\n const float *offset_data =\n reinterpret_cast(ort_.GetTensorData(offset));\n\n const OrtValue *mask = ort_.KernelContext_GetInput(context, 2);\n const float *mask_data =\n reinterpret_cast(ort_.GetTensorData(mask));\n\n const OrtValue *filter = ort_.KernelContext_GetInput(context, 3);\n const float *filter_data =\n reinterpret_cast(ort_.GetTensorData(filter));\n\n const OrtValue *bias = ort_.KernelContext_GetInput(context, 4);\n const float *bias_data =\n (bias != nullptr)\n ? reinterpret_cast(ort_.GetTensorData(bias))\n : nullptr;\n // const float *bias_data = nullptr;\n\n OrtTensorDimensions input_dims(ort_, input);\n OrtTensorDimensions filter_dims(ort_, filter);\n\n int64_t batch = input_dims[0];\n int64_t channels = input_dims[1];\n int64_t in_height = input_dims[2];\n int64_t in_width = input_dims[3];\n int64_t num_output = filter_dims[0];\n int64_t kernel_height = filter_dims[2];\n int64_t kernel_width = filter_dims[3];\n\n // get output memory\n int64_t out_height = floor((in_height + 2 * padding_height -\n dilation_height * (kernel_height - 1) - 1) /\n stride_height +\n 1);\n int64_t out_width = floor(\n (in_width + 2 * padding_width - dilation_width * (kernel_width - 1) - 1) /\n stride_width +\n 1);\n\n std::vector output_dims = {batch, num_output, out_height, out_width};\n OrtValue *output = ort_.KernelContext_GetOutput(\n context, 0, output_dims.data(), output_dims.size());\n float *out_ptr = ort_.GetTensorMutableData(output);\n\n // allocate tmp memory\n int64_t column_len = (channels / group) * kernel_height * kernel_width *\n out_height * out_width;\n float *columns = (float *)allocator_.Alloc(sizeof(float) * column_len);\n\n deformable_conv2d_ref_fp32(\n input_data, offset_data, mask_data, filter_data, bias_data, batch,\n channels, in_height, in_width, num_output, out_height, out_width, group,\n deformable_group, channels, num_output, kernel_height, kernel_width,\n stride_height, stride_width, padding_height, padding_width,\n dilation_height, dilation_width, columns, out_ptr);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/nms.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"nms.h\"\n\n#include \n\n#include \n#include \n#include \n#include \n#include // std::iota\n#include \n\n#include \"../ort_mmcv_utils.h\"\n\nNmsKernel::NmsKernel(OrtApi api, const OrtKernelInfo *info)\n : api_(api), ort_(api_), info_(info) {\n iou_threshold_ = ort_.KernelInfoGetAttribute(info, \"iou_threshold\");\n offset_ = ort_.KernelInfoGetAttribute(info, \"offset\");\n\n // create allocator\n allocator_ = Ort::AllocatorWithDefaultOptions();\n}\n\nvoid NmsKernel::Compute(OrtKernelContext *context) {\n const float iou_threshold = iou_threshold_;\n const int64_t offset = offset_;\n\n const OrtValue *boxes = ort_.KernelContext_GetInput(context, 0);\n const float *boxes_data =\n reinterpret_cast(ort_.GetTensorData(boxes));\n const OrtValue *scores = ort_.KernelContext_GetInput(context, 1);\n const float *scores_data =\n reinterpret_cast(ort_.GetTensorData(scores));\n\n OrtTensorDimensions boxes_dim(ort_, boxes);\n OrtTensorDimensions scores_dim(ort_, scores);\n\n int64_t nboxes = boxes_dim[0];\n assert(boxes_dim[1] == 4);\n\n // allocate tmp memory\n float *tmp_boxes = (float *)allocator_.Alloc(sizeof(float) * nboxes * 4);\n float *sc = (float *)allocator_.Alloc(sizeof(float) * nboxes);\n float *areas = (float *)allocator_.Alloc(sizeof(float) * nboxes);\n bool *select = (bool *)allocator_.Alloc(sizeof(bool) * nboxes);\n for (int64_t i = 0; i < nboxes; i++) {\n select[i] = true;\n }\n\n memcpy(tmp_boxes, boxes_data, sizeof(float) * nboxes * 4);\n memcpy(sc, scores_data, sizeof(float) * nboxes);\n\n // sort scores\n std::vector tmp_sc;\n for (int i = 0; i < nboxes; i++) {\n tmp_sc.push_back(sc[i]);\n }\n std::vector order(tmp_sc.size());\n std::iota(order.begin(), order.end(), 0);\n std::sort(order.begin(), order.end(), [&tmp_sc](int64_t id1, int64_t id2) {\n return tmp_sc[id1] > tmp_sc[id2];\n });\n\n // area = (x2 - x1 + offset) * (y2 - y1 + offset)\n for (int64_t i = 0; i < nboxes; i++) {\n areas[i] = (tmp_boxes[i * 4 + 2] - tmp_boxes[i * 4 + 0] + offset) *\n (tmp_boxes[i * 4 + 3] - tmp_boxes[i * 4 + 1] + offset);\n }\n\n for (int64_t _i = 0; _i < nboxes; _i++) {\n if (select[_i] == false) continue;\n auto i = order[_i];\n auto ix1 = tmp_boxes[i * 4 + 0];\n auto iy1 = tmp_boxes[i * 4 + 1];\n auto ix2 = tmp_boxes[i * 4 + 2];\n auto iy2 = tmp_boxes[i * 4 + 3];\n auto iarea = areas[i];\n\n for (int64_t _j = _i + 1; _j < nboxes; _j++) {\n if (select[_j] == false) continue;\n auto j = order[_j];\n auto xx1 = std::max(ix1, tmp_boxes[j * 4 + 0]);\n auto yy1 = std::max(iy1, tmp_boxes[j * 4 + 1]);\n auto xx2 = std::min(ix2, tmp_boxes[j * 4 + 2]);\n auto yy2 = std::min(iy2, tmp_boxes[j * 4 + 3]);\n\n auto w = std::max(0.f, xx2 - xx1 + offset);\n auto h = std::max(0.f, yy2 - yy1 + offset);\n auto inter = w * h;\n auto ovr = inter / (iarea + areas[j] - inter);\n if (ovr > iou_threshold) select[_j] = false;\n }\n }\n std::vector res_order;\n for (int i = 0; i < nboxes; i++) {\n if (select[i]) {\n res_order.push_back(order[i]);\n }\n }\n\n std::vector inds_dims({res_order.size()});\n\n OrtValue *res = ort_.KernelContext_GetOutput(context, 0, inds_dims.data(),\n inds_dims.size());\n int64_t *res_data = ort_.GetTensorMutableData(res);\n\n memcpy(res_data, res_order.data(), sizeof(int64_t) * res_order.size());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/onnxruntime_register.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"onnxruntime_register.h\"\n\n#include \"corner_pool.h\"\n#include \"deform_conv.h\"\n#include \"grid_sample.h\"\n#include \"modulated_deform_conv.h\"\n#include \"nms.h\"\n#include \"ort_mmcv_utils.h\"\n#include \"reduce_ops.h\"\n#include \"roi_align.h\"\n#include \"roi_align_rotated.h\"\n#include \"soft_nms.h\"\n\nconst char *c_MMCVOpDomain = \"mmcv\";\nSoftNmsOp c_SoftNmsOp;\nNmsOp c_NmsOp;\nMMCVRoiAlignCustomOp c_MMCVRoiAlignCustomOp;\nMMCVRoIAlignRotatedCustomOp c_MMCVRoIAlignRotatedCustomOp;\nGridSampleOp c_GridSampleOp;\nMMCVCumMaxCustomOp c_MMCVCumMaxCustomOp;\nMMCVCumMinCustomOp c_MMCVCumMinCustomOp;\nMMCVCornerPoolCustomOp c_MMCVCornerPoolCustomOp;\nMMCVModulatedDeformConvOp c_MMCVModulatedDeformConvOp;\nMMCVDeformConvOp c_MMCVDeformConvOp;\n\nOrtStatus *ORT_API_CALL RegisterCustomOps(OrtSessionOptions *options,\n const OrtApiBase *api) {\n OrtCustomOpDomain *domain = nullptr;\n const OrtApi *ortApi = api->GetApi(ORT_API_VERSION);\n\n if (auto status = ortApi->CreateCustomOpDomain(c_MMCVOpDomain, &domain)) {\n return status;\n }\n\n if (auto status = ortApi->CustomOpDomain_Add(domain, &c_SoftNmsOp)) {\n return status;\n }\n\n if (auto status = ortApi->CustomOpDomain_Add(domain, &c_NmsOp)) {\n return status;\n }\n\n if (auto status =\n ortApi->CustomOpDomain_Add(domain, &c_MMCVRoiAlignCustomOp)) {\n return status;\n }\n\n if (auto status =\n ortApi->CustomOpDomain_Add(domain, &c_MMCVRoIAlignRotatedCustomOp)) {\n return status;\n }\n\n if (auto status = ortApi->CustomOpDomain_Add(domain, &c_GridSampleOp)) {\n return status;\n }\n\n if (auto status =\n ortApi->CustomOpDomain_Add(domain, &c_MMCVCornerPoolCustomOp)) {\n return status;\n }\n\n if (auto status = ortApi->CustomOpDomain_Add(domain, &c_MMCVCumMaxCustomOp)) {\n return status;\n }\n\n if (auto status = ortApi->CustomOpDomain_Add(domain, &c_MMCVCumMinCustomOp)) {\n return status;\n }\n\n if (auto status =\n ortApi->CustomOpDomain_Add(domain, &c_MMCVModulatedDeformConvOp)) {\n return status;\n }\n\n if (auto status = ortApi->CustomOpDomain_Add(domain, &c_MMCVDeformConvOp)) {\n return status;\n }\n\n return ortApi->AddCustomOpDomain(options, domain);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/reduce_ops.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"reduce_ops.h\"\n\n#include \n\n#include \n\n#include \"../ort_mmcv_utils.h\"\n\n// modified from\n// https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/ReduceOps.cpp\n\nstatic inline int64_t maybe_wrap_dim(int64_t dim, int64_t ndims) {\n int64_t min = -ndims;\n int64_t max = ndims - 1;\n assert(dim >= min && dim <= max);\n if (dim < 0) dim += ndims;\n return dim;\n}\n\nstatic inline int64_t get_dim_stride(const int64_t dim, const int64_t ndims,\n const int64_t *reversed_dim_cumprod) {\n return dim == ndims - 1 ? 1 : reversed_dim_cumprod[dim + 1];\n}\n\nstatic inline int64_t get_dim_size(const int64_t dim, const int64_t ndims,\n const int64_t *reversed_dim_cumprod) {\n return dim == ndims - 1\n ? reversed_dim_cumprod[dim]\n : reversed_dim_cumprod[dim] / reversed_dim_cumprod[dim + 1];\n}\n\ntemplate \nvoid cummax_cummin_helper(const T1 *input, T1 *output, T2 *indices,\n const int64_t input_dim_size, const int64_t stride) {\n Operation op;\n T1 out = input[0];\n int64_t idx = 0;\n for (int64_t i = 0; i < input_dim_size; i++) {\n T1 curr_elem = input[i * stride];\n if (op(curr_elem, out)) {\n out = curr_elem;\n idx = i;\n }\n output[i * stride] = out;\n indices[i * stride] = idx;\n }\n}\n\n// modified `tensor_dim_apply3` from\n// https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/TensorDimApply.h.\n// the difference is that: (1) use `reversed_dim_cumprod` for fast computing of\n// tensor `size` and `stride`. (2) the same `stride` is used for input, output,\n// and indices, since it's unnecessary to use separate values. currently\n// `tensor_dim_apply3` is only used for `cummax` and `cummin`, according to the\n// official pytorch projects: https://github.com/pytorch/pytorch.\ntemplate \nvoid tensor_dim_apply3(const T1 *input, T1 *output, T2 *indices,\n const int64_t dim, const int64_t ndims,\n const int64_t *reversed_dim_cumprod, Function func) {\n int dim_apply_finished = 0;\n int64_t input_dim_size = get_dim_size(dim, ndims, reversed_dim_cumprod);\n // the same stride is used for input, output and indices\n int64_t stride = get_dim_stride(dim, ndims, reversed_dim_cumprod);\n std::vector counter(ndims, 0);\n\n while (!dim_apply_finished) {\n // call `func` once to update output and indices\n func(input, output, indices, input_dim_size, stride);\n if (ndims == 1) break;\n for (int64_t dim_i = 0; dim_i < ndims; dim_i++) {\n if (dim_i == dim) {\n if (dim_i == (ndims - 1)) {\n dim_apply_finished = 1;\n break;\n }\n continue;\n }\n counter[dim_i]++;\n\n // the same stride is used for input, output, and indices\n int64_t stride_dim_i = get_dim_stride(dim_i, ndims, reversed_dim_cumprod);\n input += stride_dim_i;\n output += stride_dim_i;\n indices += stride_dim_i;\n\n if (counter[dim_i] == get_dim_size(dim_i, ndims, reversed_dim_cumprod)) {\n if (dim_i == ndims - 1) {\n dim_apply_finished = 1;\n break;\n } else {\n input -= counter[dim_i] * stride_dim_i;\n output -= counter[dim_i] * stride_dim_i;\n indices -= counter[dim_i] * stride_dim_i;\n counter[dim_i] = 0;\n }\n } else {\n break;\n } // if\n } // for\n } // while\n}\n\ntemplate \nvoid CumMax_CumMin_CPU(const T1 *input, T1 *output, T2 *indices,\n int64_t *reversed_dim_cumprod, const int64_t dim,\n const OrtTensorDimensions &out_dimensions) {\n // calculate numel\n const int64_t ndims = out_dimensions.size();\n int64_t numel = 1;\n for (int64_t dim_i = 0; dim_i < ndims; dim_i++) {\n numel *= out_dimensions.data()[dim_i];\n }\n\n // cummax is only applied to input which is non-zero dim and non-empty\n if (numel) {\n // compute the cumulative production on dimension size,\n // which is then used for computing the stride or size of a specific `dim`.\n reversed_dim_cumprod[ndims - 1] = out_dimensions.data()[ndims - 1];\n for (int64_t dim_i = ndims - 2; dim_i >= 0; dim_i--) {\n reversed_dim_cumprod[dim_i] =\n reversed_dim_cumprod[dim_i + 1] * out_dimensions.data()[dim_i];\n }\n\n // do cummax or cummin based on `Operation` type\n tensor_dim_apply3(\n input, output, indices, dim, ndims, reversed_dim_cumprod,\n cummax_cummin_helper);\n }\n}\n\nvoid MMCVCumMaxKernel::Compute(OrtKernelContext *context) {\n // get input\n const OrtValue *input = ort_.KernelContext_GetInput(context, 0);\n const float *input_data =\n reinterpret_cast(ort_.GetTensorData(input));\n\n // get output\n OrtTensorDimensions out_dimensions(ort_, input);\n OrtValue *output = ort_.KernelContext_GetOutput(\n context, 0, out_dimensions.data(), out_dimensions.size());\n float *output_data = ort_.GetTensorMutableData(output);\n OrtValue *indices = ort_.KernelContext_GetOutput(\n context, 1, out_dimensions.data(), out_dimensions.size());\n int64_t *indices_data = ort_.GetTensorMutableData(indices);\n\n // allocate tmp memory for computing the cumulative production on dimension\n // size\n const int64_t ndims = out_dimensions.size();\n assert(ndims > 0);\n int64_t *reversed_dim_cumprod =\n (int64_t *)allocator_.Alloc(sizeof(int64_t) * ndims);\n\n // dim should be wrapped if it's negative (e.g. -1)\n const int64_t dim = maybe_wrap_dim(dim_, ndims);\n CumMax_CumMin_CPU>(\n input_data, output_data, indices_data, reversed_dim_cumprod, dim,\n out_dimensions);\n}\n\nvoid MMCVCumMinKernel::Compute(OrtKernelContext *context) {\n // get input\n const OrtValue *input = ort_.KernelContext_GetInput(context, 0);\n const float *input_data =\n reinterpret_cast(ort_.GetTensorData(input));\n\n // get output\n OrtTensorDimensions out_dimensions(ort_, input);\n OrtValue *output = ort_.KernelContext_GetOutput(\n context, 0, out_dimensions.data(), out_dimensions.size());\n float *output_data = ort_.GetTensorMutableData(output);\n OrtValue *indices = ort_.KernelContext_GetOutput(\n context, 1, out_dimensions.data(), out_dimensions.size());\n int64_t *indices_data = ort_.GetTensorMutableData(indices);\n\n // allocate tmp memory for computing the cumulative production on dimension\n // size\n const int64_t ndims = out_dimensions.size();\n assert(ndims > 0);\n int64_t *reversed_dim_cumprod =\n (int64_t *)allocator_.Alloc(sizeof(int64_t) * ndims);\n\n // dim should be wrapped if it's negative (e.g. -1)\n const int64_t dim = maybe_wrap_dim(dim_, ndims);\n CumMax_CumMin_CPU>(\n input_data, output_data, indices_data, reversed_dim_cumprod, dim,\n out_dimensions);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/roi_align.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"roi_align.h\"\n\n#include \"../ort_mmcv_utils.h\"\n\n// implementation taken from Caffe2\nstruct PreCalc {\n int pos1;\n int pos2;\n int pos3;\n int pos4;\n float w1;\n float w2;\n float w3;\n float w4;\n};\n\nvoid pre_calc_for_bilinear_interpolate(\n const int height, const int width, const int pooled_height,\n const int pooled_width, const int iy_upper, const int ix_upper,\n float roi_start_h, float roi_start_w, float bin_size_h, float bin_size_w,\n int roi_bin_grid_h, int roi_bin_grid_w, std::vector &pre_calc) {\n int pre_calc_index = 0;\n for (int ph = 0; ph < pooled_height; ph++) {\n for (int pw = 0; pw < pooled_width; pw++) {\n for (int iy = 0; iy < iy_upper; iy++) {\n const float yy =\n roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < ix_upper; ix++) {\n const float xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n float x = xx;\n float y = yy;\n // deal with: inverse elements are out of feature map boundary\n if (y < -1.0 || y > height || x < -1.0 || x > width) {\n // empty\n PreCalc pc;\n pc.pos1 = 0;\n pc.pos2 = 0;\n pc.pos3 = 0;\n pc.pos4 = 0;\n pc.w1 = 0;\n pc.w2 = 0;\n pc.w3 = 0;\n pc.w4 = 0;\n pre_calc[pre_calc_index] = pc;\n pre_calc_index += 1;\n continue;\n }\n\n if (y <= 0) {\n y = 0;\n }\n if (x <= 0) {\n x = 0;\n }\n\n int y_low = (int)y;\n int x_low = (int)x;\n int y_high;\n int x_high;\n\n if (y_low >= height - 1) {\n y_high = y_low = height - 1;\n y = (float)y_low;\n } else {\n y_high = y_low + 1;\n }\n\n if (x_low >= width - 1) {\n x_high = x_low = width - 1;\n x = (float)x_low;\n } else {\n x_high = x_low + 1;\n }\n\n float ly = y - y_low;\n float lx = x - x_low;\n float hy = 1. - ly, hx = 1. - lx;\n float w1 = hy * hx, w2 = hy * lx, w3 = ly * hx, w4 = ly * lx;\n\n // save weights and indices\n PreCalc pc;\n pc.pos1 = y_low * width + x_low;\n pc.pos2 = y_low * width + x_high;\n pc.pos3 = y_high * width + x_low;\n pc.pos4 = y_high * width + x_high;\n pc.w1 = w1;\n pc.w2 = w2;\n pc.w3 = w3;\n pc.w4 = w4;\n pre_calc[pre_calc_index] = pc;\n\n pre_calc_index += 1;\n }\n }\n }\n }\n}\n\nvoid ROIAlignForwardCPU(const int nthreads, const float *input,\n const float *rois, float *output, float *argmax_y,\n float *argmax_x, const int pooled_height,\n const int pooled_width, const float spatial_scale,\n const int sampling_ratio,\n const int pool_mode, // 0 - max pool, 1 - avg pool\n const bool aligned, const int channels,\n const int height, const int width) {\n int n_rois = nthreads / channels / pooled_width / pooled_height;\n // (n, c, ph, pw) is an element in the pooled output\n // can be parallelized using omp\n // #pragma omp parallel for num_threads(32)\n for (int n = 0; n < n_rois; n++) {\n int index_n = n * channels * pooled_width * pooled_height;\n\n const float *offset_rois = rois + n * 5;\n int roi_batch_ind = offset_rois[0];\n\n // Do not use rounding; this implementation detail is critical\n float offset = aligned ? (float)0.5 : (float)0.0;\n float roi_start_w = offset_rois[1] * spatial_scale - offset;\n float roi_start_h = offset_rois[2] * spatial_scale - offset;\n float roi_end_w = offset_rois[3] * spatial_scale - offset;\n float roi_end_h = offset_rois[4] * spatial_scale - offset;\n\n float roi_width = roi_end_w - roi_start_w;\n float roi_height = roi_end_h - roi_start_h;\n if (aligned) {\n /*AT_ASSERTM(roi_width >= 0 && roi_height >= 0,\n \"ROIs in ROIAlign cannot have non-negative size!\");*/\n assert(roi_width >= 0 && roi_height >= 0);\n } else { // for backward-compatibility only\n roi_width = std::max(roi_width, (float)1.);\n roi_height = std::max(roi_height, (float)1.);\n }\n float bin_size_h =\n static_cast(roi_height) / static_cast(pooled_height);\n float bin_size_w =\n static_cast(roi_width) / static_cast(pooled_width);\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (sampling_ratio > 0)\n ? sampling_ratio\n : ceil(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (sampling_ratio > 0) ? sampling_ratio : ceil(roi_width / pooled_width);\n\n // When the grid is empty, output zeros == 0/1, instead of NaN.\n const float count =\n std::max(roi_bin_grid_h * roi_bin_grid_w, 1); // e.g. = 4\n\n // we want to precalculate indices and weights shared by all channels,\n // this is the key point of optimization\n std::vector pre_calc(roi_bin_grid_h * roi_bin_grid_w *\n pooled_width * pooled_height);\n pre_calc_for_bilinear_interpolate(\n height, width, pooled_height, pooled_width, roi_bin_grid_h,\n roi_bin_grid_w, roi_start_h, roi_start_w, bin_size_h, bin_size_w,\n roi_bin_grid_h, roi_bin_grid_w, pre_calc);\n\n for (int c = 0; c < channels; c++) {\n int index_n_c = index_n + c * pooled_width * pooled_height;\n const float *offset_input =\n input + (roi_batch_ind * channels + c) * height * width;\n int pre_calc_index = 0;\n\n for (int ph = 0; ph < pooled_height; ph++) {\n for (int pw = 0; pw < pooled_width; pw++) {\n int index = index_n_c + ph * pooled_width + pw;\n\n float output_val = 0.;\n float maxval = -10000;\n float maxidx_y = -1.f, maxidx_x = -1.f;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const float y = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h);\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const float x = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n PreCalc pc = pre_calc[pre_calc_index];\n float val = pc.w1 * offset_input[pc.pos1] +\n pc.w2 * offset_input[pc.pos2] +\n pc.w3 * offset_input[pc.pos3] +\n pc.w4 * offset_input[pc.pos4];\n if (val > maxval) {\n maxval = val;\n maxidx_y = y;\n maxidx_x = x;\n }\n output_val += val;\n pre_calc_index += 1;\n }\n }\n if (pool_mode == 0) {\n // We do max pooling inside a bin\n output[index] = maxval;\n argmax_y[index] = maxidx_y;\n argmax_x[index] = maxidx_x;\n } else if (pool_mode == 1) {\n // We do average (integral) pooling inside a bin\n output[index] = output_val / count;\n } // if\n } // for pw\n } // for ph\n } // for c\n } // for n\n}\n\nvoid MMCVRoiAlignKernel::Compute(OrtKernelContext *context) {\n // Setup inputs\n const OrtValue *input_X = ort_.KernelContext_GetInput(context, 0);\n const float *X_data =\n reinterpret_cast(ort_.GetTensorData(input_X));\n const OrtValue *input_rois = ort_.KernelContext_GetInput(context, 1);\n const float *rois = reinterpret_cast(\n ort_.GetTensorData(input_rois));\n\n // Setup output\n OrtTensorDimensions out_dimensions(ort_, input_X);\n OrtTensorDimensions roi_dimensions(ort_, input_rois);\n\n int batch_size = out_dimensions.data()[0];\n int input_channels = out_dimensions.data()[1];\n int input_height = out_dimensions.data()[2];\n int input_width = out_dimensions.data()[3];\n\n out_dimensions.data()[0] = roi_dimensions.data()[0];\n out_dimensions.data()[2] = aligned_height_;\n out_dimensions.data()[3] = aligned_width_;\n\n OrtValue *output = ort_.KernelContext_GetOutput(\n context, 0, out_dimensions.data(), out_dimensions.size());\n float *out = ort_.GetTensorMutableData(output);\n OrtTensorTypeAndShapeInfo *output_info = ort_.GetTensorTypeAndShape(output);\n ort_.ReleaseTensorTypeAndShapeInfo(output_info);\n\n // TODO: forward here\n int output_size = out_dimensions.data()[0];\n for (auto i = 1; i < out_dimensions.size(); ++i) {\n output_size *= out_dimensions.data()[i];\n }\n\n int poolMod = 1;\n if (pool_mode_ == \"max\") poolMod = 0;\n\n float *argmax_x = nullptr, *argmax_y = nullptr;\n if (poolMod == 0) {\n argmax_y = new float[output_size];\n argmax_x = new float[output_size];\n }\n\n ROIAlignForwardCPU(output_size, X_data, rois, out, argmax_y, argmax_x,\n aligned_height_, aligned_width_, spatial_scale_,\n sampling_ratio_, poolMod, aligned_, input_channels,\n input_height, input_width);\n\n if (argmax_x) delete argmax_x;\n if (argmax_y) delete argmax_y;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/roi_align_rotated.cpp", "content": "// Modified from\n// https://github.com/facebookresearch/detectron2/tree/master/detectron2/layers/csrc/ROIAlignRotated\n// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n#include \"roi_align_rotated.h\"\n\n#include \"../ort_mmcv_utils.h\"\n\nstruct PreCalc {\n int pos1;\n int pos2;\n int pos3;\n int pos4;\n float w1;\n float w2;\n float w3;\n float w4;\n};\n\nvoid pre_calc_for_bilinear_interpolate(\n const int height, const int width, const int pooled_height,\n const int pooled_width, const int iy_upper, const int ix_upper,\n float roi_start_h, float roi_start_w, float bin_size_h, float bin_size_w,\n int roi_bin_grid_h, int roi_bin_grid_w, float roi_center_h,\n float roi_center_w, float cos_theta, float sin_theta,\n std::vector &pre_calc) {\n int pre_calc_index = 0;\n for (int ph = 0; ph < pooled_height; ph++) {\n for (int pw = 0; pw < pooled_width; pw++) {\n for (int iy = 0; iy < iy_upper; iy++) {\n const float yy =\n roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < ix_upper; ix++) {\n const float xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n // Rotate by theta around the center and translate\n // In image space, (y, x) is the order for Right Handed System,\n // and this is essentially multiplying the point by a rotation matrix\n // to rotate it counterclockwise through angle theta.\n float y = yy * cos_theta - xx * sin_theta + roi_center_h;\n float x = yy * sin_theta + xx * cos_theta + roi_center_w;\n // deal with: inverse elements are out of feature map boundary\n if (y < -1.0 || y > height || x < -1.0 || x > width) {\n // empty\n PreCalc pc;\n pc.pos1 = 0;\n pc.pos2 = 0;\n pc.pos3 = 0;\n pc.pos4 = 0;\n pc.w1 = 0;\n pc.w2 = 0;\n pc.w3 = 0;\n pc.w4 = 0;\n pre_calc[pre_calc_index] = pc;\n pre_calc_index += 1;\n continue;\n }\n\n if (y < 0) {\n y = 0;\n }\n if (x < 0) {\n x = 0;\n }\n\n int y_low = (int)y;\n int x_low = (int)x;\n int y_high;\n int x_high;\n\n if (y_low >= height - 1) {\n y_high = y_low = height - 1;\n y = (float)y_low;\n } else {\n y_high = y_low + 1;\n }\n\n if (x_low >= width - 1) {\n x_high = x_low = width - 1;\n x = (float)x_low;\n } else {\n x_high = x_low + 1;\n }\n\n float ly = y - y_low;\n float lx = x - x_low;\n float hy = 1. - ly, hx = 1. - lx;\n float w1 = hy * hx, w2 = hy * lx, w3 = ly * hx, w4 = ly * lx;\n\n // save weights and indices\n PreCalc pc;\n pc.pos1 = y_low * width + x_low;\n pc.pos2 = y_low * width + x_high;\n pc.pos3 = y_high * width + x_low;\n pc.pos4 = y_high * width + x_high;\n pc.w1 = w1;\n pc.w2 = w2;\n pc.w3 = w3;\n pc.w4 = w4;\n pre_calc[pre_calc_index] = pc;\n\n pre_calc_index += 1;\n }\n }\n }\n }\n}\n\nvoid ROIAlignRotatedForwardCPU(const int nthreads, const float *input,\n const float *rois, float *output,\n const float &spatial_scale, const int aligned,\n const int clockwise, const int channels,\n const int height, const int width,\n const int pooled_height, const int pooled_width,\n const int sampling_ratio) {\n int n_rois = nthreads / channels / pooled_width / pooled_height;\n // (n, c, ph, pw) is an element in the pooled output\n // can be parallelized using omp\n // #pragma omp parallel for num_threads(32)\n for (int n = 0; n < n_rois; n++) {\n int index_n = n * channels * pooled_width * pooled_height;\n\n const float *current_roi = rois + n * 6;\n int roi_batch_ind = current_roi[0];\n\n // Do not use rounding; this implementation detail is critical\n float offset = aligned ? (float)0.5 : (float)0.0;\n float roi_center_w = current_roi[1] * spatial_scale - offset;\n float roi_center_h = current_roi[2] * spatial_scale - offset;\n float roi_width = current_roi[3] * spatial_scale;\n float roi_height = current_roi[4] * spatial_scale;\n // float theta = current_roi[5] * M_PI / 180.0;\n float theta = current_roi[5]; // Radian angle by default\n if (clockwise) {\n theta = -theta;\n }\n float cos_theta = cos(theta);\n float sin_theta = sin(theta);\n if (!aligned) { // for backward-compatibility only\n roi_width = std::max(roi_width, (float)1.);\n roi_height = std::max(roi_height, (float)1.);\n }\n\n float bin_size_h =\n static_cast(roi_height) / static_cast(pooled_height);\n float bin_size_w =\n static_cast(roi_width) / static_cast(pooled_width);\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (sampling_ratio > 0)\n ? sampling_ratio\n : ceil(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (sampling_ratio > 0) ? sampling_ratio : ceil(roi_width / pooled_width);\n\n // We do average (integral) pooling inside a bin\n const float count =\n std::max(roi_bin_grid_h * roi_bin_grid_w, 1); // e.g. = 4\n\n // we want to precalculate indices and weights shared by all channels,\n // this is the key point of optimization\n std::vector pre_calc(roi_bin_grid_h * roi_bin_grid_w *\n pooled_width * pooled_height);\n\n // roi_start_h and roi_start_w are computed wrt the center of RoI (x, y).\n // Appropriate translation needs to be applied after.\n float roi_start_h = -roi_height / 2.0;\n float roi_start_w = -roi_width / 2.0;\n\n pre_calc_for_bilinear_interpolate(\n height, width, pooled_height, pooled_width, roi_bin_grid_h,\n roi_bin_grid_w, roi_start_h, roi_start_w, bin_size_h, bin_size_w,\n roi_bin_grid_h, roi_bin_grid_w, roi_center_h, roi_center_w, cos_theta,\n sin_theta, pre_calc);\n\n for (int c = 0; c < channels; c++) {\n int index_n_c = index_n + c * pooled_width * pooled_height;\n const float *offset_input =\n input + (roi_batch_ind * channels + c) * height * width;\n int pre_calc_index = 0;\n\n for (int ph = 0; ph < pooled_height; ph++) {\n for (int pw = 0; pw < pooled_width; pw++) {\n int index = index_n_c + ph * pooled_width + pw;\n\n float output_val = 0.;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n PreCalc pc = pre_calc[pre_calc_index];\n output_val += pc.w1 * offset_input[pc.pos1] +\n pc.w2 * offset_input[pc.pos2] +\n pc.w3 * offset_input[pc.pos3] +\n pc.w4 * offset_input[pc.pos4];\n\n pre_calc_index += 1;\n }\n }\n output_val /= count;\n\n output[index] = output_val;\n } // for pw\n } // for ph\n } // for c\n } // for n\n}\n\nvoid MMCVRoIAlignRotatedKernel::Compute(OrtKernelContext *context) {\n // Setup inputs\n const OrtValue *input_X = ort_.KernelContext_GetInput(context, 0);\n const float *X_data =\n reinterpret_cast(ort_.GetTensorData(input_X));\n const OrtValue *input_rois = ort_.KernelContext_GetInput(context, 1);\n const float *rois = reinterpret_cast(\n ort_.GetTensorData(input_rois));\n\n // Setup output\n OrtTensorDimensions out_dimensions(ort_, input_X);\n OrtTensorDimensions roi_dimensions(ort_, input_rois);\n\n int batch_size = out_dimensions.data()[0];\n int input_channels = out_dimensions.data()[1];\n int input_height = out_dimensions.data()[2];\n int input_width = out_dimensions.data()[3];\n\n out_dimensions.data()[0] = roi_dimensions.data()[0];\n out_dimensions.data()[2] = aligned_height_;\n out_dimensions.data()[3] = aligned_width_;\n\n OrtValue *output = ort_.KernelContext_GetOutput(\n context, 0, out_dimensions.data(), out_dimensions.size());\n float *out = ort_.GetTensorMutableData(output);\n OrtTensorTypeAndShapeInfo *output_info = ort_.GetTensorTypeAndShape(output);\n ort_.ReleaseTensorTypeAndShapeInfo(output_info);\n\n // TODO: forward here\n int output_size = out_dimensions.data()[0];\n for (auto i = 1; i < out_dimensions.size(); ++i) {\n output_size *= out_dimensions.data()[i];\n }\n ROIAlignRotatedForwardCPU(output_size, X_data, rois, out, spatial_scale_,\n aligned_, clockwise_, input_channels, input_height,\n input_width, aligned_height_, aligned_width_,\n sampling_ratio_);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/cpu/soft_nms.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"soft_nms.h\"\n\n#include \n\n#include \n#include \n\n#include \"../ort_mmcv_utils.h\"\n\nSoftNmsKernel::SoftNmsKernel(OrtApi api, const OrtKernelInfo *info)\n : api_(api), ort_(api_), info_(info) {\n iou_threshold_ = ort_.KernelInfoGetAttribute(info, \"iou_threshold\");\n sigma_ = ort_.KernelInfoGetAttribute(info, \"sigma\");\n min_score_ = ort_.KernelInfoGetAttribute(info, \"min_score\");\n method_ = ort_.KernelInfoGetAttribute(info, \"method\");\n offset_ = ort_.KernelInfoGetAttribute(info, \"offset\");\n\n // create allocator\n allocator_ = Ort::AllocatorWithDefaultOptions();\n}\n\nvoid SoftNmsKernel::Compute(OrtKernelContext *context) {\n typedef float T;\n\n const T iou_threshold = T(iou_threshold_);\n const T sigma = T(sigma_);\n const T min_score = T(min_score_);\n const int method = int(method_);\n const T offset = T(offset_);\n\n const OrtValue *boxes = ort_.KernelContext_GetInput(context, 0);\n const T *boxes_data =\n reinterpret_cast(ort_.GetTensorData(boxes));\n const OrtValue *scores = ort_.KernelContext_GetInput(context, 1);\n const T *scores_data =\n reinterpret_cast(ort_.GetTensorData(scores));\n\n OrtTensorDimensions boxes_dim(ort_, boxes);\n OrtTensorDimensions scores_dim(ort_, scores);\n\n int64_t nboxes = boxes_dim[0];\n assert(boxes_dim[1] == 4);\n\n // allocate tmp memory\n T *tmp_boxes = (T *)allocator_.Alloc(sizeof(T) * nboxes * 4);\n T *x1 = tmp_boxes;\n T *y1 = tmp_boxes + 1;\n T *x2 = tmp_boxes + 2;\n T *y2 = tmp_boxes + 3;\n T *sc = (T *)allocator_.Alloc(sizeof(T) * nboxes);\n T *areas = (T *)allocator_.Alloc(sizeof(T) * nboxes);\n T *de = (T *)allocator_.Alloc(sizeof(T) * nboxes * 5);\n int64_t *inds = (int64_t *)allocator_.Alloc(sizeof(int64_t) * nboxes);\n\n memcpy(tmp_boxes, boxes_data, sizeof(T) * nboxes * 4);\n memcpy(sc, scores_data, sizeof(T) * nboxes);\n\n // init inds as arange(nboxes)\n std::generate(inds, inds + nboxes, [n = 0]() mutable { return n++; });\n\n // area = (x2-x1+offset)*(y2-y1+offset)\n for (int64_t i = 0; i < nboxes; i++) {\n areas[i] =\n (x2[i * 4] - x1[i * 4] + offset) * (y2[i * 4] - y1[i * 4] + offset);\n }\n\n int64_t pos = 0;\n\n for (int64_t i = 0; i < nboxes; i++) {\n auto max_score = sc[i];\n auto max_pos = i;\n\n pos = i + 1;\n // get max box\n while (pos < nboxes) {\n if (max_score < sc[pos]) {\n max_score = sc[pos];\n max_pos = pos;\n }\n pos = pos + 1;\n }\n // swap\n auto ix1 = de[i * 5 + 0] = x1[max_pos * 4];\n auto iy1 = de[i * 5 + 1] = y1[max_pos * 4];\n auto ix2 = de[i * 5 + 2] = x2[max_pos * 4];\n auto iy2 = de[i * 5 + 3] = y2[max_pos * 4];\n auto iscore = de[i * 5 + 4] = sc[max_pos];\n auto iarea = areas[max_pos];\n auto iind = inds[max_pos];\n x1[max_pos * 4] = x1[i * 4];\n y1[max_pos * 4] = y1[i * 4];\n x2[max_pos * 4] = x2[i * 4];\n y2[max_pos * 4] = y2[i * 4];\n sc[max_pos] = sc[i];\n areas[max_pos] = areas[i];\n inds[max_pos] = inds[i];\n x1[i * 4] = ix1;\n y1[i * 4] = iy1;\n x2[i * 4] = ix2;\n y2[i * 4] = iy2;\n sc[i] = iscore;\n areas[i] = iarea;\n inds[i] = iind;\n\n pos = i + 1;\n while (pos < nboxes) {\n auto xx1 = std::max(ix1, x1[pos * 4]);\n auto yy1 = std::max(iy1, y1[pos * 4]);\n auto xx2 = std::min(ix2, x2[pos * 4]);\n auto yy2 = std::min(iy2, y2[pos * 4]);\n\n auto w = std::max(0.f, xx2 - xx1 + offset);\n auto h = std::max(0.f, yy2 - yy1 + offset);\n auto inter = w * h;\n auto ovr = inter / (iarea + areas[pos] - inter);\n\n float weight = 1.;\n if (method == 0) {\n if (ovr >= iou_threshold) weight = 0;\n } else if (method == 1) {\n if (ovr >= iou_threshold) weight = 1 - ovr;\n } else if (method == 2) {\n weight = std::exp(-(ovr * ovr) / sigma);\n }\n sc[pos] *= weight;\n // if box score falls below threshold, discard the box by\n // swapping with last box update N\n if (sc[pos] < min_score) {\n x1[pos * 4] = x1[(nboxes - 1) * 4];\n y1[pos * 4] = y1[(nboxes - 1) * 4];\n x2[pos * 4] = x2[(nboxes - 1) * 4];\n y2[pos * 4] = y2[(nboxes - 1) * 4];\n sc[pos] = sc[nboxes - 1];\n areas[pos] = areas[nboxes - 1];\n inds[pos] = inds[nboxes - 1];\n nboxes = nboxes - 1;\n pos = pos - 1;\n }\n pos = pos + 1;\n }\n }\n\n std::vector dets_dim({nboxes, 5});\n OrtValue *dets = ort_.KernelContext_GetOutput(context, 0, dets_dim.data(),\n dets_dim.size());\n T *dets_data = ort_.GetTensorMutableData(dets);\n\n std::vector inds_dim({nboxes});\n OrtValue *inds_ov = ort_.KernelContext_GetOutput(context, 1, inds_dim.data(),\n inds_dim.size());\n int64_t *inds_data = ort_.GetTensorMutableData(inds_ov);\n\n memcpy(dets_data, de, sizeof(T) * nboxes * 5);\n memcpy(inds_data, inds, sizeof(int64_t) * nboxes);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/deform_conv.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_DEFORM_CONV_H\n#define ONNXRUNTIME_DEFORM_CONV_H\n\n#include \n\nstruct MMCVDeformConvKernel {\n MMCVDeformConvKernel(OrtApi api, const OrtKernelInfo *info);\n\n void Compute(OrtKernelContext *context);\n\n protected:\n OrtApi api_;\n Ort::CustomOpApi ort_;\n const OrtKernelInfo *info_;\n Ort::AllocatorWithDefaultOptions allocator_;\n\n int64_t stride_height_;\n int64_t stride_width_;\n int64_t padding_height_;\n int64_t padding_width_;\n int64_t dilation_height_;\n int64_t dilation_width_;\n int64_t deformable_group_;\n int64_t group_;\n int64_t im2col_step_;\n};\n\nstruct MMCVDeformConvOp\n : Ort::CustomOpBase {\n void *CreateKernel(OrtApi api, const OrtKernelInfo *info) const {\n return new MMCVDeformConvKernel(api, info);\n }\n\n const char *GetName() const { return \"MMCVDeformConv2d\"; };\n\n size_t GetInputTypeCount() const { return 3; };\n ONNXTensorElementDataType GetInputType(size_t /*index*/) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n OrtCustomOpInputOutputCharacteristic GetInputCharacteristic(\n size_t index) const {\n return OrtCustomOpInputOutputCharacteristic::INPUT_OUTPUT_REQUIRED;\n }\n\n size_t GetOutputTypeCount() const { return 1; };\n ONNXTensorElementDataType GetOutputType(size_t /*index*/) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n // force cpu\n const char *GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n };\n};\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/grid_sample.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_GRIDSAMPLE_H\n#define ONNXRUNTIME_GRIDSAMPLE_H\n\n#include \n\nstruct GridSampleKernel {\n GridSampleKernel(OrtApi api, const OrtKernelInfo *info);\n\n void Compute(OrtKernelContext *context);\n\n protected:\n OrtApi api_;\n Ort::CustomOpApi ort_;\n const OrtKernelInfo *info_;\n Ort::AllocatorWithDefaultOptions allocator_;\n\n int64_t align_corners_;\n int64_t interpolation_mode_;\n int64_t padding_mode_;\n};\n\nstruct GridSampleOp : Ort::CustomOpBase {\n void *CreateKernel(OrtApi api, const OrtKernelInfo *info) const {\n return new GridSampleKernel(api, info);\n };\n\n const char *GetName() const { return \"grid_sampler\"; };\n\n size_t GetInputTypeCount() const { return 2; };\n ONNXTensorElementDataType GetInputType(size_t /*index*/) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n size_t GetOutputTypeCount() const { return 1; };\n ONNXTensorElementDataType GetOutputType(size_t /*index*/) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n const char *GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n };\n};\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/modulated_deform_conv.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_MODULATED_DEFORM_CONV_H\n#define ONNXRUNTIME_MODULATED_DEFORM_CONV_H\n\n#include \n\nstruct MMCVModulatedDeformConvKernel {\n MMCVModulatedDeformConvKernel(OrtApi api, const OrtKernelInfo *info);\n\n void Compute(OrtKernelContext *context);\n\n protected:\n OrtApi api_;\n Ort::CustomOpApi ort_;\n const OrtKernelInfo *info_;\n Ort::AllocatorWithDefaultOptions allocator_;\n\n int64_t stride_height_;\n int64_t stride_width_;\n int64_t padding_height_;\n int64_t padding_width_;\n int64_t dilation_height_;\n int64_t dilation_width_;\n int64_t deformable_group_;\n int64_t group_;\n};\n\nstruct MMCVModulatedDeformConvOp\n : Ort::CustomOpBase {\n void *CreateKernel(OrtApi api, const OrtKernelInfo *info) const {\n return new MMCVModulatedDeformConvKernel(api, info);\n }\n\n const char *GetName() const { return \"MMCVModulatedDeformConv2d\"; };\n\n size_t GetInputTypeCount() const { return 5; };\n ONNXTensorElementDataType GetInputType(size_t /*index*/) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n OrtCustomOpInputOutputCharacteristic GetInputCharacteristic(\n size_t index) const {\n // The last input (index == 4) is optional, which is bias\n if (index == 4)\n return OrtCustomOpInputOutputCharacteristic::INPUT_OUTPUT_OPTIONAL;\n\n return OrtCustomOpInputOutputCharacteristic::INPUT_OUTPUT_REQUIRED;\n }\n\n size_t GetOutputTypeCount() const { return 1; };\n ONNXTensorElementDataType GetOutputType(size_t /*index*/) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n // force cpu\n const char *GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n };\n};\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/nms.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_NMS_H\n#define ONNXRUNTIME_NMS_H\n\n#include \n\nstruct NmsKernel {\n NmsKernel(OrtApi api, const OrtKernelInfo *info);\n\n void Compute(OrtKernelContext *context);\n\n protected:\n OrtApi api_;\n Ort::CustomOpApi ort_;\n const OrtKernelInfo *info_;\n Ort::AllocatorWithDefaultOptions allocator_;\n\n float iou_threshold_;\n int64_t offset_;\n};\n\nstruct NmsOp : Ort::CustomOpBase {\n void *CreateKernel(OrtApi api, const OrtKernelInfo *info) const {\n return new NmsKernel(api, info);\n };\n\n const char *GetName() const { return \"NonMaxSuppression\"; };\n\n size_t GetInputTypeCount() const { return 2; };\n ONNXTensorElementDataType GetInputType(size_t /*index*/) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n size_t GetOutputTypeCount() const { return 1; };\n ONNXTensorElementDataType GetOutputType(size_t index) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64;\n }\n\n // force cpu\n const char *GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n }\n};\n\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/onnxruntime_register.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_REGISTER_H\n#define ONNXRUNTIME_REGISTER_H\n#include \n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\nOrtStatus *ORT_API_CALL RegisterCustomOps(OrtSessionOptions *options,\n const OrtApiBase *api);\n\n#ifdef __cplusplus\n}\n#endif\n#endif // ONNXRUNTIME_REGISTER_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/onnxruntime_session_options_config_keys.h", "content": "// Copyright (c) Microsoft Corporation. All rights reserved.\n// Licensed under the MIT License.\n\n#ifndef ONNXRUNTIME_SESSION_OPTIONS_CONFIG_KEYS_H\n#define ONNXRUNTIME_SESSION_OPTIONS_CONFIG_KEYS_H\n\n/*\n * This file defines SessionOptions Config Keys and format of the Config Values.\n *\n * The Naming Convention for a SessionOptions Config Key,\n * \"[Area][.[SubArea1].[SubArea2]...].[Keyname]\"\n * Such as \"ep.cuda.use_arena\"\n * The Config Key cannot be empty\n * The maximum length of the Config Key is 128\n *\n * The string format of a SessionOptions Config Value is defined individually\n * for each Config. The maximum length of the Config Value is 1024\n */\n\n// Key for disable PrePacking,\n// If the config value is set to \"1\" then the prepacking is disabled, otherwise\n// prepacking is enabled (default value)\nstatic const char* const kOrtSessionOptionsConfigDisablePrepacking =\n \"session.disable_prepacking\";\n\n// A value of \"1\" means allocators registered in the env will be used. \"0\" means\n// the allocators created in the session will be used. Use this to override the\n// usage of env allocators on a per session level.\nstatic const char* const kOrtSessionOptionsConfigUseEnvAllocators =\n \"session.use_env_allocators\";\n\n// Set to 'ORT' (case sensitive) to load an ORT format model.\n// If unset, model type will default to ONNX unless inferred from filename\n// ('.ort' == ORT format) or bytes to be ORT\nstatic const char* const kOrtSessionOptionsConfigLoadModelFormat =\n \"session.load_model_format\";\n\n// Set to 'ORT' (case sensitive) to save optimized model in ORT format when\n// SessionOptions.optimized_model_path is set. If unset, format will default to\n// ONNX unless optimized_model_filepath ends in '.ort'.\nstatic const char* const kOrtSessionOptionsConfigSaveModelFormat =\n \"session.save_model_format\";\n\n#endif // ONNXRUNTIME_SESSION_OPTIONS_CONFIG_KEYS_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/ort_mmcv_utils.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ORT_MMCV_UTILS_H\n#define ORT_MMCV_UTILS_H\n#include \n\n#include \n\nstruct OrtTensorDimensions : std::vector {\n OrtTensorDimensions(Ort::CustomOpApi ort, const OrtValue* value) {\n OrtTensorTypeAndShapeInfo* info = ort.GetTensorTypeAndShape(value);\n std::vector::operator=(ort.GetTensorShape(info));\n ort.ReleaseTensorTypeAndShapeInfo(info);\n }\n};\n#endif // ORT_MMCV_UTILS_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/reduce_ops.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_REDUCE_OPS_H\n#define ONNXRUNTIME_REDUCE_OPS_H\n\n#include \n\nstruct MMCVCumMaxKernel {\n public:\n MMCVCumMaxKernel(Ort::CustomOpApi ort, const OrtKernelInfo* info)\n : ort_(ort) {\n dim_ = ort_.KernelInfoGetAttribute(info, \"dim\");\n\n // create allocator\n allocator_ = Ort::AllocatorWithDefaultOptions();\n }\n\n void Compute(OrtKernelContext* context);\n\n private:\n Ort::CustomOpApi ort_;\n Ort::AllocatorWithDefaultOptions allocator_;\n\n int64_t dim_;\n};\n\nstruct MMCVCumMinKernel {\n public:\n MMCVCumMinKernel(Ort::CustomOpApi ort, const OrtKernelInfo* info)\n : ort_(ort) {\n dim_ = ort_.KernelInfoGetAttribute(info, \"dim\");\n\n // create allocator\n allocator_ = Ort::AllocatorWithDefaultOptions();\n }\n\n void Compute(OrtKernelContext* context);\n\n private:\n Ort::CustomOpApi ort_;\n Ort::AllocatorWithDefaultOptions allocator_;\n\n int64_t dim_;\n};\n\nstruct MMCVCumMaxCustomOp\n : Ort::CustomOpBase {\n void* CreateKernel(Ort::CustomOpApi api, const OrtKernelInfo* info) const {\n return new MMCVCumMaxKernel(api, info);\n }\n\n const char* GetName() const { return \"cummax\"; }\n\n size_t GetInputTypeCount() const { return 1; }\n ONNXTensorElementDataType GetInputType(size_t) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n size_t GetOutputTypeCount() const { return 2; }\n ONNXTensorElementDataType GetOutputType(size_t index) const {\n if (index == 1) return ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64;\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n // force cpu\n const char* GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n };\n};\n\nstruct MMCVCumMinCustomOp\n : Ort::CustomOpBase {\n void* CreateKernel(Ort::CustomOpApi api, const OrtKernelInfo* info) const {\n return new MMCVCumMinKernel(api, info);\n }\n\n const char* GetName() const { return \"cummin\"; }\n\n size_t GetInputTypeCount() const { return 1; }\n ONNXTensorElementDataType GetInputType(size_t) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n size_t GetOutputTypeCount() const { return 2; }\n ONNXTensorElementDataType GetOutputType(size_t index) const {\n if (index == 1) return ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64;\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n // force cpu\n const char* GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n };\n};\n\n#endif // ONNXRUNTIME_REDUCE_OPS_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/roi_align.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_ROI_ALIGN_H\n#define ONNXRUNTIME_ROI_ALIGN_H\n\n#include \n#include \n\n#include \n#include \n#include \n#include \n\nstruct MMCVRoiAlignKernel {\n public:\n MMCVRoiAlignKernel(Ort::CustomOpApi ort, const OrtKernelInfo* info)\n : ort_(ort) {\n aligned_ = ort_.KernelInfoGetAttribute(info, \"aligned\");\n aligned_height_ =\n ort_.KernelInfoGetAttribute(info, \"output_height\");\n aligned_width_ = ort_.KernelInfoGetAttribute(info, \"output_width\");\n pool_mode_ = ort_.KernelInfoGetAttribute(info, \"mode\");\n sampling_ratio_ =\n ort_.KernelInfoGetAttribute(info, \"sampling_ratio\");\n spatial_scale_ = ort_.KernelInfoGetAttribute(info, \"spatial_scale\");\n }\n\n void Compute(OrtKernelContext* context);\n\n private:\n Ort::CustomOpApi ort_;\n\n int aligned_height_;\n int aligned_width_;\n float spatial_scale_;\n int sampling_ratio_;\n std::string pool_mode_;\n int aligned_;\n};\n\nstruct MMCVRoiAlignCustomOp\n : Ort::CustomOpBase {\n void* CreateKernel(Ort::CustomOpApi api, const OrtKernelInfo* info) const {\n return new MMCVRoiAlignKernel(api, info);\n }\n const char* GetName() const { return \"MMCVRoiAlign\"; }\n\n size_t GetInputTypeCount() const { return 2; }\n ONNXTensorElementDataType GetInputType(size_t) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n }\n\n size_t GetOutputTypeCount() const { return 1; }\n ONNXTensorElementDataType GetOutputType(size_t) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n }\n\n // force cpu\n const char* GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n }\n};\n#endif // ONNXRUNTIME_ROI_ALIGN_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/roi_align_rotated.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_ROI_ALIGN_ROTATED_H\n#define ONNXRUNTIME_ROI_ALIGN_ROTATED_H\n\n#include \n#include \n\n#include \n#include \n#include \n#include \n\nstruct MMCVRoIAlignRotatedKernel {\n public:\n MMCVRoIAlignRotatedKernel(Ort::CustomOpApi ort, const OrtKernelInfo* info)\n : ort_(ort) {\n aligned_height_ =\n ort_.KernelInfoGetAttribute(info, \"output_height\");\n aligned_width_ = ort_.KernelInfoGetAttribute(info, \"output_width\");\n sampling_ratio_ =\n ort_.KernelInfoGetAttribute(info, \"sampling_ratio\");\n spatial_scale_ = ort_.KernelInfoGetAttribute(info, \"spatial_scale\");\n aligned_ = ort_.KernelInfoGetAttribute(info, \"aligned\");\n clockwise_ = ort_.KernelInfoGetAttribute(info, \"clockwise\");\n }\n\n void Compute(OrtKernelContext* context);\n\n private:\n Ort::CustomOpApi ort_;\n int aligned_height_;\n int aligned_width_;\n float spatial_scale_;\n int sampling_ratio_;\n int aligned_;\n int clockwise_;\n};\n\nstruct MMCVRoIAlignRotatedCustomOp\n : Ort::CustomOpBase {\n void* CreateKernel(Ort::CustomOpApi api, const OrtKernelInfo* info) const {\n return new MMCVRoIAlignRotatedKernel(api, info);\n }\n const char* GetName() const { return \"MMCVRoIAlignRotated\"; }\n\n size_t GetInputTypeCount() const { return 2; }\n ONNXTensorElementDataType GetInputType(size_t) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n }\n\n size_t GetOutputTypeCount() const { return 1; }\n ONNXTensorElementDataType GetOutputType(size_t) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n }\n\n // force cpu\n const char* GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n }\n};\n#endif // ONNXRUNTIME_ROI_ALIGN_ROTATED_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/onnxruntime/soft_nms.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ONNXRUNTIME_SOFT_NMS_H\n#define ONNXRUNTIME_SOFT_NMS_H\n#include \n\nstruct SoftNmsKernel {\n SoftNmsKernel(OrtApi api, const OrtKernelInfo *info);\n\n void Compute(OrtKernelContext *context);\n\n protected:\n OrtApi api_;\n Ort::CustomOpApi ort_;\n const OrtKernelInfo *info_;\n Ort::AllocatorWithDefaultOptions allocator_;\n\n float iou_threshold_;\n float sigma_;\n float min_score_;\n int64_t method_;\n int64_t offset_;\n};\n\nstruct SoftNmsOp : Ort::CustomOpBase {\n void *CreateKernel(OrtApi api, const OrtKernelInfo *info) const {\n return new SoftNmsKernel(api, info);\n };\n\n const char *GetName() const { return \"SoftNonMaxSuppression\"; };\n\n size_t GetInputTypeCount() const { return 2; };\n ONNXTensorElementDataType GetInputType(size_t /*index*/) const {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n size_t GetOutputTypeCount() const { return 2; };\n ONNXTensorElementDataType GetOutputType(size_t index) const {\n if (index == 1) {\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64;\n }\n return ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT;\n };\n\n // force cpu\n const char *GetExecutionProviderType() const {\n return \"CPUExecutionProvider\";\n };\n};\n#endif // ONNXRUNTIME_SOFT_NMS_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/active_rotated_filter.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/csuhan/s2anet/blob/master/mmdet/ops/orn/src/ActiveRotatingFilter.h\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid active_rotated_filter_forward_impl(const Tensor input,\n const Tensor indices, Tensor output) {\n DISPATCH_DEVICE_IMPL(active_rotated_filter_forward_impl, input, indices,\n output);\n}\n\nvoid active_rotated_filter_backward_impl(const Tensor grad_out,\n const Tensor indices, Tensor grad_in) {\n DISPATCH_DEVICE_IMPL(active_rotated_filter_backward_impl, grad_out, indices,\n grad_in);\n}\n\nvoid active_rotated_filter_forward(const Tensor input, const Tensor indices,\n Tensor output) {\n active_rotated_filter_forward_impl(input, indices, output);\n}\n\nvoid active_rotated_filter_backward(const Tensor grad_out, const Tensor indices,\n Tensor grad_in) {\n active_rotated_filter_backward_impl(grad_out, indices, grad_in);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/active_rotated_filter_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"active_rotated_filter_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid active_rotated_filter_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto input = buildATensor(ctx, ins[0]);\n auto indices = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n active_rotated_filter_forward(input, indices, output);\n}\n\nvoid active_rotated_filter_backward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto grad_out = buildATensor(ctx, ins[0]);\n auto indices = buildATensor(ctx, ins[1]);\n auto grad_in = buildATensor(ctx, outs[0]);\n active_rotated_filter_backward(grad_out, indices, grad_in);\n}\n#endif\n\nvoid active_rotated_filter_forward_cpu_parrots(\n HostContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto input = buildATensor(ctx, ins[0]);\n auto indices = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n active_rotated_filter_forward(input, indices, output);\n}\n\nvoid active_rotated_filter_backward_cpu_parrots(\n HostContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto grad_out = buildATensor(ctx, ins[0]);\n auto indices = buildATensor(ctx, ins[1]);\n auto grad_in = buildATensor(ctx, outs[0]);\n active_rotated_filter_backward(grad_out, indices, grad_in);\n}\n\nPARROTS_EXTENSION_REGISTER(active_rotated_filter_forward)\n .input(2)\n .output(1)\n .apply(active_rotated_filter_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(active_rotated_filter_forward_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(active_rotated_filter_backward)\n .input(2)\n .output(1)\n .apply(active_rotated_filter_backward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(active_rotated_filter_backward_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/active_rotated_filter_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ACTIVE_ROTATED_FILTER_PYTORCH_H\n#define ACTIVE_ROTATED_FILTER_PYTORCH_H\n#include \nusing namespace at;\n\nvoid active_rotated_filter_forward(const Tensor input, const Tensor indices,\n Tensor output);\n\nvoid active_rotated_filter_backward(const Tensor grad_out, const Tensor indices,\n Tensor grad_in);\n\n#endif // ACTIVE_ROTATED_FILTER_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/assign_score_withk.cpp", "content": "// Modified from\n// https://github.com/CVMI-Lab/PAConv/tree/main/scene_seg/lib/paconv_lib/src/gpu\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid assign_score_withk_forward_impl(int B, int N0, int N1, int M, int K, int O,\n int aggregate, const Tensor& points,\n const Tensor& centers,\n const Tensor& scores,\n const Tensor& knn_idx, Tensor& output) {\n DISPATCH_DEVICE_IMPL(assign_score_withk_forward_impl, B, N0, N1, M, K, O,\n aggregate, points, centers, scores, knn_idx, output);\n}\n\nvoid assign_score_withk_backward_impl(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores) {\n DISPATCH_DEVICE_IMPL(assign_score_withk_backward_impl, B, N0, N1, M, K, O,\n aggregate, grad_out, points, centers, scores, knn_idx,\n grad_points, grad_centers, grad_scores);\n}\n\nvoid assign_score_withk_forward(const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx,\n Tensor& output, int B, int N0, int N1, int M,\n int K, int O, int aggregate) {\n assign_score_withk_forward_impl(B, N0, N1, M, K, O, aggregate, points,\n centers, scores, knn_idx, output);\n}\n\nvoid assign_score_withk_backward(const Tensor& grad_out, const Tensor& points,\n const Tensor& centers, const Tensor& scores,\n const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores,\n int B, int N0, int N1, int M, int K, int O,\n int aggregate) {\n assign_score_withk_backward_impl(B, N0, N1, M, K, O, aggregate, grad_out,\n points, centers, scores, knn_idx,\n grad_points, grad_centers, grad_scores);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/assign_score_withk_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"assign_score_withk_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid assign_score_withk_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int B, N0, N1, M, K, O, aggregate;\n SSAttrs(attr)\n .get(\"B\", B)\n .get(\"N0\", N0)\n .get(\"N1\", N1)\n .get(\"M\", M)\n .get(\"K\", K)\n .get(\"O\", O)\n .get(\"aggregate\", aggregate)\n .done();\n\n const auto& points = buildATensor(ctx, ins[0]);\n const auto& centers = buildATensor(ctx, ins[1]);\n const auto& scores = buildATensor(ctx, ins[2]);\n const auto& knn_idx = buildATensor(ctx, ins[3]);\n\n auto output = buildATensor(ctx, outs[0]);\n assign_score_withk_forward(points, centers, scores, knn_idx, output, B, N0,\n N1, M, K, O, aggregate);\n}\n\nvoid assign_score_withk_backward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int B, N0, N1, M, K, O, aggregate;\n SSAttrs(attr)\n .get(\"B\", B)\n .get(\"N0\", N0)\n .get(\"N1\", N1)\n .get(\"M\", M)\n .get(\"K\", K)\n .get(\"O\", O)\n .get(\"aggregate\", aggregate)\n .done();\n\n const auto& grad_out = buildATensor(ctx, ins[0]);\n const auto& points = buildATensor(ctx, ins[1]);\n const auto& centers = buildATensor(ctx, ins[2]);\n const auto& scores = buildATensor(ctx, ins[3]);\n const auto& knn_idx = buildATensor(ctx, ins[4]);\n\n auto grad_points = buildATensor(ctx, outs[0]);\n auto grad_centers = buildATensor(ctx, outs[1]);\n auto grad_scores = buildATensor(ctx, outs[2]);\n assign_score_withk_backward(grad_out, points, centers, scores, knn_idx,\n grad_points, grad_centers, grad_scores, B, N0, N1,\n M, K, O, aggregate);\n}\n\nPARROTS_EXTENSION_REGISTER(assign_score_withk_forward)\n .attr(\"B\")\n .attr(\"N0\")\n .attr(\"N1\")\n .attr(\"M\")\n .attr(\"K\")\n .attr(\"O\")\n .attr(\"aggregate\")\n .input(4)\n .output(1)\n .apply(assign_score_withk_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(assign_score_withk_backward)\n .attr(\"B\")\n .attr(\"N0\")\n .attr(\"N1\")\n .attr(\"M\")\n .attr(\"K\")\n .attr(\"O\")\n .attr(\"aggregate\")\n .input(5)\n .output(3)\n .apply(assign_score_withk_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/assign_score_withk_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ASSIGN_SCORE_WITHK_PYTORCH_H\n#define ASSIGN_SCORE_WITHK_PYTORCH_H\n#include \nusing namespace at;\n\nvoid assign_score_withk_forward(const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx,\n Tensor& output, int B, int N0, int N1, int M,\n int K, int O, int aggregate);\n\nvoid assign_score_withk_backward(const Tensor& grad_out, const Tensor& points,\n const Tensor& centers, const Tensor& scores,\n const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores,\n int B, int N0, int N1, int M, int K, int O,\n int aggregate);\n\n#endif // ASSIGN_SCORE_WITHK_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/ball_query._parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"ball_query_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid ball_query_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, n, m, nsample;\n float min_radius, max_radius;\n SSAttrs(attr)\n .get(\"b\", b)\n .get(\"n\", n)\n .get(\"m\", m)\n .get(\"nsample\", nsample)\n .get(\"min_radius\", min_radius)\n .get(\"max_radius\", max_radius)\n .done();\n\n const auto& center_xyz = buildATensor(ctx, ins[0]);\n const auto& xyz = buildATensor(ctx, ins[1]);\n auto idx = buildATensor(ctx, outs[0]);\n ball_query_forward(center_xyz, xyz, idx, b, n, m, min_radius, max_radius,\n nsample);\n}\n\nPARROTS_EXTENSION_REGISTER(ball_query_forward)\n .attr(\"b\")\n .attr(\"n\")\n .attr(\"m\")\n .attr(\"nsample\")\n .attr(\"min_radius\")\n .attr(\"max_radius\")\n .input(2)\n .output(1)\n .apply(ball_query_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/ball_query.cpp", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/ball_query.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid ball_query_forward_impl(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx) {\n DISPATCH_DEVICE_IMPL(ball_query_forward_impl, b, n, m, min_radius, max_radius,\n nsample, new_xyz, xyz, idx);\n}\n\nvoid ball_query_forward(Tensor new_xyz_tensor, Tensor xyz_tensor,\n Tensor idx_tensor, int b, int n, int m,\n float min_radius, float max_radius, int nsample) {\n ball_query_forward_impl(b, n, m, min_radius, max_radius, nsample,\n new_xyz_tensor, xyz_tensor, idx_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/ball_query_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef BALL_QUERY_PYTORCH_H\n#define BALL_QUERY_PYTORCH_H\n#include \nusing namespace at;\n\nvoid ball_query_forward(const Tensor new_xyz, const Tensor xyz, Tensor idx,\n int b, int n, int m, float min_radius, float max_radius,\n int nsample);\n\n#endif // BALL_QUERY_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/bbox_overlaps.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid bbox_overlaps_impl(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset) {\n DISPATCH_DEVICE_IMPL(bbox_overlaps_impl, bboxes1, bboxes2, ious, mode,\n aligned, offset);\n}\n\nvoid bbox_overlaps(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset) {\n bbox_overlaps_impl(bboxes1, bboxes2, ious, mode, aligned, offset);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/bbox_overlaps_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"bbox_overlaps_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\n/*\n * void bbox_overlaps_cuda(const Tensor bboxes1, const Tensor bboxes2, Tensor\n * ious, const int mode, const bool aligned, const int offset);\n */\nvoid bbox_overlaps_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int mode, offset;\n bool aligned;\n SSAttrs(attr)\n .get(\"mode\", mode)\n .get(\"aligned\", aligned)\n .get(\"offset\", offset)\n .done();\n\n const auto& bboxes1 = buildATensor(ctx, ins[0]);\n const auto& bboxes2 = buildATensor(ctx, ins[1]);\n auto ious = buildATensor(ctx, outs[0]);\n bbox_overlaps_cuda(bboxes1, bboxes2, ious, mode, aligned, offset);\n}\n\nPARROTS_EXTENSION_REGISTER(bbox_overlaps)\n .attr(\"mode\")\n .attr(\"aligned\")\n .attr(\"offset\")\n .input(2)\n .output(1)\n .apply(bbox_overlaps_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/bbox_overlaps_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef BBOX_OVERLAPS_PYTORCH_H\n#define BBOX_OVERLAPS_PYTORCH_H\n#include \nusing namespace at;\n\nvoid bbox_overlaps_cuda(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset);\n\n#endif // BBOX_OVERLAPS_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/border_align.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid border_align_forward_impl(const Tensor &input, const Tensor &boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size) {\n DISPATCH_DEVICE_IMPL(border_align_forward_impl, input, boxes, output,\n argmax_idx, pool_size);\n}\n\nvoid border_align_backward_impl(const Tensor &grad_output, const Tensor &boxes,\n const Tensor &argmax_idx, Tensor grad_input,\n const int pool_size) {\n DISPATCH_DEVICE_IMPL(border_align_backward_impl, grad_output, boxes,\n argmax_idx, grad_input, pool_size);\n}\n\nvoid border_align_forward(const Tensor &input, const Tensor &boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size) {\n border_align_forward_impl(input, boxes, output, argmax_idx, pool_size);\n}\n\nvoid border_align_backward(const Tensor &grad_output, const Tensor &boxes,\n const Tensor &argmax_idx, Tensor grad_input,\n const int pool_size) {\n border_align_backward_impl(grad_output, boxes, argmax_idx, grad_input,\n pool_size);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/border_align_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"border_align_pytorch.h\"\n\nusing namespace parrots;\n\nvoid border_align_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pool_size;\n SSAttrs(attr).get(\"pool_size\", pool_size).done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& boxes = buildATensor(ctx, ins[1]);\n\n auto output = buildATensor(ctx, outs[0]);\n auto argmax_idx = buildATensor(ctx, outs[1]);\n border_align_forward_cuda(input, boxes, output, argmax_idx, pool_size);\n}\n\nvoid border_align_backward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pool_size;\n SSAttrs(attr).get(\"pool_size\", pool_size).done();\n\n const auto& top_grad = buildATensor(ctx, ins[0]);\n const auto& boxes = buildATensor(ctx, ins[1]);\n const auto& argmax_idx = buildATensor(ctx, ins[2]);\n\n auto bottom_grad = buildATensor(ctx, outs[0]);\n border_align_backward_cuda(top_grad, boxes, argmax_idx, bottom_grad,\n pool_size);\n}\n\nPARROTS_EXTENSION_REGISTER(border_align_forward)\n .attr(\"pool_size\")\n .input(2)\n .output(2)\n .apply(border_align_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(border_align_backward)\n .attr(\"pool_size\")\n .input(3)\n .output(1)\n .apply(border_align_backward_cuda_parrots)\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/border_align_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef BORDER_ALIGN_PYTORCH_H\n#define BORDER_ALIGN_PYTORCH_H\n#include \nusing namespace at;\n\n#ifdef MMCV_WITH_CUDA\nvoid border_align_forward_cuda(const Tensor &input, const Tensor &boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size);\n\nvoid border_align_backward_cuda(const Tensor &grad_output, const Tensor &boxes,\n const Tensor &argmax_idx, Tensor grad_input,\n const int pool_size);\n#endif\n\n#endif // BORDER_ALIGN_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/box_iou_rotated.cpp", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated.h\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid box_iou_rotated_impl(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned) {\n DISPATCH_DEVICE_IMPL(box_iou_rotated_impl, boxes1, boxes2, ious, mode_flag,\n aligned);\n}\n\n// Interface for Python\n// inline is needed to prevent multiple function definitions when this header is\n// included by different cpps\nvoid box_iou_rotated(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned) {\n box_iou_rotated_impl(boxes1, boxes2, ious, mode_flag, aligned);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/box_iou_rotated_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"box_iou_rotated_pytorch.h\"\n\nusing namespace parrots;\n\n/*\n * void box_iou_rotated_cpu(const Tensor boxes1, const Tensor boxes2, Tensor\n * ious, const int mode_flag, const bool aligned);\n */\nvoid box_iou_rotated_cpu_parrots(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n bool aligned;\n int mode_flag;\n SSAttrs(attr)\n .get(\"aligned\", aligned)\n .get(\"mode_flag\", mode_flag)\n .done();\n\n const auto& boxes1 = buildATensor(ctx, ins[0]);\n const auto& boxes2 = buildATensor(ctx, ins[1]);\n auto ious = buildATensor(ctx, outs[0]);\n box_iou_rotated_cpu(boxes1, boxes2, ious, mode_flag, aligned);\n}\n\n#ifdef MMCV_WITH_CUDA\n/*\n * void box_iou_rotated_cuda(const Tensor boxes1, const Tensor boxes2, Tensor\n * ious, const int mode_flag, const bool aligned);\n */\nvoid box_iou_rotated_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n bool aligned;\n int mode_flag;\n SSAttrs(attr)\n .get(\"aligned\", aligned)\n .get(\"mode_flag\", mode_flag)\n .done();\n\n const auto& boxes1 = buildATensor(ctx, ins[0]);\n const auto& boxes2 = buildATensor(ctx, ins[1]);\n auto ious = buildATensor(ctx, outs[0]);\n box_iou_rotated_cuda(boxes1, boxes2, ious, mode_flag, aligned);\n}\n#endif\n\nPARROTS_EXTENSION_REGISTER(box_iou_rotated)\n .attr(\"aligned\")\n .attr(\"mode_flag\")\n .input(2)\n .output(1)\n .apply(box_iou_rotated_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(box_iou_rotated_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/box_iou_rotated_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef BOX_IOU_ROTATED_PYTORCH_H\n#define BOX_IOU_ROTATED_PYTORCH_H\n#include \nusing namespace at;\n\nvoid box_iou_rotated_cpu(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned);\n\n#ifdef MMCV_WITH_CUDA\nvoid box_iou_rotated_cuda(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned);\n#endif\n\n#endif // BOX_IOU_ROTATED_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/carafe.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid carafe_forward_impl(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor) {\n DISPATCH_DEVICE_IMPL(carafe_forward_impl, features, masks, rfeatures, routput,\n rmasks, output, kernel_size, group_size, scale_factor);\n}\n\nvoid carafe_backward_impl(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad,\n Tensor bottom_grad, Tensor mask_grad, int kernel_size,\n int group_size, int scale_factor) {\n DISPATCH_DEVICE_IMPL(carafe_backward_impl, top_grad, rfeatures, masks,\n rtop_grad, rbottom_grad_hs, rbottom_grad, rmask_grad,\n bottom_grad, mask_grad, kernel_size, group_size,\n scale_factor);\n}\n\nvoid carafe_forward(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor) {\n carafe_forward_impl(features, masks, rfeatures, routput, rmasks, output,\n kernel_size, group_size, scale_factor);\n}\n\nvoid carafe_backward(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad, Tensor bottom_grad,\n Tensor mask_grad, int kernel_size, int group_size,\n int scale_factor) {\n carafe_backward_impl(top_grad, rfeatures, masks, rtop_grad, rbottom_grad_hs,\n rbottom_grad, rmask_grad, bottom_grad, mask_grad,\n kernel_size, group_size, scale_factor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/carafe_naive.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid carafe_naive_forward_impl(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size,\n int scale_factor) {\n DISPATCH_DEVICE_IMPL(carafe_naive_forward_impl, features, masks, output,\n kernel_size, group_size, scale_factor);\n}\n\nvoid carafe_naive_backward_impl(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size,\n int scale_factor) {\n DISPATCH_DEVICE_IMPL(carafe_naive_backward_impl, top_grad, features, masks,\n bottom_grad, mask_grad, kernel_size, group_size,\n scale_factor);\n}\n\nvoid carafe_naive_forward(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size, int scale_factor) {\n carafe_naive_forward_impl(features, masks, output, kernel_size, group_size,\n scale_factor);\n}\n\nvoid carafe_naive_backward(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size, int scale_factor) {\n carafe_naive_backward_impl(top_grad, features, masks, bottom_grad, mask_grad,\n kernel_size, group_size, scale_factor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/carafe_naive_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"carafe_naive_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\n/*void carafe_naive_forward_cuda(Tensor features, Tensor masks, Tensor output,\n * int kernel_size, int group_size,\n * int scale_factor)\n */\nvoid carafe_naive_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_size, group_size, scale_factor;\n SSAttrs(attr)\n .get(\"kernel_size\", kernel_size)\n .get(\"group_size\", group_size)\n .get(\"scale_factor\", scale_factor)\n .done();\n\n const auto& features = buildATensor(ctx, ins[0]);\n const auto& masks = buildATensor(ctx, ins[1]);\n\n auto output = buildATensor(ctx, outs[0]);\n carafe_naive_forward_cuda(features, masks, output, kernel_size, group_size,\n scale_factor);\n}\n\n/*void carafe_naive_backward_cuda(Tensor top_grad, Tensor features, Tensor\n * masks, Tensor bottom_grad, Tensor mask_grad, int kernel_size, int group_size,\n * int scale_factor);\n */\nvoid carafe_naive_backward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_size, group_size, scale_factor;\n SSAttrs(attr)\n .get(\"kernel_size\", kernel_size)\n .get(\"group_size\", group_size)\n .get(\"scale_factor\", scale_factor)\n .done();\n\n const auto& top_grad = buildATensor(ctx, ins[0]);\n const auto& features = buildATensor(ctx, ins[1]);\n const auto& masks = buildATensor(ctx, ins[2]);\n\n auto bottom_grad = buildATensor(ctx, outs[0]);\n auto mask_grad = buildATensor(ctx, outs[1]);\n carafe_naive_backward_cuda(top_grad, features, masks, bottom_grad, mask_grad,\n kernel_size, group_size, scale_factor);\n}\n\nPARROTS_EXTENSION_REGISTER(carafe_naive_forward)\n .attr(\"kernel_size\")\n .attr(\"group_size\")\n .attr(\"scale_factor\")\n .input(2)\n .output(1)\n .apply(carafe_naive_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(carafe_naive_backward)\n .attr(\"kernel_size\")\n .attr(\"group_size\")\n .attr(\"scale_factor\")\n .input(3)\n .output(2)\n .apply(carafe_naive_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/carafe_naive_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CARAFE_NAIVE_PYTORCH_H\n#define CARAFE_NAIVE_PYTORCH_H\n#include \nusing namespace at;\n\nvoid carafe_naive_forward_cuda(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size,\n int scale_factor);\n\nvoid carafe_naive_backward_cuda(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size,\n int scale_factor);\n#endif // CARAFE_NAIVE_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/carafe_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"carafe_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\n/*\n * void carafe_forward_cuda(Tensor features, Tensor masks, Tensor rfeatures,\n * Tensor routput, Tensor rmasks, Tensor output,\n * int kernel_size, int group_size, int scale_factor);\n */\nvoid carafe_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_size, group_size, scale_factor;\n SSAttrs(attr)\n .get(\"kernel_size\", kernel_size)\n .get(\"group_size\", group_size)\n .get(\"scale_factor\", scale_factor)\n .done();\n\n const auto& features = buildATensor(ctx, ins[0]);\n const auto& masks = buildATensor(ctx, ins[1]);\n\n auto rfeatures = buildATensor(ctx, outs[0]);\n auto routput = buildATensor(ctx, outs[1]);\n auto rmasks = buildATensor(ctx, outs[2]);\n auto output = buildATensor(ctx, outs[3]);\n\n carafe_forward_cuda(features, masks, rfeatures, routput, rmasks, output,\n kernel_size, group_size, scale_factor);\n}\n\n/*\n * void carafe_backward_cuda(Tensor top_grad, Tensor rfeatures, Tensor masks,\n * Tensor rtop_grad, Tensor rbottom_grad_hs,\n * Tensor rbottom_grad, Tensor rmask_grad,\n * Tensor bottom_grad, Tensor mask_grad, int\n * kernel_size, int group_size, int scale_factor);\n */\nvoid carafe_backward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_size, group_size, scale_factor;\n SSAttrs(attr)\n .get(\"kernel_size\", kernel_size)\n .get(\"group_size\", group_size)\n .get(\"scale_factor\", scale_factor)\n .done();\n\n const auto& top_grad = buildATensor(ctx, ins[0]);\n const auto& rfeatures = buildATensor(ctx, ins[1]);\n const auto& masks = buildATensor(ctx, ins[2]);\n\n auto rtop_grad = buildATensor(ctx, outs[0]);\n auto rbottom_grad_hs = buildATensor(ctx, outs[1]);\n auto rbottom_grad = buildATensor(ctx, outs[2]);\n auto rmask_grad = buildATensor(ctx, outs[3]);\n auto bottom_grad = buildATensor(ctx, outs[4]);\n auto mask_grad = buildATensor(ctx, outs[5]);\n\n carafe_backward_cuda(top_grad, rfeatures, masks, rtop_grad, rbottom_grad_hs,\n rbottom_grad, rmask_grad, bottom_grad, mask_grad,\n kernel_size, group_size, scale_factor);\n}\n\nPARROTS_EXTENSION_REGISTER(carafe_forward)\n .attr(\"kernel_size\")\n .attr(\"group_size\")\n .attr(\"scale_factor\")\n .input(2)\n .output(4)\n .apply(carafe_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(carafe_backward)\n .attr(\"kernel_size\")\n .attr(\"group_size\")\n .attr(\"scale_factor\")\n .input(3)\n .output(6)\n .apply(carafe_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/carafe_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CARAFE_PYTORCH_H\n#define CARAFE_PYTORCH_H\n#include \nusing namespace at;\n\nvoid carafe_forward_cuda(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor);\n\nvoid carafe_backward_cuda(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad,\n Tensor bottom_grad, Tensor mask_grad, int kernel_size,\n int group_size, int scale_factor);\n#endif // CARAFE_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/contour_expand.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// It is modified from https://github.com/whai362/PSENet\n#include \n#include \n\n#include \"pytorch_cpp_helper.hpp\"\n\nusing namespace std;\n\nclass Point2d {\n public:\n int x;\n int y;\n\n Point2d() : x(0), y(0) {}\n Point2d(int _x, int _y) : x(_x), y(_y) {}\n};\n\nvoid kernel_dilate(const uint8_t *data, IntArrayRef data_shape,\n const int *label_map, int &label_num, int &min_area,\n vector> &text_line) {\n std::vector area(label_num + 1);\n int kernel_num = data_shape[0];\n int height = data_shape[1];\n int width = data_shape[2];\n\n for (int x = 0; x < height; ++x) {\n for (int y = 0; y < width; ++y) {\n int label = label_map[x * width + y];\n if (label == 0) continue;\n area[label] += 1;\n }\n }\n\n queue queue, next_queue;\n for (int x = 0; x < height; ++x) {\n vector row(width);\n for (int y = 0; y < width; ++y) {\n int label = label_map[x * width + y];\n if (label == 0) continue;\n if (area[label] < min_area) continue;\n\n Point2d point(x, y);\n queue.push(point);\n row[y] = label;\n }\n text_line.emplace_back(row);\n }\n\n int dx[] = {-1, 1, 0, 0};\n int dy[] = {0, 0, -1, 1};\n vector kernel_step(kernel_num);\n std::for_each(kernel_step.begin(), kernel_step.end(),\n [=](int &k) { return k * height * width; });\n\n for (int kernel_id = kernel_num - 2; kernel_id >= 0; --kernel_id) {\n while (!queue.empty()) {\n Point2d point = queue.front();\n queue.pop();\n int x = point.x;\n int y = point.y;\n int label = text_line[x][y];\n\n bool is_edge = true;\n for (int d = 0; d < 4; ++d) {\n int tmp_x = x + dx[d];\n int tmp_y = y + dy[d];\n\n if (tmp_x < 0 || tmp_x >= height) continue;\n if (tmp_y < 0 || tmp_y >= width) continue;\n int kernel_value = data[kernel_step[kernel_id] + tmp_x * width + tmp_y];\n if (kernel_value == 0) continue;\n if (text_line[tmp_x][tmp_y] > 0) continue;\n\n Point2d point(tmp_x, tmp_y);\n queue.push(point);\n text_line[tmp_x][tmp_y] = label;\n is_edge = false;\n }\n\n if (is_edge) {\n next_queue.push(point);\n }\n }\n swap(queue, next_queue);\n }\n}\n\nstd::vector> contour_expand(Tensor kernel_mask,\n Tensor internal_kernel_label,\n int min_kernel_area,\n int kernel_num) {\n kernel_mask = kernel_mask.contiguous();\n internal_kernel_label = internal_kernel_label.contiguous();\n assert(kernel_mask.dim() == 3);\n assert(internal_kernel_label.dim() == 2);\n assert(kernel_mask.size(1) == internal_kernel_label.size(0));\n assert(kernel_mask.size(2) == internal_kernel_label.size(1));\n CHECK_CPU_INPUT(kernel_mask);\n CHECK_CPU_INPUT(internal_kernel_label);\n auto ptr_data = kernel_mask.data_ptr();\n IntArrayRef data_shape = kernel_mask.sizes();\n\n auto data_label_map = internal_kernel_label.data_ptr();\n IntArrayRef label_map_shape = internal_kernel_label.sizes();\n vector> text_line;\n\n kernel_dilate(ptr_data, data_shape, data_label_map, kernel_num,\n min_kernel_area, text_line);\n\n return text_line;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/contour_expand_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"contour_expand_pytorch.h\"\n\nusing namespace parrots;\nusing namespace std;\n\ntemplate \nvoid contour_expand_parrots(T& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int min_kernel_area, kernel_num;\n SSAttrs(attr)\n .get(\"min_kernel_area\", min_kernel_area)\n .get(\"kernel_num\", kernel_num)\n .done();\n at::Tensor kernel_mask;\n at::Tensor internal_kernel_label;\n kernel_mask = buildATensor(ctx, ins[0]);\n internal_kernel_label = buildATensor(ctx, ins[1]);\n auto out = contour_expand(kernel_mask, internal_kernel_label, min_kernel_area,\n kernel_num);\n int n = out.size(), m = 0;\n for (int i = 0; i < n; ++i)\n if (m < out[i].size()) m = out[i].size();\n auto options = torch::TensorOptions().dtype(at::kInt);\n auto tensor = torch::zeros({n, m}, options);\n for (int i = 0; i < n; i++)\n tensor.slice(0, i, i + 1) =\n torch::from_blob(out[i].data(), {out[i].size()}, options);\n updateDArray(ctx, tensor, outs[0]);\n}\n\nPARROTS_EXTENSION_REGISTER(contour_expand)\n .attr(\"min_kernel_area\")\n .attr(\"kernel_num\")\n .input(2)\n .output(1)\n .apply(contour_expand_parrots)\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/contour_expand_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CONTOUR_EXPAND_PYTORCH_H\n#define CONTOUR_EXPAND_PYTORCH_H\n#include \nusing namespace at;\n\nstd::vector> contour_expand(Tensor kernel_mask,\n Tensor internal_kernel_label,\n int min_kernel_area,\n int kernel_num);\n\n#endif // CONTOUR_EXPAND_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/convex_iou.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// modified from\n// https://github.com/SDL-GuoZonghao/BeyondBoundingBox/tree/main/mmdet/ops/iou/src\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid convex_iou_impl(const Tensor pointsets, const Tensor polygons,\n Tensor ious) {\n DISPATCH_DEVICE_IMPL(convex_iou_impl, pointsets, polygons, ious);\n}\n\nvoid convex_iou(const Tensor pointsets, const Tensor polygons, Tensor ious) {\n convex_iou_impl(pointsets, polygons, ious);\n}\n\nvoid convex_giou_impl(const Tensor pointsets, const Tensor polygons,\n Tensor output) {\n DISPATCH_DEVICE_IMPL(convex_giou_impl, pointsets, polygons, output);\n}\n\nvoid convex_giou(const Tensor pointsets, const Tensor polygons, Tensor output) {\n convex_giou_impl(pointsets, polygons, output);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/convex_iou_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"convex_iou_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid convex_iou_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto pointsets = buildATensor(ctx, ins[0]);\n auto polygons = buildATensor(ctx, ins[1]);\n auto ious = buildATensor(ctx, outs[0]);\n convex_iou(pointsets, polygons, ious);\n}\n\nvoid convex_giou_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto pointsets = buildATensor(ctx, ins[0]);\n auto polygons = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n convex_giou(pointsets, polygons, output);\n}\n\nPARROTS_EXTENSION_REGISTER(convex_iou)\n .input(2)\n .output(1)\n .apply(convex_iou_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(convex_giou)\n .input(2)\n .output(1)\n .apply(convex_giou_forward_cuda_parrots)\n .done();\n\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/convex_iou_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CONVEX_IOU_PYTORCH_H\n#define CONVEX_IOU_PYTORCH_H\n#include \nusing namespace at;\n\nvoid convex_iou(const Tensor pointsets, const Tensor polygons, Tensor ious);\n\nvoid convex_giou(const Tensor pointsets, const Tensor polygons, Tensor output);\n\n#endif // RIROI_ALIGN_ROTATED_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/corner_pool.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/princeton-vl/CornerNet-Lite/tree/master/core/models/py_utils/_cpools/src\n#include \"pytorch_cpp_helper.hpp\"\n\nTensor bottom_pool_forward(Tensor input) {\n // Initialize output\n Tensor output = at::zeros_like(input);\n // Get height\n int64_t height = input.size(2);\n output.copy_(input);\n\n for (int64_t ind = 1; ind < height; ind <<= 1) {\n Tensor max_temp = at::slice(output, 2, ind, height);\n Tensor cur_temp = at::slice(output, 2, ind, height).clone();\n Tensor next_temp = at::slice(output, 2, 0, height - ind).clone();\n at::max_out(max_temp, cur_temp, next_temp);\n }\n\n return output;\n}\n\nTensor bottom_pool_backward(Tensor input, Tensor grad_output) {\n auto output = at::zeros_like(input);\n\n int32_t batch = input.size(0);\n int32_t channel = input.size(1);\n int32_t height = input.size(2);\n int32_t width = input.size(3);\n\n auto max_val = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kFloat));\n auto max_ind = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kLong));\n\n auto input_temp = input.select(2, 0);\n max_val.copy_(input_temp);\n\n max_ind.fill_(0);\n\n auto output_temp = output.select(2, 0);\n auto grad_output_temp = grad_output.select(2, 0);\n output_temp.copy_(grad_output_temp);\n\n auto un_max_ind = max_ind.unsqueeze(2);\n auto gt_mask = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kBool));\n auto max_temp = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kFloat));\n for (int32_t ind = 0; ind < height - 1; ++ind) {\n input_temp = input.select(2, ind + 1);\n at::gt_out(gt_mask, input_temp, max_val);\n\n at::masked_select_out(max_temp, input_temp, gt_mask);\n max_val.masked_scatter_(gt_mask, max_temp);\n max_ind.masked_fill_(gt_mask, ind + 1);\n\n grad_output_temp = grad_output.select(2, ind + 1).unsqueeze(2);\n output.scatter_add_(2, un_max_ind, grad_output_temp);\n }\n\n return output;\n}\n\nTensor left_pool_forward(Tensor input) {\n // Initialize output\n Tensor output = at::zeros_like(input);\n // Get width\n int64_t width = input.size(3);\n output.copy_(input);\n\n for (int64_t ind = 1; ind < width; ind <<= 1) {\n Tensor max_temp = at::slice(output, 3, 0, width - ind);\n Tensor cur_temp = at::slice(output, 3, 0, width - ind).clone();\n Tensor next_temp = at::slice(output, 3, ind, width).clone();\n at::max_out(max_temp, cur_temp, next_temp);\n }\n\n return output;\n}\n\nTensor left_pool_backward(Tensor input, Tensor grad_output) {\n auto output = at::zeros_like(input);\n\n int32_t batch = input.size(0);\n int32_t channel = input.size(1);\n int32_t height = input.size(2);\n int32_t width = input.size(3);\n\n auto max_val = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kFloat));\n auto max_ind = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kLong));\n\n auto input_temp = input.select(3, width - 1);\n max_val.copy_(input_temp);\n\n max_ind.fill_(width - 1);\n\n auto output_temp = output.select(3, width - 1);\n auto grad_output_temp = grad_output.select(3, width - 1);\n output_temp.copy_(grad_output_temp);\n\n auto un_max_ind = max_ind.unsqueeze(3);\n auto gt_mask = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kBool));\n auto max_temp = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kFloat));\n for (int32_t ind = 1; ind < width; ++ind) {\n input_temp = input.select(3, width - ind - 1);\n at::gt_out(gt_mask, input_temp, max_val);\n\n at::masked_select_out(max_temp, input_temp, gt_mask);\n max_val.masked_scatter_(gt_mask, max_temp);\n max_ind.masked_fill_(gt_mask, width - ind - 1);\n\n grad_output_temp = grad_output.select(3, width - ind - 1).unsqueeze(3);\n output.scatter_add_(3, un_max_ind, grad_output_temp);\n }\n\n return output;\n}\n\nTensor right_pool_forward(Tensor input) {\n // Initialize output\n Tensor output = at::zeros_like(input);\n // Get width\n int64_t width = input.size(3);\n output.copy_(input);\n\n for (int64_t ind = 1; ind < width; ind <<= 1) {\n Tensor max_temp = at::slice(output, 3, ind, width);\n Tensor cur_temp = at::slice(output, 3, ind, width).clone();\n Tensor next_temp = at::slice(output, 3, 0, width - ind).clone();\n at::max_out(max_temp, cur_temp, next_temp);\n }\n\n return output;\n}\n\nTensor right_pool_backward(Tensor input, Tensor grad_output) {\n Tensor output = at::zeros_like(input);\n\n int32_t batch = input.size(0);\n int32_t channel = input.size(1);\n int32_t height = input.size(2);\n int32_t width = input.size(3);\n\n auto max_val = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kFloat));\n auto max_ind = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kLong));\n\n auto input_temp = input.select(3, 0);\n max_val.copy_(input_temp);\n\n max_ind.fill_(0);\n\n auto output_temp = output.select(3, 0);\n auto grad_output_temp = grad_output.select(3, 0);\n output_temp.copy_(grad_output_temp);\n\n auto un_max_ind = max_ind.unsqueeze(3);\n auto gt_mask = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kBool));\n auto max_temp = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kFloat));\n for (int32_t ind = 0; ind < width - 1; ++ind) {\n input_temp = input.select(3, ind + 1);\n at::gt_out(gt_mask, input_temp, max_val);\n\n at::masked_select_out(max_temp, input_temp, gt_mask);\n max_val.masked_scatter_(gt_mask, max_temp);\n max_ind.masked_fill_(gt_mask, ind + 1);\n\n grad_output_temp = grad_output.select(3, ind + 1).unsqueeze(3);\n output.scatter_add_(3, un_max_ind, grad_output_temp);\n }\n\n return output;\n}\n\nTensor top_pool_forward(Tensor input) {\n // Initialize output\n Tensor output = at::zeros_like(input);\n // Get height\n int64_t height = input.size(2);\n output.copy_(input);\n\n for (int64_t ind = 1; ind < height; ind <<= 1) {\n Tensor max_temp = at::slice(output, 2, 0, height - ind);\n Tensor cur_temp = at::slice(output, 2, 0, height - ind).clone();\n Tensor next_temp = at::slice(output, 2, ind, height).clone();\n at::max_out(max_temp, cur_temp, next_temp);\n }\n\n return output;\n}\n\nTensor top_pool_backward(Tensor input, Tensor grad_output) {\n auto output = at::zeros_like(input);\n\n int32_t batch = input.size(0);\n int32_t channel = input.size(1);\n int32_t height = input.size(2);\n int32_t width = input.size(3);\n\n auto max_val = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kFloat));\n auto max_ind = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kLong));\n\n auto input_temp = input.select(2, height - 1);\n max_val.copy_(input_temp);\n\n max_ind.fill_(height - 1);\n\n auto output_temp = output.select(2, height - 1);\n auto grad_output_temp = grad_output.select(2, height - 1);\n output_temp.copy_(grad_output_temp);\n\n auto un_max_ind = max_ind.unsqueeze(2);\n auto gt_mask = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kBool));\n auto max_temp = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kFloat));\n for (int32_t ind = 1; ind < height; ++ind) {\n input_temp = input.select(2, height - ind - 1);\n at::gt_out(gt_mask, input_temp, max_val);\n\n at::masked_select_out(max_temp, input_temp, gt_mask);\n max_val.masked_scatter_(gt_mask, max_temp);\n max_ind.masked_fill_(gt_mask, height - ind - 1);\n\n grad_output_temp = grad_output.select(2, height - ind - 1).unsqueeze(2);\n output.scatter_add_(2, un_max_ind, grad_output_temp);\n }\n\n return output;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/corner_pool_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"corner_pool_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid bottom_pool_forward_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input;\n input = buildATensor(ctx, ins[0]);\n auto out = bottom_pool_forward(input);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid bottom_pool_backward_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input, grad_output;\n input = buildATensor(ctx, ins[0]);\n grad_output = buildATensor(ctx, ins[1]);\n auto out = bottom_pool_backward(input, grad_output);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid left_pool_forward_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input;\n input = buildATensor(ctx, ins[0]);\n auto out = left_pool_forward(input);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid left_pool_backward_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input, grad_output;\n input = buildATensor(ctx, ins[0]);\n grad_output = buildATensor(ctx, ins[1]);\n auto out = left_pool_backward(input, grad_output);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid right_pool_forward_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input;\n input = buildATensor(ctx, ins[0]);\n auto out = right_pool_forward(input);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid right_pool_backward_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input, grad_output;\n input = buildATensor(ctx, ins[0]);\n grad_output = buildATensor(ctx, ins[1]);\n auto out = right_pool_backward(input, grad_output);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid top_pool_forward_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input;\n input = buildATensor(ctx, ins[0]);\n auto out = top_pool_forward(input);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid top_pool_backward_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input, grad_output;\n input = buildATensor(ctx, ins[0]);\n grad_output = buildATensor(ctx, ins[1]);\n auto out = top_pool_backward(input, grad_output);\n updateDArray(ctx, out, outs[0]);\n}\n#endif\n\nvoid bottom_pool_forward_parrots_cpu(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input;\n input = buildATensor(ctx, ins[0]);\n auto out = bottom_pool_forward(input);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid bottom_pool_backward_parrots_cpu(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input, grad_output;\n input = buildATensor(ctx, ins[0]);\n grad_output = buildATensor(ctx, ins[1]);\n auto out = bottom_pool_backward(input, grad_output);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid left_pool_forward_parrots_cpu(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input;\n input = buildATensor(ctx, ins[0]);\n auto out = left_pool_forward(input);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid left_pool_backward_parrots_cpu(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input, grad_output;\n input = buildATensor(ctx, ins[0]);\n grad_output = buildATensor(ctx, ins[1]);\n auto out = left_pool_backward(input, grad_output);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid right_pool_forward_parrots_cpu(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input;\n input = buildATensor(ctx, ins[0]);\n auto out = right_pool_forward(input);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid right_pool_backward_parrots_cpu(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input, grad_output;\n input = buildATensor(ctx, ins[0]);\n grad_output = buildATensor(ctx, ins[1]);\n auto out = right_pool_backward(input, grad_output);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid top_pool_forward_parrots_cpu(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input;\n input = buildATensor(ctx, ins[0]);\n auto out = top_pool_forward(input);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid top_pool_backward_parrots_cpu(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n at::Tensor input, grad_output;\n input = buildATensor(ctx, ins[0]);\n grad_output = buildATensor(ctx, ins[1]);\n auto out = top_pool_backward(input, grad_output);\n updateDArray(ctx, out, outs[0]);\n}\n\nPARROTS_EXTENSION_REGISTER(bottom_pool_forward)\n .input(1)\n .output(1)\n#ifdef MMCV_WITH_CUDA\n .apply(bottom_pool_forward_parrots)\n#endif\n .apply(bottom_pool_forward_parrots_cpu)\n .done();\n\nPARROTS_EXTENSION_REGISTER(bottom_pool_backward)\n .input(2)\n .output(1)\n#ifdef MMCV_WITH_CUDA\n .apply(bottom_pool_backward_parrots)\n#endif\n .apply(bottom_pool_backward_parrots_cpu)\n .done();\n\nPARROTS_EXTENSION_REGISTER(top_pool_forward)\n .input(1)\n .output(1)\n#ifdef MMCV_WITH_CUDA\n .apply(top_pool_forward_parrots)\n#endif\n .apply(top_pool_forward_parrots_cpu)\n .done();\n\nPARROTS_EXTENSION_REGISTER(top_pool_backward)\n .input(2)\n .output(1)\n#ifdef MMCV_WITH_CUDA\n .apply(top_pool_backward_parrots)\n#endif\n .apply(top_pool_backward_parrots_cpu)\n .done();\n\nPARROTS_EXTENSION_REGISTER(left_pool_forward)\n .input(1)\n .output(1)\n#ifdef MMCV_WITH_CUDA\n .apply(left_pool_forward_parrots)\n#endif\n .apply(left_pool_forward_parrots_cpu)\n .done();\n\nPARROTS_EXTENSION_REGISTER(left_pool_backward)\n .input(2)\n .output(1)\n#ifdef MMCV_WITH_CUDA\n .apply(left_pool_backward_parrots)\n#endif\n .apply(left_pool_backward_parrots_cpu)\n .done();\n\nPARROTS_EXTENSION_REGISTER(right_pool_forward)\n .input(1)\n .output(1)\n#ifdef MMCV_WITH_CUDA\n .apply(right_pool_forward_parrots)\n#endif\n .apply(right_pool_forward_parrots_cpu)\n .done();\n\nPARROTS_EXTENSION_REGISTER(right_pool_backward)\n .input(2)\n .output(1)\n#ifdef MMCV_WITH_CUDA\n .apply(right_pool_backward_parrots)\n#endif\n .apply(right_pool_backward_parrots_cpu)\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/corner_pool_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CORNER_POOL_PYTORCH_H\n#define CORNER_POOL_PYTORCH_H\n#include \n\nat::Tensor bottom_pool_forward(at::Tensor input);\nat::Tensor bottom_pool_backward(at::Tensor input, at::Tensor grad_output);\nat::Tensor left_pool_forward(at::Tensor input);\nat::Tensor left_pool_backward(at::Tensor input, at::Tensor grad_output);\nat::Tensor right_pool_forward(at::Tensor input);\nat::Tensor right_pool_backward(at::Tensor input, at::Tensor grad_output);\nat::Tensor top_pool_forward(at::Tensor input);\nat::Tensor top_pool_backward(at::Tensor input, at::Tensor grad_output);\n\n#endif // CORNER_POOL_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/correlation.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#include \n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid correlation_forward_impl(Tensor input1, Tensor input2, Tensor output,\n int kH, int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW) {\n DISPATCH_DEVICE_IMPL(correlation_forward_impl, input1, input2, output, kH, kW,\n patchH, patchW, padH, padW, dilationH, dilationW,\n dilation_patchH, dilation_patchW, dH, dW);\n}\n\nvoid correlation_backward_impl(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW) {\n DISPATCH_DEVICE_IMPL(correlation_backward_impl, grad_output, input1, input2,\n grad_input1, grad_input2, kH, kW, patchH, patchW, padH,\n padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n\nvoid correlation_forward(Tensor input1, Tensor input2, Tensor output, int kH,\n int kW, int patchH, int patchW, int padH, int padW,\n int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW) {\n correlation_forward_impl(input1, input2, output, kH, kW, patchH, patchW, padH,\n padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n\nvoid correlation_backward(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH, int padW,\n int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW) {\n correlation_backward_impl(grad_output, input1, input2, grad_input1,\n grad_input2, kH, kW, patchH, patchW, padH, padW,\n dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/correlation_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"correlation_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid correlation_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kH, kW, patchH, patchW, padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW;\n SSAttrs(attr)\n .get(\"kH\", kH)\n .get(\"kW\", kW)\n .get(\"patchH\", patchH)\n .get(\"patchW\", patchW)\n .get(\"padH\", padH)\n .get(\"padW\", padW)\n .get(\"dilationH\", dilationH)\n .get(\"dilationW\", dilationW)\n .get(\"dilation_patchH\", dilation_patchH)\n .get(\"dilation_patchW\", dilation_patchW)\n .get(\"dH\", dH)\n .get(\"dW\", dW)\n .done();\n\n auto input1 = buildATensor(ctx, ins[0]);\n auto input2 = buildATensor(ctx, ins[1]);\n\n auto output = buildATensor(ctx, outs[0]);\n\n correlation_forward(input1, input2, output, kH, kW, patchH, patchW, padH,\n padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n\nvoid correlation_backward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kH, kW, patchH, patchW, padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW;\n SSAttrs(attr)\n .get(\"kH\", kH)\n .get(\"kW\", kW)\n .get(\"patchH\", patchH)\n .get(\"patchW\", patchW)\n .get(\"padH\", padH)\n .get(\"padW\", padW)\n .get(\"dilationH\", dilationH)\n .get(\"dilationW\", dilationW)\n .get(\"dilation_patchH\", dilation_patchH)\n .get(\"dilation_patchW\", dilation_patchW)\n .get(\"dH\", dH)\n .get(\"dW\", dW)\n .done();\n\n auto grad_output = buildATensor(ctx, ins[0]);\n auto input1 = buildATensor(ctx, ins[1]);\n auto input2 = buildATensor(ctx, ins[2]);\n\n auto grad_input1 = buildATensor(ctx, outs[0]);\n auto grad_input2 = buildATensor(ctx, outs[1]);\n\n correlation_backward(grad_output, input1, input2, grad_input1, grad_input2,\n kH, kW, patchH, patchW, padH, padW, dilationH, dilationW,\n dilation_patchH, dilation_patchW, dH, dW);\n}\n#endif\n\nvoid correlation_forward_cpu_parrots(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kH, kW, patchH, patchW, padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW;\n SSAttrs(attr)\n .get(\"kH\", kH)\n .get(\"kW\", kW)\n .get(\"patchH\", patchH)\n .get(\"patchW\", patchW)\n .get(\"padH\", padH)\n .get(\"padW\", padW)\n .get(\"dilationH\", dilationH)\n .get(\"dilationW\", dilationW)\n .get(\"dilation_patchH\", dilation_patchH)\n .get(\"dilation_patchW\", dilation_patchW)\n .get(\"dH\", dH)\n .get(\"dW\", dW)\n .done();\n\n auto input1 = buildATensor(ctx, ins[0]);\n auto input2 = buildATensor(ctx, ins[1]);\n\n auto output = buildATensor(ctx, outs[0]);\n\n correlation_forward(input1, input2, output, kH, kW, patchH, patchW, padH,\n padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n\nvoid correlation_backward_cpu_parrots(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kH, kW, patchH, patchW, padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW;\n SSAttrs(attr)\n .get(\"kH\", kH)\n .get(\"kW\", kW)\n .get(\"patchH\", patchH)\n .get(\"patchW\", patchW)\n .get(\"padH\", padH)\n .get(\"padW\", padW)\n .get(\"dilationH\", dilationH)\n .get(\"dilationW\", dilationW)\n .get(\"dilation_patchH\", dilation_patchH)\n .get(\"dilation_patchW\", dilation_patchW)\n .get(\"dH\", dH)\n .get(\"dW\", dW)\n .done();\n\n auto grad_output = buildATensor(ctx, ins[0]);\n auto input1 = buildATensor(ctx, ins[1]);\n auto input2 = buildATensor(ctx, ins[2]);\n\n auto grad_input1 = buildATensor(ctx, outs[0]);\n auto grad_input2 = buildATensor(ctx, outs[1]);\n\n correlation_backward(grad_output, input1, input2, grad_input1, grad_input2,\n kH, kW, patchH, patchW, padH, padW, dilationH, dilationW,\n dilation_patchH, dilation_patchW, dH, dW);\n}\n\nPARROTS_EXTENSION_REGISTER(correlation_forward)\n .attr(\"kH\")\n .attr(\"kW\")\n .attr(\"patchH\")\n .attr(\"patchW\")\n .attr(\"padH\")\n .attr(\"padW\")\n .attr(\"dilationH\")\n .attr(\"dilationW\")\n .attr(\"dilation_patchH\")\n .attr(\"dilation_patchW\")\n .attr(\"dH\")\n .attr(\"dW\")\n .input(2)\n .output(1)\n .apply(correlation_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(correlation_forward_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(correlation_backward)\n .attr(\"kH\")\n .attr(\"kW\")\n .attr(\"patchH\")\n .attr(\"patchW\")\n .attr(\"padH\")\n .attr(\"padW\")\n .attr(\"dilationH\")\n .attr(\"dilationW\")\n .attr(\"dilation_patchH\")\n .attr(\"dilation_patchW\")\n .attr(\"dH\")\n .attr(\"dW\")\n .input(3)\n .output(2)\n .apply(correlation_backward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(correlation_backward_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/correlation_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef CORRELATION_PYTORCH_H\n#define CORRELATION_PYTORCH_H\n#include \nusing namespace at;\n\nvoid correlation_forward(Tensor input1, Tensor input2, Tensor output, int kH,\n int kW, int patchH, int patchW, int padH, int padW,\n int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW);\n\nvoid correlation_backward(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH, int padW,\n int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW);\n\n#endif // CORRELATION_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/cudabind.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid AssignScoreWithKForwardCUDAKernelLauncher(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& points, const Tensor& centers, const Tensor& scores,\n const Tensor& knn_idx, Tensor& output);\n\nvoid AssignScoreWithKBackwardCUDAKernelLauncher(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores);\n\nvoid assign_score_withk_forward_cuda(int B, int N0, int N1, int M, int K, int O,\n int aggregate, const Tensor& points,\n const Tensor& centers,\n const Tensor& scores,\n const Tensor& knn_idx, Tensor& output) {\n AssignScoreWithKForwardCUDAKernelLauncher(\n B, N0, N1, M, K, O, aggregate, points, centers, scores, knn_idx, output);\n};\n\nvoid assign_score_withk_backward_cuda(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores) {\n AssignScoreWithKBackwardCUDAKernelLauncher(\n B, N0, N1, M, K, O, aggregate, grad_out, points, centers, scores, knn_idx,\n grad_points, grad_centers, grad_scores);\n};\n\nvoid assign_score_withk_forward_impl(int B, int N0, int N1, int M, int K, int O,\n int aggregate, const Tensor& points,\n const Tensor& centers,\n const Tensor& scores,\n const Tensor& knn_idx, Tensor& output);\n\nvoid assign_score_withk_backward_impl(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores);\n\nREGISTER_DEVICE_IMPL(assign_score_withk_forward_impl, CUDA,\n assign_score_withk_forward_cuda);\nREGISTER_DEVICE_IMPL(assign_score_withk_backward_impl, CUDA,\n assign_score_withk_backward_cuda);\n\nvoid BallQueryForwardCUDAKernelLauncher(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx);\n\nvoid ball_query_forward_cuda(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx) {\n BallQueryForwardCUDAKernelLauncher(b, n, m, min_radius, max_radius, nsample,\n new_xyz, xyz, idx);\n};\n\nvoid ball_query_forward_impl(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx);\nREGISTER_DEVICE_IMPL(ball_query_forward_impl, CUDA, ball_query_forward_cuda);\n\nvoid BBoxOverlapsCUDAKernelLauncher(const Tensor bboxes1, const Tensor bboxes2,\n Tensor ious, const int mode,\n const bool aligned, const int offset);\n\nvoid bbox_overlaps_cuda(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset) {\n BBoxOverlapsCUDAKernelLauncher(bboxes1, bboxes2, ious, mode, aligned, offset);\n}\n\nvoid bbox_overlaps_impl(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset);\nREGISTER_DEVICE_IMPL(bbox_overlaps_impl, CUDA, bbox_overlaps_cuda);\n\nvoid BorderAlignForwardCUDAKernelLauncher(const Tensor& input,\n const Tensor& boxes, Tensor output,\n Tensor argmax_idx,\n const int pool_size);\n\nvoid BorderAlignBackwardCUDAKernelLauncher(const Tensor& grad_output,\n const Tensor& boxes,\n const Tensor& argmax_idx,\n Tensor grad_input,\n const int pool_size);\n\nvoid border_align_forward_cuda(const Tensor& input, const Tensor& boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size) {\n BorderAlignForwardCUDAKernelLauncher(input, boxes, output, argmax_idx,\n pool_size);\n}\n\nvoid border_align_backward_cuda(const Tensor& grad_output, const Tensor& boxes,\n const Tensor& argmax_idx, Tensor grad_input,\n const int pool_size) {\n BorderAlignBackwardCUDAKernelLauncher(grad_output, boxes, argmax_idx,\n grad_input, pool_size);\n}\n\nvoid border_align_forward_impl(const Tensor& input, const Tensor& boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size);\n\nvoid border_align_backward_impl(const Tensor& grad_output, const Tensor& boxes,\n const Tensor& argmax_idx, Tensor grad_input,\n const int pool_size);\n\nREGISTER_DEVICE_IMPL(border_align_forward_impl, CUDA,\n border_align_forward_cuda);\nREGISTER_DEVICE_IMPL(border_align_backward_impl, CUDA,\n border_align_backward_cuda);\n\nvoid box_iou_rotated_cuda(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned);\n\nvoid box_iou_rotated_impl(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned);\nREGISTER_DEVICE_IMPL(box_iou_rotated_impl, CUDA, box_iou_rotated_cuda);\n\nvoid CARAFEForwardCUDAKernelLauncher(const Tensor features, const Tensor masks,\n Tensor rfeatures, Tensor routput,\n Tensor rmasks, Tensor output,\n const int kernel_size,\n const int group_size,\n const int scale_factor);\n\nvoid CARAFEBackwardCUDAKernelLauncher(\n const Tensor top_grad, const Tensor rfeatures, const Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs, Tensor rbottom_grad,\n Tensor rmask_grad, Tensor bottom_grad, Tensor mask_grad,\n const int kernel_size, const int group_size, const int scale_factor);\n\nvoid carafe_forward_cuda(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor) {\n CARAFEForwardCUDAKernelLauncher(features, masks, rfeatures, routput, rmasks,\n output, kernel_size, group_size,\n scale_factor);\n}\n\nvoid carafe_backward_cuda(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad,\n Tensor bottom_grad, Tensor mask_grad, int kernel_size,\n int group_size, int scale_factor) {\n CARAFEBackwardCUDAKernelLauncher(top_grad, rfeatures, masks, rtop_grad,\n rbottom_grad_hs, rbottom_grad, rmask_grad,\n bottom_grad, mask_grad, kernel_size,\n group_size, scale_factor);\n}\n\nvoid carafe_forward_impl(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor);\n\nvoid carafe_backward_impl(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad,\n Tensor bottom_grad, Tensor mask_grad, int kernel_size,\n int group_size, int scale_factor);\n\nREGISTER_DEVICE_IMPL(carafe_forward_impl, CUDA, carafe_forward_cuda);\nREGISTER_DEVICE_IMPL(carafe_backward_impl, CUDA, carafe_backward_cuda);\n\nvoid CARAFENAIVEForwardCUDAKernelLauncher(const Tensor features,\n const Tensor masks, Tensor output,\n const int kernel_size,\n const int group_size,\n const int scale_factor);\n\nvoid CARAFENAIVEBackwardCUDAKernelLauncher(\n const Tensor top_grad, const Tensor features, const Tensor masks,\n Tensor bottom_grad, Tensor mask_grad, const int kernel_size,\n const int group_size, const int scale_factor);\n\nvoid carafe_naive_forward_cuda(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size,\n int scale_factor) {\n CARAFENAIVEForwardCUDAKernelLauncher(features, masks, output, kernel_size,\n group_size, scale_factor);\n}\n\nvoid carafe_naive_backward_cuda(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size,\n int scale_factor) {\n CARAFENAIVEBackwardCUDAKernelLauncher(top_grad, features, masks, bottom_grad,\n mask_grad, kernel_size, group_size,\n scale_factor);\n}\nvoid carafe_naive_forward_impl(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size,\n int scale_factor);\n\nvoid carafe_naive_backward_impl(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size,\n int scale_factor);\n\nREGISTER_DEVICE_IMPL(carafe_naive_forward_impl, CUDA,\n carafe_naive_forward_cuda);\nREGISTER_DEVICE_IMPL(carafe_naive_backward_impl, CUDA,\n carafe_naive_backward_cuda);\n\nvoid CorrelationForwardCUDAKernelLauncher(Tensor input1, Tensor input2,\n Tensor output, int kH, int kW,\n int patchH, int patchW, int padH,\n int padW, int dilationH,\n int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW);\n\nvoid CorrelationBackwardCUDAKernelLauncher(Tensor grad_output, Tensor input1,\n Tensor input2, Tensor grad_input1,\n Tensor grad_input2, int kH, int kW,\n int patchH, int patchW, int padH,\n int padW, int dilationH,\n int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW);\n\nvoid correlation_forward_cuda(Tensor input1, Tensor input2, Tensor output,\n int kH, int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW) {\n CorrelationForwardCUDAKernelLauncher(\n input1, input2, output, kH, kW, patchH, patchW, padH, padW, dilationH,\n dilationW, dilation_patchH, dilation_patchW, dH, dW);\n}\n\nvoid correlation_backward_cuda(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW) {\n CorrelationBackwardCUDAKernelLauncher(\n grad_output, input1, input2, grad_input1, grad_input2, kH, kW, patchH,\n patchW, padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n\nvoid correlation_forward_impl(Tensor input1, Tensor input2, Tensor output,\n int kH, int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW);\n\nvoid correlation_backward_impl(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW);\n\nREGISTER_DEVICE_IMPL(correlation_forward_impl, CUDA, correlation_forward_cuda);\nREGISTER_DEVICE_IMPL(correlation_backward_impl, CUDA,\n correlation_backward_cuda);\n\nvoid deformable_im2col_cuda(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col);\n\nvoid deformable_col2im_cuda(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im);\n\nvoid deformable_col2im_coord_cuda(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset);\n\nvoid deformable_im2col_impl(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col);\n\nvoid deformable_col2im_impl(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im);\n\nvoid deformable_col2im_coord_impl(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset);\n\nREGISTER_DEVICE_IMPL(deformable_im2col_impl, CUDA, deformable_im2col_cuda);\nREGISTER_DEVICE_IMPL(deformable_col2im_impl, CUDA, deformable_col2im_cuda);\nREGISTER_DEVICE_IMPL(deformable_col2im_coord_impl, CUDA,\n deformable_col2im_coord_cuda);\n\nvoid DeformRoIPoolForwardCUDAKernelLauncher(Tensor input, Tensor rois,\n Tensor offset, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale,\n int sampling_ratio, float gamma);\n\nvoid DeformRoIPoolBackwardCUDAKernelLauncher(\n Tensor grad_output, Tensor input, Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset, int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio, float gamma);\n\nvoid deform_roi_pool_forward_cuda(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma) {\n DeformRoIPoolForwardCUDAKernelLauncher(input, rois, offset, output,\n pooled_height, pooled_width,\n spatial_scale, sampling_ratio, gamma);\n}\n\nvoid deform_roi_pool_backward_cuda(Tensor grad_output, Tensor input,\n Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset,\n int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma) {\n DeformRoIPoolBackwardCUDAKernelLauncher(\n grad_output, input, rois, offset, grad_input, grad_offset, pooled_height,\n pooled_width, spatial_scale, sampling_ratio, gamma);\n}\n\nvoid deform_roi_pool_forward_impl(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma);\n\nvoid deform_roi_pool_backward_impl(Tensor grad_output, Tensor input,\n Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset,\n int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma);\n\nREGISTER_DEVICE_IMPL(deform_roi_pool_forward_impl, CUDA,\n deform_roi_pool_forward_cuda);\nREGISTER_DEVICE_IMPL(deform_roi_pool_backward_impl, CUDA,\n deform_roi_pool_backward_cuda);\n\nvoid SigmoidFocalLossForwardCUDAKernelLauncher(Tensor input, Tensor target,\n Tensor weight, Tensor output,\n const float gamma,\n const float alpha);\n\nvoid SigmoidFocalLossBackwardCUDAKernelLauncher(Tensor input, Tensor target,\n Tensor weight,\n Tensor grad_input,\n const float gamma,\n const float alpha);\n\nvoid SoftmaxFocalLossForwardCUDAKernelLauncher(Tensor softmax, Tensor target,\n Tensor weight, Tensor output,\n const float gamma,\n const float alpha);\n\nvoid SoftmaxFocalLossBackwardCUDAKernelLauncher(Tensor softmax, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input,\n const float gamma,\n const float alpha);\n\nvoid sigmoid_focal_loss_forward_cuda(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n SigmoidFocalLossForwardCUDAKernelLauncher(input, target, weight, output,\n gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_backward_cuda(Tensor input, Tensor target,\n Tensor weight, Tensor grad_input,\n float gamma, float alpha) {\n SigmoidFocalLossBackwardCUDAKernelLauncher(input, target, weight, grad_input,\n gamma, alpha);\n}\n\nvoid softmax_focal_loss_forward_cuda(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n SoftmaxFocalLossForwardCUDAKernelLauncher(input, target, weight, output,\n gamma, alpha);\n}\n\nvoid softmax_focal_loss_backward_cuda(Tensor input, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input, float gamma,\n float alpha) {\n SoftmaxFocalLossBackwardCUDAKernelLauncher(input, target, weight, buff,\n grad_input, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_forward_impl(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha);\n\nvoid sigmoid_focal_loss_backward_impl(Tensor input, Tensor target,\n Tensor weight, Tensor grad_input,\n float gamma, float alpha);\n\nvoid softmax_focal_loss_forward_impl(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha);\n\nvoid softmax_focal_loss_backward_impl(Tensor input, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input, float gamma,\n float alpha);\n\nREGISTER_DEVICE_IMPL(sigmoid_focal_loss_forward_impl, CUDA,\n sigmoid_focal_loss_forward_cuda);\nREGISTER_DEVICE_IMPL(sigmoid_focal_loss_backward_impl, CUDA,\n sigmoid_focal_loss_backward_cuda);\nREGISTER_DEVICE_IMPL(softmax_focal_loss_forward_impl, CUDA,\n softmax_focal_loss_forward_cuda);\nREGISTER_DEVICE_IMPL(softmax_focal_loss_backward_impl, CUDA,\n softmax_focal_loss_backward_cuda);\n\nvoid FurthestPointSamplingForwardCUDAKernelLauncher(int b, int n, int m,\n const float* dataset,\n float* temp, int* idxs);\n\nvoid FurthestPointSamplingWithDistForwardCUDAKernelLauncher(\n int b, int n, int m, const float* dataset, float* temp, int* idxs);\n\nvoid furthest_point_sampling_forward_cuda(Tensor points_tensor,\n Tensor temp_tensor, Tensor idx_tensor,\n int b, int n, int m) {\n const float* dataset = points_tensor.data_ptr();\n float* temp = temp_tensor.data_ptr();\n int* idxs = idx_tensor.data_ptr();\n FurthestPointSamplingForwardCUDAKernelLauncher(b, n, m, dataset, temp, idxs);\n}\n\nvoid furthest_point_sampling_with_dist_forward_cuda(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b,\n int n, int m) {\n const float* dataset = points_tensor.data_ptr();\n float* temp = temp_tensor.data_ptr();\n int* idxs = idx_tensor.data_ptr();\n FurthestPointSamplingWithDistForwardCUDAKernelLauncher(b, n, m, dataset, temp,\n idxs);\n}\n\nvoid furthest_point_sampling_forward_impl(Tensor points_tensor,\n Tensor temp_tensor, Tensor idx_tensor,\n int b, int n, int m);\n\nvoid furthest_point_sampling_with_dist_forward_impl(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b,\n int n, int m);\n\nREGISTER_DEVICE_IMPL(furthest_point_sampling_forward_impl, CUDA,\n furthest_point_sampling_forward_cuda);\nREGISTER_DEVICE_IMPL(furthest_point_sampling_with_dist_forward_impl, CUDA,\n furthest_point_sampling_with_dist_forward_cuda);\n\ntorch::Tensor fused_bias_leakyrelu_op(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale);\n\ntorch::Tensor fused_bias_leakyrelu_op_impl(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale);\nREGISTER_DEVICE_IMPL(fused_bias_leakyrelu_op_impl, CUDA,\n fused_bias_leakyrelu_op);\n\nvoid GatherPointsForwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n const Tensor points,\n const Tensor idx, Tensor out);\n\nvoid GatherPointsBackwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n const Tensor grad_out,\n const Tensor idx,\n Tensor grad_points);\n\nvoid gather_points_forward_cuda(int b, int c, int n, int npoints,\n const Tensor points, const Tensor idx,\n Tensor out) {\n GatherPointsForwardCUDAKernelLauncher(b, c, n, npoints, points, idx, out);\n};\n\nvoid gather_points_backward_cuda(int b, int c, int n, int npoints,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points) {\n GatherPointsBackwardCUDAKernelLauncher(b, c, n, npoints, grad_out, idx,\n grad_points);\n};\n\nvoid gather_points_forward_impl(int b, int c, int n, int npoints,\n const Tensor points, const Tensor idx,\n Tensor out);\n\nvoid gather_points_backward_impl(int b, int c, int n, int npoints,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points);\n\nREGISTER_DEVICE_IMPL(gather_points_forward_impl, CUDA,\n gather_points_forward_cuda);\nREGISTER_DEVICE_IMPL(gather_points_backward_impl, CUDA,\n gather_points_backward_cuda);\n\nvoid GroupPointsForwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n int nsample, const Tensor points,\n const Tensor idx, Tensor out);\n\nvoid GroupPointsBackwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n int nsample, const Tensor grad_out,\n const Tensor idx,\n Tensor grad_points);\n\nvoid group_points_forward_cuda(int b, int c, int n, int npoints, int nsample,\n const Tensor points, const Tensor idx,\n Tensor out) {\n GroupPointsForwardCUDAKernelLauncher(b, c, n, npoints, nsample, points, idx,\n out);\n};\n\nvoid group_points_backward_cuda(int b, int c, int n, int npoints, int nsample,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points) {\n GroupPointsBackwardCUDAKernelLauncher(b, c, n, npoints, nsample, grad_out,\n idx, grad_points);\n};\n\nvoid group_points_forward_impl(int b, int c, int n, int npoints, int nsample,\n const Tensor points, const Tensor idx,\n Tensor out);\n\nvoid group_points_backward_impl(int b, int c, int n, int npoints, int nsample,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points);\n\nREGISTER_DEVICE_IMPL(group_points_forward_impl, CUDA,\n group_points_forward_cuda);\nREGISTER_DEVICE_IMPL(group_points_backward_impl, CUDA,\n group_points_backward_cuda);\n\nvoid IoU3DBoxesOverlapBevForwardCUDAKernelLauncher(const int num_a,\n const Tensor boxes_a,\n const int num_b,\n const Tensor boxes_b,\n Tensor ans_overlap);\n\nvoid IoU3DBoxesIoUBevForwardCUDAKernelLauncher(const int num_a,\n const Tensor boxes_a,\n const int num_b,\n const Tensor boxes_b,\n Tensor ans_iou);\n\nvoid IoU3DNMSForwardCUDAKernelLauncher(const Tensor boxes,\n unsigned long long* mask, int boxes_num,\n float nms_overlap_thresh);\n\nvoid IoU3DNMSNormalForwardCUDAKernelLauncher(const Tensor boxes,\n unsigned long long* mask,\n int boxes_num,\n float nms_overlap_thresh);\n\nvoid iou3d_boxes_overlap_bev_forward_cuda(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_overlap) {\n IoU3DBoxesOverlapBevForwardCUDAKernelLauncher(num_a, boxes_a, num_b, boxes_b,\n ans_overlap);\n};\n\nvoid iou3d_boxes_iou_bev_forward_cuda(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_iou) {\n IoU3DBoxesIoUBevForwardCUDAKernelLauncher(num_a, boxes_a, num_b, boxes_b,\n ans_iou);\n};\n\nvoid iou3d_nms_forward_cuda(const Tensor boxes, unsigned long long* mask,\n int boxes_num, float nms_overlap_thresh) {\n IoU3DNMSForwardCUDAKernelLauncher(boxes, mask, boxes_num, nms_overlap_thresh);\n};\n\nvoid iou3d_nms_normal_forward_cuda(const Tensor boxes, unsigned long long* mask,\n int boxes_num, float nms_overlap_thresh) {\n IoU3DNMSNormalForwardCUDAKernelLauncher(boxes, mask, boxes_num,\n nms_overlap_thresh);\n};\n\nvoid iou3d_boxes_overlap_bev_forward_impl(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_overlap);\n\nvoid iou3d_boxes_iou_bev_forward_impl(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_iou);\n\nvoid iou3d_nms_forward_impl(const Tensor boxes, unsigned long long* mask,\n int boxes_num, float nms_overlap_thresh);\n\nvoid iou3d_nms_normal_forward_impl(const Tensor boxes, unsigned long long* mask,\n int boxes_num, float nms_overlap_thresh);\n\nREGISTER_DEVICE_IMPL(iou3d_boxes_overlap_bev_forward_impl, CUDA,\n iou3d_boxes_overlap_bev_forward_cuda);\nREGISTER_DEVICE_IMPL(iou3d_boxes_iou_bev_forward_impl, CUDA,\n iou3d_boxes_iou_bev_forward_cuda);\nREGISTER_DEVICE_IMPL(iou3d_nms_forward_impl, CUDA, iou3d_nms_forward_cuda);\nREGISTER_DEVICE_IMPL(iou3d_nms_normal_forward_impl, CUDA,\n iou3d_nms_normal_forward_cuda);\n\nvoid KNNForwardCUDAKernelLauncher(int b, int n, int m, int nsample,\n const Tensor xyz, const Tensor new_xyz,\n Tensor idx, Tensor dist2);\n\nvoid knn_forward_cuda(int b, int n, int m, int nsample, const Tensor xyz,\n const Tensor new_xyz, Tensor idx, Tensor dist2) {\n KNNForwardCUDAKernelLauncher(b, n, m, nsample, xyz, new_xyz, idx, dist2);\n}\n\nvoid knn_forward_impl(int b, int n, int m, int nsample, const Tensor xyz,\n const Tensor new_xyz, Tensor idx, Tensor dist2);\nREGISTER_DEVICE_IMPL(knn_forward_impl, CUDA, knn_forward_cuda);\n\nvoid MaskedIm2colForwardCUDAKernelLauncher(const Tensor bottom_data,\n const Tensor mask_h_idx,\n const Tensor mask_w_idx,\n Tensor top_data, const int kernel_h,\n const int kernel_w, const int pad_h,\n const int pad_w);\n\nvoid MaskedCol2imForwardCUDAKernelLauncher(const Tensor bottom_data,\n const Tensor mask_h_idx,\n const Tensor mask_w_idx,\n Tensor top_data, const int height,\n const int width, const int channels);\n\nvoid masked_im2col_forward_cuda(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w) {\n // im: (n, ic, h, w), kernel size (kh, kw)\n // kernel: (oc, ic * kh * kw), col: (kh * kw * ic, ow * oh)\n MaskedIm2colForwardCUDAKernelLauncher(im, mask_h_idx, mask_w_idx, col,\n kernel_h, kernel_w, pad_h, pad_w);\n}\n\nvoid masked_col2im_forward_cuda(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels) {\n // im: (n, ic, h, w), kernel size (kh, kw)\n // kernel: (oc, ic * kh * kh), col: (kh * kw * ic, ow * oh)\n MaskedCol2imForwardCUDAKernelLauncher(col, mask_h_idx, mask_w_idx, im, height,\n width, channels);\n}\n\nvoid masked_im2col_forward_impl(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w);\n\nvoid masked_col2im_forward_impl(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels);\n\nREGISTER_DEVICE_IMPL(masked_im2col_forward_impl, CUDA,\n masked_im2col_forward_cuda);\nREGISTER_DEVICE_IMPL(masked_col2im_forward_impl, CUDA,\n masked_col2im_forward_cuda);\n\nvoid modulated_deformable_im2col_cuda(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col);\n\nvoid modulated_deformable_col2im_cuda(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im);\n\nvoid modulated_deformable_col2im_coord_cuda(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask);\n\nvoid modulated_deformable_im2col_impl(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col);\n\nvoid modulated_deformable_col2im_impl(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im);\n\nvoid modulated_deformable_col2im_coord_impl(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask);\n\nREGISTER_DEVICE_IMPL(modulated_deformable_im2col_impl, CUDA,\n modulated_deformable_im2col_cuda);\nREGISTER_DEVICE_IMPL(modulated_deformable_col2im_impl, CUDA,\n modulated_deformable_col2im_cuda);\nREGISTER_DEVICE_IMPL(modulated_deformable_col2im_coord_impl, CUDA,\n modulated_deformable_col2im_coord_cuda);\n\nTensor ms_deform_attn_cuda_forward(const Tensor& value,\n const Tensor& spatial_shapes,\n const Tensor& level_start_index,\n const Tensor& sampling_loc,\n const Tensor& attn_weight,\n const int im2col_step);\n\nvoid ms_deform_attn_cuda_backward(\n const Tensor& value, const Tensor& spatial_shapes,\n const Tensor& level_start_index, const Tensor& sampling_loc,\n const Tensor& attn_weight, const Tensor& grad_output, Tensor& grad_value,\n Tensor& grad_sampling_loc, Tensor& grad_attn_weight, const int im2col_step);\n\nTensor ms_deform_attn_impl_forward(const Tensor& value,\n const Tensor& spatial_shapes,\n const Tensor& level_start_index,\n const Tensor& sampling_loc,\n const Tensor& attn_weight,\n const int im2col_step);\n\nvoid ms_deform_attn_impl_backward(\n const Tensor& value, const Tensor& spatial_shapes,\n const Tensor& level_start_index, const Tensor& sampling_loc,\n const Tensor& attn_weight, const Tensor& grad_output, Tensor& grad_value,\n Tensor& grad_sampling_loc, Tensor& grad_attn_weight, const int im2col_step);\n\nREGISTER_DEVICE_IMPL(ms_deform_attn_impl_forward, CUDA,\n ms_deform_attn_cuda_forward);\nREGISTER_DEVICE_IMPL(ms_deform_attn_impl_backward, CUDA,\n ms_deform_attn_cuda_backward);\n\nTensor NMSCUDAKernelLauncher(Tensor boxes, Tensor scores, float iou_threshold,\n int offset);\n\nTensor nms_cuda(Tensor boxes, Tensor scores, float iou_threshold, int offset) {\n return NMSCUDAKernelLauncher(boxes, scores, iou_threshold, offset);\n}\n\nTensor nms_impl(Tensor boxes, Tensor scores, float iou_threshold, int offset);\nREGISTER_DEVICE_IMPL(nms_impl, CUDA, nms_cuda);\n\nvoid PointsInBoxesPartForwardCUDAKernelLauncher(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points);\n\nvoid PointsInBoxesAllForwardCUDAKernelLauncher(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points);\n\nvoid points_in_boxes_part_forward_cuda(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n PointsInBoxesPartForwardCUDAKernelLauncher(batch_size, boxes_num, pts_num,\n boxes, pts, box_idx_of_points);\n};\n\nvoid points_in_boxes_all_forward_cuda(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n PointsInBoxesAllForwardCUDAKernelLauncher(batch_size, boxes_num, pts_num,\n boxes, pts, box_idx_of_points);\n};\n\nvoid points_in_boxes_part_forward_impl(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points);\n\nvoid points_in_boxes_all_forward_impl(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points);\nREGISTER_DEVICE_IMPL(points_in_boxes_part_forward_impl, CUDA,\n points_in_boxes_part_forward_cuda);\nREGISTER_DEVICE_IMPL(points_in_boxes_all_forward_impl, CUDA,\n points_in_boxes_all_forward_cuda);\n\nvoid PSAMaskForwardCUDAKernelLauncher(const int psa_type, const Tensor input,\n Tensor output, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask,\n const int half_w_mask);\n\nvoid PSAMaskBackwardCUDAKernelLauncher(\n const int psa_type, const Tensor grad_output, Tensor grad_input,\n const int num_, const int h_feature, const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask, const int half_w_mask);\n\nvoid psamask_forward_cuda(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask) {\n PSAMaskForwardCUDAKernelLauncher(psa_type, input, output, num_, h_feature,\n w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask);\n}\n\nvoid psamask_backward_cuda(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask) {\n PSAMaskBackwardCUDAKernelLauncher(psa_type, grad_output, grad_input, num_,\n h_feature, w_feature, h_mask, w_mask,\n half_h_mask, half_w_mask);\n}\n\nvoid psamask_forward_impl(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask);\n\nvoid psamask_backward_impl(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask);\nREGISTER_DEVICE_IMPL(psamask_forward_impl, CUDA, psamask_forward_cuda);\nREGISTER_DEVICE_IMPL(psamask_backward_impl, CUDA, psamask_backward_cuda);\n\nvoid ROIAlignForwardCUDAKernelLauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nvoid ROIAlignBackwardCUDAKernelLauncher(Tensor grad_output, Tensor rois,\n Tensor argmax_y, Tensor argmax_x,\n Tensor grad_input, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode,\n bool aligned);\n\nvoid roi_align_forward_cuda(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n ROIAlignForwardCUDAKernelLauncher(\n input, rois, output, argmax_y, argmax_x, aligned_height, aligned_width,\n spatial_scale, sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_backward_cuda(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n ROIAlignBackwardCUDAKernelLauncher(\n grad_output, rois, argmax_y, argmax_x, grad_input, aligned_height,\n aligned_width, spatial_scale, sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nvoid roi_align_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nREGISTER_DEVICE_IMPL(roi_align_forward_impl, CUDA, roi_align_forward_cuda);\nREGISTER_DEVICE_IMPL(roi_align_backward_impl, CUDA, roi_align_backward_cuda);\n\nvoid ROIAlignRotatedForwardCUDAKernelLauncher(\n const at::Tensor features, const at::Tensor rois, const float spatial_scale,\n const int sample_num, const bool aligned, const bool clockwise,\n const int channels, const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, at::Tensor output);\n\nvoid ROIAlignRotatedBackwardCUDAKernelLauncher(\n const at::Tensor top_grad, const at::Tensor rois, const float spatial_scale,\n const int sample_num, const bool aligned, const bool clockwise,\n const int channels, const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, at::Tensor bottom_grad);\n\nvoid roi_align_rotated_forward_cuda(Tensor features, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sample_ratio,\n bool aligned, bool clockwise) {\n // Number of ROIs\n int num_rois = rois.size(0);\n int size_rois = rois.size(1);\n\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n\n int num_channels = features.size(1);\n int data_height = features.size(2);\n int data_width = features.size(3);\n ROIAlignRotatedForwardCUDAKernelLauncher(\n features, rois, spatial_scale, sample_ratio, aligned, clockwise,\n num_channels, data_height, data_width, num_rois, aligned_height,\n aligned_width, output);\n}\n\nvoid roi_align_rotated_backward_cuda(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sample_ratio, bool aligned,\n bool clockwise) {\n // Number of ROIs\n int num_rois = rois.size(0);\n int size_rois = rois.size(1);\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n\n int num_channels = bottom_grad.size(1);\n int data_height = bottom_grad.size(2);\n int data_width = bottom_grad.size(3);\n ROIAlignRotatedBackwardCUDAKernelLauncher(\n top_grad, rois, spatial_scale, sample_ratio, aligned, clockwise,\n num_channels, data_height, data_width, num_rois, aligned_height,\n aligned_width, bottom_grad);\n}\n\nvoid roi_align_rotated_forward_impl(Tensor features, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sample_ratio,\n bool aligned, bool clockwise);\n\nvoid roi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sample_ratio, bool aligned,\n bool clockwise);\nREGISTER_DEVICE_IMPL(roi_align_rotated_forward_impl, CUDA,\n roi_align_rotated_forward_cuda);\nREGISTER_DEVICE_IMPL(roi_align_rotated_backward_impl, CUDA,\n roi_align_rotated_backward_cuda);\n\nvoid RiROIAlignRotatedForwardCUDAKernelLauncher(\n const at::Tensor features, const at::Tensor rois, const float spatial_scale,\n const int num_samples, const bool clockwise, const int channels,\n const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, const int num_orientations,\n at::Tensor output);\n\nvoid RiROIAlignRotatedBackwardCUDAKernelLauncher(\n const at::Tensor top_grad, const at::Tensor rois, const float spatial_scale,\n const int num_samples, const bool clockwise, const int channels,\n const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, const int num_orientations,\n at::Tensor bottom_grad);\n\nvoid riroi_align_rotated_forward_cuda(Tensor features, Tensor rois,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n // Number of ROIs\n int num_rois = rois.size(0);\n int size_rois = rois.size(1);\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n CHECK_CONTIGUOUS(features);\n CHECK_CONTIGUOUS(rois);\n int num_channels = features.size(1) / num_orientations;\n int data_height = features.size(2);\n int data_width = features.size(3);\n RiROIAlignRotatedForwardCUDAKernelLauncher(\n features, rois, spatial_scale, num_samples, clockwise, num_channels,\n data_height, data_width, num_rois, pooled_height, pooled_width,\n num_orientations, output);\n}\n\nvoid riroi_align_rotated_backward_cuda(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n // Number of ROIs\n int num_rois = rois.size(0);\n int size_rois = rois.size(1);\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n CHECK_CONTIGUOUS(top_grad);\n CHECK_CONTIGUOUS(rois);\n int num_channels = bottom_grad.size(1) / num_orientations;\n int data_height = bottom_grad.size(2);\n int data_width = bottom_grad.size(3);\n RiROIAlignRotatedBackwardCUDAKernelLauncher(\n top_grad, rois, spatial_scale, num_samples, clockwise, num_channels,\n data_height, data_width, num_rois, pooled_height, pooled_width,\n num_orientations, bottom_grad);\n}\n\nvoid riroi_align_rotated_forward_impl(Tensor features, Tensor rois,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise);\n\nvoid riroi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise);\n\nREGISTER_DEVICE_IMPL(riroi_align_rotated_forward_impl, CUDA,\n riroi_align_rotated_forward_cuda);\nREGISTER_DEVICE_IMPL(riroi_align_rotated_backward_impl, CUDA,\n riroi_align_rotated_backward_cuda);\n\nvoid RoiawarePool3dForwardCUDAKernelLauncher(\n int boxes_num, int pts_num, int channels, int max_pts_each_voxel, int out_x,\n int out_y, int out_z, const Tensor rois, const Tensor pts,\n const Tensor pts_feature, Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method);\n\nvoid RoiawarePool3dBackwardCUDAKernelLauncher(\n int boxes_num, int out_x, int out_y, int out_z, int channels,\n int max_pts_each_voxel, const Tensor pts_idx_of_voxels, const Tensor argmax,\n const Tensor grad_out, Tensor grad_in, int pool_method);\n\nvoid roiaware_pool3d_forward_cuda(int boxes_num, int pts_num, int channels,\n int max_pts_each_voxel, int out_x, int out_y,\n int out_z, const Tensor rois,\n const Tensor pts, const Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method) {\n RoiawarePool3dForwardCUDAKernelLauncher(\n boxes_num, pts_num, channels, max_pts_each_voxel, out_x, out_y, out_z,\n rois, pts, pts_feature, argmax, pts_idx_of_voxels, pooled_features,\n pool_method);\n};\n\nvoid roiaware_pool3d_backward_cuda(int boxes_num, int out_x, int out_y,\n int out_z, int channels,\n int max_pts_each_voxel,\n const Tensor pts_idx_of_voxels,\n const Tensor argmax, const Tensor grad_out,\n Tensor grad_in, int pool_method) {\n RoiawarePool3dBackwardCUDAKernelLauncher(\n boxes_num, out_x, out_y, out_z, channels, max_pts_each_voxel,\n pts_idx_of_voxels, argmax, grad_out, grad_in, pool_method);\n};\n\nvoid roiaware_pool3d_forward_impl(int boxes_num, int pts_num, int channels,\n int max_pts_each_voxel, int out_x, int out_y,\n int out_z, const Tensor rois,\n const Tensor pts, const Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method);\n\nvoid roiaware_pool3d_backward_impl(int boxes_num, int out_x, int out_y,\n int out_z, int channels,\n int max_pts_each_voxel,\n const Tensor pts_idx_of_voxels,\n const Tensor argmax, const Tensor grad_out,\n Tensor grad_in, int pool_method);\n\nREGISTER_DEVICE_IMPL(roiaware_pool3d_forward_impl, CUDA,\n roiaware_pool3d_forward_cuda);\nREGISTER_DEVICE_IMPL(roiaware_pool3d_backward_impl, CUDA,\n roiaware_pool3d_backward_cuda);\n\nvoid RoIPointPool3dForwardCUDAKernelLauncher(\n int batch_size, int pts_num, int boxes_num, int feature_in_len,\n int sampled_pts_num, const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature, Tensor pooled_features, Tensor pooled_empty_flag);\n\nvoid roipoint_pool3d_forward_cuda(int batch_size, int pts_num, int boxes_num,\n int feature_in_len, int sampled_pts_num,\n const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature,\n Tensor pooled_features,\n Tensor pooled_empty_flag) {\n RoIPointPool3dForwardCUDAKernelLauncher(\n batch_size, pts_num, boxes_num, feature_in_len, sampled_pts_num, xyz,\n boxes3d, pts_feature, pooled_features, pooled_empty_flag);\n};\n\nvoid roipoint_pool3d_forward_impl(int batch_size, int pts_num, int boxes_num,\n int feature_in_len, int sampled_pts_num,\n const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature,\n Tensor pooled_features,\n Tensor pooled_empty_flag);\nREGISTER_DEVICE_IMPL(roipoint_pool3d_forward_impl, CUDA,\n roipoint_pool3d_forward_cuda);\n\nvoid ROIPoolForwardCUDAKernelLauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height,\n int pooled_width, float spatial_scale);\n\nvoid ROIPoolBackwardCUDAKernelLauncher(Tensor grad_output, Tensor rois,\n Tensor argmax, Tensor grad_input,\n int pooled_height, int pooled_width,\n float spatial_scale);\n\nvoid roi_pool_forward_cuda(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height, int pooled_width,\n float spatial_scale) {\n ROIPoolForwardCUDAKernelLauncher(input, rois, output, argmax, pooled_height,\n pooled_width, spatial_scale);\n}\n\nvoid roi_pool_backward_cuda(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height,\n int pooled_width, float spatial_scale) {\n ROIPoolBackwardCUDAKernelLauncher(grad_output, rois, argmax, grad_input,\n pooled_height, pooled_width, spatial_scale);\n}\n\nvoid roi_pool_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height, int pooled_width,\n float spatial_scale);\nvoid roi_pool_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height,\n int pooled_width, float spatial_scale);\nREGISTER_DEVICE_IMPL(roi_pool_forward_impl, CUDA, roi_pool_forward_cuda);\nREGISTER_DEVICE_IMPL(roi_pool_backward_impl, CUDA, roi_pool_backward_cuda);\n\ntypedef enum { SUM = 0, MEAN = 1, MAX = 2 } reduce_t;\n\nstd::vector DynamicPointToVoxelForwardCUDAKernelLauncher(\n const at::Tensor& feats, const at::Tensor& coors,\n const reduce_t reduce_type);\n\nvoid DynamicPointToVoxelBackwardCUDAKernelLauncher(\n at::Tensor& grad_feats, const at::Tensor& grad_reduced_feats,\n const at::Tensor& feats, const at::Tensor& reduced_feats,\n const at::Tensor& coors_map, const at::Tensor& reduce_count,\n const reduce_t reduce_type);\n\nstd::vector dynamic_point_to_voxel_forward_cuda(\n const torch::Tensor& feats, const torch::Tensor& coors,\n const reduce_t reduce_type) {\n return DynamicPointToVoxelForwardCUDAKernelLauncher(feats, coors,\n reduce_type);\n};\n\nvoid dynamic_point_to_voxel_backward_cuda(\n torch::Tensor& grad_feats, const torch::Tensor& grad_reduced_feats,\n const torch::Tensor& feats, const torch::Tensor& reduced_feats,\n const torch::Tensor& coors_idx, const torch::Tensor& reduce_count,\n const reduce_t reduce_type) {\n DynamicPointToVoxelBackwardCUDAKernelLauncher(grad_feats, grad_reduced_feats,\n feats, reduced_feats, coors_idx,\n reduce_count, reduce_type);\n};\n\nstd::vector dynamic_point_to_voxel_forward_impl(\n const torch::Tensor& feats, const torch::Tensor& coors,\n const reduce_t reduce_type);\n\nvoid dynamic_point_to_voxel_backward_impl(\n torch::Tensor& grad_feats, const torch::Tensor& grad_reduced_feats,\n const torch::Tensor& feats, const torch::Tensor& reduced_feats,\n const torch::Tensor& coors_idx, const torch::Tensor& reduce_count,\n const reduce_t reduce_type);\n\nREGISTER_DEVICE_IMPL(dynamic_point_to_voxel_forward_impl, CUDA,\n dynamic_point_to_voxel_forward_cuda);\nREGISTER_DEVICE_IMPL(dynamic_point_to_voxel_backward_impl, CUDA,\n dynamic_point_to_voxel_backward_cuda);\n\nvoid SyncBNForwardMeanCUDAKernelLauncher(const Tensor input, Tensor mean);\n\nvoid SyncBNForwardVarCUDAKernelLauncher(const Tensor input, const Tensor mean,\n Tensor var);\n\nvoid SyncBNForwardOutputCUDAKernelLauncher(\n const Tensor input, const Tensor mean, const Tensor var,\n Tensor running_mean, Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std, Tensor output, float eps,\n float momentum, int group_size);\n\nvoid SyncBNBackwardParamCUDAKernelLauncher(const Tensor grad_output,\n const Tensor norm,\n Tensor grad_weight,\n Tensor grad_bias);\n\nvoid SyncBNBackwardDataCUDAKernelLauncher(const Tensor grad_output,\n const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias,\n const Tensor norm, const Tensor std,\n Tensor grad_input);\n\nvoid sync_bn_forward_mean_cuda(const Tensor input, Tensor mean) {\n SyncBNForwardMeanCUDAKernelLauncher(input, mean);\n}\n\nvoid sync_bn_forward_var_cuda(const Tensor input, const Tensor mean,\n Tensor var) {\n SyncBNForwardVarCUDAKernelLauncher(input, mean, var);\n}\n\nvoid sync_bn_forward_output_cuda(const Tensor input, const Tensor mean,\n const Tensor var, Tensor running_mean,\n Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size) {\n SyncBNForwardOutputCUDAKernelLauncher(input, mean, var, running_mean,\n running_var, weight, bias, norm, std,\n output, eps, momentum, group_size);\n}\n\nvoid sync_bn_backward_param_cuda(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias) {\n SyncBNBackwardParamCUDAKernelLauncher(grad_output, norm, grad_weight,\n grad_bias);\n}\n\nvoid sync_bn_backward_data_cuda(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias, const Tensor norm,\n const Tensor std, Tensor grad_input) {\n SyncBNBackwardDataCUDAKernelLauncher(grad_output, weight, grad_weight,\n grad_bias, norm, std, grad_input);\n}\n\nvoid sync_bn_forward_mean_impl(const Tensor input, Tensor mean);\n\nvoid sync_bn_forward_var_impl(const Tensor input, const Tensor mean,\n Tensor var);\n\nvoid sync_bn_forward_output_impl(const Tensor input, const Tensor mean,\n const Tensor var, Tensor running_mean,\n Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size);\n\nvoid sync_bn_backward_param_impl(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias);\n\nvoid sync_bn_backward_data_impl(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias, const Tensor norm,\n const Tensor std, Tensor grad_input);\n\nREGISTER_DEVICE_IMPL(sync_bn_forward_mean_impl, CUDA,\n sync_bn_forward_mean_cuda);\nREGISTER_DEVICE_IMPL(sync_bn_forward_var_impl, CUDA, sync_bn_forward_var_cuda);\nREGISTER_DEVICE_IMPL(sync_bn_forward_output_impl, CUDA,\n sync_bn_forward_output_cuda);\nREGISTER_DEVICE_IMPL(sync_bn_backward_param_impl, CUDA,\n sync_bn_backward_param_cuda);\nREGISTER_DEVICE_IMPL(sync_bn_backward_data_impl, CUDA,\n sync_bn_backward_data_cuda);\n\nvoid ThreeInterpolateForwardCUDAKernelLauncher(int b, int c, int m, int n,\n const Tensor points,\n const Tensor idx,\n const Tensor weight, Tensor out);\n\nvoid ThreeInterpolateBackwardCUDAKernelLauncher(int b, int c, int n, int m,\n const Tensor grad_out,\n const Tensor idx,\n const Tensor weight,\n Tensor grad_points);\n\nvoid three_interpolate_forward_cuda(int b, int c, int m, int n,\n const Tensor points, const Tensor idx,\n const Tensor weight, Tensor out) {\n ThreeInterpolateForwardCUDAKernelLauncher(b, c, m, n, points, idx, weight,\n out);\n};\n\nvoid three_interpolate_backward_cuda(int b, int c, int n, int m,\n const Tensor grad_out, const Tensor idx,\n const Tensor weight, Tensor grad_points) {\n ThreeInterpolateBackwardCUDAKernelLauncher(b, c, n, m, grad_out, idx, weight,\n grad_points);\n};\n\nvoid three_interpolate_forward_impl(int b, int c, int m, int n,\n const Tensor points, const Tensor idx,\n const Tensor weight, Tensor out);\n\nvoid three_interpolate_backward_impl(int b, int c, int n, int m,\n const Tensor grad_out, const Tensor idx,\n const Tensor weight, Tensor grad_points);\nREGISTER_DEVICE_IMPL(three_interpolate_forward_impl, CUDA,\n three_interpolate_forward_cuda);\nREGISTER_DEVICE_IMPL(three_interpolate_backward_impl, CUDA,\n three_interpolate_backward_cuda);\n\nvoid ThreeNNForwardCUDAKernelLauncher(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2,\n Tensor idx);\n\nvoid three_nn_forward_cuda(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2, Tensor idx) {\n ThreeNNForwardCUDAKernelLauncher(b, n, m, unknown, known, dist2, idx);\n};\n\nvoid three_nn_forward_impl(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2, Tensor idx);\nREGISTER_DEVICE_IMPL(three_nn_forward_impl, CUDA, three_nn_forward_cuda);\n\nvoid TINShiftForwardCUDAKernelLauncher(Tensor input, Tensor shift,\n Tensor output);\n\nvoid TINShiftBackwardCUDAKernelLauncher(Tensor grad_output, Tensor shift,\n Tensor grad_input);\n\nvoid tin_shift_forward_cuda(Tensor input, Tensor shift, Tensor output) {\n TINShiftForwardCUDAKernelLauncher(input, shift, output);\n}\n\nvoid tin_shift_backward_cuda(Tensor grad_output, Tensor shift,\n Tensor grad_input) {\n TINShiftBackwardCUDAKernelLauncher(grad_output, shift, grad_input);\n}\n\nvoid tin_shift_forward_impl(Tensor input, Tensor shift, Tensor output);\nvoid tin_shift_backward_impl(Tensor grad_output, Tensor shift,\n Tensor grad_input);\nREGISTER_DEVICE_IMPL(tin_shift_forward_impl, CUDA, tin_shift_forward_cuda);\nREGISTER_DEVICE_IMPL(tin_shift_backward_impl, CUDA, tin_shift_backward_cuda);\n\ntorch::Tensor upfirdn2d_op(const torch::Tensor& input,\n const torch::Tensor& kernel, int up_x, int up_y,\n int down_x, int down_y, int pad_x0, int pad_x1,\n int pad_y0, int pad_y1);\n\ntorch::Tensor upfirdn2d_op_impl(const torch::Tensor& input,\n const torch::Tensor& kernel, int up_x, int up_y,\n int down_x, int down_y, int pad_x0, int pad_x1,\n int pad_y0, int pad_y1);\nREGISTER_DEVICE_IMPL(upfirdn2d_op_impl, CUDA, upfirdn2d_op);\n\nint HardVoxelizeForwardCUDAKernelLauncher(\n const at::Tensor& points, at::Tensor& voxels, at::Tensor& coors,\n at::Tensor& num_points_per_voxel, const std::vector voxel_size,\n const std::vector coors_range, const int max_points,\n const int max_voxels, const int NDim = 3);\n\nvoid DynamicVoxelizeForwardCUDAKernelLauncher(\n const at::Tensor& points, at::Tensor& coors,\n const std::vector voxel_size, const std::vector coors_range,\n const int NDim = 3);\n\nint hard_voxelize_forward_cuda(const at::Tensor& points, at::Tensor& voxels,\n at::Tensor& coors,\n at::Tensor& num_points_per_voxel,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int max_points, const int max_voxels,\n const int NDim) {\n return HardVoxelizeForwardCUDAKernelLauncher(\n points, voxels, coors, num_points_per_voxel, voxel_size, coors_range,\n max_points, max_voxels, NDim);\n};\n\nvoid dynamic_voxelize_forward_cuda(const at::Tensor& points, at::Tensor& coors,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int NDim) {\n DynamicVoxelizeForwardCUDAKernelLauncher(points, coors, voxel_size,\n coors_range, NDim);\n};\n\nint hard_voxelize_forward_impl(const at::Tensor& points, at::Tensor& voxels,\n at::Tensor& coors,\n at::Tensor& num_points_per_voxel,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int max_points, const int max_voxels,\n const int NDim);\n\nvoid dynamic_voxelize_forward_impl(const at::Tensor& points, at::Tensor& coors,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int NDim);\n\nREGISTER_DEVICE_IMPL(hard_voxelize_forward_impl, CUDA,\n hard_voxelize_forward_cuda);\nREGISTER_DEVICE_IMPL(dynamic_voxelize_forward_impl, CUDA,\n dynamic_voxelize_forward_cuda);\n\nvoid RotatedFeatureAlignForwardCUDAKernelLauncher(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points,\n Tensor output);\n\nvoid RotatedFeatureAlignBackwardCUDAKernelLauncher(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points,\n Tensor bottom_grad);\n\nvoid rotated_feature_align_forward_cuda(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor output) {\n RotatedFeatureAlignForwardCUDAKernelLauncher(features, best_bboxes,\n spatial_scale, points, output);\n};\n\nvoid rotated_feature_align_backward_cuda(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor bottom_grad) {\n RotatedFeatureAlignBackwardCUDAKernelLauncher(\n top_grad, best_bboxes, spatial_scale, points, bottom_grad);\n};\n\nvoid rotated_feature_align_forward_impl(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor output);\n\nvoid rotated_feature_align_backward_impl(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor bottom_grad);\n\nREGISTER_DEVICE_IMPL(rotated_feature_align_forward_impl, CUDA,\n rotated_feature_align_forward_cuda);\nREGISTER_DEVICE_IMPL(rotated_feature_align_backward_impl, CUDA,\n rotated_feature_align_backward_cuda);\n\nvoid PointsInPolygonsForwardCUDAKernelLauncher(const at::Tensor points,\n const at::Tensor polygons,\n const int rows, const int cols,\n at::Tensor output);\n\nvoid points_in_polygons_forward_cuda(const Tensor points, const Tensor polygons,\n Tensor output, const int rows,\n const int cols) {\n PointsInPolygonsForwardCUDAKernelLauncher(points, polygons, rows, cols,\n output);\n};\n\nvoid points_in_polygons_forward_impl(const Tensor points, const Tensor polygons,\n Tensor output, const int rows,\n const int cols);\n\nREGISTER_DEVICE_IMPL(points_in_polygons_forward_impl, CUDA,\n points_in_polygons_forward_cuda);\n\nvoid MinAreaPolygonsCUDAKernelLauncher(const Tensor pointsets, Tensor polygons);\n\nvoid min_area_polygons_cuda(const Tensor pointsets, Tensor polygons) {\n MinAreaPolygonsCUDAKernelLauncher(pointsets, polygons);\n}\n\nvoid min_area_polygons_impl(const Tensor pointsets, Tensor polygons);\n\nREGISTER_DEVICE_IMPL(min_area_polygons_impl, CUDA, min_area_polygons_cuda);\n\nvoid ActiveRotatedFilterForwardCUDAKernelLauncher(const Tensor input,\n const Tensor indices,\n Tensor output);\n\nvoid ActiveRotatedFilterBackwardCUDAKernelLauncher(const Tensor grad_out,\n const Tensor indices,\n Tensor grad_in);\n\nvoid active_rotated_filter_forward_cuda(const Tensor input,\n const Tensor indices, Tensor output) {\n ActiveRotatedFilterForwardCUDAKernelLauncher(input, indices, output);\n};\n\nvoid active_rotated_filter_backward_cuda(const Tensor grad_out,\n const Tensor indices, Tensor grad_in) {\n ActiveRotatedFilterBackwardCUDAKernelLauncher(grad_out, indices, grad_in);\n};\n\nvoid active_rotated_filter_forward_impl(const Tensor input,\n const Tensor indices, Tensor output);\n\nvoid active_rotated_filter_backward_impl(const Tensor grad_out,\n const Tensor indices, Tensor grad_in);\n\nREGISTER_DEVICE_IMPL(active_rotated_filter_forward_impl, CUDA,\n active_rotated_filter_forward_cuda);\nREGISTER_DEVICE_IMPL(active_rotated_filter_backward_impl, CUDA,\n active_rotated_filter_backward_cuda);\n\nvoid ConvexIoUCUDAKernelLauncher(const Tensor pointsets, const Tensor polygons,\n Tensor ious);\n\nvoid ConvexGIoUCUDAKernelLauncher(const Tensor pointsets, const Tensor polygons,\n Tensor output);\n\nvoid convex_iou_cuda(const Tensor pointsets, const Tensor polygons,\n Tensor ious) {\n ConvexIoUCUDAKernelLauncher(pointsets, polygons, ious);\n}\n\nvoid convex_giou_cuda(const Tensor pointsets, const Tensor polygons,\n Tensor output) {\n ConvexGIoUCUDAKernelLauncher(pointsets, polygons, output);\n}\n\nvoid convex_iou_impl(const Tensor pointsets, const Tensor polygons,\n Tensor ious);\n\nvoid convex_giou_impl(const Tensor pointsets, const Tensor polygons,\n Tensor output);\n\nREGISTER_DEVICE_IMPL(convex_iou_impl, CUDA, convex_iou_cuda);\nREGISTER_DEVICE_IMPL(convex_giou_impl, CUDA, convex_giou_cuda);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid deformable_im2col_impl(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col) {\n DISPATCH_DEVICE_IMPL(deformable_im2col_impl, data_im, data_offset, channels,\n height, width, ksize_h, ksize_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, parallel_imgs,\n deformable_group, data_col);\n}\n\nvoid deformable_col2im_impl(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im) {\n DISPATCH_DEVICE_IMPL(deformable_col2im_impl, data_col, data_offset, channels,\n height, width, ksize_h, ksize_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, parallel_imgs,\n deformable_group, grad_im);\n}\n\nvoid deformable_col2im_coord_impl(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset) {\n DISPATCH_DEVICE_IMPL(deformable_col2im_coord_impl, data_col, data_im,\n data_offset, channels, height, width, ksize_h, ksize_w,\n pad_h, pad_w, stride_h, stride_w, dilation_h, dilation_w,\n parallel_imgs, deformable_group, grad_offset);\n}\n\nvoid deform_conv_shape_check(at::Tensor input, at::Tensor offset,\n at::Tensor *gradOutput, at::Tensor weight, int kH,\n int kW, int dH, int dW, int padH, int padW,\n int dilationH, int dilationW, int group,\n int deformable_group) {\n TORCH_CHECK(\n weight.ndimension() == 4,\n \"4D weight tensor (nOutputPlane,nInputPlane,kH,kW) expected, but got: %s\",\n weight.ndimension());\n\n TORCH_CHECK(weight.is_contiguous(), \"weight tensor has to be contiguous\");\n\n TORCH_CHECK(kW > 0 && kH > 0,\n \"kernel size should be greater than zero, but got kH: %d kW: %d\",\n kH, kW);\n\n TORCH_CHECK((weight.size(2) == kH && weight.size(3) == kW),\n \"kernel size should be consistent with weight, \",\n \"but got kH: %d kW: %d weight.size(2): %d, weight.size(3): %d\",\n kH, kW, weight.size(2), weight.size(3));\n\n TORCH_CHECK(dW > 0 && dH > 0,\n \"stride should be greater than zero, but got dH: %d dW: %d\", dH,\n dW);\n\n TORCH_CHECK(\n dilationW > 0 && dilationH > 0,\n \"dilation should be greater than 0, but got dilationH: %d dilationW: %d\",\n dilationH, dilationW);\n\n int ndim = input.ndimension();\n int dimf = 0;\n int dimh = 1;\n int dimw = 2;\n\n if (ndim == 4) {\n dimf++;\n dimh++;\n dimw++;\n }\n\n TORCH_CHECK(ndim == 3 || ndim == 4,\n \"3D or 4D input tensor expected but got: %s\", ndim);\n\n long nInputPlane = weight.size(1) * group;\n long inputHeight = input.size(dimh);\n long inputWidth = input.size(dimw);\n long nOutputPlane = weight.size(0);\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n\n TORCH_CHECK(nInputPlane % deformable_group == 0,\n \"input channels must divide deformable group size\");\n\n if (outputWidth < 1 || outputHeight < 1)\n AT_ERROR(\n \"Given input size: (%ld x %ld x %ld). \"\n \"Calculated output size: (%ld x %ld x %ld). Output size is too small\",\n nInputPlane, inputHeight, inputWidth, nOutputPlane, outputHeight,\n outputWidth);\n\n TORCH_CHECK(input.size(1) == nInputPlane,\n \"invalid number of input planes, expected: %d, but got: %d\",\n nInputPlane, input.size(1));\n\n TORCH_CHECK((inputHeight >= kH && inputWidth >= kW),\n \"input image is smaller than kernel\");\n\n TORCH_CHECK(\n (offset.size(2) == outputHeight && offset.size(3) == outputWidth),\n \"invalid spatial size of offset, expected height: %d width: %d, but \"\n \"got height: %d width: %d\",\n outputHeight, outputWidth, offset.size(2), offset.size(3));\n\n TORCH_CHECK((offset.size(1) == deformable_group * 2 * kH * kW),\n \"invalid number of channels of offset\");\n\n if (gradOutput != NULL) {\n TORCH_CHECK(\n gradOutput->size(dimf) == nOutputPlane,\n \"invalid number of gradOutput planes, expected: %d, but got: %d\",\n nOutputPlane, gradOutput->size(dimf));\n\n TORCH_CHECK(\n (gradOutput->size(dimh) == outputHeight &&\n gradOutput->size(dimw) == outputWidth),\n \"invalid size of gradOutput, expected height: %d width: %d , but \"\n \"got height: %d width: %d\",\n outputHeight, outputWidth, gradOutput->size(dimh),\n gradOutput->size(dimw));\n }\n}\n\nvoid deform_conv_forward(Tensor input, Tensor weight, Tensor offset,\n Tensor output, Tensor columns, Tensor ones, int kW,\n int kH, int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step) {\n if (input.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n CHECK_CUDA_INPUT(input);\n CHECK_CUDA_INPUT(offset);\n CHECK_CUDA_INPUT(weight);\n CHECK_CUDA_INPUT(output);\n CHECK_CUDA_INPUT(columns);\n CHECK_CUDA_INPUT(ones);\n#else\n AT_ERROR(\"DeformConv is not compiled with GPU support\");\n#endif\n } else {\n CHECK_CPU_INPUT(input);\n CHECK_CPU_INPUT(offset);\n CHECK_CPU_INPUT(weight);\n CHECK_CPU_INPUT(output);\n CHECK_CPU_INPUT(columns);\n CHECK_CPU_INPUT(ones);\n }\n\n deform_conv_shape_check(input, offset, NULL, weight, kH, kW, dH, dW, padH,\n padW, dilationH, dilationW, group, deformable_group);\n at::DeviceGuard guard(input.device());\n\n int batch = 1;\n if (input.ndimension() == 3) {\n // Force batch\n batch = 0;\n input.unsqueeze_(0);\n offset.unsqueeze_(0);\n }\n\n // todo: assert batchsize dividable by im2col_step\n\n long batchSize = input.size(0);\n long nInputPlane = input.size(1);\n long inputHeight = input.size(2);\n long inputWidth = input.size(3);\n\n long nOutputPlane = weight.size(0);\n\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n\n TORCH_CHECK((offset.size(0) == batchSize), \"invalid batch size of offset\");\n\n output = output.view({batchSize / im2col_step, im2col_step, nOutputPlane,\n outputHeight, outputWidth});\n columns = at::zeros(\n {nInputPlane * kW * kH, im2col_step * outputHeight * outputWidth},\n input.options());\n\n if (ones.ndimension() != 2 ||\n ones.size(0) * ones.size(1) < outputHeight * outputWidth) {\n ones = at::ones({outputHeight, outputWidth}, input.options());\n }\n\n input = input.view({batchSize / im2col_step, im2col_step, nInputPlane,\n inputHeight, inputWidth});\n offset =\n offset.view({batchSize / im2col_step, im2col_step,\n deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n Tensor output_buffer = at::zeros({batchSize / im2col_step, nOutputPlane,\n im2col_step * outputHeight, outputWidth},\n output.options());\n\n output_buffer = output_buffer.view(\n {output_buffer.size(0), group, output_buffer.size(1) / group,\n output_buffer.size(2), output_buffer.size(3)});\n\n for (int elt = 0; elt < batchSize / im2col_step; elt++) {\n deformable_im2col_impl(input[elt], offset[elt], nInputPlane, inputHeight,\n inputWidth, kH, kW, padH, padW, dH, dW, dilationH,\n dilationW, im2col_step, deformable_group, columns);\n\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n weight = weight.view({group, weight.size(0) / group, weight.size(1),\n weight.size(2), weight.size(3)});\n\n for (int g = 0; g < group; g++) {\n output_buffer[elt][g] = output_buffer[elt][g]\n .flatten(1)\n .addmm_(weight[g].flatten(1), columns[g])\n .view_as(output_buffer[elt][g]);\n }\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n weight = weight.view({weight.size(0) * weight.size(1), weight.size(2),\n weight.size(3), weight.size(4)});\n }\n\n output_buffer = output_buffer.view(\n {output_buffer.size(0), output_buffer.size(1) * output_buffer.size(2),\n output_buffer.size(3), output_buffer.size(4)});\n\n output_buffer = output_buffer.view({batchSize / im2col_step, nOutputPlane,\n im2col_step, outputHeight, outputWidth});\n output_buffer.transpose_(1, 2);\n output.copy_(output_buffer);\n output = output.view({batchSize, nOutputPlane, outputHeight, outputWidth});\n\n input = input.view({batchSize, nInputPlane, inputHeight, inputWidth});\n offset = offset.view(\n {batchSize, deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n if (batch == 0) {\n output = output.view({nOutputPlane, outputHeight, outputWidth});\n input = input.view({nInputPlane, inputHeight, inputWidth});\n offset = offset.view({offset.size(1), offset.size(2), offset.size(3)});\n }\n}\n\nvoid deform_conv_backward_input(Tensor input, Tensor offset, Tensor gradOutput,\n Tensor gradInput, Tensor gradOffset,\n Tensor weight, Tensor columns, int kW, int kH,\n int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step) {\n if (input.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n CHECK_CUDA_INPUT(input);\n CHECK_CUDA_INPUT(offset);\n CHECK_CUDA_INPUT(gradOutput);\n CHECK_CUDA_INPUT(gradInput);\n CHECK_CUDA_INPUT(gradOffset);\n CHECK_CUDA_INPUT(weight);\n CHECK_CUDA_INPUT(columns);\n#else\n AT_ERROR(\"DeformConv is not compiled with GPU support\");\n#endif\n } else {\n CHECK_CPU_INPUT(input);\n CHECK_CPU_INPUT(offset);\n CHECK_CPU_INPUT(gradOutput);\n CHECK_CPU_INPUT(gradInput);\n CHECK_CPU_INPUT(gradOffset);\n CHECK_CPU_INPUT(weight);\n CHECK_CPU_INPUT(columns);\n }\n deform_conv_shape_check(input, offset, &gradOutput, weight, kH, kW, dH, dW,\n padH, padW, dilationH, dilationW, group,\n deformable_group);\n\n at::DeviceGuard guard(input.device());\n\n int batch = 1;\n if (input.ndimension() == 3) {\n // Force batch\n batch = 0;\n input = input.view({1, input.size(0), input.size(1), input.size(2)});\n offset = offset.view({1, offset.size(0), offset.size(1), offset.size(2)});\n gradOutput = gradOutput.view(\n {1, gradOutput.size(0), gradOutput.size(1), gradOutput.size(2)});\n }\n\n long batchSize = input.size(0);\n long nInputPlane = input.size(1);\n long inputHeight = input.size(2);\n long inputWidth = input.size(3);\n\n long nOutputPlane = weight.size(0);\n\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n\n TORCH_CHECK((offset.size(0) == batchSize), 3, \"invalid batch size of offset\");\n gradInput = gradInput.view({batchSize, nInputPlane, inputHeight, inputWidth});\n columns = at::zeros(\n {nInputPlane * kW * kH, im2col_step * outputHeight * outputWidth},\n input.options());\n\n // change order of grad output\n gradOutput = gradOutput.view({batchSize / im2col_step, im2col_step,\n nOutputPlane, outputHeight, outputWidth});\n gradOutput.transpose_(1, 2);\n\n gradInput = gradInput.view({batchSize / im2col_step, im2col_step, nInputPlane,\n inputHeight, inputWidth});\n input = input.view({batchSize / im2col_step, im2col_step, nInputPlane,\n inputHeight, inputWidth});\n gradOffset = gradOffset.view({batchSize / im2col_step, im2col_step,\n deformable_group * 2 * kH * kW, outputHeight,\n outputWidth});\n offset =\n offset.view({batchSize / im2col_step, im2col_step,\n deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n for (int elt = 0; elt < batchSize / im2col_step; elt++) {\n // divide into groups\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n weight = weight.view({group, weight.size(0) / group, weight.size(1),\n weight.size(2), weight.size(3)});\n gradOutput = gradOutput.view(\n {gradOutput.size(0), group, gradOutput.size(1) / group,\n gradOutput.size(2), gradOutput.size(3), gradOutput.size(4)});\n\n for (int g = 0; g < group; g++) {\n columns[g] = columns[g].addmm_(weight[g].flatten(1).transpose(0, 1),\n gradOutput[elt][g].flatten(1), 0.0f, 1.0f);\n }\n\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n gradOutput = gradOutput.view(\n {gradOutput.size(0), gradOutput.size(1) * gradOutput.size(2),\n gradOutput.size(3), gradOutput.size(4), gradOutput.size(5)});\n\n deformable_col2im_coord_impl(columns, input[elt], offset[elt], nInputPlane,\n inputHeight, inputWidth, kH, kW, padH, padW,\n dH, dW, dilationH, dilationW, im2col_step,\n deformable_group, gradOffset[elt]);\n\n deformable_col2im_impl(columns, offset[elt], nInputPlane, inputHeight,\n inputWidth, kH, kW, padH, padW, dH, dW, dilationH,\n dilationW, im2col_step, deformable_group,\n gradInput[elt]);\n\n weight = weight.view({weight.size(0) * weight.size(1), weight.size(2),\n weight.size(3), weight.size(4)});\n }\n\n gradOutput.transpose_(1, 2);\n gradOutput =\n gradOutput.view({batchSize, nOutputPlane, outputHeight, outputWidth});\n\n gradInput = gradInput.view({batchSize, nInputPlane, inputHeight, inputWidth});\n input = input.view({batchSize, nInputPlane, inputHeight, inputWidth});\n gradOffset = gradOffset.view(\n {batchSize, deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n offset = offset.view(\n {batchSize, deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n if (batch == 0) {\n gradOutput = gradOutput.view({nOutputPlane, outputHeight, outputWidth});\n input = input.view({nInputPlane, inputHeight, inputWidth});\n gradInput = gradInput.view({nInputPlane, inputHeight, inputWidth});\n offset = offset.view({offset.size(1), offset.size(2), offset.size(3)});\n gradOffset =\n gradOffset.view({offset.size(1), offset.size(2), offset.size(3)});\n }\n}\n\nvoid deform_conv_backward_parameters(Tensor input, Tensor offset,\n Tensor gradOutput, Tensor gradWeight,\n Tensor columns, Tensor ones, int kW,\n int kH, int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, float scale,\n int im2col_step) {\n if (input.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n CHECK_CUDA_INPUT(input);\n CHECK_CUDA_INPUT(offset);\n CHECK_CUDA_INPUT(gradOutput);\n CHECK_CUDA_INPUT(gradWeight);\n CHECK_CUDA_INPUT(columns);\n CHECK_CUDA_INPUT(ones);\n#else\n AT_ERROR(\"DeformConv is not compiled with GPU support\");\n#endif\n } else {\n CHECK_CPU_INPUT(input);\n CHECK_CPU_INPUT(offset);\n CHECK_CPU_INPUT(gradOutput);\n CHECK_CPU_INPUT(gradWeight);\n CHECK_CPU_INPUT(columns);\n CHECK_CPU_INPUT(ones);\n }\n\n deform_conv_shape_check(input, offset, &gradOutput, gradWeight, kH, kW, dH,\n dW, padH, padW, dilationH, dilationW, group,\n deformable_group);\n at::DeviceGuard guard(input.device());\n\n int batch = 1;\n\n if (input.ndimension() == 3) {\n // Force batch\n batch = 0;\n input = input.view(\n at::IntList({1, input.size(0), input.size(1), input.size(2)}));\n gradOutput = gradOutput.view(\n {1, gradOutput.size(0), gradOutput.size(1), gradOutput.size(2)});\n }\n\n long batchSize = input.size(0);\n long nInputPlane = input.size(1);\n long inputHeight = input.size(2);\n long inputWidth = input.size(3);\n\n long nOutputPlane = gradWeight.size(0);\n\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n\n TORCH_CHECK((offset.size(0) == batchSize), \"invalid batch size of offset\");\n\n columns = at::zeros(\n {nInputPlane * kW * kH, im2col_step * outputHeight * outputWidth},\n input.options());\n\n gradOutput = gradOutput.view({batchSize / im2col_step, im2col_step,\n nOutputPlane, outputHeight, outputWidth});\n gradOutput.transpose_(1, 2);\n\n Tensor gradOutputBuffer = at::zeros_like(gradOutput);\n gradOutputBuffer =\n gradOutputBuffer.view({batchSize / im2col_step, nOutputPlane, im2col_step,\n outputHeight, outputWidth});\n gradOutputBuffer = gradOutputBuffer.contiguous();\n gradOutputBuffer.copy_(gradOutput);\n gradOutputBuffer =\n gradOutputBuffer.view({batchSize / im2col_step, nOutputPlane,\n im2col_step * outputHeight, outputWidth});\n\n gradOutput.transpose_(1, 2);\n gradOutput =\n gradOutput.view({batchSize, nOutputPlane, outputHeight, outputWidth});\n\n input = input.view({batchSize / im2col_step, im2col_step, nInputPlane,\n inputHeight, inputWidth});\n offset =\n offset.view({batchSize / im2col_step, im2col_step,\n deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n for (int elt = 0; elt < batchSize / im2col_step; elt++) {\n deformable_im2col_impl(input[elt], offset[elt], nInputPlane, inputHeight,\n inputWidth, kH, kW, padH, padW, dH, dW, dilationH,\n dilationW, im2col_step, deformable_group, columns);\n\n // divide into group\n gradOutputBuffer = gradOutputBuffer.view(\n {gradOutputBuffer.size(0), group, gradOutputBuffer.size(1) / group,\n gradOutputBuffer.size(2), gradOutputBuffer.size(3)});\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n gradWeight =\n gradWeight.view({group, gradWeight.size(0) / group, gradWeight.size(1),\n gradWeight.size(2), gradWeight.size(3)});\n\n for (int g = 0; g < group; g++) {\n gradWeight[g] = gradWeight[g]\n .flatten(1)\n .addmm_(gradOutputBuffer[elt][g].flatten(1),\n columns[g].transpose(1, 0), 1.0, scale)\n .view_as(gradWeight[g]);\n }\n gradOutputBuffer = gradOutputBuffer.view(\n {gradOutputBuffer.size(0),\n gradOutputBuffer.size(1) * gradOutputBuffer.size(2),\n gradOutputBuffer.size(3), gradOutputBuffer.size(4)});\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n gradWeight = gradWeight.view({gradWeight.size(0) * gradWeight.size(1),\n gradWeight.size(2), gradWeight.size(3),\n gradWeight.size(4)});\n }\n\n input = input.view({batchSize, nInputPlane, inputHeight, inputWidth});\n offset = offset.view(\n {batchSize, deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n if (batch == 0) {\n gradOutput = gradOutput.view({nOutputPlane, outputHeight, outputWidth});\n input = input.view({nInputPlane, inputHeight, inputWidth});\n }\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/deform_conv_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"deform_conv_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid deform_conv_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kW, kH, dW, dH, padW, padH, dilationW, dilationH, group, deformable_group,\n im2col_step;\n SSAttrs(attr)\n .get(\"kW\", kW)\n .get(\"kH\", kH)\n .get(\"dW\", dW)\n .get(\"dH\", dH)\n .get(\"padW\", padW)\n .get(\"padH\", padH)\n .get(\"dilationW\", dilationW)\n .get(\"dilationH\", dilationH)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"im2col_step\", im2col_step)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& weight = buildATensor(ctx, ins[1]);\n const auto& offset = buildATensor(ctx, ins[2]);\n\n auto output = buildATensor(ctx, outs[0]);\n auto columns = buildATensor(ctx, outs[1]);\n auto ones = buildATensor(ctx, outs[2]);\n\n deform_conv_forward(input, weight, offset, output, columns, ones, kW, kH, dW,\n dH, padW, padH, dilationW, dilationH, group,\n deformable_group, im2col_step);\n}\n\nvoid deform_conv_backward_input_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kW, kH, dW, dH, padW, padH, dilationW, dilationH, group, deformable_group,\n im2col_step;\n SSAttrs(attr)\n .get(\"kW\", kW)\n .get(\"kH\", kH)\n .get(\"dW\", dW)\n .get(\"dH\", dH)\n .get(\"padW\", padW)\n .get(\"padH\", padH)\n .get(\"dilationW\", dilationW)\n .get(\"dilationH\", dilationH)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"im2col_step\", im2col_step)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& offset = buildATensor(ctx, ins[1]);\n const auto& gradOutput = buildATensor(ctx, ins[2]);\n\n auto gradInput = buildATensor(ctx, outs[0]);\n auto gradOffset = buildATensor(ctx, outs[1]);\n auto weight = buildATensor(ctx, outs[2]);\n auto columns = buildATensor(ctx, outs[3]);\n\n deform_conv_backward_input(input, offset, gradOutput, gradInput, gradOffset,\n weight, columns, kW, kH, dW, dH, padW, padH,\n dilationW, dilationH, group, deformable_group,\n im2col_step);\n}\n\nvoid deform_conv_backward_parameters_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kW, kH, dW, dH, padW, padH, dilationW, dilationH, group, deformable_group,\n im2col_step;\n float scale;\n SSAttrs(attr)\n .get(\"kW\", kW)\n .get(\"kH\", kH)\n .get(\"dW\", dW)\n .get(\"dH\", dH)\n .get(\"padW\", padW)\n .get(\"padH\", padH)\n .get(\"dilationW\", dilationW)\n .get(\"dilationH\", dilationH)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"scale\", scale)\n .get(\"im2col_step\", im2col_step)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& offset = buildATensor(ctx, ins[1]);\n const auto& gradOutput = buildATensor(ctx, ins[2]);\n\n auto gradWeight = buildATensor(ctx, outs[0]);\n auto columns = buildATensor(ctx, outs[1]);\n auto ones = buildATensor(ctx, outs[2]);\n deform_conv_backward_parameters(input, offset, gradOutput, gradWeight,\n columns, ones, kW, kH, dW, dH, padW, padH,\n dilationW, dilationH, group, deformable_group,\n scale, im2col_step);\n}\n#endif\n\nvoid deform_conv_forward_cpu_parrots(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kW, kH, dW, dH, padW, padH, dilationW, dilationH, group, deformable_group,\n im2col_step;\n SSAttrs(attr)\n .get(\"kW\", kW)\n .get(\"kH\", kH)\n .get(\"dW\", dW)\n .get(\"dH\", dH)\n .get(\"padW\", padW)\n .get(\"padH\", padH)\n .get(\"dilationW\", dilationW)\n .get(\"dilationH\", dilationH)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"im2col_step\", im2col_step)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& weight = buildATensor(ctx, ins[1]);\n const auto& offset = buildATensor(ctx, ins[2]);\n\n auto output = buildATensor(ctx, outs[0]);\n auto columns = buildATensor(ctx, outs[1]);\n auto ones = buildATensor(ctx, outs[2]);\n\n deform_conv_forward(input, weight, offset, output, columns, ones, kW, kH, dW,\n dH, padW, padH, dilationW, dilationH, group,\n deformable_group, im2col_step);\n}\n\nvoid deform_conv_backward_input_cpu_parrots(HostContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kW, kH, dW, dH, padW, padH, dilationW, dilationH, group, deformable_group,\n im2col_step;\n SSAttrs(attr)\n .get(\"kW\", kW)\n .get(\"kH\", kH)\n .get(\"dW\", dW)\n .get(\"dH\", dH)\n .get(\"padW\", padW)\n .get(\"padH\", padH)\n .get(\"dilationW\", dilationW)\n .get(\"dilationH\", dilationH)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"im2col_step\", im2col_step)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& offset = buildATensor(ctx, ins[1]);\n const auto& gradOutput = buildATensor(ctx, ins[2]);\n\n auto gradInput = buildATensor(ctx, outs[0]);\n auto gradOffset = buildATensor(ctx, outs[1]);\n auto weight = buildATensor(ctx, outs[2]);\n auto columns = buildATensor(ctx, outs[3]);\n\n deform_conv_backward_input(input, offset, gradOutput, gradInput, gradOffset,\n weight, columns, kW, kH, dW, dH, padW, padH,\n dilationW, dilationH, group, deformable_group,\n im2col_step);\n}\n\nvoid deform_conv_backward_parameters_cpu_parrots(\n HostContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kW, kH, dW, dH, padW, padH, dilationW, dilationH, group, deformable_group,\n im2col_step;\n float scale;\n SSAttrs(attr)\n .get(\"kW\", kW)\n .get(\"kH\", kH)\n .get(\"dW\", dW)\n .get(\"dH\", dH)\n .get(\"padW\", padW)\n .get(\"padH\", padH)\n .get(\"dilationW\", dilationW)\n .get(\"dilationH\", dilationH)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"scale\", scale)\n .get(\"im2col_step\", im2col_step)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& offset = buildATensor(ctx, ins[1]);\n const auto& gradOutput = buildATensor(ctx, ins[2]);\n\n auto gradWeight = buildATensor(ctx, outs[0]);\n auto columns = buildATensor(ctx, outs[1]);\n auto ones = buildATensor(ctx, outs[2]);\n deform_conv_backward_parameters(input, offset, gradOutput, gradWeight,\n columns, ones, kW, kH, dW, dH, padW, padH,\n dilationW, dilationH, group, deformable_group,\n scale, im2col_step);\n}\n\nPARROTS_EXTENSION_REGISTER(deform_conv_forward)\n .attr(\"kW\")\n .attr(\"kH\")\n .attr(\"dW\")\n .attr(\"dH\")\n .attr(\"padW\")\n .attr(\"padH\")\n .attr(\"dilationW\")\n .attr(\"dilationH\")\n .attr(\"group\")\n .attr(\"deformable_group\")\n .attr(\"im2col_step\")\n .input(3)\n .output(3)\n .apply(deform_conv_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(deform_conv_forward_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(deform_conv_backward_input)\n .attr(\"kW\")\n .attr(\"kH\")\n .attr(\"dW\")\n .attr(\"dH\")\n .attr(\"padW\")\n .attr(\"padH\")\n .attr(\"dilationW\")\n .attr(\"dilationH\")\n .attr(\"group\")\n .attr(\"deformable_group\")\n .attr(\"im2col_step\")\n .input(3)\n .output(4)\n .apply(deform_conv_backward_input_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(deform_conv_backward_input_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(deform_conv_backward_parameters)\n .attr(\"kW\")\n .attr(\"kH\")\n .attr(\"dW\")\n .attr(\"dH\")\n .attr(\"padW\")\n .attr(\"padH\")\n .attr(\"dilationW\")\n .attr(\"dilationH\")\n .attr(\"group\")\n .attr(\"deformable_group\")\n .attr(\"scale\")\n .attr(\"im2col_step\")\n .input(3)\n .output(3)\n .apply(deform_conv_backward_parameters_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(deform_conv_backward_parameters_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/deform_conv_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef DEFORM_CONV_PYTORCH_H\n#define DEFORM_CONV_PYTORCH_H\n#include \nusing namespace at;\n\nvoid deform_conv_forward(Tensor input, Tensor weight, Tensor offset,\n Tensor output, Tensor columns, Tensor ones, int kW,\n int kH, int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step);\n\nvoid deform_conv_backward_input(Tensor input, Tensor offset, Tensor gradOutput,\n Tensor gradInput, Tensor gradOffset,\n Tensor weight, Tensor columns, int kW, int kH,\n int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step);\n\nvoid deform_conv_backward_parameters(Tensor input, Tensor offset,\n Tensor gradOutput, Tensor gradWeight,\n Tensor columns, Tensor ones, int kW,\n int kH, int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, float scale,\n int im2col_step);\n\n#endif // DEFORM_CONV_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/deform_roi_pool.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid deform_roi_pool_forward_impl(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma) {\n DISPATCH_DEVICE_IMPL(deform_roi_pool_forward_impl, input, rois, offset,\n output, pooled_height, pooled_width, spatial_scale,\n sampling_ratio, gamma);\n}\n\nvoid deform_roi_pool_backward_impl(Tensor grad_output, Tensor input,\n Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset,\n int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma) {\n DISPATCH_DEVICE_IMPL(deform_roi_pool_backward_impl, grad_output, input, rois,\n offset, grad_input, grad_offset, pooled_height,\n pooled_width, spatial_scale, sampling_ratio, gamma);\n}\n\nvoid deform_roi_pool_forward(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma) {\n deform_roi_pool_forward_impl(input, rois, offset, output, pooled_height,\n pooled_width, spatial_scale, sampling_ratio,\n gamma);\n}\n\nvoid deform_roi_pool_backward(Tensor grad_output, Tensor input, Tensor rois,\n Tensor offset, Tensor grad_input,\n Tensor grad_offset, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma) {\n deform_roi_pool_backward_impl(grad_output, input, rois, offset, grad_input,\n grad_offset, pooled_height, pooled_width,\n spatial_scale, sampling_ratio, gamma);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/deform_roi_pool_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"deform_roi_pool_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\n/*void deform_roi_pool_forward_cuda(Tensor input, Tensor rois, Tensor offset,\n * Tensor output, int pooled_height,\n * int pooled_width, float spatial_scale,\n * int sampling_ratio, float gamma);\n */\nvoid deform_roi_pool_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n int sampling_ratio;\n float gamma;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sampling_ratio\", sampling_ratio)\n .get(\"gamma\", gamma)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n const auto& offset = buildATensor(ctx, ins[2]);\n\n auto output = buildATensor(ctx, outs[0]);\n deform_roi_pool_forward_cuda(input, rois, offset, output, pooled_height,\n pooled_width, spatial_scale, sampling_ratio,\n gamma);\n}\n\n/*void deform_roi_pool_backward_cuda(Tensor grad_output, Tensor input,\n * Tensor rois, Tensor offset,\n * Tensor grad_input, Tensor grad_offset,\n * int pooled_height, int pooled_width,\n * float spatial_scale, int sampling_ratio,\n * float gamma);\n */\nvoid deform_roi_pool_backward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n int sampling_ratio;\n float gamma;\n\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sampling_ratio\", sampling_ratio)\n .get(\"gamma\", gamma)\n .done();\n\n const auto& grad_output = buildATensor(ctx, ins[0]);\n const auto& input = buildATensor(ctx, ins[1]);\n const auto& rois = buildATensor(ctx, ins[2]);\n const auto& offset = buildATensor(ctx, ins[3]);\n\n auto grad_input = buildATensor(ctx, outs[0]);\n auto grad_offset = buildATensor(ctx, outs[1]);\n\n deform_roi_pool_backward_cuda(grad_output, input, rois, offset, grad_input,\n grad_offset, pooled_height, pooled_width,\n spatial_scale, sampling_ratio, gamma);\n}\n\nPARROTS_EXTENSION_REGISTER(deform_roi_pool_forward)\n .attr(\"pooled_height\")\n .attr(\"pooled_width\")\n .attr(\"spatial_scale\")\n .attr(\"sampling_ratio\")\n .attr(\"gamma\")\n .input(3)\n .output(1)\n .apply(deform_roi_pool_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(deform_roi_pool_backward)\n .attr(\"pooled_height\")\n .attr(\"pooled_width\")\n .attr(\"spatial_scale\")\n .attr(\"sampling_ratio\")\n .attr(\"gamma\")\n .input(4)\n .output(2)\n .apply(deform_roi_pool_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/deform_roi_pool_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef DEFORM_ROI_POOL_PYTORCH_H\n#define DEFORM_ROI_POOL_PYTORCH_H\n#include \nusing namespace at;\n\nvoid deform_roi_pool_forward_cuda(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma);\n\nvoid deform_roi_pool_backward_cuda(Tensor grad_output, Tensor input,\n Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset,\n int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma);\n#endif // DEFORM_ROI_POOL_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/focal_loss.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid sigmoid_focal_loss_forward_impl(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n DISPATCH_DEVICE_IMPL(sigmoid_focal_loss_forward_impl, input, target, weight,\n output, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_backward_impl(Tensor input, Tensor target,\n Tensor weight, Tensor grad_input,\n float gamma, float alpha) {\n DISPATCH_DEVICE_IMPL(sigmoid_focal_loss_backward_impl, input, target, weight,\n grad_input, gamma, alpha);\n}\n\nvoid softmax_focal_loss_forward_impl(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n DISPATCH_DEVICE_IMPL(softmax_focal_loss_forward_impl, input, target, weight,\n output, gamma, alpha);\n}\n\nvoid softmax_focal_loss_backward_impl(Tensor input, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input, float gamma,\n float alpha) {\n DISPATCH_DEVICE_IMPL(softmax_focal_loss_backward_impl, input, target, weight,\n buff, grad_input, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_forward(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n sigmoid_focal_loss_forward_impl(input, target, weight, output, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_backward(Tensor input, Tensor target, Tensor weight,\n Tensor grad_input, float gamma, float alpha) {\n sigmoid_focal_loss_backward_impl(input, target, weight, grad_input, gamma,\n alpha);\n}\n\nvoid softmax_focal_loss_forward(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n softmax_focal_loss_forward_impl(input, target, weight, output, gamma, alpha);\n}\n\nvoid softmax_focal_loss_backward(Tensor input, Tensor target, Tensor weight,\n Tensor buff, Tensor grad_input, float gamma,\n float alpha) {\n softmax_focal_loss_backward_impl(input, target, weight, buff, grad_input,\n gamma, alpha);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/focal_loss_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"focal_loss_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid sigmoid_focal_loss_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float gamma;\n float alpha;\n SSAttrs(attr).get(\"gamma\", gamma).get(\"alpha\", alpha).done();\n\n // get inputs and outputs\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& target = buildATensor(ctx, ins[1]);\n const auto& weight = buildATensor(ctx, ins[2]);\n\n auto output = buildATensor(ctx, outs[0]);\n\n sigmoid_focal_loss_forward_cuda(input, target, weight, output, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_backward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float gamma;\n float alpha;\n SSAttrs(attr).get(\"gamma\", gamma).get(\"alpha\", alpha).done();\n\n // get inputs and outputs\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& target = buildATensor(ctx, ins[1]);\n const auto& weight = buildATensor(ctx, ins[2]);\n\n auto grad_input = buildATensor(ctx, outs[0]);\n\n sigmoid_focal_loss_backward_cuda(input, target, weight, grad_input, gamma,\n alpha);\n}\n\nvoid softmax_focal_loss_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float gamma;\n float alpha;\n SSAttrs(attr).get(\"gamma\", gamma).get(\"alpha\", alpha).done();\n\n // get inputs and outputs\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& target = buildATensor(ctx, ins[1]);\n const auto& weight = buildATensor(ctx, ins[2]);\n\n auto output = buildATensor(ctx, outs[0]);\n softmax_focal_loss_forward_cuda(input, target, weight, output, gamma, alpha);\n}\n\nvoid softmax_focal_loss_backward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float gamma;\n float alpha;\n SSAttrs(attr).get(\"gamma\", gamma).get(\"alpha\", alpha).done();\n\n // get inputs and outputs\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& target = buildATensor(ctx, ins[1]);\n const auto& weight = buildATensor(ctx, ins[2]);\n\n auto buff = buildATensor(ctx, outs[0]);\n auto grad_input = buildATensor(ctx, outs[1]);\n softmax_focal_loss_backward_cuda(input, target, weight, buff, grad_input,\n gamma, alpha);\n}\n\nPARROTS_EXTENSION_REGISTER(sigmoid_focal_loss_forward)\n .attr(\"gamma\")\n .attr(\"alpha\")\n .input(3)\n .output(1)\n .apply(sigmoid_focal_loss_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(sigmoid_focal_loss_backward)\n .attr(\"gamma\")\n .attr(\"alpha\")\n .input(3)\n .output(1)\n .apply(sigmoid_focal_loss_backward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(softmax_focal_loss_forward)\n .attr(\"gamma\")\n .attr(\"alpha\")\n .input(3)\n .output(1)\n .apply(softmax_focal_loss_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(softmax_focal_loss_backward)\n .attr(\"gamma\")\n .attr(\"alpha\")\n .input(3)\n .output(2)\n .apply(softmax_focal_loss_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/focal_loss_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef FOCAL_LOSS_PYTORCH_H\n#define FOCAL_LOSS_PYTORCH_H\n#include \nusing namespace at;\n\nvoid sigmoid_focal_loss_forward_cuda(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha);\n\nvoid sigmoid_focal_loss_backward_cuda(Tensor input, Tensor target,\n Tensor weight, Tensor grad_input,\n float gamma, float alpha);\n\nvoid softmax_focal_loss_forward_cuda(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha);\n\nvoid softmax_focal_loss_backward_cuda(Tensor input, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input, float gamma,\n float alpha);\n#endif // FOCAL_LOSS_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/furthest_point_sample.cpp", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/sampling.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid furthest_point_sampling_forward_impl(Tensor points_tensor,\n Tensor temp_tensor, Tensor idx_tensor,\n int b, int n, int m) {\n DISPATCH_DEVICE_IMPL(furthest_point_sampling_forward_impl, points_tensor,\n temp_tensor, idx_tensor, b, n, m);\n}\n\nvoid furthest_point_sampling_with_dist_forward_impl(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b,\n int n, int m) {\n DISPATCH_DEVICE_IMPL(furthest_point_sampling_with_dist_forward_impl,\n points_tensor, temp_tensor, idx_tensor, b, n, m);\n}\n\nvoid furthest_point_sampling_forward(Tensor points_tensor, Tensor temp_tensor,\n Tensor idx_tensor, int b, int n, int m) {\n furthest_point_sampling_forward_impl(points_tensor, temp_tensor, idx_tensor,\n b, n, m);\n}\n\nvoid furthest_point_sampling_with_dist_forward(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b, int n,\n int m) {\n furthest_point_sampling_with_dist_forward_impl(points_tensor, temp_tensor,\n idx_tensor, b, n, m);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/furthest_point_sample_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"furthest_point_sample_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid furthest_point_sample_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, n, m;\n SSAttrs(attr).get(\"b\", b).get(\"n\", n).get(\"m\", m).done();\n\n auto points_tensor = buildATensor(ctx, ins[0]);\n auto temp_tensor = buildATensor(ctx, ins[1]);\n\n auto idx_tensor = buildATensor(ctx, outs[0]);\n\n furthest_point_sampling_forward(points_tensor, temp_tensor, idx_tensor, b, n,\n m);\n}\n\nvoid furthest_point_sampling_with_dist_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, n, m;\n SSAttrs(attr).get(\"b\", b).get(\"n\", n).get(\"m\", m).done();\n\n auto points_tensor = buildATensor(ctx, ins[0]);\n auto temp_tensor = buildATensor(ctx, ins[1]);\n\n auto idx_tensor = buildATensor(ctx, outs[0]);\n\n furthest_point_sampling_with_dist_forward(points_tensor, temp_tensor,\n idx_tensor, b, n, m);\n}\nPARROTS_EXTENSION_REGISTER(furthest_point_sampling_forward)\n .attr(\"b\")\n .attr(\"n\")\n .attr(\"m\")\n .input(2)\n .output(1)\n .apply(furthest_point_sample_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(furthest_point_sampling_with_dist_forward)\n .attr(\"b\")\n .attr(\"n\")\n .attr(\"m\")\n .input(2)\n .output(1)\n .apply(furthest_point_sampling_with_dist_forward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/furthest_point_sample_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef FURTHEST_POINT_SAMPLE_PYTORCH_H\n#define FURTHEST_POINT_SAMPLE_PYTORCH_H\n#include \nusing namespace at;\n\nvoid furthest_point_sampling_forward(Tensor points_tensor, Tensor temp_tensor,\n Tensor idx_tensor, int b, int n, int m);\n\nvoid furthest_point_sampling_with_dist_forward(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b, int n,\n int m);\n#endif // FURTHEST_POINT_SAMPLE_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/fused_bias_leakyrelu.cpp", "content": "// Modified from\n// https://github.com/rosinality/stylegan2-pytorch/blob/master/op/fused_bias_act.cpp\n\n/*\nCopyright (c) 2021, NVIDIA Corporation. All rights reserved.\n\nNVIDIA Source Code License for StyleGAN2 with Adaptive Discriminator\nAugmentation (ADA)\n=======================================================================\n\n1. Definitions\n\n\"Licensor\" means any person or entity that distributes its Work.\n\n\"Software\" means the original work of authorship made available under\nthis License.\n\n\"Work\" means the Software and any additions to or derivative works of\nthe Software that are made available under this License.\n\nThe terms \"reproduce,\" \"reproduction,\" \"derivative works,\" and\n\"distribution\" have the meaning as provided under U.S. copyright law;\nprovided, however, that for the purposes of this License, derivative\nworks shall not include works that remain separable from, or merely\nlink (or bind by name) to the interfaces of, the Work.\n\nWorks, including the Software, are \"made available\" under this License\nby including in or with the Work either (a) a copyright notice\nreferencing the applicability of this License to the Work, or (b) a\ncopy of this License.\n\n2. License Grants\n\n 2.1 Copyright Grant. Subject to the terms and conditions of this\n License, each Licensor grants to you a perpetual, worldwide,\n non-exclusive, royalty-free, copyright license to reproduce,\n prepare derivative works of, publicly display, publicly perform,\n sublicense and distribute its Work and any resulting derivative\n works in any form.\n\n3. Limitations\n\n 3.1 Redistribution. You may reproduce or distribute the Work only\n if (a) you do so under this License, (b) you include a complete\n copy of this License with your distribution, and (c) you retain\n without modification any copyright, patent, trademark, or\n attribution notices that are present in the Work.\n\n 3.2 Derivative Works. You may specify that additional or different\n terms apply to the use, reproduction, and distribution of your\n derivative works of the Work (\"Your Terms\") only if (a) Your Terms\n provide that the use limitation in Section 3.3 applies to your\n derivative works, and (b) you identify the specific derivative\n works that are subject to Your Terms. Notwithstanding Your Terms,\n this License (including the redistribution requirements in Section\n 3.1) will continue to apply to the Work itself.\n\n 3.3 Use Limitation. The Work and any derivative works thereof only\n may be used or intended for use non-commercially. Notwithstanding\n the foregoing, NVIDIA and its affiliates may use the Work and any\n derivative works commercially. As used herein, \"non-commercially\"\n means for research or evaluation purposes only.\n\n 3.4 Patent Claims. If you bring or threaten to bring a patent claim\n against any Licensor (including any claim, cross-claim or\n counterclaim in a lawsuit) to enforce any patents that you allege\n are infringed by any Work, then your rights under this License from\n such Licensor (including the grant in Section 2.1) will terminate\n immediately.\n\n 3.5 Trademarks. This License does not grant any rights to use any\n Licensor’s or its affiliates’ names, logos, or trademarks, except\n as necessary to reproduce the notices described in this License.\n\n 3.6 Termination. If you violate any term of this License, then your\n rights under this License (including the grant in Section 2.1) will\n terminate immediately.\n\n4. Disclaimer of Warranty.\n\nTHE WORK IS PROVIDED \"AS IS\" WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR\nNON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER\nTHIS LICENSE.\n\n5. Limitation of Liability.\n\nEXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL\nTHEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE\nSHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT,\nINDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF\nOR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK\n(INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION,\nLOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER\nCOMMERCIAL DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF\nTHE POSSIBILITY OF SUCH DAMAGES.\n\n=======================================================================\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntorch::Tensor fused_bias_leakyrelu_op_impl(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale) {\n return DISPATCH_DEVICE_IMPL(fused_bias_leakyrelu_op_impl, input, bias, refer,\n act, grad, alpha, scale);\n}\n\ntorch::Tensor fused_bias_leakyrelu(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale) {\n return fused_bias_leakyrelu_op_impl(input, bias, refer, act, grad, alpha,\n scale);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/fused_bias_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n\n#include \n#include \n#include \nusing namespace at;\nusing namespace parrots;\n\ntorch::Tensor fused_bias_leakyrelu(const torch::Tensor &input,\n const torch::Tensor &bias,\n const torch::Tensor &refer, int act,\n int grad, float alpha, float scale);\n\nvoid fused_bias_leakyrelu_parrots(CudaContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n int act, grad;\n float alpha, scale;\n SSAttrs(attr)\n .get(\"act\", act)\n .get(\"grad\", grad)\n .get(\"alpha\", alpha)\n .get(\"scale\", scale)\n .done();\n const auto &input = buildATensor(ctx, ins[0]);\n const auto &bias = buildATensor(ctx, ins[1]);\n const auto &refer = buildATensor(ctx, ins[2]);\n auto out = fused_bias_leakyrelu(input, bias, refer, act, grad, alpha, scale);\n updateDArray(ctx, out, outs[0]);\n}\n\nPARROTS_EXTENSION_REGISTER(fused_bias_leakyrelu)\n .attr(\"act\")\n .attr(\"grad\")\n .attr(\"alpha\")\n .attr(\"scale\")\n .input(3)\n .output(1)\n .apply(fused_bias_leakyrelu_parrots)\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/gather_points.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid gather_points_forward_impl(int b, int c, int n, int npoints,\n const Tensor points, const Tensor idx,\n Tensor out) {\n DISPATCH_DEVICE_IMPL(gather_points_forward_impl, b, c, n, npoints, points,\n idx, out);\n}\n\nvoid gather_points_backward_impl(int b, int c, int n, int npoints,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points) {\n DISPATCH_DEVICE_IMPL(gather_points_backward_impl, b, c, n, npoints, grad_out,\n idx, grad_points);\n}\n\nvoid gather_points_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor out_tensor, int b, int c, int n,\n int npoints) {\n gather_points_forward_impl(b, c, n, npoints, points_tensor, idx_tensor,\n out_tensor);\n}\n\nvoid gather_points_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor grad_points_tensor, int b, int c, int n,\n int npoints) {\n gather_points_backward_impl(b, c, n, npoints, grad_out_tensor, idx_tensor,\n grad_points_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/gather_points_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"gather_points_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid gather_points_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, c, n, npoints;\n SSAttrs(attr)\n .get(\"b\", b)\n .get(\"c\", c)\n .get(\"n\", n)\n .get(\"npoints\", npoints)\n .done();\n\n auto points_tensor = buildATensor(ctx, ins[0]);\n auto idx_tensor = buildATensor(ctx, ins[1]);\n\n auto out_tensor = buildATensor(ctx, outs[0]);\n\n gather_points_forward(points_tensor, idx_tensor, out_tensor, b, c, n,\n npoints);\n}\n\nvoid gather_points_backward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, c, n, npoints;\n SSAttrs(attr)\n .get(\"b\", b)\n .get(\"c\", c)\n .get(\"n\", n)\n .get(\"npoints\", npoints)\n .done();\n\n auto grad_out_tensor = buildATensor(ctx, ins[0]);\n auto idx_tensor = buildATensor(ctx, ins[1]);\n\n auto grad_points_tensor = buildATensor(ctx, outs[0]);\n\n gather_points_backward(grad_out_tensor, idx_tensor, grad_points_tensor, b, c,\n n, npoints);\n}\n\nPARROTS_EXTENSION_REGISTER(gather_points_forward)\n .attr(\"b\")\n .attr(\"c\")\n .attr(\"n\")\n .attr(\"npoints\")\n .input(2)\n .output(1)\n .apply(gather_points_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(gather_points_backward)\n .attr(\"b\")\n .attr(\"c\")\n .attr(\"n\")\n .attr(\"npoints\")\n .input(2)\n .output(1)\n .apply(gather_points_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/gather_points_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef GATHER_POINTS_PYTORCH_H\n#define GATHER_POINTS_PYTORCH_H\n#include \nusing namespace at;\n\nvoid gather_points_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor out_tensor, int b, int c, int n, int npoints);\n\nvoid gather_points_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor grad_points_tensor, int b, int c, int n,\n int npoints);\n#endif // GATHER_POINTS_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/group_points.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/group_points.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid group_points_forward_impl(int b, int c, int n, int npoints, int nsample,\n const Tensor points, const Tensor idx,\n Tensor out) {\n DISPATCH_DEVICE_IMPL(group_points_forward_impl, b, c, n, npoints, nsample,\n points, idx, out);\n}\n\nvoid group_points_backward_impl(int b, int c, int n, int npoints, int nsample,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points) {\n DISPATCH_DEVICE_IMPL(group_points_backward_impl, b, c, n, npoints, nsample,\n grad_out, idx, grad_points);\n}\n\nvoid group_points_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor out_tensor, int b, int c, int n, int npoints,\n int nsample) {\n DISPATCH_DEVICE_IMPL(group_points_forward_impl, b, c, n, npoints, nsample,\n points_tensor, idx_tensor, out_tensor);\n}\n\nvoid group_points_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor grad_points_tensor, int b, int c, int n,\n int npoints, int nsample) {\n group_points_backward_impl(b, c, n, npoints, nsample, grad_out_tensor,\n idx_tensor, grad_points_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/group_points_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"group_points_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid group_points_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, c, n, npoints, nsample;\n SSAttrs(attr)\n .get(\"b\", b)\n .get(\"c\", c)\n .get(\"n\", n)\n .get(\"npoints\", npoints)\n .get(\"nsample\", nsample)\n .done();\n auto points_tensor = buildATensor(ctx, ins[0]);\n auto idx_tensor = buildATensor(ctx, ins[1]);\n\n auto out_tensor = buildATensor(ctx, outs[0]);\n\n group_points_forward(points_tensor, idx_tensor, out_tensor, b, c, n, npoints,\n nsample);\n}\n\nvoid group_points_backward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, c, n, npoints, nsample;\n SSAttrs(attr)\n .get(\"b\", b)\n .get(\"c\", c)\n .get(\"n\", n)\n .get(\"npoints\", npoints)\n .get(\"nsample\", nsample)\n .done();\n auto grad_out_tensor = buildATensor(ctx, ins[0]);\n auto idx_tensor = buildATensor(ctx, ins[1]);\n\n auto grad_points_tensor = buildATensor(ctx, outs[0]);\n\n group_points_backward(grad_out_tensor, idx_tensor, grad_points_tensor, b, c,\n n, npoints, nsample);\n}\n\nPARROTS_EXTENSION_REGISTER(group_points_forward)\n .attr(\"b\")\n .attr(\"c\")\n .attr(\"n\")\n .attr(\"npoints\")\n .attr(\"nsample\")\n .input(2)\n .output(1)\n .apply(group_points_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(group_points_backward)\n .attr(\"b\")\n .attr(\"c\")\n .attr(\"n\")\n .attr(\"npoints\")\n .attr(\"nsample\")\n .input(2)\n .output(1)\n .apply(group_points_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/group_points_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef GROUP_POINTS_PYTORCH_H\n#define GROUP_POINTS_PYTORCH_H\n#include \nusing namespace at;\n\nvoid group_points_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor out_tensor, int b, int c, int n, int npoints,\n int nsample);\n\nvoid group_points_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor grad_points_tensor, int b, int c, int n,\n int npoints, int nsample);\n\n#endif // GROUP_POINTS_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/info.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/vision.cpp\n#include \"pytorch_cpp_helper.hpp\"\n\n#ifdef MMCV_WITH_CUDA\n#ifndef HIP_DIFF\n#include \nint get_cudart_version() { return CUDART_VERSION; }\n#endif\n#endif\n\nstd::string get_compiling_cuda_version() {\n#ifdef MMCV_WITH_CUDA\n#ifndef HIP_DIFF\n std::ostringstream oss;\n // copied from\n // https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/cuda/detail/CUDAHooks.cpp#L231\n auto printCudaStyleVersion = [&](int v) {\n oss << (v / 1000) << \".\" << (v / 10 % 100);\n if (v % 10 != 0) {\n oss << \".\" << (v % 10);\n }\n };\n printCudaStyleVersion(get_cudart_version());\n return oss.str();\n#else\n return std::string(\"rocm not available\");\n#endif\n#else\n return std::string(\"not available\");\n#endif\n}\n\n// similar to\n// https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/Version.cpp\nstd::string get_compiler_version() {\n std::ostringstream ss;\n#if defined(__GNUC__)\n#ifndef __clang__\n { ss << \"GCC \" << __GNUC__ << \".\" << __GNUC_MINOR__; }\n#endif\n#endif\n\n#if defined(__clang_major__)\n {\n ss << \"clang \" << __clang_major__ << \".\" << __clang_minor__ << \".\"\n << __clang_patchlevel__;\n }\n#endif\n\n#if defined(_MSC_VER)\n { ss << \"MSVC \" << _MSC_FULL_VER; }\n#endif\n return ss.str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/iou3d.cpp", "content": "// Modified from\n// https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/ops/iou3d_nms/src/iou3d_nms.cpp\n\n/*\n3D IoU Calculation and Rotated NMS(modified from 2D NMS written by others)\nWritten by Shaoshuai Shi\nAll Rights Reserved 2019-2020.\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nconst int THREADS_PER_BLOCK_NMS = sizeof(unsigned long long) * 8;\n\nvoid iou3d_boxes_overlap_bev_forward_impl(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_overlap) {\n DISPATCH_DEVICE_IMPL(iou3d_boxes_overlap_bev_forward_impl, num_a, boxes_a,\n num_b, boxes_b, ans_overlap);\n}\n\nvoid iou3d_boxes_iou_bev_forward_impl(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_iou) {\n DISPATCH_DEVICE_IMPL(iou3d_boxes_iou_bev_forward_impl, num_a, boxes_a, num_b,\n boxes_b, ans_iou);\n}\n\nvoid iou3d_nms_forward_impl(const Tensor boxes, unsigned long long *mask,\n int boxes_num, float nms_overlap_thresh) {\n DISPATCH_DEVICE_IMPL(iou3d_nms_forward_impl, boxes, mask, boxes_num,\n nms_overlap_thresh);\n}\n\nvoid iou3d_nms_normal_forward_impl(const Tensor boxes, unsigned long long *mask,\n int boxes_num, float nms_overlap_thresh) {\n DISPATCH_DEVICE_IMPL(iou3d_nms_normal_forward_impl, boxes, mask, boxes_num,\n nms_overlap_thresh);\n}\n\nvoid iou3d_boxes_overlap_bev_forward(Tensor boxes_a, Tensor boxes_b,\n Tensor ans_overlap) {\n // params boxes_a: (N, 5) [x1, y1, x2, y2, ry]\n // params boxes_b: (M, 5)\n // params ans_overlap: (N, M)\n\n int num_a = boxes_a.size(0);\n int num_b = boxes_b.size(0);\n\n iou3d_boxes_overlap_bev_forward_impl(num_a, boxes_a, num_b, boxes_b,\n ans_overlap);\n}\n\nvoid iou3d_boxes_iou_bev_forward(Tensor boxes_a, Tensor boxes_b,\n Tensor ans_iou) {\n // params boxes_a: (N, 5) [x1, y1, x2, y2, ry]\n // params boxes_b: (M, 5)\n // params ans_overlap: (N, M)\n int num_a = boxes_a.size(0);\n int num_b = boxes_b.size(0);\n\n iou3d_boxes_iou_bev_forward_impl(num_a, boxes_a, num_b, boxes_b, ans_iou);\n}\n\nvoid iou3d_nms_forward(Tensor boxes, Tensor keep, Tensor keep_num,\n float nms_overlap_thresh) {\n // params boxes: (N, 5) [x1, y1, x2, y2, ry]\n // params keep: (N)\n CHECK_CONTIGUOUS(boxes);\n CHECK_CONTIGUOUS(keep);\n\n int boxes_num = boxes.size(0);\n int64_t *keep_data = keep.data_ptr();\n int64_t *keep_num_data = keep_num.data_ptr();\n\n const int col_blocks =\n (boxes_num + THREADS_PER_BLOCK_NMS - 1) / THREADS_PER_BLOCK_NMS;\n\n Tensor mask =\n at::empty({boxes_num, col_blocks}, boxes.options().dtype(at::kLong));\n unsigned long long *mask_data =\n (unsigned long long *)mask.data_ptr();\n iou3d_nms_forward_impl(boxes, mask_data, boxes_num, nms_overlap_thresh);\n\n at::Tensor mask_cpu = mask.to(at::kCPU);\n unsigned long long *mask_host =\n (unsigned long long *)mask_cpu.data_ptr();\n\n std::vector remv_cpu(col_blocks);\n memset(&remv_cpu[0], 0, sizeof(unsigned long long) * col_blocks);\n\n int num_to_keep = 0;\n\n for (int i = 0; i < boxes_num; i++) {\n int nblock = i / THREADS_PER_BLOCK_NMS;\n int inblock = i % THREADS_PER_BLOCK_NMS;\n\n if (!(remv_cpu[nblock] & (1ULL << inblock))) {\n keep_data[num_to_keep++] = i;\n unsigned long long *p = &mask_host[0] + i * col_blocks;\n for (int j = nblock; j < col_blocks; j++) {\n remv_cpu[j] |= p[j];\n }\n }\n *keep_num_data = num_to_keep;\n }\n}\n\nvoid iou3d_nms_normal_forward(Tensor boxes, Tensor keep, Tensor keep_num,\n float nms_overlap_thresh) {\n // params boxes: (N, 5) [x1, y1, x2, y2, ry]\n // params keep: (N)\n\n CHECK_CONTIGUOUS(boxes);\n CHECK_CONTIGUOUS(keep);\n\n int boxes_num = boxes.size(0);\n int64_t *keep_data = keep.data_ptr();\n int64_t *keep_num_data = keep_num.data_ptr();\n\n const int col_blocks =\n (boxes_num + THREADS_PER_BLOCK_NMS - 1) / THREADS_PER_BLOCK_NMS;\n\n Tensor mask =\n at::empty({boxes_num, col_blocks}, boxes.options().dtype(at::kLong));\n unsigned long long *mask_data =\n (unsigned long long *)mask.data_ptr();\n iou3d_nms_normal_forward_impl(boxes, mask_data, boxes_num,\n nms_overlap_thresh);\n\n at::Tensor mask_cpu = mask.to(at::kCPU);\n unsigned long long *mask_host =\n (unsigned long long *)mask_cpu.data_ptr();\n\n std::vector remv_cpu(col_blocks);\n memset(&remv_cpu[0], 0, sizeof(unsigned long long) * col_blocks);\n int num_to_keep = 0;\n\n for (int i = 0; i < boxes_num; i++) {\n int nblock = i / THREADS_PER_BLOCK_NMS;\n int inblock = i % THREADS_PER_BLOCK_NMS;\n\n if (!(remv_cpu[nblock] & (1ULL << inblock))) {\n keep_data[num_to_keep++] = i;\n unsigned long long *p = &mask_host[0] + i * col_blocks;\n for (int j = nblock; j < col_blocks; j++) {\n remv_cpu[j] |= p[j];\n }\n }\n }\n\n *keep_num_data = num_to_keep;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/iou3d_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"iou3d_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid iou3d_boxes_iou_bev_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto boxes_a = buildATensor(ctx, ins[0]);\n auto boxes_b = buildATensor(ctx, ins[1]);\n\n auto ans_iou = buildATensor(ctx, outs[0]);\n\n iou3d_boxes_iou_bev_forward(boxes_a, boxes_b, ans_iou);\n}\n\nvoid iou3d_nms_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float nms_overlap_thresh;\n SSAttrs(attr).get(\"nms_overlap_thresh\", nms_overlap_thresh).done();\n\n auto boxes = buildATensor(ctx, ins[0]);\n\n auto keep = buildATensor(ctx, outs[0]);\n auto keep_num = buildATensor(ctx, outs[1]);\n\n iou3d_nms_forward(boxes, keep, keep_num, nms_overlap_thresh);\n}\n\nvoid iou3d_nms_normal_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float nms_overlap_thresh;\n SSAttrs(attr).get(\"nms_overlap_thresh\", nms_overlap_thresh).done();\n\n auto boxes = buildATensor(ctx, ins[0]);\n\n auto keep = buildATensor(ctx, outs[0]);\n auto keep_num = buildATensor(ctx, outs[1]);\n\n iou3d_nms_normal_forward(boxes, keep, keep_num, nms_overlap_thresh);\n}\n\nPARROTS_EXTENSION_REGISTER(iou3d_boxes_iou_bev_forward)\n .input(2)\n .output(1)\n .apply(iou3d_boxes_iou_bev_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(iou3d_nms_forward)\n .attr(\"nms_overlap_thresh\")\n .input(1)\n .output(2)\n .apply(iou3d_nms_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(iou3d_nms_normal_forward)\n .attr(\"nms_overlap_thresh\")\n .input(1)\n .output(2)\n .apply(iou3d_nms_normal_forward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/iou3d_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef IOU_3D_PYTORCH_H\n#define IOU_3D_PYTORCH_H\n#include \nusing namespace at;\n\nvoid iou3d_boxes_iou_bev_forward(Tensor boxes_a, Tensor boxes_b,\n Tensor ans_iou);\n\nvoid iou3d_nms_forward(Tensor boxes, Tensor keep, Tensor keep_num,\n float nms_overlap_thresh);\n\nvoid iou3d_nms_normal_forward(Tensor boxes, Tensor keep, Tensor keep_num,\n float nms_overlap_thresh);\n\n#endif // IOU_3D_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/knn.cpp", "content": "// Modified from\n// https://github.com/CVMI-Lab/PAConv/tree/main/scene_seg/lib/pointops/src/knnquery_heap\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid knn_forward_impl(int b, int n, int m, int nsample, const Tensor xyz,\n const Tensor new_xyz, Tensor idx, Tensor dist2) {\n DISPATCH_DEVICE_IMPL(knn_forward_impl, b, n, m, nsample, xyz, new_xyz, idx,\n dist2);\n}\n\nvoid knn_forward(Tensor xyz_tensor, Tensor new_xyz_tensor, Tensor idx_tensor,\n Tensor dist2_tensor, int b, int n, int m, int nsample) {\n knn_forward_impl(b, n, m, nsample, xyz_tensor, new_xyz_tensor, idx_tensor,\n dist2_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/knn_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"knn_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid knn_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, n, m, nsample;\n SSAttrs(attr)\n .get(\"b\", b)\n .get(\"n\", n)\n .get(\"m\", m)\n .get(\"nsample\", nsample)\n .done();\n\n auto xyz_tensor = buildATensor(ctx, ins[0]);\n auto new_xyz_tensor = buildATensor(ctx, ins[1]);\n\n auto idx_tensor = buildATensor(ctx, outs[0]);\n auto dist2_tensor = buildATensor(ctx, outs[1]);\n\n knn_forward(xyz_tensor, new_xyz_tensor, idx_tensor, dist2_tensor, b, n, m,\n nsample);\n}\n\nPARROTS_EXTENSION_REGISTER(knn_forward)\n .attr(\"b\")\n .attr(\"n\")\n .attr(\"m\")\n .attr(\"nsample\")\n .input(2)\n .output(2)\n .apply(knn_forward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/knn_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef KNN_PYTORCH_H\n#define KNN_PYTORCH_H\n#include \nusing namespace at;\n\nvoid knn_forward(Tensor xyz_tensor, Tensor new_xyz_tensor, Tensor idx_tensor,\n Tensor dist2_tensor, int b, int n, int m, int nsample);\n#endif // KNN_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/masked_conv2d.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid masked_im2col_forward_impl(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w) {\n DISPATCH_DEVICE_IMPL(masked_im2col_forward_impl, im, mask_h_idx, mask_w_idx,\n col, kernel_h, kernel_w, pad_h, pad_w);\n}\n\nvoid masked_col2im_forward_impl(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels) {\n DISPATCH_DEVICE_IMPL(masked_col2im_forward_impl, col, mask_h_idx, mask_w_idx,\n im, height, width, channels);\n}\n\nvoid masked_im2col_forward(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w) {\n masked_im2col_forward_impl(im, mask_h_idx, mask_w_idx, col, kernel_h,\n kernel_w, pad_h, pad_w);\n}\n\nvoid masked_col2im_forward(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels) {\n masked_col2im_forward_impl(col, mask_h_idx, mask_w_idx, im, height, width,\n channels);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/masked_conv2d_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"masked_conv2d_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid masked_im2col_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n // im: (n, ic, h, w), kernel size (kh, kw)\n // kernel: (oc, ic * kh * kw), col: (kh * kw * ic, ow * oh)\n int kernel_h, kernel_w, pad_h, pad_w;\n SSAttrs(attr)\n .get(\"kernel_h\", kernel_h)\n .get(\"kernel_w\", kernel_w)\n .get(\"pad_h\", pad_h)\n .get(\"pad_w\", pad_w)\n .done();\n\n const auto& im = buildATensor(ctx, ins[0]);\n const auto& mask_h_idx = buildATensor(ctx, ins[1]);\n const auto& mask_w_idx = buildATensor(ctx, ins[2]);\n\n auto col = buildATensor(ctx, outs[0]);\n masked_im2col_forward_cuda(im, mask_h_idx, mask_w_idx, col, kernel_h,\n kernel_w, pad_h, pad_w);\n}\n\nvoid masked_col2im_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n // im: (n, ic, h, w), kernel size (kh, kw)\n // kernel: (oc, ic * kh * kh), col: (kh * kw * ic, ow * oh)\n int height, width, channels;\n SSAttrs(attr)\n .get(\"height\", height)\n .get(\"width\", width)\n .get(\"channels\", channels)\n .done();\n\n const auto& col = buildATensor(ctx, ins[0]);\n const auto& mask_h_idx = buildATensor(ctx, ins[1]);\n const auto& mask_w_idx = buildATensor(ctx, ins[2]);\n\n auto im = buildATensor(ctx, outs[0]);\n masked_col2im_forward_cuda(col, mask_h_idx, mask_w_idx, im, height, width,\n channels);\n}\n\nPARROTS_EXTENSION_REGISTER(masked_im2col_forward)\n .attr(\"kernel_h\")\n .attr(\"kernel_w\")\n .attr(\"pad_h\")\n .attr(\"pad_w\")\n .input(3)\n .output(1)\n .apply(masked_im2col_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(masked_col2im_forward)\n .attr(\"height\")\n .attr(\"width\")\n .attr(\"channels\")\n .input(3)\n .output(1)\n .apply(masked_col2im_forward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/masked_conv2d_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef MASKED_CONV2D_PYTORCH_H\n#define MASKED_CONV2D_PYTORCH_H\n#include \nusing namespace at;\n\nvoid masked_im2col_forward_cuda(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w);\n\nvoid masked_col2im_forward_cuda(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels);\n#endif // MASKED_CONV2D_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/min_area_polygons.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid min_area_polygons_impl(const Tensor pointsets, Tensor polygons) {\n DISPATCH_DEVICE_IMPL(min_area_polygons_impl, pointsets, polygons);\n}\n\nvoid min_area_polygons(const Tensor pointsets, Tensor polygons) {\n min_area_polygons_impl(pointsets, polygons);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/min_area_polygons_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"min_area_polygons_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid min_area_polygons_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto pointsets = buildATensor(ctx, ins[0]);\n\n auto polygons = buildATensor(ctx, outs[0]);\n min_area_polygons(pointsets, polygons);\n}\n\nPARROTS_EXTENSION_REGISTER(min_area_polygons)\n .input(1)\n .output(1)\n .apply(min_area_polygons_cuda_parrots)\n .done();\n\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/min_area_polygons_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef MIN_AREA_POLYGONS_PYTORCH_H\n#define MIN_AREA_POLYGONS_PYTORCH_H\n#include \nusing namespace at;\n\nvoid min_area_polygons(const Tensor pointsets, Tensor polygons);\n\n#endif // MIN_AREA_POLYGONS_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/modulated_deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid modulated_deformable_im2col_impl(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col) {\n DISPATCH_DEVICE_IMPL(modulated_deformable_im2col_impl, data_im, data_offset,\n data_mask, batch_size, channels, height_im, width_im,\n height_col, width_col, kernel_h, kernel_w, pad_h, pad_w,\n stride_h, stride_w, dilation_h, dilation_w,\n deformable_group, data_col);\n}\n\nvoid modulated_deformable_col2im_impl(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im) {\n DISPATCH_DEVICE_IMPL(modulated_deformable_col2im_impl, data_col, data_offset,\n data_mask, batch_size, channels, height_im, width_im,\n height_col, width_col, kernel_h, kernel_w, pad_h, pad_w,\n stride_h, stride_w, dilation_h, dilation_w,\n deformable_group, grad_im);\n}\n\nvoid modulated_deformable_col2im_coord_impl(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask) {\n DISPATCH_DEVICE_IMPL(modulated_deformable_col2im_coord_impl, data_col,\n data_im, data_offset, data_mask, batch_size, channels,\n height_im, width_im, height_col, width_col, kernel_h,\n kernel_w, pad_h, pad_w, stride_h, stride_w, dilation_h,\n dilation_w, deformable_group, grad_offset, grad_mask);\n}\n\nvoid modulated_deform_conv_forward(\n Tensor input, Tensor weight, Tensor bias, Tensor ones, Tensor offset,\n Tensor mask, Tensor output, Tensor columns, int kernel_h, int kernel_w,\n const int stride_h, const int stride_w, const int pad_h, const int pad_w,\n const int dilation_h, const int dilation_w, const int group,\n const int deformable_group, const bool with_bias) {\n at::DeviceGuard guard(input.device());\n\n const int batch = input.size(0);\n const int channels = input.size(1);\n const int height = input.size(2);\n const int width = input.size(3);\n\n const int channels_out = weight.size(0);\n const int channels_kernel = weight.size(1);\n const int kernel_h_ = weight.size(2);\n const int kernel_w_ = weight.size(3);\n\n if (kernel_h_ != kernel_h || kernel_w_ != kernel_w)\n AT_ERROR(\"Input shape and kernel shape won't match: (%d x %d vs %d x %d).\",\n kernel_h_, kernel_w, kernel_h_, kernel_w_);\n if (channels != channels_kernel * group)\n AT_ERROR(\"Input shape and kernel channels won't match: (%d vs %d).\",\n channels, channels_kernel * group);\n\n const int height_out =\n (height + 2 * pad_h - (dilation_h * (kernel_h - 1) + 1)) / stride_h + 1;\n const int width_out =\n (width + 2 * pad_w - (dilation_w * (kernel_w - 1) + 1)) / stride_w + 1;\n\n if (ones.ndimension() != 2 ||\n ones.size(0) * ones.size(1) < height_out * width_out) {\n // Resize plane and fill with ones...\n ones = at::ones({height_out, width_out}, input.options());\n }\n\n // resize output\n output = output.view({batch, channels_out, height_out, width_out}).zero_();\n // resize temporary columns\n columns =\n at::zeros({channels * kernel_h * kernel_w, 1 * height_out * width_out},\n input.options());\n\n output = output.view({output.size(0), group, output.size(1) / group,\n output.size(2), output.size(3)});\n\n for (int b = 0; b < batch; b++) {\n modulated_deformable_im2col_impl(\n input[b], offset[b], mask[b], 1, channels, height, width, height_out,\n width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, deformable_group, columns);\n\n // divide into group\n weight = weight.view({group, weight.size(0) / group, weight.size(1),\n weight.size(2), weight.size(3)});\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n\n for (int g = 0; g < group; g++) {\n output[b][g] = output[b][g]\n .flatten(1)\n .addmm_(weight[g].flatten(1), columns[g])\n .view_as(output[b][g]);\n }\n\n weight = weight.view({weight.size(0) * weight.size(1), weight.size(2),\n weight.size(3), weight.size(4)});\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n }\n\n output = output.view({output.size(0), output.size(1) * output.size(2),\n output.size(3), output.size(4)});\n\n if (with_bias) {\n output += bias.view({1, bias.size(0), 1, 1});\n }\n}\n\nvoid modulated_deform_conv_backward(\n Tensor input, Tensor weight, Tensor bias, Tensor ones, Tensor offset,\n Tensor mask, Tensor columns, Tensor grad_input, Tensor grad_weight,\n Tensor grad_bias, Tensor grad_offset, Tensor grad_mask, Tensor grad_output,\n int kernel_h, int kernel_w, int stride_h, int stride_w, int pad_h,\n int pad_w, int dilation_h, int dilation_w, int group, int deformable_group,\n const bool with_bias) {\n at::DeviceGuard guard(input.device());\n\n const int batch = input.size(0);\n const int channels = input.size(1);\n const int height = input.size(2);\n const int width = input.size(3);\n\n const int channels_kernel = weight.size(1);\n const int kernel_h_ = weight.size(2);\n const int kernel_w_ = weight.size(3);\n if (kernel_h_ != kernel_h || kernel_w_ != kernel_w)\n AT_ERROR(\"Input shape and kernel shape won't match: (%d x %d vs %d x %d).\",\n kernel_h_, kernel_w, kernel_h_, kernel_w_);\n if (channels != channels_kernel * group)\n AT_ERROR(\"Input shape and kernel channels won't match: (%d vs %d).\",\n channels, channels_kernel * group);\n\n const int height_out =\n (height + 2 * pad_h - (dilation_h * (kernel_h - 1) + 1)) / stride_h + 1;\n const int width_out =\n (width + 2 * pad_w - (dilation_w * (kernel_w - 1) + 1)) / stride_w + 1;\n\n if (ones.ndimension() != 2 ||\n ones.size(0) * ones.size(1) < height_out * width_out) {\n // Resize plane and fill with ones...\n ones = at::ones({height_out, width_out}, input.options());\n }\n\n grad_input = grad_input.view({batch, channels, height, width});\n columns = at::zeros({channels * kernel_h * kernel_w, height_out * width_out},\n input.options());\n\n grad_output =\n grad_output.view({grad_output.size(0), group, grad_output.size(1) / group,\n grad_output.size(2), grad_output.size(3)});\n\n for (int b = 0; b < batch; b++) {\n // divide int group\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n weight = weight.view({group, weight.size(0) / group, weight.size(1),\n weight.size(2), weight.size(3)});\n\n for (int g = 0; g < group; g++) {\n columns[g].addmm_(weight[g].flatten(1).transpose(0, 1),\n grad_output[b][g].flatten(1), 0.0f, 1.0f);\n }\n\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n weight = weight.view({weight.size(0) * weight.size(1), weight.size(2),\n weight.size(3), weight.size(4)});\n\n // gradient w.r.t. input coordinate data\n modulated_deformable_col2im_coord_impl(\n columns, input[b], offset[b], mask[b], 1, channels, height, width,\n height_out, width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, deformable_group, grad_offset[b],\n grad_mask[b]);\n // gradient w.r.t. input data\n modulated_deformable_col2im_impl(\n columns, offset[b], mask[b], 1, channels, height, width, height_out,\n width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, deformable_group, grad_input[b]);\n\n // gradient w.r.t. weight, dWeight should accumulate across the batch and\n // group\n modulated_deformable_im2col_impl(\n input[b], offset[b], mask[b], 1, channels, height, width, height_out,\n width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, deformable_group, columns);\n\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n grad_weight = grad_weight.view({group, grad_weight.size(0) / group,\n grad_weight.size(1), grad_weight.size(2),\n grad_weight.size(3)});\n if (with_bias)\n grad_bias = grad_bias.view({group, grad_bias.size(0) / group});\n\n for (int g = 0; g < group; g++) {\n grad_weight[g] =\n grad_weight[g]\n .flatten(1)\n .addmm_(grad_output[b][g].flatten(1), columns[g].transpose(0, 1))\n .view_as(grad_weight[g]);\n if (with_bias) {\n grad_bias[g] =\n grad_bias[g]\n .view({-1, 1})\n .addmm_(grad_output[b][g].flatten(1), ones.view({-1, 1}))\n .view(-1);\n }\n }\n\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n grad_weight = grad_weight.view({grad_weight.size(0) * grad_weight.size(1),\n grad_weight.size(2), grad_weight.size(3),\n grad_weight.size(4)});\n if (with_bias)\n grad_bias = grad_bias.view({grad_bias.size(0) * grad_bias.size(1)});\n }\n grad_output = grad_output.view({grad_output.size(0) * grad_output.size(1),\n grad_output.size(2), grad_output.size(3),\n grad_output.size(4)});\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/modulated_deform_conv_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"modulated_deform_conv_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid modulated_deform_conv_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_h, kernel_w, stride_h, stride_w, pad_h, pad_w, dilation_h,\n dilation_w, group, deformable_group, with_bias;\n SSAttrs(attr)\n .get(\"kernel_h\", kernel_h)\n .get(\"kernel_w\", kernel_w)\n .get(\"stride_h\", stride_h)\n .get(\"stride_w\", stride_w)\n .get(\"pad_h\", pad_h)\n .get(\"pad_w\", pad_w)\n .get(\"dilation_h\", dilation_h)\n .get(\"dilation_w\", dilation_w)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"with_bias\", with_bias)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& weight = buildATensor(ctx, ins[1]);\n const auto& bias = buildATensor(ctx, ins[2]);\n const auto& ones = buildATensor(ctx, ins[3]);\n const auto& offset = buildATensor(ctx, ins[4]);\n const auto& mask = buildATensor(ctx, ins[5]);\n\n auto output = buildATensor(ctx, outs[0]);\n auto columns = buildATensor(ctx, outs[1]);\n\n modulated_deform_conv_forward(input, weight, bias, ones, offset, mask, output,\n columns, kernel_h, kernel_w, stride_h, stride_w,\n pad_h, pad_w, dilation_h, dilation_w, group,\n deformable_group, with_bias);\n}\n\nvoid modulated_deform_conv_backward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_h, kernel_w, stride_h, stride_w, pad_h, pad_w, dilation_h,\n dilation_w, group, deformable_group, with_bias;\n SSAttrs(attr)\n .get(\"kernel_h\", kernel_h)\n .get(\"kernel_w\", kernel_w)\n .get(\"stride_h\", stride_h)\n .get(\"stride_w\", stride_w)\n .get(\"pad_h\", pad_h)\n .get(\"pad_w\", pad_w)\n .get(\"dilation_h\", dilation_h)\n .get(\"dilation_w\", dilation_w)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"with_bias\", with_bias)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& weight = buildATensor(ctx, ins[1]);\n const auto& bias = buildATensor(ctx, ins[2]);\n const auto& ones = buildATensor(ctx, ins[3]);\n const auto& offset = buildATensor(ctx, ins[4]);\n const auto& mask = buildATensor(ctx, ins[5]);\n\n auto columns = buildATensor(ctx, outs[0]);\n auto grad_input = buildATensor(ctx, outs[1]);\n auto grad_weight = buildATensor(ctx, outs[2]);\n auto grad_bias = buildATensor(ctx, outs[3]);\n auto grad_offset = buildATensor(ctx, outs[4]);\n auto grad_mask = buildATensor(ctx, outs[5]);\n auto grad_output = buildATensor(ctx, outs[6]);\n modulated_deform_conv_backward(\n input, weight, bias, ones, offset, mask, columns, grad_input, grad_weight,\n grad_bias, grad_offset, grad_mask, grad_output, kernel_h, kernel_w,\n stride_h, stride_w, pad_h, pad_w, dilation_h, dilation_w, group,\n deformable_group, with_bias);\n}\n#endif\n\nvoid modulated_deform_conv_forward_cpu_parrots(\n HostContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_h, kernel_w, stride_h, stride_w, pad_h, pad_w, dilation_h,\n dilation_w, group, deformable_group, with_bias;\n SSAttrs(attr)\n .get(\"kernel_h\", kernel_h)\n .get(\"kernel_w\", kernel_w)\n .get(\"stride_h\", stride_h)\n .get(\"stride_w\", stride_w)\n .get(\"pad_h\", pad_h)\n .get(\"pad_w\", pad_w)\n .get(\"dilation_h\", dilation_h)\n .get(\"dilation_w\", dilation_w)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"with_bias\", with_bias)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& weight = buildATensor(ctx, ins[1]);\n const auto& bias = buildATensor(ctx, ins[2]);\n const auto& ones = buildATensor(ctx, ins[3]);\n const auto& offset = buildATensor(ctx, ins[4]);\n const auto& mask = buildATensor(ctx, ins[5]);\n\n auto output = buildATensor(ctx, outs[0]);\n auto columns = buildATensor(ctx, outs[1]);\n\n modulated_deform_conv_forward(input, weight, bias, ones, offset, mask, output,\n columns, kernel_h, kernel_w, stride_h, stride_w,\n pad_h, pad_w, dilation_h, dilation_w, group,\n deformable_group, with_bias);\n}\n\nvoid modulated_deform_conv_backward_cpu_parrots(\n HostContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_h, kernel_w, stride_h, stride_w, pad_h, pad_w, dilation_h,\n dilation_w, group, deformable_group, with_bias;\n SSAttrs(attr)\n .get(\"kernel_h\", kernel_h)\n .get(\"kernel_w\", kernel_w)\n .get(\"stride_h\", stride_h)\n .get(\"stride_w\", stride_w)\n .get(\"pad_h\", pad_h)\n .get(\"pad_w\", pad_w)\n .get(\"dilation_h\", dilation_h)\n .get(\"dilation_w\", dilation_w)\n .get(\"group\", group)\n .get(\"deformable_group\", deformable_group)\n .get(\"with_bias\", with_bias)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& weight = buildATensor(ctx, ins[1]);\n const auto& bias = buildATensor(ctx, ins[2]);\n const auto& ones = buildATensor(ctx, ins[3]);\n const auto& offset = buildATensor(ctx, ins[4]);\n const auto& mask = buildATensor(ctx, ins[5]);\n\n auto columns = buildATensor(ctx, outs[0]);\n auto grad_input = buildATensor(ctx, outs[1]);\n auto grad_weight = buildATensor(ctx, outs[2]);\n auto grad_bias = buildATensor(ctx, outs[3]);\n auto grad_offset = buildATensor(ctx, outs[4]);\n auto grad_mask = buildATensor(ctx, outs[5]);\n auto grad_output = buildATensor(ctx, outs[6]);\n modulated_deform_conv_backward(\n input, weight, bias, ones, offset, mask, columns, grad_input, grad_weight,\n grad_bias, grad_offset, grad_mask, grad_output, kernel_h, kernel_w,\n stride_h, stride_w, pad_h, pad_w, dilation_h, dilation_w, group,\n deformable_group, with_bias);\n}\nPARROTS_EXTENSION_REGISTER(modulated_deform_conv_forward)\n .attr(\"kernel_h\")\n .attr(\"kernel_w\")\n .attr(\"stride_h\")\n .attr(\"stride_w\")\n .attr(\"pad_h\")\n .attr(\"pad_w\")\n .attr(\"dilation_h\")\n .attr(\"dilation_w\")\n .attr(\"group\")\n .attr(\"deformable_group\")\n .attr(\"with_bias\")\n .input(6)\n .output(2)\n .apply(modulated_deform_conv_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(modulated_deform_conv_forward_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(modulated_deform_conv_backward)\n .attr(\"kernel_h\")\n .attr(\"kernel_w\")\n .attr(\"stride_h\")\n .attr(\"stride_w\")\n .attr(\"pad_h\")\n .attr(\"pad_w\")\n .attr(\"dilation_h\")\n .attr(\"dilation_w\")\n .attr(\"group\")\n .attr(\"deformable_group\")\n .attr(\"with_bias\")\n .input(6)\n .output(7)\n .apply(modulated_deform_conv_backward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(modulated_deform_conv_backward_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/modulated_deform_conv_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef MODULATED_DEFORM_CONV_PYTORCH_H\n#define MODULATED_DEFORM_CONV_PYTORCH_H\n#include \nusing namespace at;\n\nvoid modulated_deform_conv_forward(\n Tensor input, Tensor weight, Tensor bias, Tensor ones, Tensor offset,\n Tensor mask, Tensor output, Tensor columns, int kernel_h, int kernel_w,\n const int stride_h, const int stride_w, const int pad_h, const int pad_w,\n const int dilation_h, const int dilation_w, const int group,\n const int deformable_group, const bool with_bias);\n\nvoid modulated_deform_conv_backward(\n Tensor input, Tensor weight, Tensor bias, Tensor ones, Tensor offset,\n Tensor mask, Tensor columns, Tensor grad_input, Tensor grad_weight,\n Tensor grad_bias, Tensor grad_offset, Tensor grad_mask, Tensor grad_output,\n int kernel_h, int kernel_w, int stride_h, int stride_w, int pad_h,\n int pad_w, int dilation_h, int dilation_w, int group, int deformable_group,\n const bool with_bias);\n#endif // MODULATED_DEFORM_CONV_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/ms_deform_attn.cpp", "content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from\n*https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nTensor ms_deform_attn_impl_forward(const Tensor &value,\n const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight,\n const int im2col_step) {\n return DISPATCH_DEVICE_IMPL(ms_deform_attn_impl_forward, value,\n spatial_shapes, level_start_index, sampling_loc,\n attn_weight, im2col_step);\n}\n\nvoid ms_deform_attn_impl_backward(\n const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index, const Tensor &sampling_loc,\n const Tensor &attn_weight, const Tensor &grad_output, Tensor &grad_value,\n Tensor &grad_sampling_loc, Tensor &grad_attn_weight,\n const int im2col_step) {\n DISPATCH_DEVICE_IMPL(ms_deform_attn_impl_backward, value, spatial_shapes,\n level_start_index, sampling_loc, attn_weight,\n grad_output, grad_value, grad_sampling_loc,\n grad_attn_weight, im2col_step);\n}\n\nTensor ms_deform_attn_forward(const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight,\n const int im2col_step) {\n at::DeviceGuard guard(value.device());\n return ms_deform_attn_impl_forward(value, spatial_shapes, level_start_index,\n sampling_loc, attn_weight, im2col_step);\n}\n\nvoid ms_deform_attn_backward(const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight,\n const Tensor &grad_output, Tensor &grad_value,\n Tensor &grad_sampling_loc,\n Tensor &grad_attn_weight, const int im2col_step) {\n at::DeviceGuard guard(value.device());\n ms_deform_attn_impl_backward(value, spatial_shapes, level_start_index,\n sampling_loc, attn_weight, grad_output,\n grad_value, grad_sampling_loc, grad_attn_weight,\n im2col_step);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/ms_deform_attn_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n\n#include \n#include \n#include \nusing namespace at;\nusing namespace parrots;\n\nTensor ms_deform_attn_forward(const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight, const int im2col_step);\n\nvoid ms_deform_attn_backward(const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight,\n const Tensor &grad_output, Tensor &grad_value,\n Tensor &grad_sampling_loc,\n Tensor &grad_attn_weight, const int im2col_step);\n\nvoid ms_deform_attn_forward_parrots(CudaContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n int im2col_step;\n SSAttrs(attr).get(\"im2col_step\", im2col_step).done();\n const auto &value = buildATensor(ctx, ins[0]);\n const auto &spatial_shapes = buildATensor(ctx, ins[1]);\n const auto &level_start_index = buildATensor(ctx, ins[2]);\n const auto &sampling_loc = buildATensor(ctx, ins[3]);\n const auto &attn_weight = buildATensor(ctx, ins[4]);\n auto out = ms_deform_attn_forward(value, spatial_shapes, level_start_index,\n sampling_loc, attn_weight, im2col_step);\n updateDArray(ctx, out, outs[0]);\n}\n\nvoid ms_deform_attn_backward_parrots(CudaContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n int im2col_step;\n SSAttrs(attr).get(\"im2col_step\", im2col_step).done();\n const auto &value = buildATensor(ctx, ins[0]);\n const auto &spatial_shapes = buildATensor(ctx, ins[1]);\n const auto &level_start_index = buildATensor(ctx, ins[2]);\n const auto &sampling_loc = buildATensor(ctx, ins[3]);\n const auto &attn_weight = buildATensor(ctx, ins[4]);\n const auto &grad_output = buildATensor(ctx, ins[5]);\n auto grad_value = buildATensor(ctx, outs[0]);\n auto grad_sampling_loc = buildATensor(ctx, outs[1]);\n auto grad_attn_weight = buildATensor(ctx, outs[2]);\n ms_deform_attn_backward(value, spatial_shapes, level_start_index,\n sampling_loc, attn_weight, grad_output, grad_value,\n grad_sampling_loc, grad_attn_weight, im2col_step);\n}\n\nPARROTS_EXTENSION_REGISTER(ms_deform_attn_forward)\n .attr(\"im2col_step\")\n .input(5)\n .output(1)\n .apply(ms_deform_attn_forward_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(ms_deform_attn_backward)\n .attr(\"im2col_step\")\n .input(6)\n .output(3)\n .apply(ms_deform_attn_backward_parrots)\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/nms.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nTensor nms_impl(Tensor boxes, Tensor scores, float iou_threshold, int offset) {\n return DISPATCH_DEVICE_IMPL(nms_impl, boxes, scores, iou_threshold, offset);\n}\n\nTensor softnms_impl(Tensor boxes, Tensor scores, Tensor dets,\n float iou_threshold, float sigma, float min_score,\n int method, int offset) {\n return DISPATCH_DEVICE_IMPL(softnms_impl, boxes, scores, dets, iou_threshold,\n sigma, min_score, method, offset);\n}\n\nstd::vector > nms_match_impl(Tensor dets,\n float iou_threshold) {\n return DISPATCH_DEVICE_IMPL(nms_match_impl, dets, iou_threshold);\n}\n\nTensor nms(Tensor boxes, Tensor scores, float iou_threshold, int offset) {\n return nms_impl(boxes, scores, iou_threshold, offset);\n}\n\nTensor softnms(Tensor boxes, Tensor scores, Tensor dets, float iou_threshold,\n float sigma, float min_score, int method, int offset) {\n return softnms_impl(boxes, scores, dets, iou_threshold, sigma, min_score,\n method, offset);\n}\n\nstd::vector > nms_match(Tensor dets, float iou_threshold) {\n return nms_match_impl(dets, iou_threshold);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/nms_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"nms_pytorch.h\"\n\nusing namespace parrots;\n\n// Tensor nms(Tensor boxes, Tensor scores, float iou_threshold, int offset);\ntemplate \nvoid nms_parrots(T& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float iou_threshold;\n int offset;\n SSAttrs(attr)\n .get(\"iou_threshold\", iou_threshold)\n .get(\"offset\", offset)\n .done();\n at::Tensor boxes, scores;\n boxes = buildATensor(ctx, ins[0]);\n scores = buildATensor(ctx, ins[1]);\n auto out = nms(boxes, scores, iou_threshold, offset);\n updateDArray(ctx, out, outs[0]);\n}\n\n/*Tensor softnms(Tensor boxes, Tensor scores, Tensor dets, float iou_threshold,\n * float sigma, float min_score, int method, int offset);*/\ntemplate \nvoid softnms_parrots(T& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float iou_threshold, sigma, min_score;\n int method, offset;\n SSAttrs(attr)\n .get(\"iou_threshold\", iou_threshold)\n .get(\"sigma\", sigma)\n .get(\"min_score\", min_score)\n .get(\"method\", method)\n .get(\"offset\", offset)\n .done();\n at::Tensor boxes, scores, dets;\n boxes = buildATensor(ctx, ins[0]);\n scores = buildATensor(ctx, ins[1]);\n dets = buildATensor(ctx, ins[2]);\n auto out = softnms(boxes, scores, dets, iou_threshold, sigma, min_score,\n method, offset);\n updateDArray(ctx, out, outs[0]);\n}\n\n// std::vector > nms_match(Tensor dets, float iou_threshold);\ntemplate \nvoid nms_match_parrots(T& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float iou_threshold;\n SSAttrs(attr).get(\"iou_threshold\", iou_threshold).done();\n at::Tensor dets;\n dets = buildATensor(ctx, ins[0]);\n auto out = nms_match(dets, iou_threshold);\n int n = out.size(), m = 0;\n for (int i = 0; i < n; ++i)\n if (m < out[i].size()) m = out[i].size();\n auto options = torch::TensorOptions().dtype(at::kInt);\n auto tensor = torch::zeros({n, m}, options);\n for (int i = 0; i < n; i++)\n tensor.slice(0, i, i + 1) =\n torch::from_blob(out[i].data(), {out[i].size()}, options);\n updateDArray(ctx, tensor, outs[0]);\n}\n\n/*Tensor nms_rotated(const Tensor dets, const Tensor scores, const Tensor order,\n * const Tensor dets_sorted, const float iou_threshold,\n * const int multi_label);*/\ntemplate \nvoid nms_rotated_parrots(T& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float iou_threshold;\n int multi_label;\n SSAttrs(attr)\n .get(\"iou_threshold\", iou_threshold)\n .get(\"multi_label\", multi_label)\n .done();\n at::Tensor dets, scores, order, dets_sorted;\n dets = buildATensor(ctx, ins[0]);\n scores = buildATensor(ctx, ins[1]);\n order = buildATensor(ctx, ins[2]);\n dets_sorted = buildATensor(ctx, ins[3]);\n auto out =\n nms_rotated(dets, scores, order, dets_sorted, iou_threshold, multi_label);\n updateDArray(ctx, out, outs[0]);\n}\n\nPARROTS_EXTENSION_REGISTER(nms)\n .attr(\"iou_threshold\")\n .attr(\"offset\")\n .input(2)\n .output(1)\n .apply(nms_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(nms_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(softnms)\n .attr(\"iou_threshold\")\n .attr(\"sigma\")\n .attr(\"min_score\")\n .attr(\"method\")\n .attr(\"offset\")\n .input(3)\n .output(1)\n .apply(softnms_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(softnms_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(nms_match)\n .attr(\"iou_threshold\")\n .input(1)\n .output(1)\n .apply(nms_match_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(nms_match_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(nms_rotated)\n .attr(\"multi_label\")\n .attr(\"iou_threshold\")\n .input(4)\n .output(1)\n .apply(nms_rotated_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(nms_rotated_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/nms_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef NMS_PYTORCH_H\n#define NMS_PYTORCH_H\n#include \n\nat::Tensor nms(at::Tensor boxes, at::Tensor scores, float iou_threshold,\n int offset);\n\nat::Tensor softnms(at::Tensor boxes, at::Tensor scores, at::Tensor dets,\n float iou_threshold, float sigma, float min_score,\n int method, int offset);\n\nstd::vector > nms_match(at::Tensor dets, float iou_threshold);\n\nat::Tensor nms_rotated(const at::Tensor dets, const at::Tensor scores,\n const at::Tensor order, const at::Tensor dets_sorted,\n const float iou_threshold, const int multi_label);\n#endif // NMS_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/nms_rotated.cpp", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/nms_rotated/nms_rotated.h\n#include \"pytorch_cpp_helper.hpp\"\n\nTensor nms_rotated_cpu(const Tensor dets, const Tensor scores,\n const float iou_threshold);\n\n#ifdef MMCV_WITH_CUDA\nTensor nms_rotated_cuda(const Tensor dets, const Tensor scores,\n const Tensor order, const Tensor dets_sorted,\n const float iou_threshold, const int multi_label);\n#endif\n\n// Interface for Python\n// inline is needed to prevent multiple function definitions when this header is\n// included by different cpps\nTensor nms_rotated(const Tensor dets, const Tensor scores, const Tensor order,\n const Tensor dets_sorted, const float iou_threshold,\n const int multi_label) {\n assert(dets.device().is_cuda() == scores.device().is_cuda());\n if (dets.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n return nms_rotated_cuda(dets, scores, order, dets_sorted, iou_threshold,\n multi_label);\n#else\n AT_ERROR(\"Not compiled with GPU support\");\n#endif\n }\n\n return nms_rotated_cpu(dets, scores, iou_threshold);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/pixel_group.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// It is modified from https://github.com/WenmuZhou/PAN.pytorch\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nstd::vector> pixel_group_impl(\n Tensor score, Tensor mask, Tensor embedding, Tensor kernel_label,\n Tensor kernel_contour, int kernel_region_num, float dis_threshold) {\n return DISPATCH_DEVICE_IMPL(pixel_group_impl, score, mask, embedding,\n kernel_label, kernel_contour, kernel_region_num,\n dis_threshold);\n}\n\nstd::vector> pixel_group(\n Tensor score, Tensor mask, Tensor embedding, Tensor kernel_label,\n Tensor kernel_contour, int kernel_region_num, float distance_threshold) {\n score = score.contiguous();\n mask = mask.contiguous();\n embedding = embedding.contiguous();\n kernel_label = kernel_label.contiguous();\n kernel_contour = kernel_contour.contiguous();\n\n return pixel_group_impl(score, mask, embedding, kernel_label, kernel_contour,\n kernel_region_num, distance_threshold);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/pixel_group_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"pixel_group_pytorch.h\"\n\nusing namespace parrots;\nusing namespace std;\n\ntemplate \nvoid pixel_group_parrots(T& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int kernel_region_num;\n float distance_threshold;\n SSAttrs(attr)\n .get(\"kernel_region_num\", kernel_region_num)\n .get(\"distance_threshold\", distance_threshold)\n .done();\n at::Tensor score;\n at::Tensor mask;\n at::Tensor embedding;\n at::Tensor kernel_label;\n at::Tensor kernel_contour;\n score = buildATensor(ctx, ins[0]);\n mask = buildATensor(ctx, ins[1]);\n embedding = buildATensor(ctx, ins[2]);\n kernel_label = buildATensor(ctx, ins[3]);\n kernel_contour = buildATensor(ctx, ins[4]);\n auto out = pixel_group(score, mask, embedding, kernel_label, kernel_contour,\n kernel_region_num, distance_threshold);\n int n = out.size();\n std::vector out_tensor;\n for (int i = 0; i < n; ++i) out_tensor.push_back(float(out[i].size()));\n for (int i = 0; i < n; ++i)\n out_tensor.insert(out_tensor.end(), out[i].begin(), out[i].end());\n auto options = torch::TensorOptions().dtype(at::kFloat);\n auto tensor = torch::zeros({1, out_tensor.size()}, options);\n tensor.slice(0, 0, 1) =\n torch::from_blob(out_tensor.data(), {out_tensor.size()}, options);\n updateDArray(ctx, tensor, outs[0]);\n}\n\nPARROTS_EXTENSION_REGISTER(pixel_group)\n .attr(\"kernel_region_num\")\n .attr(\"distance_threshold\")\n .input(5)\n .output(1)\n .apply(pixel_group_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(pixel_group_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/pixel_group_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef PIXEL_GROUP_PYTORCH_H\n#define PIXEL_GROUP_PYTORCH_H\n#include \nusing namespace at;\n\nstd::vector> pixel_group(\n Tensor score, Tensor mask, Tensor embedding, Tensor kernel_label,\n Tensor kernel_contour, int kernel_region_num, float distance_threshold);\n\n#endif // PIXEL_GROUP_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/points_in_boxes.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid points_in_boxes_part_forward_impl(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n DISPATCH_DEVICE_IMPL(points_in_boxes_part_forward_impl, batch_size, boxes_num,\n pts_num, boxes, pts, box_idx_of_points);\n}\n\nvoid points_in_boxes_all_forward_impl(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n DISPATCH_DEVICE_IMPL(points_in_boxes_all_forward_impl, batch_size, boxes_num,\n pts_num, boxes, pts, box_idx_of_points);\n}\n\nvoid points_in_boxes_part_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor box_idx_of_points_tensor) {\n // params boxes: (B, N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is the bottom center, each box params pts: (B, npoints, 3)\n // [x, y, z] in LiDAR coordinate params boxes_idx_of_points: (B, npoints),\n // default -1\n int batch_size = boxes_tensor.size(0);\n int boxes_num = boxes_tensor.size(1);\n int pts_num = pts_tensor.size(1);\n points_in_boxes_part_forward_impl(batch_size, boxes_num, pts_num,\n boxes_tensor, pts_tensor,\n box_idx_of_points_tensor);\n}\n\nvoid points_in_boxes_all_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor box_idx_of_points_tensor) {\n // params boxes: (B, N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is the bottom center. params pts: (B, npoints, 3) [x, y, z]\n // in LiDAR coordinate params boxes_idx_of_points: (B, npoints), default -1\n int batch_size = boxes_tensor.size(0);\n int boxes_num = boxes_tensor.size(1);\n int pts_num = pts_tensor.size(1);\n points_in_boxes_all_forward_impl(batch_size, boxes_num, pts_num, boxes_tensor,\n pts_tensor, box_idx_of_points_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/points_in_boxes_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"points_in_boxes_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid points_in_boxes_part_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto boxes_tensor = buildATensor(ctx, ins[0]);\n auto pts_tensor = buildATensor(ctx, ins[1]);\n\n auto box_idx_of_points_tensor = buildATensor(ctx, outs[0]);\n\n points_in_boxes_part_forward(boxes_tensor, pts_tensor,\n box_idx_of_points_tensor);\n}\n\nvoid points_in_boxes_all_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto boxes_tensor = buildATensor(ctx, ins[0]);\n auto pts_tensor = buildATensor(ctx, ins[1]);\n\n auto box_idx_of_points_tensor = buildATensor(ctx, outs[0]);\n\n points_in_boxes_all_forward(boxes_tensor, pts_tensor,\n box_idx_of_points_tensor);\n}\n\nPARROTS_EXTENSION_REGISTER(points_in_boxes_part_forward)\n .input(2)\n .output(1)\n .apply(points_in_boxes_part_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(points_in_boxes_all_forward)\n .input(2)\n .output(1)\n .apply(points_in_boxes_all_forward_cuda_parrots)\n .done();\n#endif\n\nvoid points_in_boxes_forward_cpu_parrots(HostContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto boxes_tensor = buildATensor(ctx, ins[0]);\n auto pts_tensor = buildATensor(ctx, ins[1]);\n\n auto pts_indices_tensor = buildATensor(ctx, outs[0]);\n\n points_in_boxes_cpu_forward(boxes_tensor, pts_tensor, pts_indices_tensor);\n}\n\nPARROTS_EXTENSION_REGISTER(points_in_boxes_cpu_forward)\n .input(2)\n .output(1)\n .apply(points_in_boxes_forward_cpu_parrots)\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/points_in_boxes_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef POINTS_IN_BOXES_PYTORCH_H\n#define POINTS_IN_BOXES_PYTORCH_H\n#include \nusing namespace at;\n\nvoid points_in_boxes_part_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor box_idx_of_points_tensor);\n\nvoid points_in_boxes_all_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor box_idx_of_points_tensor);\n\nvoid points_in_boxes_cpu_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor pts_indices_tensor);\n\n#endif // POINTS_IN_BOXES_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/points_in_polygons.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid points_in_polygons_forward_impl(const Tensor points, const Tensor polygons,\n Tensor output, const int rows,\n const int cols) {\n DISPATCH_DEVICE_IMPL(points_in_polygons_forward_impl, points, polygons,\n output, rows, cols);\n}\n\nvoid points_in_polygons_forward(Tensor points, Tensor polygons, Tensor output) {\n int rows = points.size(0);\n int cols = polygons.size(0);\n points_in_polygons_forward_impl(points, polygons, output, rows, cols);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/points_in_polygons_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"points_in_polygons_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid points_in_polygons_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto points = buildATensor(ctx, ins[0]);\n auto polygons = buildATensor(ctx, ins[1]);\n\n auto output = buildATensor(ctx, outs[0]);\n\n points_in_polygons_forward(points, polygons, output);\n}\n\nPARROTS_EXTENSION_REGISTER(points_in_polygons_forward)\n .input(2)\n .output(1)\n .apply(points_in_polygons_cuda_parrots)\n .done();\n\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/points_in_polygons_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef POINTS_IN_POLYGONS_PYTORCH_H\n#define POINTS_IN_POLYGONS_PYTORCH_H\n#include \nusing namespace at;\n\nvoid points_in_polygons_forward(Tensor points, Tensor polygons, Tensor output);\n\n#endif // POINTS_IN_POLYGONS_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/psamask.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/hszhao/semseg/blob/master/lib/psa/src\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid psamask_forward_impl(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask) {\n DISPATCH_DEVICE_IMPL(psamask_forward_impl, psa_type, input, output, num_,\n h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask);\n}\n\nvoid psamask_backward_impl(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask) {\n DISPATCH_DEVICE_IMPL(psamask_backward_impl, psa_type, grad_output, grad_input,\n num_, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask);\n}\n\nvoid psamask_forward(const Tensor input, Tensor output, const int psa_type,\n const int num_, const int h_feature, const int w_feature,\n const int h_mask, const int w_mask, const int half_h_mask,\n const int half_w_mask) {\n psamask_forward_impl(psa_type, input, output, num_, h_feature, w_feature,\n h_mask, w_mask, half_h_mask, half_w_mask);\n}\n\nvoid psamask_backward(Tensor grad_output, const Tensor grad_input,\n const int psa_type, const int num_, const int h_feature,\n const int w_feature, const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask) {\n psamask_backward_impl(psa_type, grad_output, grad_input, num_, h_feature,\n w_feature, h_mask, w_mask, half_h_mask, half_w_mask);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/psamask_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"psamask_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid psamask_forward_cuda_parrots(CudaContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n int psa_type, num_, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask;\n SSAttrs(attr)\n .get(\"psa_type\", psa_type)\n .get(\"num_\", num_)\n .get(\"h_feature\", h_feature)\n .get(\"w_feature\", w_feature)\n .get(\"h_mask\", h_mask)\n .get(\"w_mask\", w_mask)\n .get(\"half_h_mask\", half_h_mask)\n .get(\"half_w_mask\", half_w_mask)\n .done();\n const auto &input = buildATensor(ctx, ins[0]);\n auto output = buildATensor(ctx, outs[0]);\n psamask_forward_cuda(psa_type, input, output, num_, h_feature, w_feature,\n h_mask, w_mask, half_h_mask, half_w_mask);\n}\n\nvoid psamask_backward_cuda_parrots(CudaContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n int psa_type, num_, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask;\n SSAttrs(attr)\n .get(\"psa_type\", psa_type)\n .get(\"num_\", num_)\n .get(\"h_feature\", h_feature)\n .get(\"w_feature\", w_feature)\n .get(\"h_mask\", h_mask)\n .get(\"w_mask\", w_mask)\n .get(\"half_h_mask\", half_h_mask)\n .get(\"half_w_mask\", half_w_mask)\n .done();\n\n const auto &grad_output = buildATensor(ctx, ins[0]);\n auto grad_input = buildATensor(ctx, outs[0]);\n psamask_backward_cuda(psa_type, grad_output, grad_input, num_, h_feature,\n w_feature, h_mask, w_mask, half_h_mask, half_w_mask);\n}\n#endif\n\nvoid psamask_forward_cpu_parrots(HostContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n int psa_type, num_, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask;\n SSAttrs(attr)\n .get(\"psa_type\", psa_type)\n .get(\"num_\", num_)\n .get(\"h_feature\", h_feature)\n .get(\"w_feature\", w_feature)\n .get(\"h_mask\", h_mask)\n .get(\"w_mask\", w_mask)\n .get(\"half_h_mask\", half_h_mask)\n .get(\"half_w_mask\", half_w_mask)\n .done();\n const auto &input = buildATensor(ctx, ins[0]);\n auto output = buildATensor(ctx, outs[0]);\n psamask_forward_cpu(psa_type, input, output, num_, h_feature, w_feature,\n h_mask, w_mask, half_h_mask, half_w_mask);\n}\n\nvoid psamask_backward_cpu_parrots(HostContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n int psa_type, num_, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask;\n SSAttrs(attr)\n .get(\"psa_type\", psa_type)\n .get(\"num_\", num_)\n .get(\"h_feature\", h_feature)\n .get(\"w_feature\", w_feature)\n .get(\"h_mask\", h_mask)\n .get(\"w_mask\", w_mask)\n .get(\"half_h_mask\", half_h_mask)\n .get(\"half_w_mask\", half_w_mask)\n .done();\n\n const auto &grad_output = buildATensor(ctx, ins[0]);\n auto grad_input = buildATensor(ctx, outs[0]);\n psamask_backward_cpu(psa_type, grad_output, grad_input, num_, h_feature,\n w_feature, h_mask, w_mask, half_h_mask, half_w_mask);\n}\n\nPARROTS_EXTENSION_REGISTER(psamask_forward)\n .attr(\"psa_type\")\n .attr(\"num_\")\n .attr(\"h_feature\")\n .attr(\"w_feature\")\n .attr(\"h_mask\")\n .attr(\"w_mask\")\n .attr(\"half_h_mask\")\n .attr(\"half_w_mask\")\n .input(1)\n .output(1)\n .apply(psamask_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(psamask_forward_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(psamask_backward)\n .attr(\"psa_type\")\n .attr(\"num_\")\n .attr(\"h_feature\")\n .attr(\"w_feature\")\n .attr(\"h_mask\")\n .attr(\"w_mask\")\n .attr(\"half_h_mask\")\n .attr(\"half_w_mask\")\n .input(1)\n .output(1)\n .apply(psamask_backward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(psamask_backward_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/psamask_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef PSAMASK_PYTORCH_H\n#define PSAMASK_PYTORCH_H\n#include \nusing namespace at;\n\n#ifdef MMCV_WITH_CUDA\nvoid psamask_forward_cuda(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask);\n\nvoid psamask_backward_cuda(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask);\n#endif\nvoid psamask_forward_cpu(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask);\n\nvoid psamask_backward_cpu(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask);\n#endif // PSAMASK_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/riroi_align_rotated.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid riroi_align_rotated_forward_impl(Tensor features, Tensor rois,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n DISPATCH_DEVICE_IMPL(riroi_align_rotated_forward_impl, features, rois, output,\n pooled_height, pooled_width, spatial_scale, num_samples,\n num_orientations, clockwise);\n}\n\nvoid riroi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n DISPATCH_DEVICE_IMPL(riroi_align_rotated_backward_impl, top_grad, rois,\n bottom_grad, pooled_height, pooled_width, spatial_scale,\n num_samples, num_orientations, clockwise);\n}\n\nvoid riroi_align_rotated_forward(Tensor features, Tensor rois, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale, int num_samples,\n int num_orientations, bool clockwise) {\n riroi_align_rotated_forward_impl(features, rois, output, pooled_height,\n pooled_width, spatial_scale, num_samples,\n num_orientations, clockwise);\n}\n\nvoid riroi_align_rotated_backward(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n riroi_align_rotated_backward_impl(top_grad, rois, bottom_grad, pooled_height,\n pooled_width, spatial_scale, num_samples,\n num_orientations, clockwise);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/riroi_align_rotated_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"riroi_align_rotated_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid riroi_align_rotated_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n int sample_num;\n int num_orientations;\n bool clockwise;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"num_samples\", sample_num)\n .get(\"num_orientations\", num_orientations)\n .get(\"clockwise\", clockwise)\n .done();\n\n auto input = buildATensor(ctx, ins[0]);\n auto rois = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n riroi_align_rotated_forward(input, rois, output, pooled_height, pooled_width,\n spatial_scale, sample_num, num_orientations,\n clockwise);\n}\n\nvoid riroi_align_rotated_backward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n int sample_num;\n int num_orientations;\n bool clockwise;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"num_samples\", sample_num)\n .get(\"num_orientations\", num_orientations)\n .get(\"clockwise\", clockwise)\n .done();\n\n auto grad_output = buildATensor(ctx, ins[0]);\n auto rois = buildATensor(ctx, ins[1]);\n auto grad_input = buildATensor(ctx, outs[0]);\n riroi_align_rotated_backward(grad_output, rois, grad_input, pooled_height,\n pooled_width, spatial_scale, sample_num,\n num_orientations, clockwise);\n}\n\nPARROTS_EXTENSION_REGISTER(riroi_align_rotated_forward)\n .attr(\"pooled_height\")\n .attr(\"pooled_width\")\n .attr(\"spatial_scale\")\n .attr(\"num_samples\")\n .attr(\"num_orientations\")\n .attr(\"clockwise\")\n .input(2)\n .output(1)\n .apply(riroi_align_rotated_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(riroi_align_rotated_backward)\n .attr(\"pooled_height\")\n .attr(\"pooled_width\")\n .attr(\"spatial_scale\")\n .attr(\"num_samples\")\n .attr(\"num_orientations\")\n .attr(\"clockwise\")\n .input(2)\n .output(1)\n .apply(riroi_align_rotated_backward_cuda_parrots)\n .done();\n\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/riroi_align_rotated_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef RIROI_ALIGN_ROTATED_PYTORCH_H\n#define RIROI_ALIGN_ROTATED_PYTORCH_H\n#include \nusing namespace at;\n\nvoid riroi_align_rotated_forward(Tensor features, Tensor rois, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale, int num_samples,\n int num_orientations, bool clockwise);\n\nvoid riroi_align_rotated_backward(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise);\n\n#endif // RIROI_ALIGN_ROTATED_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_align.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n DISPATCH_DEVICE_IMPL(roi_align_forward_impl, input, rois, output, argmax_y,\n argmax_x, aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n DISPATCH_DEVICE_IMPL(roi_align_backward_impl, grad_output, rois, argmax_y,\n argmax_x, grad_input, aligned_height, aligned_width,\n spatial_scale, sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_forward(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned) {\n roi_align_forward_impl(input, rois, output, argmax_y, argmax_x,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_backward(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned) {\n roi_align_backward_impl(grad_output, rois, argmax_y, argmax_x, grad_input,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_align_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"roi_align_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid roi_align_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int aligned_height;\n int aligned_width;\n float spatial_scale;\n int sampling_ratio;\n int pool_mode;\n bool aligned;\n SSAttrs(attr)\n .get(\"aligned_height\", aligned_height)\n .get(\"aligned_width\", aligned_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sampling_ratio\", sampling_ratio)\n .get(\"pool_mode\", pool_mode)\n .get(\"aligned\", aligned)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n auto argmax_y = buildATensor(ctx, outs[1]);\n auto argmax_x = buildATensor(ctx, outs[2]);\n roi_align_forward_cuda(input, rois, output, argmax_y, argmax_x,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_backward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int aligned_height;\n int aligned_width;\n float spatial_scale;\n int sampling_ratio;\n int pool_mode;\n bool aligned;\n SSAttrs(attr)\n .get(\"aligned_height\", aligned_height)\n .get(\"aligned_width\", aligned_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sampling_ratio\", sampling_ratio)\n .get(\"pool_mode\", pool_mode)\n .get(\"aligned\", aligned)\n .done();\n\n const auto& grad_output = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n const auto& argmax_y = buildATensor(ctx, ins[2]);\n const auto& argmax_x = buildATensor(ctx, ins[3]);\n auto grad_input = buildATensor(ctx, outs[0]);\n roi_align_backward_cuda(grad_output, rois, argmax_y, argmax_x, grad_input,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n#endif\n\nvoid roi_align_forward_cpu_parrots(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int aligned_height;\n int aligned_width;\n float spatial_scale;\n int sampling_ratio;\n int pool_mode;\n bool aligned;\n SSAttrs(attr)\n .get(\"aligned_height\", aligned_height)\n .get(\"aligned_width\", aligned_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sampling_ratio\", sampling_ratio)\n .get(\"pool_mode\", pool_mode)\n .get(\"aligned\", aligned)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n auto argmax_y = buildATensor(ctx, outs[1]);\n auto argmax_x = buildATensor(ctx, outs[2]);\n roi_align_forward_cpu(input, rois, output, argmax_y, argmax_x, aligned_height,\n aligned_width, spatial_scale, sampling_ratio, pool_mode,\n aligned);\n}\n\nvoid roi_align_backward_cpu_parrots(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int aligned_height;\n int aligned_width;\n float spatial_scale;\n int sampling_ratio;\n int pool_mode;\n bool aligned;\n SSAttrs(attr)\n .get(\"aligned_height\", aligned_height)\n .get(\"aligned_width\", aligned_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sampling_ratio\", sampling_ratio)\n .get(\"pool_mode\", pool_mode)\n .get(\"aligned\", aligned)\n .done();\n\n const auto& grad_output = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n const auto& argmax_y = buildATensor(ctx, ins[2]);\n const auto& argmax_x = buildATensor(ctx, ins[3]);\n auto grad_input = buildATensor(ctx, outs[0]);\n roi_align_backward_cpu(grad_output, rois, argmax_y, argmax_x, grad_input,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\nPARROTS_EXTENSION_REGISTER(roi_align_forward)\n .attr(\"aligned_height\")\n .attr(\"aligned_width\")\n .attr(\"spatial_scale\")\n .attr(\"sampling_ratio\")\n .attr(\"pool_mode\")\n .attr(\"aligned\")\n .input(2)\n .output(3)\n .apply(roi_align_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(roi_align_forward_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(roi_align_backward)\n .attr(\"aligned_height\")\n .attr(\"aligned_width\")\n .attr(\"spatial_scale\")\n .attr(\"sampling_ratio\")\n .attr(\"pool_mode\")\n .attr(\"aligned\")\n .input(4)\n .output(1)\n .apply(roi_align_backward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(roi_align_backward_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_align_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROI_ALIGN_PYTORCH_H\n#define ROI_ALIGN_PYTORCH_H\n#include \nusing namespace at;\n\n#ifdef MMCV_WITH_CUDA\nvoid roi_align_forward_cuda(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nvoid roi_align_backward_cuda(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n#endif\n\nvoid roi_align_forward_cpu(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned);\n\nvoid roi_align_backward_cpu(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\n#endif // ROI_ALIGN_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_align_rotated.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roi_align_rotated_forward_impl(Tensor features, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sample_ratio,\n bool aligned, bool clockwise) {\n DISPATCH_DEVICE_IMPL(roi_align_rotated_forward_impl, features, rois, output,\n aligned_height, aligned_width, spatial_scale,\n sample_ratio, aligned, clockwise);\n}\n\nvoid roi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sample_ratio, bool aligned,\n bool clockwise) {\n DISPATCH_DEVICE_IMPL(roi_align_rotated_backward_impl, top_grad, rois,\n bottom_grad, aligned_height, aligned_width,\n spatial_scale, sample_ratio, aligned, clockwise);\n}\n\nvoid roi_align_rotated_forward(Tensor input, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n bool aligned, bool clockwise) {\n roi_align_rotated_forward_impl(input, rois, output, aligned_height,\n aligned_width, spatial_scale, sampling_ratio,\n aligned, clockwise);\n}\n\nvoid roi_align_rotated_backward(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, bool aligned,\n bool clockwise) {\n roi_align_rotated_backward_impl(top_grad, rois, bottom_grad, aligned_height,\n aligned_width, spatial_scale, sampling_ratio,\n aligned, clockwise);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_align_rotated_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"roi_align_rotated_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid roi_align_rotated_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n int sample_num;\n bool aligned;\n bool clockwise;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sample_num\", sample_num)\n .get(\"aligned\", aligned)\n .get(\"clockwise\", clockwise)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n roi_align_rotated_forward_cuda(input, rois, output, pooled_height,\n pooled_width, spatial_scale, sample_num,\n aligned, clockwise);\n}\n\nvoid roi_align_rotated_backward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n int sample_num;\n bool aligned;\n bool clockwise;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sample_num\", sample_num)\n .get(\"aligned\", aligned)\n .get(\"clockwise\", clockwise)\n .done();\n\n const auto& grad_output = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n auto grad_input = buildATensor(ctx, outs[0]);\n roi_align_rotated_backward_cuda(grad_output, rois, grad_input, pooled_height,\n pooled_width, spatial_scale, sample_num,\n aligned, clockwise);\n}\n#endif\n\nvoid roi_align_rotated_forward_cpu_parrots(HostContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n int sample_num;\n bool aligned;\n bool clockwise;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sample_num\", sample_num)\n .get(\"aligned\", aligned)\n .get(\"clockwise\", clockwise)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n roi_align_rotated_forward_cpu(input, rois, output, pooled_height,\n pooled_width, spatial_scale, sample_num,\n aligned, clockwise);\n}\n\nvoid roi_align_rotated_backward_cpu_parrots(HostContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n int sample_num;\n bool aligned;\n bool clockwise;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"sample_num\", sample_num)\n .get(\"aligned\", aligned)\n .get(\"clockwise\", clockwise)\n .done();\n\n const auto& grad_output = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n auto grad_input = buildATensor(ctx, outs[0]);\n roi_align_rotated_backward_cpu(grad_output, rois, grad_input, pooled_height,\n pooled_width, spatial_scale, sample_num,\n aligned, clockwise);\n}\n\nPARROTS_EXTENSION_REGISTER(roi_align_rotated_forward)\n .attr(\"pooled_height\")\n .attr(\"pooled_width\")\n .attr(\"spatial_scale\")\n .attr(\"sample_num\")\n .attr(\"aligned\")\n .attr(\"clockwise\")\n .input(2)\n .output(1)\n .apply(roi_align_rotated_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(roi_align_rotated_forward_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(roi_align_rotated_backward)\n .attr(\"pooled_height\")\n .attr(\"pooled_width\")\n .attr(\"spatial_scale\")\n .attr(\"sample_num\")\n .attr(\"aligned\")\n .attr(\"clockwise\")\n .input(2)\n .output(1)\n .apply(roi_align_rotated_backward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(roi_align_rotated_backward_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_align_rotated_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROI_ALIGN_ROTATED_PYTORCH_H\n#define ROI_ALIGN_ROTATED_PYTORCH_H\n#include \nusing namespace at;\n\n#ifdef MMCV_WITH_CUDA\nvoid roi_align_rotated_forward_cuda(Tensor features, Tensor rois, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale, int sample_num,\n bool aligned, bool clockwise);\n\nvoid roi_align_rotated_backward_cuda(Tensor grad_output, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int sample_num, bool aligned,\n bool clockwise);\n#endif\n\nvoid roi_align_rotated_forward_cpu(Tensor features, Tensor rois, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale, int sample_num,\n bool aligned, bool clockwise);\n\nvoid roi_align_rotated_backward_cpu(Tensor grad_output, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int sample_num, bool aligned,\n bool clockwise);\n\n#endif // ROI_ALIGN_ROTATED_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_pool.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roi_pool_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height, int pooled_width,\n float spatial_scale) {\n DISPATCH_DEVICE_IMPL(roi_pool_forward_impl, input, rois, output, argmax,\n pooled_height, pooled_width, spatial_scale);\n}\n\nvoid roi_pool_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height,\n int pooled_width, float spatial_scale) {\n DISPATCH_DEVICE_IMPL(roi_pool_backward_impl, grad_output, rois, argmax,\n grad_input, pooled_height, pooled_width, spatial_scale);\n}\n\nvoid roi_pool_forward(Tensor input, Tensor rois, Tensor output, Tensor argmax,\n int pooled_height, int pooled_width,\n float spatial_scale) {\n roi_pool_forward_impl(input, rois, output, argmax, pooled_height,\n pooled_width, spatial_scale);\n}\n\nvoid roi_pool_backward(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height, int pooled_width,\n float spatial_scale) {\n roi_pool_backward_impl(grad_output, rois, argmax, grad_input, pooled_height,\n pooled_width, spatial_scale);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_pool_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"roi_pool_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid roi_pool_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n auto argmax = buildATensor(ctx, outs[1]);\n roi_pool_forward_cuda(input, rois, output, argmax, pooled_height,\n pooled_width, spatial_scale);\n}\n\nvoid roi_pool_backward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pooled_height;\n int pooled_width;\n float spatial_scale;\n SSAttrs(attr)\n .get(\"pooled_height\", pooled_height)\n .get(\"pooled_width\", pooled_width)\n .get(\"spatial_scale\", spatial_scale)\n .done();\n\n const auto& grad_output = buildATensor(ctx, ins[0]);\n const auto& rois = buildATensor(ctx, ins[1]);\n const auto& argmax = buildATensor(ctx, ins[2]);\n auto grad_input = buildATensor(ctx, outs[0]);\n roi_pool_backward_cuda(grad_output, rois, argmax, grad_input, pooled_height,\n pooled_width, spatial_scale);\n}\n\nPARROTS_EXTENSION_REGISTER(roi_pool_forward)\n .attr(\"pooled_height\")\n .attr(\"pooled_width\")\n .attr(\"spatial_scale\")\n .input(2)\n .output(2)\n .apply(roi_pool_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(roi_pool_backward)\n .attr(\"pooled_height\")\n .attr(\"pooled_width\")\n .attr(\"spatial_scale\")\n .input(3)\n .output(1)\n .apply(roi_pool_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roi_pool_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROI_POOL_PYTORCH_H\n#define ROI_POOL_PYTORCH_H\n#include \nusing namespace at;\n\n#ifdef MMCV_WITH_CUDA\nvoid roi_pool_forward_cuda(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height, int pooled_width,\n float spatial_scale);\n\nvoid roi_pool_backward_cuda(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height,\n int pooled_width, float spatial_scale);\n#endif\n#endif // ROI_POOL_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roiaware_pool3d.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roiaware_pool3d_forward_impl(int boxes_num, int pts_num, int channels,\n int max_pts_each_voxel, int out_x, int out_y,\n int out_z, const Tensor rois,\n const Tensor pts, const Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method) {\n DISPATCH_DEVICE_IMPL(roiaware_pool3d_forward_impl, boxes_num, pts_num,\n channels, max_pts_each_voxel, out_x, out_y, out_z, rois,\n pts, pts_feature, argmax, pts_idx_of_voxels,\n pooled_features, pool_method);\n}\n\nvoid roiaware_pool3d_backward_impl(int boxes_num, int out_x, int out_y,\n int out_z, int channels,\n int max_pts_each_voxel,\n const Tensor pts_idx_of_voxels,\n const Tensor argmax, const Tensor grad_out,\n Tensor grad_in, int pool_method) {\n DISPATCH_DEVICE_IMPL(roiaware_pool3d_backward_impl, boxes_num, out_x, out_y,\n out_z, channels, max_pts_each_voxel, pts_idx_of_voxels,\n argmax, grad_out, grad_in, pool_method);\n}\n\nvoid roiaware_pool3d_forward(Tensor rois, Tensor pts, Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method) {\n // params rois: (N, 7) [x, y, z, x_size, y_size, z_size, ry] in LiDAR\n // coordinate\n // params pts: (npoints, 3) [x, y, z] in LiDAR coordinate\n // params pts_feature: (npoints, C)\n // params argmax: (N, out_x, out_y, out_z, C)\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel)\n // params pooled_features: (N, out_x, out_y, out_z, C)\n // params pool_method: 0: max_pool 1: avg_pool\n int boxes_num = rois.size(0);\n int pts_num = pts.size(0);\n int channels = pts_feature.size(1);\n int max_pts_each_voxel = pts_idx_of_voxels.size(4); // index 0 is the counter\n int out_x = pts_idx_of_voxels.size(1);\n int out_y = pts_idx_of_voxels.size(2);\n int out_z = pts_idx_of_voxels.size(3);\n assert((out_x < 256) && (out_y < 256) &&\n (out_z < 256)); // we encode index with 8bit\n\n roiaware_pool3d_forward_impl(boxes_num, pts_num, channels, max_pts_each_voxel,\n out_x, out_y, out_z, rois, pts, pts_feature,\n argmax, pts_idx_of_voxels, pooled_features,\n pool_method);\n}\n\nvoid roiaware_pool3d_backward(Tensor pts_idx_of_voxels, Tensor argmax,\n Tensor grad_out, Tensor grad_in,\n int pool_method) {\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel)\n // params argmax: (N, out_x, out_y, out_z, C)\n // params grad_out: (N, out_x, out_y, out_z, C)\n // params grad_in: (npoints, C), return value\n // params pool_method: 0: max_pool 1: avg_pool\n int boxes_num = pts_idx_of_voxels.size(0);\n int out_x = pts_idx_of_voxels.size(1);\n int out_y = pts_idx_of_voxels.size(2);\n int out_z = pts_idx_of_voxels.size(3);\n int max_pts_each_voxel = pts_idx_of_voxels.size(4); // index 0 is the counter\n int channels = grad_out.size(4);\n\n roiaware_pool3d_backward_impl(boxes_num, out_x, out_y, out_z, channels,\n max_pts_each_voxel, pts_idx_of_voxels, argmax,\n grad_out, grad_in, pool_method);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roiaware_pool3d_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"roiaware_pool3d_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid roiaware_pool3d_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pool_method;\n SSAttrs(attr).get(\"pool_method\", pool_method).done();\n auto rois = buildATensor(ctx, ins[0]);\n auto pts = buildATensor(ctx, ins[1]);\n auto pts_feature = buildATensor(ctx, ins[2]);\n\n auto argmax = buildATensor(ctx, outs[0]);\n auto pts_idx_of_voxels = buildATensor(ctx, outs[1]);\n auto pooled_features = buildATensor(ctx, outs[2]);\n\n roiaware_pool3d_forward(rois, pts, pts_feature, argmax, pts_idx_of_voxels,\n pooled_features, pool_method);\n}\n\nvoid roiaware_pool3d_backward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int pool_method;\n SSAttrs(attr).get(\"pool_method\", pool_method).done();\n auto pts_idx_of_voxels = buildATensor(ctx, ins[0]);\n auto argmax = buildATensor(ctx, ins[1]);\n auto grad_out = buildATensor(ctx, ins[2]);\n\n auto grad_in = buildATensor(ctx, outs[0]);\n\n roiaware_pool3d_backward(pts_idx_of_voxels, argmax, grad_out, grad_in,\n pool_method);\n}\n\nPARROTS_EXTENSION_REGISTER(roiaware_pool3d_forward)\n .attr(\"pool_method\")\n .input(3)\n .output(3)\n .apply(roiaware_pool3d_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(roiaware_pool3d_backward)\n .attr(\"pool_method\")\n .input(3)\n .output(1)\n .apply(roiaware_pool3d_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roiaware_pool3d_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROIAWARE_POOL3D_PYTORCH_H\n#define ROIAWARE_POOL3D_PYTORCH_H\n#include \nusing namespace at;\n\nvoid roiaware_pool3d_forward(Tensor rois, Tensor pts, Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method);\n\nvoid roiaware_pool3d_backward(Tensor pts_idx_of_voxels, Tensor argmax,\n Tensor grad_out, Tensor grad_in, int pool_method);\n\n#endif // ROIAWARE_POOL3D_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roipoint_pool3d.cpp", "content": "/*\nModified from\nhttps://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/ops/roipoint_pool3d/src/roipoint_pool3d.cpp\nPoint cloud feature pooling\nWritten by Shaoshuai Shi\nAll Rights Reserved 2018.\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roipoint_pool3d_forward_impl(int batch_size, int pts_num, int boxes_num,\n int feature_in_len, int sampled_pts_num,\n const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature,\n Tensor pooled_features,\n Tensor pooled_empty_flag) {\n DISPATCH_DEVICE_IMPL(roipoint_pool3d_forward_impl, batch_size, pts_num,\n boxes_num, feature_in_len, sampled_pts_num, xyz, boxes3d,\n pts_feature, pooled_features, pooled_empty_flag);\n}\n\nvoid roipoint_pool3d_forward(Tensor xyz, Tensor boxes3d, Tensor pts_feature,\n Tensor pooled_features, Tensor pooled_empty_flag) {\n // params xyz: (B, N, 3)\n // params boxes3d: (B, M, 7)\n // params pts_feature: (B, N, C)\n // params pooled_features: (B, M, 512, 3+C)\n // params pooled_empty_flag: (B, M)\n int batch_size = xyz.size(0);\n int pts_num = xyz.size(1);\n int boxes_num = boxes3d.size(1);\n int feature_in_len = pts_feature.size(2);\n int sampled_pts_num = pooled_features.size(2);\n\n roipoint_pool3d_forward_impl(batch_size, pts_num, boxes_num, feature_in_len,\n sampled_pts_num, xyz, boxes3d, pts_feature,\n pooled_features, pooled_empty_flag);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roipoint_pool3d_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"roipoint_pool3d_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid roipoint_pool3d_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n auto xyz = buildATensor(ctx, ins[0]);\n auto boxes3d = buildATensor(ctx, ins[1]);\n auto pts_feature = buildATensor(ctx, ins[2]);\n\n auto pooled_features = buildATensor(ctx, outs[0]);\n auto pooled_empty_flag = buildATensor(ctx, outs[1]);\n\n roipoint_pool3d_forward(xyz, boxes3d, pts_feature, pooled_features,\n pooled_empty_flag);\n}\n\nPARROTS_EXTENSION_REGISTER(roipoint_pool3d_forward)\n .input(3)\n .output(2)\n .apply(roipoint_pool3d_forward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/roipoint_pool3d_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROIPOINT_POOL3D_PYTORCH_H\n#define ROIPOINT_POOL3D_PYTORCH_H\n#include \nusing namespace at;\n\nvoid roipoint_pool3d_forward(Tensor xyz, Tensor boxes3d, Tensor pts_feature,\n Tensor pooled_features, Tensor pooled_empty_flag);\n\n#endif // ROIPOINT_POOL3D_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/rotated_feature_align.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/SJTU-Thinklab-Det/r3det-on-mmdetection/blob/master/mmdet/ops/fr/src/feature_refine_cuda.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid rotated_feature_align_forward_impl(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor output) {\n DISPATCH_DEVICE_IMPL(rotated_feature_align_forward_impl, features,\n best_bboxes, spatial_scale, points, output);\n}\n\nvoid rotated_feature_align_backward_impl(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor bottom_grad) {\n DISPATCH_DEVICE_IMPL(rotated_feature_align_backward_impl, top_grad,\n best_bboxes, spatial_scale, points, bottom_grad);\n}\n\nvoid rotated_feature_align_forward(const Tensor features,\n const Tensor best_bboxes, Tensor output,\n const float spatial_scale,\n const int points) {\n rotated_feature_align_forward_impl(features, best_bboxes, spatial_scale,\n points, output);\n}\n\nvoid rotated_feature_align_backward(const Tensor top_grad,\n const Tensor best_bboxes,\n Tensor bottom_grad,\n const float spatial_scale,\n const int points) {\n rotated_feature_align_backward_impl(top_grad, best_bboxes, spatial_scale,\n points, bottom_grad);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/rotated_feature_align_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"rotated_feature_align_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid rotated_feature_align_forward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float spatial_scale;\n int points;\n SSAttrs(attr)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"points\", points)\n .done();\n\n auto features = buildATensor(ctx, ins[0]);\n auto best_bboxes = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n rotated_feature_align_forward(features, best_bboxes, output, spatial_scale,\n points);\n}\n\nvoid rotated_feature_align_backward_cuda_parrots(\n CudaContext& ctx, const SSElement& attr, const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n float spatial_scale;\n int points;\n SSAttrs(attr)\n .get(\"spatial_scale\", spatial_scale)\n .get(\"points\", points)\n .done();\n\n auto grad_output = buildATensor(ctx, ins[0]);\n auto best_bboxes = buildATensor(ctx, ins[1]);\n auto grad_input = buildATensor(ctx, outs[0]);\n rotated_feature_align_backward(grad_output, best_bboxes, grad_input,\n spatial_scale, points);\n}\n\nPARROTS_EXTENSION_REGISTER(rotated_feature_align_forward)\n .attr(\"spatial_scale\")\n .attr(\"points\")\n .input(2)\n .output(1)\n .apply(rotated_feature_align_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(rotated_feature_align_backward)\n .attr(\"spatial_scale\")\n .attr(\"points\")\n .input(2)\n .output(1)\n .apply(rotated_feature_align_backward_cuda_parrots)\n .done();\n\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/rotated_feature_align_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef ROTATED_FEATURE_ALIGN_PYTORCH_H\n#define ROTATED_FEATURE_ALIGN_PYTORCH_H\n#include \nusing namespace at;\n\nvoid rotated_feature_align_forward(const Tensor features,\n const Tensor best_bboxes, Tensor output,\n const float spatial_scale, const int points);\n\nvoid rotated_feature_align_backward(const Tensor top_grad,\n const Tensor best_bboxes,\n Tensor bottom_grad,\n const float spatial_scale,\n const int points);\n\n#endif // ROTATED_FEATURE_ALIGN_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/sync_bn.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid sync_bn_forward_mean_impl(const Tensor input, Tensor mean) {\n DISPATCH_DEVICE_IMPL(sync_bn_forward_mean_impl, input, mean);\n}\n\nvoid sync_bn_forward_var_impl(const Tensor input, const Tensor mean,\n Tensor var) {\n DISPATCH_DEVICE_IMPL(sync_bn_forward_var_impl, input, mean, var);\n}\n\nvoid sync_bn_forward_output_impl(const Tensor input, const Tensor mean,\n const Tensor var, Tensor running_mean,\n Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size) {\n DISPATCH_DEVICE_IMPL(sync_bn_forward_output_impl, input, mean, var,\n running_mean, running_var, weight, bias, norm, std,\n output, eps, momentum, group_size);\n}\n\nvoid sync_bn_backward_param_impl(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias) {\n DISPATCH_DEVICE_IMPL(sync_bn_backward_param_impl, grad_output, norm,\n grad_weight, grad_bias);\n}\n\nvoid sync_bn_backward_data_impl(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias, const Tensor norm,\n const Tensor std, Tensor grad_input) {\n DISPATCH_DEVICE_IMPL(sync_bn_backward_data_impl, grad_output, weight,\n grad_weight, grad_bias, norm, std, grad_input);\n}\n\nvoid sync_bn_forward_mean(const Tensor input, Tensor mean) {\n sync_bn_forward_mean_impl(input, mean);\n}\n\nvoid sync_bn_forward_var(const Tensor input, const Tensor mean, Tensor var) {\n sync_bn_forward_var_impl(input, mean, var);\n}\n\nvoid sync_bn_forward_output(const Tensor input, const Tensor mean,\n const Tensor var, const Tensor weight,\n const Tensor bias, Tensor running_mean,\n Tensor running_var, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size) {\n sync_bn_forward_output_impl(input, mean, var, running_mean, running_var,\n weight, bias, norm, std, output, eps, momentum,\n group_size);\n}\n\nvoid sync_bn_backward_param(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias) {\n sync_bn_backward_param_impl(grad_output, norm, grad_weight, grad_bias);\n}\n\nvoid sync_bn_backward_data(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight, const Tensor grad_bias,\n const Tensor norm, const Tensor std,\n Tensor grad_input) {\n sync_bn_backward_data_impl(grad_output, weight, grad_weight, grad_bias, norm,\n std, grad_input);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/sync_bn_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"sync_bn_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid sync_bn_forward_mean_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n const auto& input = buildATensor(ctx, ins[0]);\n auto mean = buildATensor(ctx, outs[0]);\n sync_bn_forward_mean_cuda(input, mean);\n}\n\nvoid sync_bn_forward_var_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& mean = buildATensor(ctx, ins[1]);\n auto var = buildATensor(ctx, outs[0]);\n sync_bn_forward_var_cuda(input, mean, var);\n}\n\nvoid sync_bn_forward_output_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n size_t group_size;\n float eps, momentum;\n SSAttrs(attr)\n .get(\"eps\", eps)\n .get(\"momentum\", momentum)\n .get(\"group_size\", group_size)\n .done();\n\n const auto& input = buildATensor(ctx, ins[0]);\n const auto& mean = buildATensor(ctx, ins[1]);\n const auto& var = buildATensor(ctx, ins[2]);\n const auto& weight = buildATensor(ctx, ins[3]);\n const auto& bias = buildATensor(ctx, ins[4]);\n auto running_mean = buildATensor(ctx, outs[0]);\n auto running_var = buildATensor(ctx, outs[1]);\n auto norm = buildATensor(ctx, outs[2]);\n auto std = buildATensor(ctx, outs[3]);\n auto output = buildATensor(ctx, outs[4]);\n sync_bn_forward_output_cuda(input, mean, var, running_mean, running_var,\n weight, bias, norm, std, output, eps, momentum,\n group_size);\n}\n\nvoid sync_bn_backward_param_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n const auto& grad_output = buildATensor(ctx, ins[0]);\n const auto& norm = buildATensor(ctx, ins[1]);\n auto grad_weight = buildATensor(ctx, outs[0]);\n auto grad_bias = buildATensor(ctx, outs[1]);\n sync_bn_backward_param_cuda(grad_output, norm, grad_weight, grad_bias);\n}\n\nvoid sync_bn_backward_data_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n const auto& grad_output = buildATensor(ctx, ins[0]);\n const auto& weight = buildATensor(ctx, ins[1]);\n const auto& grad_weight = buildATensor(ctx, ins[2]);\n const auto& grad_bias = buildATensor(ctx, ins[3]);\n const auto& norm = buildATensor(ctx, ins[4]);\n const auto& std = buildATensor(ctx, ins[5]);\n auto grad_input = buildATensor(ctx, outs[0]);\n sync_bn_backward_data_cuda(grad_output, weight, grad_weight, grad_bias, norm,\n std, grad_input);\n}\n\nPARROTS_EXTENSION_REGISTER(sync_bn_forward_mean)\n .input(1)\n .output(1)\n .apply(sync_bn_forward_mean_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(sync_bn_forward_var)\n .input(2)\n .output(1)\n .apply(sync_bn_forward_var_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(sync_bn_forward_output)\n .attr(\"eps\")\n .attr(\"momentum\")\n .attr(\"group_size\")\n .input(5)\n .output(5)\n .apply(sync_bn_forward_output_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(sync_bn_backward_param)\n .input(2)\n .output(2)\n .apply(sync_bn_backward_param_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(sync_bn_backward_data)\n .input(6)\n .output(1)\n .apply(sync_bn_backward_data_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/sync_bn_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef SYNC_BN_PYTORCH_H\n#define SYNC_BN_PYTORCH_H\n#include \nusing namespace at;\n\nvoid sync_bn_forward_mean_cuda(const Tensor input, Tensor mean);\n\nvoid sync_bn_forward_var_cuda(const Tensor input, const Tensor mean,\n Tensor var);\n\nvoid sync_bn_forward_output_cuda(const Tensor input, const Tensor mean,\n const Tensor var, Tensor running_mean,\n Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size);\n\nvoid sync_bn_backward_param_cuda(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias);\n\nvoid sync_bn_backward_data_cuda(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias, const Tensor norm,\n const Tensor std, Tensor grad_input);\n#endif // SYNC_BN_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/three_interpolate.cpp", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/interpolate.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid three_interpolate_forward_impl(int b, int c, int m, int n,\n const Tensor points, const Tensor idx,\n const Tensor weight, Tensor out) {\n DISPATCH_DEVICE_IMPL(three_interpolate_forward_impl, b, c, m, n, points, idx,\n weight, out);\n}\n\nvoid three_interpolate_backward_impl(int b, int c, int n, int m,\n const Tensor grad_out, const Tensor idx,\n const Tensor weight, Tensor grad_points) {\n DISPATCH_DEVICE_IMPL(three_interpolate_backward_impl, b, c, n, m, grad_out,\n idx, weight, grad_points);\n}\n\nvoid three_interpolate_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor weight_tensor, Tensor out_tensor, int b,\n int c, int m, int n) {\n three_interpolate_forward_impl(b, c, m, n, points_tensor, idx_tensor,\n weight_tensor, out_tensor);\n}\n\nvoid three_interpolate_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor weight_tensor, Tensor grad_points_tensor,\n int b, int c, int n, int m) {\n three_interpolate_backward_impl(b, c, n, m, grad_out_tensor, idx_tensor,\n weight_tensor, grad_points_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/three_interpolate_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"three_interpolate_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid three_interpolate_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, c, m, n;\n SSAttrs(attr)\n .get(\"b\", b)\n .get(\"c\", c)\n .get(\"m\", m)\n .get(\"n\", n)\n .done();\n\n auto points_tensor = buildATensor(ctx, ins[0]);\n auto idx_tensor = buildATensor(ctx, ins[1]);\n auto weight_tensor = buildATensor(ctx, ins[2]);\n\n auto out_tensor = buildATensor(ctx, outs[0]);\n\n three_interpolate_forward(points_tensor, idx_tensor, weight_tensor,\n out_tensor, b, c, m, n);\n}\n\nvoid three_interpolate_backward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, c, n, m;\n SSAttrs(attr)\n .get(\"b\", b)\n .get(\"c\", c)\n .get(\"n\", n)\n .get(\"m\", m)\n .done();\n\n auto grad_out_tensor = buildATensor(ctx, ins[0]);\n auto idx_tensor = buildATensor(ctx, ins[1]);\n auto weight_tensor = buildATensor(ctx, ins[2]);\n\n auto grad_points_tensor = buildATensor(ctx, outs[0]);\n\n three_interpolate_backward(grad_out_tensor, idx_tensor, weight_tensor,\n grad_points_tensor, b, c, n, m);\n}\n\nPARROTS_EXTENSION_REGISTER(three_interpolate_forward)\n .attr(\"b\")\n .attr(\"c\")\n .attr(\"m\")\n .attr(\"n\")\n .input(3)\n .output(1)\n .apply(three_interpolate_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(three_interpolate_backward)\n .attr(\"b\")\n .attr(\"c\")\n .attr(\"n\")\n .attr(\"m\")\n .input(3)\n .output(1)\n .apply(three_interpolate_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/three_interpolate_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef THREE_INTERPOLATE_PYTORCH_H\n#define THREE_INTERPOLATE_PYTORCH_H\n#include \nusing namespace at;\n\nvoid three_interpolate_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor weight_tensor, Tensor out_tensor, int b,\n int c, int m, int n);\n\nvoid three_interpolate_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor weight_tensor, Tensor grad_points_tensor,\n int b, int c, int n, int m);\n#endif // THREE_INTERPOLATE_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/three_nn.cpp", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/interpolate.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid three_nn_forward_impl(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2, Tensor idx) {\n DISPATCH_DEVICE_IMPL(three_nn_forward_impl, b, n, m, unknown, known, dist2,\n idx);\n}\n\nvoid three_nn_forward(Tensor unknown_tensor, Tensor known_tensor,\n Tensor dist2_tensor, Tensor idx_tensor, int b, int n,\n int m) {\n three_nn_forward_impl(b, n, m, unknown_tensor, known_tensor, dist2_tensor,\n idx_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/three_nn_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"three_nn_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid three_nn_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int b, n, m;\n SSAttrs(attr).get(\"b\", b).get(\"n\", n).get(\"m\", m).done();\n\n auto unknown_tensor = buildATensor(ctx, ins[0]);\n auto known_tensor = buildATensor(ctx, ins[1]);\n\n auto dist2_tensor = buildATensor(ctx, outs[0]);\n auto idx_tensor = buildATensor(ctx, outs[1]);\n\n three_nn_forward(unknown_tensor, known_tensor, dist2_tensor, idx_tensor, b, n,\n m);\n}\n\nPARROTS_EXTENSION_REGISTER(three_nn_forward)\n .attr(\"b\")\n .attr(\"n\")\n .attr(\"m\")\n .input(2)\n .output(2)\n .apply(three_nn_forward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/three_nn_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef THREE_NN_PYTORCH_H\n#define THREE_NN_PYTORCH_H\n#include \nusing namespace at;\n\nvoid three_nn_forward(Tensor unknown_tensor, Tensor known_tensor,\n Tensor dist2_tensor, Tensor idx_tensor, int b, int n,\n int m);\n#endif // THREE_NN_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/tin_shift.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid tin_shift_forward_impl(Tensor input, Tensor shift, Tensor output) {\n DISPATCH_DEVICE_IMPL(tin_shift_forward_impl, input, shift, output);\n}\n\nvoid tin_shift_backward_impl(Tensor grad_output, Tensor shift,\n Tensor grad_input) {\n DISPATCH_DEVICE_IMPL(tin_shift_backward_impl, grad_output, shift, grad_input);\n}\n\nvoid tin_shift_forward(Tensor input, Tensor shift, Tensor output) {\n tin_shift_forward_impl(input, shift, output);\n}\n\nvoid tin_shift_backward(Tensor grad_output, Tensor shift, Tensor grad_input) {\n tin_shift_backward_impl(grad_output, shift, grad_input);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/tin_shift_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"tin_shift_pytorch.h\"\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid tin_shift_forward_cuda_parrots(CudaContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n const auto &input = buildATensor(ctx, ins[0]);\n const auto &shift = buildATensor(ctx, ins[1]);\n auto output = buildATensor(ctx, outs[0]);\n tin_shift_forward_cuda(input, shift, output);\n}\n\nvoid tin_shift_backward_cuda_parrots(CudaContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n const auto &grad_output = buildATensor(ctx, ins[0]);\n const auto &shift = buildATensor(ctx, ins[1]);\n auto grad_input = buildATensor(ctx, outs[0]);\n tin_shift_backward_cuda(grad_output, shift, grad_input);\n}\n\nPARROTS_EXTENSION_REGISTER(tin_shift_forward)\n .input(2)\n .output(1)\n .apply(tin_shift_forward_cuda_parrots)\n .done();\n\nPARROTS_EXTENSION_REGISTER(tin_shift_backward)\n .input(2)\n .output(1)\n .apply(tin_shift_backward_cuda_parrots)\n .done();\n#endif\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/tin_shift_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef TIN_SHIFT_PYTORCH_H\n#define TIN_SHIFT_PYTORCH_H\n#include \nusing namespace at;\n\nvoid tin_shift_forward_cuda(Tensor input, Tensor shift, Tensor output);\n\nvoid tin_shift_backward_cuda(Tensor grad_output, Tensor shift,\n Tensor grad_input);\n#endif // TIN_SHIFT_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/upfirdn2d.cpp", "content": "// Modified from\n// https://github.com/rosinality/stylegan2-pytorch/blob/master/op/upfirdn2d.cpp\n\n/*\nCopyright (c) 2021, NVIDIA Corporation. All rights reserved.\n\nNVIDIA Source Code License for StyleGAN2 with Adaptive Discriminator\nAugmentation (ADA)\n=======================================================================\n\n1. Definitions\n\n\"Licensor\" means any person or entity that distributes its Work.\n\n\"Software\" means the original work of authorship made available under\nthis License.\n\n\"Work\" means the Software and any additions to or derivative works of\nthe Software that are made available under this License.\n\nThe terms \"reproduce,\" \"reproduction,\" \"derivative works,\" and\n\"distribution\" have the meaning as provided under U.S. copyright law;\nprovided, however, that for the purposes of this License, derivative\nworks shall not include works that remain separable from, or merely\nlink (or bind by name) to the interfaces of, the Work.\n\nWorks, including the Software, are \"made available\" under this License\nby including in or with the Work either (a) a copyright notice\nreferencing the applicability of this License to the Work, or (b) a\ncopy of this License.\n\n2. License Grants\n\n 2.1 Copyright Grant. Subject to the terms and conditions of this\n License, each Licensor grants to you a perpetual, worldwide,\n non-exclusive, royalty-free, copyright license to reproduce,\n prepare derivative works of, publicly display, publicly perform,\n sublicense and distribute its Work and any resulting derivative\n works in any form.\n\n3. Limitations\n\n 3.1 Redistribution. You may reproduce or distribute the Work only\n if (a) you do so under this License, (b) you include a complete\n copy of this License with your distribution, and (c) you retain\n without modification any copyright, patent, trademark, or\n attribution notices that are present in the Work.\n\n 3.2 Derivative Works. You may specify that additional or different\n terms apply to the use, reproduction, and distribution of your\n derivative works of the Work (\"Your Terms\") only if (a) Your Terms\n provide that the use limitation in Section 3.3 applies to your\n derivative works, and (b) you identify the specific derivative\n works that are subject to Your Terms. Notwithstanding Your Terms,\n this License (including the redistribution requirements in Section\n 3.1) will continue to apply to the Work itself.\n\n 3.3 Use Limitation. The Work and any derivative works thereof only\n may be used or intended for use non-commercially. Notwithstanding\n the foregoing, NVIDIA and its affiliates may use the Work and any\n derivative works commercially. As used herein, \"non-commercially\"\n means for research or evaluation purposes only.\n\n 3.4 Patent Claims. If you bring or threaten to bring a patent claim\n against any Licensor (including any claim, cross-claim or\n counterclaim in a lawsuit) to enforce any patents that you allege\n are infringed by any Work, then your rights under this License from\n such Licensor (including the grant in Section 2.1) will terminate\n immediately.\n\n 3.5 Trademarks. This License does not grant any rights to use any\n Licensor’s or its affiliates’ names, logos, or trademarks, except\n as necessary to reproduce the notices described in this License.\n\n 3.6 Termination. If you violate any term of this License, then your\n rights under this License (including the grant in Section 2.1) will\n terminate immediately.\n\n4. Disclaimer of Warranty.\n\nTHE WORK IS PROVIDED \"AS IS\" WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR\nNON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER\nTHIS LICENSE.\n\n5. Limitation of Liability.\n\nEXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL\nTHEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE\nSHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT,\nINDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF\nOR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK\n(INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION,\nLOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER\nCOMMERCIAL DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF\nTHE POSSIBILITY OF SUCH DAMAGES.\n\n=======================================================================\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntorch::Tensor upfirdn2d_op_impl(const torch::Tensor& input,\n const torch::Tensor& kernel, int up_x, int up_y,\n int down_x, int down_y, int pad_x0, int pad_x1,\n int pad_y0, int pad_y1) {\n return DISPATCH_DEVICE_IMPL(upfirdn2d_op_impl, input, kernel, up_x, up_y,\n down_x, down_y, pad_x0, pad_x1, pad_y0, pad_y1);\n}\n\ntorch::Tensor upfirdn2d(const torch::Tensor& input, const torch::Tensor& kernel,\n int up_x, int up_y, int down_x, int down_y, int pad_x0,\n int pad_x1, int pad_y0, int pad_y1) {\n return upfirdn2d_op_impl(input, kernel, up_x, up_y, down_x, down_y, pad_x0,\n pad_x1, pad_y0, pad_y1);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/upfirdn2d_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n\n#include \n#include \n#include \nusing namespace at;\nusing namespace parrots;\n\ntorch::Tensor upfirdn2d(const Tensor &input, const Tensor &kernel, int up_x,\n int up_y, int down_x, int down_y, int pad_x0,\n int pad_x1, int pad_y0, int pad_y1);\n\nvoid upfirdn2d_parrots(CudaContext &ctx, const SSElement &attr,\n const OperatorBase::in_list_t &ins,\n OperatorBase::out_list_t &outs) {\n int up_x, up_y, down_x, down_y, pad_x0, pad_x1, pad_y0, pad_y1;\n const auto &input = buildATensor(ctx, ins[0]);\n const auto &kernel = buildATensor(ctx, ins[1]);\n SSAttrs(attr)\n .get(\"up_x\", up_x)\n .get(\"up_y\", up_y)\n .get(\"down_x\", down_x)\n .get(\"down_y\", down_y)\n .get(\"pad_x0\", pad_x0)\n .get(\"pad_x1\", pad_x1)\n .get(\"pad_y0\", pad_y0)\n .get(\"pad_y1\", pad_y1)\n .done();\n auto out = upfirdn2d(input, kernel, up_x, up_y, down_x, down_y, pad_x0,\n pad_x1, pad_y0, pad_y1);\n updateDArray(ctx, out, outs[0]);\n}\n\nPARROTS_EXTENSION_REGISTER(upfirdn2d)\n .attr(\"up_x\")\n .attr(\"up_y\")\n .attr(\"down_x\")\n .attr(\"down_y\")\n .attr(\"pad_x0\")\n .attr(\"pad_x1\")\n .attr(\"pad_y0\")\n .attr(\"pad_y1\")\n .input(2)\n .output(1)\n .apply(upfirdn2d_parrots)\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/voxelization.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nint hard_voxelize_forward_impl(const at::Tensor &points, at::Tensor &voxels,\n at::Tensor &coors,\n at::Tensor &num_points_per_voxel,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int max_points, const int max_voxels,\n const int NDim = 3) {\n return DISPATCH_DEVICE_IMPL(hard_voxelize_forward_impl, points, voxels, coors,\n num_points_per_voxel, voxel_size, coors_range,\n max_points, max_voxels, NDim);\n}\n\nvoid dynamic_voxelize_forward_impl(const at::Tensor &points, at::Tensor &coors,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int NDim = 3) {\n DISPATCH_DEVICE_IMPL(dynamic_voxelize_forward_impl, points, coors, voxel_size,\n coors_range, NDim);\n}\n\nvoid hard_voxelize_forward(const at::Tensor &points,\n const at::Tensor &voxel_size,\n const at::Tensor &coors_range, at::Tensor &voxels,\n at::Tensor &coors, at::Tensor &num_points_per_voxel,\n at::Tensor &voxel_num, const int max_points,\n const int max_voxels, const int NDim = 3) {\n int64_t *voxel_num_data = voxel_num.data_ptr();\n std::vector voxel_size_v(\n voxel_size.data_ptr(),\n voxel_size.data_ptr() + voxel_size.numel());\n std::vector coors_range_v(\n coors_range.data_ptr(),\n coors_range.data_ptr() + coors_range.numel());\n\n *voxel_num_data = hard_voxelize_forward_impl(\n points, voxels, coors, num_points_per_voxel, voxel_size_v, coors_range_v,\n max_points, max_voxels, NDim);\n}\n\nvoid dynamic_voxelize_forward(const at::Tensor &points,\n const at::Tensor &voxel_size,\n const at::Tensor &coors_range, at::Tensor &coors,\n const int NDim = 3) {\n std::vector voxel_size_v(\n voxel_size.data_ptr(),\n voxel_size.data_ptr() + voxel_size.numel());\n std::vector coors_range_v(\n coors_range.data_ptr(),\n coors_range.data_ptr() + coors_range.numel());\n dynamic_voxelize_forward_impl(points, coors, voxel_size_v, coors_range_v,\n NDim);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/voxelization_parrots.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n\n#include \"voxelization_pytorch.h\"\n\nusing namespace parrots;\n\n#ifdef MMCV_WITH_CUDA\nvoid hard_voxelize_forward_cuda_parrots(CudaContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int max_points, max_voxels, NDim;\n SSAttrs(attr)\n .get(\"max_points\", max_points)\n .get(\"max_voxels\", max_voxels)\n .get(\"NDim\", NDim)\n .done();\n const auto& points = buildATensor(ctx, ins[0]);\n const auto& voxel_size = buildATensor(ctx, ins[1]);\n const auto& coors_range = buildATensor(ctx, ins[2]);\n\n auto voxels = buildATensor(ctx, outs[0]);\n auto coors = buildATensor(ctx, outs[1]);\n auto num_points_per_voxel = buildATensor(ctx, outs[2]);\n auto voxel_num = buildATensor(ctx, outs[3]);\n\n hard_voxelize_forward(points, voxel_size, coors_range, voxels, coors,\n num_points_per_voxel, voxel_num, max_points, max_voxels,\n NDim);\n}\n\nvoid dynamic_voxelize_forward_cuda_parrots(CudaContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int NDim;\n SSAttrs(attr).get(\"NDim\", NDim).done();\n const auto& points = buildATensor(ctx, ins[0]);\n const auto& voxel_size = buildATensor(ctx, ins[1]);\n const auto& coors_range = buildATensor(ctx, ins[2]);\n\n auto coors = buildATensor(ctx, outs[0]);\n\n dynamic_voxelize_forward(points, voxel_size, coors_range, coors, NDim);\n}\n#endif\n\nvoid hard_voxelize_forward_cpu_parrots(HostContext& ctx, const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int max_points, max_voxels, NDim;\n SSAttrs(attr)\n .get(\"max_points\", max_points)\n .get(\"max_voxels\", max_voxels)\n .get(\"NDim\", NDim)\n .done();\n const auto& points = buildATensor(ctx, ins[0]);\n const auto& voxel_size = buildATensor(ctx, ins[1]);\n const auto& coors_range = buildATensor(ctx, ins[2]);\n\n auto voxels = buildATensor(ctx, outs[0]);\n auto coors = buildATensor(ctx, outs[1]);\n auto num_points_per_voxel = buildATensor(ctx, outs[2]);\n auto voxel_num = buildATensor(ctx, outs[3]);\n\n hard_voxelize_forward(points, voxel_size, coors_range, voxels, coors,\n num_points_per_voxel, voxel_num, max_points, max_voxels,\n NDim);\n}\n\nvoid dynamic_voxelize_forward_cpu_parrots(HostContext& ctx,\n const SSElement& attr,\n const OperatorBase::in_list_t& ins,\n OperatorBase::out_list_t& outs) {\n int NDim;\n SSAttrs(attr).get(\"NDim\", NDim).done();\n const auto& points = buildATensor(ctx, ins[0]);\n const auto& voxel_size = buildATensor(ctx, ins[1]);\n const auto& coors_range = buildATensor(ctx, ins[2]);\n\n auto coors = buildATensor(ctx, outs[0]);\n\n dynamic_voxelize_forward(points, voxel_size, coors_range, coors, NDim);\n}\n\nPARROTS_EXTENSION_REGISTER(hard_voxelize_forward)\n .attr(\"max_points\")\n .attr(\"max_voxels\")\n .attr(\"NDim\")\n .input(3)\n .output(4)\n .apply(hard_voxelize_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(hard_voxelize_forward_cuda_parrots)\n#endif\n .done();\n\nPARROTS_EXTENSION_REGISTER(dynamic_voxelize_forward)\n .attr(\"NDim\")\n .input(3)\n .output(1)\n .apply(dynamic_voxelize_forward_cpu_parrots)\n#ifdef MMCV_WITH_CUDA\n .apply(dynamic_voxelize_forward_cuda_parrots)\n#endif\n .done();\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/parrots/voxelization_pytorch.h", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef VOXELIZATION_PYTORCH_H\n#define VOXELIZATION_PYTORCH_H\n#include \nusing namespace at;\n\nvoid hard_voxelize_forward(const at::Tensor &points,\n const at::Tensor &voxel_size,\n const at::Tensor &coors_range, at::Tensor &voxels,\n at::Tensor &coors, at::Tensor &num_points_per_voxel,\n at::Tensor &voxel_num, const int max_points,\n const int max_voxels, const int NDim = 3);\n\nvoid dynamic_voxelize_forward(const at::Tensor &points,\n const at::Tensor &voxel_size,\n const at::Tensor &coors_range, at::Tensor &coors,\n const int NDim = 3);\n\n#endif // VOXELIZATION_PYTORCH_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/active_rotated_filter.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/csuhan/s2anet/blob/master/mmdet/ops/orn/src/ActiveRotatingFilter.h\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid active_rotated_filter_forward_impl(const Tensor input,\n const Tensor indices, Tensor output) {\n DISPATCH_DEVICE_IMPL(active_rotated_filter_forward_impl, input, indices,\n output);\n}\n\nvoid active_rotated_filter_backward_impl(const Tensor grad_out,\n const Tensor indices, Tensor grad_in) {\n DISPATCH_DEVICE_IMPL(active_rotated_filter_backward_impl, grad_out, indices,\n grad_in);\n}\n\nvoid active_rotated_filter_forward(const Tensor input, const Tensor indices,\n Tensor output) {\n active_rotated_filter_forward_impl(input, indices, output);\n}\n\nvoid active_rotated_filter_backward(const Tensor grad_out, const Tensor indices,\n Tensor grad_in) {\n active_rotated_filter_backward_impl(grad_out, indices, grad_in);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/assign_score_withk.cpp", "content": "// Modified from\n// https://github.com/CVMI-Lab/PAConv/tree/main/scene_seg/lib/paconv_lib/src/gpu\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid assign_score_withk_forward_impl(int B, int N0, int N1, int M, int K, int O,\n int aggregate, const Tensor& points,\n const Tensor& centers,\n const Tensor& scores,\n const Tensor& knn_idx, Tensor& output) {\n DISPATCH_DEVICE_IMPL(assign_score_withk_forward_impl, B, N0, N1, M, K, O,\n aggregate, points, centers, scores, knn_idx, output);\n}\n\nvoid assign_score_withk_backward_impl(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores) {\n DISPATCH_DEVICE_IMPL(assign_score_withk_backward_impl, B, N0, N1, M, K, O,\n aggregate, grad_out, points, centers, scores, knn_idx,\n grad_points, grad_centers, grad_scores);\n}\n\nvoid assign_score_withk_forward(const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx,\n Tensor& output, int B, int N0, int N1, int M,\n int K, int O, int aggregate) {\n assign_score_withk_forward_impl(B, N0, N1, M, K, O, aggregate, points,\n centers, scores, knn_idx, output);\n}\n\nvoid assign_score_withk_backward(const Tensor& grad_out, const Tensor& points,\n const Tensor& centers, const Tensor& scores,\n const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores,\n int B, int N0, int N1, int M, int K, int O,\n int aggregate) {\n assign_score_withk_backward_impl(B, N0, N1, M, K, O, aggregate, grad_out,\n points, centers, scores, knn_idx,\n grad_points, grad_centers, grad_scores);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/ball_query.cpp", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/ball_query.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid ball_query_forward_impl(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx) {\n DISPATCH_DEVICE_IMPL(ball_query_forward_impl, b, n, m, min_radius, max_radius,\n nsample, new_xyz, xyz, idx);\n}\n\nvoid ball_query_forward(Tensor new_xyz_tensor, Tensor xyz_tensor,\n Tensor idx_tensor, int b, int n, int m,\n float min_radius, float max_radius, int nsample) {\n ball_query_forward_impl(b, n, m, min_radius, max_radius, nsample,\n new_xyz_tensor, xyz_tensor, idx_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/bbox_overlaps.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid bbox_overlaps_impl(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset) {\n DISPATCH_DEVICE_IMPL(bbox_overlaps_impl, bboxes1, bboxes2, ious, mode,\n aligned, offset);\n}\n\nvoid bbox_overlaps(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset) {\n bbox_overlaps_impl(bboxes1, bboxes2, ious, mode, aligned, offset);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/border_align.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid border_align_forward_impl(const Tensor &input, const Tensor &boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size) {\n DISPATCH_DEVICE_IMPL(border_align_forward_impl, input, boxes, output,\n argmax_idx, pool_size);\n}\n\nvoid border_align_backward_impl(const Tensor &grad_output, const Tensor &boxes,\n const Tensor &argmax_idx, Tensor grad_input,\n const int pool_size) {\n DISPATCH_DEVICE_IMPL(border_align_backward_impl, grad_output, boxes,\n argmax_idx, grad_input, pool_size);\n}\n\nvoid border_align_forward(const Tensor &input, const Tensor &boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size) {\n border_align_forward_impl(input, boxes, output, argmax_idx, pool_size);\n}\n\nvoid border_align_backward(const Tensor &grad_output, const Tensor &boxes,\n const Tensor &argmax_idx, Tensor grad_input,\n const int pool_size) {\n border_align_backward_impl(grad_output, boxes, argmax_idx, grad_input,\n pool_size);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/box_iou_rotated.cpp", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated.h\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid box_iou_rotated_impl(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned) {\n DISPATCH_DEVICE_IMPL(box_iou_rotated_impl, boxes1, boxes2, ious, mode_flag,\n aligned);\n}\n\n// Interface for Python\n// inline is needed to prevent multiple function definitions when this header is\n// included by different cpps\nvoid box_iou_rotated(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned) {\n box_iou_rotated_impl(boxes1, boxes2, ious, mode_flag, aligned);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/carafe.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid carafe_forward_impl(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor) {\n DISPATCH_DEVICE_IMPL(carafe_forward_impl, features, masks, rfeatures, routput,\n rmasks, output, kernel_size, group_size, scale_factor);\n}\n\nvoid carafe_backward_impl(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad,\n Tensor bottom_grad, Tensor mask_grad, int kernel_size,\n int group_size, int scale_factor) {\n DISPATCH_DEVICE_IMPL(carafe_backward_impl, top_grad, rfeatures, masks,\n rtop_grad, rbottom_grad_hs, rbottom_grad, rmask_grad,\n bottom_grad, mask_grad, kernel_size, group_size,\n scale_factor);\n}\n\nvoid carafe_forward(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor) {\n carafe_forward_impl(features, masks, rfeatures, routput, rmasks, output,\n kernel_size, group_size, scale_factor);\n}\n\nvoid carafe_backward(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad, Tensor bottom_grad,\n Tensor mask_grad, int kernel_size, int group_size,\n int scale_factor) {\n carafe_backward_impl(top_grad, rfeatures, masks, rtop_grad, rbottom_grad_hs,\n rbottom_grad, rmask_grad, bottom_grad, mask_grad,\n kernel_size, group_size, scale_factor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/carafe_naive.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid carafe_naive_forward_impl(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size,\n int scale_factor) {\n DISPATCH_DEVICE_IMPL(carafe_naive_forward_impl, features, masks, output,\n kernel_size, group_size, scale_factor);\n}\n\nvoid carafe_naive_backward_impl(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size,\n int scale_factor) {\n DISPATCH_DEVICE_IMPL(carafe_naive_backward_impl, top_grad, features, masks,\n bottom_grad, mask_grad, kernel_size, group_size,\n scale_factor);\n}\n\nvoid carafe_naive_forward(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size, int scale_factor) {\n carafe_naive_forward_impl(features, masks, output, kernel_size, group_size,\n scale_factor);\n}\n\nvoid carafe_naive_backward(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size, int scale_factor) {\n carafe_naive_backward_impl(top_grad, features, masks, bottom_grad, mask_grad,\n kernel_size, group_size, scale_factor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/contour_expand.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// It is modified from https://github.com/whai362/PSENet\n#include \n#include \n\n#include \"pytorch_cpp_helper.hpp\"\n\nusing namespace std;\n\nclass Point2d {\n public:\n int x;\n int y;\n\n Point2d() : x(0), y(0) {}\n Point2d(int _x, int _y) : x(_x), y(_y) {}\n};\n\nvoid kernel_dilate(const uint8_t *data, IntArrayRef data_shape,\n const int *label_map, int &label_num, int &min_area,\n vector> &text_line) {\n std::vector area(label_num + 1);\n int kernel_num = data_shape[0];\n int height = data_shape[1];\n int width = data_shape[2];\n\n for (int x = 0; x < height; ++x) {\n for (int y = 0; y < width; ++y) {\n int label = label_map[x * width + y];\n if (label == 0) continue;\n area[label] += 1;\n }\n }\n\n queue queue, next_queue;\n for (int x = 0; x < height; ++x) {\n vector row(width);\n for (int y = 0; y < width; ++y) {\n int label = label_map[x * width + y];\n if (label == 0) continue;\n if (area[label] < min_area) continue;\n\n Point2d point(x, y);\n queue.push(point);\n row[y] = label;\n }\n text_line.emplace_back(row);\n }\n\n int dx[] = {-1, 1, 0, 0};\n int dy[] = {0, 0, -1, 1};\n vector kernel_step(kernel_num);\n std::for_each(kernel_step.begin(), kernel_step.end(),\n [=](int &k) { return k * height * width; });\n\n for (int kernel_id = kernel_num - 2; kernel_id >= 0; --kernel_id) {\n while (!queue.empty()) {\n Point2d point = queue.front();\n queue.pop();\n int x = point.x;\n int y = point.y;\n int label = text_line[x][y];\n\n bool is_edge = true;\n for (int d = 0; d < 4; ++d) {\n int tmp_x = x + dx[d];\n int tmp_y = y + dy[d];\n\n if (tmp_x < 0 || tmp_x >= height) continue;\n if (tmp_y < 0 || tmp_y >= width) continue;\n int kernel_value = data[kernel_step[kernel_id] + tmp_x * width + tmp_y];\n if (kernel_value == 0) continue;\n if (text_line[tmp_x][tmp_y] > 0) continue;\n\n Point2d point(tmp_x, tmp_y);\n queue.push(point);\n text_line[tmp_x][tmp_y] = label;\n is_edge = false;\n }\n\n if (is_edge) {\n next_queue.push(point);\n }\n }\n swap(queue, next_queue);\n }\n}\n\nstd::vector> contour_expand(Tensor kernel_mask,\n Tensor internal_kernel_label,\n int min_kernel_area,\n int kernel_num) {\n kernel_mask = kernel_mask.contiguous();\n internal_kernel_label = internal_kernel_label.contiguous();\n assert(kernel_mask.dim() == 3);\n assert(internal_kernel_label.dim() == 2);\n assert(kernel_mask.size(1) == internal_kernel_label.size(0));\n assert(kernel_mask.size(2) == internal_kernel_label.size(1));\n CHECK_CPU_INPUT(kernel_mask);\n CHECK_CPU_INPUT(internal_kernel_label);\n auto ptr_data = kernel_mask.data_ptr();\n IntArrayRef data_shape = kernel_mask.sizes();\n\n auto data_label_map = internal_kernel_label.data_ptr();\n vector> text_line;\n\n kernel_dilate(ptr_data, data_shape, data_label_map, kernel_num,\n min_kernel_area, text_line);\n\n return text_line;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/convex_iou.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// modified from\n// https://github.com/SDL-GuoZonghao/BeyondBoundingBox/tree/main/mmdet/ops/iou/src\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid convex_iou_impl(const Tensor pointsets, const Tensor polygons,\n Tensor ious) {\n DISPATCH_DEVICE_IMPL(convex_iou_impl, pointsets, polygons, ious);\n}\n\nvoid convex_iou(const Tensor pointsets, const Tensor polygons, Tensor ious) {\n convex_iou_impl(pointsets, polygons, ious);\n}\n\nvoid convex_giou_impl(const Tensor pointsets, const Tensor polygons,\n Tensor output) {\n DISPATCH_DEVICE_IMPL(convex_giou_impl, pointsets, polygons, output);\n}\n\nvoid convex_giou(const Tensor pointsets, const Tensor polygons, Tensor output) {\n convex_giou_impl(pointsets, polygons, output);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/corner_pool.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/princeton-vl/CornerNet-Lite/tree/master/core/models/py_utils/_cpools/src\n#include \"pytorch_cpp_helper.hpp\"\n\nTensor bottom_pool_forward(Tensor input) {\n // Initialize output\n Tensor output = at::zeros_like(input);\n // Get height\n int64_t height = input.size(2);\n output.copy_(input);\n\n for (int64_t ind = 1; ind < height; ind <<= 1) {\n Tensor max_temp = at::slice(output, 2, ind, height);\n Tensor cur_temp = at::slice(output, 2, ind, height).clone();\n Tensor next_temp = at::slice(output, 2, 0, height - ind).clone();\n at::max_out(max_temp, cur_temp, next_temp);\n }\n\n return output;\n}\n\nTensor bottom_pool_backward(Tensor input, Tensor grad_output) {\n auto output = at::zeros_like(input);\n\n int32_t batch = input.size(0);\n int32_t channel = input.size(1);\n int32_t height = input.size(2);\n int32_t width = input.size(3);\n\n auto max_val = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kFloat));\n auto max_ind = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kLong));\n\n auto input_temp = input.select(2, 0);\n max_val.copy_(input_temp);\n\n max_ind.fill_(0);\n\n auto output_temp = output.select(2, 0);\n auto grad_output_temp = grad_output.select(2, 0);\n output_temp.copy_(grad_output_temp);\n\n auto un_max_ind = max_ind.unsqueeze(2);\n auto gt_mask = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kBool));\n auto max_temp = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kFloat));\n for (int32_t ind = 0; ind < height - 1; ++ind) {\n input_temp = input.select(2, ind + 1);\n at::gt_out(gt_mask, input_temp, max_val);\n\n at::masked_select_out(max_temp, input_temp, gt_mask);\n max_val.masked_scatter_(gt_mask, max_temp);\n max_ind.masked_fill_(gt_mask, ind + 1);\n\n grad_output_temp = grad_output.select(2, ind + 1).unsqueeze(2);\n output.scatter_add_(2, un_max_ind, grad_output_temp);\n }\n\n return output;\n}\n\nTensor left_pool_forward(Tensor input) {\n // Initialize output\n Tensor output = at::zeros_like(input);\n // Get width\n int64_t width = input.size(3);\n output.copy_(input);\n\n for (int64_t ind = 1; ind < width; ind <<= 1) {\n Tensor max_temp = at::slice(output, 3, 0, width - ind);\n Tensor cur_temp = at::slice(output, 3, 0, width - ind).clone();\n Tensor next_temp = at::slice(output, 3, ind, width).clone();\n at::max_out(max_temp, cur_temp, next_temp);\n }\n\n return output;\n}\n\nTensor left_pool_backward(Tensor input, Tensor grad_output) {\n auto output = at::zeros_like(input);\n\n int32_t batch = input.size(0);\n int32_t channel = input.size(1);\n int32_t height = input.size(2);\n int32_t width = input.size(3);\n\n auto max_val = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kFloat));\n auto max_ind = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kLong));\n\n auto input_temp = input.select(3, width - 1);\n max_val.copy_(input_temp);\n\n max_ind.fill_(width - 1);\n\n auto output_temp = output.select(3, width - 1);\n auto grad_output_temp = grad_output.select(3, width - 1);\n output_temp.copy_(grad_output_temp);\n\n auto un_max_ind = max_ind.unsqueeze(3);\n auto gt_mask = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kBool));\n auto max_temp = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kFloat));\n for (int32_t ind = 1; ind < width; ++ind) {\n input_temp = input.select(3, width - ind - 1);\n at::gt_out(gt_mask, input_temp, max_val);\n\n at::masked_select_out(max_temp, input_temp, gt_mask);\n max_val.masked_scatter_(gt_mask, max_temp);\n max_ind.masked_fill_(gt_mask, width - ind - 1);\n\n grad_output_temp = grad_output.select(3, width - ind - 1).unsqueeze(3);\n output.scatter_add_(3, un_max_ind, grad_output_temp);\n }\n\n return output;\n}\n\nTensor right_pool_forward(Tensor input) {\n // Initialize output\n Tensor output = at::zeros_like(input);\n // Get width\n int64_t width = input.size(3);\n output.copy_(input);\n\n for (int64_t ind = 1; ind < width; ind <<= 1) {\n Tensor max_temp = at::slice(output, 3, ind, width);\n Tensor cur_temp = at::slice(output, 3, ind, width).clone();\n Tensor next_temp = at::slice(output, 3, 0, width - ind).clone();\n at::max_out(max_temp, cur_temp, next_temp);\n }\n\n return output;\n}\n\nTensor right_pool_backward(Tensor input, Tensor grad_output) {\n Tensor output = at::zeros_like(input);\n\n int32_t batch = input.size(0);\n int32_t channel = input.size(1);\n int32_t height = input.size(2);\n int32_t width = input.size(3);\n\n auto max_val = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kFloat));\n auto max_ind = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kLong));\n\n auto input_temp = input.select(3, 0);\n max_val.copy_(input_temp);\n\n max_ind.fill_(0);\n\n auto output_temp = output.select(3, 0);\n auto grad_output_temp = grad_output.select(3, 0);\n output_temp.copy_(grad_output_temp);\n\n auto un_max_ind = max_ind.unsqueeze(3);\n auto gt_mask = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kBool));\n auto max_temp = torch::zeros({batch, channel, height},\n at::device(at::kCUDA).dtype(at::kFloat));\n for (int32_t ind = 0; ind < width - 1; ++ind) {\n input_temp = input.select(3, ind + 1);\n at::gt_out(gt_mask, input_temp, max_val);\n\n at::masked_select_out(max_temp, input_temp, gt_mask);\n max_val.masked_scatter_(gt_mask, max_temp);\n max_ind.masked_fill_(gt_mask, ind + 1);\n\n grad_output_temp = grad_output.select(3, ind + 1).unsqueeze(3);\n output.scatter_add_(3, un_max_ind, grad_output_temp);\n }\n\n return output;\n}\n\nTensor top_pool_forward(Tensor input) {\n // Initialize output\n Tensor output = at::zeros_like(input);\n // Get height\n int64_t height = input.size(2);\n output.copy_(input);\n\n for (int64_t ind = 1; ind < height; ind <<= 1) {\n Tensor max_temp = at::slice(output, 2, 0, height - ind);\n Tensor cur_temp = at::slice(output, 2, 0, height - ind).clone();\n Tensor next_temp = at::slice(output, 2, ind, height).clone();\n at::max_out(max_temp, cur_temp, next_temp);\n }\n\n return output;\n}\n\nTensor top_pool_backward(Tensor input, Tensor grad_output) {\n auto output = at::zeros_like(input);\n\n int32_t batch = input.size(0);\n int32_t channel = input.size(1);\n int32_t height = input.size(2);\n int32_t width = input.size(3);\n\n auto max_val = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kFloat));\n auto max_ind = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kLong));\n\n auto input_temp = input.select(2, height - 1);\n max_val.copy_(input_temp);\n\n max_ind.fill_(height - 1);\n\n auto output_temp = output.select(2, height - 1);\n auto grad_output_temp = grad_output.select(2, height - 1);\n output_temp.copy_(grad_output_temp);\n\n auto un_max_ind = max_ind.unsqueeze(2);\n auto gt_mask = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kBool));\n auto max_temp = torch::zeros({batch, channel, width},\n at::device(at::kCUDA).dtype(at::kFloat));\n for (int32_t ind = 1; ind < height; ++ind) {\n input_temp = input.select(2, height - ind - 1);\n at::gt_out(gt_mask, input_temp, max_val);\n\n at::masked_select_out(max_temp, input_temp, gt_mask);\n max_val.masked_scatter_(gt_mask, max_temp);\n max_ind.masked_fill_(gt_mask, height - ind - 1);\n\n grad_output_temp = grad_output.select(2, height - ind - 1).unsqueeze(2);\n output.scatter_add_(2, un_max_ind, grad_output_temp);\n }\n\n return output;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/correlation.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#include \n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid correlation_forward_impl(Tensor input1, Tensor input2, Tensor output,\n int kH, int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW) {\n DISPATCH_DEVICE_IMPL(correlation_forward_impl, input1, input2, output, kH, kW,\n patchH, patchW, padH, padW, dilationH, dilationW,\n dilation_patchH, dilation_patchW, dH, dW);\n}\n\nvoid correlation_backward_impl(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW) {\n DISPATCH_DEVICE_IMPL(correlation_backward_impl, grad_output, input1, input2,\n grad_input1, grad_input2, kH, kW, patchH, patchW, padH,\n padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n\nvoid correlation_forward(Tensor input1, Tensor input2, Tensor output, int kH,\n int kW, int patchH, int patchW, int padH, int padW,\n int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW) {\n correlation_forward_impl(input1, input2, output, kH, kW, patchH, patchW, padH,\n padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n\nvoid correlation_backward(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH, int padW,\n int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW) {\n correlation_backward_impl(grad_output, input1, input2, grad_input1,\n grad_input2, kH, kW, patchH, patchW, padH, padW,\n dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/active_rotated_filter.cpp", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/csuhan/s2anet/blob/master/mmdet/ops/orn/src/cpu/ActiveRotatingFilter_cpu.cpp\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntemplate \nvoid active_rotated_filter_forward_cpu_kernel(\n const T* weightData, const int* indicesData, const int num_output_planes,\n const int num_input_planes, const int num_orientations, const int kH,\n const int kW, const int num_rotations, T* outputData) {\n const int nEntry = num_orientations * kH * kW;\n int i, j, l;\n int k;\n\n#pragma omp parallel for private(i, j, l, k)\n for (i = 0; i < num_output_planes; i++) {\n for (j = 0; j < num_input_planes; j++) {\n for (l = 0; l < nEntry; l++) {\n int weightIndex = i * num_input_planes * nEntry + j * nEntry + l;\n T val = *(weightData + weightIndex);\n for (k = 0; k < num_rotations; k++) {\n int index = (int)(*(indicesData + l * num_rotations + k)) - 1;\n T* target = outputData +\n i * (num_rotations * num_input_planes * nEntry) +\n k * (num_input_planes * nEntry) + j * (nEntry) + index;\n *target = val;\n }\n }\n }\n }\n}\n\ntemplate \nvoid active_rotated_filter_backward_cpu_kernel(\n const T* gradOutputData, const int* indicesData,\n const int num_output_planes, const int num_input_planes,\n const int num_orientations, const int kH, const int kW,\n const int num_rotations, T* gradInputData) {\n const int nEntry = num_orientations * kH * kW;\n int i, j, l;\n int k;\n\n#pragma omp parallel for private(i, j, l, k)\n for (i = 0; i < num_output_planes; i++) {\n for (j = 0; j < num_input_planes; j++) {\n for (l = 0; l < nEntry; l++) {\n int gradInputIndex = i * num_input_planes * nEntry + j * nEntry + l;\n T* val = gradInputData + gradInputIndex;\n *val = 0;\n for (k = 0; k < num_rotations; k++) {\n int index = (int)(*(indicesData + l * num_rotations + k)) - 1;\n const T* target =\n gradOutputData + i * (num_rotations * num_input_planes * nEntry) +\n k * (num_input_planes * nEntry) + j * (nEntry) + index;\n *val = *val + *target;\n }\n }\n }\n }\n}\n\nvoid ActiveRotatedFilterForwardCPULauncher(const Tensor input,\n const Tensor indices,\n Tensor output) {\n const int num_output_planes = input.size(0);\n const int num_input_planes = input.size(1);\n const int num_orientations = input.size(2);\n const int kH = input.size(3);\n const int kW = input.size(4);\n const int num_rotations = indices.size(3);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"active_rotated_filter_forward_cpu_kernel\", [&] {\n active_rotated_filter_forward_cpu_kernel(\n input.data_ptr(), indices.data_ptr(),\n num_output_planes, num_input_planes, num_orientations, kH, kW,\n num_rotations, output.data_ptr());\n });\n}\n\nvoid ActiveRotatedFilterBackwardCPULauncher(const Tensor grad_out,\n const Tensor indices,\n Tensor grad_in) {\n const int num_orientations = indices.size(0);\n const int kH = indices.size(1);\n const int kW = indices.size(2);\n const int num_rotations = indices.size(3);\n const int num_output_planes = grad_out.size(0) / num_rotations;\n const int num_input_planes = grad_out.size(1) / num_orientations;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_out.scalar_type(), \"active_rotated_filter_backward_cpu_kernel\", [&] {\n active_rotated_filter_backward_cpu_kernel(\n grad_out.data_ptr(), indices.data_ptr(),\n num_output_planes, num_input_planes, num_orientations, kH, kW,\n num_rotations, grad_in.data_ptr());\n });\n}\n\nvoid active_rotated_filter_forward_cpu(const Tensor input, const Tensor indices,\n Tensor output) {\n ActiveRotatedFilterForwardCPULauncher(input, indices, output);\n}\n\nvoid active_rotated_filter_backward_cpu(const Tensor grad_out,\n const Tensor indices, Tensor grad_in) {\n ActiveRotatedFilterBackwardCPULauncher(grad_out, indices, grad_in);\n}\n\nvoid active_rotated_filter_forward_impl(const Tensor input,\n const Tensor indices, Tensor output);\n\nvoid active_rotated_filter_backward_impl(const Tensor grad_out,\n const Tensor indices, Tensor grad_in);\n\nREGISTER_DEVICE_IMPL(active_rotated_filter_forward_impl, CPU,\n active_rotated_filter_forward_cpu);\nREGISTER_DEVICE_IMPL(active_rotated_filter_backward_impl, CPU,\n active_rotated_filter_backward_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/box_iou_rotated.cpp", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated_cpu.cpp\n#include \"box_iou_rotated_utils.hpp\"\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntemplate \nvoid box_iou_rotated_cpu_kernel(const Tensor boxes1, const Tensor boxes2,\n Tensor ious, const int mode_flag,\n const bool aligned) {\n int output_size = ious.numel();\n auto num_boxes1 = boxes1.size(0);\n auto num_boxes2 = boxes2.size(0);\n\n if (aligned) {\n for (int i = 0; i < output_size; i++) {\n ious[i] = single_box_iou_rotated(boxes1[i].data_ptr(),\n boxes2[i].data_ptr(), mode_flag);\n }\n } else {\n for (int i = 0; i < num_boxes1; i++) {\n for (int j = 0; j < num_boxes2; j++) {\n ious[i * num_boxes2 + j] = single_box_iou_rotated(\n boxes1[i].data_ptr(), boxes2[j].data_ptr(), mode_flag);\n }\n }\n }\n}\n\nvoid box_iou_rotated_cpu(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned) {\n box_iou_rotated_cpu_kernel(boxes1, boxes2, ious, mode_flag, aligned);\n}\n\nvoid box_iou_rotated_impl(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned);\nREGISTER_DEVICE_IMPL(box_iou_rotated_impl, CPU, box_iou_rotated_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntemplate \nT deformable_im2col_bilinear_cpu(const T *input, const int data_width,\n const int height, const int width, T h, T w) {\n if (h <= -1 || height <= h || w <= -1 || width <= w) {\n return 0;\n }\n\n int h_low = floor(h);\n int w_low = floor(w);\n int h_high = h_low + 1;\n int w_high = w_low + 1;\n\n T lh = h - h_low;\n T lw = w - w_low;\n T hh = 1 - lh, hw = 1 - lw;\n\n T v1 = 0;\n if (h_low >= 0 && w_low >= 0) v1 = input[h_low * data_width + w_low];\n T v2 = 0;\n if (h_low >= 0 && w_high <= width - 1)\n v2 = input[h_low * data_width + w_high];\n T v3 = 0;\n if (h_high <= height - 1 && w_low >= 0)\n v3 = input[h_high * data_width + w_low];\n T v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1)\n v4 = input[h_high * data_width + w_high];\n\n T w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n\n T val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n return val;\n}\n\ntemplate \nT get_gradient_weight_cpu(T argmax_h, T argmax_w, const int h, const int w,\n const int height, const int width) {\n if (argmax_h <= -1 || argmax_h >= height || argmax_w <= -1 ||\n argmax_w >= width) {\n // empty\n return 0;\n }\n\n int argmax_h_low = floor(argmax_h);\n int argmax_w_low = floor(argmax_w);\n int argmax_h_high = argmax_h_low + 1;\n int argmax_w_high = argmax_w_low + 1;\n\n T weight = 0;\n if (h == argmax_h_low && w == argmax_w_low)\n weight = (h + 1 - argmax_h) * (w + 1 - argmax_w);\n if (h == argmax_h_low && w == argmax_w_high)\n weight = (h + 1 - argmax_h) * (argmax_w + 1 - w);\n if (h == argmax_h_high && w == argmax_w_low)\n weight = (argmax_h + 1 - h) * (w + 1 - argmax_w);\n if (h == argmax_h_high && w == argmax_w_high)\n weight = (argmax_h + 1 - h) * (argmax_w + 1 - w);\n return weight;\n}\n\ntemplate \nT get_coordinate_weight_cpu(T argmax_h, T argmax_w, const int height,\n const int width, const T *im_data,\n const int data_width, const int bp_dir) {\n if (argmax_h <= -1 || argmax_h >= height || argmax_w <= -1 ||\n argmax_w >= width) {\n // empty\n return 0;\n }\n\n int argmax_h_low = floor(argmax_h);\n int argmax_w_low = floor(argmax_w);\n int argmax_h_high = argmax_h_low + 1;\n int argmax_w_high = argmax_w_low + 1;\n\n T weight = 0;\n\n if (bp_dir == 0) {\n if (argmax_h_low >= 0 && argmax_w_low >= 0)\n weight += -1 * (argmax_w_low + 1 - argmax_w) *\n im_data[argmax_h_low * data_width + argmax_w_low];\n if (argmax_h_low >= 0 && argmax_w_high <= width - 1)\n weight += -1 * (argmax_w - argmax_w_low) *\n im_data[argmax_h_low * data_width + argmax_w_high];\n if (argmax_h_high <= height - 1 && argmax_w_low >= 0)\n weight += (argmax_w_low + 1 - argmax_w) *\n im_data[argmax_h_high * data_width + argmax_w_low];\n if (argmax_h_high <= height - 1 && argmax_w_high <= width - 1)\n weight += (argmax_w - argmax_w_low) *\n im_data[argmax_h_high * data_width + argmax_w_high];\n } else if (bp_dir == 1) {\n if (argmax_h_low >= 0 && argmax_w_low >= 0)\n weight += -1 * (argmax_h_low + 1 - argmax_h) *\n im_data[argmax_h_low * data_width + argmax_w_low];\n if (argmax_h_low >= 0 && argmax_w_high <= width - 1)\n weight += (argmax_h_low + 1 - argmax_h) *\n im_data[argmax_h_low * data_width + argmax_w_high];\n if (argmax_h_high <= height - 1 && argmax_w_low >= 0)\n weight += -1 * (argmax_h - argmax_h_low) *\n im_data[argmax_h_high * data_width + argmax_w_low];\n if (argmax_h_high <= height - 1 && argmax_w_high <= width - 1)\n weight += (argmax_h - argmax_h_low) *\n im_data[argmax_h_high * data_width + argmax_w_high];\n }\n\n return weight;\n}\n\ntemplate \nvoid deformable_im2col_cpu_kernel(\n const int n, const T *data_im, const T *data_offset, const int height,\n const int width, const int kernel_h, const int kernel_w, const int pad_h,\n const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int num_channels, const int deformable_group, const int height_col,\n const int width_col, T *data_col) {\n for (int index = 0; index < n; index++) {\n // index index of output matrix\n const int w_col = index % width_col;\n const int h_col = (index / width_col) % height_col;\n const int b_col = (index / width_col / height_col) % batch_size;\n const int c_im = (index / width_col / height_col) / batch_size;\n const int c_col = c_im * kernel_h * kernel_w;\n\n // compute deformable group index\n const int deformable_group_index = c_im / channel_per_deformable_group;\n\n const int h_in = h_col * stride_h - pad_h;\n const int w_in = w_col * stride_w - pad_w;\n T *data_col_ptr =\n data_col +\n ((c_col * batch_size + b_col) * height_col + h_col) * width_col + w_col;\n const T *data_im_ptr =\n data_im + (b_col * num_channels + c_im) * height * width;\n const T *data_offset_ptr =\n data_offset + (b_col * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n\n for (int i = 0; i < kernel_h; ++i) {\n for (int j = 0; j < kernel_w; ++j) {\n const int data_offset_h_ptr =\n ((2 * (i * kernel_w + j)) * height_col + h_col) * width_col + w_col;\n const int data_offset_w_ptr =\n ((2 * (i * kernel_w + j) + 1) * height_col + h_col) * width_col +\n w_col;\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n T val = static_cast(0);\n const T h_im = h_in + i * dilation_h + offset_h;\n const T w_im = w_in + j * dilation_w + offset_w;\n if (h_im > -1 && w_im > -1 && h_im < height && w_im < width)\n val = deformable_im2col_bilinear_cpu(data_im_ptr, width, height,\n width, h_im, w_im);\n *data_col_ptr = val;\n data_col_ptr += batch_size * height_col * width_col;\n }\n }\n }\n}\n\ntemplate \nvoid deformable_col2im_cpu_kernel(\n const int n, const T *data_col, const T *data_offset, const int channels,\n const int height, const int width, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int deformable_group, const int height_col, const int width_col,\n T *grad_im) {\n for (int index = 0; index < n; index++) {\n const int j = (index / width_col / height_col / batch_size) % kernel_w;\n const int i =\n (index / width_col / height_col / batch_size / kernel_w) % kernel_h;\n const int c =\n index / width_col / height_col / batch_size / kernel_w / kernel_h;\n // compute the start and end of the output\n\n const int deformable_group_index = c / channel_per_deformable_group;\n\n int w_out = index % width_col;\n int h_out = (index / width_col) % height_col;\n int b = (index / width_col / height_col) % batch_size;\n int w_in = w_out * stride_w - pad_w;\n int h_in = h_out * stride_h - pad_h;\n\n const T *data_offset_ptr =\n data_offset + (b * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n const int data_offset_h_ptr =\n ((2 * (i * kernel_w + j)) * height_col + h_out) * width_col + w_out;\n const int data_offset_w_ptr =\n ((2 * (i * kernel_w + j) + 1) * height_col + h_out) * width_col + w_out;\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n const T cur_inv_h_data = h_in + i * dilation_h + offset_h;\n const T cur_inv_w_data = w_in + j * dilation_w + offset_w;\n\n const T cur_top_grad = data_col[index];\n const int cur_h = (int)cur_inv_h_data;\n const int cur_w = (int)cur_inv_w_data;\n for (int dy = -2; dy <= 2; dy++) {\n for (int dx = -2; dx <= 2; dx++) {\n if (cur_h + dy >= 0 && cur_h + dy < height && cur_w + dx >= 0 &&\n cur_w + dx < width && abs(cur_inv_h_data - (cur_h + dy)) < 1 &&\n abs(cur_inv_w_data - (cur_w + dx)) < 1) {\n int cur_bottom_grad_pos =\n ((b * channels + c) * height + cur_h + dy) * width + cur_w + dx;\n T weight =\n get_gradient_weight_cpu(cur_inv_h_data, cur_inv_w_data,\n cur_h + dy, cur_w + dx, height, width);\n *(grad_im + cur_bottom_grad_pos) += weight * cur_top_grad;\n }\n }\n }\n }\n}\n\ntemplate \nvoid deformable_col2im_coord_cpu_kernel(\n const int n, const T *data_col, const T *data_im, const T *data_offset,\n const int channels, const int height, const int width, const int kernel_h,\n const int kernel_w, const int pad_h, const int pad_w, const int stride_h,\n const int stride_w, const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int offset_channels, const int deformable_group, const int height_col,\n const int width_col, T *grad_offset) {\n for (int index = 0; index < n; index++) {\n T val = 0;\n int w = index % width_col;\n int h = (index / width_col) % height_col;\n int c = (index / width_col / height_col) % offset_channels;\n int b = (index / width_col / height_col) / offset_channels;\n // compute the start and end of the output\n\n const int deformable_group_index = c / (2 * kernel_h * kernel_w);\n const int col_step = kernel_h * kernel_w;\n int cnt = 0;\n const T *data_col_ptr = data_col + deformable_group_index *\n channel_per_deformable_group *\n batch_size * width_col * height_col;\n const T *data_im_ptr =\n data_im + (b * deformable_group + deformable_group_index) *\n channel_per_deformable_group / kernel_h / kernel_w *\n height * width;\n const T *data_offset_ptr =\n data_offset + (b * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n\n const int offset_c = c - deformable_group_index * 2 * kernel_h * kernel_w;\n\n for (int col_c = (offset_c / 2); col_c < channel_per_deformable_group;\n col_c += col_step) {\n const int col_pos =\n (((col_c * batch_size + b) * height_col) + h) * width_col + w;\n const int bp_dir = offset_c % 2;\n\n int j = (col_pos / width_col / height_col / batch_size) % kernel_w;\n int i =\n (col_pos / width_col / height_col / batch_size / kernel_w) % kernel_h;\n int w_out = col_pos % width_col;\n int h_out = (col_pos / width_col) % height_col;\n int w_in = w_out * stride_w - pad_w;\n int h_in = h_out * stride_h - pad_h;\n const int data_offset_h_ptr =\n (((2 * (i * kernel_w + j)) * height_col + h_out) * width_col + w_out);\n const int data_offset_w_ptr =\n (((2 * (i * kernel_w + j) + 1) * height_col + h_out) * width_col +\n w_out);\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n T inv_h = h_in + i * dilation_h + offset_h;\n T inv_w = w_in + j * dilation_w + offset_w;\n if (inv_h <= -1 || inv_w <= -1 || inv_h >= height || inv_w >= width)\n inv_h = inv_w = -2;\n const T weight = get_coordinate_weight_cpu(\n inv_h, inv_w, height, width, data_im_ptr + cnt * height * width,\n width, bp_dir);\n val += weight * data_col_ptr[col_pos];\n cnt += 1;\n }\n\n grad_offset[index] = val;\n }\n}\n\nvoid deformable_im2col_cpu(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col) {\n int height_col =\n (height + 2 * pad_h - (dilation_h * (ksize_h - 1) + 1)) / stride_h + 1;\n int width_col =\n (width + 2 * pad_w - (dilation_w * (ksize_w - 1) + 1)) / stride_w + 1;\n int num_kernels = channels * height_col * width_col * parallel_imgs;\n int channel_per_deformable_group = channels / deformable_group;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_im.scalar_type(), \"deformable_im2col_cpu\", [&] {\n deformable_im2col_cpu_kernel(\n num_kernels, data_im.data_ptr(),\n data_offset.data_ptr(), height, width, ksize_h, ksize_w,\n pad_h, pad_w, stride_h, stride_w, dilation_h, dilation_w,\n channel_per_deformable_group, parallel_imgs, channels,\n deformable_group, height_col, width_col,\n data_col.data_ptr());\n });\n}\n\nvoid deformable_col2im_cpu(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im) {\n // todo: make sure parallel_imgs is passed in correctly\n int height_col =\n (height + 2 * pad_h - (dilation_h * (ksize_h - 1) + 1)) / stride_h + 1;\n int width_col =\n (width + 2 * pad_w - (dilation_w * (ksize_w - 1) + 1)) / stride_w + 1;\n int num_kernels =\n channels * ksize_h * ksize_w * height_col * width_col * parallel_imgs;\n int channel_per_deformable_group = channels / deformable_group;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_col.scalar_type(), \"deformable_col2im_gpu\", ([&] {\n const scalar_t *data_col_ = data_col.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n scalar_t *grad_im_ = grad_im.data_ptr();\n\n deformable_col2im_cpu_kernel(\n num_kernels, data_col_, data_offset_, channels, height, width,\n ksize_h, ksize_w, pad_h, pad_w, stride_h, stride_w, dilation_h,\n dilation_w, channel_per_deformable_group, parallel_imgs,\n deformable_group, height_col, width_col, grad_im_);\n }));\n}\n\nvoid deformable_col2im_coord_cpu(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset) {\n int height_col =\n (height + 2 * pad_h - (dilation_h * (ksize_h - 1) + 1)) / stride_h + 1;\n int width_col =\n (width + 2 * pad_w - (dilation_w * (ksize_w - 1) + 1)) / stride_w + 1;\n int num_kernels = height_col * width_col * 2 * ksize_h * ksize_w *\n deformable_group * parallel_imgs;\n int channel_per_deformable_group =\n channels * ksize_h * ksize_w / deformable_group;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_col.scalar_type(), \"deformable_col2im_coord_cpu\", ([&] {\n const scalar_t *data_col_ = data_col.data_ptr();\n const scalar_t *data_im_ = data_im.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n scalar_t *grad_offset_ = grad_offset.data_ptr();\n\n deformable_col2im_coord_cpu_kernel(\n num_kernels, data_col_, data_im_, data_offset_, channels, height,\n width, ksize_h, ksize_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, channel_per_deformable_group, parallel_imgs,\n 2 * ksize_h * ksize_w * deformable_group, deformable_group,\n height_col, width_col, grad_offset_);\n }));\n}\n\nvoid deformable_im2col_impl(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col);\n\nvoid deformable_col2im_impl(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im);\n\nvoid deformable_col2im_coord_impl(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset);\n\nREGISTER_DEVICE_IMPL(deformable_im2col_impl, CPU, deformable_im2col_cpu);\nREGISTER_DEVICE_IMPL(deformable_col2im_impl, CPU, deformable_col2im_cpu);\nREGISTER_DEVICE_IMPL(deformable_col2im_coord_impl, CPU,\n deformable_col2im_coord_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/modulated_deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntemplate \nT dmcn_im2col_bilinear_cpu(const T *input, const int data_width,\n const int height, const int width, T h, T w) {\n int h_low = floorf(h);\n int w_low = floorf(w);\n int h_high = h_low + 1;\n int w_high = w_low + 1;\n\n T lh = h - h_low;\n T lw = w - w_low;\n T hh = 1 - lh, hw = 1 - lw;\n\n T v1 = 0;\n if (h_low >= 0 && w_low >= 0) v1 = input[h_low * data_width + w_low];\n T v2 = 0;\n if (h_low >= 0 && w_high <= width - 1)\n v2 = input[h_low * data_width + w_high];\n T v3 = 0;\n if (h_high <= height - 1 && w_low >= 0)\n v3 = input[h_high * data_width + w_low];\n T v4 = 0;\n if (h_high <= height - 1 && w_high <= width - 1)\n v4 = input[h_high * data_width + w_high];\n\n T w1 = hh * hw, w2 = hh * lw, w3 = lh * hw, w4 = lh * lw;\n\n T val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n return val;\n}\n\ntemplate \nT dmcn_get_gradient_weight_cpu(T argmax_h, T argmax_w, const int h, const int w,\n const int height, const int width) {\n if (argmax_h <= -1 || argmax_h >= height || argmax_w <= -1 ||\n argmax_w >= width) {\n // empty\n return 0;\n }\n\n int argmax_h_low = floorf(argmax_h);\n int argmax_w_low = floorf(argmax_w);\n int argmax_h_high = argmax_h_low + 1;\n int argmax_w_high = argmax_w_low + 1;\n\n T weight = 0;\n if (h == argmax_h_low && w == argmax_w_low)\n weight = (h + 1 - argmax_h) * (w + 1 - argmax_w);\n if (h == argmax_h_low && w == argmax_w_high)\n weight = (h + 1 - argmax_h) * (argmax_w + 1 - w);\n if (h == argmax_h_high && w == argmax_w_low)\n weight = (argmax_h + 1 - h) * (w + 1 - argmax_w);\n if (h == argmax_h_high && w == argmax_w_high)\n weight = (argmax_h + 1 - h) * (argmax_w + 1 - w);\n return weight;\n}\n\ntemplate \nT dmcn_get_coordinate_weight_cpu(T argmax_h, T argmax_w, const int height,\n const int width, const T *im_data,\n const int data_width, const int bp_dir) {\n if (argmax_h <= -1 || argmax_h >= height || argmax_w <= -1 ||\n argmax_w >= width) {\n // empty\n return 0;\n }\n\n int argmax_h_low = floorf(argmax_h);\n int argmax_w_low = floorf(argmax_w);\n int argmax_h_high = argmax_h_low + 1;\n int argmax_w_high = argmax_w_low + 1;\n\n T weight = 0;\n\n if (bp_dir == 0) {\n if (argmax_h_low >= 0 && argmax_w_low >= 0)\n weight += -1 * (argmax_w_low + 1 - argmax_w) *\n im_data[argmax_h_low * data_width + argmax_w_low];\n if (argmax_h_low >= 0 && argmax_w_high <= width - 1)\n weight += -1 * (argmax_w - argmax_w_low) *\n im_data[argmax_h_low * data_width + argmax_w_high];\n if (argmax_h_high <= height - 1 && argmax_w_low >= 0)\n weight += (argmax_w_low + 1 - argmax_w) *\n im_data[argmax_h_high * data_width + argmax_w_low];\n if (argmax_h_high <= height - 1 && argmax_w_high <= width - 1)\n weight += (argmax_w - argmax_w_low) *\n im_data[argmax_h_high * data_width + argmax_w_high];\n } else if (bp_dir == 1) {\n if (argmax_h_low >= 0 && argmax_w_low >= 0)\n weight += -1 * (argmax_h_low + 1 - argmax_h) *\n im_data[argmax_h_low * data_width + argmax_w_low];\n if (argmax_h_low >= 0 && argmax_w_high <= width - 1)\n weight += (argmax_h_low + 1 - argmax_h) *\n im_data[argmax_h_low * data_width + argmax_w_high];\n if (argmax_h_high <= height - 1 && argmax_w_low >= 0)\n weight += -1 * (argmax_h - argmax_h_low) *\n im_data[argmax_h_high * data_width + argmax_w_low];\n if (argmax_h_high <= height - 1 && argmax_w_high <= width - 1)\n weight += (argmax_h - argmax_h_low) *\n im_data[argmax_h_high * data_width + argmax_w_high];\n }\n\n return weight;\n}\n\ntemplate \nvoid modulated_deformable_im2col_cpu_kernel(\n const int n, const T *data_im, const T *data_offset, const T *data_mask,\n const int height, const int width, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int num_channels, const int deformable_group, const int height_col,\n const int width_col, T *data_col) {\n for (int index = 0; index < n; index++) {\n // index index of output matrix\n const int w_col = index % width_col;\n const int h_col = (index / width_col) % height_col;\n const int b_col = (index / width_col / height_col) % batch_size;\n const int c_im = (index / width_col / height_col) / batch_size;\n const int c_col = c_im * kernel_h * kernel_w;\n\n // compute deformable group index\n const int deformable_group_index = c_im / channel_per_deformable_group;\n\n const int h_in = h_col * stride_h - pad_h;\n const int w_in = w_col * stride_w - pad_w;\n\n T *data_col_ptr =\n data_col +\n ((c_col * batch_size + b_col) * height_col + h_col) * width_col + w_col;\n const T *data_im_ptr =\n data_im + (b_col * num_channels + c_im) * height * width;\n const T *data_offset_ptr =\n data_offset + (b_col * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n\n const T *data_mask_ptr =\n data_mask + (b_col * deformable_group + deformable_group_index) *\n kernel_h * kernel_w * height_col * width_col;\n\n for (int i = 0; i < kernel_h; ++i) {\n for (int j = 0; j < kernel_w; ++j) {\n const int data_offset_h_ptr =\n ((2 * (i * kernel_w + j)) * height_col + h_col) * width_col + w_col;\n const int data_offset_w_ptr =\n ((2 * (i * kernel_w + j) + 1) * height_col + h_col) * width_col +\n w_col;\n const int data_mask_hw_ptr =\n ((i * kernel_w + j) * height_col + h_col) * width_col + w_col;\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n const T mask = data_mask_ptr[data_mask_hw_ptr];\n T val = static_cast(0);\n const T h_im = h_in + i * dilation_h + offset_h;\n const T w_im = w_in + j * dilation_w + offset_w;\n if (h_im > -1 && w_im > -1 && h_im < height && w_im < width)\n val = dmcn_im2col_bilinear_cpu(data_im_ptr, width, height, width,\n h_im, w_im);\n *data_col_ptr = val * mask;\n data_col_ptr += batch_size * height_col * width_col;\n }\n }\n }\n}\n\ntemplate \nvoid modulated_deformable_col2im_cpu_kernel(\n const int n, const T *data_col, const T *data_offset, const T *data_mask,\n const int channels, const int height, const int width, const int kernel_h,\n const int kernel_w, const int pad_h, const int pad_w, const int stride_h,\n const int stride_w, const int dilation_h, const int dilation_w,\n const int channel_per_deformable_group, const int batch_size,\n const int deformable_group, const int height_col, const int width_col,\n T *grad_im) {\n for (int index = 0; index < n; index++) {\n const int j = (index / width_col / height_col / batch_size) % kernel_w;\n const int i =\n (index / width_col / height_col / batch_size / kernel_w) % kernel_h;\n const int c =\n index / width_col / height_col / batch_size / kernel_w / kernel_h;\n // compute the start and end of the output\n\n const int deformable_group_index = c / channel_per_deformable_group;\n\n int w_out = index % width_col;\n int h_out = (index / width_col) % height_col;\n int b = (index / width_col / height_col) % batch_size;\n int w_in = w_out * stride_w - pad_w;\n int h_in = h_out * stride_h - pad_h;\n\n const T *data_offset_ptr =\n data_offset + (b * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n const T *data_mask_ptr =\n data_mask + (b * deformable_group + deformable_group_index) * kernel_h *\n kernel_w * height_col * width_col;\n const int data_offset_h_ptr =\n ((2 * (i * kernel_w + j)) * height_col + h_out) * width_col + w_out;\n const int data_offset_w_ptr =\n ((2 * (i * kernel_w + j) + 1) * height_col + h_out) * width_col + w_out;\n const int data_mask_hw_ptr =\n ((i * kernel_w + j) * height_col + h_out) * width_col + w_out;\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n const T mask = data_mask_ptr[data_mask_hw_ptr];\n const T cur_inv_h_data = h_in + i * dilation_h + offset_h;\n const T cur_inv_w_data = w_in + j * dilation_w + offset_w;\n\n const T cur_top_grad = data_col[index] * mask;\n const int cur_h = (int)cur_inv_h_data;\n const int cur_w = (int)cur_inv_w_data;\n for (int dy = -2; dy <= 2; dy++) {\n for (int dx = -2; dx <= 2; dx++) {\n if (cur_h + dy >= 0 && cur_h + dy < height && cur_w + dx >= 0 &&\n cur_w + dx < width && abs(cur_inv_h_data - (cur_h + dy)) < 1 &&\n abs(cur_inv_w_data - (cur_w + dx)) < 1) {\n int cur_bottom_grad_pos =\n ((b * channels + c) * height + cur_h + dy) * width + cur_w + dx;\n T weight = dmcn_get_gradient_weight_cpu(cur_inv_h_data,\n cur_inv_w_data, cur_h + dy,\n cur_w + dx, height, width);\n *(grad_im + cur_bottom_grad_pos) += weight * cur_top_grad;\n }\n }\n }\n }\n}\n\ntemplate \nvoid modulated_deformable_col2im_coord_cpu_kernel(\n const int n, const T *data_col, const T *data_im, const T *data_offset,\n const T *data_mask, const int channels, const int height, const int width,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int channel_per_deformable_group,\n const int batch_size, const int offset_channels, const int deformable_group,\n const int height_col, const int width_col, T *grad_offset, T *grad_mask) {\n for (int index = 0; index < n; index++) {\n T val = 0, mval = 0;\n int w = index % width_col;\n int h = (index / width_col) % height_col;\n int c = (index / width_col / height_col) % offset_channels;\n int b = (index / width_col / height_col) / offset_channels;\n // compute the start and end of the output\n\n const int deformable_group_index = c / (2 * kernel_h * kernel_w);\n const int col_step = kernel_h * kernel_w;\n int cnt = 0;\n const T *data_col_ptr = data_col + deformable_group_index *\n channel_per_deformable_group *\n batch_size * width_col * height_col;\n const T *data_im_ptr =\n data_im + (b * deformable_group + deformable_group_index) *\n channel_per_deformable_group / kernel_h / kernel_w *\n height * width;\n const T *data_offset_ptr =\n data_offset + (b * deformable_group + deformable_group_index) * 2 *\n kernel_h * kernel_w * height_col * width_col;\n const T *data_mask_ptr =\n data_mask + (b * deformable_group + deformable_group_index) * kernel_h *\n kernel_w * height_col * width_col;\n\n const int offset_c = c - deformable_group_index * 2 * kernel_h * kernel_w;\n\n for (int col_c = (offset_c / 2); col_c < channel_per_deformable_group;\n col_c += col_step) {\n const int col_pos =\n (((col_c * batch_size + b) * height_col) + h) * width_col + w;\n const int bp_dir = offset_c % 2;\n\n int j = (col_pos / width_col / height_col / batch_size) % kernel_w;\n int i =\n (col_pos / width_col / height_col / batch_size / kernel_w) % kernel_h;\n int w_out = col_pos % width_col;\n int h_out = (col_pos / width_col) % height_col;\n int w_in = w_out * stride_w - pad_w;\n int h_in = h_out * stride_h - pad_h;\n const int data_offset_h_ptr =\n (((2 * (i * kernel_w + j)) * height_col + h_out) * width_col + w_out);\n const int data_offset_w_ptr =\n (((2 * (i * kernel_w + j) + 1) * height_col + h_out) * width_col +\n w_out);\n const int data_mask_hw_ptr =\n (((i * kernel_w + j) * height_col + h_out) * width_col + w_out);\n const T offset_h = data_offset_ptr[data_offset_h_ptr];\n const T offset_w = data_offset_ptr[data_offset_w_ptr];\n const T mask = data_mask_ptr[data_mask_hw_ptr];\n T inv_h = h_in + i * dilation_h + offset_h;\n T inv_w = w_in + j * dilation_w + offset_w;\n if (inv_h <= -1 || inv_w <= -1 || inv_h >= height || inv_w >= width)\n inv_h = inv_w = -2;\n else\n mval += data_col_ptr[col_pos] *\n dmcn_im2col_bilinear_cpu(data_im_ptr + cnt * height * width,\n width, height, width, inv_h, inv_w);\n const T weight = dmcn_get_coordinate_weight_cpu(\n inv_h, inv_w, height, width, data_im_ptr + cnt * height * width,\n width, bp_dir);\n val += weight * data_col_ptr[col_pos] * mask;\n cnt += 1;\n }\n // KERNEL_ASSIGN(grad_offset[index], offset_req, val);\n grad_offset[index] = val;\n if (offset_c % 2 == 0)\n // KERNEL_ASSIGN(grad_mask[(((b * deformable_group +\n // deformable_group_index) * kernel_h * kernel_w + offset_c / 2) *\n // height_col + h) * width_col + w], mask_req, mval);\n grad_mask[(((b * deformable_group + deformable_group_index) * kernel_h *\n kernel_w +\n offset_c / 2) *\n height_col +\n h) *\n width_col +\n w] = mval;\n }\n}\n\nvoid modulated_deformable_im2col_cpu(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col) {\n // num_axes should be smaller than block size\n const int channel_per_deformable_group = channels / deformable_group;\n const int num_kernels = channels * batch_size * height_col * width_col;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_im.scalar_type(), \"modulated_deformable_im2col_cpu\", ([&] {\n const scalar_t *data_im_ = data_im.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n const scalar_t *data_mask_ = data_mask.data_ptr();\n scalar_t *data_col_ = data_col.data_ptr();\n\n modulated_deformable_im2col_cpu_kernel(\n num_kernels, data_im_, data_offset_, data_mask_, height_im,\n width_im, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, channel_per_deformable_group, batch_size,\n channels, deformable_group, height_col, width_col, data_col_);\n }));\n}\n\nvoid modulated_deformable_col2im_cpu(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im) {\n const int channel_per_deformable_group = channels / deformable_group;\n const int num_kernels =\n channels * kernel_h * kernel_w * batch_size * height_col * width_col;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_col.scalar_type(), \"modulated_deformable_col2im_cpu\", ([&] {\n const scalar_t *data_col_ = data_col.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n const scalar_t *data_mask_ = data_mask.data_ptr();\n scalar_t *grad_im_ = grad_im.data_ptr();\n\n modulated_deformable_col2im_cpu_kernel(\n num_kernels, data_col_, data_offset_, data_mask_, channels,\n height_im, width_im, kernel_h, kernel_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, channel_per_deformable_group,\n batch_size, deformable_group, height_col, width_col, grad_im_);\n }));\n}\n\nvoid modulated_deformable_col2im_coord_cpu(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask) {\n const int num_kernels = batch_size * height_col * width_col * 2 * kernel_h *\n kernel_w * deformable_group;\n const int channel_per_deformable_group =\n channels * kernel_h * kernel_w / deformable_group;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_col.scalar_type(), \"modulated_deformable_col2im_coord_cpu\", ([&] {\n const scalar_t *data_col_ = data_col.data_ptr();\n const scalar_t *data_im_ = data_im.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n const scalar_t *data_mask_ = data_mask.data_ptr();\n scalar_t *grad_offset_ = grad_offset.data_ptr();\n scalar_t *grad_mask_ = grad_mask.data_ptr();\n\n modulated_deformable_col2im_coord_cpu_kernel(\n num_kernels, data_col_, data_im_, data_offset_, data_mask_,\n channels, height_im, width_im, kernel_h, kernel_w, pad_h, pad_w,\n stride_h, stride_w, dilation_h, dilation_w,\n channel_per_deformable_group, batch_size,\n 2 * kernel_h * kernel_w * deformable_group, deformable_group,\n height_col, width_col, grad_offset_, grad_mask_);\n }));\n}\n\nvoid modulated_deformable_im2col_impl(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col);\n\nvoid modulated_deformable_col2im_impl(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im);\n\nvoid modulated_deformable_col2im_coord_impl(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask);\n\nREGISTER_DEVICE_IMPL(modulated_deformable_im2col_impl, CPU,\n modulated_deformable_im2col_cpu);\nREGISTER_DEVICE_IMPL(modulated_deformable_col2im_impl, CPU,\n modulated_deformable_col2im_cpu);\nREGISTER_DEVICE_IMPL(modulated_deformable_col2im_coord_impl, CPU,\n modulated_deformable_col2im_coord_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/nms.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nTensor nms_cpu(Tensor boxes, Tensor scores, float iou_threshold, int offset) {\n if (boxes.numel() == 0) {\n return at::empty({0}, boxes.options().dtype(at::kLong));\n }\n auto x1_t = boxes.select(1, 0).contiguous();\n auto y1_t = boxes.select(1, 1).contiguous();\n auto x2_t = boxes.select(1, 2).contiguous();\n auto y2_t = boxes.select(1, 3).contiguous();\n\n Tensor areas_t = (x2_t - x1_t + offset) * (y2_t - y1_t + offset);\n\n auto order_t = std::get<1>(scores.sort(0, /* descending=*/true));\n\n auto nboxes = boxes.size(0);\n Tensor select_t = at::ones({nboxes}, boxes.options().dtype(at::kBool));\n\n auto select = select_t.data_ptr();\n auto order = order_t.data_ptr();\n auto x1 = x1_t.data_ptr();\n auto y1 = y1_t.data_ptr();\n auto x2 = x2_t.data_ptr();\n auto y2 = y2_t.data_ptr();\n auto areas = areas_t.data_ptr();\n\n for (int64_t _i = 0; _i < nboxes; _i++) {\n if (select[_i] == false) continue;\n auto i = order[_i];\n auto ix1 = x1[i];\n auto iy1 = y1[i];\n auto ix2 = x2[i];\n auto iy2 = y2[i];\n auto iarea = areas[i];\n\n for (int64_t _j = _i + 1; _j < nboxes; _j++) {\n if (select[_j] == false) continue;\n auto j = order[_j];\n auto xx1 = std::max(ix1, x1[j]);\n auto yy1 = std::max(iy1, y1[j]);\n auto xx2 = std::min(ix2, x2[j]);\n auto yy2 = std::min(iy2, y2[j]);\n\n auto w = std::max(0.f, xx2 - xx1 + offset);\n auto h = std::max(0.f, yy2 - yy1 + offset);\n auto inter = w * h;\n auto ovr = inter / (iarea + areas[j] - inter);\n if (ovr > iou_threshold) select[_j] = false;\n }\n }\n return order_t.masked_select(select_t);\n}\n\nTensor nms_impl(Tensor boxes, Tensor scores, float iou_threshold, int offset);\nREGISTER_DEVICE_IMPL(nms_impl, CPU, nms_cpu);\n\nTensor softnms_cpu(Tensor boxes, Tensor scores, Tensor dets,\n float iou_threshold, float sigma, float min_score,\n int method, int offset) {\n if (boxes.numel() == 0) {\n return at::empty({0}, boxes.options().dtype(at::kLong));\n }\n\n auto x1_t = boxes.select(1, 0).contiguous();\n auto y1_t = boxes.select(1, 1).contiguous();\n auto x2_t = boxes.select(1, 2).contiguous();\n auto y2_t = boxes.select(1, 3).contiguous();\n auto scores_t = scores.clone();\n\n Tensor areas_t = (x2_t - x1_t + offset) * (y2_t - y1_t + offset);\n\n auto nboxes = boxes.size(0);\n auto x1 = x1_t.data_ptr();\n auto y1 = y1_t.data_ptr();\n auto x2 = x2_t.data_ptr();\n auto y2 = y2_t.data_ptr();\n auto sc = scores_t.data_ptr();\n auto areas = areas_t.data_ptr();\n auto de = dets.data_ptr();\n\n int64_t pos = 0;\n Tensor inds_t = at::arange(nboxes, boxes.options().dtype(at::kLong));\n auto inds = inds_t.data_ptr();\n\n for (int64_t i = 0; i < nboxes; i++) {\n auto max_score = sc[i];\n auto max_pos = i;\n\n pos = i + 1;\n // get max box\n while (pos < nboxes) {\n if (max_score < sc[pos]) {\n max_score = sc[pos];\n max_pos = pos;\n }\n pos = pos + 1;\n }\n // swap\n auto ix1 = de[i * 5 + 0] = x1[max_pos];\n auto iy1 = de[i * 5 + 1] = y1[max_pos];\n auto ix2 = de[i * 5 + 2] = x2[max_pos];\n auto iy2 = de[i * 5 + 3] = y2[max_pos];\n auto iscore = de[i * 5 + 4] = sc[max_pos];\n auto iarea = areas[max_pos];\n auto iind = inds[max_pos];\n x1[max_pos] = x1[i];\n y1[max_pos] = y1[i];\n x2[max_pos] = x2[i];\n y2[max_pos] = y2[i];\n sc[max_pos] = sc[i];\n areas[max_pos] = areas[i];\n inds[max_pos] = inds[i];\n x1[i] = ix1;\n y1[i] = iy1;\n x2[i] = ix2;\n y2[i] = iy2;\n sc[i] = iscore;\n areas[i] = iarea;\n inds[i] = iind;\n\n pos = i + 1;\n while (pos < nboxes) {\n auto xx1 = std::max(ix1, x1[pos]);\n auto yy1 = std::max(iy1, y1[pos]);\n auto xx2 = std::min(ix2, x2[pos]);\n auto yy2 = std::min(iy2, y2[pos]);\n\n auto w = std::max(0.f, xx2 - xx1 + offset);\n auto h = std::max(0.f, yy2 - yy1 + offset);\n auto inter = w * h;\n auto ovr = inter / (iarea + areas[pos] - inter);\n\n float weight = 1.;\n if (method == 0) {\n if (ovr >= iou_threshold) weight = 0;\n } else if (method == 1) {\n if (ovr >= iou_threshold) weight = 1 - ovr;\n } else if (method == 2) {\n weight = std::exp(-(ovr * ovr) / sigma);\n }\n sc[pos] *= weight;\n // if box score falls below threshold, discard the box by\n // swapping with last box update N\n if (sc[pos] < min_score) {\n x1[pos] = x1[nboxes - 1];\n y1[pos] = y1[nboxes - 1];\n x2[pos] = x2[nboxes - 1];\n y2[pos] = y2[nboxes - 1];\n sc[pos] = sc[nboxes - 1];\n areas[pos] = areas[nboxes - 1];\n inds[pos] = inds[nboxes - 1];\n nboxes = nboxes - 1;\n pos = pos - 1;\n }\n pos = pos + 1;\n }\n }\n return inds_t.slice(0, 0, nboxes);\n}\n\nTensor softnms_impl(Tensor boxes, Tensor scores, Tensor dets,\n float iou_threshold, float sigma, float min_score,\n int method, int offset);\nREGISTER_DEVICE_IMPL(softnms_impl, CPU, softnms_cpu);\n\nstd::vector > nms_match_cpu(Tensor dets, float iou_threshold) {\n auto x1_t = dets.select(1, 0).contiguous();\n auto y1_t = dets.select(1, 1).contiguous();\n auto x2_t = dets.select(1, 2).contiguous();\n auto y2_t = dets.select(1, 3).contiguous();\n auto scores = dets.select(1, 4).contiguous();\n\n at::Tensor areas_t = (x2_t - x1_t) * (y2_t - y1_t);\n\n auto order_t = std::get<1>(scores.sort(0, /* descending=*/true));\n\n auto ndets = dets.size(0);\n at::Tensor suppressed_t =\n at::zeros({ndets}, dets.options().dtype(at::kByte).device(at::kCPU));\n\n auto suppressed = suppressed_t.data_ptr();\n auto order = order_t.data_ptr();\n auto x1 = x1_t.data_ptr();\n auto y1 = y1_t.data_ptr();\n auto x2 = x2_t.data_ptr();\n auto y2 = y2_t.data_ptr();\n auto areas = areas_t.data_ptr();\n\n std::vector keep;\n std::vector > matched;\n\n for (int64_t _i = 0; _i < ndets; _i++) {\n auto i = order[_i];\n if (suppressed[i] == 1) continue;\n keep.push_back(i);\n std::vector v_i;\n auto ix1 = x1[i];\n auto iy1 = y1[i];\n auto ix2 = x2[i];\n auto iy2 = y2[i];\n auto iarea = areas[i];\n\n for (int64_t _j = _i + 1; _j < ndets; _j++) {\n auto j = order[_j];\n if (suppressed[j] == 1) continue;\n auto xx1 = std::max(ix1, x1[j]);\n auto yy1 = std::max(iy1, y1[j]);\n auto xx2 = std::min(ix2, x2[j]);\n auto yy2 = std::min(iy2, y2[j]);\n\n auto w = std::max(static_cast(0), xx2 - xx1);\n auto h = std::max(static_cast(0), yy2 - yy1);\n auto inter = w * h;\n auto ovr = inter / (iarea + areas[j] - inter);\n if (ovr >= iou_threshold) {\n suppressed[j] = 1;\n v_i.push_back(j);\n }\n }\n matched.push_back(v_i);\n }\n for (size_t i = 0; i < keep.size(); i++)\n matched[i].insert(matched[i].begin(), keep[i]);\n return matched;\n}\n\nstd::vector > nms_match_impl(Tensor dets, float iou_threshold);\nREGISTER_DEVICE_IMPL(nms_match_impl, CPU, nms_match_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/nms_rotated.cpp", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/nms_rotated/nms_rotated_cpu.cpp\n#include \"box_iou_rotated_utils.hpp\"\n#include \"pytorch_cpp_helper.hpp\"\n\ntemplate \nTensor nms_rotated_cpu_kernel(const Tensor dets, const Tensor scores,\n const float iou_threshold) {\n // nms_rotated_cpu_kernel is modified from torchvision's nms_cpu_kernel,\n // however, the code in this function is much shorter because\n // we delegate the IoU computation for rotated boxes to\n // the single_box_iou_rotated function in box_iou_rotated_utils.h\n AT_ASSERTM(!dets.is_cuda(), \"dets must be a CPU tensor\");\n AT_ASSERTM(!scores.is_cuda(), \"scores must be a CPU tensor\");\n AT_ASSERTM(dets.scalar_type() == scores.scalar_type(),\n \"dets should have the same type as scores\");\n\n if (dets.numel() == 0) {\n return at::empty({0}, dets.options().dtype(at::kLong));\n }\n\n auto order_t = std::get<1>(scores.sort(0, /* descending=*/true));\n\n auto ndets = dets.size(0);\n Tensor suppressed_t = at::zeros({ndets}, dets.options().dtype(at::kByte));\n Tensor keep_t = at::zeros({ndets}, dets.options().dtype(at::kLong));\n\n auto suppressed = suppressed_t.data_ptr();\n auto keep = keep_t.data_ptr();\n auto order = order_t.data_ptr();\n\n int64_t num_to_keep = 0;\n\n for (int64_t _i = 0; _i < ndets; _i++) {\n auto i = order[_i];\n if (suppressed[i] == 1) {\n continue;\n }\n\n keep[num_to_keep++] = i;\n\n for (int64_t _j = _i + 1; _j < ndets; _j++) {\n auto j = order[_j];\n if (suppressed[j] == 1) {\n continue;\n }\n\n auto ovr = single_box_iou_rotated(\n dets[i].data_ptr(), dets[j].data_ptr(), 0);\n if (ovr >= iou_threshold) {\n suppressed[j] = 1;\n }\n }\n }\n return keep_t.narrow(/*dim=*/0, /*start=*/0, /*length=*/num_to_keep);\n}\n\nTensor nms_rotated_cpu(const Tensor dets, const Tensor scores,\n const float iou_threshold) {\n auto result = at::empty({0}, dets.options());\n AT_DISPATCH_FLOATING_TYPES(dets.scalar_type(), \"nms_rotated\", [&] {\n result = nms_rotated_cpu_kernel(dets, scores, iou_threshold);\n });\n return result;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/pixel_group.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// It is modified from https://github.com/WenmuZhou/PAN.pytorch\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nstd::vector> estimate_confidence(int32_t* label,\n float* score, int label_num,\n int height, int width) {\n std::vector> point_vector;\n for (int i = 0; i < label_num; i++) {\n std::vector point;\n point.push_back(0);\n point.push_back(0);\n point_vector.push_back(point);\n }\n for (int y = 0; y < height; y++) {\n auto label_tmp = label + y * width;\n auto score_tmp = score + y * width;\n for (int x = 0; x < width; x++) {\n auto l = label_tmp[x];\n if (l > 0) {\n float confidence = score_tmp[x];\n point_vector[l].push_back(x);\n point_vector[l].push_back(y);\n point_vector[l][0] += confidence;\n point_vector[l][1] += 1;\n }\n }\n }\n for (size_t l = 0; l < point_vector.size(); l++)\n if (point_vector[l][1] > 0) {\n point_vector[l][0] /= point_vector[l][1];\n }\n return point_vector;\n}\nstd::vector> pixel_group_cpu(\n Tensor score, Tensor mask, Tensor embedding, Tensor kernel_label,\n Tensor kernel_contour, int kernel_region_num, float dis_threshold) {\n assert(score.dim() == 2);\n assert(mask.dim() == 2);\n assert(embedding_dim.dim() == 3);\n int height = score.size(0);\n int width = score.size(1);\n assert(height == mask.size(0) == embedding.size(1) == kernel_label.size(1));\n assert(width == mask.size(1) == embedding.size(2) == kernel_label.size(2));\n\n auto threshold_square = dis_threshold * dis_threshold;\n auto ptr_score = score.data_ptr();\n auto ptr_mask = mask.data_ptr();\n auto ptr_kernel_contour = kernel_contour.data_ptr();\n auto ptr_embedding = embedding.data_ptr();\n auto ptr_kernel_label = kernel_label.data_ptr();\n std::queue> contour_pixels;\n auto embedding_dim = embedding.size(2);\n std::vector> kernel_vector(\n kernel_region_num, std::vector(embedding_dim + 1, 0));\n\n Tensor text_label;\n text_label = kernel_label.clone();\n auto ptr_text_label = text_label.data_ptr();\n\n for (int i = 0; i < height; i++) {\n auto ptr_embedding_tmp = ptr_embedding + i * width * embedding_dim;\n auto ptr_kernel_label_tmp = ptr_kernel_label + i * width;\n auto ptr_kernel_contour_tmp = ptr_kernel_contour + i * width;\n\n for (int j = 0, k = 0; j < width && k < width * embedding_dim;\n j++, k += embedding_dim) {\n int32_t label = ptr_kernel_label_tmp[j];\n if (label > 0) {\n for (int d = 0; d < embedding_dim; d++)\n kernel_vector[label][d] += ptr_embedding_tmp[k + d];\n kernel_vector[label][embedding_dim] += 1;\n // kernel pixel number\n if (ptr_kernel_contour_tmp[j]) {\n contour_pixels.push(std::make_tuple(i, j, label));\n }\n }\n }\n }\n for (int i = 0; i < kernel_region_num; i++) {\n for (int j = 0; j < embedding_dim; j++) {\n kernel_vector[i][j] /= kernel_vector[i][embedding_dim];\n }\n }\n int dx[4] = {-1, 1, 0, 0};\n int dy[4] = {0, 0, -1, 1};\n while (!contour_pixels.empty()) {\n auto query_pixel = contour_pixels.front();\n contour_pixels.pop();\n int y = std::get<0>(query_pixel);\n int x = std::get<1>(query_pixel);\n int32_t l = std::get<2>(query_pixel);\n auto kernel_cv = kernel_vector[l];\n for (int idx = 0; idx < 4; idx++) {\n int tmpy = y + dy[idx];\n int tmpx = x + dx[idx];\n auto ptr_text_label_tmp = ptr_text_label + tmpy * width;\n if (tmpy < 0 || tmpy >= height || tmpx < 0 || tmpx >= width) continue;\n if (!ptr_mask[tmpy * width + tmpx] || ptr_text_label_tmp[tmpx] > 0)\n continue;\n\n float dis = 0;\n auto ptr_embedding_tmp = ptr_embedding + tmpy * width * embedding_dim;\n for (size_t i = 0; i < embedding_dim; i++) {\n dis +=\n pow(kernel_cv[i] - ptr_embedding_tmp[tmpx * embedding_dim + i], 2);\n // ignore further computing if dis is big enough\n if (dis >= threshold_square) break;\n }\n if (dis >= threshold_square) continue;\n contour_pixels.push(std::make_tuple(tmpy, tmpx, l));\n ptr_text_label_tmp[tmpx] = l;\n }\n }\n\n return estimate_confidence(ptr_text_label, ptr_score, kernel_region_num,\n height, width);\n}\nstd::vector> pixel_group_impl(\n Tensor score, Tensor mask, Tensor embedding, Tensor kernel_label,\n Tensor kernel_contour, int kernel_region_num, float dis_threshold);\nREGISTER_DEVICE_IMPL(pixel_group_impl, CPU, pixel_group_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/points_in_boxes.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n\ninline void lidar_to_local_coords_cpu(float shift_x, float shift_y, float rz,\n float &local_x, float &local_y) {\n float cosa = cos(-rz), sina = sin(-rz);\n local_x = shift_x * cosa + shift_y * (-sina);\n local_y = shift_x * sina + shift_y * cosa;\n}\n\ninline int check_pt_in_box3d_cpu(const float *pt, const float *box3d,\n float &local_x, float &local_y) {\n // param pt: (x, y, z)\n // param box3d: (cx, cy, cz, x_size, y_size, z_size, rz) in LiDAR coordinate,\n // cz in the bottom center\n float x = pt[0], y = pt[1], z = pt[2];\n float cx = box3d[0], cy = box3d[1], cz = box3d[2];\n float x_size = box3d[3], y_size = box3d[4], z_size = box3d[5], rz = box3d[6];\n cz += z_size /\n 2.0; // shift to the center since cz in box3d is the bottom center\n\n if (fabsf(z - cz) > z_size / 2.0) return 0;\n lidar_to_local_coords_cpu(x - cx, y - cy, rz, local_x, local_y);\n float in_flag = (local_x > -x_size / 2.0) & (local_x < x_size / 2.0) &\n (local_y > -y_size / 2.0) & (local_y < y_size / 2.0);\n return in_flag;\n}\n\nvoid points_in_boxes_cpu_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor pts_indices_tensor) {\n // params boxes: (N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is the bottom center, each box DO NOT overlaps params pts:\n // (npoints, 3) [x, y, z] in LiDAR coordinate params pts_indices: (N, npoints)\n\n CHECK_CONTIGUOUS(boxes_tensor);\n CHECK_CONTIGUOUS(pts_tensor);\n CHECK_CONTIGUOUS(pts_indices_tensor);\n\n int boxes_num = boxes_tensor.size(0);\n int pts_num = pts_tensor.size(0);\n\n const float *boxes = boxes_tensor.data_ptr();\n const float *pts = pts_tensor.data_ptr();\n int *pts_indices = pts_indices_tensor.data_ptr();\n\n float local_x = 0, local_y = 0;\n for (int i = 0; i < boxes_num; i++) {\n for (int j = 0; j < pts_num; j++) {\n int cur_in_flag =\n check_pt_in_box3d_cpu(pts + j * 3, boxes + i * 7, local_x, local_y);\n pts_indices[i * pts_num + j] = cur_in_flag;\n }\n }\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/psamask.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/hszhao/semseg/blob/master/lib/psa/src\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\n#ifndef min\n#define min(a, b) (((a) < (b)) ? (a) : (b))\n#endif\n#ifndef max\n#define max(a, b) (((a) > (b)) ? (a) : (b))\n#endif\n\nvoid psamask_collect_forward(const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask, const Tensor mask_data,\n Tensor buffer_data) {\n for (int n = 0; n < num_; n++) {\n for (int h = 0; h < h_feature; h++) {\n for (int w = 0; w < w_feature; w++) {\n // effective mask region : [hstart, hend) x [wstart, wend) with\n // mask-indexed\n const int hstart = max(0, half_h_mask - h);\n const int hend = min(h_mask, h_feature + half_h_mask - h);\n const int wstart = max(0, half_w_mask - w);\n const int wend = min(w_mask, w_feature + half_w_mask - w);\n // (hidx, widx ) with mask-indexed\n // (hidx + h - half_h_mask, widx + w - half_w_mask) with\n // feature-indexed\n for (int hidx = hstart; hidx < hend; hidx++) {\n for (int widx = wstart; widx < wend; widx++) {\n buffer_data.view({-1})[(n * h_feature * w_feature +\n (hidx + h - half_h_mask) * w_feature +\n (widx + w - half_w_mask)) *\n h_feature * w_feature +\n h * w_feature + w] =\n mask_data.view(\n {-1})[((n * h_mask * w_mask + hidx * w_mask + widx) *\n h_feature +\n h) *\n w_feature +\n w];\n }\n }\n }\n }\n }\n}\n\nvoid psamask_distribute_forward(const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask, const Tensor mask_data,\n Tensor buffer_data) {\n for (int n = 0; n < num_; n++) {\n for (int h = 0; h < h_feature; h++) {\n for (int w = 0; w < w_feature; w++) {\n // effective mask region : [hstart, hend) x [wstart, wend) with\n // mask-indexed\n const int hstart = max(0, half_h_mask - h);\n const int hend = min(h_mask, h_feature + half_h_mask - h);\n const int wstart = max(0, half_w_mask - w);\n const int wend = min(w_mask, w_feature + half_w_mask - w);\n // (hidx, widx ) with mask-indexed\n // (hidx + h - half_h_mask, widx + w - half_w_mask) with\n // feature-indexed\n for (int hidx = hstart; hidx < hend; hidx++) {\n for (int widx = wstart; widx < wend; widx++) {\n buffer_data.view(\n {-1})[(n * h_feature * w_feature + h * w_feature + w) *\n h_feature * w_feature +\n (hidx + h - half_h_mask) * w_feature +\n (widx + w - half_w_mask)] =\n mask_data.view(\n {-1})[((n * h_mask * w_mask + hidx * w_mask + widx) *\n h_feature +\n h) *\n w_feature +\n w];\n }\n }\n }\n }\n }\n}\n\nvoid psamask_collect_backward(const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask, const Tensor buffer_diff,\n Tensor mask_diff) {\n for (int n = 0; n < num_; n++) {\n for (int h = 0; h < h_feature; h++) {\n for (int w = 0; w < w_feature; w++) {\n // effective mask region : [hstart, hend) x [wstart, wend) with\n // mask-indexed\n const int hstart = max(0, half_h_mask - h);\n const int hend = min(h_mask, h_feature + half_h_mask - h);\n const int wstart = max(0, half_w_mask - w);\n const int wend = min(w_mask, w_feature + half_w_mask - w);\n // (hidx, widx ) with mask-indexed\n // (hidx + h - half_h_mask, widx + w - half_w_mask) with\n // feature-indexed\n for (int hidx = hstart; hidx < hend; hidx++) {\n for (int widx = wstart; widx < wend; widx++) {\n mask_diff.view({-1})[((n * h_mask * w_mask + hidx * w_mask + widx) *\n h_feature +\n h) *\n w_feature +\n w] =\n buffer_diff.view({-1})[(n * h_feature * w_feature +\n (hidx + h - half_h_mask) * w_feature +\n (widx + w - half_w_mask)) *\n h_feature * w_feature +\n h * w_feature + w];\n }\n }\n }\n }\n }\n}\n\nvoid psamask_distribute_backward(const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask,\n const Tensor buffer_diff, Tensor mask_diff) {\n for (int n = 0; n < num_; n++) {\n for (int h = 0; h < h_feature; h++) {\n for (int w = 0; w < w_feature; w++) {\n // effective mask region : [hstart, hend) x [wstart, wend) with\n // mask-indexed\n const int hstart = max(0, half_h_mask - h);\n const int hend = min(h_mask, h_feature + half_h_mask - h);\n const int wstart = max(0, half_w_mask - w);\n const int wend = min(w_mask, w_feature + half_w_mask - w);\n // (hidx, widx ) with mask-indexed\n // (hidx + h - half_h_mask, widx + w - half_w_mask) with\n // feature-indexed\n for (int hidx = hstart; hidx < hend; hidx++) {\n for (int widx = wstart; widx < wend; widx++) {\n mask_diff.view({-1})[((n * h_mask * w_mask + hidx * w_mask + widx) *\n h_feature +\n h) *\n w_feature +\n w] =\n buffer_diff.view(\n {-1})[(n * h_feature * w_feature + h * w_feature + w) *\n h_feature * w_feature +\n (hidx + h - half_h_mask) * w_feature +\n (widx + w - half_w_mask)];\n }\n }\n }\n }\n }\n}\n\nvoid psamask_forward_cpu(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask) {\n if (psa_type == 0)\n psamask_collect_forward(num_, h_feature, w_feature, h_mask, w_mask,\n half_h_mask, half_w_mask, input, output);\n else\n psamask_distribute_forward(num_, h_feature, w_feature, h_mask, w_mask,\n half_h_mask, half_w_mask, input, output);\n}\n\nvoid psamask_backward_cpu(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask) {\n if (psa_type == 0)\n psamask_collect_backward(num_, h_feature, w_feature, h_mask, w_mask,\n half_h_mask, half_w_mask, grad_output, grad_input);\n else\n psamask_distribute_backward(num_, h_feature, w_feature, h_mask, w_mask,\n half_h_mask, half_w_mask, grad_output,\n grad_input);\n}\n\nvoid psamask_forward_impl(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask);\n\nvoid psamask_backward_impl(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask);\nREGISTER_DEVICE_IMPL(psamask_forward_impl, CPU, psamask_forward_cpu);\nREGISTER_DEVICE_IMPL(psamask_backward_impl, CPU, psamask_backward_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/roi_align.cpp", "content": "// Modified from\n// https://github.com/facebookresearch/detectron2/tree/master/detectron2/layers/csrc/ROIAlign\n// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n#include \n#include \n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\n// implementation taken from Caffe2\ntemplate \nstruct PreCalc {\n int pos1;\n int pos2;\n int pos3;\n int pos4;\n T w1;\n T w2;\n T w3;\n T w4;\n};\n\ntemplate \nvoid pre_calc_for_bilinear_interpolate(\n const int height, const int width, const int pooled_height,\n const int pooled_width, const int iy_upper, const int ix_upper,\n T roi_start_h, T roi_start_w, T bin_size_h, T bin_size_w,\n int roi_bin_grid_h, int roi_bin_grid_w, std::vector>& pre_calc) {\n int pre_calc_index = 0;\n for (int ph = 0; ph < pooled_height; ph++) {\n for (int pw = 0; pw < pooled_width; pw++) {\n for (int iy = 0; iy < iy_upper; iy++) {\n const T yy = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < ix_upper; ix++) {\n const T xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n T x = xx;\n T y = yy;\n // deal with: inverse elements are out of feature map boundary\n if (y < -1.0 || y > height || x < -1.0 || x > width) {\n // empty\n PreCalc pc;\n pc.pos1 = 0;\n pc.pos2 = 0;\n pc.pos3 = 0;\n pc.pos4 = 0;\n pc.w1 = 0;\n pc.w2 = 0;\n pc.w3 = 0;\n pc.w4 = 0;\n pre_calc[pre_calc_index] = pc;\n pre_calc_index += 1;\n continue;\n }\n\n if (y <= 0) {\n y = 0;\n }\n if (x <= 0) {\n x = 0;\n }\n\n int y_low = (int)y;\n int x_low = (int)x;\n int y_high;\n int x_high;\n\n if (y_low >= height - 1) {\n y_high = y_low = height - 1;\n y = (T)y_low;\n } else {\n y_high = y_low + 1;\n }\n\n if (x_low >= width - 1) {\n x_high = x_low = width - 1;\n x = (T)x_low;\n } else {\n x_high = x_low + 1;\n }\n\n T ly = y - y_low;\n T lx = x - x_low;\n T hy = 1. - ly, hx = 1. - lx;\n T w1 = hy * hx, w2 = hy * lx, w3 = ly * hx, w4 = ly * lx;\n\n // save weights and indices\n PreCalc pc;\n pc.pos1 = y_low * width + x_low;\n pc.pos2 = y_low * width + x_high;\n pc.pos3 = y_high * width + x_low;\n pc.pos4 = y_high * width + x_high;\n pc.w1 = w1;\n pc.w2 = w2;\n pc.w3 = w3;\n pc.w4 = w4;\n pre_calc[pre_calc_index] = pc;\n\n pre_calc_index += 1;\n }\n }\n }\n }\n}\n\ntemplate \nvoid ROIAlignForward(const int nthreads, const T* input, const T* rois,\n T* output, T* argmax_y, T* argmax_x,\n const int pooled_height, const int pooled_width,\n const T spatial_scale, const int sampling_ratio,\n const int pool_mode, // 0 - max pool, 1 - avg pool\n const bool aligned, const int channels, const int height,\n const int width) {\n int n_rois = nthreads / channels / pooled_width / pooled_height;\n // (n, c, ph, pw) is an element in the pooled output\n // can be parallelized using omp\n // #pragma omp parallel for num_threads(32)\n for (int n = 0; n < n_rois; n++) {\n int index_n = n * channels * pooled_width * pooled_height;\n\n const T* offset_rois = rois + n * 5;\n int roi_batch_ind = offset_rois[0];\n\n // Do not use rounding; this implementation detail is critical\n T offset = aligned ? (T)0.5 : (T)0.0;\n T roi_start_w = offset_rois[1] * spatial_scale - offset;\n T roi_start_h = offset_rois[2] * spatial_scale - offset;\n T roi_end_w = offset_rois[3] * spatial_scale - offset;\n T roi_end_h = offset_rois[4] * spatial_scale - offset;\n\n T roi_width = roi_end_w - roi_start_w;\n T roi_height = roi_end_h - roi_start_h;\n if (aligned) {\n AT_ASSERTM(roi_width >= 0 && roi_height >= 0,\n \"ROIs in ROIAlign cannot have non-negative size!\");\n } else { // for backward-compatibility only\n roi_width = std::max(roi_width, (T)1.);\n roi_height = std::max(roi_height, (T)1.);\n }\n T bin_size_h = static_cast(roi_height) / static_cast(pooled_height);\n T bin_size_w = static_cast(roi_width) / static_cast(pooled_width);\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (sampling_ratio > 0)\n ? sampling_ratio\n : ceilf(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (sampling_ratio > 0) ? sampling_ratio : ceilf(roi_width / pooled_width);\n\n // When the grid is empty, output zeros == 0/1, instead of NaN.\n const T count = std::max(roi_bin_grid_h * roi_bin_grid_w, 1); // e.g. = 4\n\n // we want to precalculate indices and weights shared by all channels,\n // this is the key point of optimization\n std::vector> pre_calc(roi_bin_grid_h * roi_bin_grid_w *\n pooled_width * pooled_height);\n pre_calc_for_bilinear_interpolate(\n height, width, pooled_height, pooled_width, roi_bin_grid_h,\n roi_bin_grid_w, roi_start_h, roi_start_w, bin_size_h, bin_size_w,\n roi_bin_grid_h, roi_bin_grid_w, pre_calc);\n\n for (int c = 0; c < channels; c++) {\n int index_n_c = index_n + c * pooled_width * pooled_height;\n const T* offset_input =\n input + (roi_batch_ind * channels + c) * height * width;\n int pre_calc_index = 0;\n\n for (int ph = 0; ph < pooled_height; ph++) {\n for (int pw = 0; pw < pooled_width; pw++) {\n int index = index_n_c + ph * pooled_width + pw;\n\n T output_val = 0.;\n T maxval = -10000;\n T maxidx_y = -1.f, maxidx_x = -1.f;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const T y = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h);\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const T x = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n PreCalc pc = pre_calc[pre_calc_index];\n T val = pc.w1 * offset_input[pc.pos1] +\n pc.w2 * offset_input[pc.pos2] +\n pc.w3 * offset_input[pc.pos3] +\n pc.w4 * offset_input[pc.pos4];\n if (val > maxval) {\n maxval = val;\n maxidx_y = y;\n maxidx_x = x;\n }\n output_val += val;\n pre_calc_index += 1;\n }\n }\n if (pool_mode == 0) {\n // We do max pooling inside a bin\n output[index] = maxval;\n argmax_y[index] = maxidx_y;\n argmax_x[index] = maxidx_x;\n } else if (pool_mode == 1) {\n // We do average (integral) pooling inside a bin\n output[index] = output_val / count;\n } // if\n } // for pw\n } // for ph\n } // for c\n } // for n\n}\n\ntemplate \nvoid bilinear_interpolate_gradient(const int height, const int width, T y, T x,\n T& w1, T& w2, T& w3, T& w4, int& x_low,\n int& x_high, int& y_low, int& y_high,\n const int index /* index for debug only*/) {\n // deal with cases that inverse elements are out of feature map boundary\n if (y < -1.0 || y > height || x < -1.0 || x > width) {\n // empty\n w1 = w2 = w3 = w4 = 0.;\n x_low = x_high = y_low = y_high = -1;\n return;\n }\n\n if (y <= 0) y = 0;\n if (x <= 0) x = 0;\n\n y_low = (int)y;\n x_low = (int)x;\n\n if (y_low >= height - 1) {\n y_high = y_low = height - 1;\n y = (T)y_low;\n } else {\n y_high = y_low + 1;\n }\n\n if (x_low >= width - 1) {\n x_high = x_low = width - 1;\n x = (T)x_low;\n } else {\n x_high = x_low + 1;\n }\n\n T ly = y - y_low;\n T lx = x - x_low;\n T hy = 1. - ly, hx = 1. - lx;\n\n // reference in forward\n // T v1 = input[y_low * width + x_low];\n // T v2 = input[y_low * width + x_high];\n // T v3 = input[y_high * width + x_low];\n // T v4 = input[y_high * width + x_high];\n // T val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n\n w1 = hy * hx, w2 = hy * lx, w3 = ly * hx, w4 = ly * lx;\n\n return;\n}\n\ntemplate \ninline void add(T* address, const T& val) {\n *address += val;\n}\n\ntemplate \nvoid ROIAlignBackward(const int nthreads, const T* grad_output, const T* rois,\n const T* argmax_y, const T* argmax_x, T* grad_input,\n const int pooled_height, const int pooled_width,\n const T spatial_scale, const int sampling_ratio,\n const int pool_mode, // 0 - max pool, 1 - avg pool\n const bool aligned, const int channels, const int height,\n const int width, const int n_stride, const int c_stride,\n const int h_stride, const int w_stride) {\n for (int index = 0; index < nthreads; index++) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const T* offset_rois = rois + n * 5;\n int roi_batch_ind = offset_rois[0];\n\n // Do not use rounding; this implementation detail is critical\n T offset = aligned ? (T)0.5 : (T)0.0;\n T roi_start_w = offset_rois[1] * spatial_scale - offset;\n T roi_start_h = offset_rois[2] * spatial_scale - offset;\n T roi_end_w = offset_rois[3] * spatial_scale - offset;\n T roi_end_h = offset_rois[4] * spatial_scale - offset;\n\n T roi_width = roi_end_w - roi_start_w;\n T roi_height = roi_end_h - roi_start_h;\n if (aligned) {\n AT_ASSERTM(roi_width >= 0 && roi_height >= 0,\n \"ROIs in ROIAlign do not have non-negative size!\");\n } else { // for backward-compatibility only\n roi_width = std::max(roi_width, (T)1.);\n roi_height = std::max(roi_height, (T)1.);\n }\n T bin_size_h = static_cast(roi_height) / static_cast(pooled_height);\n T bin_size_w = static_cast(roi_width) / static_cast(pooled_width);\n\n T* offset_grad_input =\n grad_input + ((roi_batch_ind * channels + c) * height * width);\n\n int output_offset = n * n_stride + c * c_stride;\n const T* offset_grad_output = grad_output + output_offset;\n const T grad_output_this_bin =\n offset_grad_output[ph * h_stride + pw * w_stride];\n\n if (pool_mode == 0) {\n // We do max pooling inside a bin\n T y = argmax_y[index], x = argmax_x[index];\n if (y != -1.f) {\n T w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3, w4,\n x_low, x_high, y_low, y_high, index);\n\n T g1 = grad_output_this_bin * w1;\n T g2 = grad_output_this_bin * w2;\n T g3 = grad_output_this_bin * w3;\n T g4 = grad_output_this_bin * w4;\n\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n // atomic add is not needed for now since it is single threaded\n add(offset_grad_input + y_low * width + x_low, static_cast(g1));\n add(offset_grad_input + y_low * width + x_high, static_cast(g2));\n add(offset_grad_input + y_high * width + x_low, static_cast(g3));\n add(offset_grad_input + y_high * width + x_high, static_cast(g4));\n } // if\n } // mode\n } else if (pool_mode == 1) {\n // We do average (integral) pooling inside a bin\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h =\n (sampling_ratio > 0)\n ? sampling_ratio\n : ceilf(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w = (sampling_ratio > 0)\n ? sampling_ratio\n : ceilf(roi_width / pooled_width);\n\n const T count = roi_bin_grid_h * roi_bin_grid_w; // e.g. = 4\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const T y = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const T x = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n T w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3, w4,\n x_low, x_high, y_low, y_high, index);\n\n T g1 = grad_output_this_bin * w1 / count;\n T g2 = grad_output_this_bin * w2 / count;\n T g3 = grad_output_this_bin * w3 / count;\n T g4 = grad_output_this_bin * w4 / count;\n\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n // atomic add is not needed for now since it is single threaded\n add(offset_grad_input + y_low * width + x_low, static_cast(g1));\n add(offset_grad_input + y_low * width + x_high, static_cast(g2));\n add(offset_grad_input + y_high * width + x_low, static_cast(g3));\n add(offset_grad_input + y_high * width + x_high,\n static_cast(g4));\n } // if\n } // ix\n } // iy\n } // mode\n } // for\n} // ROIAlignBackward\n\nvoid ROIAlignForwardCPULauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n int output_size = output.numel();\n int channels = input.size(1);\n int height = input.size(2);\n int width = input.size(3);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"ROIAlign_forward\", [&] {\n ROIAlignForward(\n output_size, input.data_ptr(), rois.data_ptr(),\n output.data_ptr(), argmax_y.data_ptr(),\n argmax_x.data_ptr(), aligned_height, aligned_width,\n static_cast(spatial_scale), sampling_ratio, pool_mode,\n aligned, channels, height, width);\n });\n}\n\nvoid ROIAlignBackwardCPULauncher(Tensor grad_output, Tensor rois,\n Tensor argmax_y, Tensor argmax_x,\n Tensor grad_input, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode,\n bool aligned) {\n int output_size = grad_output.numel();\n int channels = grad_input.size(1);\n int height = grad_input.size(2);\n int width = grad_input.size(3);\n\n // get stride values to ensure indexing into gradients is correct.\n int n_stride = grad_output.stride(0);\n int c_stride = grad_output.stride(1);\n int h_stride = grad_output.stride(2);\n int w_stride = grad_output.stride(3);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"ROIAlign_backward\", [&] {\n ROIAlignBackward(\n output_size, grad_output.data_ptr(),\n rois.data_ptr(), argmax_y.data_ptr(),\n argmax_x.data_ptr(), grad_input.data_ptr(),\n aligned_height, aligned_width, static_cast(spatial_scale),\n sampling_ratio, pool_mode, aligned, channels, height, width,\n n_stride, c_stride, h_stride, w_stride);\n });\n}\n\nvoid roi_align_forward_cpu(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned) {\n ROIAlignForwardCPULauncher(input, rois, output, argmax_y, argmax_x,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_backward_cpu(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n ROIAlignBackwardCPULauncher(grad_output, rois, argmax_y, argmax_x, grad_input,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nvoid roi_align_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nREGISTER_DEVICE_IMPL(roi_align_forward_impl, CPU, roi_align_forward_cpu);\nREGISTER_DEVICE_IMPL(roi_align_backward_impl, CPU, roi_align_backward_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/roi_align_rotated.cpp", "content": "// Modified from\n// https://github.com/facebookresearch/detectron2/tree/master/detectron2/layers/csrc/ROIAlignRotated\n// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n#include \n#include \n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\n// implementation taken from Caffe2\ntemplate \nstruct PreCalc {\n int pos1;\n int pos2;\n int pos3;\n int pos4;\n T w1;\n T w2;\n T w3;\n T w4;\n};\n\ntemplate \nvoid pre_calc_for_bilinear_interpolate(\n const int height, const int width, const int pooled_height,\n const int pooled_width, const int iy_upper, const int ix_upper,\n T roi_start_h, T roi_start_w, T bin_size_h, T bin_size_w,\n int roi_bin_grid_h, int roi_bin_grid_w, T roi_center_h, T roi_center_w,\n T cos_theta, T sin_theta, std::vector>& pre_calc) {\n int pre_calc_index = 0;\n for (int ph = 0; ph < pooled_height; ph++) {\n for (int pw = 0; pw < pooled_width; pw++) {\n for (int iy = 0; iy < iy_upper; iy++) {\n const T yy = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < ix_upper; ix++) {\n const T xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n // Rotate by theta around the center and translate\n // In image space, (y, x) is the order for Right Handed System,\n // and this is essentially multiplying the point by a rotation matrix\n // to rotate it counterclockwise through angle theta.\n T y = yy * cos_theta - xx * sin_theta + roi_center_h;\n T x = yy * sin_theta + xx * cos_theta + roi_center_w;\n // deal with: inverse elements are out of feature map boundary\n if (y < -1.0 || y > height || x < -1.0 || x > width) {\n // empty\n PreCalc pc;\n pc.pos1 = 0;\n pc.pos2 = 0;\n pc.pos3 = 0;\n pc.pos4 = 0;\n pc.w1 = 0;\n pc.w2 = 0;\n pc.w3 = 0;\n pc.w4 = 0;\n pre_calc[pre_calc_index] = pc;\n pre_calc_index += 1;\n continue;\n }\n\n if (y < 0) {\n y = 0;\n }\n if (x < 0) {\n x = 0;\n }\n\n int y_low = (int)y;\n int x_low = (int)x;\n int y_high;\n int x_high;\n\n if (y_low >= height - 1) {\n y_high = y_low = height - 1;\n y = (T)y_low;\n } else {\n y_high = y_low + 1;\n }\n\n if (x_low >= width - 1) {\n x_high = x_low = width - 1;\n x = (T)x_low;\n } else {\n x_high = x_low + 1;\n }\n\n T ly = y - y_low;\n T lx = x - x_low;\n T hy = 1. - ly, hx = 1. - lx;\n T w1 = hy * hx, w2 = hy * lx, w3 = ly * hx, w4 = ly * lx;\n\n // save weights and indices\n PreCalc pc;\n pc.pos1 = y_low * width + x_low;\n pc.pos2 = y_low * width + x_high;\n pc.pos3 = y_high * width + x_low;\n pc.pos4 = y_high * width + x_high;\n pc.w1 = w1;\n pc.w2 = w2;\n pc.w3 = w3;\n pc.w4 = w4;\n pre_calc[pre_calc_index] = pc;\n\n pre_calc_index += 1;\n }\n }\n }\n }\n}\n\ntemplate \nvoid ROIAlignRotatedForward(const int nthreads, const T* input,\n const T& spatial_scale, const bool aligned,\n const bool clockwise, const int channels,\n const int height, const int width,\n const int pooled_height, const int pooled_width,\n const int sampling_ratio, const T* rois,\n T* output) {\n int n_rois = nthreads / channels / pooled_width / pooled_height;\n // (n, c, ph, pw) is an element in the pooled output\n // can be parallelized using omp\n // #pragma omp parallel for num_threads(32)\n for (int n = 0; n < n_rois; n++) {\n int index_n = n * channels * pooled_width * pooled_height;\n\n const T* current_roi = rois + n * 6;\n int roi_batch_ind = current_roi[0];\n\n // Do not use rounding; this implementation detail is critical\n T offset = aligned ? (T)0.5 : (T)0.0;\n T roi_center_w = current_roi[1] * spatial_scale - offset;\n T roi_center_h = current_roi[2] * spatial_scale - offset;\n T roi_width = current_roi[3] * spatial_scale;\n T roi_height = current_roi[4] * spatial_scale;\n T theta = current_roi[5];\n if (clockwise) {\n theta = -theta; // If clockwise, the angle needs to be reversed.\n }\n T cos_theta = cos(theta);\n T sin_theta = sin(theta);\n\n if (aligned) {\n AT_ASSERTM(roi_width >= 0 && roi_height >= 0,\n \"ROIs in ROIAlignRotated do not have non-negative size!\");\n } else { // for backward-compatibility only\n roi_width = std::max(roi_width, (T)1.);\n roi_height = std::max(roi_height, (T)1.);\n }\n\n T bin_size_h = static_cast(roi_height) / static_cast(pooled_height);\n T bin_size_w = static_cast(roi_width) / static_cast(pooled_width);\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (sampling_ratio > 0)\n ? sampling_ratio\n : ceilf(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (sampling_ratio > 0) ? sampling_ratio : ceilf(roi_width / pooled_width);\n\n // We do average (integral) pooling inside a bin\n const T count = std::max(roi_bin_grid_h * roi_bin_grid_w, 1); // e.g. = 4\n\n // we want to precalculate indices and weights shared by all channels,\n // this is the key point of optimization\n std::vector> pre_calc(roi_bin_grid_h * roi_bin_grid_w *\n pooled_width * pooled_height);\n\n // roi_start_h and roi_start_w are computed wrt the center of RoI (x, y).\n // Appropriate translation needs to be applied after.\n T roi_start_h = -roi_height / 2.0;\n T roi_start_w = -roi_width / 2.0;\n\n pre_calc_for_bilinear_interpolate(\n height, width, pooled_height, pooled_width, roi_bin_grid_h,\n roi_bin_grid_w, roi_start_h, roi_start_w, bin_size_h, bin_size_w,\n roi_bin_grid_h, roi_bin_grid_w, roi_center_h, roi_center_w, cos_theta,\n sin_theta, pre_calc);\n\n for (int c = 0; c < channels; c++) {\n int index_n_c = index_n + c * pooled_width * pooled_height;\n const T* offset_input =\n input + (roi_batch_ind * channels + c) * height * width;\n int pre_calc_index = 0;\n\n for (int ph = 0; ph < pooled_height; ph++) {\n for (int pw = 0; pw < pooled_width; pw++) {\n int index = index_n_c + ph * pooled_width + pw;\n\n T output_val = 0.;\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n PreCalc pc = pre_calc[pre_calc_index];\n output_val += pc.w1 * offset_input[pc.pos1] +\n pc.w2 * offset_input[pc.pos2] +\n pc.w3 * offset_input[pc.pos3] +\n pc.w4 * offset_input[pc.pos4];\n\n pre_calc_index += 1;\n }\n }\n output_val /= count;\n\n output[index] = output_val;\n } // for pw\n } // for ph\n } // for c\n } // for n\n}\n\ntemplate \nvoid bilinear_interpolate_gradient(const int height, const int width, T y, T x,\n T& w1, T& w2, T& w3, T& w4, int& x_low,\n int& x_high, int& y_low, int& y_high) {\n // deal with cases that inverse elements are out of feature map boundary\n if (y < -1.0 || y > height || x < -1.0 || x > width) {\n // empty\n w1 = w2 = w3 = w4 = 0.;\n x_low = x_high = y_low = y_high = -1;\n return;\n }\n\n if (y < 0) {\n y = 0;\n }\n\n if (x < 0) {\n x = 0;\n }\n\n y_low = (int)y;\n x_low = (int)x;\n\n if (y_low >= height - 1) {\n y_high = y_low = height - 1;\n y = (T)y_low;\n } else {\n y_high = y_low + 1;\n }\n\n if (x_low >= width - 1) {\n x_high = x_low = width - 1;\n x = (T)x_low;\n } else {\n x_high = x_low + 1;\n }\n\n T ly = y - y_low;\n T lx = x - x_low;\n T hy = 1. - ly, hx = 1. - lx;\n\n // reference in forward\n // T v1 = input[y_low * width + x_low];\n // T v2 = input[y_low * width + x_high];\n // T v3 = input[y_high * width + x_low];\n // T v4 = input[y_high * width + x_high];\n // T val = (w1 * v1 + w2 * v2 + w3 * v3 + w4 * v4);\n\n w1 = hy * hx, w2 = hy * lx, w3 = ly * hx, w4 = ly * lx;\n\n return;\n}\n\ntemplate \ninline void add(T* address, const T& val) {\n *address += val;\n}\n\ntemplate \nvoid ROIAlignRotatedBackward(\n const int nthreads,\n // may not be contiguous. should index using n_stride, etc\n const T* grad_output, const T& spatial_scale, const bool aligned,\n const bool clockwise, const int channels, const int height, const int width,\n const int pooled_height, const int pooled_width, const int sampling_ratio,\n T* grad_input, const T* rois, const int n_stride, const int c_stride,\n const int h_stride, const int w_stride) {\n for (int index = 0; index < nthreads; index++) {\n // (n, c, ph, pw) is an element in the pooled output\n int pw = index % pooled_width;\n int ph = (index / pooled_width) % pooled_height;\n int c = (index / pooled_width / pooled_height) % channels;\n int n = index / pooled_width / pooled_height / channels;\n\n const T* current_roi = rois + n * 6;\n int roi_batch_ind = current_roi[0];\n\n // Do not use rounding; this implementation detail is critical\n T offset = aligned ? (T)0.5 : (T)0.0;\n T roi_center_w = current_roi[1] * spatial_scale - offset;\n T roi_center_h = current_roi[2] * spatial_scale - offset;\n T roi_width = current_roi[3] * spatial_scale;\n T roi_height = current_roi[4] * spatial_scale;\n T theta = current_roi[5];\n if (clockwise) {\n theta = -theta; // If clockwise, the angle needs to be reversed.\n }\n T cos_theta = cos(theta);\n T sin_theta = sin(theta);\n\n if (aligned) {\n AT_ASSERTM(roi_width >= 0 && roi_height >= 0,\n \"ROIs in ROIAlignRotated do not have non-negative size!\");\n } else { // for backward-compatibility only\n roi_width = std::max(roi_width, (T)1.);\n roi_height = std::max(roi_height, (T)1.);\n }\n\n T bin_size_h = static_cast(roi_height) / static_cast(pooled_height);\n T bin_size_w = static_cast(roi_width) / static_cast(pooled_width);\n\n T* offset_grad_input =\n grad_input + ((roi_batch_ind * channels + c) * height * width);\n\n int output_offset = n * n_stride + c * c_stride;\n const T* offset_grad_output = grad_output + output_offset;\n const T grad_output_this_bin =\n offset_grad_output[ph * h_stride + pw * w_stride];\n\n // We use roi_bin_grid to sample the grid and mimic integral\n int roi_bin_grid_h = (sampling_ratio > 0)\n ? sampling_ratio\n : ceilf(roi_height / pooled_height); // e.g., = 2\n int roi_bin_grid_w =\n (sampling_ratio > 0) ? sampling_ratio : ceilf(roi_width / pooled_width);\n\n // roi_start_h and roi_start_w are computed wrt the center of RoI (x, y).\n // Appropriate translation needs to be applied after.\n T roi_start_h = -roi_height / 2.0;\n T roi_start_w = -roi_width / 2.0;\n\n // We do average (integral) pooling inside a bin\n const T count = roi_bin_grid_h * roi_bin_grid_w; // e.g. = 4\n\n for (int iy = 0; iy < roi_bin_grid_h; iy++) {\n const T yy = roi_start_h + ph * bin_size_h +\n static_cast(iy + .5f) * bin_size_h /\n static_cast(roi_bin_grid_h); // e.g., 0.5, 1.5\n for (int ix = 0; ix < roi_bin_grid_w; ix++) {\n const T xx = roi_start_w + pw * bin_size_w +\n static_cast(ix + .5f) * bin_size_w /\n static_cast(roi_bin_grid_w);\n\n // Rotate by theta around the center and translate\n T y = yy * cos_theta - xx * sin_theta + roi_center_h;\n T x = yy * sin_theta + xx * cos_theta + roi_center_w;\n\n T w1, w2, w3, w4;\n int x_low, x_high, y_low, y_high;\n\n bilinear_interpolate_gradient(height, width, y, x, w1, w2, w3, w4,\n x_low, x_high, y_low, y_high);\n\n T g1 = grad_output_this_bin * w1 / count;\n T g2 = grad_output_this_bin * w2 / count;\n T g3 = grad_output_this_bin * w3 / count;\n T g4 = grad_output_this_bin * w4 / count;\n\n if (x_low >= 0 && x_high >= 0 && y_low >= 0 && y_high >= 0) {\n // atomic add is not needed for now since it is single threaded\n add(offset_grad_input + y_low * width + x_low, static_cast(g1));\n add(offset_grad_input + y_low * width + x_high, static_cast(g2));\n add(offset_grad_input + y_high * width + x_low, static_cast(g3));\n add(offset_grad_input + y_high * width + x_high, static_cast(g4));\n } // if\n } // ix\n } // iy\n } // for\n} // ROIAlignRotatedBackward\n\nvoid ROIAlignRotatedForwardCPULauncher(Tensor input, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n bool aligned, bool clockwise) {\n int output_size = output.numel();\n int channels = input.size(1);\n int height = input.size(2);\n int width = input.size(3);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"ROIAlignRotated_forward\", [&] {\n ROIAlignRotatedForward(\n output_size, input.data_ptr(),\n static_cast(spatial_scale), aligned, clockwise, channels,\n height, width, aligned_height, aligned_width, sampling_ratio,\n rois.data_ptr(), output.data_ptr());\n });\n}\n\nvoid ROIAlignRotatedBackwardCPULauncher(Tensor grad_output, Tensor rois,\n Tensor grad_input, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, bool aligned,\n bool clockwise) {\n int channels = grad_input.size(1);\n int height = grad_input.size(2);\n int width = grad_input.size(3);\n\n // get stride values to ensure indexing into gradients is correct.\n int n_stride = grad_output.stride(0);\n int c_stride = grad_output.stride(1);\n int h_stride = grad_output.stride(2);\n int w_stride = grad_output.stride(3);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"ROIAlignRotated_backward\", [&] {\n ROIAlignRotatedBackward(\n grad_output.numel(), grad_output.data_ptr(),\n static_cast(spatial_scale), aligned, clockwise, channels,\n height, width, aligned_height, aligned_width, sampling_ratio,\n grad_input.data_ptr(), rois.data_ptr(),\n n_stride, c_stride, h_stride, w_stride);\n });\n}\n\nvoid roi_align_rotated_forward_cpu(Tensor input, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n bool aligned, bool clockwise) {\n ROIAlignRotatedForwardCPULauncher(input, rois, output, aligned_height,\n aligned_width, spatial_scale,\n sampling_ratio, aligned, clockwise);\n}\n\nvoid roi_align_rotated_backward_cpu(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, bool aligned,\n bool clockwise) {\n int size_rois = rois.size(1);\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n ROIAlignRotatedBackwardCPULauncher(\n top_grad, rois, bottom_grad, aligned_height, aligned_width, spatial_scale,\n sampling_ratio, aligned, clockwise);\n}\n\nvoid roi_align_rotated_forward_impl(Tensor features, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sample_ratio,\n bool aligned, bool clockwise);\n\nvoid roi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sample_ratio, bool aligned,\n bool clockwise);\nREGISTER_DEVICE_IMPL(roi_align_rotated_forward_impl, CPU,\n roi_align_rotated_forward_cpu);\nREGISTER_DEVICE_IMPL(roi_align_rotated_backward_impl, CPU,\n roi_align_rotated_backward_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cpu/voxelization.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntemplate \nvoid dynamic_voxelize_forward_cpu_kernel(\n const torch::TensorAccessor points,\n torch::TensorAccessor coors, const std::vector voxel_size,\n const std::vector coors_range, const std::vector grid_size,\n const int num_points, const int num_features, const int NDim) {\n const int ndim_minus_1 = NDim - 1;\n bool failed = false;\n // int coor[NDim];\n int* coor = new int[NDim]();\n int c;\n\n for (int i = 0; i < num_points; ++i) {\n failed = false;\n for (int j = 0; j < NDim; ++j) {\n c = floor((points[i][j] - coors_range[j]) / voxel_size[j]);\n // necessary to rm points out of range\n if ((c < 0 || c >= grid_size[j])) {\n failed = true;\n break;\n }\n coor[ndim_minus_1 - j] = c;\n }\n\n if (failed)\n memset(&coors[i][0], -1, NDim * sizeof(T_int));\n else\n memcpy(&coors[i][0], &coor[0], NDim * sizeof(T_int));\n }\n\n delete[] coor;\n}\n\ntemplate \nvoid hard_voxelize_forward_cpu_kernel(\n const torch::TensorAccessor points,\n torch::TensorAccessor voxels, torch::TensorAccessor coors,\n torch::TensorAccessor num_points_per_voxel,\n torch::TensorAccessor coor_to_voxelidx, int& voxel_num,\n const std::vector voxel_size, const std::vector coors_range,\n const std::vector grid_size, const int max_points,\n const int max_voxels, const int num_points, const int num_features,\n const int NDim) {\n // declare a temp coors\n at::Tensor temp_coors = at::zeros(\n {num_points, NDim}, at::TensorOptions().dtype(at::kInt).device(at::kCPU));\n\n // First use dynamic voxelization to get coors,\n // then check max points/voxels constraints\n dynamic_voxelize_forward_cpu_kernel(\n points, temp_coors.accessor(), voxel_size, coors_range, grid_size,\n num_points, num_features, NDim);\n\n int voxelidx, num;\n auto coor = temp_coors.accessor();\n\n for (int i = 0; i < num_points; ++i) {\n // T_int* coor = temp_coors.data_ptr() + i * NDim;\n\n if (coor[i][0] == -1) continue;\n\n voxelidx = coor_to_voxelidx[coor[i][0]][coor[i][1]][coor[i][2]];\n\n // record voxel\n if (voxelidx == -1) {\n voxelidx = voxel_num;\n if (max_voxels != -1 && voxel_num >= max_voxels) continue;\n voxel_num += 1;\n\n coor_to_voxelidx[coor[i][0]][coor[i][1]][coor[i][2]] = voxelidx;\n memcpy(&coors[voxelidx][0], &coor[i][0], NDim * sizeof(T_int));\n }\n\n // put points into voxel\n num = num_points_per_voxel[voxelidx];\n if (max_points == -1 || num < max_points) {\n memcpy(&voxels[voxelidx][num][0], &points[i][0],\n num_features * sizeof(T));\n num_points_per_voxel[voxelidx] += 1;\n }\n }\n\n return;\n}\n\nvoid dynamic_voxelize_forward_cpu(const at::Tensor& points, at::Tensor& coors,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int NDim = 3) {\n // check device\n AT_ASSERTM(points.device().is_cpu(), \"points must be a CPU tensor\");\n\n std::vector grid_size(NDim);\n const int num_points = points.size(0);\n const int num_features = points.size(1);\n\n for (int i = 0; i < NDim; ++i) {\n grid_size[i] =\n round((coors_range[NDim + i] - coors_range[i]) / voxel_size[i]);\n }\n\n // coors, num_points_per_voxel, coor_to_voxelidx are int Tensor\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n points.scalar_type(), \"dynamic_voxelize_forward_cpu_kernel\", [&] {\n dynamic_voxelize_forward_cpu_kernel(\n points.accessor(), coors.accessor(),\n voxel_size, coors_range, grid_size, num_points, num_features, NDim);\n });\n}\n\nint hard_voxelize_forward_cpu(const at::Tensor& points, at::Tensor& voxels,\n at::Tensor& coors,\n at::Tensor& num_points_per_voxel,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int max_points, const int max_voxels,\n const int NDim = 3) {\n // current version tooks about 0.02s_0.03s for one frame on cpu\n // check device\n AT_ASSERTM(points.device().is_cpu(), \"points must be a CPU tensor\");\n\n std::vector grid_size(NDim);\n const int num_points = points.size(0);\n const int num_features = points.size(1);\n\n for (int i = 0; i < NDim; ++i) {\n grid_size[i] =\n round((coors_range[NDim + i] - coors_range[i]) / voxel_size[i]);\n }\n\n // coors, num_points_per_voxel, coor_to_voxelidx are int Tensor\n // printf(\"cpu coor_to_voxelidx size: [%d, %d, %d]\\n\", grid_size[2],\n // grid_size[1], grid_size[0]);\n at::Tensor coor_to_voxelidx =\n -at::ones({grid_size[2], grid_size[1], grid_size[0]}, coors.options());\n\n int voxel_num = 0;\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n points.scalar_type(), \"hard_voxelize_forward_cpu_kernel\", [&] {\n hard_voxelize_forward_cpu_kernel(\n points.accessor(), voxels.accessor(),\n coors.accessor(), num_points_per_voxel.accessor(),\n coor_to_voxelidx.accessor(), voxel_num, voxel_size,\n coors_range, grid_size, max_points, max_voxels, num_points,\n num_features, NDim);\n });\n\n return voxel_num;\n}\n\nint hard_voxelize_forward_impl(const at::Tensor& points, at::Tensor& voxels,\n at::Tensor& coors,\n at::Tensor& num_points_per_voxel,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int max_points, const int max_voxels,\n const int NDim);\n\nvoid dynamic_voxelize_forward_impl(const at::Tensor& points, at::Tensor& coors,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int NDim);\nREGISTER_DEVICE_IMPL(hard_voxelize_forward_impl, CPU,\n hard_voxelize_forward_cpu);\nREGISTER_DEVICE_IMPL(dynamic_voxelize_forward_impl, CPU,\n dynamic_voxelize_forward_cpu);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/active_rotated_filter_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/csuhan/s2anet/blob/master/mmdet/ops/orn/src/cuda/ActiveRotatingFilter_cuda.cu\n#include \"active_rotated_filter_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid ActiveRotatedFilterForwardCUDAKernelLauncher(const Tensor input,\n const Tensor indices,\n Tensor output) {\n int num_output_planes = input.size(0);\n int num_input_planes = input.size(1);\n int num_orientations = input.size(2);\n int kH = input.size(3);\n int kW = input.size(4);\n int num_rotations = indices.size(3);\n int nEntry = num_orientations * kH * kW;\n int output_size = input.numel();\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"active_rotated_filter_forward_cuda_kernel\", [&] {\n active_rotated_filter_forward_cuda_kernel\n <<>>(\n output_size, input.data_ptr(),\n indices.data_ptr(), num_input_planes, num_output_planes,\n num_orientations, num_rotations, nEntry,\n output.data_ptr());\n });\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid ActiveRotatedFilterBackwardCUDAKernelLauncher(const Tensor grad_out,\n const Tensor indices,\n Tensor grad_in) {\n int num_orientations = indices.size(0);\n int kH = indices.size(1);\n int kW = indices.size(2);\n int num_rotations = indices.size(3);\n int num_output_planes = grad_out.size(0) / num_rotations;\n int num_input_planes = grad_out.size(1) / num_orientations;\n int nEntry = num_orientations * kH * kW;\n int output_size = grad_in.numel();\n\n at::cuda::CUDAGuard device_guard(indices.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_out.scalar_type(), \"active_rotated_filter_backward_cuda_kernel\",\n [&] {\n active_rotated_filter_backward_cuda_kernel\n <<>>(\n output_size, grad_out.data_ptr(),\n indices.data_ptr(), num_input_planes, num_output_planes,\n num_orientations, num_rotations, nEntry,\n grad_in.data_ptr());\n });\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/assign_score_withk_cuda.cu", "content": "// Modified from\n// https://github.com/CVMI-Lab/PAConv/tree/main/scene_seg/lib/paconv_lib/src/gpu\n#include \n#include \n\n#include \"assign_score_withk_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid AssignScoreWithKForwardCUDAKernelLauncher(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& points, const Tensor& centers, const Tensor& scores,\n const Tensor& knn_idx, Tensor& output) {\n at::cuda::CUDAGuard device_guard(points.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n dim3 blocks(GET_BLOCKS(B * O * N1 * K, THREADS_PER_BLOCK));\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n points.scalar_type(), \"assign_score_withk_forward_cuda_kernel\", [&] {\n assign_score_withk_forward_cuda_kernel\n <<>>(\n B, N0, N1, M, K, O, aggregate, points.data_ptr(),\n centers.data_ptr(), scores.data_ptr(),\n knn_idx.data_ptr(), output.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid AssignScoreWithKBackwardCUDAKernelLauncher(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores) {\n at::cuda::CUDAGuard device_guard(grad_out.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n dim3 blocks1(GET_BLOCKS(B * M * O, THREADS_PER_BLOCK));\n dim3 threads1(THREADS_PER_BLOCK);\n dim3 blocks2(GET_BLOCKS(B * N1 * K * M, THREADS_PER_BLOCK));\n dim3 threads2(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_out.scalar_type(), \"assign_score_withk_points_backward_cuda_kernel\",\n [&] {\n assign_score_withk_points_backward_cuda_kernel\n <<>>(\n B, N0, N1, M, K, O, aggregate, grad_out.data_ptr(),\n scores.data_ptr(), knn_idx.data_ptr(),\n grad_points.data_ptr(),\n grad_centers.data_ptr());\n });\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_out.scalar_type(), \"assign_score_withk_scores_backward_cuda_kernel\",\n [&] {\n assign_score_withk_scores_backward_cuda_kernel\n <<>>(\n B, N0, N1, M, K, O, aggregate, grad_out.data_ptr(),\n points.data_ptr(), centers.data_ptr(),\n knn_idx.data_ptr(), grad_scores.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/ball_query_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/ball_query_gpu.cu\n\n#include \n#include \n#include \n\n#include \"ball_query_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid BallQueryForwardCUDAKernelLauncher(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx) {\n // new_xyz: (B, M, 3)\n // xyz: (B, N, 3)\n // output:\n // idx: (B, M, nsample)\n\n at::cuda::CUDAGuard device_guard(new_xyz.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(m, THREADS_PER_BLOCK), b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n new_xyz.scalar_type(), \"ball_query_forward_cuda_kernel\", [&] {\n ball_query_forward_cuda_kernel\n <<>>(\n b, n, m, min_radius, max_radius, nsample,\n new_xyz.data_ptr(), xyz.data_ptr(),\n idx.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/bbox_overlaps_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"bbox_overlaps_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid BBoxOverlapsCUDAKernelLauncher(const Tensor bboxes1, const Tensor bboxes2,\n Tensor ious, const int mode,\n const bool aligned, const int offset) {\n int output_size = ious.numel();\n int num_bbox1 = bboxes1.size(0);\n int num_bbox2 = bboxes2.size(0);\n\n at::cuda::CUDAGuard device_guard(bboxes1.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n bboxes1.scalar_type(), \"bbox_overlaps_cuda_kernel\", ([&] {\n bbox_overlaps_cuda_kernel\n <<>>(\n bboxes1.data_ptr(), bboxes2.data_ptr(),\n ious.data_ptr(), num_bbox1, num_bbox2, mode, aligned,\n offset);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/border_align_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"border_align_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid BorderAlignForwardCUDAKernelLauncher(const Tensor &input,\n const Tensor &boxes, Tensor output,\n Tensor argmax_idx,\n const int pool_size) {\n // shape assertion\n AT_ASSERTM(input.ndimension() == 4,\n \"non-empty 4D(batch mode) tensor expected for input feature\");\n AT_ASSERTM(boxes.ndimension() == 3,\n \"boxes must be 3D tensor with size of [B, H*W, 4]\");\n\n int batch_size = input.size(0);\n int feat_channels = input.size(1);\n int channels = feat_channels / 4;\n int height = input.size(2);\n int width = input.size(3);\n // shape [N, box_size, 4] for boxes. (x1, y1, x2, y2) format\n int box_size = boxes.size(1);\n // shape [N, channels, box_size, 4] for output\n int nthreads = batch_size * channels * box_size;\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n dim3 block(128, 4);\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"border_align_forward_cuda_kernel\", [&] {\n border_align_forward_cuda_kernel\n <<>>(\n nthreads, input.data_ptr(),\n boxes.data_ptr(), output.data_ptr(),\n argmax_idx.data_ptr(), channels, box_size, height, width,\n pool_size);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid BorderAlignBackwardCUDAKernelLauncher(const Tensor &grad_output,\n const Tensor &boxes,\n const Tensor &argmax_idx,\n Tensor grad_input,\n const int pool_size) {\n int batch_size = grad_input.size(0);\n int feat_channels = grad_input.size(1);\n int channels = feat_channels / 4;\n int height = grad_input.size(2);\n int width = grad_input.size(3);\n int box_size = boxes.size(1);\n int nthreads = batch_size * channels * box_size;\n\n at::cuda::CUDAGuard device_guard(grad_output.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n dim3 block(128, 4);\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"border_align_backward_cuda_kernel\", [&] {\n border_align_backward_cuda_kernel\n <<>>(\n nthreads, grad_output.data_ptr(),\n boxes.data_ptr(), argmax_idx.data_ptr(),\n grad_input.data_ptr(), channels, box_size, height,\n width, pool_size);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/box_iou_rotated_cuda.cu", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated_cuda.cu\n#include \"box_iou_rotated_cuda.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid box_iou_rotated_cuda(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned) {\n using scalar_t = float;\n AT_ASSERTM(boxes1.is_cuda(), \"boxes1 must be a CUDA tensor\");\n AT_ASSERTM(boxes2.is_cuda(), \"boxes2 must be a CUDA tensor\");\n\n int output_size = ious.numel();\n int num_boxes1 = boxes1.size(0);\n int num_boxes2 = boxes2.size(0);\n\n at::cuda::CUDAGuard device_guard(boxes1.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n box_iou_rotated_cuda_kernel\n <<>>(\n num_boxes1, num_boxes2, boxes1.data_ptr(),\n boxes2.data_ptr(), (scalar_t*)ious.data_ptr(),\n mode_flag, aligned);\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/carafe_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"carafe_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid CARAFEForwardCUDAKernelLauncher(const Tensor features, const Tensor masks,\n Tensor rfeatures, Tensor routput,\n Tensor rmasks, Tensor output,\n const int kernel_size,\n const int group_size,\n const int scale_factor) {\n const int batch_size = output.size(0);\n const int channels = output.size(1);\n const int output_height = output.size(2);\n const int output_width = output.size(3);\n\n const int input_height = features.size(2);\n const int input_width = features.size(3);\n\n const int mask_channels = masks.size(1);\n\n rfeatures.resize_({batch_size, input_height, input_width, channels});\n routput.resize_({batch_size, output_height, output_width, channels});\n rmasks.resize_({batch_size, output_height, output_width, mask_channels});\n\n // one warp per pixel\n at::cuda::CUDAGuard device_guard(features.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n features.scalar_type(), \"NCHW2NHWC_Feature\", ([&] {\n const scalar_t *bottom_data = features.data_ptr();\n scalar_t *top_data = rfeatures.data_ptr();\n const int dh = divideUP(channels, kTileDim);\n const int dw = divideUP(input_height * input_width, kTileDim);\n BatchTranspose2DCUDAKernel\n <<>>(\n batch_size, channels, input_height * input_width, dh, dw,\n bottom_data, top_data);\n }));\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n features.scalar_type(), \"NCHW2NHWC_Masks\", ([&] {\n const scalar_t *bottom_data = masks.data_ptr();\n scalar_t *top_data = rmasks.data_ptr();\n const int dh = divideUP(mask_channels, kTileDim);\n const int dw = divideUP(output_height * output_width, kTileDim);\n BatchTranspose2DCUDAKernel\n <<>>(\n batch_size, mask_channels, output_height * output_width, dh, dw,\n bottom_data, top_data);\n }));\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n features.scalar_type(), \"CARAFELaucherForward\", ([&] {\n const int num_kernels =\n batch_size * output_height * output_width * THREADS_PER_PIXEL;\n const scalar_t *bottom_data = rfeatures.data_ptr();\n const scalar_t *bottom_masks = rmasks.data_ptr();\n scalar_t *top_data = routput.data_ptr();\n\n CARAFEForward<<>>(\n num_kernels, bottom_data, bottom_masks, kernel_size, group_size,\n scale_factor, channels, input_height, input_width, output_height,\n output_width, mask_channels, top_data);\n }));\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n features.scalar_type(), \"NHWC2NCHW\", ([&] {\n const scalar_t *bottom_data = routput.data_ptr();\n scalar_t *top_data = output.data_ptr();\n const int dh = divideUP(output_height * output_width, kTileDim);\n const int dw = divideUP(channels, kTileDim);\n BatchTranspose2DCUDAKernel\n <<>>(\n batch_size, output_height * output_width, channels, dh, dw,\n bottom_data, top_data);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid CARAFEBackwardCUDAKernelLauncher(\n const Tensor top_grad, const Tensor rfeatures, const Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs, Tensor rbottom_grad,\n Tensor rmask_grad, Tensor bottom_grad, Tensor mask_grad,\n const int kernel_size, const int group_size, const int scale_factor) {\n const int batch_size = top_grad.size(0);\n const int channels = top_grad.size(1);\n const int output_height = top_grad.size(2);\n const int output_width = top_grad.size(3);\n\n const int input_height = bottom_grad.size(2);\n const int input_width = bottom_grad.size(3);\n\n const int mask_channels = masks.size(1);\n\n rtop_grad.resize_({batch_size, output_height, output_width, channels});\n rbottom_grad.resize_({batch_size, input_height, input_width, channels});\n rbottom_grad_hs.resize_({batch_size, output_height, output_width, channels});\n rmask_grad.resize_({batch_size, output_height, output_width, mask_channels});\n\n at::cuda::CUDAGuard device_guard(top_grad.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"NCHW2NHWC_Top_Grad\", ([&] {\n const scalar_t *bottom_data = top_grad.data_ptr();\n scalar_t *top_data = rtop_grad.data_ptr();\n const int dh = divideUP(channels, kTileDim);\n const int dw = divideUP(output_height * output_width, kTileDim);\n BatchTranspose2DCUDAKernel\n <<>>(\n batch_size, channels, output_height * output_width, dh, dw,\n bottom_data, top_data);\n }));\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"CARAFELaucherBackward_Feature\", ([&] {\n const int num_kernels =\n batch_size * output_height * output_width * THREADS_PER_PIXEL;\n const scalar_t *top_diff = rtop_grad.data_ptr();\n const scalar_t *bottom_masks = masks.data_ptr();\n scalar_t *bottom_diff = rbottom_grad_hs.data_ptr();\n\n CARAFEBackward_Feature\n <<>>(num_kernels, top_diff, bottom_masks, kernel_size,\n group_size, scale_factor, channels, input_height,\n input_width, output_height, output_width,\n mask_channels, bottom_diff);\n }));\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"FeatureSum\", ([&] {\n const int num_kernels =\n batch_size * input_height * input_width * THREADS_PER_PIXEL;\n const scalar_t *bottom_diff_hs = rbottom_grad_hs.data_ptr();\n scalar_t *bottom_diff = rbottom_grad.data_ptr();\n\n FeatureSum\n <<>>(num_kernels, bottom_diff_hs, scale_factor, channels,\n input_height, input_width, bottom_diff);\n }));\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"NHWC2NCHW_Bottom_Grad\", ([&] {\n const scalar_t *bottom_data = rbottom_grad.data_ptr();\n scalar_t *top_data = bottom_grad.data_ptr();\n const int dh = divideUP(input_height * input_width, kTileDim);\n const int dw = divideUP(channels, kTileDim);\n BatchTranspose2DCUDAKernel\n <<>>(\n batch_size, input_height * input_width, channels, dh, dw,\n bottom_data, top_data);\n }));\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"CARAFELaucherBackward_Mask\", ([&] {\n const int num_kernels = batch_size * output_height * output_width *\n mask_channels * WARP_SIZE;\n const scalar_t *top_diff = rtop_grad.data_ptr();\n const scalar_t *bottom_data = rfeatures.data_ptr();\n scalar_t *mask_diff = rmask_grad.data_ptr();\n\n CARAFEBackward_Mask\n <<>>(num_kernels, top_diff, bottom_data, kernel_size,\n group_size, scale_factor, channels, input_height,\n input_width, output_height, output_width,\n mask_channels, mask_diff);\n }));\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"NHWC2NCHW_Mask_Grad\", ([&] {\n const scalar_t *bottom_data = rmask_grad.data_ptr();\n scalar_t *top_data = mask_grad.data_ptr();\n const int dh = divideUP(output_height * output_width, kTileDim);\n const int dw = divideUP(mask_channels, kTileDim);\n BatchTranspose2DCUDAKernel\n <<>>(\n batch_size, output_height * output_width, mask_channels, dh, dw,\n bottom_data, top_data);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/carafe_naive_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"carafe_naive_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid CARAFENAIVEForwardCUDAKernelLauncher(const Tensor features,\n const Tensor masks, Tensor output,\n const int kernel_size,\n const int group_size,\n const int scale_factor) {\n int output_size = output.numel();\n int channels = output.size(1);\n int height = output.size(2);\n int width = output.size(3);\n\n at::cuda::CUDAGuard device_guard(features.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n features.scalar_type(), \"CARAFENAIVEForward\", ([&] {\n carafe_naive_forward_cuda_kernel\n <<>>(\n output_size, features.data_ptr(),\n masks.data_ptr(), output.data_ptr(),\n kernel_size, group_size, scale_factor, channels, height, width);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid CARAFENAIVEBackwardCUDAKernelLauncher(\n const Tensor top_grad, const Tensor features, const Tensor masks,\n Tensor bottom_grad, Tensor mask_grad, const int kernel_size,\n const int group_size, const int scale_factor) {\n int output_size = top_grad.numel();\n int channels = top_grad.size(1);\n int height = top_grad.size(2);\n int width = top_grad.size(3);\n\n at::cuda::CUDAGuard device_guard(top_grad.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"CARAFENAIVEBackward\", ([&] {\n carafe_naive_backward_cuda_kernel\n <<>>(\n output_size, top_grad.data_ptr(),\n features.data_ptr(), masks.data_ptr(),\n bottom_grad.data_ptr(),\n mask_grad.data_ptr(), kernel_size, group_size,\n scale_factor, channels, height, width);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/convex_iou.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// modified from\n// https://github.com/SDL-GuoZonghao/BeyondBoundingBox/blob/main/mmdet/ops/iou/src/convex_iou_kernel.cu\n#include \"convex_iou_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid ConvexIoUCUDAKernelLauncher(const Tensor pointsets, const Tensor polygons,\n Tensor ious) {\n int output_size = ious.numel();\n int num_pointsets = pointsets.size(0);\n int num_polygons = polygons.size(0);\n\n at::cuda::CUDAGuard device_guard(pointsets.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n pointsets.scalar_type(), \"convex_iou_cuda_kernel\", ([&] {\n convex_iou_cuda_kernel\n <<>>(\n num_pointsets, num_polygons, pointsets.data_ptr(),\n polygons.data_ptr(), ious.data_ptr());\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid ConvexGIoUCUDAKernelLauncher(const Tensor pointsets, const Tensor polygons,\n Tensor output) {\n int output_size = output.numel();\n int num_pointsets = pointsets.size(0);\n int num_polygons = polygons.size(0);\n\n at::cuda::CUDAGuard device_guard(pointsets.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n pointsets.scalar_type(), \"convex_giou_cuda_kernel\", ([&] {\n convex_giou_cuda_kernel\n <<>>(\n num_pointsets, num_polygons, pointsets.data_ptr(),\n polygons.data_ptr(), output.data_ptr());\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/correlation_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/ClementPinard/Pytorch-Correlation-extension/blob/master/Correlation_Module/correlation_cuda_kernel.cu\n// Original licence: Under MIT License\n\n#include \"correlation_cuda.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid CorrelationForwardCUDAKernelLauncher(Tensor input1, Tensor input2,\n Tensor output, int kH, int kW,\n int patchH, int patchW, int padH,\n int padW, int dilationH,\n int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW) {\n const int batch_size = input1.size(0);\n const int iH = input1.size(2);\n const int iW = input1.size(3);\n const int dilatedKH = (kH - 1) * dilationH + 1;\n const int dilatedKW = (kW - 1) * dilationW + 1;\n\n const auto oH = (iH + 2 * padH - dilatedKH) / dH + 1;\n const auto oW = (iW + 2 * padW - dilatedKW) / dW + 1;\n\n auto trInput1 = input1.permute({0, 2, 3, 1}).contiguous();\n auto trInput2 = input2.permute({0, 2, 3, 1}).contiguous();\n\n const int threads = THREADS_FORWARD;\n const dim3 blocks(batch_size, oH, oW);\n\n at::cuda::CUDAGuard device_guard(input1.device());\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input1.scalar_type(), \"correlation_forward_cuda\", ([&] {\n TensorAcc4R trInput1_acc =\n trInput1.packed_accessor32();\n TensorAcc4R trInput2_acc =\n trInput2.packed_accessor32();\n TensorAcc5R output_acc =\n output.packed_accessor32();\n\n correlation_forward_cuda_kernel\n <<>>(\n trInput1_acc, trInput2_acc, output_acc, kH, kW, patchH, patchW,\n padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n }));\n}\n\nvoid CorrelationBackwardCUDAKernelLauncher(\n Tensor grad_output, Tensor input1, Tensor input2, Tensor grad_input1,\n Tensor grad_input2, int kH, int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW) {\n const int batch_size = input1.size(0);\n const int iH = input1.size(2);\n const int iW = input1.size(3);\n const int C = input1.size(1);\n\n const dim3 blocks(C, iH, iW);\n const dim3 threads(THREADS_BACKWARD, THREADS_BACKWARD);\n\n at::cuda::CUDAGuard device_guard(input1.device());\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input1.scalar_type(), \"correlation_backward_cuda\", ([&] {\n TensorAcc4R input1_acc =\n input1.packed_accessor32();\n TensorAcc4R input2_acc =\n input2.packed_accessor32();\n TensorAcc4R grad_input1_acc =\n grad_input1.packed_accessor32();\n TensorAcc4R grad_input2_acc =\n grad_input2.packed_accessor32();\n TensorAcc5R grad_output_acc =\n grad_output.packed_accessor32();\n\n for (int n = 0; n < batch_size; ++n) {\n correlation_backward_cuda_kernel_input1\n <<>>(\n grad_output_acc, input2_acc, grad_input1_acc, kH, kW, patchH,\n patchW, padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW, n);\n }\n\n for (int n = 0; n < batch_size; ++n) {\n correlation_backward_cuda_kernel_input2\n <<>>(\n grad_output_acc, input1_acc, grad_input2_acc, kH, kW, patchH,\n patchW, padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW, n);\n }\n }));\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/cudabind.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid AssignScoreWithKForwardCUDAKernelLauncher(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& points, const Tensor& centers, const Tensor& scores,\n const Tensor& knn_idx, Tensor& output);\n\nvoid AssignScoreWithKBackwardCUDAKernelLauncher(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores);\n\nvoid assign_score_withk_forward_cuda(int B, int N0, int N1, int M, int K, int O,\n int aggregate, const Tensor& points,\n const Tensor& centers,\n const Tensor& scores,\n const Tensor& knn_idx, Tensor& output) {\n AssignScoreWithKForwardCUDAKernelLauncher(\n B, N0, N1, M, K, O, aggregate, points, centers, scores, knn_idx, output);\n};\n\nvoid assign_score_withk_backward_cuda(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores) {\n AssignScoreWithKBackwardCUDAKernelLauncher(\n B, N0, N1, M, K, O, aggregate, grad_out, points, centers, scores, knn_idx,\n grad_points, grad_centers, grad_scores);\n};\n\nvoid assign_score_withk_forward_impl(int B, int N0, int N1, int M, int K, int O,\n int aggregate, const Tensor& points,\n const Tensor& centers,\n const Tensor& scores,\n const Tensor& knn_idx, Tensor& output);\n\nvoid assign_score_withk_backward_impl(\n int B, int N0, int N1, int M, int K, int O, int aggregate,\n const Tensor& grad_out, const Tensor& points, const Tensor& centers,\n const Tensor& scores, const Tensor& knn_idx, Tensor& grad_points,\n Tensor& grad_centers, Tensor& grad_scores);\n\nREGISTER_DEVICE_IMPL(assign_score_withk_forward_impl, CUDA,\n assign_score_withk_forward_cuda);\nREGISTER_DEVICE_IMPL(assign_score_withk_backward_impl, CUDA,\n assign_score_withk_backward_cuda);\n\nvoid BallQueryForwardCUDAKernelLauncher(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx);\n\nvoid ball_query_forward_cuda(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx) {\n BallQueryForwardCUDAKernelLauncher(b, n, m, min_radius, max_radius, nsample,\n new_xyz, xyz, idx);\n};\n\nvoid ball_query_forward_impl(int b, int n, int m, float min_radius,\n float max_radius, int nsample,\n const Tensor new_xyz, const Tensor xyz,\n Tensor idx);\nREGISTER_DEVICE_IMPL(ball_query_forward_impl, CUDA, ball_query_forward_cuda);\n\nvoid BBoxOverlapsCUDAKernelLauncher(const Tensor bboxes1, const Tensor bboxes2,\n Tensor ious, const int mode,\n const bool aligned, const int offset);\n\nvoid bbox_overlaps_cuda(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset) {\n BBoxOverlapsCUDAKernelLauncher(bboxes1, bboxes2, ious, mode, aligned, offset);\n}\n\nvoid bbox_overlaps_impl(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset);\nREGISTER_DEVICE_IMPL(bbox_overlaps_impl, CUDA, bbox_overlaps_cuda);\n\nvoid BorderAlignForwardCUDAKernelLauncher(const Tensor& input,\n const Tensor& boxes, Tensor output,\n Tensor argmax_idx,\n const int pool_size);\n\nvoid BorderAlignBackwardCUDAKernelLauncher(const Tensor& grad_output,\n const Tensor& boxes,\n const Tensor& argmax_idx,\n Tensor grad_input,\n const int pool_size);\n\nvoid border_align_forward_cuda(const Tensor& input, const Tensor& boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size) {\n BorderAlignForwardCUDAKernelLauncher(input, boxes, output, argmax_idx,\n pool_size);\n}\n\nvoid border_align_backward_cuda(const Tensor& grad_output, const Tensor& boxes,\n const Tensor& argmax_idx, Tensor grad_input,\n const int pool_size) {\n BorderAlignBackwardCUDAKernelLauncher(grad_output, boxes, argmax_idx,\n grad_input, pool_size);\n}\n\nvoid border_align_forward_impl(const Tensor& input, const Tensor& boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size);\n\nvoid border_align_backward_impl(const Tensor& grad_output, const Tensor& boxes,\n const Tensor& argmax_idx, Tensor grad_input,\n const int pool_size);\n\nREGISTER_DEVICE_IMPL(border_align_forward_impl, CUDA,\n border_align_forward_cuda);\nREGISTER_DEVICE_IMPL(border_align_backward_impl, CUDA,\n border_align_backward_cuda);\n\nvoid box_iou_rotated_cuda(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned);\n\nvoid box_iou_rotated_impl(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned);\nREGISTER_DEVICE_IMPL(box_iou_rotated_impl, CUDA, box_iou_rotated_cuda);\n\nvoid CARAFEForwardCUDAKernelLauncher(const Tensor features, const Tensor masks,\n Tensor rfeatures, Tensor routput,\n Tensor rmasks, Tensor output,\n const int kernel_size,\n const int group_size,\n const int scale_factor);\n\nvoid CARAFEBackwardCUDAKernelLauncher(\n const Tensor top_grad, const Tensor rfeatures, const Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs, Tensor rbottom_grad,\n Tensor rmask_grad, Tensor bottom_grad, Tensor mask_grad,\n const int kernel_size, const int group_size, const int scale_factor);\n\nvoid carafe_forward_cuda(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor) {\n CARAFEForwardCUDAKernelLauncher(features, masks, rfeatures, routput, rmasks,\n output, kernel_size, group_size,\n scale_factor);\n}\n\nvoid carafe_backward_cuda(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad,\n Tensor bottom_grad, Tensor mask_grad, int kernel_size,\n int group_size, int scale_factor) {\n CARAFEBackwardCUDAKernelLauncher(top_grad, rfeatures, masks, rtop_grad,\n rbottom_grad_hs, rbottom_grad, rmask_grad,\n bottom_grad, mask_grad, kernel_size,\n group_size, scale_factor);\n}\n\nvoid carafe_forward_impl(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor);\n\nvoid carafe_backward_impl(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad,\n Tensor bottom_grad, Tensor mask_grad, int kernel_size,\n int group_size, int scale_factor);\n\nREGISTER_DEVICE_IMPL(carafe_forward_impl, CUDA, carafe_forward_cuda);\nREGISTER_DEVICE_IMPL(carafe_backward_impl, CUDA, carafe_backward_cuda);\n\nvoid CARAFENAIVEForwardCUDAKernelLauncher(const Tensor features,\n const Tensor masks, Tensor output,\n const int kernel_size,\n const int group_size,\n const int scale_factor);\n\nvoid CARAFENAIVEBackwardCUDAKernelLauncher(\n const Tensor top_grad, const Tensor features, const Tensor masks,\n Tensor bottom_grad, Tensor mask_grad, const int kernel_size,\n const int group_size, const int scale_factor);\n\nvoid carafe_naive_forward_cuda(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size,\n int scale_factor) {\n CARAFENAIVEForwardCUDAKernelLauncher(features, masks, output, kernel_size,\n group_size, scale_factor);\n}\n\nvoid carafe_naive_backward_cuda(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size,\n int scale_factor) {\n CARAFENAIVEBackwardCUDAKernelLauncher(top_grad, features, masks, bottom_grad,\n mask_grad, kernel_size, group_size,\n scale_factor);\n}\nvoid carafe_naive_forward_impl(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size,\n int scale_factor);\n\nvoid carafe_naive_backward_impl(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size,\n int scale_factor);\n\nREGISTER_DEVICE_IMPL(carafe_naive_forward_impl, CUDA,\n carafe_naive_forward_cuda);\nREGISTER_DEVICE_IMPL(carafe_naive_backward_impl, CUDA,\n carafe_naive_backward_cuda);\n\nvoid CorrelationForwardCUDAKernelLauncher(Tensor input1, Tensor input2,\n Tensor output, int kH, int kW,\n int patchH, int patchW, int padH,\n int padW, int dilationH,\n int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW);\n\nvoid CorrelationBackwardCUDAKernelLauncher(Tensor grad_output, Tensor input1,\n Tensor input2, Tensor grad_input1,\n Tensor grad_input2, int kH, int kW,\n int patchH, int patchW, int padH,\n int padW, int dilationH,\n int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW);\n\nvoid correlation_forward_cuda(Tensor input1, Tensor input2, Tensor output,\n int kH, int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW) {\n CorrelationForwardCUDAKernelLauncher(\n input1, input2, output, kH, kW, patchH, patchW, padH, padW, dilationH,\n dilationW, dilation_patchH, dilation_patchW, dH, dW);\n}\n\nvoid correlation_backward_cuda(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW) {\n CorrelationBackwardCUDAKernelLauncher(\n grad_output, input1, input2, grad_input1, grad_input2, kH, kW, patchH,\n patchW, padH, padW, dilationH, dilationW, dilation_patchH,\n dilation_patchW, dH, dW);\n}\n\nvoid correlation_forward_impl(Tensor input1, Tensor input2, Tensor output,\n int kH, int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW);\n\nvoid correlation_backward_impl(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH,\n int padW, int dilationH, int dilationW,\n int dilation_patchH, int dilation_patchW, int dH,\n int dW);\n\nREGISTER_DEVICE_IMPL(correlation_forward_impl, CUDA, correlation_forward_cuda);\nREGISTER_DEVICE_IMPL(correlation_backward_impl, CUDA,\n correlation_backward_cuda);\n\nvoid deformable_im2col_cuda(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col);\n\nvoid deformable_col2im_cuda(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im);\n\nvoid deformable_col2im_coord_cuda(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset);\n\nvoid deformable_im2col_impl(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col);\n\nvoid deformable_col2im_impl(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im);\n\nvoid deformable_col2im_coord_impl(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset);\n\nREGISTER_DEVICE_IMPL(deformable_im2col_impl, CUDA, deformable_im2col_cuda);\nREGISTER_DEVICE_IMPL(deformable_col2im_impl, CUDA, deformable_col2im_cuda);\nREGISTER_DEVICE_IMPL(deformable_col2im_coord_impl, CUDA,\n deformable_col2im_coord_cuda);\n\nvoid DeformRoIPoolForwardCUDAKernelLauncher(Tensor input, Tensor rois,\n Tensor offset, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale,\n int sampling_ratio, float gamma);\n\nvoid DeformRoIPoolBackwardCUDAKernelLauncher(\n Tensor grad_output, Tensor input, Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset, int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio, float gamma);\n\nvoid deform_roi_pool_forward_cuda(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma) {\n DeformRoIPoolForwardCUDAKernelLauncher(input, rois, offset, output,\n pooled_height, pooled_width,\n spatial_scale, sampling_ratio, gamma);\n}\n\nvoid deform_roi_pool_backward_cuda(Tensor grad_output, Tensor input,\n Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset,\n int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma) {\n DeformRoIPoolBackwardCUDAKernelLauncher(\n grad_output, input, rois, offset, grad_input, grad_offset, pooled_height,\n pooled_width, spatial_scale, sampling_ratio, gamma);\n}\n\nvoid deform_roi_pool_forward_impl(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma);\n\nvoid deform_roi_pool_backward_impl(Tensor grad_output, Tensor input,\n Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset,\n int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma);\n\nREGISTER_DEVICE_IMPL(deform_roi_pool_forward_impl, CUDA,\n deform_roi_pool_forward_cuda);\nREGISTER_DEVICE_IMPL(deform_roi_pool_backward_impl, CUDA,\n deform_roi_pool_backward_cuda);\n\nvoid SigmoidFocalLossForwardCUDAKernelLauncher(Tensor input, Tensor target,\n Tensor weight, Tensor output,\n const float gamma,\n const float alpha);\n\nvoid SigmoidFocalLossBackwardCUDAKernelLauncher(Tensor input, Tensor target,\n Tensor weight,\n Tensor grad_input,\n const float gamma,\n const float alpha);\n\nvoid SoftmaxFocalLossForwardCUDAKernelLauncher(Tensor softmax, Tensor target,\n Tensor weight, Tensor output,\n const float gamma,\n const float alpha);\n\nvoid SoftmaxFocalLossBackwardCUDAKernelLauncher(Tensor softmax, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input,\n const float gamma,\n const float alpha);\n\nvoid sigmoid_focal_loss_forward_cuda(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n SigmoidFocalLossForwardCUDAKernelLauncher(input, target, weight, output,\n gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_backward_cuda(Tensor input, Tensor target,\n Tensor weight, Tensor grad_input,\n float gamma, float alpha) {\n SigmoidFocalLossBackwardCUDAKernelLauncher(input, target, weight, grad_input,\n gamma, alpha);\n}\n\nvoid softmax_focal_loss_forward_cuda(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n SoftmaxFocalLossForwardCUDAKernelLauncher(input, target, weight, output,\n gamma, alpha);\n}\n\nvoid softmax_focal_loss_backward_cuda(Tensor input, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input, float gamma,\n float alpha) {\n SoftmaxFocalLossBackwardCUDAKernelLauncher(input, target, weight, buff,\n grad_input, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_forward_impl(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha);\n\nvoid sigmoid_focal_loss_backward_impl(Tensor input, Tensor target,\n Tensor weight, Tensor grad_input,\n float gamma, float alpha);\n\nvoid softmax_focal_loss_forward_impl(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha);\n\nvoid softmax_focal_loss_backward_impl(Tensor input, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input, float gamma,\n float alpha);\n\nREGISTER_DEVICE_IMPL(sigmoid_focal_loss_forward_impl, CUDA,\n sigmoid_focal_loss_forward_cuda);\nREGISTER_DEVICE_IMPL(sigmoid_focal_loss_backward_impl, CUDA,\n sigmoid_focal_loss_backward_cuda);\nREGISTER_DEVICE_IMPL(softmax_focal_loss_forward_impl, CUDA,\n softmax_focal_loss_forward_cuda);\nREGISTER_DEVICE_IMPL(softmax_focal_loss_backward_impl, CUDA,\n softmax_focal_loss_backward_cuda);\n\nvoid FurthestPointSamplingForwardCUDAKernelLauncher(int b, int n, int m,\n const float* dataset,\n float* temp, int* idxs);\n\nvoid FurthestPointSamplingWithDistForwardCUDAKernelLauncher(\n int b, int n, int m, const float* dataset, float* temp, int* idxs);\n\nvoid furthest_point_sampling_forward_cuda(Tensor points_tensor,\n Tensor temp_tensor, Tensor idx_tensor,\n int b, int n, int m) {\n const float* dataset = points_tensor.data_ptr();\n float* temp = temp_tensor.data_ptr();\n int* idxs = idx_tensor.data_ptr();\n FurthestPointSamplingForwardCUDAKernelLauncher(b, n, m, dataset, temp, idxs);\n}\n\nvoid furthest_point_sampling_with_dist_forward_cuda(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b,\n int n, int m) {\n const float* dataset = points_tensor.data_ptr();\n float* temp = temp_tensor.data_ptr();\n int* idxs = idx_tensor.data_ptr();\n FurthestPointSamplingWithDistForwardCUDAKernelLauncher(b, n, m, dataset, temp,\n idxs);\n}\n\nvoid furthest_point_sampling_forward_impl(Tensor points_tensor,\n Tensor temp_tensor, Tensor idx_tensor,\n int b, int n, int m);\n\nvoid furthest_point_sampling_with_dist_forward_impl(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b,\n int n, int m);\n\nREGISTER_DEVICE_IMPL(furthest_point_sampling_forward_impl, CUDA,\n furthest_point_sampling_forward_cuda);\nREGISTER_DEVICE_IMPL(furthest_point_sampling_with_dist_forward_impl, CUDA,\n furthest_point_sampling_with_dist_forward_cuda);\n\ntorch::Tensor fused_bias_leakyrelu_op(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale);\n\ntorch::Tensor fused_bias_leakyrelu_op_impl(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale);\nREGISTER_DEVICE_IMPL(fused_bias_leakyrelu_op_impl, CUDA,\n fused_bias_leakyrelu_op);\n\nvoid GatherPointsForwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n const Tensor points,\n const Tensor idx, Tensor out);\n\nvoid GatherPointsBackwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n const Tensor grad_out,\n const Tensor idx,\n Tensor grad_points);\n\nvoid gather_points_forward_cuda(int b, int c, int n, int npoints,\n const Tensor points, const Tensor idx,\n Tensor out) {\n GatherPointsForwardCUDAKernelLauncher(b, c, n, npoints, points, idx, out);\n};\n\nvoid gather_points_backward_cuda(int b, int c, int n, int npoints,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points) {\n GatherPointsBackwardCUDAKernelLauncher(b, c, n, npoints, grad_out, idx,\n grad_points);\n};\n\nvoid gather_points_forward_impl(int b, int c, int n, int npoints,\n const Tensor points, const Tensor idx,\n Tensor out);\n\nvoid gather_points_backward_impl(int b, int c, int n, int npoints,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points);\n\nREGISTER_DEVICE_IMPL(gather_points_forward_impl, CUDA,\n gather_points_forward_cuda);\nREGISTER_DEVICE_IMPL(gather_points_backward_impl, CUDA,\n gather_points_backward_cuda);\n\nvoid GroupPointsForwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n int nsample, const Tensor points,\n const Tensor idx, Tensor out);\n\nvoid GroupPointsBackwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n int nsample, const Tensor grad_out,\n const Tensor idx,\n Tensor grad_points);\n\nvoid group_points_forward_cuda(int b, int c, int n, int npoints, int nsample,\n const Tensor points, const Tensor idx,\n Tensor out) {\n GroupPointsForwardCUDAKernelLauncher(b, c, n, npoints, nsample, points, idx,\n out);\n};\n\nvoid group_points_backward_cuda(int b, int c, int n, int npoints, int nsample,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points) {\n GroupPointsBackwardCUDAKernelLauncher(b, c, n, npoints, nsample, grad_out,\n idx, grad_points);\n};\n\nvoid group_points_forward_impl(int b, int c, int n, int npoints, int nsample,\n const Tensor points, const Tensor idx,\n Tensor out);\n\nvoid group_points_backward_impl(int b, int c, int n, int npoints, int nsample,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points);\n\nREGISTER_DEVICE_IMPL(group_points_forward_impl, CUDA,\n group_points_forward_cuda);\nREGISTER_DEVICE_IMPL(group_points_backward_impl, CUDA,\n group_points_backward_cuda);\n\nvoid IoU3DBoxesOverlapBevForwardCUDAKernelLauncher(const int num_a,\n const Tensor boxes_a,\n const int num_b,\n const Tensor boxes_b,\n Tensor ans_overlap);\n\nvoid IoU3DBoxesIoUBevForwardCUDAKernelLauncher(const int num_a,\n const Tensor boxes_a,\n const int num_b,\n const Tensor boxes_b,\n Tensor ans_iou);\n\nvoid IoU3DNMSForwardCUDAKernelLauncher(const Tensor boxes,\n unsigned long long* mask, int boxes_num,\n float nms_overlap_thresh);\n\nvoid IoU3DNMSNormalForwardCUDAKernelLauncher(const Tensor boxes,\n unsigned long long* mask,\n int boxes_num,\n float nms_overlap_thresh);\n\nvoid iou3d_boxes_overlap_bev_forward_cuda(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_overlap) {\n IoU3DBoxesOverlapBevForwardCUDAKernelLauncher(num_a, boxes_a, num_b, boxes_b,\n ans_overlap);\n};\n\nvoid iou3d_boxes_iou_bev_forward_cuda(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_iou) {\n IoU3DBoxesIoUBevForwardCUDAKernelLauncher(num_a, boxes_a, num_b, boxes_b,\n ans_iou);\n};\n\nvoid iou3d_nms_forward_cuda(const Tensor boxes, unsigned long long* mask,\n int boxes_num, float nms_overlap_thresh) {\n IoU3DNMSForwardCUDAKernelLauncher(boxes, mask, boxes_num, nms_overlap_thresh);\n};\n\nvoid iou3d_nms_normal_forward_cuda(const Tensor boxes, unsigned long long* mask,\n int boxes_num, float nms_overlap_thresh) {\n IoU3DNMSNormalForwardCUDAKernelLauncher(boxes, mask, boxes_num,\n nms_overlap_thresh);\n};\n\nvoid iou3d_boxes_overlap_bev_forward_impl(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_overlap);\n\nvoid iou3d_boxes_iou_bev_forward_impl(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_iou);\n\nvoid iou3d_nms_forward_impl(const Tensor boxes, unsigned long long* mask,\n int boxes_num, float nms_overlap_thresh);\n\nvoid iou3d_nms_normal_forward_impl(const Tensor boxes, unsigned long long* mask,\n int boxes_num, float nms_overlap_thresh);\n\nREGISTER_DEVICE_IMPL(iou3d_boxes_overlap_bev_forward_impl, CUDA,\n iou3d_boxes_overlap_bev_forward_cuda);\nREGISTER_DEVICE_IMPL(iou3d_boxes_iou_bev_forward_impl, CUDA,\n iou3d_boxes_iou_bev_forward_cuda);\nREGISTER_DEVICE_IMPL(iou3d_nms_forward_impl, CUDA, iou3d_nms_forward_cuda);\nREGISTER_DEVICE_IMPL(iou3d_nms_normal_forward_impl, CUDA,\n iou3d_nms_normal_forward_cuda);\n\nvoid KNNForwardCUDAKernelLauncher(int b, int n, int m, int nsample,\n const Tensor xyz, const Tensor new_xyz,\n Tensor idx, Tensor dist2);\n\nvoid knn_forward_cuda(int b, int n, int m, int nsample, const Tensor xyz,\n const Tensor new_xyz, Tensor idx, Tensor dist2) {\n KNNForwardCUDAKernelLauncher(b, n, m, nsample, xyz, new_xyz, idx, dist2);\n}\n\nvoid knn_forward_impl(int b, int n, int m, int nsample, const Tensor xyz,\n const Tensor new_xyz, Tensor idx, Tensor dist2);\nREGISTER_DEVICE_IMPL(knn_forward_impl, CUDA, knn_forward_cuda);\n\nvoid MaskedIm2colForwardCUDAKernelLauncher(const Tensor bottom_data,\n const Tensor mask_h_idx,\n const Tensor mask_w_idx,\n Tensor top_data, const int kernel_h,\n const int kernel_w, const int pad_h,\n const int pad_w);\n\nvoid MaskedCol2imForwardCUDAKernelLauncher(const Tensor bottom_data,\n const Tensor mask_h_idx,\n const Tensor mask_w_idx,\n Tensor top_data, const int height,\n const int width, const int channels);\n\nvoid masked_im2col_forward_cuda(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w) {\n // im: (n, ic, h, w), kernel size (kh, kw)\n // kernel: (oc, ic * kh * kw), col: (kh * kw * ic, ow * oh)\n MaskedIm2colForwardCUDAKernelLauncher(im, mask_h_idx, mask_w_idx, col,\n kernel_h, kernel_w, pad_h, pad_w);\n}\n\nvoid masked_col2im_forward_cuda(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels) {\n // im: (n, ic, h, w), kernel size (kh, kw)\n // kernel: (oc, ic * kh * kh), col: (kh * kw * ic, ow * oh)\n MaskedCol2imForwardCUDAKernelLauncher(col, mask_h_idx, mask_w_idx, im, height,\n width, channels);\n}\n\nvoid masked_im2col_forward_impl(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w);\n\nvoid masked_col2im_forward_impl(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels);\n\nREGISTER_DEVICE_IMPL(masked_im2col_forward_impl, CUDA,\n masked_im2col_forward_cuda);\nREGISTER_DEVICE_IMPL(masked_col2im_forward_impl, CUDA,\n masked_col2im_forward_cuda);\n\nvoid modulated_deformable_im2col_cuda(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col);\n\nvoid modulated_deformable_col2im_cuda(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im);\n\nvoid modulated_deformable_col2im_coord_cuda(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask);\n\nvoid modulated_deformable_im2col_impl(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col);\n\nvoid modulated_deformable_col2im_impl(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im);\n\nvoid modulated_deformable_col2im_coord_impl(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask);\n\nREGISTER_DEVICE_IMPL(modulated_deformable_im2col_impl, CUDA,\n modulated_deformable_im2col_cuda);\nREGISTER_DEVICE_IMPL(modulated_deformable_col2im_impl, CUDA,\n modulated_deformable_col2im_cuda);\nREGISTER_DEVICE_IMPL(modulated_deformable_col2im_coord_impl, CUDA,\n modulated_deformable_col2im_coord_cuda);\n\nTensor ms_deform_attn_cuda_forward(const Tensor& value,\n const Tensor& spatial_shapes,\n const Tensor& level_start_index,\n const Tensor& sampling_loc,\n const Tensor& attn_weight,\n const int im2col_step);\n\nvoid ms_deform_attn_cuda_backward(\n const Tensor& value, const Tensor& spatial_shapes,\n const Tensor& level_start_index, const Tensor& sampling_loc,\n const Tensor& attn_weight, const Tensor& grad_output, Tensor& grad_value,\n Tensor& grad_sampling_loc, Tensor& grad_attn_weight, const int im2col_step);\n\nTensor ms_deform_attn_impl_forward(const Tensor& value,\n const Tensor& spatial_shapes,\n const Tensor& level_start_index,\n const Tensor& sampling_loc,\n const Tensor& attn_weight,\n const int im2col_step);\n\nvoid ms_deform_attn_impl_backward(\n const Tensor& value, const Tensor& spatial_shapes,\n const Tensor& level_start_index, const Tensor& sampling_loc,\n const Tensor& attn_weight, const Tensor& grad_output, Tensor& grad_value,\n Tensor& grad_sampling_loc, Tensor& grad_attn_weight, const int im2col_step);\n\nREGISTER_DEVICE_IMPL(ms_deform_attn_impl_forward, CUDA,\n ms_deform_attn_cuda_forward);\nREGISTER_DEVICE_IMPL(ms_deform_attn_impl_backward, CUDA,\n ms_deform_attn_cuda_backward);\n\nTensor NMSCUDAKernelLauncher(Tensor boxes, Tensor scores, float iou_threshold,\n int offset);\n\nTensor nms_cuda(Tensor boxes, Tensor scores, float iou_threshold, int offset) {\n return NMSCUDAKernelLauncher(boxes, scores, iou_threshold, offset);\n}\n\nTensor nms_impl(Tensor boxes, Tensor scores, float iou_threshold, int offset);\nREGISTER_DEVICE_IMPL(nms_impl, CUDA, nms_cuda);\n\nvoid PointsInBoxesPartForwardCUDAKernelLauncher(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points);\n\nvoid PointsInBoxesAllForwardCUDAKernelLauncher(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points);\n\nvoid points_in_boxes_part_forward_cuda(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n PointsInBoxesPartForwardCUDAKernelLauncher(batch_size, boxes_num, pts_num,\n boxes, pts, box_idx_of_points);\n};\n\nvoid points_in_boxes_all_forward_cuda(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n PointsInBoxesAllForwardCUDAKernelLauncher(batch_size, boxes_num, pts_num,\n boxes, pts, box_idx_of_points);\n};\n\nvoid points_in_boxes_part_forward_impl(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points);\n\nvoid points_in_boxes_all_forward_impl(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points);\nREGISTER_DEVICE_IMPL(points_in_boxes_part_forward_impl, CUDA,\n points_in_boxes_part_forward_cuda);\nREGISTER_DEVICE_IMPL(points_in_boxes_all_forward_impl, CUDA,\n points_in_boxes_all_forward_cuda);\n\nvoid PSAMaskForwardCUDAKernelLauncher(const int psa_type, const Tensor input,\n Tensor output, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask,\n const int half_w_mask);\n\nvoid PSAMaskBackwardCUDAKernelLauncher(\n const int psa_type, const Tensor grad_output, Tensor grad_input,\n const int num_, const int h_feature, const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask, const int half_w_mask);\n\nvoid psamask_forward_cuda(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask) {\n PSAMaskForwardCUDAKernelLauncher(psa_type, input, output, num_, h_feature,\n w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask);\n}\n\nvoid psamask_backward_cuda(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask) {\n PSAMaskBackwardCUDAKernelLauncher(psa_type, grad_output, grad_input, num_,\n h_feature, w_feature, h_mask, w_mask,\n half_h_mask, half_w_mask);\n}\n\nvoid psamask_forward_impl(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask);\n\nvoid psamask_backward_impl(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask);\nREGISTER_DEVICE_IMPL(psamask_forward_impl, CUDA, psamask_forward_cuda);\nREGISTER_DEVICE_IMPL(psamask_backward_impl, CUDA, psamask_backward_cuda);\n\nvoid ROIAlignForwardCUDAKernelLauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nvoid ROIAlignBackwardCUDAKernelLauncher(Tensor grad_output, Tensor rois,\n Tensor argmax_y, Tensor argmax_x,\n Tensor grad_input, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode,\n bool aligned);\n\nvoid roi_align_forward_cuda(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n ROIAlignForwardCUDAKernelLauncher(\n input, rois, output, argmax_y, argmax_x, aligned_height, aligned_width,\n spatial_scale, sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_backward_cuda(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n ROIAlignBackwardCUDAKernelLauncher(\n grad_output, rois, argmax_y, argmax_x, grad_input, aligned_height,\n aligned_width, spatial_scale, sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nvoid roi_align_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned);\n\nREGISTER_DEVICE_IMPL(roi_align_forward_impl, CUDA, roi_align_forward_cuda);\nREGISTER_DEVICE_IMPL(roi_align_backward_impl, CUDA, roi_align_backward_cuda);\n\nvoid ROIAlignRotatedForwardCUDAKernelLauncher(\n const at::Tensor features, const at::Tensor rois, const float spatial_scale,\n const int sample_num, const bool aligned, const bool clockwise,\n const int channels, const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, at::Tensor output);\n\nvoid ROIAlignRotatedBackwardCUDAKernelLauncher(\n const at::Tensor top_grad, const at::Tensor rois, const float spatial_scale,\n const int sample_num, const bool aligned, const bool clockwise,\n const int channels, const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, at::Tensor bottom_grad);\n\nvoid roi_align_rotated_forward_cuda(Tensor features, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sample_ratio,\n bool aligned, bool clockwise) {\n // Number of ROIs\n int num_rois = rois.size(0);\n int size_rois = rois.size(1);\n\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n\n int num_channels = features.size(1);\n int data_height = features.size(2);\n int data_width = features.size(3);\n ROIAlignRotatedForwardCUDAKernelLauncher(\n features, rois, spatial_scale, sample_ratio, aligned, clockwise,\n num_channels, data_height, data_width, num_rois, aligned_height,\n aligned_width, output);\n}\n\nvoid roi_align_rotated_backward_cuda(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sample_ratio, bool aligned,\n bool clockwise) {\n // Number of ROIs\n int num_rois = rois.size(0);\n int size_rois = rois.size(1);\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n\n int num_channels = bottom_grad.size(1);\n int data_height = bottom_grad.size(2);\n int data_width = bottom_grad.size(3);\n ROIAlignRotatedBackwardCUDAKernelLauncher(\n top_grad, rois, spatial_scale, sample_ratio, aligned, clockwise,\n num_channels, data_height, data_width, num_rois, aligned_height,\n aligned_width, bottom_grad);\n}\n\nvoid roi_align_rotated_forward_impl(Tensor features, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sample_ratio,\n bool aligned, bool clockwise);\n\nvoid roi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sample_ratio, bool aligned,\n bool clockwise);\nREGISTER_DEVICE_IMPL(roi_align_rotated_forward_impl, CUDA,\n roi_align_rotated_forward_cuda);\nREGISTER_DEVICE_IMPL(roi_align_rotated_backward_impl, CUDA,\n roi_align_rotated_backward_cuda);\n\nvoid RiROIAlignRotatedForwardCUDAKernelLauncher(\n const at::Tensor features, const at::Tensor rois, const float spatial_scale,\n const int num_samples, const bool clockwise, const int channels,\n const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, const int num_orientations,\n at::Tensor output);\n\nvoid RiROIAlignRotatedBackwardCUDAKernelLauncher(\n const at::Tensor top_grad, const at::Tensor rois, const float spatial_scale,\n const int num_samples, const bool clockwise, const int channels,\n const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, const int num_orientations,\n at::Tensor bottom_grad);\n\nvoid riroi_align_rotated_forward_cuda(Tensor features, Tensor rois,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n // Number of ROIs\n int num_rois = rois.size(0);\n int size_rois = rois.size(1);\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n CHECK_CONTIGUOUS(features);\n CHECK_CONTIGUOUS(rois);\n int num_channels = features.size(1) / num_orientations;\n int data_height = features.size(2);\n int data_width = features.size(3);\n RiROIAlignRotatedForwardCUDAKernelLauncher(\n features, rois, spatial_scale, num_samples, clockwise, num_channels,\n data_height, data_width, num_rois, pooled_height, pooled_width,\n num_orientations, output);\n}\n\nvoid riroi_align_rotated_backward_cuda(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n // Number of ROIs\n int num_rois = rois.size(0);\n int size_rois = rois.size(1);\n if (size_rois != 6) {\n AT_ERROR(\"wrong roi size\");\n }\n CHECK_CONTIGUOUS(top_grad);\n CHECK_CONTIGUOUS(rois);\n int num_channels = bottom_grad.size(1) / num_orientations;\n int data_height = bottom_grad.size(2);\n int data_width = bottom_grad.size(3);\n RiROIAlignRotatedBackwardCUDAKernelLauncher(\n top_grad, rois, spatial_scale, num_samples, clockwise, num_channels,\n data_height, data_width, num_rois, pooled_height, pooled_width,\n num_orientations, bottom_grad);\n}\n\nvoid riroi_align_rotated_forward_impl(Tensor features, Tensor rois,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise);\n\nvoid riroi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise);\n\nREGISTER_DEVICE_IMPL(riroi_align_rotated_forward_impl, CUDA,\n riroi_align_rotated_forward_cuda);\nREGISTER_DEVICE_IMPL(riroi_align_rotated_backward_impl, CUDA,\n riroi_align_rotated_backward_cuda);\n\nvoid RoiawarePool3dForwardCUDAKernelLauncher(\n int boxes_num, int pts_num, int channels, int max_pts_each_voxel, int out_x,\n int out_y, int out_z, const Tensor rois, const Tensor pts,\n const Tensor pts_feature, Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method);\n\nvoid RoiawarePool3dBackwardCUDAKernelLauncher(\n int boxes_num, int out_x, int out_y, int out_z, int channels,\n int max_pts_each_voxel, const Tensor pts_idx_of_voxels, const Tensor argmax,\n const Tensor grad_out, Tensor grad_in, int pool_method);\n\nvoid roiaware_pool3d_forward_cuda(int boxes_num, int pts_num, int channels,\n int max_pts_each_voxel, int out_x, int out_y,\n int out_z, const Tensor rois,\n const Tensor pts, const Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method) {\n RoiawarePool3dForwardCUDAKernelLauncher(\n boxes_num, pts_num, channels, max_pts_each_voxel, out_x, out_y, out_z,\n rois, pts, pts_feature, argmax, pts_idx_of_voxels, pooled_features,\n pool_method);\n};\n\nvoid roiaware_pool3d_backward_cuda(int boxes_num, int out_x, int out_y,\n int out_z, int channels,\n int max_pts_each_voxel,\n const Tensor pts_idx_of_voxels,\n const Tensor argmax, const Tensor grad_out,\n Tensor grad_in, int pool_method) {\n RoiawarePool3dBackwardCUDAKernelLauncher(\n boxes_num, out_x, out_y, out_z, channels, max_pts_each_voxel,\n pts_idx_of_voxels, argmax, grad_out, grad_in, pool_method);\n};\n\nvoid roiaware_pool3d_forward_impl(int boxes_num, int pts_num, int channels,\n int max_pts_each_voxel, int out_x, int out_y,\n int out_z, const Tensor rois,\n const Tensor pts, const Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method);\n\nvoid roiaware_pool3d_backward_impl(int boxes_num, int out_x, int out_y,\n int out_z, int channels,\n int max_pts_each_voxel,\n const Tensor pts_idx_of_voxels,\n const Tensor argmax, const Tensor grad_out,\n Tensor grad_in, int pool_method);\n\nREGISTER_DEVICE_IMPL(roiaware_pool3d_forward_impl, CUDA,\n roiaware_pool3d_forward_cuda);\nREGISTER_DEVICE_IMPL(roiaware_pool3d_backward_impl, CUDA,\n roiaware_pool3d_backward_cuda);\n\nvoid RoIPointPool3dForwardCUDAKernelLauncher(\n int batch_size, int pts_num, int boxes_num, int feature_in_len,\n int sampled_pts_num, const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature, Tensor pooled_features, Tensor pooled_empty_flag);\n\nvoid roipoint_pool3d_forward_cuda(int batch_size, int pts_num, int boxes_num,\n int feature_in_len, int sampled_pts_num,\n const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature,\n Tensor pooled_features,\n Tensor pooled_empty_flag) {\n RoIPointPool3dForwardCUDAKernelLauncher(\n batch_size, pts_num, boxes_num, feature_in_len, sampled_pts_num, xyz,\n boxes3d, pts_feature, pooled_features, pooled_empty_flag);\n};\n\nvoid roipoint_pool3d_forward_impl(int batch_size, int pts_num, int boxes_num,\n int feature_in_len, int sampled_pts_num,\n const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature,\n Tensor pooled_features,\n Tensor pooled_empty_flag);\nREGISTER_DEVICE_IMPL(roipoint_pool3d_forward_impl, CUDA,\n roipoint_pool3d_forward_cuda);\n\nvoid ROIPoolForwardCUDAKernelLauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height,\n int pooled_width, float spatial_scale);\n\nvoid ROIPoolBackwardCUDAKernelLauncher(Tensor grad_output, Tensor rois,\n Tensor argmax, Tensor grad_input,\n int pooled_height, int pooled_width,\n float spatial_scale);\n\nvoid roi_pool_forward_cuda(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height, int pooled_width,\n float spatial_scale) {\n ROIPoolForwardCUDAKernelLauncher(input, rois, output, argmax, pooled_height,\n pooled_width, spatial_scale);\n}\n\nvoid roi_pool_backward_cuda(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height,\n int pooled_width, float spatial_scale) {\n ROIPoolBackwardCUDAKernelLauncher(grad_output, rois, argmax, grad_input,\n pooled_height, pooled_width, spatial_scale);\n}\n\nvoid roi_pool_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height, int pooled_width,\n float spatial_scale);\nvoid roi_pool_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height,\n int pooled_width, float spatial_scale);\nREGISTER_DEVICE_IMPL(roi_pool_forward_impl, CUDA, roi_pool_forward_cuda);\nREGISTER_DEVICE_IMPL(roi_pool_backward_impl, CUDA, roi_pool_backward_cuda);\n\ntypedef enum { SUM = 0, MEAN = 1, MAX = 2 } reduce_t;\n\nstd::vector DynamicPointToVoxelForwardCUDAKernelLauncher(\n const at::Tensor& feats, const at::Tensor& coors,\n const reduce_t reduce_type);\n\nvoid DynamicPointToVoxelBackwardCUDAKernelLauncher(\n at::Tensor& grad_feats, const at::Tensor& grad_reduced_feats,\n const at::Tensor& feats, const at::Tensor& reduced_feats,\n const at::Tensor& coors_map, const at::Tensor& reduce_count,\n const reduce_t reduce_type);\n\nstd::vector dynamic_point_to_voxel_forward_cuda(\n const torch::Tensor& feats, const torch::Tensor& coors,\n const reduce_t reduce_type) {\n return DynamicPointToVoxelForwardCUDAKernelLauncher(feats, coors,\n reduce_type);\n};\n\nvoid dynamic_point_to_voxel_backward_cuda(\n torch::Tensor& grad_feats, const torch::Tensor& grad_reduced_feats,\n const torch::Tensor& feats, const torch::Tensor& reduced_feats,\n const torch::Tensor& coors_idx, const torch::Tensor& reduce_count,\n const reduce_t reduce_type) {\n DynamicPointToVoxelBackwardCUDAKernelLauncher(grad_feats, grad_reduced_feats,\n feats, reduced_feats, coors_idx,\n reduce_count, reduce_type);\n};\n\nstd::vector dynamic_point_to_voxel_forward_impl(\n const torch::Tensor& feats, const torch::Tensor& coors,\n const reduce_t reduce_type);\n\nvoid dynamic_point_to_voxel_backward_impl(\n torch::Tensor& grad_feats, const torch::Tensor& grad_reduced_feats,\n const torch::Tensor& feats, const torch::Tensor& reduced_feats,\n const torch::Tensor& coors_idx, const torch::Tensor& reduce_count,\n const reduce_t reduce_type);\n\nREGISTER_DEVICE_IMPL(dynamic_point_to_voxel_forward_impl, CUDA,\n dynamic_point_to_voxel_forward_cuda);\nREGISTER_DEVICE_IMPL(dynamic_point_to_voxel_backward_impl, CUDA,\n dynamic_point_to_voxel_backward_cuda);\n\nvoid SyncBNForwardMeanCUDAKernelLauncher(const Tensor input, Tensor mean);\n\nvoid SyncBNForwardVarCUDAKernelLauncher(const Tensor input, const Tensor mean,\n Tensor var);\n\nvoid SyncBNForwardOutputCUDAKernelLauncher(\n const Tensor input, const Tensor mean, const Tensor var,\n Tensor running_mean, Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std, Tensor output, float eps,\n float momentum, int group_size);\n\nvoid SyncBNBackwardParamCUDAKernelLauncher(const Tensor grad_output,\n const Tensor norm,\n Tensor grad_weight,\n Tensor grad_bias);\n\nvoid SyncBNBackwardDataCUDAKernelLauncher(const Tensor grad_output,\n const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias,\n const Tensor norm, const Tensor std,\n Tensor grad_input);\n\nvoid sync_bn_forward_mean_cuda(const Tensor input, Tensor mean) {\n SyncBNForwardMeanCUDAKernelLauncher(input, mean);\n}\n\nvoid sync_bn_forward_var_cuda(const Tensor input, const Tensor mean,\n Tensor var) {\n SyncBNForwardVarCUDAKernelLauncher(input, mean, var);\n}\n\nvoid sync_bn_forward_output_cuda(const Tensor input, const Tensor mean,\n const Tensor var, Tensor running_mean,\n Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size) {\n SyncBNForwardOutputCUDAKernelLauncher(input, mean, var, running_mean,\n running_var, weight, bias, norm, std,\n output, eps, momentum, group_size);\n}\n\nvoid sync_bn_backward_param_cuda(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias) {\n SyncBNBackwardParamCUDAKernelLauncher(grad_output, norm, grad_weight,\n grad_bias);\n}\n\nvoid sync_bn_backward_data_cuda(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias, const Tensor norm,\n const Tensor std, Tensor grad_input) {\n SyncBNBackwardDataCUDAKernelLauncher(grad_output, weight, grad_weight,\n grad_bias, norm, std, grad_input);\n}\n\nvoid sync_bn_forward_mean_impl(const Tensor input, Tensor mean);\n\nvoid sync_bn_forward_var_impl(const Tensor input, const Tensor mean,\n Tensor var);\n\nvoid sync_bn_forward_output_impl(const Tensor input, const Tensor mean,\n const Tensor var, Tensor running_mean,\n Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size);\n\nvoid sync_bn_backward_param_impl(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias);\n\nvoid sync_bn_backward_data_impl(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias, const Tensor norm,\n const Tensor std, Tensor grad_input);\n\nREGISTER_DEVICE_IMPL(sync_bn_forward_mean_impl, CUDA,\n sync_bn_forward_mean_cuda);\nREGISTER_DEVICE_IMPL(sync_bn_forward_var_impl, CUDA, sync_bn_forward_var_cuda);\nREGISTER_DEVICE_IMPL(sync_bn_forward_output_impl, CUDA,\n sync_bn_forward_output_cuda);\nREGISTER_DEVICE_IMPL(sync_bn_backward_param_impl, CUDA,\n sync_bn_backward_param_cuda);\nREGISTER_DEVICE_IMPL(sync_bn_backward_data_impl, CUDA,\n sync_bn_backward_data_cuda);\n\nvoid ThreeInterpolateForwardCUDAKernelLauncher(int b, int c, int m, int n,\n const Tensor points,\n const Tensor idx,\n const Tensor weight, Tensor out);\n\nvoid ThreeInterpolateBackwardCUDAKernelLauncher(int b, int c, int n, int m,\n const Tensor grad_out,\n const Tensor idx,\n const Tensor weight,\n Tensor grad_points);\n\nvoid three_interpolate_forward_cuda(int b, int c, int m, int n,\n const Tensor points, const Tensor idx,\n const Tensor weight, Tensor out) {\n ThreeInterpolateForwardCUDAKernelLauncher(b, c, m, n, points, idx, weight,\n out);\n};\n\nvoid three_interpolate_backward_cuda(int b, int c, int n, int m,\n const Tensor grad_out, const Tensor idx,\n const Tensor weight, Tensor grad_points) {\n ThreeInterpolateBackwardCUDAKernelLauncher(b, c, n, m, grad_out, idx, weight,\n grad_points);\n};\n\nvoid three_interpolate_forward_impl(int b, int c, int m, int n,\n const Tensor points, const Tensor idx,\n const Tensor weight, Tensor out);\n\nvoid three_interpolate_backward_impl(int b, int c, int n, int m,\n const Tensor grad_out, const Tensor idx,\n const Tensor weight, Tensor grad_points);\nREGISTER_DEVICE_IMPL(three_interpolate_forward_impl, CUDA,\n three_interpolate_forward_cuda);\nREGISTER_DEVICE_IMPL(three_interpolate_backward_impl, CUDA,\n three_interpolate_backward_cuda);\n\nvoid ThreeNNForwardCUDAKernelLauncher(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2,\n Tensor idx);\n\nvoid three_nn_forward_cuda(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2, Tensor idx) {\n ThreeNNForwardCUDAKernelLauncher(b, n, m, unknown, known, dist2, idx);\n};\n\nvoid three_nn_forward_impl(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2, Tensor idx);\nREGISTER_DEVICE_IMPL(three_nn_forward_impl, CUDA, three_nn_forward_cuda);\n\nvoid TINShiftForwardCUDAKernelLauncher(Tensor input, Tensor shift,\n Tensor output);\n\nvoid TINShiftBackwardCUDAKernelLauncher(Tensor grad_output, Tensor shift,\n Tensor grad_input);\n\nvoid tin_shift_forward_cuda(Tensor input, Tensor shift, Tensor output) {\n TINShiftForwardCUDAKernelLauncher(input, shift, output);\n}\n\nvoid tin_shift_backward_cuda(Tensor grad_output, Tensor shift,\n Tensor grad_input) {\n TINShiftBackwardCUDAKernelLauncher(grad_output, shift, grad_input);\n}\n\nvoid tin_shift_forward_impl(Tensor input, Tensor shift, Tensor output);\nvoid tin_shift_backward_impl(Tensor grad_output, Tensor shift,\n Tensor grad_input);\nREGISTER_DEVICE_IMPL(tin_shift_forward_impl, CUDA, tin_shift_forward_cuda);\nREGISTER_DEVICE_IMPL(tin_shift_backward_impl, CUDA, tin_shift_backward_cuda);\n\ntorch::Tensor upfirdn2d_op(const torch::Tensor& input,\n const torch::Tensor& kernel, int up_x, int up_y,\n int down_x, int down_y, int pad_x0, int pad_x1,\n int pad_y0, int pad_y1);\n\ntorch::Tensor upfirdn2d_op_impl(const torch::Tensor& input,\n const torch::Tensor& kernel, int up_x, int up_y,\n int down_x, int down_y, int pad_x0, int pad_x1,\n int pad_y0, int pad_y1);\nREGISTER_DEVICE_IMPL(upfirdn2d_op_impl, CUDA, upfirdn2d_op);\n\nint HardVoxelizeForwardCUDAKernelLauncher(\n const at::Tensor& points, at::Tensor& voxels, at::Tensor& coors,\n at::Tensor& num_points_per_voxel, const std::vector voxel_size,\n const std::vector coors_range, const int max_points,\n const int max_voxels, const int NDim = 3);\n\nvoid DynamicVoxelizeForwardCUDAKernelLauncher(\n const at::Tensor& points, at::Tensor& coors,\n const std::vector voxel_size, const std::vector coors_range,\n const int NDim = 3);\n\nint hard_voxelize_forward_cuda(const at::Tensor& points, at::Tensor& voxels,\n at::Tensor& coors,\n at::Tensor& num_points_per_voxel,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int max_points, const int max_voxels,\n const int NDim) {\n return HardVoxelizeForwardCUDAKernelLauncher(\n points, voxels, coors, num_points_per_voxel, voxel_size, coors_range,\n max_points, max_voxels, NDim);\n};\n\nvoid dynamic_voxelize_forward_cuda(const at::Tensor& points, at::Tensor& coors,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int NDim) {\n DynamicVoxelizeForwardCUDAKernelLauncher(points, coors, voxel_size,\n coors_range, NDim);\n};\n\nint hard_voxelize_forward_impl(const at::Tensor& points, at::Tensor& voxels,\n at::Tensor& coors,\n at::Tensor& num_points_per_voxel,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int max_points, const int max_voxels,\n const int NDim);\n\nvoid dynamic_voxelize_forward_impl(const at::Tensor& points, at::Tensor& coors,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int NDim);\n\nREGISTER_DEVICE_IMPL(hard_voxelize_forward_impl, CUDA,\n hard_voxelize_forward_cuda);\nREGISTER_DEVICE_IMPL(dynamic_voxelize_forward_impl, CUDA,\n dynamic_voxelize_forward_cuda);\n\nvoid RotatedFeatureAlignForwardCUDAKernelLauncher(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points,\n Tensor output);\n\nvoid RotatedFeatureAlignBackwardCUDAKernelLauncher(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points,\n Tensor bottom_grad);\n\nvoid rotated_feature_align_forward_cuda(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor output) {\n RotatedFeatureAlignForwardCUDAKernelLauncher(features, best_bboxes,\n spatial_scale, points, output);\n};\n\nvoid rotated_feature_align_backward_cuda(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor bottom_grad) {\n RotatedFeatureAlignBackwardCUDAKernelLauncher(\n top_grad, best_bboxes, spatial_scale, points, bottom_grad);\n};\n\nvoid rotated_feature_align_forward_impl(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor output);\n\nvoid rotated_feature_align_backward_impl(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor bottom_grad);\n\nREGISTER_DEVICE_IMPL(rotated_feature_align_forward_impl, CUDA,\n rotated_feature_align_forward_cuda);\nREGISTER_DEVICE_IMPL(rotated_feature_align_backward_impl, CUDA,\n rotated_feature_align_backward_cuda);\n\nvoid PointsInPolygonsForwardCUDAKernelLauncher(const at::Tensor points,\n const at::Tensor polygons,\n const int rows, const int cols,\n at::Tensor output);\n\nvoid points_in_polygons_forward_cuda(const Tensor points, const Tensor polygons,\n Tensor output, const int rows,\n const int cols) {\n PointsInPolygonsForwardCUDAKernelLauncher(points, polygons, rows, cols,\n output);\n};\n\nvoid points_in_polygons_forward_impl(const Tensor points, const Tensor polygons,\n Tensor output, const int rows,\n const int cols);\n\nREGISTER_DEVICE_IMPL(points_in_polygons_forward_impl, CUDA,\n points_in_polygons_forward_cuda);\n\nvoid MinAreaPolygonsCUDAKernelLauncher(const Tensor pointsets, Tensor polygons);\n\nvoid min_area_polygons_cuda(const Tensor pointsets, Tensor polygons) {\n MinAreaPolygonsCUDAKernelLauncher(pointsets, polygons);\n}\n\nvoid min_area_polygons_impl(const Tensor pointsets, Tensor polygons);\n\nREGISTER_DEVICE_IMPL(min_area_polygons_impl, CUDA, min_area_polygons_cuda);\n\nvoid ActiveRotatedFilterForwardCUDAKernelLauncher(const Tensor input,\n const Tensor indices,\n Tensor output);\n\nvoid ActiveRotatedFilterBackwardCUDAKernelLauncher(const Tensor grad_out,\n const Tensor indices,\n Tensor grad_in);\n\nvoid active_rotated_filter_forward_cuda(const Tensor input,\n const Tensor indices, Tensor output) {\n ActiveRotatedFilterForwardCUDAKernelLauncher(input, indices, output);\n};\n\nvoid active_rotated_filter_backward_cuda(const Tensor grad_out,\n const Tensor indices, Tensor grad_in) {\n ActiveRotatedFilterBackwardCUDAKernelLauncher(grad_out, indices, grad_in);\n};\n\nvoid active_rotated_filter_forward_impl(const Tensor input,\n const Tensor indices, Tensor output);\n\nvoid active_rotated_filter_backward_impl(const Tensor grad_out,\n const Tensor indices, Tensor grad_in);\n\nREGISTER_DEVICE_IMPL(active_rotated_filter_forward_impl, CUDA,\n active_rotated_filter_forward_cuda);\nREGISTER_DEVICE_IMPL(active_rotated_filter_backward_impl, CUDA,\n active_rotated_filter_backward_cuda);\n\nvoid ConvexIoUCUDAKernelLauncher(const Tensor pointsets, const Tensor polygons,\n Tensor ious);\n\nvoid ConvexGIoUCUDAKernelLauncher(const Tensor pointsets, const Tensor polygons,\n Tensor output);\n\nvoid convex_iou_cuda(const Tensor pointsets, const Tensor polygons,\n Tensor ious) {\n ConvexIoUCUDAKernelLauncher(pointsets, polygons, ious);\n}\n\nvoid convex_giou_cuda(const Tensor pointsets, const Tensor polygons,\n Tensor output) {\n ConvexGIoUCUDAKernelLauncher(pointsets, polygons, output);\n}\n\nvoid convex_iou_impl(const Tensor pointsets, const Tensor polygons,\n Tensor ious);\n\nvoid convex_giou_impl(const Tensor pointsets, const Tensor polygons,\n Tensor output);\n\nREGISTER_DEVICE_IMPL(convex_iou_impl, CUDA, convex_iou_cuda);\nREGISTER_DEVICE_IMPL(convex_giou_impl, CUDA, convex_giou_cuda);\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/deform_conv_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"deform_conv_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid deformable_im2col_cuda(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col) {\n // num_axes should be smaller than block size\n // todo: check parallel_imgs is correctly passed in\n int height_col =\n (height + 2 * pad_h - (dilation_h * (ksize_h - 1) + 1)) / stride_h + 1;\n int width_col =\n (width + 2 * pad_w - (dilation_w * (ksize_w - 1) + 1)) / stride_w + 1;\n int num_kernels = channels * height_col * width_col * parallel_imgs;\n int channel_per_deformable_group = channels / deformable_group;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_im.scalar_type(), \"deformable_im2col_gpu\", ([&] {\n const scalar_t *data_im_ = data_im.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n scalar_t *data_col_ = data_col.data_ptr();\n\n deformable_im2col_gpu_kernel<<>>(\n num_kernels, data_im_, data_offset_, height, width, ksize_h,\n ksize_w, pad_h, pad_w, stride_h, stride_w, dilation_h, dilation_w,\n channel_per_deformable_group, parallel_imgs, channels,\n deformable_group, height_col, width_col, data_col_);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid deformable_col2im_cuda(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im) {\n // todo: make sure parallel_imgs is passed in correctly\n int height_col =\n (height + 2 * pad_h - (dilation_h * (ksize_h - 1) + 1)) / stride_h + 1;\n int width_col =\n (width + 2 * pad_w - (dilation_w * (ksize_w - 1) + 1)) / stride_w + 1;\n int num_kernels =\n channels * ksize_h * ksize_w * height_col * width_col * parallel_imgs;\n int channel_per_deformable_group = channels / deformable_group;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_col.scalar_type(), \"deformable_col2im_gpu\", ([&] {\n const scalar_t *data_col_ = data_col.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n scalar_t *grad_im_ = grad_im.data_ptr();\n\n deformable_col2im_gpu_kernel<<>>(\n num_kernels, data_col_, data_offset_, channels, height, width,\n ksize_h, ksize_w, pad_h, pad_w, stride_h, stride_w, dilation_h,\n dilation_w, channel_per_deformable_group, parallel_imgs,\n deformable_group, height_col, width_col, grad_im_);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid deformable_col2im_coord_cuda(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset) {\n int height_col =\n (height + 2 * pad_h - (dilation_h * (ksize_h - 1) + 1)) / stride_h + 1;\n int width_col =\n (width + 2 * pad_w - (dilation_w * (ksize_w - 1) + 1)) / stride_w + 1;\n int num_kernels = height_col * width_col * 2 * ksize_h * ksize_w *\n deformable_group * parallel_imgs;\n int channel_per_deformable_group =\n channels * ksize_h * ksize_w / deformable_group;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_col.scalar_type(), \"deformable_col2im_coord_gpu\", ([&] {\n const scalar_t *data_col_ = data_col.data_ptr();\n const scalar_t *data_im_ = data_im.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n scalar_t *grad_offset_ = grad_offset.data_ptr();\n\n deformable_col2im_coord_gpu_kernel<<<\n GET_BLOCKS(num_kernels), THREADS_PER_BLOCK, 0,\n at::cuda::getCurrentCUDAStream()>>>(\n num_kernels, data_col_, data_im_, data_offset_, channels, height,\n width, ksize_h, ksize_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, channel_per_deformable_group, parallel_imgs,\n 2 * ksize_h * ksize_w * deformable_group, deformable_group,\n height_col, width_col, grad_offset_);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/deform_roi_pool_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"deform_roi_pool_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid DeformRoIPoolForwardCUDAKernelLauncher(Tensor input, Tensor rois,\n Tensor offset, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale,\n int sampling_ratio, float gamma) {\n int output_size = output.numel();\n int channels = input.size(1);\n int height = input.size(2);\n int width = input.size(3);\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"deform_roi_pool_forward_cuda_kernel\", [&] {\n deform_roi_pool_forward_cuda_kernel\n <<>>(\n output_size, input.data_ptr(),\n rois.data_ptr(), offset.data_ptr(),\n output.data_ptr(), pooled_height, pooled_width,\n static_cast(spatial_scale), sampling_ratio,\n static_cast(gamma), channels, height, width);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid DeformRoIPoolBackwardCUDAKernelLauncher(\n Tensor grad_output, Tensor input, Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset, int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio, float gamma) {\n int output_size = grad_output.numel();\n int channels = grad_input.size(1);\n int height = grad_input.size(2);\n int width = grad_input.size(3);\n\n at::cuda::CUDAGuard device_guard(grad_output.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"deform_roi_pool_backward_cuda_kernel\", [&] {\n deform_roi_pool_backward_cuda_kernel\n <<>>(\n output_size, grad_output.data_ptr(),\n input.data_ptr(), rois.data_ptr(),\n offset.data_ptr(), grad_input.data_ptr(),\n grad_offset.data_ptr(), pooled_height, pooled_width,\n static_cast(spatial_scale), sampling_ratio,\n static_cast(gamma), channels, height, width);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/focal_loss_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cuda_helper.hpp\"\n#include \"sigmoid_focal_loss_cuda_kernel.cuh\"\n#include \"softmax_focal_loss_cuda_kernel.cuh\"\n\nvoid SigmoidFocalLossForwardCUDAKernelLauncher(Tensor input, Tensor target,\n Tensor weight, Tensor output,\n const float gamma,\n const float alpha) {\n int output_size = output.numel();\n int num_classes = input.size(1);\n AT_ASSERTM(target.max().item() <= (int64_t)num_classes,\n \"target label should smaller or equal than num classes\");\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"sigmoid_focal_loss_forward_cuda_kernel\", [&] {\n sigmoid_focal_loss_forward_cuda_kernel\n <<>>(\n output_size, input.data_ptr(),\n target.data_ptr(), weight.data_ptr(),\n output.data_ptr(), gamma, alpha, num_classes);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid SigmoidFocalLossBackwardCUDAKernelLauncher(Tensor input, Tensor target,\n Tensor weight,\n Tensor grad_input,\n const float gamma,\n const float alpha) {\n int output_size = grad_input.numel();\n int num_classes = input.size(1);\n\n at::cuda::CUDAGuard device_guard(grad_input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"sigmoid_focal_loss_backward_cuda_kernel\", [&] {\n sigmoid_focal_loss_backward_cuda_kernel\n <<>>(\n output_size, input.data_ptr(),\n target.data_ptr(), weight.data_ptr(),\n grad_input.data_ptr(), gamma, alpha, num_classes);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid SoftmaxFocalLossForwardCUDAKernelLauncher(Tensor softmax, Tensor target,\n Tensor weight, Tensor output,\n const float gamma,\n const float alpha) {\n int output_size = output.numel();\n int num_classes = softmax.size(1);\n\n AT_ASSERTM(target.max().item() <= (int64_t)num_classes,\n \"target label should smaller or equal than num classes\");\n at::cuda::CUDAGuard device_guard(softmax.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n softmax.scalar_type(), \"softmax_focal_loss_forward_cuda_kernel\", [&] {\n softmax_focal_loss_forward_cuda_kernel\n <<>>(\n output_size, softmax.data_ptr(),\n target.data_ptr(), weight.data_ptr(),\n output.data_ptr(), gamma, alpha, num_classes);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid SoftmaxFocalLossBackwardCUDAKernelLauncher(Tensor softmax, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input,\n const float gamma,\n const float alpha) {\n int num_classes = softmax.size(1);\n\n int output_size = buff.numel();\n at::cuda::CUDAGuard device_guard(grad_input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_input.scalar_type(),\n \"softmax_focal_loss_backward_cuda1_\"\n \"kernel\",\n [&] {\n softmax_focal_loss_backward_cuda1_kernel\n <<>>(\n output_size, softmax.data_ptr(),\n target.data_ptr(), weight.data_ptr(),\n buff.data_ptr(), gamma, alpha, num_classes);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n output_size = grad_input.numel();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_input.scalar_type(),\n \"softmax_focal_loss_backward_cuda2_\"\n \"kernel\",\n [&] {\n softmax_focal_loss_backward_cuda2_kernel\n <<>>(\n output_size, softmax.data_ptr(),\n target.data_ptr(), buff.data_ptr(),\n grad_input.data_ptr(), num_classes);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/furthest_point_sample_cuda.cu", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/sampling_gpu.cu\n\n#include \n#include \n\n#include \"furthest_point_sample_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\ninline int opt_n_threads(int work_size) {\n const int pow_2 = std::log(static_cast(work_size)) / std::log(2.0);\n\n return max(min(1 << pow_2, 1024), 1);\n}\n\nvoid FurthestPointSamplingForwardCUDAKernelLauncher(int b, int n, int m,\n const float* dataset,\n float* temp, int* idxs) {\n // dataset: (B, N, 3)\n // tmp: (B, N)\n // output:\n // idx: (B, M)\n\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n unsigned int n_threads = opt_n_threads(n);\n\n switch (n_threads) {\n case 1024:\n furthest_point_sampling_forward_cuda_kernel<1024>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 512:\n furthest_point_sampling_forward_cuda_kernel<512>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 256:\n furthest_point_sampling_forward_cuda_kernel<256>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 128:\n furthest_point_sampling_forward_cuda_kernel<128>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 64:\n furthest_point_sampling_forward_cuda_kernel<64>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 32:\n furthest_point_sampling_forward_cuda_kernel<32>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 16:\n furthest_point_sampling_forward_cuda_kernel<16>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 8:\n furthest_point_sampling_forward_cuda_kernel<8>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 4:\n furthest_point_sampling_forward_cuda_kernel<4>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 2:\n furthest_point_sampling_forward_cuda_kernel<2>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 1:\n furthest_point_sampling_forward_cuda_kernel<1>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n default:\n furthest_point_sampling_forward_cuda_kernel<512>\n <<>>(b, n, m, dataset, temp, idxs);\n }\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid FurthestPointSamplingWithDistForwardCUDAKernelLauncher(\n int b, int n, int m, const float* dataset, float* temp, int* idxs) {\n // dataset: (B, N, N)\n // temp: (B, N)\n // output:\n // idx: (B, M)\n\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n unsigned int n_threads = opt_n_threads(n);\n\n switch (n_threads) {\n case 1024:\n furthest_point_sampling_with_dist_forward_cuda_kernel<1024>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 512:\n furthest_point_sampling_with_dist_forward_cuda_kernel<512>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 256:\n furthest_point_sampling_with_dist_forward_cuda_kernel<256>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 128:\n furthest_point_sampling_with_dist_forward_cuda_kernel<128>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 64:\n furthest_point_sampling_with_dist_forward_cuda_kernel<64>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 32:\n furthest_point_sampling_with_dist_forward_cuda_kernel<32>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 16:\n furthest_point_sampling_with_dist_forward_cuda_kernel<16>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 8:\n furthest_point_sampling_with_dist_forward_cuda_kernel<8>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 4:\n furthest_point_sampling_with_dist_forward_cuda_kernel<4>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 2:\n furthest_point_sampling_with_dist_forward_cuda_kernel<2>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n case 1:\n furthest_point_sampling_with_dist_forward_cuda_kernel<1>\n <<>>(b, n, m, dataset, temp, idxs);\n break;\n default:\n furthest_point_sampling_with_dist_forward_cuda_kernel<512>\n <<>>(b, n, m, dataset, temp, idxs);\n }\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/fused_bias_leakyrelu_cuda.cu", "content": "// Modified from\n// https://github.com/rosinality/stylegan2-pytorch/blob/master/op/fused_bias_act_kernel.cu\n// Copyright (c) 2019, NVIDIA Corporation. All rights reserved.\n//\n// This work is made available under the Nvidia Source Code License-NC.\n// To view a copy of this license, visit\n// https://nvlabs.github.io/stylegan2/license.html\n\n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \n\ntemplate \nstatic __global__ void fused_bias_act_kernel(\n scalar_t* out, const scalar_t* p_x, const scalar_t* p_b,\n const scalar_t* p_ref, int act, int grad, scalar_t alpha, scalar_t scale,\n int loop_x, int size_x, int step_b, int size_b, int use_bias, int use_ref) {\n int xi = blockIdx.x * loop_x * blockDim.x + threadIdx.x;\n\n scalar_t zero = 0.0;\n\n for (int loop_idx = 0; loop_idx < loop_x && xi < size_x;\n loop_idx++, xi += blockDim.x) {\n scalar_t x = p_x[xi];\n\n if (use_bias) {\n x += p_b[(xi / step_b) % size_b];\n }\n\n scalar_t ref = use_ref ? p_ref[xi] : zero;\n\n scalar_t y;\n\n // act = 1: linear layer\n // act = 3: leaky relu layer\n // grad = 0: direct forward path\n // grad = 1: first order deviation\n // grad = 2: second order deviation\n switch (act * 10 + grad) {\n default:\n case 10:\n y = x;\n break;\n case 11:\n y = x;\n break;\n case 12:\n y = 0.0;\n break;\n\n case 30:\n y = (x > 0.0) ? x : x * alpha;\n break;\n case 31:\n y = (ref > 0.0) ? x : x * alpha;\n break;\n case 32:\n y = 0.0;\n break;\n }\n\n out[xi] = y * scale;\n }\n}\n\ntorch::Tensor fused_bias_leakyrelu_op(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale) {\n int curDevice = -1;\n cudaGetDevice(&curDevice);\n cudaStream_t stream = at::cuda::getCurrentCUDAStream(curDevice);\n\n auto x = input.contiguous();\n auto b = bias.contiguous();\n auto ref = refer.contiguous();\n\n int use_bias = b.numel() ? 1 : 0;\n int use_ref = ref.numel() ? 1 : 0;\n\n int size_x = x.numel();\n int size_b = b.numel();\n int step_b = 1;\n\n for (int i = 1 + 1; i < x.dim(); i++) {\n step_b *= x.size(i);\n }\n\n int loop_x = 4;\n int block_size = 4 * 32;\n int grid_size = (size_x - 1) / (loop_x * block_size) + 1;\n\n auto y = torch::empty_like(x);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n x.scalar_type(), \"fused_bias_act_kernel\", [&] {\n fused_bias_act_kernel<<>>(\n y.data_ptr(), x.data_ptr(),\n b.data_ptr(), ref.data_ptr(), act, grad, alpha,\n scale, loop_x, size_x, step_b, size_b, use_bias, use_ref);\n });\n\n return y;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/gather_points_cuda.cu", "content": "#include \n#include \n\n#include \"gather_points_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid GatherPointsForwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n const Tensor points,\n const Tensor idx, Tensor out) {\n // points: (B, C, N)\n // idx: (B, npoints)\n // output:\n // out: (B, C, npoints)\n\n at::cuda::CUDAGuard device_guard(points.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(npoints, THREADS_PER_BLOCK), c, b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n points.scalar_type(), \"gather_points_forward_cuda_kernel\", [&] {\n gather_points_forward_cuda_kernel\n <<>>(\n b, c, n, npoints, points.data_ptr(),\n idx.data_ptr(), out.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid GatherPointsBackwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n const Tensor grad_out,\n const Tensor idx,\n Tensor grad_points) {\n // grad_out: (B, C, npoints)\n // idx: (B, npoints)\n // output:\n // grad_points: (B, C, N)\n\n at::cuda::CUDAGuard device_guard(grad_out.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(npoints, THREADS_PER_BLOCK), c, b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_out.scalar_type(), \"gather_points_backward_cuda_kernel\", [&] {\n gather_points_backward_cuda_kernel\n <<>>(\n b, c, n, npoints, grad_out.data_ptr(),\n idx.data_ptr(), grad_points.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/group_points_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/group_points_gpu.cu\n#include \n#include \n\n#include \"group_points_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid GroupPointsForwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n int nsample, const Tensor points,\n const Tensor idx, Tensor out) {\n // points: (B, C, N)\n // idx: (B, npoints, nsample)\n // output:\n // out: (B, C, npoints, nsample)\n\n at::cuda::CUDAGuard device_guard(points.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(npoints * nsample, THREADS_PER_BLOCK), c, b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n points.scalar_type(), \"group_points_forward_cuda_kernel\", [&] {\n group_points_forward_cuda_kernel\n <<>>(\n b, c, n, npoints, nsample, points.data_ptr(),\n idx.data_ptr(), out.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid GroupPointsBackwardCUDAKernelLauncher(int b, int c, int n, int npoints,\n int nsample, const Tensor grad_out,\n const Tensor idx,\n Tensor grad_points) {\n // grad_out: (B, C, npoints, nsample)\n // idx: (B, npoints, nsample)\n // output:\n // grad_points: (B, C, N)\n\n at::cuda::CUDAGuard device_guard(grad_out.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(npoints * nsample, THREADS_PER_BLOCK), c, b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_out.scalar_type(), \"group_points_backward_cuda_kernel\", [&] {\n group_points_backward_cuda_kernel\n <<>>(\n b, c, n, npoints, nsample, grad_out.data_ptr(),\n idx.data_ptr(), grad_points.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/iou3d_cuda.cu", "content": "// Modified from\n// https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/ops/iou3d_nms/src/iou3d_nms_kernel.cu\n\n/*\n3D IoU Calculation and Rotated NMS(modified from 2D NMS written by others)\nWritten by Shaoshuai Shi\nAll Rights Reserved 2019-2020.\n*/\n\n#include \n\n#include \"iou3d_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid IoU3DBoxesOverlapBevForwardCUDAKernelLauncher(const int num_a,\n const Tensor boxes_a,\n const int num_b,\n const Tensor boxes_b,\n Tensor ans_overlap) {\n at::cuda::CUDAGuard device_guard(boxes_a.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(num_b, THREADS_PER_BLOCK_IOU3D),\n GET_BLOCKS(num_a, THREADS_PER_BLOCK_IOU3D));\n dim3 threads(THREADS_PER_BLOCK_IOU3D, THREADS_PER_BLOCK_IOU3D);\n\n iou3d_boxes_overlap_bev_forward_cuda_kernel<<>>(\n num_a, boxes_a.data_ptr(), num_b, boxes_b.data_ptr(),\n ans_overlap.data_ptr());\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid IoU3DBoxesIoUBevForwardCUDAKernelLauncher(const int num_a,\n const Tensor boxes_a,\n const int num_b,\n const Tensor boxes_b,\n Tensor ans_iou) {\n at::cuda::CUDAGuard device_guard(boxes_a.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(num_b, THREADS_PER_BLOCK_IOU3D),\n GET_BLOCKS(num_a, THREADS_PER_BLOCK_IOU3D));\n dim3 threads(THREADS_PER_BLOCK_IOU3D, THREADS_PER_BLOCK_IOU3D);\n\n iou3d_boxes_iou_bev_forward_cuda_kernel<<>>(\n num_a, boxes_a.data_ptr(), num_b, boxes_b.data_ptr(),\n ans_iou.data_ptr());\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid IoU3DNMSForwardCUDAKernelLauncher(const Tensor boxes,\n unsigned long long *mask, int boxes_num,\n float nms_overlap_thresh) {\n at::cuda::CUDAGuard device_guard(boxes.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n dim3 blocks(GET_BLOCKS(boxes_num, THREADS_PER_BLOCK_NMS),\n GET_BLOCKS(boxes_num, THREADS_PER_BLOCK_NMS));\n dim3 threads(THREADS_PER_BLOCK_NMS);\n\n nms_forward_cuda_kernel<<>>(\n boxes_num, nms_overlap_thresh, boxes.data_ptr(), mask);\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid IoU3DNMSNormalForwardCUDAKernelLauncher(const Tensor boxes,\n unsigned long long *mask,\n int boxes_num,\n float nms_overlap_thresh) {\n at::cuda::CUDAGuard device_guard(boxes.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n dim3 blocks(GET_BLOCKS(boxes_num, THREADS_PER_BLOCK_NMS),\n GET_BLOCKS(boxes_num, THREADS_PER_BLOCK_NMS));\n dim3 threads(THREADS_PER_BLOCK_NMS);\n\n nms_normal_forward_cuda_kernel<<>>(\n boxes_num, nms_overlap_thresh, boxes.data_ptr(), mask);\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/knn_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/CVMI-Lab/PAConv/tree/main/scene_seg/lib/pointops/src/knnquery_heap\n\n#include \n#include \n\n#include \"knn_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid KNNForwardCUDAKernelLauncher(int b, int n, int m, int nsample,\n const Tensor xyz, const Tensor new_xyz,\n Tensor idx, Tensor dist2) {\n // param new_xyz: (B, m, 3)\n // param xyz: (B, n, 3)\n // param idx: (B, m, nsample)\n\n at::cuda::CUDAGuard device_guard(new_xyz.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(m, THREADS_PER_BLOCK), b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n new_xyz.scalar_type(), \"knn_forward_cuda_kernel\", [&] {\n knn_forward_cuda_kernel<<>>(\n b, n, m, nsample, xyz.data_ptr(),\n new_xyz.data_ptr(), idx.data_ptr(),\n dist2.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/masked_conv2d_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"masked_conv2d_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid MaskedIm2colForwardCUDAKernelLauncher(const Tensor bottom_data,\n const Tensor mask_h_idx,\n const Tensor mask_w_idx,\n Tensor top_data, const int kernel_h,\n const int kernel_w, const int pad_h,\n const int pad_w) {\n int channels = bottom_data.size(1);\n int height = bottom_data.size(2);\n int width = bottom_data.size(3);\n int mask_cnt = mask_h_idx.size(0);\n int output_size = mask_cnt * channels;\n\n at::cuda::CUDAGuard device_guard(bottom_data.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n bottom_data.scalar_type(), \"MaskedIm2colLaucherForward\", ([&] {\n const scalar_t *bottom_data_ = bottom_data.data_ptr();\n const int64_t *mask_h_idx_ = mask_h_idx.data_ptr();\n const int64_t *mask_w_idx_ = mask_w_idx.data_ptr();\n scalar_t *top_data_ = top_data.data_ptr();\n MaskedIm2colForward\n <<>>(\n output_size, bottom_data_, height, width, kernel_h, kernel_w,\n pad_h, pad_w, mask_h_idx_, mask_w_idx_, mask_cnt, top_data_);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid MaskedCol2imForwardCUDAKernelLauncher(\n const Tensor bottom_data, const Tensor mask_h_idx, const Tensor mask_w_idx,\n Tensor top_data, const int height, const int width, const int channels) {\n int mask_cnt = mask_h_idx.size(0);\n int output_size = mask_cnt * channels;\n\n at::cuda::CUDAGuard device_guard(bottom_data.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n bottom_data.scalar_type(), \"MaskedCol2imLaucherForward\", ([&] {\n const scalar_t *bottom_data_ = bottom_data.data_ptr();\n const int64_t *mask_h_idx_ = mask_h_idx.data_ptr();\n const int64_t *mask_w_idx_ = mask_w_idx.data_ptr();\n scalar_t *top_data_ = top_data.data_ptr();\n\n MaskedCol2imForward\n <<>>(\n output_size, bottom_data_, height, width, channels, mask_h_idx_,\n mask_w_idx_, mask_cnt, top_data_);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/min_area_polygons.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// modified from\n// https://github.com/SDL-GuoZonghao/BeyondBoundingBox/blob/main/mmdet/ops/minareabbox/src/minareabbox_kernel.cu\n#include \"min_area_polygons_cuda.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid MinAreaPolygonsCUDAKernelLauncher(const Tensor pointsets,\n Tensor polygons) {\n int num_pointsets = pointsets.size(0);\n const int output_size = polygons.numel();\n at::cuda::CUDAGuard device_guard(pointsets.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n pointsets.scalar_type(), \"min_area_polygons_cuda_kernel\", ([&] {\n min_area_polygons_cuda_kernel\n <<>>(\n num_pointsets, pointsets.data_ptr(),\n polygons.data_ptr());\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/modulated_deform_conv_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"modulated_deform_conv_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid modulated_deformable_im2col_cuda(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col) {\n // num_axes should be smaller than block size\n const int channel_per_deformable_group = channels / deformable_group;\n const int num_kernels = channels * batch_size * height_col * width_col;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_im.scalar_type(), \"modulated_deformable_im2col_gpu\", ([&] {\n const scalar_t *data_im_ = data_im.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n const scalar_t *data_mask_ = data_mask.data_ptr();\n scalar_t *data_col_ = data_col.data_ptr();\n\n modulated_deformable_im2col_gpu_kernel<<<\n GET_BLOCKS(num_kernels), THREADS_PER_BLOCK, 0,\n at::cuda::getCurrentCUDAStream()>>>(\n num_kernels, data_im_, data_offset_, data_mask_, height_im,\n width_im, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, channel_per_deformable_group, batch_size,\n channels, deformable_group, height_col, width_col, data_col_);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid modulated_deformable_col2im_cuda(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im) {\n const int channel_per_deformable_group = channels / deformable_group;\n const int num_kernels =\n channels * kernel_h * kernel_w * batch_size * height_col * width_col;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_col.scalar_type(), \"modulated_deformable_col2im_gpu\", ([&] {\n const scalar_t *data_col_ = data_col.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n const scalar_t *data_mask_ = data_mask.data_ptr();\n scalar_t *grad_im_ = grad_im.data_ptr();\n\n modulated_deformable_col2im_gpu_kernel<<<\n GET_BLOCKS(num_kernels), THREADS_PER_BLOCK, 0,\n at::cuda::getCurrentCUDAStream()>>>(\n num_kernels, data_col_, data_offset_, data_mask_, channels,\n height_im, width_im, kernel_h, kernel_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, channel_per_deformable_group,\n batch_size, deformable_group, height_col, width_col, grad_im_);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid modulated_deformable_col2im_coord_cuda(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask) {\n const int num_kernels = batch_size * height_col * width_col * 2 * kernel_h *\n kernel_w * deformable_group;\n const int channel_per_deformable_group =\n channels * kernel_h * kernel_w / deformable_group;\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n data_col.scalar_type(), \"modulated_deformable_col2im_coord_gpu\", ([&] {\n const scalar_t *data_col_ = data_col.data_ptr();\n const scalar_t *data_im_ = data_im.data_ptr();\n const scalar_t *data_offset_ = data_offset.data_ptr();\n const scalar_t *data_mask_ = data_mask.data_ptr();\n scalar_t *grad_offset_ = grad_offset.data_ptr();\n scalar_t *grad_mask_ = grad_mask.data_ptr();\n\n modulated_deformable_col2im_coord_gpu_kernel<<<\n GET_BLOCKS(num_kernels), THREADS_PER_BLOCK, 0,\n at::cuda::getCurrentCUDAStream()>>>(\n num_kernels, data_col_, data_im_, data_offset_, data_mask_,\n channels, height_im, width_im, kernel_h, kernel_w, pad_h, pad_w,\n stride_h, stride_w, dilation_h, dilation_w,\n channel_per_deformable_group, batch_size,\n 2 * kernel_h * kernel_w * deformable_group, deformable_group,\n height_col, width_col, grad_offset_, grad_mask_);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/ms_deform_attn_cuda.cu", "content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from\n*https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#include \n#include \n#include \n#include \n\n#include \n#include \n\n#include \"ms_deform_attn_cuda_kernel.cuh\"\n\ntemplate \nvoid ms_deformable_im2col_cuda(cudaStream_t stream, const scalar_t *data_value,\n const int64_t *data_spatial_shapes,\n const int64_t *data_level_start_index,\n const scalar_t *data_sampling_loc,\n const scalar_t *data_attn_weight,\n const int batch_size, const int spatial_size,\n const int num_heads, const int channels,\n const int num_levels, const int num_query,\n const int num_point, scalar_t *data_col) {\n const int num_kernels = batch_size * num_query * num_heads * channels;\n const int num_actual_kernels = batch_size * num_query * num_heads * channels;\n const int num_threads = CUDA_NUM_THREADS;\n ms_deformable_im2col_gpu_kernel\n <<>>(\n num_kernels, data_value, data_spatial_shapes, data_level_start_index,\n data_sampling_loc, data_attn_weight, batch_size, spatial_size,\n num_heads, channels, num_levels, num_query, num_point, data_col);\n\n cudaError_t err = cudaGetLastError();\n if (err != cudaSuccess) {\n printf(\"error in ms_deformable_im2col_cuda: %s\\n\", cudaGetErrorString(err));\n }\n}\n\ntemplate \nvoid ms_deformable_col2im_cuda(\n cudaStream_t stream, const scalar_t *grad_col, const scalar_t *data_value,\n const int64_t *data_spatial_shapes, const int64_t *data_level_start_index,\n const scalar_t *data_sampling_loc, const scalar_t *data_attn_weight,\n const int batch_size, const int spatial_size, const int num_heads,\n const int channels, const int num_levels, const int num_query,\n const int num_point, scalar_t *grad_value, scalar_t *grad_sampling_loc,\n scalar_t *grad_attn_weight) {\n const int num_threads =\n (channels > CUDA_NUM_THREADS) ? CUDA_NUM_THREADS : channels;\n const int num_kernels = batch_size * num_query * num_heads * channels;\n const int num_actual_kernels = batch_size * num_query * num_heads * channels;\n if (channels > 1024) {\n if ((channels & 1023) == 0) {\n ms_deformable_col2im_gpu_kernel_shm_reduce_v2_multi_blocks\n <<>>(\n num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc, data_attn_weight,\n batch_size, spatial_size, num_heads, channels, num_levels,\n num_query, num_point, grad_value, grad_sampling_loc,\n grad_attn_weight);\n } else {\n ms_deformable_col2im_gpu_kernel_gm\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n }\n } else {\n switch (channels) {\n case 1:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 2:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 4:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 8:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 16:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 32:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v1\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 64:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 128:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 256:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 512:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n case 1024:\n ms_deformable_col2im_gpu_kernel_shm_blocksize_aware_reduce_v2\n <<>>(num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc,\n data_attn_weight, batch_size, spatial_size, num_heads,\n channels, num_levels, num_query, num_point, grad_value,\n grad_sampling_loc, grad_attn_weight);\n break;\n default:\n if (channels < 64) {\n ms_deformable_col2im_gpu_kernel_shm_reduce_v1\n <<>>(\n num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc, data_attn_weight,\n batch_size, spatial_size, num_heads, channels, num_levels,\n num_query, num_point, grad_value, grad_sampling_loc,\n grad_attn_weight);\n } else {\n ms_deformable_col2im_gpu_kernel_shm_reduce_v2\n <<>>(\n num_kernels, grad_col, data_value, data_spatial_shapes,\n data_level_start_index, data_sampling_loc, data_attn_weight,\n batch_size, spatial_size, num_heads, channels, num_levels,\n num_query, num_point, grad_value, grad_sampling_loc,\n grad_attn_weight);\n }\n }\n }\n cudaError_t err = cudaGetLastError();\n if (err != cudaSuccess) {\n printf(\"error in ms_deformable_col2im_cuda: %s\\n\", cudaGetErrorString(err));\n }\n}\n\nat::Tensor ms_deform_attn_cuda_forward(const at::Tensor &value,\n const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index,\n const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight,\n const int im2col_step) {\n AT_ASSERTM(value.is_contiguous(), \"value tensor has to be contiguous\");\n AT_ASSERTM(spatial_shapes.is_contiguous(),\n \"spatial_shapes tensor has to be contiguous\");\n AT_ASSERTM(level_start_index.is_contiguous(),\n \"level_start_index tensor has to be contiguous\");\n AT_ASSERTM(sampling_loc.is_contiguous(),\n \"sampling_loc tensor has to be contiguous\");\n AT_ASSERTM(attn_weight.is_contiguous(),\n \"attn_weight tensor has to be contiguous\");\n\n AT_ASSERTM(value.is_cuda(), \"value must be a CUDA tensor\");\n AT_ASSERTM(spatial_shapes.is_cuda(), \"spatial_shapes must be a CUDA tensor\");\n AT_ASSERTM(level_start_index.is_cuda(),\n \"level_start_index must be a CUDA tensor\");\n AT_ASSERTM(sampling_loc.is_cuda(), \"sampling_loc must be a CUDA tensor\");\n AT_ASSERTM(attn_weight.is_cuda(), \"attn_weight must be a CUDA tensor\");\n\n const int batch = value.size(0);\n const int spatial_size = value.size(1);\n const int num_heads = value.size(2);\n const int channels = value.size(3);\n\n const int num_levels = spatial_shapes.size(0);\n\n const int num_query = sampling_loc.size(1);\n const int num_point = sampling_loc.size(4);\n\n const int im2col_step_ = std::min(batch, im2col_step);\n\n AT_ASSERTM(batch % im2col_step_ == 0, \"batch(%d) must divide im2col_step(%d)\",\n batch, im2col_step_);\n\n auto output =\n at::zeros({batch, num_query, num_heads, channels}, value.options());\n\n const int batch_n = im2col_step_;\n auto output_n = output.view(\n {batch / im2col_step_, batch_n, num_query, num_heads, channels});\n auto per_value_size = spatial_size * num_heads * channels;\n auto per_sample_loc_size = num_query * num_heads * num_levels * num_point * 2;\n auto per_attn_weight_size = num_query * num_heads * num_levels * num_point;\n for (int n = 0; n < batch / im2col_step_; ++n) {\n auto columns = output_n.select(0, n);\n AT_DISPATCH_FLOATING_TYPES(\n value.scalar_type(), \"ms_deform_attn_forward_cuda\", ([&] {\n ms_deformable_im2col_cuda(\n at::cuda::getCurrentCUDAStream(),\n value.data_ptr() + n * im2col_step_ * per_value_size,\n spatial_shapes.data_ptr(),\n level_start_index.data_ptr(),\n sampling_loc.data_ptr() +\n n * im2col_step_ * per_sample_loc_size,\n attn_weight.data_ptr() +\n n * im2col_step_ * per_attn_weight_size,\n batch_n, spatial_size, num_heads, channels, num_levels, num_query,\n num_point, columns.data_ptr());\n }));\n }\n\n output = output.view({batch, num_query, num_heads * channels});\n\n return output;\n}\n\nvoid ms_deform_attn_cuda_backward(\n const at::Tensor &value, const at::Tensor &spatial_shapes,\n const at::Tensor &level_start_index, const at::Tensor &sampling_loc,\n const at::Tensor &attn_weight, const at::Tensor &grad_output,\n at::Tensor &grad_value, at::Tensor &grad_sampling_loc,\n at::Tensor &grad_attn_weight, const int im2col_step) {\n AT_ASSERTM(value.is_contiguous(), \"value tensor has to be contiguous\");\n AT_ASSERTM(spatial_shapes.is_contiguous(),\n \"spatial_shapes tensor has to be contiguous\");\n AT_ASSERTM(level_start_index.is_contiguous(),\n \"level_start_index tensor has to be contiguous\");\n AT_ASSERTM(sampling_loc.is_contiguous(),\n \"sampling_loc tensor has to be contiguous\");\n AT_ASSERTM(attn_weight.is_contiguous(),\n \"attn_weight tensor has to be contiguous\");\n AT_ASSERTM(grad_output.is_contiguous(),\n \"grad_output tensor has to be contiguous\");\n\n AT_ASSERTM(value.is_cuda(), \"value must be a CUDA tensor\");\n AT_ASSERTM(spatial_shapes.is_cuda(), \"spatial_shapes must be a CUDA tensor\");\n AT_ASSERTM(level_start_index.is_cuda(),\n \"level_start_index must be a CUDA tensor\");\n AT_ASSERTM(sampling_loc.is_cuda(), \"sampling_loc must be a CUDA tensor\");\n AT_ASSERTM(attn_weight.is_cuda(), \"attn_weight must be a CUDA tensor\");\n AT_ASSERTM(grad_output.is_cuda(), \"grad_output must be a CUDA tensor\");\n\n const int batch = value.size(0);\n const int spatial_size = value.size(1);\n const int num_heads = value.size(2);\n const int channels = value.size(3);\n\n const int num_levels = spatial_shapes.size(0);\n\n const int num_query = sampling_loc.size(1);\n const int num_point = sampling_loc.size(4);\n\n const int im2col_step_ = std::min(batch, im2col_step);\n\n AT_ASSERTM(batch % im2col_step_ == 0, \"batch(%d) must divide im2col_step(%d)\",\n batch, im2col_step_);\n\n const int batch_n = im2col_step_;\n auto per_value_size = spatial_size * num_heads * channels;\n auto per_sample_loc_size = num_query * num_heads * num_levels * num_point * 2;\n auto per_attn_weight_size = num_query * num_heads * num_levels * num_point;\n auto grad_output_n = grad_output.view(\n {batch / im2col_step_, batch_n, num_query, num_heads, channels});\n\n for (int n = 0; n < batch / im2col_step_; ++n) {\n auto grad_output_g = grad_output_n.select(0, n);\n AT_DISPATCH_FLOATING_TYPES(\n value.scalar_type(), \"ms_deform_attn_backward_cuda\", ([&] {\n ms_deformable_col2im_cuda(\n at::cuda::getCurrentCUDAStream(),\n grad_output_g.data_ptr(),\n value.data_ptr() + n * im2col_step_ * per_value_size,\n spatial_shapes.data_ptr(),\n level_start_index.data_ptr(),\n sampling_loc.data_ptr() +\n n * im2col_step_ * per_sample_loc_size,\n attn_weight.data_ptr() +\n n * im2col_step_ * per_attn_weight_size,\n batch_n, spatial_size, num_heads, channels, num_levels, num_query,\n num_point,\n grad_value.data_ptr() +\n n * im2col_step_ * per_value_size,\n grad_sampling_loc.data_ptr() +\n n * im2col_step_ * per_sample_loc_size,\n grad_attn_weight.data_ptr() +\n n * im2col_step_ * per_attn_weight_size);\n }));\n }\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/nms_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"nms_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nTensor NMSCUDAKernelLauncher(Tensor boxes, Tensor scores, float iou_threshold,\n int offset) {\n at::cuda::CUDAGuard device_guard(boxes.device());\n\n if (boxes.numel() == 0) {\n return at::empty({0}, boxes.options().dtype(at::kLong));\n }\n auto order_t = std::get<1>(scores.sort(0, /*descending=*/true));\n auto boxes_sorted = boxes.index_select(0, order_t);\n\n int boxes_num = boxes.size(0);\n const int col_blocks = (boxes_num + threadsPerBlock - 1) / threadsPerBlock;\n const int col_blocks_alloc = GET_BLOCKS(boxes_num, threadsPerBlock);\n Tensor mask =\n at::empty({boxes_num, col_blocks}, boxes.options().dtype(at::kLong));\n dim3 blocks(col_blocks_alloc, col_blocks_alloc);\n dim3 threads(threadsPerBlock);\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n nms_cuda<<>>(\n boxes_num, iou_threshold, offset, boxes_sorted.data_ptr(),\n (unsigned long long*)mask.data_ptr());\n\n at::Tensor mask_cpu = mask.to(at::kCPU);\n unsigned long long* mask_host =\n (unsigned long long*)mask_cpu.data_ptr();\n\n std::vector remv(col_blocks);\n memset(&remv[0], 0, sizeof(unsigned long long) * col_blocks);\n\n at::Tensor keep_t =\n at::zeros({boxes_num}, boxes.options().dtype(at::kBool).device(at::kCPU));\n bool* keep = keep_t.data_ptr();\n\n for (int i = 0; i < boxes_num; i++) {\n int nblock = i / threadsPerBlock;\n int inblock = i % threadsPerBlock;\n\n if (!(remv[nblock] & (1ULL << inblock))) {\n keep[i] = true;\n // set every overlap box with bit 1 in remv\n unsigned long long* p = mask_host + i * col_blocks;\n for (int j = nblock; j < col_blocks; j++) {\n remv[j] |= p[j];\n }\n }\n }\n\n AT_CUDA_CHECK(cudaGetLastError());\n return order_t.masked_select(keep_t.to(at::kCUDA));\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/nms_rotated_cuda.cu", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/nms_rotated/nms_rotated_cuda.cu\n#include \"nms_rotated_cuda.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nTensor nms_rotated_cuda(const Tensor dets, const Tensor scores,\n const Tensor order_t, const Tensor dets_sorted,\n float iou_threshold, const int multi_label) {\n // using scalar_t = float;\n AT_ASSERTM(dets.is_cuda(), \"dets must be a CUDA tensor\");\n AT_ASSERTM(scores.is_cuda(), \"scores must be a CUDA tensor\");\n at::cuda::CUDAGuard device_guard(dets.device());\n\n int dets_num = dets.size(0);\n\n const int col_blocks = at::cuda::ATenCeilDiv(dets_num, threadsPerBlock);\n\n Tensor mask =\n at::empty({dets_num * col_blocks}, dets.options().dtype(at::kLong));\n\n dim3 blocks(col_blocks, col_blocks);\n dim3 threads(threadsPerBlock);\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n dets_sorted.scalar_type(), \"nms_rotated_kernel_cuda\", [&] {\n nms_rotated_cuda_kernel<<>>(\n dets_num, iou_threshold, dets_sorted.data_ptr(),\n (unsigned long long*)mask.data_ptr(), multi_label);\n });\n\n Tensor mask_cpu = mask.to(at::kCPU);\n unsigned long long* mask_host =\n (unsigned long long*)mask_cpu.data_ptr();\n\n std::vector remv(col_blocks);\n memset(&remv[0], 0, sizeof(unsigned long long) * col_blocks);\n\n Tensor keep =\n at::empty({dets_num}, dets.options().dtype(at::kLong).device(at::kCPU));\n int64_t* keep_out = keep.data_ptr();\n\n int num_to_keep = 0;\n for (int i = 0; i < dets_num; i++) {\n int nblock = i / threadsPerBlock;\n int inblock = i % threadsPerBlock;\n\n if (!(remv[nblock] & (1ULL << inblock))) {\n keep_out[num_to_keep++] = i;\n unsigned long long* p = mask_host + i * col_blocks;\n for (int j = nblock; j < col_blocks; j++) {\n remv[j] |= p[j];\n }\n }\n }\n\n AT_CUDA_CHECK(cudaGetLastError());\n return order_t.index(\n {keep.narrow(/*dim=*/0, /*start=*/0, /*length=*/num_to_keep)\n .to(order_t.device(), keep.scalar_type())});\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/points_in_boxes_cuda.cu", "content": "// Modified from\n// https://github.com/sshaoshuai/PCDet/blob/master/pcdet/ops/roiaware_pool3d/src/roiaware_pool3d_kernel.cu\n// Written by Shaoshuai Shi\n// All Rights Reserved 2019.\n\n#include \n\n#include \"points_in_boxes_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid PointsInBoxesPartForwardCUDAKernelLauncher(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n // params boxes: (B, N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is\n // the bottom center, each box DO NOT overlaps params pts: (B, npoints, 3) [x,\n // y, z] in LiDAR coordinate params boxes_idx_of_points: (B, npoints), default\n // -1\n\n at::cuda::CUDAGuard device_guard(boxes.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n dim3 blocks(GET_BLOCKS(pts_num, THREADS_PER_BLOCK), batch_size);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n boxes.scalar_type(), \"points_in_boxes_part_forward_cuda_kernel\", [&] {\n points_in_boxes_part_forward_cuda_kernel\n <<>>(\n batch_size, boxes_num, pts_num, boxes.data_ptr(),\n pts.data_ptr(), box_idx_of_points.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid PointsInBoxesAllForwardCUDAKernelLauncher(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n // params boxes: (B, N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is the bottom center, each box params pts: (B, npoints, 3)\n // [x, y, z] in LiDAR coordinate params boxes_idx_of_points: (B, npoints),\n // default -1\n\n at::cuda::CUDAGuard device_guard(boxes.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n dim3 blocks(GET_BLOCKS(pts_num, THREADS_PER_BLOCK), batch_size);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n boxes.scalar_type(), \"points_in_boxes_all_forward_cuda_kernel\", [&] {\n points_in_boxes_all_forward_cuda_kernel\n <<>>(\n batch_size, boxes_num, pts_num, boxes.data_ptr(),\n pts.data_ptr(), box_idx_of_points.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/points_in_polygons_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/ming71/CUDA/blob/master/point_justify/points_justify_kernel.cu\n\n#include \n\n#include \"points_in_polygons_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid PointsInPolygonsForwardCUDAKernelLauncher(const at::Tensor points,\n const at::Tensor polygons,\n const int rows, const int cols,\n at::Tensor output) {\n const int output_size = rows * cols;\n at::cuda::CUDAGuard device_guard(points.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n points.scalar_type(), \"points_in_polygons_forward_cuda_kernel\", ([&] {\n const scalar_t *vertex1 = points.data_ptr();\n const scalar_t *vertex2 = polygons.data_ptr();\n scalar_t *inside_flag = output.data_ptr();\n\n points_in_polygons_forward_cuda_kernel\n <<>>(\n output_size, vertex1, vertex2, rows, cols, inside_flag);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/psamask_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/hszhao/semseg/blob/master/lib/psa/src\n\n#include \n#include \n\n#include \n\n#include \"psamask_cuda_kernel.cuh\"\n#include \"pytorch_cuda_helper.hpp\"\n\nvoid PSAMaskForwardCUDAKernelLauncher(const int psa_type, const Tensor input,\n Tensor output, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask,\n const int half_w_mask) {\n int nthreads = num_ * h_feature * w_feature;\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n if (psa_type == 0)\n AT_DISPATCH_FLOATING_TYPES(\n input.scalar_type(), \"psamask_collect_forward_cuda\", [&] {\n psamask_collect_forward_cuda<<>>(\n nthreads, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask, input.data_ptr(),\n output.data_ptr());\n });\n else\n AT_DISPATCH_FLOATING_TYPES(\n input.scalar_type(), \"psamask_distribute_forward_cuda\", [&] {\n psamask_distribute_forward_cuda\n <<>>(\n nthreads, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask, input.data_ptr(),\n output.data_ptr());\n });\n}\n\nvoid PSAMaskBackwardCUDAKernelLauncher(\n const int psa_type, const Tensor grad_output, Tensor grad_input,\n const int num_, const int h_feature, const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask, const int half_w_mask) {\n int nthreads = num_ * h_feature * w_feature;\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n if (psa_type == 0)\n AT_DISPATCH_FLOATING_TYPES(\n grad_input.scalar_type(), \"psamask_collect_backward_cuda\", [&] {\n psamask_collect_backward_cuda<<>>(\n nthreads, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask, grad_output.data_ptr(),\n grad_input.data_ptr());\n });\n else\n AT_DISPATCH_FLOATING_TYPES(\n grad_input.scalar_type(), \"psamask_distribute_backward_cuda\", [&] {\n psamask_distribute_backward_cuda\n <<>>(\n nthreads, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask, grad_output.data_ptr(),\n grad_input.data_ptr());\n });\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/riroi_align_rotated_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cuda_helper.hpp\"\n#include \"riroi_align_rotated_cuda_kernel.cuh\"\n\nvoid RiROIAlignRotatedForwardCUDAKernelLauncher(\n const at::Tensor features, const at::Tensor rois, const float spatial_scale,\n const int num_samples, const bool clockwise, const int channels,\n const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, const int num_orientations,\n at::Tensor output) {\n const int output_size =\n num_rois * pooled_height * pooled_width * channels * num_orientations;\n at::cuda::CUDAGuard device_guard(features.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n features.scalar_type(), \"riroi_align_rotated_forward_cuda_kernel\", ([&] {\n const scalar_t *bottom_data = features.data_ptr();\n const scalar_t *rois_data = rois.data_ptr();\n scalar_t *top_data = output.data_ptr();\n\n riroi_align_rotated_forward_cuda_kernel\n <<>>(\n output_size, bottom_data, rois_data, scalar_t(spatial_scale),\n num_samples, clockwise, channels, height, width, pooled_height,\n pooled_width, num_orientations, top_data);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid RiROIAlignRotatedBackwardCUDAKernelLauncher(\n const at::Tensor top_grad, const at::Tensor rois, const float spatial_scale,\n const int num_samples, const bool clockwise, const int channels,\n const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, const int num_orientations,\n at::Tensor bottom_grad) {\n const int output_size =\n num_rois * pooled_height * pooled_width * channels * num_orientations;\n at::cuda::CUDAGuard device_guard(top_grad.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"riroi_align_rotated_backward_cuda_kernel\", ([&] {\n const scalar_t *top_diff = top_grad.data_ptr();\n const scalar_t *rois_data = rois.data_ptr();\n scalar_t *bottom_diff = bottom_grad.data_ptr();\n riroi_align_rotated_backward_cuda_kernel\n <<>>(\n output_size, top_diff, rois_data, spatial_scale, num_samples,\n clockwise, channels, height, width, pooled_height, pooled_width,\n num_orientations, bottom_diff);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/roi_align_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cuda_helper.hpp\"\n#include \"roi_align_cuda_kernel.cuh\"\n\nvoid ROIAlignForwardCUDAKernelLauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n int output_size = output.numel();\n int channels = input.size(1);\n int height = input.size(2);\n int width = input.size(3);\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"roi_align_forward_cuda_kernel\", [&] {\n roi_align_forward_cuda_kernel\n <<>>(\n output_size, input.data_ptr(),\n rois.data_ptr(), output.data_ptr(),\n argmax_y.data_ptr(), argmax_x.data_ptr(),\n aligned_height, aligned_width,\n static_cast(spatial_scale), sampling_ratio, pool_mode,\n aligned, channels, height, width);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid ROIAlignBackwardCUDAKernelLauncher(Tensor grad_output, Tensor rois,\n Tensor argmax_y, Tensor argmax_x,\n Tensor grad_input, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode,\n bool aligned) {\n int output_size = grad_output.numel();\n int channels = grad_input.size(1);\n int height = grad_input.size(2);\n int width = grad_input.size(3);\n\n at::cuda::CUDAGuard device_guard(grad_output.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"roi_align_backward_cuda_kernel\", [&] {\n roi_align_backward_cuda_kernel\n <<>>(\n output_size, grad_output.data_ptr(),\n rois.data_ptr(), argmax_y.data_ptr(),\n argmax_x.data_ptr(), grad_input.data_ptr(),\n aligned_height, aligned_width,\n static_cast(spatial_scale), sampling_ratio, pool_mode,\n aligned, channels, height, width);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/roi_align_rotated_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cuda_helper.hpp\"\n#include \"roi_align_rotated_cuda_kernel.cuh\"\n\nvoid ROIAlignRotatedForwardCUDAKernelLauncher(\n const at::Tensor features, const at::Tensor rois, const float spatial_scale,\n const int sample_num, const bool aligned, const bool clockwise,\n const int channels, const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, at::Tensor output) {\n const int output_size = num_rois * pooled_height * pooled_width * channels;\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n features.scalar_type(), \"ROIAlignRotatedLaucherForward\", ([&] {\n const scalar_t *bottom_data = features.data_ptr();\n const scalar_t *rois_data = rois.data_ptr();\n scalar_t *top_data = output.data_ptr();\n\n roi_align_rotated_forward_cuda_kernel\n <<>>(\n output_size, bottom_data, rois_data, scalar_t(spatial_scale),\n sample_num, aligned, clockwise, channels, height, width,\n pooled_height, pooled_width, top_data);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid ROIAlignRotatedBackwardCUDAKernelLauncher(\n const at::Tensor top_grad, const at::Tensor rois, const float spatial_scale,\n const int sample_num, const bool aligned, const bool clockwise,\n const int channels, const int height, const int width, const int num_rois,\n const int pooled_height, const int pooled_width, at::Tensor bottom_grad) {\n const int output_size = num_rois * pooled_height * pooled_width * channels;\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"ROIAlignLaucherBackward\", ([&] {\n const scalar_t *top_diff = top_grad.data_ptr();\n const scalar_t *rois_data = rois.data_ptr();\n scalar_t *bottom_diff = bottom_grad.data_ptr();\n roi_align_rotated_backward_cuda_kernel\n <<>>(\n output_size, top_diff, rois_data, spatial_scale, sample_num,\n aligned, clockwise, channels, height, width, pooled_height,\n pooled_width, bottom_diff);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/roi_pool_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cuda_helper.hpp\"\n#include \"roi_pool_cuda_kernel.cuh\"\n\nvoid ROIPoolForwardCUDAKernelLauncher(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height,\n int pooled_width, float spatial_scale) {\n int output_size = output.numel();\n int channels = input.size(1);\n int height = input.size(2);\n int width = input.size(3);\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"roi_pool_forward_cuda_kernel\", [&] {\n roi_pool_forward_cuda_kernel\n <<>>(\n output_size, input.data_ptr(),\n rois.data_ptr(), output.data_ptr(),\n argmax.data_ptr(), pooled_height, pooled_width,\n static_cast(spatial_scale), channels, height, width);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid ROIPoolBackwardCUDAKernelLauncher(Tensor grad_output, Tensor rois,\n Tensor argmax, Tensor grad_input,\n int pooled_height, int pooled_width,\n float spatial_scale) {\n int output_size = grad_output.numel();\n int channels = grad_input.size(1);\n int height = grad_input.size(2);\n int width = grad_input.size(3);\n\n at::cuda::CUDAGuard device_guard(grad_output.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"roi_pool_backward_cuda_kernel\", [&] {\n roi_pool_backward_cuda_kernel\n <<>>(\n output_size, grad_output.data_ptr(),\n rois.data_ptr(), argmax.data_ptr(),\n grad_input.data_ptr(), pooled_height, pooled_width,\n channels, height, width);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/roiaware_pool3d_cuda.cu", "content": "// Modified from\n// https://github.com/sshaoshuai/PCDet/blob/master/pcdet/ops/roiaware_pool3d/src/roiaware_pool3d_kernel.cu\n// Written by Shaoshuai Shi\n// All Rights Reserved 2019.\n\n#include \n\n#include \"pytorch_cuda_helper.hpp\"\n#include \"roiaware_pool3d_cuda_kernel.cuh\"\n\nvoid RoiawarePool3dForwardCUDAKernelLauncher(\n int boxes_num, int pts_num, int channels, int max_pts_each_voxel, int out_x,\n int out_y, int out_z, const Tensor rois, const Tensor pts,\n const Tensor pts_feature, Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method) {\n // params rois: (N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate params pts: (npoints, 3) [x, y, z] in LiDAR coordinate params\n // pts_feature: (npoints, C) params argmax: (N, out_x, out_y, out_z, C) params\n // pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel) params\n // pooled_features: (N, out_x, out_y, out_z, C) params pool_method: 0:\n // max_pool 1: avg_pool\n\n at::cuda::CUDAGuard device_guard(pts_feature.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n Tensor pts_mask =\n -at::ones({boxes_num, pts_num}, pts_feature.options().dtype(at::kInt));\n\n dim3 blocks_mask(GET_BLOCKS(pts_num, THREADS_PER_BLOCK), boxes_num);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n rois.scalar_type(), \"generate_pts_mask_for_box3d\", [&] {\n generate_pts_mask_for_box3d\n <<>>(\n boxes_num, pts_num, out_x, out_y, out_z,\n rois.data_ptr(), pts.data_ptr(),\n pts_mask.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n // TODO: Merge the collect and pool functions, SS\n\n dim3 blocks_collect(GET_BLOCKS(boxes_num, THREADS_PER_BLOCK));\n\n AT_DISPATCH_INTEGRAL_TYPES(\n pts_idx_of_voxels.scalar_type(), \"collect_inside_pts_for_box3d\", [&] {\n collect_inside_pts_for_box3d\n <<>>(\n boxes_num, pts_num, max_pts_each_voxel, out_x, out_y, out_z,\n pts_mask.data_ptr(),\n pts_idx_of_voxels.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n dim3 blocks_pool(GET_BLOCKS(out_x * out_y * out_z, THREADS_PER_BLOCK),\n channels, boxes_num);\n if (pool_method == 0) {\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n pts_feature.scalar_type(), \"roiaware_maxpool3d\", [&] {\n roiaware_maxpool3d<<>>(\n boxes_num, pts_num, channels, max_pts_each_voxel, out_x, out_y,\n out_z, pts_feature.data_ptr(),\n pts_idx_of_voxels.data_ptr(),\n pooled_features.data_ptr(), argmax.data_ptr());\n });\n } else if (pool_method == 1) {\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n pts_feature.scalar_type(), \"roiaware_avgpool3d\", [&] {\n roiaware_avgpool3d<<>>(\n boxes_num, pts_num, channels, max_pts_each_voxel, out_x, out_y,\n out_z, pts_feature.data_ptr(),\n pts_idx_of_voxels.data_ptr(),\n pooled_features.data_ptr());\n });\n }\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid RoiawarePool3dBackwardCUDAKernelLauncher(\n int boxes_num, int out_x, int out_y, int out_z, int channels,\n int max_pts_each_voxel, const Tensor pts_idx_of_voxels, const Tensor argmax,\n const Tensor grad_out, Tensor grad_in, int pool_method) {\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel)\n // params argmax: (N, out_x, out_y, out_z, C)\n // params grad_out: (N, out_x, out_y, out_z, C)\n // params grad_in: (npoints, C), return value\n // params pool_method: 0: max_pool, 1: avg_pool\n\n at::cuda::CUDAGuard device_guard(grad_out.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n dim3 blocks(GET_BLOCKS(out_x * out_y * out_z, THREADS_PER_BLOCK), channels,\n boxes_num);\n dim3 threads(THREADS_PER_BLOCK);\n\n if (pool_method == 0) {\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_in.scalar_type(), \"roiaware_maxpool3d_backward\", [&] {\n roiaware_maxpool3d_backward<<>>(\n boxes_num, channels, out_x, out_y, out_z, argmax.data_ptr(),\n grad_out.data_ptr(), grad_in.data_ptr());\n });\n } else if (pool_method == 1) {\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_in.scalar_type(), \"roiaware_avgpool3d_backward\", [&] {\n roiaware_avgpool3d_backward<<>>(\n boxes_num, channels, out_x, out_y, out_z, max_pts_each_voxel,\n pts_idx_of_voxels.data_ptr(), grad_out.data_ptr(),\n grad_in.data_ptr());\n });\n }\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/roipoint_pool3d_cuda.cu", "content": "/*\nModified from\nhttps://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/ops/roipoint_pool3d/src/roipoint_pool3d_kernel.cu\nPoint cloud feature pooling\nWritten by Shaoshuai Shi\nAll Rights Reserved 2018.\n*/\n\n#include \n#include \n\n#include \"pytorch_cuda_helper.hpp\"\n#include \"roipoint_pool3d_cuda_kernel.cuh\"\n\nvoid RoIPointPool3dForwardCUDAKernelLauncher(\n int batch_size, int pts_num, int boxes_num, int feature_in_len,\n int sampled_pts_num, const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature, Tensor pooled_features,\n Tensor pooled_empty_flag) {\n Tensor pts_assign = at::empty({batch_size, pts_num, boxes_num},\n boxes3d.options().dtype(at::kInt));\n\n at::cuda::CUDAGuard device_guard(xyz.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(pts_num, THREADS_PER_BLOCK), boxes_num, batch_size);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n xyz.scalar_type(), \"assign_pts_to_box3d\", [&] {\n assign_pts_to_box3d<<>>(\n batch_size, pts_num, boxes_num, xyz.data_ptr(),\n boxes3d.data_ptr(), pts_assign.data_ptr());\n });\n\n Tensor pts_idx = at::empty({batch_size, boxes_num, sampled_pts_num},\n boxes3d.options().dtype(at::kInt));\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks2(GET_BLOCKS(boxes_num, THREADS_PER_BLOCK), batch_size);\n\n get_pooled_idx<<>>(\n batch_size, pts_num, boxes_num, sampled_pts_num,\n pts_assign.data_ptr(), pts_idx.data_ptr(),\n pooled_empty_flag.data_ptr());\n\n dim3 blocks_pool(GET_BLOCKS(sampled_pts_num, THREADS_PER_BLOCK), boxes_num,\n batch_size);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n xyz.scalar_type(), \"roipoint_pool3d_forward\", [&] {\n roipoint_pool3d_forward<<>>(\n batch_size, pts_num, boxes_num, feature_in_len, sampled_pts_num,\n xyz.data_ptr(), pts_idx.data_ptr(),\n pts_feature.data_ptr(),\n pooled_features.data_ptr(),\n pooled_empty_flag.data_ptr());\n });\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/rotated_feature_align_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/SJTU-Thinklab-Det/r3det-on-mmdetection/blob/master/mmdet/ops/fr/src/feature_refine_kernel.cu\n#include \"pytorch_cuda_helper.hpp\"\n#include \"rotated_feature_align_cuda_kernel.cuh\"\n\nvoid RotatedFeatureAlignForwardCUDAKernelLauncher(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points,\n Tensor output) {\n at::cuda::CUDAGuard device_guard(features.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n const int output_size = features.numel();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n features.scalar_type(), \"rotated_feature_align_forward_cuda_kernel\",\n ([&] {\n const scalar_t* bottom_data = features.data_ptr();\n const scalar_t* bboxes_data = best_bboxes.data_ptr();\n scalar_t* top_data = output.data_ptr();\n\n rotated_feature_align_forward_kernel\n <<>>(\n output_size, points, bottom_data, bboxes_data,\n scalar_t(spatial_scale), features.size(1), features.size(2),\n features.size(3), top_data);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid RotatedFeatureAlignBackwardCUDAKernelLauncher(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points,\n Tensor bottom_grad) {\n at::cuda::CUDAGuard device_guard(top_grad.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n const int output_size = top_grad.numel();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n top_grad.scalar_type(), \"rotated_feature_align_backward_cuda_kernel\",\n ([&] {\n const scalar_t* top_diff = top_grad.data_ptr();\n const scalar_t* bboxes_data = best_bboxes.data_ptr();\n scalar_t* bottom_diff = bottom_grad.data_ptr();\n\n rotated_feature_align_backward_kernel\n <<>>(\n output_size, points, top_diff, bboxes_data,\n scalar_t(spatial_scale), top_grad.size(1), top_grad.size(2),\n top_grad.size(3), bottom_diff);\n }));\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/scatter_points_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#include \n#include \n#include \n\n#include \"pytorch_cuda_helper.hpp\"\n#include \"scatter_points_cuda_kernel.cuh\"\n\nstd::vector DynamicPointToVoxelForwardCUDAKernelLauncher(\n const at::Tensor &feats, const at::Tensor &coors,\n const reduce_t reduce_type) {\n const int num_input = feats.size(0);\n const int num_feats = feats.size(1);\n\n if (num_input == 0)\n return {feats.clone().detach(), coors.clone().detach(),\n coors.new_empty({0}, torch::kInt32),\n coors.new_empty({0}, torch::kInt32)};\n\n at::Tensor out_coors;\n at::Tensor coors_map;\n at::Tensor reduce_count;\n\n auto coors_clean = coors.masked_fill(coors.lt(0).any(-1, true), -1);\n\n std::tie(out_coors, coors_map, reduce_count) =\n at::unique_dim(coors_clean, 0, true, true, true);\n\n // the first element of out_coors is always (-1,-1,-1) and should be removed\n out_coors = out_coors.slice(0, 1);\n reduce_count = reduce_count.slice(0, 1).to(torch::kInt32);\n coors_map = coors_map.to(torch::kInt32) - 1;\n\n auto reduced_feats =\n at::empty({out_coors.size(0), num_feats}, feats.options());\n\n at::cuda::CUDAGuard device_guard(feats.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n AT_DISPATCH_FLOATING_TYPES(\n feats.scalar_type(), \"feats_reduce_kernel\", ([&] {\n if (reduce_type == reduce_t::MAX)\n reduced_feats.fill_(-std::numeric_limits::infinity());\n else\n reduced_feats.fill_(static_cast(0));\n\n dim3 blocks(std::min(\n at::cuda::ATenCeilDiv(num_input, THREADS_PER_BLOCK), maxGridDim));\n dim3 threads(THREADS_PER_BLOCK);\n feats_reduce_kernel<<>>(\n feats.data_ptr(), coors_map.data_ptr(),\n reduced_feats.data_ptr(), num_input, num_feats,\n reduce_type);\n if (reduce_type == reduce_t::MEAN)\n reduced_feats /= reduce_count.unsqueeze(-1).to(reduced_feats.dtype());\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n return {reduced_feats, out_coors, coors_map, reduce_count};\n}\n\nvoid DynamicPointToVoxelBackwardCUDAKernelLauncher(\n at::Tensor &grad_feats, const at::Tensor &grad_reduced_feats,\n const at::Tensor &feats, const at::Tensor &reduced_feats,\n const at::Tensor &coors_map, const at::Tensor &reduce_count,\n const reduce_t reduce_type) {\n const int num_input = feats.size(0);\n const int num_reduced = reduced_feats.size(0);\n const int num_feats = feats.size(1);\n\n grad_feats.fill_(0);\n // copy voxel grad to points\n\n if (num_input == 0 || num_reduced == 0) return;\n at::cuda::CUDAGuard device_guard(feats.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n if (reduce_type == reduce_t::MEAN || reduce_type == reduce_t::SUM) {\n AT_DISPATCH_FLOATING_TYPES(\n grad_reduced_feats.scalar_type(), \"add_reduce_traceback_grad_kernel\",\n ([&] {\n dim3 blocks(std::min(\n at::cuda::ATenCeilDiv(num_input, THREADS_PER_BLOCK), maxGridDim));\n dim3 threads(THREADS_PER_BLOCK);\n add_reduce_traceback_grad_kernel<<>>(\n grad_feats.data_ptr(),\n grad_reduced_feats.data_ptr(),\n coors_map.data_ptr(), reduce_count.data_ptr(),\n num_input, num_feats, reduce_type);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n } else {\n auto reduce_from = at::full({num_reduced, num_feats}, num_input,\n coors_map.options().dtype(torch::kInt32));\n AT_DISPATCH_FLOATING_TYPES(\n grad_reduced_feats.scalar_type(),\n \"max_reduce_traceback_scatter_idx_kernel\", ([&] {\n dim3 blocks(std::min(\n at::cuda::ATenCeilDiv(num_input, THREADS_PER_BLOCK), maxGridDim));\n dim3 threads(THREADS_PER_BLOCK);\n max_reduce_traceback_scatter_idx_kernel<<>>(\n feats.data_ptr(), reduced_feats.data_ptr(),\n reduce_from.data_ptr(), coors_map.data_ptr(),\n num_input, num_feats);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n AT_DISPATCH_FLOATING_TYPES(\n grad_reduced_feats.scalar_type(),\n \"max_reduce_traceback_scatter_idx_kernel\", ([&] {\n dim3 blocks(\n std::min(at::cuda::ATenCeilDiv(num_reduced, THREADS_PER_BLOCK),\n maxGridDim));\n dim3 threads(THREADS_PER_BLOCK);\n max_reduce_scatter_grad_kernel<<>>(\n grad_feats.data_ptr(),\n grad_reduced_feats.data_ptr(),\n reduce_from.data_ptr(), num_reduced, num_feats);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n }\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/sync_bn_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cuda_helper.hpp\"\n#include \"sync_bn_cuda_kernel.cuh\"\n\nvoid SyncBNForwardMeanCUDAKernelLauncher(const Tensor input, Tensor mean) {\n int num = input.size(0);\n int channels = input.size(1);\n int spatial = input.size(2);\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"sync_bn_forward_mean_cuda_kernel\", [&] {\n sync_bn_forward_mean_cuda_kernel\n <<>>(\n input.data_ptr(), mean.data_ptr(), num,\n channels, spatial);\n });\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid SyncBNForwardVarCUDAKernelLauncher(const Tensor input, const Tensor mean,\n Tensor var) {\n int num = input.size(0);\n int channels = input.size(1);\n int spatial = input.size(2);\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"sync_bn_forward_mean_cuda_kernel\", [&] {\n sync_bn_forward_var_cuda_kernel\n <<>>(\n input.data_ptr(), mean.data_ptr(),\n var.data_ptr(), num, channels, spatial);\n });\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid SyncBNForwardOutputCUDAKernelLauncher(\n const Tensor input, const Tensor mean, const Tensor var,\n Tensor running_mean, Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std, Tensor output, float eps,\n float momentum, int group_size) {\n int num = input.size(0);\n int channels = input.size(1);\n int spatial = input.size(2);\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"sync_bn_forward_mean_cuda_kernel\", [&] {\n sync_bn_forward_output_cuda_kernel\n <<>>(\n input.data_ptr(), mean.data_ptr(),\n var.data_ptr(), running_mean.data_ptr(),\n running_var.data_ptr(), weight.data_ptr(),\n bias.data_ptr(), norm.data_ptr(),\n std.data_ptr(), output.data_ptr(), num,\n channels, spatial, eps, momentum, group_size);\n });\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid SyncBNBackwardParamCUDAKernelLauncher(const Tensor grad_output,\n const Tensor norm,\n Tensor grad_weight,\n Tensor grad_bias) {\n int num = grad_output.size(0);\n int channels = grad_output.size(1);\n int spatial = grad_output.size(2);\n\n at::cuda::CUDAGuard device_guard(grad_output.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"sync_bn_backward_param_cuda_kernel\", [&] {\n sync_bn_backward_param_cuda_kernel\n <<>>(\n grad_output.data_ptr(), norm.data_ptr(),\n grad_weight.data_ptr(), grad_bias.data_ptr(), num,\n channels, spatial);\n });\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid SyncBNBackwardDataCUDAKernelLauncher(const Tensor grad_output,\n const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias,\n const Tensor norm, const Tensor std,\n Tensor grad_input) {\n int output_size = grad_input.numel();\n int num = grad_input.size(0);\n int channels = grad_input.size(1);\n int spatial = grad_input.size(2);\n\n at::cuda::CUDAGuard device_guard(grad_input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"sync_bn_backward_data_cuda_kernel\", [&] {\n sync_bn_backward_data_cuda_kernel\n <<>>(\n output_size, grad_output.data_ptr(),\n weight.data_ptr(), grad_weight.data_ptr(),\n grad_bias.data_ptr(), norm.data_ptr(),\n std.data_ptr(), grad_input.data_ptr(), num,\n channels, spatial);\n });\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/three_interpolate_cuda.cu", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/interpolate_gpu.cu\n\n#include \n#include \n#include \n\n#include \"pytorch_cuda_helper.hpp\"\n#include \"three_interpolate_cuda_kernel.cuh\"\n\nvoid ThreeInterpolateForwardCUDAKernelLauncher(int b, int c, int m, int n,\n const Tensor points,\n const Tensor idx,\n const Tensor weight,\n Tensor out) {\n // points: (B, C, M)\n // idx: (B, N, 3)\n // weight: (B, N, 3)\n // output:\n // out: (B, C, N)\n\n at::cuda::CUDAGuard device_guard(points.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(n, THREADS_PER_BLOCK), c, b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n points.scalar_type(), \"three_interpolate_forward_cuda_kernel\", [&] {\n three_interpolate_forward_cuda_kernel\n <<>>(\n b, c, m, n, points.data_ptr(), idx.data_ptr(),\n weight.data_ptr(), out.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid ThreeInterpolateBackwardCUDAKernelLauncher(int b, int c, int n, int m,\n const Tensor grad_out,\n const Tensor idx,\n const Tensor weight,\n Tensor grad_points) {\n // grad_out: (B, C, N)\n // weight: (B, N, 3)\n // output:\n // grad_points: (B, C, M)\n\n at::cuda::CUDAGuard device_guard(grad_out.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(n, THREADS_PER_BLOCK), c, b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_out.scalar_type(), \"three_interpolate_backward_cuda_kernel\", [&] {\n three_interpolate_backward_cuda_kernel\n <<>>(\n b, c, n, m, grad_out.data_ptr(), idx.data_ptr(),\n weight.data_ptr(), grad_points.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/three_nn_cuda.cu", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/interpolate_gpu.cu\n\n#include \n#include \n#include \n\n#include \"pytorch_cuda_helper.hpp\"\n#include \"three_nn_cuda_kernel.cuh\"\n\nvoid ThreeNNForwardCUDAKernelLauncher(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2,\n Tensor idx) {\n // unknown: (B, N, 3)\n // known: (B, M, 3)\n // output:\n // dist2: (B, N, 3)\n // idx: (B, N, 3)\n\n at::cuda::CUDAGuard device_guard(unknown.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n // blockIdx.x(col), blockIdx.y(row)\n dim3 blocks(GET_BLOCKS(n, THREADS_PER_BLOCK), b);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n unknown.scalar_type(), \"three_nn_forward_cuda_kernel\", [&] {\n three_nn_forward_cuda_kernel<<>>(\n b, n, m, unknown.data_ptr(), known.data_ptr(),\n dist2.data_ptr(), idx.data_ptr());\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/tin_shift_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cuda_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n#include \"tin_shift_cuda_kernel.cuh\"\n\nvoid TINShiftForwardCUDAKernelLauncher(Tensor input, Tensor shift,\n Tensor output) {\n int output_size = output.numel();\n int batch_size = input.size(0);\n int t_size = input.size(1);\n int channels = input.size(2);\n int hw_size = input.size(3);\n int group_size = shift.size(1);\n int group_channel = channels / group_size;\n int num_kernels = batch_size * hw_size * channels;\n\n at::cuda::CUDAGuard device_guard(input.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n input.scalar_type(), \"tin_shift_forward_cuda_kernel\", [&] {\n tin_shift_forward_cuda_kernel\n <<>>(\n output_size, input.data_ptr(), shift.data_ptr(),\n output.data_ptr(), batch_size, channels, t_size,\n hw_size, group_size, group_channel);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n\nvoid TINShiftBackwardCUDAKernelLauncher(Tensor grad_output, Tensor shift,\n Tensor grad_input) {\n int output_size = grad_output.numel();\n int batch_size = grad_output.size(0);\n int t_size = grad_output.size(1);\n int channels = grad_output.size(2);\n int hw_size = grad_output.size(3);\n int group_size = shift.size(1);\n int group_channel = channels / group_size;\n int num_kernels = batch_size * hw_size * channels;\n\n at::cuda::CUDAGuard device_guard(grad_output.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(\n grad_output.scalar_type(), \"tin_shift_backward_cuda_kernel\", [&] {\n tin_shift_backward_cuda_kernel\n <<>>(\n output_size, grad_output.data_ptr(),\n shift.data_ptr(), grad_input.data_ptr(),\n batch_size, channels, t_size, hw_size, group_size,\n group_channel);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/upfirdn2d_kernel.cu", "content": "// Modified from\n// https://github.com/rosinality/stylegan2-pytorch/blob/master/op/upfirdn2d_kernel.cu\n// Copyright (c) 2019, NVIDIA Corporation. All rights reserved.\n//\n// This work is made available under the Nvidia Source Code License-NC.\n// To view a copy of this license, visit\n// https://nvlabs.github.io/stylegan2/license.html\n\n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \n\nstatic __host__ __device__ __forceinline__ int floor_div(int a, int b) {\n int c = a / b;\n\n if (c * b > a) {\n c--;\n }\n\n return c;\n}\n\nstruct UpFirDn2DKernelParams {\n int up_x;\n int up_y;\n int down_x;\n int down_y;\n int pad_x0;\n int pad_x1;\n int pad_y0;\n int pad_y1;\n\n int major_dim;\n int in_h;\n int in_w;\n int minor_dim;\n int kernel_h;\n int kernel_w;\n int out_h;\n int out_w;\n int loop_major;\n int loop_x;\n};\n\ntemplate \n__global__ void upfirdn2d_kernel_large(scalar_t *out, const scalar_t *input,\n const scalar_t *kernel,\n const UpFirDn2DKernelParams p) {\n int minor_idx = blockIdx.x * blockDim.x + threadIdx.x;\n int out_y = minor_idx / p.minor_dim;\n minor_idx -= out_y * p.minor_dim;\n int out_x_base = blockIdx.y * p.loop_x * blockDim.y + threadIdx.y;\n int major_idx_base = blockIdx.z * p.loop_major;\n\n if (out_x_base >= p.out_w || out_y >= p.out_h ||\n major_idx_base >= p.major_dim) {\n return;\n }\n\n int mid_y = out_y * p.down_y + p.up_y - 1 - p.pad_y0;\n int in_y = min(max(floor_div(mid_y, p.up_y), 0), p.in_h);\n int h = min(max(floor_div(mid_y + p.kernel_h, p.up_y), 0), p.in_h) - in_y;\n int kernel_y = mid_y + p.kernel_h - (in_y + 1) * p.up_y;\n\n for (int loop_major = 0, major_idx = major_idx_base;\n loop_major < p.loop_major && major_idx < p.major_dim;\n loop_major++, major_idx++) {\n for (int loop_x = 0, out_x = out_x_base;\n loop_x < p.loop_x && out_x < p.out_w; loop_x++, out_x += blockDim.y) {\n int mid_x = out_x * p.down_x + p.up_x - 1 - p.pad_x0;\n int in_x = min(max(floor_div(mid_x, p.up_x), 0), p.in_w);\n int w = min(max(floor_div(mid_x + p.kernel_w, p.up_x), 0), p.in_w) - in_x;\n int kernel_x = mid_x + p.kernel_w - (in_x + 1) * p.up_x;\n\n const scalar_t *x_p =\n &input[((major_idx * p.in_h + in_y) * p.in_w + in_x) * p.minor_dim +\n minor_idx];\n const scalar_t *k_p = &kernel[kernel_y * p.kernel_w + kernel_x];\n int x_px = p.minor_dim;\n int k_px = -p.up_x;\n int x_py = p.in_w * p.minor_dim;\n int k_py = -p.up_y * p.kernel_w;\n\n scalar_t v = 0.0f;\n\n for (int y = 0; y < h; y++) {\n for (int x = 0; x < w; x++) {\n v += static_cast(*x_p) * static_cast(*k_p);\n x_p += x_px;\n k_p += k_px;\n }\n\n x_p += x_py - w * x_px;\n k_p += k_py - w * k_px;\n }\n\n out[((major_idx * p.out_h + out_y) * p.out_w + out_x) * p.minor_dim +\n minor_idx] = v;\n }\n }\n}\n\ntemplate \n__global__ void upfirdn2d_kernel(scalar_t *out, const scalar_t *input,\n const scalar_t *kernel,\n const UpFirDn2DKernelParams p) {\n const int tile_in_h = ((tile_out_h - 1) * down_y + kernel_h - 1) / up_y + 1;\n const int tile_in_w = ((tile_out_w - 1) * down_x + kernel_w - 1) / up_x + 1;\n\n __shared__ volatile float sk[kernel_h][kernel_w];\n __shared__ volatile float sx[tile_in_h][tile_in_w];\n\n int minor_idx = blockIdx.x;\n int tile_out_y = minor_idx / p.minor_dim;\n minor_idx -= tile_out_y * p.minor_dim;\n tile_out_y *= tile_out_h;\n int tile_out_x_base = blockIdx.y * p.loop_x * tile_out_w;\n int major_idx_base = blockIdx.z * p.loop_major;\n\n if (tile_out_x_base >= p.out_w | tile_out_y >= p.out_h |\n major_idx_base >= p.major_dim) {\n return;\n }\n\n for (int tap_idx = threadIdx.x; tap_idx < kernel_h * kernel_w;\n tap_idx += blockDim.x) {\n int ky = tap_idx / kernel_w;\n int kx = tap_idx - ky * kernel_w;\n scalar_t v = 0.0;\n\n if (kx < p.kernel_w & ky < p.kernel_h) {\n v = kernel[(p.kernel_h - 1 - ky) * p.kernel_w + (p.kernel_w - 1 - kx)];\n }\n\n sk[ky][kx] = v;\n }\n\n for (int loop_major = 0, major_idx = major_idx_base;\n loop_major < p.loop_major & major_idx < p.major_dim;\n loop_major++, major_idx++) {\n for (int loop_x = 0, tile_out_x = tile_out_x_base;\n loop_x < p.loop_x & tile_out_x < p.out_w;\n loop_x++, tile_out_x += tile_out_w) {\n int tile_mid_x = tile_out_x * down_x + up_x - 1 - p.pad_x0;\n int tile_mid_y = tile_out_y * down_y + up_y - 1 - p.pad_y0;\n int tile_in_x = floor_div(tile_mid_x, up_x);\n int tile_in_y = floor_div(tile_mid_y, up_y);\n\n __syncthreads();\n\n for (int in_idx = threadIdx.x; in_idx < tile_in_h * tile_in_w;\n in_idx += blockDim.x) {\n int rel_in_y = in_idx / tile_in_w;\n int rel_in_x = in_idx - rel_in_y * tile_in_w;\n int in_x = rel_in_x + tile_in_x;\n int in_y = rel_in_y + tile_in_y;\n\n scalar_t v = 0.0;\n\n if (in_x >= 0 & in_y >= 0 & in_x < p.in_w & in_y < p.in_h) {\n v = input[((major_idx * p.in_h + in_y) * p.in_w + in_x) *\n p.minor_dim +\n minor_idx];\n }\n\n sx[rel_in_y][rel_in_x] = v;\n }\n\n __syncthreads();\n for (int out_idx = threadIdx.x; out_idx < tile_out_h * tile_out_w;\n out_idx += blockDim.x) {\n int rel_out_y = out_idx / tile_out_w;\n int rel_out_x = out_idx - rel_out_y * tile_out_w;\n int out_x = rel_out_x + tile_out_x;\n int out_y = rel_out_y + tile_out_y;\n\n int mid_x = tile_mid_x + rel_out_x * down_x;\n int mid_y = tile_mid_y + rel_out_y * down_y;\n int in_x = floor_div(mid_x, up_x);\n int in_y = floor_div(mid_y, up_y);\n int rel_in_x = in_x - tile_in_x;\n int rel_in_y = in_y - tile_in_y;\n int kernel_x = (in_x + 1) * up_x - mid_x - 1;\n int kernel_y = (in_y + 1) * up_y - mid_y - 1;\n\n scalar_t v = 0.0;\n\n#pragma unroll\n for (int y = 0; y < kernel_h / up_y; y++)\n#pragma unroll\n for (int x = 0; x < kernel_w / up_x; x++)\n v += sx[rel_in_y + y][rel_in_x + x] *\n sk[kernel_y + y * up_y][kernel_x + x * up_x];\n\n if (out_x < p.out_w & out_y < p.out_h) {\n out[((major_idx * p.out_h + out_y) * p.out_w + out_x) * p.minor_dim +\n minor_idx] = v;\n }\n }\n }\n }\n}\n\ntorch::Tensor upfirdn2d_op(const torch::Tensor &input,\n const torch::Tensor &kernel, int up_x, int up_y,\n int down_x, int down_y, int pad_x0, int pad_x1,\n int pad_y0, int pad_y1) {\n int curDevice = -1;\n cudaGetDevice(&curDevice);\n cudaStream_t stream = at::cuda::getCurrentCUDAStream(curDevice);\n\n UpFirDn2DKernelParams p;\n\n auto x = input.contiguous();\n auto k = kernel.contiguous();\n\n p.major_dim = x.size(0);\n p.in_h = x.size(1);\n p.in_w = x.size(2);\n p.minor_dim = x.size(3);\n p.kernel_h = k.size(0);\n p.kernel_w = k.size(1);\n p.up_x = up_x;\n p.up_y = up_y;\n p.down_x = down_x;\n p.down_y = down_y;\n p.pad_x0 = pad_x0;\n p.pad_x1 = pad_x1;\n p.pad_y0 = pad_y0;\n p.pad_y1 = pad_y1;\n\n p.out_h = (p.in_h * p.up_y + p.pad_y0 + p.pad_y1 - p.kernel_h + p.down_y) /\n p.down_y;\n p.out_w = (p.in_w * p.up_x + p.pad_x0 + p.pad_x1 - p.kernel_w + p.down_x) /\n p.down_x;\n\n auto out =\n at::empty({p.major_dim, p.out_h, p.out_w, p.minor_dim}, x.options());\n\n int mode = -1;\n\n int tile_out_h = -1;\n int tile_out_w = -1;\n\n if (p.up_x == 1 && p.up_y == 1 && p.down_x == 1 && p.down_y == 1 &&\n p.kernel_h <= 4 && p.kernel_w <= 4) {\n mode = 1;\n tile_out_h = 16;\n tile_out_w = 64;\n }\n\n if (p.up_x == 1 && p.up_y == 1 && p.down_x == 1 && p.down_y == 1 &&\n p.kernel_h <= 3 && p.kernel_w <= 3) {\n mode = 2;\n tile_out_h = 16;\n tile_out_w = 64;\n }\n\n if (p.up_x == 2 && p.up_y == 2 && p.down_x == 1 && p.down_y == 1 &&\n p.kernel_h <= 4 && p.kernel_w <= 4) {\n mode = 3;\n tile_out_h = 16;\n tile_out_w = 64;\n }\n\n if (p.up_x == 2 && p.up_y == 2 && p.down_x == 1 && p.down_y == 1 &&\n p.kernel_h <= 2 && p.kernel_w <= 2) {\n mode = 4;\n tile_out_h = 16;\n tile_out_w = 64;\n }\n\n if (p.up_x == 1 && p.up_y == 1 && p.down_x == 2 && p.down_y == 2 &&\n p.kernel_h <= 4 && p.kernel_w <= 4) {\n mode = 5;\n tile_out_h = 8;\n tile_out_w = 32;\n }\n\n if (p.up_x == 1 && p.up_y == 1 && p.down_x == 2 && p.down_y == 2 &&\n p.kernel_h <= 2 && p.kernel_w <= 2) {\n mode = 6;\n tile_out_h = 8;\n tile_out_w = 32;\n }\n\n dim3 block_size;\n dim3 grid_size;\n\n if (tile_out_h > 0 && tile_out_w > 0) {\n p.loop_major = (p.major_dim - 1) / 16384 + 1;\n p.loop_x = 1;\n block_size = dim3(32 * 8, 1, 1);\n grid_size = dim3(((p.out_h - 1) / tile_out_h + 1) * p.minor_dim,\n (p.out_w - 1) / (p.loop_x * tile_out_w) + 1,\n (p.major_dim - 1) / p.loop_major + 1);\n } else {\n p.loop_major = (p.major_dim - 1) / 16384 + 1;\n p.loop_x = 4;\n block_size = dim3(4, 32, 1);\n grid_size = dim3((p.out_h * p.minor_dim - 1) / block_size.x + 1,\n (p.out_w - 1) / (p.loop_x * block_size.y) + 1,\n (p.major_dim - 1) / p.loop_major + 1);\n }\n\n AT_DISPATCH_FLOATING_TYPES_AND_HALF(x.scalar_type(), \"upfirdn2d_cuda\", [&] {\n switch (mode) {\n case 1:\n upfirdn2d_kernel\n <<>>(out.data_ptr(),\n x.data_ptr(),\n k.data_ptr(), p);\n\n break;\n\n case 2:\n upfirdn2d_kernel\n <<>>(out.data_ptr(),\n x.data_ptr(),\n k.data_ptr(), p);\n\n break;\n\n case 3:\n upfirdn2d_kernel\n <<>>(out.data_ptr(),\n x.data_ptr(),\n k.data_ptr(), p);\n\n break;\n\n case 4:\n upfirdn2d_kernel\n <<>>(out.data_ptr(),\n x.data_ptr(),\n k.data_ptr(), p);\n\n break;\n\n case 5:\n upfirdn2d_kernel\n <<>>(out.data_ptr(),\n x.data_ptr(),\n k.data_ptr(), p);\n\n break;\n\n case 6:\n upfirdn2d_kernel\n <<>>(out.data_ptr(),\n x.data_ptr(),\n k.data_ptr(), p);\n\n break;\n\n default:\n upfirdn2d_kernel_large<<>>(\n out.data_ptr(), x.data_ptr(),\n k.data_ptr(), p);\n }\n });\n\n return out;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/cuda/voxelization_cuda.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#include \n#include \n\n#include \"pytorch_cuda_helper.hpp\"\n#include \"voxelization_cuda_kernel.cuh\"\n\nint HardVoxelizeForwardCUDAKernelLauncher(\n const at::Tensor &points, at::Tensor &voxels, at::Tensor &coors,\n at::Tensor &num_points_per_voxel, const std::vector voxel_size,\n const std::vector coors_range, const int max_points,\n const int max_voxels, const int NDim = 3) {\n // current version tooks about 0.04s for one frame on cpu\n // check device\n\n at::cuda::CUDAGuard device_guard(points.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n const int num_points = points.size(0);\n const int num_features = points.size(1);\n\n const float voxel_x = voxel_size[0];\n const float voxel_y = voxel_size[1];\n const float voxel_z = voxel_size[2];\n const float coors_x_min = coors_range[0];\n const float coors_y_min = coors_range[1];\n const float coors_z_min = coors_range[2];\n const float coors_x_max = coors_range[3];\n const float coors_y_max = coors_range[4];\n const float coors_z_max = coors_range[5];\n\n const int grid_x = round((coors_x_max - coors_x_min) / voxel_x);\n const int grid_y = round((coors_y_max - coors_y_min) / voxel_y);\n const int grid_z = round((coors_z_max - coors_z_min) / voxel_z);\n\n // map points to voxel coors\n at::Tensor temp_coors =\n at::zeros({num_points, NDim}, points.options().dtype(at::kInt));\n\n dim3 grid(std::min(at::cuda::ATenCeilDiv(num_points, 512), 4096));\n dim3 block(512);\n\n // 1. link point to corresponding voxel coors\n AT_DISPATCH_ALL_TYPES(\n points.scalar_type(), \"hard_voxelize_kernel\", ([&] {\n dynamic_voxelize_kernel<<>>(\n points.contiguous().data_ptr(),\n temp_coors.contiguous().data_ptr(), voxel_x, voxel_y, voxel_z,\n coors_x_min, coors_y_min, coors_z_min, coors_x_max, coors_y_max,\n coors_z_max, grid_x, grid_y, grid_z, num_points, num_features,\n NDim);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n // 2. map point to the idx of the corresponding voxel, find duplicate coor\n // create some temporary variables\n auto point_to_pointidx = -at::ones(\n {\n num_points,\n },\n points.options().dtype(at::kInt));\n auto point_to_voxelidx = -at::ones(\n {\n num_points,\n },\n points.options().dtype(at::kInt));\n\n dim3 map_grid(std::min(at::cuda::ATenCeilDiv(num_points, 512), 4096));\n dim3 map_block(512);\n\n AT_DISPATCH_ALL_TYPES(\n temp_coors.scalar_type(), \"determin_duplicate\", ([&] {\n point_to_voxelidx_kernel<<>>(\n temp_coors.contiguous().data_ptr(),\n point_to_voxelidx.contiguous().data_ptr(),\n point_to_pointidx.contiguous().data_ptr(), max_points,\n max_voxels, num_points, NDim);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n // 3. determine voxel num and voxel's coor index\n // make the logic in the CUDA device could accelerate about 10 times\n auto coor_to_voxelidx = -at::ones(\n {\n num_points,\n },\n points.options().dtype(at::kInt));\n auto voxel_num = at::zeros(\n {\n 1,\n },\n points.options().dtype(at::kInt)); // must be zero from the beginning\n\n AT_DISPATCH_ALL_TYPES(temp_coors.scalar_type(), \"determin_duplicate\", ([&] {\n determin_voxel_num<<<1, 1, 0, stream>>>(\n num_points_per_voxel.contiguous().data_ptr(),\n point_to_voxelidx.contiguous().data_ptr(),\n point_to_pointidx.contiguous().data_ptr(),\n coor_to_voxelidx.contiguous().data_ptr(),\n voxel_num.contiguous().data_ptr(),\n max_points, max_voxels, num_points);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n // 4. copy point features to voxels\n // Step 4 & 5 could be parallel\n auto pts_output_size = num_points * num_features;\n dim3 cp_grid(std::min(at::cuda::ATenCeilDiv(pts_output_size, 512), 4096));\n dim3 cp_block(512);\n AT_DISPATCH_ALL_TYPES(\n points.scalar_type(), \"assign_point_to_voxel\", ([&] {\n assign_point_to_voxel<<>>(\n pts_output_size, points.contiguous().data_ptr(),\n point_to_voxelidx.contiguous().data_ptr(),\n coor_to_voxelidx.contiguous().data_ptr(),\n voxels.contiguous().data_ptr(), max_points, num_features,\n num_points, NDim);\n }));\n // cudaDeviceSynchronize();\n // AT_CUDA_CHECK(cudaGetLastError());\n\n // 5. copy coors of each voxels\n auto coors_output_size = num_points * NDim;\n dim3 coors_cp_grid(\n std::min(at::cuda::ATenCeilDiv(coors_output_size, 512), 4096));\n dim3 coors_cp_block(512);\n AT_DISPATCH_ALL_TYPES(\n points.scalar_type(), \"assign_point_to_voxel\", ([&] {\n assign_voxel_coors\n <<>>(\n coors_output_size, temp_coors.contiguous().data_ptr(),\n point_to_voxelidx.contiguous().data_ptr(),\n coor_to_voxelidx.contiguous().data_ptr(),\n coors.contiguous().data_ptr(), num_points, NDim);\n }));\n\n AT_CUDA_CHECK(cudaGetLastError());\n\n auto voxel_num_cpu = voxel_num.to(at::kCPU);\n int voxel_num_int = voxel_num_cpu.data_ptr()[0];\n\n return voxel_num_int;\n}\n\nvoid DynamicVoxelizeForwardCUDAKernelLauncher(\n const at::Tensor &points, at::Tensor &coors,\n const std::vector voxel_size, const std::vector coors_range,\n const int NDim = 3) {\n // current version tooks about 0.04s for one frame on cpu\n // check device\n\n at::cuda::CUDAGuard device_guard(points.device());\n cudaStream_t stream = at::cuda::getCurrentCUDAStream();\n\n const int num_points = points.size(0);\n const int num_features = points.size(1);\n\n const float voxel_x = voxel_size[0];\n const float voxel_y = voxel_size[1];\n const float voxel_z = voxel_size[2];\n const float coors_x_min = coors_range[0];\n const float coors_y_min = coors_range[1];\n const float coors_z_min = coors_range[2];\n const float coors_x_max = coors_range[3];\n const float coors_y_max = coors_range[4];\n const float coors_z_max = coors_range[5];\n\n const int grid_x = round((coors_x_max - coors_x_min) / voxel_x);\n const int grid_y = round((coors_y_max - coors_y_min) / voxel_y);\n const int grid_z = round((coors_z_max - coors_z_min) / voxel_z);\n\n const int col_blocks = at::cuda::ATenCeilDiv(num_points, THREADS_PER_BLOCK);\n dim3 blocks(col_blocks);\n dim3 threads(THREADS_PER_BLOCK);\n\n AT_DISPATCH_ALL_TYPES(points.scalar_type(), \"dynamic_voxelize_kernel\", [&] {\n dynamic_voxelize_kernel<<>>(\n points.contiguous().data_ptr(),\n coors.contiguous().data_ptr(), voxel_x, voxel_y, voxel_z,\n coors_x_min, coors_y_min, coors_z_min, coors_x_max, coors_y_max,\n coors_z_max, grid_x, grid_y, grid_z, num_points, num_features, NDim);\n });\n\n AT_CUDA_CHECK(cudaGetLastError());\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid deformable_im2col_impl(Tensor data_im, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor data_col) {\n DISPATCH_DEVICE_IMPL(deformable_im2col_impl, data_im, data_offset, channels,\n height, width, ksize_h, ksize_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, parallel_imgs,\n deformable_group, data_col);\n}\n\nvoid deformable_col2im_impl(Tensor data_col, Tensor data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n Tensor grad_im) {\n DISPATCH_DEVICE_IMPL(deformable_col2im_impl, data_col, data_offset, channels,\n height, width, ksize_h, ksize_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, parallel_imgs,\n deformable_group, grad_im);\n}\n\nvoid deformable_col2im_coord_impl(\n Tensor data_col, Tensor data_im, Tensor data_offset, const int channels,\n const int height, const int width, const int ksize_h, const int ksize_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int parallel_imgs,\n const int deformable_group, Tensor grad_offset) {\n DISPATCH_DEVICE_IMPL(deformable_col2im_coord_impl, data_col, data_im,\n data_offset, channels, height, width, ksize_h, ksize_w,\n pad_h, pad_w, stride_h, stride_w, dilation_h, dilation_w,\n parallel_imgs, deformable_group, grad_offset);\n}\n\nvoid deform_conv_shape_check(at::Tensor input, at::Tensor offset,\n at::Tensor *gradOutput, at::Tensor weight, int kH,\n int kW, int dH, int dW, int padH, int padW,\n int dilationH, int dilationW, int group,\n int deformable_group) {\n TORCH_CHECK(\n weight.ndimension() == 4,\n \"4D weight tensor (nOutputPlane,nInputPlane,kH,kW) expected, but got: %s\",\n weight.ndimension());\n\n TORCH_CHECK(weight.is_contiguous(), \"weight tensor has to be contiguous\");\n\n TORCH_CHECK(kW > 0 && kH > 0,\n \"kernel size should be greater than zero, but got kH: %d kW: %d\",\n kH, kW);\n\n TORCH_CHECK((weight.size(2) == kH && weight.size(3) == kW),\n \"kernel size should be consistent with weight, \",\n \"but got kH: %d kW: %d weight.size(2): %d, weight.size(3): %d\",\n kH, kW, weight.size(2), weight.size(3));\n\n TORCH_CHECK(dW > 0 && dH > 0,\n \"stride should be greater than zero, but got dH: %d dW: %d\", dH,\n dW);\n\n TORCH_CHECK(\n dilationW > 0 && dilationH > 0,\n \"dilation should be greater than 0, but got dilationH: %d dilationW: %d\",\n dilationH, dilationW);\n\n int ndim = input.ndimension();\n int dimf = 0;\n int dimh = 1;\n int dimw = 2;\n\n if (ndim == 4) {\n dimf++;\n dimh++;\n dimw++;\n }\n\n TORCH_CHECK(ndim == 3 || ndim == 4,\n \"3D or 4D input tensor expected but got: %s\", ndim);\n\n long nInputPlane = weight.size(1) * group;\n long inputHeight = input.size(dimh);\n long inputWidth = input.size(dimw);\n long nOutputPlane = weight.size(0);\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n\n TORCH_CHECK(nInputPlane % deformable_group == 0,\n \"input channels must divide deformable group size\");\n\n if (outputWidth < 1 || outputHeight < 1)\n AT_ERROR(\n \"Given input size: (%ld x %ld x %ld). \"\n \"Calculated output size: (%ld x %ld x %ld). Output size is too small\",\n nInputPlane, inputHeight, inputWidth, nOutputPlane, outputHeight,\n outputWidth);\n\n TORCH_CHECK(input.size(1) == nInputPlane,\n \"invalid number of input planes, expected: %d, but got: %d\",\n nInputPlane, input.size(1));\n\n TORCH_CHECK((inputHeight >= kH && inputWidth >= kW),\n \"input image is smaller than kernel\");\n\n TORCH_CHECK(\n (offset.size(2) == outputHeight && offset.size(3) == outputWidth),\n \"invalid spatial size of offset, expected height: %d width: %d, but \"\n \"got height: %d width: %d\",\n outputHeight, outputWidth, offset.size(2), offset.size(3));\n\n TORCH_CHECK((offset.size(1) == deformable_group * 2 * kH * kW),\n \"invalid number of channels of offset\");\n\n if (gradOutput != NULL) {\n TORCH_CHECK(\n gradOutput->size(dimf) == nOutputPlane,\n \"invalid number of gradOutput planes, expected: %d, but got: %d\",\n nOutputPlane, gradOutput->size(dimf));\n\n TORCH_CHECK(\n (gradOutput->size(dimh) == outputHeight &&\n gradOutput->size(dimw) == outputWidth),\n \"invalid size of gradOutput, expected height: %d width: %d , but \"\n \"got height: %d width: %d\",\n outputHeight, outputWidth, gradOutput->size(dimh),\n gradOutput->size(dimw));\n }\n}\n\nvoid deform_conv_forward(Tensor input, Tensor weight, Tensor offset,\n Tensor output, Tensor columns, Tensor ones, int kW,\n int kH, int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step) {\n if (input.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n CHECK_CUDA_INPUT(input);\n CHECK_CUDA_INPUT(offset);\n CHECK_CUDA_INPUT(weight);\n CHECK_CUDA_INPUT(output);\n CHECK_CUDA_INPUT(columns);\n CHECK_CUDA_INPUT(ones);\n#else\n AT_ERROR(\"DeformConv is not compiled with GPU support\");\n#endif\n } else {\n CHECK_CPU_INPUT(input);\n CHECK_CPU_INPUT(offset);\n CHECK_CPU_INPUT(weight);\n CHECK_CPU_INPUT(output);\n CHECK_CPU_INPUT(columns);\n CHECK_CPU_INPUT(ones);\n }\n\n deform_conv_shape_check(input, offset, NULL, weight, kH, kW, dH, dW, padH,\n padW, dilationH, dilationW, group, deformable_group);\n at::DeviceGuard guard(input.device());\n\n int batch = 1;\n if (input.ndimension() == 3) {\n // Force batch\n batch = 0;\n input.unsqueeze_(0);\n offset.unsqueeze_(0);\n }\n\n // todo: assert batchsize dividable by im2col_step\n\n long batchSize = input.size(0);\n long nInputPlane = input.size(1);\n long inputHeight = input.size(2);\n long inputWidth = input.size(3);\n\n long nOutputPlane = weight.size(0);\n\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n\n TORCH_CHECK((offset.size(0) == batchSize), \"invalid batch size of offset\");\n\n output = output.view({batchSize / im2col_step, im2col_step, nOutputPlane,\n outputHeight, outputWidth});\n columns = at::zeros(\n {nInputPlane * kW * kH, im2col_step * outputHeight * outputWidth},\n input.options());\n\n if (ones.ndimension() != 2 ||\n ones.size(0) * ones.size(1) < outputHeight * outputWidth) {\n ones = at::ones({outputHeight, outputWidth}, input.options());\n }\n\n input = input.view({batchSize / im2col_step, im2col_step, nInputPlane,\n inputHeight, inputWidth});\n offset =\n offset.view({batchSize / im2col_step, im2col_step,\n deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n Tensor output_buffer = at::zeros({batchSize / im2col_step, nOutputPlane,\n im2col_step * outputHeight, outputWidth},\n output.options());\n\n output_buffer = output_buffer.view(\n {output_buffer.size(0), group, output_buffer.size(1) / group,\n output_buffer.size(2), output_buffer.size(3)});\n\n for (int elt = 0; elt < batchSize / im2col_step; elt++) {\n deformable_im2col_impl(input[elt], offset[elt], nInputPlane, inputHeight,\n inputWidth, kH, kW, padH, padW, dH, dW, dilationH,\n dilationW, im2col_step, deformable_group, columns);\n\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n weight = weight.view({group, weight.size(0) / group, weight.size(1),\n weight.size(2), weight.size(3)});\n\n for (int g = 0; g < group; g++) {\n output_buffer[elt][g] = output_buffer[elt][g]\n .flatten(1)\n .addmm_(weight[g].flatten(1), columns[g])\n .view_as(output_buffer[elt][g]);\n }\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n weight = weight.view({weight.size(0) * weight.size(1), weight.size(2),\n weight.size(3), weight.size(4)});\n }\n\n output_buffer = output_buffer.view(\n {output_buffer.size(0), output_buffer.size(1) * output_buffer.size(2),\n output_buffer.size(3), output_buffer.size(4)});\n\n output_buffer = output_buffer.view({batchSize / im2col_step, nOutputPlane,\n im2col_step, outputHeight, outputWidth});\n output_buffer.transpose_(1, 2);\n output.copy_(output_buffer);\n output = output.view({batchSize, nOutputPlane, outputHeight, outputWidth});\n\n input = input.view({batchSize, nInputPlane, inputHeight, inputWidth});\n offset = offset.view(\n {batchSize, deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n if (batch == 0) {\n output = output.view({nOutputPlane, outputHeight, outputWidth});\n input = input.view({nInputPlane, inputHeight, inputWidth});\n offset = offset.view({offset.size(1), offset.size(2), offset.size(3)});\n }\n}\n\nvoid deform_conv_backward_input(Tensor input, Tensor offset, Tensor gradOutput,\n Tensor gradInput, Tensor gradOffset,\n Tensor weight, Tensor columns, int kW, int kH,\n int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step) {\n if (input.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n CHECK_CUDA_INPUT(input);\n CHECK_CUDA_INPUT(offset);\n CHECK_CUDA_INPUT(gradOutput);\n CHECK_CUDA_INPUT(gradInput);\n CHECK_CUDA_INPUT(gradOffset);\n CHECK_CUDA_INPUT(weight);\n CHECK_CUDA_INPUT(columns);\n#else\n AT_ERROR(\"DeformConv is not compiled with GPU support\");\n#endif\n } else {\n CHECK_CPU_INPUT(input);\n CHECK_CPU_INPUT(offset);\n CHECK_CPU_INPUT(gradOutput);\n CHECK_CPU_INPUT(gradInput);\n CHECK_CPU_INPUT(gradOffset);\n CHECK_CPU_INPUT(weight);\n CHECK_CPU_INPUT(columns);\n }\n deform_conv_shape_check(input, offset, &gradOutput, weight, kH, kW, dH, dW,\n padH, padW, dilationH, dilationW, group,\n deformable_group);\n\n at::DeviceGuard guard(input.device());\n\n int batch = 1;\n if (input.ndimension() == 3) {\n // Force batch\n batch = 0;\n input = input.view({1, input.size(0), input.size(1), input.size(2)});\n offset = offset.view({1, offset.size(0), offset.size(1), offset.size(2)});\n gradOutput = gradOutput.view(\n {1, gradOutput.size(0), gradOutput.size(1), gradOutput.size(2)});\n }\n\n long batchSize = input.size(0);\n long nInputPlane = input.size(1);\n long inputHeight = input.size(2);\n long inputWidth = input.size(3);\n\n long nOutputPlane = weight.size(0);\n\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n\n TORCH_CHECK((offset.size(0) == batchSize), 3, \"invalid batch size of offset\");\n gradInput = gradInput.view({batchSize, nInputPlane, inputHeight, inputWidth});\n columns = at::zeros(\n {nInputPlane * kW * kH, im2col_step * outputHeight * outputWidth},\n input.options());\n\n // change order of grad output\n gradOutput = gradOutput.view({batchSize / im2col_step, im2col_step,\n nOutputPlane, outputHeight, outputWidth});\n gradOutput.transpose_(1, 2);\n\n gradInput = gradInput.view({batchSize / im2col_step, im2col_step, nInputPlane,\n inputHeight, inputWidth});\n input = input.view({batchSize / im2col_step, im2col_step, nInputPlane,\n inputHeight, inputWidth});\n gradOffset = gradOffset.view({batchSize / im2col_step, im2col_step,\n deformable_group * 2 * kH * kW, outputHeight,\n outputWidth});\n offset =\n offset.view({batchSize / im2col_step, im2col_step,\n deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n for (int elt = 0; elt < batchSize / im2col_step; elt++) {\n // divide into groups\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n weight = weight.view({group, weight.size(0) / group, weight.size(1),\n weight.size(2), weight.size(3)});\n gradOutput = gradOutput.view(\n {gradOutput.size(0), group, gradOutput.size(1) / group,\n gradOutput.size(2), gradOutput.size(3), gradOutput.size(4)});\n\n for (int g = 0; g < group; g++) {\n columns[g] = columns[g].addmm_(weight[g].flatten(1).transpose(0, 1),\n gradOutput[elt][g].flatten(1), 0.0f, 1.0f);\n }\n\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n gradOutput = gradOutput.view(\n {gradOutput.size(0), gradOutput.size(1) * gradOutput.size(2),\n gradOutput.size(3), gradOutput.size(4), gradOutput.size(5)});\n\n deformable_col2im_coord_impl(columns, input[elt], offset[elt], nInputPlane,\n inputHeight, inputWidth, kH, kW, padH, padW,\n dH, dW, dilationH, dilationW, im2col_step,\n deformable_group, gradOffset[elt]);\n\n deformable_col2im_impl(columns, offset[elt], nInputPlane, inputHeight,\n inputWidth, kH, kW, padH, padW, dH, dW, dilationH,\n dilationW, im2col_step, deformable_group,\n gradInput[elt]);\n\n weight = weight.view({weight.size(0) * weight.size(1), weight.size(2),\n weight.size(3), weight.size(4)});\n }\n\n gradOutput.transpose_(1, 2);\n gradOutput =\n gradOutput.view({batchSize, nOutputPlane, outputHeight, outputWidth});\n\n gradInput = gradInput.view({batchSize, nInputPlane, inputHeight, inputWidth});\n input = input.view({batchSize, nInputPlane, inputHeight, inputWidth});\n gradOffset = gradOffset.view(\n {batchSize, deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n offset = offset.view(\n {batchSize, deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n if (batch == 0) {\n gradOutput = gradOutput.view({nOutputPlane, outputHeight, outputWidth});\n input = input.view({nInputPlane, inputHeight, inputWidth});\n gradInput = gradInput.view({nInputPlane, inputHeight, inputWidth});\n offset = offset.view({offset.size(1), offset.size(2), offset.size(3)});\n gradOffset =\n gradOffset.view({offset.size(1), offset.size(2), offset.size(3)});\n }\n}\n\nvoid deform_conv_backward_parameters(Tensor input, Tensor offset,\n Tensor gradOutput, Tensor gradWeight,\n Tensor columns, Tensor ones, int kW,\n int kH, int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, float scale,\n int im2col_step) {\n if (input.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n CHECK_CUDA_INPUT(input);\n CHECK_CUDA_INPUT(offset);\n CHECK_CUDA_INPUT(gradOutput);\n CHECK_CUDA_INPUT(gradWeight);\n CHECK_CUDA_INPUT(columns);\n CHECK_CUDA_INPUT(ones);\n#else\n AT_ERROR(\"DeformConv is not compiled with GPU support\");\n#endif\n } else {\n CHECK_CPU_INPUT(input);\n CHECK_CPU_INPUT(offset);\n CHECK_CPU_INPUT(gradOutput);\n CHECK_CPU_INPUT(gradWeight);\n CHECK_CPU_INPUT(columns);\n CHECK_CPU_INPUT(ones);\n }\n\n deform_conv_shape_check(input, offset, &gradOutput, gradWeight, kH, kW, dH,\n dW, padH, padW, dilationH, dilationW, group,\n deformable_group);\n at::DeviceGuard guard(input.device());\n\n int batch = 1;\n\n if (input.ndimension() == 3) {\n // Force batch\n batch = 0;\n input = input.view(\n at::IntList({1, input.size(0), input.size(1), input.size(2)}));\n gradOutput = gradOutput.view(\n {1, gradOutput.size(0), gradOutput.size(1), gradOutput.size(2)});\n }\n\n long batchSize = input.size(0);\n long nInputPlane = input.size(1);\n long inputHeight = input.size(2);\n long inputWidth = input.size(3);\n\n long nOutputPlane = gradWeight.size(0);\n\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n\n TORCH_CHECK((offset.size(0) == batchSize), \"invalid batch size of offset\");\n\n columns = at::zeros(\n {nInputPlane * kW * kH, im2col_step * outputHeight * outputWidth},\n input.options());\n\n gradOutput = gradOutput.view({batchSize / im2col_step, im2col_step,\n nOutputPlane, outputHeight, outputWidth});\n gradOutput.transpose_(1, 2);\n\n Tensor gradOutputBuffer = at::zeros_like(gradOutput);\n gradOutputBuffer =\n gradOutputBuffer.view({batchSize / im2col_step, nOutputPlane, im2col_step,\n outputHeight, outputWidth});\n gradOutputBuffer = gradOutputBuffer.contiguous();\n gradOutputBuffer.copy_(gradOutput);\n gradOutputBuffer =\n gradOutputBuffer.view({batchSize / im2col_step, nOutputPlane,\n im2col_step * outputHeight, outputWidth});\n\n gradOutput.transpose_(1, 2);\n gradOutput =\n gradOutput.view({batchSize, nOutputPlane, outputHeight, outputWidth});\n\n input = input.view({batchSize / im2col_step, im2col_step, nInputPlane,\n inputHeight, inputWidth});\n offset =\n offset.view({batchSize / im2col_step, im2col_step,\n deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n for (int elt = 0; elt < batchSize / im2col_step; elt++) {\n deformable_im2col_impl(input[elt], offset[elt], nInputPlane, inputHeight,\n inputWidth, kH, kW, padH, padW, dH, dW, dilationH,\n dilationW, im2col_step, deformable_group, columns);\n\n // divide into group\n gradOutputBuffer = gradOutputBuffer.view(\n {gradOutputBuffer.size(0), group, gradOutputBuffer.size(1) / group,\n gradOutputBuffer.size(2), gradOutputBuffer.size(3)});\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n gradWeight =\n gradWeight.view({group, gradWeight.size(0) / group, gradWeight.size(1),\n gradWeight.size(2), gradWeight.size(3)});\n\n for (int g = 0; g < group; g++) {\n gradWeight[g] = gradWeight[g]\n .flatten(1)\n .addmm_(gradOutputBuffer[elt][g].flatten(1),\n columns[g].transpose(1, 0), 1.0, scale)\n .view_as(gradWeight[g]);\n }\n gradOutputBuffer = gradOutputBuffer.view(\n {gradOutputBuffer.size(0),\n gradOutputBuffer.size(1) * gradOutputBuffer.size(2),\n gradOutputBuffer.size(3), gradOutputBuffer.size(4)});\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n gradWeight = gradWeight.view({gradWeight.size(0) * gradWeight.size(1),\n gradWeight.size(2), gradWeight.size(3),\n gradWeight.size(4)});\n }\n\n input = input.view({batchSize, nInputPlane, inputHeight, inputWidth});\n offset = offset.view(\n {batchSize, deformable_group * 2 * kH * kW, outputHeight, outputWidth});\n\n if (batch == 0) {\n gradOutput = gradOutput.view({nOutputPlane, outputHeight, outputWidth});\n input = input.view({nInputPlane, inputHeight, inputWidth});\n }\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/deform_roi_pool.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid deform_roi_pool_forward_impl(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma) {\n DISPATCH_DEVICE_IMPL(deform_roi_pool_forward_impl, input, rois, offset,\n output, pooled_height, pooled_width, spatial_scale,\n sampling_ratio, gamma);\n}\n\nvoid deform_roi_pool_backward_impl(Tensor grad_output, Tensor input,\n Tensor rois, Tensor offset,\n Tensor grad_input, Tensor grad_offset,\n int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma) {\n DISPATCH_DEVICE_IMPL(deform_roi_pool_backward_impl, grad_output, input, rois,\n offset, grad_input, grad_offset, pooled_height,\n pooled_width, spatial_scale, sampling_ratio, gamma);\n}\n\nvoid deform_roi_pool_forward(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma) {\n deform_roi_pool_forward_impl(input, rois, offset, output, pooled_height,\n pooled_width, spatial_scale, sampling_ratio,\n gamma);\n}\n\nvoid deform_roi_pool_backward(Tensor grad_output, Tensor input, Tensor rois,\n Tensor offset, Tensor grad_input,\n Tensor grad_offset, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma) {\n deform_roi_pool_backward_impl(grad_output, input, rois, offset, grad_input,\n grad_offset, pooled_height, pooled_width,\n spatial_scale, sampling_ratio, gamma);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/focal_loss.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid sigmoid_focal_loss_forward_impl(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n DISPATCH_DEVICE_IMPL(sigmoid_focal_loss_forward_impl, input, target, weight,\n output, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_backward_impl(Tensor input, Tensor target,\n Tensor weight, Tensor grad_input,\n float gamma, float alpha) {\n DISPATCH_DEVICE_IMPL(sigmoid_focal_loss_backward_impl, input, target, weight,\n grad_input, gamma, alpha);\n}\n\nvoid softmax_focal_loss_forward_impl(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n DISPATCH_DEVICE_IMPL(softmax_focal_loss_forward_impl, input, target, weight,\n output, gamma, alpha);\n}\n\nvoid softmax_focal_loss_backward_impl(Tensor input, Tensor target,\n Tensor weight, Tensor buff,\n Tensor grad_input, float gamma,\n float alpha) {\n DISPATCH_DEVICE_IMPL(softmax_focal_loss_backward_impl, input, target, weight,\n buff, grad_input, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_forward(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n sigmoid_focal_loss_forward_impl(input, target, weight, output, gamma, alpha);\n}\n\nvoid sigmoid_focal_loss_backward(Tensor input, Tensor target, Tensor weight,\n Tensor grad_input, float gamma, float alpha) {\n sigmoid_focal_loss_backward_impl(input, target, weight, grad_input, gamma,\n alpha);\n}\n\nvoid softmax_focal_loss_forward(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha) {\n softmax_focal_loss_forward_impl(input, target, weight, output, gamma, alpha);\n}\n\nvoid softmax_focal_loss_backward(Tensor input, Tensor target, Tensor weight,\n Tensor buff, Tensor grad_input, float gamma,\n float alpha) {\n softmax_focal_loss_backward_impl(input, target, weight, buff, grad_input,\n gamma, alpha);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/furthest_point_sample.cpp", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/sampling.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid furthest_point_sampling_forward_impl(Tensor points_tensor,\n Tensor temp_tensor, Tensor idx_tensor,\n int b, int n, int m) {\n DISPATCH_DEVICE_IMPL(furthest_point_sampling_forward_impl, points_tensor,\n temp_tensor, idx_tensor, b, n, m);\n}\n\nvoid furthest_point_sampling_with_dist_forward_impl(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b,\n int n, int m) {\n DISPATCH_DEVICE_IMPL(furthest_point_sampling_with_dist_forward_impl,\n points_tensor, temp_tensor, idx_tensor, b, n, m);\n}\n\nvoid furthest_point_sampling_forward(Tensor points_tensor, Tensor temp_tensor,\n Tensor idx_tensor, int b, int n, int m) {\n furthest_point_sampling_forward_impl(points_tensor, temp_tensor, idx_tensor,\n b, n, m);\n}\n\nvoid furthest_point_sampling_with_dist_forward(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b, int n,\n int m) {\n furthest_point_sampling_with_dist_forward_impl(points_tensor, temp_tensor,\n idx_tensor, b, n, m);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/fused_bias_leakyrelu.cpp", "content": "// Modified from\n// https://github.com/rosinality/stylegan2-pytorch/blob/master/op/fused_bias_act.cpp\n\n/*\nCopyright (c) 2021, NVIDIA Corporation. All rights reserved.\n\nNVIDIA Source Code License for StyleGAN2 with Adaptive Discriminator\nAugmentation (ADA)\n=======================================================================\n\n1. Definitions\n\n\"Licensor\" means any person or entity that distributes its Work.\n\n\"Software\" means the original work of authorship made available under\nthis License.\n\n\"Work\" means the Software and any additions to or derivative works of\nthe Software that are made available under this License.\n\nThe terms \"reproduce,\" \"reproduction,\" \"derivative works,\" and\n\"distribution\" have the meaning as provided under U.S. copyright law;\nprovided, however, that for the purposes of this License, derivative\nworks shall not include works that remain separable from, or merely\nlink (or bind by name) to the interfaces of, the Work.\n\nWorks, including the Software, are \"made available\" under this License\nby including in or with the Work either (a) a copyright notice\nreferencing the applicability of this License to the Work, or (b) a\ncopy of this License.\n\n2. License Grants\n\n 2.1 Copyright Grant. Subject to the terms and conditions of this\n License, each Licensor grants to you a perpetual, worldwide,\n non-exclusive, royalty-free, copyright license to reproduce,\n prepare derivative works of, publicly display, publicly perform,\n sublicense and distribute its Work and any resulting derivative\n works in any form.\n\n3. Limitations\n\n 3.1 Redistribution. You may reproduce or distribute the Work only\n if (a) you do so under this License, (b) you include a complete\n copy of this License with your distribution, and (c) you retain\n without modification any copyright, patent, trademark, or\n attribution notices that are present in the Work.\n\n 3.2 Derivative Works. You may specify that additional or different\n terms apply to the use, reproduction, and distribution of your\n derivative works of the Work (\"Your Terms\") only if (a) Your Terms\n provide that the use limitation in Section 3.3 applies to your\n derivative works, and (b) you identify the specific derivative\n works that are subject to Your Terms. Notwithstanding Your Terms,\n this License (including the redistribution requirements in Section\n 3.1) will continue to apply to the Work itself.\n\n 3.3 Use Limitation. The Work and any derivative works thereof only\n may be used or intended for use non-commercially. Notwithstanding\n the foregoing, NVIDIA and its affiliates may use the Work and any\n derivative works commercially. As used herein, \"non-commercially\"\n means for research or evaluation purposes only.\n\n 3.4 Patent Claims. If you bring or threaten to bring a patent claim\n against any Licensor (including any claim, cross-claim or\n counterclaim in a lawsuit) to enforce any patents that you allege\n are infringed by any Work, then your rights under this License from\n such Licensor (including the grant in Section 2.1) will terminate\n immediately.\n\n 3.5 Trademarks. This License does not grant any rights to use any\n Licensor’s or its affiliates’ names, logos, or trademarks, except\n as necessary to reproduce the notices described in this License.\n\n 3.6 Termination. If you violate any term of this License, then your\n rights under this License (including the grant in Section 2.1) will\n terminate immediately.\n\n4. Disclaimer of Warranty.\n\nTHE WORK IS PROVIDED \"AS IS\" WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR\nNON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER\nTHIS LICENSE.\n\n5. Limitation of Liability.\n\nEXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL\nTHEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE\nSHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT,\nINDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF\nOR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK\n(INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION,\nLOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER\nCOMMERCIAL DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF\nTHE POSSIBILITY OF SUCH DAMAGES.\n\n=======================================================================\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntorch::Tensor fused_bias_leakyrelu_op_impl(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale) {\n return DISPATCH_DEVICE_IMPL(fused_bias_leakyrelu_op_impl, input, bias, refer,\n act, grad, alpha, scale);\n}\n\ntorch::Tensor fused_bias_leakyrelu(const torch::Tensor& input,\n const torch::Tensor& bias,\n const torch::Tensor& refer, int act,\n int grad, float alpha, float scale) {\n return fused_bias_leakyrelu_op_impl(input, bias, refer, act, grad, alpha,\n scale);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/gather_points.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid gather_points_forward_impl(int b, int c, int n, int npoints,\n const Tensor points, const Tensor idx,\n Tensor out) {\n DISPATCH_DEVICE_IMPL(gather_points_forward_impl, b, c, n, npoints, points,\n idx, out);\n}\n\nvoid gather_points_backward_impl(int b, int c, int n, int npoints,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points) {\n DISPATCH_DEVICE_IMPL(gather_points_backward_impl, b, c, n, npoints, grad_out,\n idx, grad_points);\n}\n\nvoid gather_points_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor out_tensor, int b, int c, int n,\n int npoints) {\n gather_points_forward_impl(b, c, n, npoints, points_tensor, idx_tensor,\n out_tensor);\n}\n\nvoid gather_points_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor grad_points_tensor, int b, int c, int n,\n int npoints) {\n gather_points_backward_impl(b, c, n, npoints, grad_out_tensor, idx_tensor,\n grad_points_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/group_points.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/group_points.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid group_points_forward_impl(int b, int c, int n, int npoints, int nsample,\n const Tensor points, const Tensor idx,\n Tensor out) {\n DISPATCH_DEVICE_IMPL(group_points_forward_impl, b, c, n, npoints, nsample,\n points, idx, out);\n}\n\nvoid group_points_backward_impl(int b, int c, int n, int npoints, int nsample,\n const Tensor grad_out, const Tensor idx,\n Tensor grad_points) {\n DISPATCH_DEVICE_IMPL(group_points_backward_impl, b, c, n, npoints, nsample,\n grad_out, idx, grad_points);\n}\n\nvoid group_points_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor out_tensor, int b, int c, int n, int npoints,\n int nsample) {\n DISPATCH_DEVICE_IMPL(group_points_forward_impl, b, c, n, npoints, nsample,\n points_tensor, idx_tensor, out_tensor);\n}\n\nvoid group_points_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor grad_points_tensor, int b, int c, int n,\n int npoints, int nsample) {\n group_points_backward_impl(b, c, n, npoints, nsample, grad_out_tensor,\n idx_tensor, grad_points_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/info.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/vision.cpp\n#include \"pytorch_cpp_helper.hpp\"\n\n#ifdef MMCV_WITH_CUDA\n#ifndef HIP_DIFF\n#include \nint get_cudart_version() { return CUDART_VERSION; }\n#endif\n#endif\n\nstd::string get_compiling_cuda_version() {\n#ifdef MMCV_WITH_CUDA\n#ifndef HIP_DIFF\n std::ostringstream oss;\n // copied from\n // https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/cuda/detail/CUDAHooks.cpp#L231\n auto printCudaStyleVersion = [&](int v) {\n oss << (v / 1000) << \".\" << (v / 10 % 100);\n if (v % 10 != 0) {\n oss << \".\" << (v % 10);\n }\n };\n printCudaStyleVersion(get_cudart_version());\n return oss.str();\n#else\n return std::string(\"rocm not available\");\n#endif\n#else\n return std::string(\"not available\");\n#endif\n}\n\n// similar to\n// https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/Version.cpp\nstd::string get_compiler_version() {\n std::ostringstream ss;\n#if defined(__GNUC__)\n#ifndef __clang__\n { ss << \"GCC \" << __GNUC__ << \".\" << __GNUC_MINOR__; }\n#endif\n#endif\n\n#if defined(__clang_major__)\n {\n ss << \"clang \" << __clang_major__ << \".\" << __clang_minor__ << \".\"\n << __clang_patchlevel__;\n }\n#endif\n\n#if defined(_MSC_VER)\n { ss << \"MSVC \" << _MSC_FULL_VER; }\n#endif\n return ss.str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/iou3d.cpp", "content": "// Modified from\n// https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/ops/iou3d_nms/src/iou3d_nms.cpp\n\n/*\n3D IoU Calculation and Rotated NMS(modified from 2D NMS written by others)\nWritten by Shaoshuai Shi\nAll Rights Reserved 2019-2020.\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nconst int THREADS_PER_BLOCK_NMS = sizeof(unsigned long long) * 8;\n\nvoid iou3d_boxes_overlap_bev_forward_impl(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_overlap) {\n DISPATCH_DEVICE_IMPL(iou3d_boxes_overlap_bev_forward_impl, num_a, boxes_a,\n num_b, boxes_b, ans_overlap);\n}\n\nvoid iou3d_boxes_iou_bev_forward_impl(const int num_a, const Tensor boxes_a,\n const int num_b, const Tensor boxes_b,\n Tensor ans_iou) {\n DISPATCH_DEVICE_IMPL(iou3d_boxes_iou_bev_forward_impl, num_a, boxes_a, num_b,\n boxes_b, ans_iou);\n}\n\nvoid iou3d_nms_forward_impl(const Tensor boxes, unsigned long long *mask,\n int boxes_num, float nms_overlap_thresh) {\n DISPATCH_DEVICE_IMPL(iou3d_nms_forward_impl, boxes, mask, boxes_num,\n nms_overlap_thresh);\n}\n\nvoid iou3d_nms_normal_forward_impl(const Tensor boxes, unsigned long long *mask,\n int boxes_num, float nms_overlap_thresh) {\n DISPATCH_DEVICE_IMPL(iou3d_nms_normal_forward_impl, boxes, mask, boxes_num,\n nms_overlap_thresh);\n}\n\nvoid iou3d_boxes_overlap_bev_forward(Tensor boxes_a, Tensor boxes_b,\n Tensor ans_overlap) {\n // params boxes_a: (N, 5) [x1, y1, x2, y2, ry]\n // params boxes_b: (M, 5)\n // params ans_overlap: (N, M)\n\n int num_a = boxes_a.size(0);\n int num_b = boxes_b.size(0);\n\n iou3d_boxes_overlap_bev_forward_impl(num_a, boxes_a, num_b, boxes_b,\n ans_overlap);\n}\n\nvoid iou3d_boxes_iou_bev_forward(Tensor boxes_a, Tensor boxes_b,\n Tensor ans_iou) {\n // params boxes_a: (N, 5) [x1, y1, x2, y2, ry]\n // params boxes_b: (M, 5)\n // params ans_overlap: (N, M)\n int num_a = boxes_a.size(0);\n int num_b = boxes_b.size(0);\n\n iou3d_boxes_iou_bev_forward_impl(num_a, boxes_a, num_b, boxes_b, ans_iou);\n}\n\nvoid iou3d_nms_forward(Tensor boxes, Tensor keep, Tensor keep_num,\n float nms_overlap_thresh) {\n // params boxes: (N, 5) [x1, y1, x2, y2, ry]\n // params keep: (N)\n CHECK_CONTIGUOUS(boxes);\n CHECK_CONTIGUOUS(keep);\n\n int boxes_num = boxes.size(0);\n int64_t *keep_data = keep.data_ptr();\n int64_t *keep_num_data = keep_num.data_ptr();\n\n const int col_blocks =\n (boxes_num + THREADS_PER_BLOCK_NMS - 1) / THREADS_PER_BLOCK_NMS;\n\n Tensor mask =\n at::empty({boxes_num, col_blocks}, boxes.options().dtype(at::kLong));\n unsigned long long *mask_data =\n (unsigned long long *)mask.data_ptr();\n iou3d_nms_forward_impl(boxes, mask_data, boxes_num, nms_overlap_thresh);\n\n at::Tensor mask_cpu = mask.to(at::kCPU);\n unsigned long long *mask_host =\n (unsigned long long *)mask_cpu.data_ptr();\n\n std::vector remv_cpu(col_blocks);\n memset(&remv_cpu[0], 0, sizeof(unsigned long long) * col_blocks);\n\n int num_to_keep = 0;\n\n for (int i = 0; i < boxes_num; i++) {\n int nblock = i / THREADS_PER_BLOCK_NMS;\n int inblock = i % THREADS_PER_BLOCK_NMS;\n\n if (!(remv_cpu[nblock] & (1ULL << inblock))) {\n keep_data[num_to_keep++] = i;\n unsigned long long *p = &mask_host[0] + i * col_blocks;\n for (int j = nblock; j < col_blocks; j++) {\n remv_cpu[j] |= p[j];\n }\n }\n *keep_num_data = num_to_keep;\n }\n}\n\nvoid iou3d_nms_normal_forward(Tensor boxes, Tensor keep, Tensor keep_num,\n float nms_overlap_thresh) {\n // params boxes: (N, 5) [x1, y1, x2, y2, ry]\n // params keep: (N)\n\n CHECK_CONTIGUOUS(boxes);\n CHECK_CONTIGUOUS(keep);\n\n int boxes_num = boxes.size(0);\n int64_t *keep_data = keep.data_ptr();\n int64_t *keep_num_data = keep_num.data_ptr();\n\n const int col_blocks =\n (boxes_num + THREADS_PER_BLOCK_NMS - 1) / THREADS_PER_BLOCK_NMS;\n\n Tensor mask =\n at::empty({boxes_num, col_blocks}, boxes.options().dtype(at::kLong));\n unsigned long long *mask_data =\n (unsigned long long *)mask.data_ptr();\n iou3d_nms_normal_forward_impl(boxes, mask_data, boxes_num,\n nms_overlap_thresh);\n\n at::Tensor mask_cpu = mask.to(at::kCPU);\n unsigned long long *mask_host =\n (unsigned long long *)mask_cpu.data_ptr();\n\n std::vector remv_cpu(col_blocks);\n memset(&remv_cpu[0], 0, sizeof(unsigned long long) * col_blocks);\n int num_to_keep = 0;\n\n for (int i = 0; i < boxes_num; i++) {\n int nblock = i / THREADS_PER_BLOCK_NMS;\n int inblock = i % THREADS_PER_BLOCK_NMS;\n\n if (!(remv_cpu[nblock] & (1ULL << inblock))) {\n keep_data[num_to_keep++] = i;\n unsigned long long *p = &mask_host[0] + i * col_blocks;\n for (int j = nblock; j < col_blocks; j++) {\n remv_cpu[j] |= p[j];\n }\n }\n }\n\n *keep_num_data = num_to_keep;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/knn.cpp", "content": "// Modified from\n// https://github.com/CVMI-Lab/PAConv/tree/main/scene_seg/lib/pointops/src/knnquery_heap\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid knn_forward_impl(int b, int n, int m, int nsample, const Tensor xyz,\n const Tensor new_xyz, Tensor idx, Tensor dist2) {\n DISPATCH_DEVICE_IMPL(knn_forward_impl, b, n, m, nsample, xyz, new_xyz, idx,\n dist2);\n}\n\nvoid knn_forward(Tensor xyz_tensor, Tensor new_xyz_tensor, Tensor idx_tensor,\n Tensor dist2_tensor, int b, int n, int m, int nsample) {\n knn_forward_impl(b, n, m, nsample, xyz_tensor, new_xyz_tensor, idx_tensor,\n dist2_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/masked_conv2d.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid masked_im2col_forward_impl(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w) {\n DISPATCH_DEVICE_IMPL(masked_im2col_forward_impl, im, mask_h_idx, mask_w_idx,\n col, kernel_h, kernel_w, pad_h, pad_w);\n}\n\nvoid masked_col2im_forward_impl(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels) {\n DISPATCH_DEVICE_IMPL(masked_col2im_forward_impl, col, mask_h_idx, mask_w_idx,\n im, height, width, channels);\n}\n\nvoid masked_im2col_forward(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w) {\n masked_im2col_forward_impl(im, mask_h_idx, mask_w_idx, col, kernel_h,\n kernel_w, pad_h, pad_w);\n}\n\nvoid masked_col2im_forward(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels) {\n masked_col2im_forward_impl(col, mask_h_idx, mask_w_idx, im, height, width,\n channels);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/min_area_polygons.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid min_area_polygons_impl(const Tensor pointsets, Tensor polygons) {\n DISPATCH_DEVICE_IMPL(min_area_polygons_impl, pointsets, polygons);\n}\n\nvoid min_area_polygons(const Tensor pointsets, Tensor polygons) {\n min_area_polygons_impl(pointsets, polygons);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/modulated_deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid modulated_deformable_im2col_impl(\n const Tensor data_im, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor data_col) {\n DISPATCH_DEVICE_IMPL(modulated_deformable_im2col_impl, data_im, data_offset,\n data_mask, batch_size, channels, height_im, width_im,\n height_col, width_col, kernel_h, kernel_w, pad_h, pad_w,\n stride_h, stride_w, dilation_h, dilation_w,\n deformable_group, data_col);\n}\n\nvoid modulated_deformable_col2im_impl(\n const Tensor data_col, const Tensor data_offset, const Tensor data_mask,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, Tensor grad_im) {\n DISPATCH_DEVICE_IMPL(modulated_deformable_col2im_impl, data_col, data_offset,\n data_mask, batch_size, channels, height_im, width_im,\n height_col, width_col, kernel_h, kernel_w, pad_h, pad_w,\n stride_h, stride_w, dilation_h, dilation_w,\n deformable_group, grad_im);\n}\n\nvoid modulated_deformable_col2im_coord_impl(\n const Tensor data_col, const Tensor data_im, const Tensor data_offset,\n const Tensor data_mask, const int batch_size, const int channels,\n const int height_im, const int width_im, const int height_col,\n const int width_col, const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w, const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w, const int deformable_group,\n Tensor grad_offset, Tensor grad_mask) {\n DISPATCH_DEVICE_IMPL(modulated_deformable_col2im_coord_impl, data_col,\n data_im, data_offset, data_mask, batch_size, channels,\n height_im, width_im, height_col, width_col, kernel_h,\n kernel_w, pad_h, pad_w, stride_h, stride_w, dilation_h,\n dilation_w, deformable_group, grad_offset, grad_mask);\n}\n\nvoid modulated_deform_conv_forward(\n Tensor input, Tensor weight, Tensor bias, Tensor ones, Tensor offset,\n Tensor mask, Tensor output, Tensor columns, int kernel_h, int kernel_w,\n const int stride_h, const int stride_w, const int pad_h, const int pad_w,\n const int dilation_h, const int dilation_w, const int group,\n const int deformable_group, const bool with_bias) {\n at::DeviceGuard guard(input.device());\n\n const int batch = input.size(0);\n const int channels = input.size(1);\n const int height = input.size(2);\n const int width = input.size(3);\n\n const int channels_out = weight.size(0);\n const int channels_kernel = weight.size(1);\n const int kernel_h_ = weight.size(2);\n const int kernel_w_ = weight.size(3);\n\n if (kernel_h_ != kernel_h || kernel_w_ != kernel_w)\n AT_ERROR(\"Input shape and kernel shape won't match: (%d x %d vs %d x %d).\",\n kernel_h_, kernel_w, kernel_h_, kernel_w_);\n if (channels != channels_kernel * group)\n AT_ERROR(\"Input shape and kernel channels won't match: (%d vs %d).\",\n channels, channels_kernel * group);\n\n const int height_out =\n (height + 2 * pad_h - (dilation_h * (kernel_h - 1) + 1)) / stride_h + 1;\n const int width_out =\n (width + 2 * pad_w - (dilation_w * (kernel_w - 1) + 1)) / stride_w + 1;\n\n if (ones.ndimension() != 2 ||\n ones.size(0) * ones.size(1) < height_out * width_out) {\n // Resize plane and fill with ones...\n ones = at::ones({height_out, width_out}, input.options());\n }\n\n // resize output\n output = output.view({batch, channels_out, height_out, width_out}).zero_();\n // resize temporary columns\n columns =\n at::zeros({channels * kernel_h * kernel_w, 1 * height_out * width_out},\n input.options());\n\n output = output.view({output.size(0), group, output.size(1) / group,\n output.size(2), output.size(3)});\n\n for (int b = 0; b < batch; b++) {\n modulated_deformable_im2col_impl(\n input[b], offset[b], mask[b], 1, channels, height, width, height_out,\n width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, deformable_group, columns);\n\n // divide into group\n weight = weight.view({group, weight.size(0) / group, weight.size(1),\n weight.size(2), weight.size(3)});\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n\n for (int g = 0; g < group; g++) {\n output[b][g] = output[b][g]\n .flatten(1)\n .addmm_(weight[g].flatten(1), columns[g])\n .view_as(output[b][g]);\n }\n\n weight = weight.view({weight.size(0) * weight.size(1), weight.size(2),\n weight.size(3), weight.size(4)});\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n }\n\n output = output.view({output.size(0), output.size(1) * output.size(2),\n output.size(3), output.size(4)});\n\n if (with_bias) {\n output += bias.view({1, bias.size(0), 1, 1});\n }\n}\n\nvoid modulated_deform_conv_backward(\n Tensor input, Tensor weight, Tensor bias, Tensor ones, Tensor offset,\n Tensor mask, Tensor columns, Tensor grad_input, Tensor grad_weight,\n Tensor grad_bias, Tensor grad_offset, Tensor grad_mask, Tensor grad_output,\n int kernel_h, int kernel_w, int stride_h, int stride_w, int pad_h,\n int pad_w, int dilation_h, int dilation_w, int group, int deformable_group,\n const bool with_bias) {\n at::DeviceGuard guard(input.device());\n\n const int batch = input.size(0);\n const int channels = input.size(1);\n const int height = input.size(2);\n const int width = input.size(3);\n\n const int channels_kernel = weight.size(1);\n const int kernel_h_ = weight.size(2);\n const int kernel_w_ = weight.size(3);\n if (kernel_h_ != kernel_h || kernel_w_ != kernel_w)\n AT_ERROR(\"Input shape and kernel shape won't match: (%d x %d vs %d x %d).\",\n kernel_h_, kernel_w, kernel_h_, kernel_w_);\n if (channels != channels_kernel * group)\n AT_ERROR(\"Input shape and kernel channels won't match: (%d vs %d).\",\n channels, channels_kernel * group);\n\n const int height_out =\n (height + 2 * pad_h - (dilation_h * (kernel_h - 1) + 1)) / stride_h + 1;\n const int width_out =\n (width + 2 * pad_w - (dilation_w * (kernel_w - 1) + 1)) / stride_w + 1;\n\n if (ones.ndimension() != 2 ||\n ones.size(0) * ones.size(1) < height_out * width_out) {\n // Resize plane and fill with ones...\n ones = at::ones({height_out, width_out}, input.options());\n }\n\n grad_input = grad_input.view({batch, channels, height, width});\n columns = at::zeros({channels * kernel_h * kernel_w, height_out * width_out},\n input.options());\n\n grad_output =\n grad_output.view({grad_output.size(0), group, grad_output.size(1) / group,\n grad_output.size(2), grad_output.size(3)});\n\n for (int b = 0; b < batch; b++) {\n // divide int group\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n weight = weight.view({group, weight.size(0) / group, weight.size(1),\n weight.size(2), weight.size(3)});\n\n for (int g = 0; g < group; g++) {\n columns[g].addmm_(weight[g].flatten(1).transpose(0, 1),\n grad_output[b][g].flatten(1), 0.0f, 1.0f);\n }\n\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n weight = weight.view({weight.size(0) * weight.size(1), weight.size(2),\n weight.size(3), weight.size(4)});\n\n // gradient w.r.t. input coordinate data\n modulated_deformable_col2im_coord_impl(\n columns, input[b], offset[b], mask[b], 1, channels, height, width,\n height_out, width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, deformable_group, grad_offset[b],\n grad_mask[b]);\n // gradient w.r.t. input data\n modulated_deformable_col2im_impl(\n columns, offset[b], mask[b], 1, channels, height, width, height_out,\n width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, deformable_group, grad_input[b]);\n\n // gradient w.r.t. weight, dWeight should accumulate across the batch and\n // group\n modulated_deformable_im2col_impl(\n input[b], offset[b], mask[b], 1, channels, height, width, height_out,\n width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h, stride_w,\n dilation_h, dilation_w, deformable_group, columns);\n\n columns = columns.view({group, columns.size(0) / group, columns.size(1)});\n grad_weight = grad_weight.view({group, grad_weight.size(0) / group,\n grad_weight.size(1), grad_weight.size(2),\n grad_weight.size(3)});\n if (with_bias)\n grad_bias = grad_bias.view({group, grad_bias.size(0) / group});\n\n for (int g = 0; g < group; g++) {\n grad_weight[g] =\n grad_weight[g]\n .flatten(1)\n .addmm_(grad_output[b][g].flatten(1), columns[g].transpose(0, 1))\n .view_as(grad_weight[g]);\n if (with_bias) {\n grad_bias[g] =\n grad_bias[g]\n .view({-1, 1})\n .addmm_(grad_output[b][g].flatten(1), ones.view({-1, 1}))\n .view(-1);\n }\n }\n\n columns =\n columns.view({columns.size(0) * columns.size(1), columns.size(2)});\n grad_weight = grad_weight.view({grad_weight.size(0) * grad_weight.size(1),\n grad_weight.size(2), grad_weight.size(3),\n grad_weight.size(4)});\n if (with_bias)\n grad_bias = grad_bias.view({grad_bias.size(0) * grad_bias.size(1)});\n }\n grad_output = grad_output.view({grad_output.size(0) * grad_output.size(1),\n grad_output.size(2), grad_output.size(3),\n grad_output.size(4)});\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/ms_deform_attn.cpp", "content": "/*!\n**************************************************************************************************\n* Deformable DETR\n* Copyright (c) 2020 SenseTime. All Rights Reserved.\n* Licensed under the Apache License, Version 2.0 [see LICENSE for details]\n**************************************************************************************************\n* Modified from\n*https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0\n**************************************************************************************************\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nTensor ms_deform_attn_impl_forward(const Tensor &value,\n const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight,\n const int im2col_step) {\n return DISPATCH_DEVICE_IMPL(ms_deform_attn_impl_forward, value,\n spatial_shapes, level_start_index, sampling_loc,\n attn_weight, im2col_step);\n}\n\nvoid ms_deform_attn_impl_backward(\n const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index, const Tensor &sampling_loc,\n const Tensor &attn_weight, const Tensor &grad_output, Tensor &grad_value,\n Tensor &grad_sampling_loc, Tensor &grad_attn_weight,\n const int im2col_step) {\n DISPATCH_DEVICE_IMPL(ms_deform_attn_impl_backward, value, spatial_shapes,\n level_start_index, sampling_loc, attn_weight,\n grad_output, grad_value, grad_sampling_loc,\n grad_attn_weight, im2col_step);\n}\n\nTensor ms_deform_attn_forward(const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight,\n const int im2col_step) {\n at::DeviceGuard guard(value.device());\n return ms_deform_attn_impl_forward(value, spatial_shapes, level_start_index,\n sampling_loc, attn_weight, im2col_step);\n}\n\nvoid ms_deform_attn_backward(const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight,\n const Tensor &grad_output, Tensor &grad_value,\n Tensor &grad_sampling_loc,\n Tensor &grad_attn_weight, const int im2col_step) {\n at::DeviceGuard guard(value.device());\n ms_deform_attn_impl_backward(value, spatial_shapes, level_start_index,\n sampling_loc, attn_weight, grad_output,\n grad_value, grad_sampling_loc, grad_attn_weight,\n im2col_step);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/nms.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nTensor nms_impl(Tensor boxes, Tensor scores, float iou_threshold, int offset) {\n return DISPATCH_DEVICE_IMPL(nms_impl, boxes, scores, iou_threshold, offset);\n}\n\nTensor softnms_impl(Tensor boxes, Tensor scores, Tensor dets,\n float iou_threshold, float sigma, float min_score,\n int method, int offset) {\n return DISPATCH_DEVICE_IMPL(softnms_impl, boxes, scores, dets, iou_threshold,\n sigma, min_score, method, offset);\n}\n\nstd::vector > nms_match_impl(Tensor dets,\n float iou_threshold) {\n return DISPATCH_DEVICE_IMPL(nms_match_impl, dets, iou_threshold);\n}\n\nTensor nms(Tensor boxes, Tensor scores, float iou_threshold, int offset) {\n return nms_impl(boxes, scores, iou_threshold, offset);\n}\n\nTensor softnms(Tensor boxes, Tensor scores, Tensor dets, float iou_threshold,\n float sigma, float min_score, int method, int offset) {\n return softnms_impl(boxes, scores, dets, iou_threshold, sigma, min_score,\n method, offset);\n}\n\nstd::vector > nms_match(Tensor dets, float iou_threshold) {\n return nms_match_impl(dets, iou_threshold);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/nms_rotated.cpp", "content": "// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved\n// modified from\n// https://github.com/facebookresearch/detectron2/blob/master/detectron2/layers/csrc/nms_rotated/nms_rotated.h\n#include \"pytorch_cpp_helper.hpp\"\n\nTensor nms_rotated_cpu(const Tensor dets, const Tensor scores,\n const float iou_threshold);\n\n#ifdef MMCV_WITH_CUDA\nTensor nms_rotated_cuda(const Tensor dets, const Tensor scores,\n const Tensor order, const Tensor dets_sorted,\n const float iou_threshold, const int multi_label);\n#endif\n\n// Interface for Python\n// inline is needed to prevent multiple function definitions when this header is\n// included by different cpps\nTensor nms_rotated(const Tensor dets, const Tensor scores, const Tensor order,\n const Tensor dets_sorted, const float iou_threshold,\n const int multi_label) {\n assert(dets.device().is_cuda() == scores.device().is_cuda());\n if (dets.device().is_cuda()) {\n#ifdef MMCV_WITH_CUDA\n return nms_rotated_cuda(dets, scores, order, dets_sorted, iou_threshold,\n multi_label);\n#else\n AT_ERROR(\"Not compiled with GPU support\");\n#endif\n }\n\n return nms_rotated_cpu(dets, scores, iou_threshold);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/pixel_group.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// It is modified from https://github.com/WenmuZhou/PAN.pytorch\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nstd::vector> pixel_group_impl(\n Tensor score, Tensor mask, Tensor embedding, Tensor kernel_label,\n Tensor kernel_contour, int kernel_region_num, float dis_threshold) {\n return DISPATCH_DEVICE_IMPL(pixel_group_impl, score, mask, embedding,\n kernel_label, kernel_contour, kernel_region_num,\n dis_threshold);\n}\n\nstd::vector> pixel_group(\n Tensor score, Tensor mask, Tensor embedding, Tensor kernel_label,\n Tensor kernel_contour, int kernel_region_num, float distance_threshold) {\n score = score.contiguous();\n mask = mask.contiguous();\n embedding = embedding.contiguous();\n kernel_label = kernel_label.contiguous();\n kernel_contour = kernel_contour.contiguous();\n\n return pixel_group_impl(score, mask, embedding, kernel_label, kernel_contour,\n kernel_region_num, distance_threshold);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/points_in_boxes.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid points_in_boxes_part_forward_impl(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n DISPATCH_DEVICE_IMPL(points_in_boxes_part_forward_impl, batch_size, boxes_num,\n pts_num, boxes, pts, box_idx_of_points);\n}\n\nvoid points_in_boxes_all_forward_impl(int batch_size, int boxes_num,\n int pts_num, const Tensor boxes,\n const Tensor pts,\n Tensor box_idx_of_points) {\n DISPATCH_DEVICE_IMPL(points_in_boxes_all_forward_impl, batch_size, boxes_num,\n pts_num, boxes, pts, box_idx_of_points);\n}\n\nvoid points_in_boxes_part_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor box_idx_of_points_tensor) {\n // params boxes: (B, N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is the bottom center, each box params pts: (B, npoints, 3)\n // [x, y, z] in LiDAR coordinate params boxes_idx_of_points: (B, npoints),\n // default -1\n int batch_size = boxes_tensor.size(0);\n int boxes_num = boxes_tensor.size(1);\n int pts_num = pts_tensor.size(1);\n points_in_boxes_part_forward_impl(batch_size, boxes_num, pts_num,\n boxes_tensor, pts_tensor,\n box_idx_of_points_tensor);\n}\n\nvoid points_in_boxes_all_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor box_idx_of_points_tensor) {\n // params boxes: (B, N, 7) [x, y, z, x_size, y_size, z_size, rz] in LiDAR\n // coordinate, z is the bottom center. params pts: (B, npoints, 3) [x, y, z]\n // in LiDAR coordinate params boxes_idx_of_points: (B, npoints), default -1\n int batch_size = boxes_tensor.size(0);\n int boxes_num = boxes_tensor.size(1);\n int pts_num = pts_tensor.size(1);\n points_in_boxes_all_forward_impl(batch_size, boxes_num, pts_num, boxes_tensor,\n pts_tensor, box_idx_of_points_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/points_in_polygons.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid points_in_polygons_forward_impl(const Tensor points, const Tensor polygons,\n Tensor output, const int rows,\n const int cols) {\n DISPATCH_DEVICE_IMPL(points_in_polygons_forward_impl, points, polygons,\n output, rows, cols);\n}\n\nvoid points_in_polygons_forward(Tensor points, Tensor polygons, Tensor output) {\n int rows = points.size(0);\n int cols = polygons.size(0);\n points_in_polygons_forward_impl(points, polygons, output, rows, cols);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/psamask.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from\n// https://github.com/hszhao/semseg/blob/master/lib/psa/src\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid psamask_forward_impl(const int psa_type, const Tensor input, Tensor output,\n const int num_, const int h_feature,\n const int w_feature, const int h_mask,\n const int w_mask, const int half_h_mask,\n const int half_w_mask) {\n DISPATCH_DEVICE_IMPL(psamask_forward_impl, psa_type, input, output, num_,\n h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask);\n}\n\nvoid psamask_backward_impl(const int psa_type, const Tensor grad_output,\n Tensor grad_input, const int num_,\n const int h_feature, const int w_feature,\n const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask) {\n DISPATCH_DEVICE_IMPL(psamask_backward_impl, psa_type, grad_output, grad_input,\n num_, h_feature, w_feature, h_mask, w_mask, half_h_mask,\n half_w_mask);\n}\n\nvoid psamask_forward(const Tensor input, Tensor output, const int psa_type,\n const int num_, const int h_feature, const int w_feature,\n const int h_mask, const int w_mask, const int half_h_mask,\n const int half_w_mask) {\n psamask_forward_impl(psa_type, input, output, num_, h_feature, w_feature,\n h_mask, w_mask, half_h_mask, half_w_mask);\n}\n\nvoid psamask_backward(Tensor grad_output, const Tensor grad_input,\n const int psa_type, const int num_, const int h_feature,\n const int w_feature, const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask) {\n psamask_backward_impl(psa_type, grad_output, grad_input, num_, h_feature,\n w_feature, h_mask, w_mask, half_h_mask, half_w_mask);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/pybind.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n\nstd::string get_compiler_version();\nstd::string get_compiling_cuda_version();\n\nvoid assign_score_withk_forward(const Tensor &points, const Tensor ¢ers,\n const Tensor &scores, const Tensor &knn_idx,\n Tensor &output, int B, int N0, int N1, int M,\n int K, int O, int aggregate);\n\nvoid assign_score_withk_backward(const Tensor &grad_out, const Tensor &points,\n const Tensor ¢ers, const Tensor &scores,\n const Tensor &knn_idx, Tensor &grad_points,\n Tensor &grad_centers, Tensor &grad_scores,\n int B, int N0, int N1, int M, int K, int O,\n int aggregate);\n\nvoid carafe_naive_forward(Tensor features, Tensor masks, Tensor output,\n int kernel_size, int group_size, int scale_factor);\n\nvoid carafe_naive_backward(Tensor top_grad, Tensor features, Tensor masks,\n Tensor bottom_grad, Tensor mask_grad,\n int kernel_size, int group_size, int scale_factor);\n\nvoid carafe_forward(Tensor features, Tensor masks, Tensor rfeatures,\n Tensor routput, Tensor rmasks, Tensor output,\n int kernel_size, int group_size, int scale_factor);\n\nvoid carafe_backward(Tensor top_grad, Tensor rfeatures, Tensor masks,\n Tensor rtop_grad, Tensor rbottom_grad_hs,\n Tensor rbottom_grad, Tensor rmask_grad, Tensor bottom_grad,\n Tensor mask_grad, int kernel_size, int group_size,\n int scale_factor);\n\nvoid deform_conv_forward(Tensor input, Tensor weight, Tensor offset,\n Tensor output, Tensor columns, Tensor ones, int kW,\n int kH, int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step);\n\nvoid deform_conv_backward_input(Tensor input, Tensor offset, Tensor gradOutput,\n Tensor gradInput, Tensor gradOffset,\n Tensor weight, Tensor columns, int kW, int kH,\n int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step);\n\nvoid deform_conv_backward_parameters(Tensor input, Tensor offset,\n Tensor gradOutput, Tensor gradWeight,\n Tensor columns, Tensor ones, int kW,\n int kH, int dW, int dH, int padW, int padH,\n int dilationW, int dilationH, int group,\n int deformable_group, float scale,\n int im2col_step);\n\nvoid deform_roi_pool_forward(Tensor input, Tensor rois, Tensor offset,\n Tensor output, int pooled_height, int pooled_width,\n float spatial_scale, int sampling_ratio,\n float gamma);\n\nvoid deform_roi_pool_backward(Tensor grad_output, Tensor input, Tensor rois,\n Tensor offset, Tensor grad_input,\n Tensor grad_offset, int pooled_height,\n int pooled_width, float spatial_scale,\n int sampling_ratio, float gamma);\n\nvoid group_points_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor out_tensor, int b, int c, int n, int npoints,\n int nsample);\n\nvoid group_points_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor grad_points_tensor, int b, int c, int n,\n int npoints, int nsample);\n\nvoid roipoint_pool3d_forward(Tensor xyz, Tensor boxes3d, Tensor pts_feature,\n Tensor pooled_features, Tensor pooled_empty_flag);\n\nvoid gather_points_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor out_tensor, int b, int c, int n, int npoints);\n\nvoid gather_points_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor grad_points_tensor, int b, int c, int n,\n int npoints);\n\nvoid sigmoid_focal_loss_forward(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha);\n\nvoid sigmoid_focal_loss_backward(Tensor input, Tensor target, Tensor weight,\n Tensor grad_input, float gamma, float alpha);\n\nvoid softmax_focal_loss_forward(Tensor input, Tensor target, Tensor weight,\n Tensor output, float gamma, float alpha);\n\nvoid softmax_focal_loss_backward(Tensor input, Tensor target, Tensor weight,\n Tensor buff, Tensor grad_input, float gamma,\n float alpha);\n\nvoid three_interpolate_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor weight_tensor, Tensor out_tensor, int b,\n int c, int m, int n);\n\nvoid three_interpolate_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor weight_tensor, Tensor grad_points_tensor,\n int b, int c, int n, int m);\n\nvoid three_nn_forward(Tensor unknown_tensor, Tensor known_tensor,\n Tensor dist2_tensor, Tensor idx_tensor, int b, int n,\n int m);\n\nvoid bbox_overlaps(const Tensor bboxes1, const Tensor bboxes2, Tensor ious,\n const int mode, const bool aligned, const int offset);\n\nvoid knn_forward(Tensor xyz_tensor, Tensor new_xyz_tensor, Tensor idx_tensor,\n Tensor dist2_tensor, int b, int n, int m, int nsample);\nvoid iou3d_boxes_overlap_bev_forward(Tensor boxes_a, Tensor boxes_b,\n Tensor ans_overlap);\n\nvoid iou3d_boxes_iou_bev_forward(Tensor boxes_a, Tensor boxes_b,\n Tensor ans_iou);\n\nvoid iou3d_nms_forward(Tensor boxes, Tensor keep, Tensor keep_num,\n float nms_overlap_thresh);\n\nvoid iou3d_nms_normal_forward(Tensor boxes, Tensor keep, Tensor keep_num,\n float nms_overlap_thresh);\n\nvoid furthest_point_sampling_forward(Tensor points_tensor, Tensor temp_tensor,\n Tensor idx_tensor, int b, int n, int m);\n\nvoid furthest_point_sampling_with_dist_forward(Tensor points_tensor,\n Tensor temp_tensor,\n Tensor idx_tensor, int b, int n,\n int m);\n\nvoid masked_im2col_forward(const Tensor im, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor col,\n const int kernel_h, const int kernel_w,\n const int pad_h, const int pad_w);\n\nvoid masked_col2im_forward(const Tensor col, const Tensor mask_h_idx,\n const Tensor mask_w_idx, Tensor im, int height,\n int width, int channels);\n\nvoid modulated_deform_conv_forward(\n Tensor input, Tensor weight, Tensor bias, Tensor ones, Tensor offset,\n Tensor mask, Tensor output, Tensor columns, int kernel_h, int kernel_w,\n const int stride_h, const int stride_w, const int pad_h, const int pad_w,\n const int dilation_h, const int dilation_w, const int group,\n const int deformable_group, const bool with_bias);\n\nvoid modulated_deform_conv_backward(\n Tensor input, Tensor weight, Tensor bias, Tensor ones, Tensor offset,\n Tensor mask, Tensor columns, Tensor grad_input, Tensor grad_weight,\n Tensor grad_bias, Tensor grad_offset, Tensor grad_mask, Tensor grad_output,\n int kernel_h, int kernel_w, int stride_h, int stride_w, int pad_h,\n int pad_w, int dilation_h, int dilation_w, int group, int deformable_group,\n const bool with_bias);\n\nTensor ms_deform_attn_forward(const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight, const int im2col_step);\n\nvoid ms_deform_attn_backward(const Tensor &value, const Tensor &spatial_shapes,\n const Tensor &level_start_index,\n const Tensor &sampling_loc,\n const Tensor &attn_weight,\n const Tensor &grad_output, Tensor &grad_value,\n Tensor &grad_sampling_loc,\n Tensor &grad_attn_weight, const int im2col_step);\n\nTensor nms(Tensor boxes, Tensor scores, float iou_threshold, int offset);\n\nTensor softnms(Tensor boxes, Tensor scores, Tensor dets, float iou_threshold,\n float sigma, float min_score, int method, int offset);\n\nstd::vector> nms_match(Tensor dets, float iou_threshold);\n\nstd::vector> pixel_group(\n Tensor score, Tensor mask, Tensor embedding, Tensor kernel_label,\n Tensor kernel_contour, int kernel_region_num, float distance_threshold);\n\nstd::vector> contour_expand(Tensor kernel_mask,\n Tensor internal_kernel_label,\n int min_kernel_area,\n int kernel_num);\n\nvoid roi_align_forward(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned);\n\nvoid roi_align_backward(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned);\n\nvoid roi_pool_forward(Tensor input, Tensor rois, Tensor output, Tensor argmax,\n int pooled_height, int pooled_width, float spatial_scale);\n\nvoid roi_pool_backward(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height, int pooled_width,\n float spatial_scale);\n\nvoid sync_bn_forward_mean(const Tensor input, Tensor mean);\n\nvoid sync_bn_forward_var(const Tensor input, const Tensor mean, Tensor var);\n\nvoid sync_bn_forward_output(const Tensor input, const Tensor mean,\n const Tensor var, const Tensor weight,\n const Tensor bias, Tensor running_mean,\n Tensor running_var, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size);\n\nvoid sync_bn_backward_param(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias);\n\nvoid sync_bn_backward_data(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight, const Tensor grad_bias,\n const Tensor norm, const Tensor std,\n Tensor grad_input);\n\nvoid psamask_forward(const Tensor input, Tensor output, const int psa_type,\n const int num_, const int h_feature, const int w_feature,\n const int h_mask, const int w_mask, const int half_h_mask,\n const int half_w_mask);\n\nvoid psamask_backward(Tensor grad_output, const Tensor grad_input,\n const int psa_type, const int num_, const int h_feature,\n const int w_feature, const int h_mask, const int w_mask,\n const int half_h_mask, const int half_w_mask);\n\nvoid tin_shift_forward(Tensor input, Tensor shift, Tensor output);\n\nvoid tin_shift_backward(Tensor grad_output, Tensor shift, Tensor grad_input);\n\nvoid ball_query_forward(Tensor new_xyz_tensor, Tensor xyz_tensor,\n Tensor idx_tensor, int b, int n, int m,\n float min_radius, float max_radius, int nsample);\n\nTensor bottom_pool_forward(Tensor input);\n\nTensor bottom_pool_backward(Tensor input, Tensor grad_output);\n\nTensor left_pool_forward(Tensor input);\n\nTensor left_pool_backward(Tensor input, Tensor grad_output);\n\nTensor right_pool_forward(Tensor input);\n\nTensor right_pool_backward(Tensor input, Tensor grad_output);\n\nTensor top_pool_forward(Tensor input);\n\nTensor top_pool_backward(Tensor input, Tensor grad_output);\n\nvoid box_iou_rotated(const Tensor boxes1, const Tensor boxes2, Tensor ious,\n const int mode_flag, const bool aligned);\n\nTensor nms_rotated(const Tensor dets, const Tensor scores, const Tensor order,\n const Tensor dets_sorted, const float iou_threshold,\n const int multi_label);\n\nTensor upfirdn2d(const Tensor &input, const Tensor &kernel, int up_x, int up_y,\n int down_x, int down_y, int pad_x0, int pad_x1, int pad_y0,\n int pad_y1);\n\nTensor fused_bias_leakyrelu(const Tensor &input, const Tensor &bias,\n const Tensor &refer, int act, int grad, float alpha,\n float scale);\n\nvoid roi_align_rotated_forward(Tensor input, Tensor rois, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale, int sample_num,\n bool aligned, bool clockwise);\n\nvoid roi_align_rotated_backward(Tensor grad_output, Tensor rois,\n Tensor grad_input, int pooled_height,\n int pooled_width, float spatial_scale,\n int sample_num, bool aligned, bool clockwise);\n\nstd::vector dynamic_point_to_voxel_forward(\n const torch::Tensor &feats, const torch::Tensor &coors,\n const std::string &reduce_type);\n\nvoid dynamic_point_to_voxel_backward(torch::Tensor &grad_feats,\n const torch::Tensor &grad_reduced_feats,\n const torch::Tensor &feats,\n const torch::Tensor &reduced_feats,\n const torch::Tensor &coors_idx,\n const torch::Tensor &reduce_count,\n const std::string &reduce_type);\n\nvoid hard_voxelize_forward(const at::Tensor &points,\n const at::Tensor &voxel_size,\n const at::Tensor &coors_range, at::Tensor &voxels,\n at::Tensor &coors, at::Tensor &num_points_per_voxel,\n at::Tensor &voxel_num, const int max_points,\n const int max_voxels, const int NDim);\n\nvoid dynamic_voxelize_forward(const at::Tensor &points,\n const at::Tensor &voxel_size,\n const at::Tensor &coors_range, at::Tensor &coors,\n const int NDim);\n\nvoid border_align_forward(const Tensor &input, const Tensor &boxes,\n Tensor output, Tensor argmax_idx,\n const int pool_size);\n\nvoid border_align_backward(const Tensor &grad_output, const Tensor &boxes,\n const Tensor &argmax_idx, Tensor grad_input,\n const int pool_size);\n\nvoid points_in_boxes_cpu_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor pts_indices_tensor);\n\nvoid points_in_boxes_part_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor box_idx_of_points_tensor);\n\nvoid points_in_boxes_all_forward(Tensor boxes_tensor, Tensor pts_tensor,\n Tensor box_idx_of_points_tensor);\n\nvoid roiaware_pool3d_forward(Tensor rois, Tensor pts, Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method);\n\nvoid roiaware_pool3d_backward(Tensor pts_idx_of_voxels, Tensor argmax,\n Tensor grad_out, Tensor grad_in, int pool_method);\n\nvoid correlation_forward(Tensor input1, Tensor input2, Tensor output, int kH,\n int kW, int patchH, int patchW, int padH, int padW,\n int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW);\n\nvoid correlation_backward(Tensor grad_output, Tensor input1, Tensor input2,\n Tensor grad_input1, Tensor grad_input2, int kH,\n int kW, int patchH, int patchW, int padH, int padW,\n int dilationH, int dilationW, int dilation_patchH,\n int dilation_patchW, int dH, int dW);\n\nvoid rotated_feature_align_forward(const Tensor features,\n const Tensor best_bboxes, Tensor output,\n const float spatial_scale, const int points);\n\nvoid rotated_feature_align_backward(const Tensor top_grad,\n const Tensor best_bboxes,\n Tensor bottom_grad,\n const float spatial_scale,\n const int points);\n\nvoid riroi_align_rotated_forward(Tensor features, Tensor rois, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale, int num_samples,\n int num_orientations, bool clockwise);\n\nvoid riroi_align_rotated_backward(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise);\n\nvoid points_in_polygons_forward(Tensor points, Tensor polygons, Tensor output);\n\nvoid min_area_polygons(const Tensor pointsets, Tensor polygons);\n\nvoid active_rotated_filter_forward(const Tensor input, const Tensor indices,\n Tensor output);\n\nvoid active_rotated_filter_backward(const Tensor grad_out, const Tensor indices,\n Tensor grad_in);\n\nvoid convex_iou(const Tensor pointsets, const Tensor polygons, Tensor ious);\n\nvoid convex_giou(const Tensor pointsets, const Tensor polygons, Tensor output);\n\nPYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {\n m.def(\"upfirdn2d\", &upfirdn2d, \"upfirdn2d (CUDA)\", py::arg(\"input\"),\n py::arg(\"kernel\"), py::arg(\"up_x\"), py::arg(\"up_y\"), py::arg(\"down_x\"),\n py::arg(\"down_y\"), py::arg(\"pad_x0\"), py::arg(\"pad_x1\"),\n py::arg(\"pad_y0\"), py::arg(\"pad_y1\"));\n m.def(\"fused_bias_leakyrelu\", &fused_bias_leakyrelu,\n \"fused_bias_leakyrelu (CUDA)\", py::arg(\"input\"), py::arg(\"bias\"),\n py::arg(\"empty\"), py::arg(\"act\"), py::arg(\"grad\"), py::arg(\"alpha\"),\n py::arg(\"scale\"));\n m.def(\"gather_points_forward\", &gather_points_forward,\n \"gather_points_forward\", py::arg(\"points_tensor\"),\n py::arg(\"idx_tensor\"), py::arg(\"out_tensor\"), py::arg(\"b\"),\n py::arg(\"c\"), py::arg(\"n\"), py::arg(\"npoints\"));\n m.def(\"gather_points_backward\", &gather_points_backward,\n \"gather_points_backward\", py::arg(\"grad_out_tensor\"),\n py::arg(\"idx_tensor\"), py::arg(\"grad_points_tensor\"), py::arg(\"b\"),\n py::arg(\"c\"), py::arg(\"n\"), py::arg(\"npoints\"));\n m.def(\"get_compiler_version\", &get_compiler_version, \"get_compiler_version\");\n m.def(\"get_compiling_cuda_version\", &get_compiling_cuda_version,\n \"get_compiling_cuda_version\");\n m.def(\"assign_score_withk_forward\", &assign_score_withk_forward,\n \"assign_score_withk_forward\", py::arg(\"points\"), py::arg(\"centers\"),\n py::arg(\"scores\"), py::arg(\"knn_idx\"), py::arg(\"output\"), py::arg(\"B\"),\n py::arg(\"N0\"), py::arg(\"N1\"), py::arg(\"M\"), py::arg(\"K\"), py::arg(\"O\"),\n py::arg(\"aggregate\"));\n m.def(\"assign_score_withk_backward\", &assign_score_withk_backward,\n \"assign_score_withk_backward\", py::arg(\"grad_out\"), py::arg(\"points\"),\n py::arg(\"centers\"), py::arg(\"scores\"), py::arg(\"knn_idx\"),\n py::arg(\"grad_points\"), py::arg(\"grad_centers\"), py::arg(\"grad_scores\"),\n py::arg(\"B\"), py::arg(\"N0\"), py::arg(\"N1\"), py::arg(\"M\"), py::arg(\"K\"),\n py::arg(\"O\"), py::arg(\"aggregate\"));\n m.def(\"knn_forward\", &knn_forward, \"knn_forward\", py::arg(\"xyz_tensor\"),\n py::arg(\"new_xyz_tensor\"), py::arg(\"idx_tensor\"),\n py::arg(\"dist2_tensor\"), py::arg(\"b\"), py::arg(\"n\"), py::arg(\"m\"),\n py::arg(\"nsample\"));\n m.def(\"carafe_naive_forward\", &carafe_naive_forward, \"carafe_naive_forward\",\n py::arg(\"features\"), py::arg(\"masks\"), py::arg(\"output\"),\n py::arg(\"kernel_size\"), py::arg(\"group_size\"), py::arg(\"scale_factor\"));\n m.def(\"carafe_naive_backward\", &carafe_naive_backward,\n \"carafe_naive_backward\", py::arg(\"top_grad\"), py::arg(\"features\"),\n py::arg(\"masks\"), py::arg(\"bottom_grad\"), py::arg(\"mask_grad\"),\n py::arg(\"kernel_size\"), py::arg(\"group_size\"), py::arg(\"scale_factor\"));\n m.def(\"carafe_forward\", &carafe_forward, \"carafe_forward\",\n py::arg(\"features\"), py::arg(\"masks\"), py::arg(\"rfeatures\"),\n py::arg(\"routput\"), py::arg(\"rmasks\"), py::arg(\"output\"),\n py::arg(\"kernel_size\"), py::arg(\"group_size\"), py::arg(\"scale_factor\"));\n m.def(\"carafe_backward\", &carafe_backward, \"carafe_backward\",\n py::arg(\"top_grad\"), py::arg(\"rfeatures\"), py::arg(\"masks\"),\n py::arg(\"rtop_grad\"), py::arg(\"rbottom_grad_hs\"),\n py::arg(\"rbottom_grad\"), py::arg(\"rmask_grad\"), py::arg(\"bottom_grad\"),\n py::arg(\"mask_grad\"), py::arg(\"kernel_size\"), py::arg(\"group_size\"),\n py::arg(\"scale_factor\"));\n m.def(\"deform_conv_forward\", &deform_conv_forward, \"deform_conv_forward\",\n py::arg(\"input\"), py::arg(\"weight\"), py::arg(\"offset\"),\n py::arg(\"output\"), py::arg(\"columns\"), py::arg(\"ones\"), py::arg(\"kW\"),\n py::arg(\"kH\"), py::arg(\"dW\"), py::arg(\"dH\"), py::arg(\"padH\"),\n py::arg(\"padW\"), py::arg(\"dilationW\"), py::arg(\"dilationH\"),\n py::arg(\"group\"), py::arg(\"deformable_group\"), py::arg(\"im2col_step\"));\n m.def(\"deform_conv_backward_input\", &deform_conv_backward_input,\n \"deform_conv_backward_input\", py::arg(\"input\"), py::arg(\"offset\"),\n py::arg(\"gradOutput\"), py::arg(\"gradInput\"), py::arg(\"gradOffset\"),\n py::arg(\"weight\"), py::arg(\"columns\"), py::arg(\"kW\"), py::arg(\"kH\"),\n py::arg(\"dW\"), py::arg(\"dH\"), py::arg(\"padH\"), py::arg(\"padW\"),\n py::arg(\"dilationW\"), py::arg(\"dilationH\"), py::arg(\"group\"),\n py::arg(\"deformable_group\"), py::arg(\"im2col_step\"));\n m.def(\"deform_conv_backward_parameters\", &deform_conv_backward_parameters,\n \"deform_conv_backward_parameters\", py::arg(\"input\"), py::arg(\"offset\"),\n py::arg(\"gradOutput\"), py::arg(\"gradWeight\"), py::arg(\"columns\"),\n py::arg(\"ones\"), py::arg(\"kW\"), py::arg(\"kH\"), py::arg(\"dW\"),\n py::arg(\"dH\"), py::arg(\"padH\"), py::arg(\"padW\"), py::arg(\"dilationW\"),\n py::arg(\"dilationH\"), py::arg(\"group\"), py::arg(\"deformable_group\"),\n py::arg(\"scale\"), py::arg(\"im2col_step\"));\n m.def(\"deform_roi_pool_forward\", &deform_roi_pool_forward,\n \"deform roi pool forward\", py::arg(\"input\"), py::arg(\"rois\"),\n py::arg(\"offset\"), py::arg(\"output\"), py::arg(\"pooled_height\"),\n py::arg(\"pooled_width\"), py::arg(\"spatial_scale\"),\n py::arg(\"sampling_ratio\"), py::arg(\"gamma\"));\n m.def(\"deform_roi_pool_backward\", &deform_roi_pool_backward,\n \"deform roi pool backward\", py::arg(\"grad_output\"), py::arg(\"input\"),\n py::arg(\"rois\"), py::arg(\"offset\"), py::arg(\"grad_input\"),\n py::arg(\"grad_offset\"), py::arg(\"pooled_height\"),\n py::arg(\"pooled_width\"), py::arg(\"spatial_scale\"),\n py::arg(\"sampling_ratio\"), py::arg(\"gamma\"));\n m.def(\"roipoint_pool3d_forward\", &roipoint_pool3d_forward,\n \"roipoint_pool3d_forward\", py::arg(\"xyz\"), py::arg(\"boxes3d\"),\n py::arg(\"pts_feature\"), py::arg(\"pooled_features\"),\n py::arg(\"pooled_empty_flag\"));\n m.def(\"sigmoid_focal_loss_forward\", &sigmoid_focal_loss_forward,\n \"sigmoid_focal_loss_forward \", py::arg(\"input\"), py::arg(\"target\"),\n py::arg(\"weight\"), py::arg(\"output\"), py::arg(\"gamma\"),\n py::arg(\"alpha\"));\n m.def(\"sigmoid_focal_loss_backward\", &sigmoid_focal_loss_backward,\n \"sigmoid_focal_loss_backward\", py::arg(\"input\"), py::arg(\"target\"),\n py::arg(\"weight\"), py::arg(\"grad_input\"), py::arg(\"gamma\"),\n py::arg(\"alpha\"));\n m.def(\"softmax_focal_loss_forward\", &softmax_focal_loss_forward,\n \"softmax_focal_loss_forward\", py::arg(\"input\"), py::arg(\"target\"),\n py::arg(\"weight\"), py::arg(\"output\"), py::arg(\"gamma\"),\n py::arg(\"alpha\"));\n m.def(\"softmax_focal_loss_backward\", &softmax_focal_loss_backward,\n \"softmax_focal_loss_backward\", py::arg(\"input\"), py::arg(\"target\"),\n py::arg(\"weight\"), py::arg(\"buff\"), py::arg(\"grad_input\"),\n py::arg(\"gamma\"), py::arg(\"alpha\"));\n m.def(\"three_interpolate_forward\", &three_interpolate_forward,\n \"three_interpolate_forward\", py::arg(\"points_tensor\"),\n py::arg(\"idx_tensor\"), py::arg(\"weight_tensor\"), py::arg(\"out_tensor\"),\n py::arg(\"b\"), py::arg(\"c\"), py::arg(\"m\"), py::arg(\"n\"));\n m.def(\"three_interpolate_backward\", &three_interpolate_backward,\n \"three_interpolate_backward\", py::arg(\"grad_out_tensor\"),\n py::arg(\"idx_tensor\"), py::arg(\"weight_tensor\"),\n py::arg(\"grad_points_tensor\"), py::arg(\"b\"), py::arg(\"c\"), py::arg(\"n\"),\n py::arg(\"m\"));\n m.def(\"three_nn_forward\", &three_nn_forward, \"three_nn_forward\",\n py::arg(\"unknown_tensor\"), py::arg(\"known_tensor\"),\n py::arg(\"dist2_tensor\"), py::arg(\"idx_tensor\"), py::arg(\"b\"),\n py::arg(\"n\"), py::arg(\"m\"));\n m.def(\"bbox_overlaps\", &bbox_overlaps, \"bbox_overlaps\", py::arg(\"bboxes1\"),\n py::arg(\"bboxes2\"), py::arg(\"ious\"), py::arg(\"mode\"),\n py::arg(\"aligned\"), py::arg(\"offset\"));\n m.def(\"group_points_forward\", &group_points_forward, \"group_points_forward\",\n py::arg(\"points_tensor\"), py::arg(\"idx_tensor\"), py::arg(\"out_tensor\"),\n py::arg(\"b\"), py::arg(\"c\"), py::arg(\"n\"), py::arg(\"npoints\"),\n py::arg(\"nsample\"));\n m.def(\"group_points_backward\", &group_points_backward,\n \"group_points_backward\", py::arg(\"grad_out_tensor\"),\n py::arg(\"idx_tensor\"), py::arg(\"grad_points_tensor\"), py::arg(\"b\"),\n py::arg(\"c\"), py::arg(\"n\"), py::arg(\"npoints\"), py::arg(\"nsample\"));\n m.def(\"knn_forward\", &knn_forward, \"knn_forward\", py::arg(\"b\"), py::arg(\"n\"),\n py::arg(\"m\"), py::arg(\"nsample\"), py::arg(\"xyz_tensor\"),\n py::arg(\"new_xyz_tensor\"), py::arg(\"idx_tensor\"),\n py::arg(\"dist2_tensor\"));\n m.def(\"iou3d_boxes_overlap_bev_forward\", &iou3d_boxes_overlap_bev_forward,\n \"iou3d_boxes_overlap_bev_forward\", py::arg(\"boxes_a\"),\n py::arg(\"boxes_b\"), py::arg(\"ans_overlap\"));\n m.def(\"iou3d_boxes_iou_bev_forward\", &iou3d_boxes_iou_bev_forward,\n \"iou3d_boxes_iou_bev_forward\", py::arg(\"boxes_a\"), py::arg(\"boxes_b\"),\n py::arg(\"ans_iou\"));\n m.def(\"iou3d_nms_forward\", &iou3d_nms_forward, \"iou3d_nms_forward\",\n py::arg(\"boxes\"), py::arg(\"keep\"), py::arg(\"num_out\"),\n py::arg(\"nms_overlap_thresh\"));\n m.def(\"iou3d_nms_normal_forward\", &iou3d_nms_normal_forward,\n \"iou3d_nms_normal_forward\", py::arg(\"boxes\"), py::arg(\"keep\"),\n py::arg(\"num_out\"), py::arg(\"nms_overlap_thresh\"));\n m.def(\"furthest_point_sampling_forward\", &furthest_point_sampling_forward,\n \"furthest_point_sampling_forward\", py::arg(\"points_tensor\"),\n py::arg(\"temp_tensor\"), py::arg(\"idx_tensor\"), py::arg(\"b\"),\n py::arg(\"n\"), py::arg(\"m\"));\n m.def(\"furthest_point_sampling_with_dist_forward\",\n &furthest_point_sampling_with_dist_forward,\n \"furthest_point_sampling_with_dist_forward\", py::arg(\"points_tensor\"),\n py::arg(\"temp_tensor\"), py::arg(\"idx_tensor\"), py::arg(\"b\"),\n py::arg(\"n\"), py::arg(\"m\"));\n m.def(\"masked_im2col_forward\", &masked_im2col_forward,\n \"masked_im2col_forward\", py::arg(\"im\"), py::arg(\"mask_h_idx\"),\n py::arg(\"mask_w_idx\"), py::arg(\"col\"), py::arg(\"kernel_h\"),\n py::arg(\"kernel_w\"), py::arg(\"pad_h\"), py::arg(\"pad_w\"));\n m.def(\"masked_col2im_forward\", &masked_col2im_forward,\n \"masked_col2im_forward\", py::arg(\"col\"), py::arg(\"mask_h_idx\"),\n py::arg(\"mask_w_idx\"), py::arg(\"im\"), py::arg(\"height\"),\n py::arg(\"width\"), py::arg(\"channels\"));\n m.def(\"modulated_deform_conv_forward\", &modulated_deform_conv_forward,\n \"modulated deform conv forward\", py::arg(\"input\"), py::arg(\"weight\"),\n py::arg(\"bias\"), py::arg(\"ones\"), py::arg(\"offset\"), py::arg(\"mask\"),\n py::arg(\"output\"), py::arg(\"columns\"), py::arg(\"kernel_h\"),\n py::arg(\"kernel_w\"), py::arg(\"stride_h\"), py::arg(\"stride_w\"),\n py::arg(\"pad_h\"), py::arg(\"pad_w\"), py::arg(\"dilation_h\"),\n py::arg(\"dilation_w\"), py::arg(\"group\"), py::arg(\"deformable_group\"),\n py::arg(\"with_bias\"));\n m.def(\"modulated_deform_conv_backward\", &modulated_deform_conv_backward,\n \"modulated deform conv backward\", py::arg(\"input\"), py::arg(\"weight\"),\n py::arg(\"bias\"), py::arg(\"ones\"), py::arg(\"offset\"), py::arg(\"mask\"),\n py::arg(\"columns\"), py::arg(\"grad_input\"), py::arg(\"grad_weight\"),\n py::arg(\"grad_bias\"), py::arg(\"grad_offset\"), py::arg(\"grad_mask\"),\n py::arg(\"grad_output\"), py::arg(\"kernel_h\"), py::arg(\"kernel_w\"),\n py::arg(\"stride_h\"), py::arg(\"stride_w\"), py::arg(\"pad_h\"),\n py::arg(\"pad_w\"), py::arg(\"dilation_h\"), py::arg(\"dilation_w\"),\n py::arg(\"group\"), py::arg(\"deformable_group\"), py::arg(\"with_bias\"));\n m.def(\"nms\", &nms, \"nms (CPU/CUDA) \", py::arg(\"boxes\"), py::arg(\"scores\"),\n py::arg(\"iou_threshold\"), py::arg(\"offset\"));\n m.def(\"softnms\", &softnms, \"softnms (CPU) \", py::arg(\"boxes\"),\n py::arg(\"scores\"), py::arg(\"dets\"), py::arg(\"iou_threshold\"),\n py::arg(\"sigma\"), py::arg(\"min_score\"), py::arg(\"method\"),\n py::arg(\"offset\"));\n m.def(\"nms_match\", &nms_match, \"nms_match (CPU) \", py::arg(\"dets\"),\n py::arg(\"iou_threshold\"));\n m.def(\"pixel_group\", &pixel_group, \"pixel group (CPU) \", py::arg(\"score\"),\n py::arg(\"mask\"), py::arg(\"embedding\"), py::arg(\"kernel_label\"),\n py::arg(\"kernel_contour\"), py::arg(\"kernel_region_label\"),\n py::arg(\"distance_threshold\"));\n m.def(\"contour_expand\", &contour_expand, \"contour exapnd (CPU) \",\n py::arg(\"kernel_mask\"), py::arg(\"internal_kernel_label\"),\n py::arg(\"min_kernel_area\"), py::arg(\"kernel_num\"));\n m.def(\"roi_align_forward\", &roi_align_forward, \"roi_align forward\",\n py::arg(\"input\"), py::arg(\"rois\"), py::arg(\"output\"),\n py::arg(\"argmax_y\"), py::arg(\"argmax_x\"), py::arg(\"aligned_height\"),\n py::arg(\"aligned_width\"), py::arg(\"spatial_scale\"),\n py::arg(\"sampling_ratio\"), py::arg(\"pool_mode\"), py::arg(\"aligned\"));\n m.def(\"roi_align_backward\", &roi_align_backward, \"roi_align backward\",\n py::arg(\"grad_output\"), py::arg(\"rois\"), py::arg(\"argmax_y\"),\n py::arg(\"argmax_x\"), py::arg(\"grad_input\"), py::arg(\"aligned_height\"),\n py::arg(\"aligned_width\"), py::arg(\"spatial_scale\"),\n py::arg(\"sampling_ratio\"), py::arg(\"pool_mode\"), py::arg(\"aligned\"));\n m.def(\"roi_pool_forward\", &roi_pool_forward, \"roi_pool forward\",\n py::arg(\"input\"), py::arg(\"rois\"), py::arg(\"output\"), py::arg(\"argmax\"),\n py::arg(\"pooled_height\"), py::arg(\"pooled_width\"),\n py::arg(\"spatial_scale\"));\n m.def(\"roi_pool_backward\", &roi_pool_backward, \"roi_pool backward\",\n py::arg(\"grad_output\"), py::arg(\"rois\"), py::arg(\"argmax\"),\n py::arg(\"grad_input\"), py::arg(\"pooled_height\"),\n py::arg(\"pooled_width\"), py::arg(\"spatial_scale\"));\n m.def(\"sync_bn_forward_mean\", &sync_bn_forward_mean, \"sync_bn forward_mean\",\n py::arg(\"input\"), py::arg(\"mean\"));\n m.def(\"sync_bn_forward_var\", &sync_bn_forward_var, \"sync_bn forward_var\",\n py::arg(\"input\"), py::arg(\"mean\"), py::arg(\"var\"));\n m.def(\"sync_bn_forward_output\", &sync_bn_forward_output,\n \"sync_bn forward_output\", py::arg(\"input\"), py::arg(\"mean\"),\n py::arg(\"var\"), py::arg(\"weight\"), py::arg(\"bias\"),\n py::arg(\"running_mean\"), py::arg(\"running_var\"), py::arg(\"norm\"),\n py::arg(\"std\"), py::arg(\"output\"), py::arg(\"eps\"), py::arg(\"momentum\"),\n py::arg(\"group_size\"));\n m.def(\"sync_bn_backward_param\", &sync_bn_backward_param,\n \"sync_bn backward_param\", py::arg(\"grad_output\"), py::arg(\"norm\"),\n py::arg(\"grad_weight\"), py::arg(\"grad_bias\"));\n m.def(\"sync_bn_backward_data\", &sync_bn_backward_data,\n \"sync_bn backward_data\", py::arg(\"grad_output\"), py::arg(\"weight\"),\n py::arg(\"grad_weight\"), py::arg(\"grad_bias\"), py::arg(\"norm\"),\n py::arg(\"std\"), py::arg(\"grad_input\"));\n m.def(\"psamask_forward\", &psamask_forward, \"PSAMASK forward (CPU/CUDA)\",\n py::arg(\"input\"), py::arg(\"output\"), py::arg(\"psa_type\"),\n py::arg(\"num_\"), py::arg(\"h_feature\"), py::arg(\"w_feature\"),\n py::arg(\"h_mask\"), py::arg(\"w_mask\"), py::arg(\"half_h_mask\"),\n py::arg(\"half_w_mask\"));\n m.def(\"psamask_backward\", &psamask_backward, \"PSAMASK backward (CPU/CUDA)\",\n py::arg(\"grad_output\"), py::arg(\"grad_input\"), py::arg(\"psa_type\"),\n py::arg(\"num_\"), py::arg(\"h_feature\"), py::arg(\"w_feature\"),\n py::arg(\"h_mask\"), py::arg(\"w_mask\"), py::arg(\"half_h_mask\"),\n py::arg(\"half_w_mask\"));\n m.def(\"tin_shift_forward\", &tin_shift_forward, \"tin_shift forward\",\n py::arg(\"input\"), py::arg(\"shift\"), py::arg(\"output\"));\n m.def(\"tin_shift_backward\", &tin_shift_backward, \"tin_shift backward\",\n py::arg(\"grad_output\"), py::arg(\"shift\"), py::arg(\"grad_input\"));\n m.def(\"bottom_pool_forward\", &bottom_pool_forward, \"Bottom Pool Forward\",\n py::arg(\"input\"), py::call_guard());\n m.def(\"bottom_pool_backward\", &bottom_pool_backward, \"Bottom Pool Backward\",\n py::arg(\"input\"), py::arg(\"grad_output\"),\n py::call_guard());\n m.def(\"left_pool_forward\", &left_pool_forward, \"Left Pool Forward\",\n py::arg(\"input\"), py::call_guard());\n m.def(\"left_pool_backward\", &left_pool_backward, \"Left Pool Backward\",\n py::arg(\"input\"), py::arg(\"grad_output\"),\n py::call_guard());\n m.def(\"right_pool_forward\", &right_pool_forward, \"Right Pool Forward\",\n py::arg(\"input\"), py::call_guard());\n m.def(\"right_pool_backward\", &right_pool_backward, \"Right Pool Backward\",\n py::arg(\"input\"), py::arg(\"grad_output\"),\n py::call_guard());\n m.def(\"top_pool_forward\", &top_pool_forward, \"Top Pool Forward\",\n py::arg(\"input\"), py::call_guard());\n m.def(\"top_pool_backward\", &top_pool_backward, \"Top Pool Backward\",\n py::arg(\"input\"), py::arg(\"grad_output\"),\n py::call_guard());\n m.def(\"box_iou_rotated\", &box_iou_rotated, \"IoU for rotated boxes\",\n py::arg(\"boxes1\"), py::arg(\"boxes2\"), py::arg(\"ious\"),\n py::arg(\"mode_flag\"), py::arg(\"aligned\"));\n m.def(\"nms_rotated\", &nms_rotated, \"NMS for rotated boxes\", py::arg(\"dets\"),\n py::arg(\"scores\"), py::arg(\"order\"), py::arg(\"dets_sorted\"),\n py::arg(\"iou_threshold\"), py::arg(\"multi_label\"));\n m.def(\"ball_query_forward\", &ball_query_forward, \"ball_query_forward\",\n py::arg(\"new_xyz_tensor\"), py::arg(\"xyz_tensor\"), py::arg(\"idx_tensor\"),\n py::arg(\"b\"), py::arg(\"n\"), py::arg(\"m\"), py::arg(\"min_radius\"),\n py::arg(\"max_radius\"), py::arg(\"nsample\"));\n m.def(\"roi_align_rotated_forward\", &roi_align_rotated_forward,\n \"roi_align_rotated forward\", py::arg(\"input\"), py::arg(\"rois\"),\n py::arg(\"output\"), py::arg(\"pooled_height\"), py::arg(\"pooled_width\"),\n py::arg(\"spatial_scale\"), py::arg(\"sample_num\"), py::arg(\"aligned\"),\n py::arg(\"clockwise\"));\n m.def(\"roi_align_rotated_backward\", &roi_align_rotated_backward,\n \"roi_align_rotated backward\", py::arg(\"rois\"), py::arg(\"grad_input\"),\n py::arg(\"grad_output\"), py::arg(\"pooled_height\"),\n py::arg(\"pooled_width\"), py::arg(\"spatial_scale\"),\n py::arg(\"sample_num\"), py::arg(\"aligned\"), py::arg(\"clockwise\"));\n m.def(\"dynamic_point_to_voxel_forward\", &dynamic_point_to_voxel_forward,\n \"dynamic_point_to_voxel_forward\", py::arg(\"feats\"), py::arg(\"coors\"),\n py::arg(\"reduce_type\"));\n m.def(\"dynamic_point_to_voxel_backward\", &dynamic_point_to_voxel_backward,\n \"dynamic_point_to_voxel_backward\", py::arg(\"grad_feats\"),\n py::arg(\"grad_reduced_feats\"), py::arg(\"feats\"),\n py::arg(\"reduced_feats\"), py::arg(\"coors_idx\"), py::arg(\"reduce_count\"),\n py::arg(\"reduce_type\"));\n m.def(\"hard_voxelize_forward\", &hard_voxelize_forward,\n \"hard_voxelize_forward\", py::arg(\"points\"), py::arg(\"voxel_size\"),\n py::arg(\"coors_range\"), py::arg(\"voxels\"), py::arg(\"coors\"),\n py::arg(\"num_points_per_voxel\"), py::arg(\"voxel_num\"),\n py::arg(\"max_points\"), py::arg(\"max_voxels\"), py::arg(\"NDim\"));\n m.def(\"dynamic_voxelize_forward\", &dynamic_voxelize_forward,\n \"dynamic_voxelize_forward\", py::arg(\"points\"), py::arg(\"voxel_size\"),\n py::arg(\"coors_range\"), py::arg(\"coors\"), py::arg(\"NDim\"));\n m.def(\"ms_deform_attn_forward\", &ms_deform_attn_forward,\n \"forward function of multi-scale deformable attention\",\n py::arg(\"value\"), py::arg(\"value_spatial_shapes\"),\n py::arg(\"value_level_start_index\"), py::arg(\"sampling_locations\"),\n py::arg(\"attention_weights\"), py::arg(\"im2col_step\"));\n m.def(\"ms_deform_attn_backward\", &ms_deform_attn_backward,\n \"backward function of multi-scale deformable attention\",\n py::arg(\"value\"), py::arg(\"value_spatial_shapes\"),\n py::arg(\"value_level_start_index\"), py::arg(\"sampling_locations\"),\n py::arg(\"attention_weights\"), py::arg(\"grad_output\"),\n py::arg(\"grad_value\"), py::arg(\"grad_sampling_loc\"),\n py::arg(\"grad_attn_weight\"), py::arg(\"im2col_step\"));\n m.def(\"border_align_forward\", &border_align_forward,\n \"forward function of border_align\", py::arg(\"input\"), py::arg(\"boxes\"),\n py::arg(\"output\"), py::arg(\"argmax_idx\"), py::arg(\"pool_size\"));\n m.def(\"border_align_backward\", &border_align_backward,\n \"backward function of border_align\", py::arg(\"grad_output\"),\n py::arg(\"boxes\"), py::arg(\"argmax_idx\"), py::arg(\"grad_input\"),\n py::arg(\"pool_size\"));\n m.def(\"correlation_forward\", &correlation_forward, \"Correlation forward\",\n py::arg(\"input1\"), py::arg(\"input2\"), py::arg(\"output\"), py::arg(\"kH\"),\n py::arg(\"kW\"), py::arg(\"patchH\"), py::arg(\"patchW\"), py::arg(\"padH\"),\n py::arg(\"padW\"), py::arg(\"dilationH\"), py::arg(\"dilationW\"),\n py::arg(\"dilation_patchH\"), py::arg(\"dilation_patchW\"), py::arg(\"dH\"),\n py::arg(\"dW\"));\n m.def(\"correlation_backward\", &correlation_backward, \"Correlation backward\",\n py::arg(\"grad_output\"), py::arg(\"input1\"), py::arg(\"input2\"),\n py::arg(\"grad_input1\"), py::arg(\"grad_input2\"), py::arg(\"kH\"),\n py::arg(\"kW\"), py::arg(\"patchH\"), py::arg(\"patchW\"), py::arg(\"padH\"),\n py::arg(\"padW\"), py::arg(\"dilationH\"), py::arg(\"dilationW\"),\n py::arg(\"dilation_patchH\"), py::arg(\"dilation_patchW\"), py::arg(\"dH\"),\n py::arg(\"dW\"));\n m.def(\"points_in_boxes_cpu_forward\", &points_in_boxes_cpu_forward,\n \"points_in_boxes_cpu_forward\", py::arg(\"boxes_tensor\"),\n py::arg(\"pts_tensor\"), py::arg(\"pts_indices_tensor\"));\n m.def(\"points_in_boxes_part_forward\", &points_in_boxes_part_forward,\n \"points_in_boxes_part_forward\", py::arg(\"boxes_tensor\"),\n py::arg(\"pts_tensor\"), py::arg(\"box_idx_of_points_tensor\"));\n m.def(\"points_in_boxes_all_forward\", &points_in_boxes_all_forward,\n \"points_in_boxes_all_forward\", py::arg(\"boxes_tensor\"),\n py::arg(\"pts_tensor\"), py::arg(\"box_idx_of_points_tensor\"));\n m.def(\"roiaware_pool3d_forward\", &roiaware_pool3d_forward,\n \"roiaware_pool3d_forward\", py::arg(\"rois\"), py::arg(\"pts\"),\n py::arg(\"pts_feature\"), py::arg(\"argmax\"), py::arg(\"pts_idx_of_voxels\"),\n py::arg(\"pooled_features\"), py::arg(\"pool_method\"));\n m.def(\"roiaware_pool3d_backward\", &roiaware_pool3d_backward,\n \"roiaware_pool3d_backward\", py::arg(\"pts_idx_of_voxels\"),\n py::arg(\"argmax\"), py::arg(\"grad_out\"), py::arg(\"grad_in\"),\n py::arg(\"pool_method\"));\n m.def(\"rotated_feature_align_forward\", &rotated_feature_align_forward,\n \"Feature Refine forward (CUDA)\", py::arg(\"features\"),\n py::arg(\"best_bboxes\"), py::arg(\"output\"), py::arg(\"spatial_scale\"),\n py::arg(\"points\"));\n m.def(\"rotated_feature_align_backward\", &rotated_feature_align_backward,\n \"Feature Refine backward (CUDA)\", py::arg(\"top_grad\"),\n py::arg(\"best_bboxes\"), py::arg(\"bottom_grad\"),\n py::arg(\"spatial_scale\"), py::arg(\"points\"));\n m.def(\"riroi_align_rotated_forward\", &riroi_align_rotated_forward,\n \"riroi_align_rotated forward\", py::arg(\"features\"), py::arg(\"rois\"),\n py::arg(\"output\"), py::arg(\"pooled_height\"), py::arg(\"pooled_width\"),\n py::arg(\"spatial_scale\"), py::arg(\"num_samples\"),\n py::arg(\"num_orientations\"), py::arg(\"clockwise\"));\n m.def(\"riroi_align_rotated_backward\", &riroi_align_rotated_backward,\n \"riroi_align_rotated backward\", py::arg(\"top_grad\"), py::arg(\"rois\"),\n py::arg(\"bottom_grad\"), py::arg(\"pooled_height\"),\n py::arg(\"pooled_width\"), py::arg(\"spatial_scale\"),\n py::arg(\"num_samples\"), py::arg(\"num_orientations\"),\n py::arg(\"clockwise\"));\n m.def(\"points_in_polygons_forward\", &points_in_polygons_forward,\n \"points_in_polygons_forward\", py::arg(\"points\"), py::arg(\"polygons\"),\n py::arg(\"output\"));\n m.def(\"min_area_polygons\", &min_area_polygons, \"min_area_polygons\",\n py::arg(\"pointsets\"), py::arg(\"polygons\"));\n m.def(\"active_rotated_filter_forward\", &active_rotated_filter_forward,\n \"active_rotated_filter_forward\", py::arg(\"input\"), py::arg(\"indices\"),\n py::arg(\"output\"));\n m.def(\"active_rotated_filter_backward\", &active_rotated_filter_backward,\n \"active_rotated_filter_backward\", py::arg(\"grad_out\"),\n py::arg(\"indices\"), py::arg(\"grad_in\"));\n m.def(\"convex_iou\", &convex_iou, \"convex_iou\", py::arg(\"pointsets\"),\n py::arg(\"polygons\"), py::arg(\"ious\"));\n m.def(\"convex_giou\", &convex_giou, \"convex_giou\", py::arg(\"pointsets\"),\n py::arg(\"polygons\"), py::arg(\"output\"));\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/riroi_align_rotated.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid riroi_align_rotated_forward_impl(Tensor features, Tensor rois,\n Tensor output, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n DISPATCH_DEVICE_IMPL(riroi_align_rotated_forward_impl, features, rois, output,\n pooled_height, pooled_width, spatial_scale, num_samples,\n num_orientations, clockwise);\n}\n\nvoid riroi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n DISPATCH_DEVICE_IMPL(riroi_align_rotated_backward_impl, top_grad, rois,\n bottom_grad, pooled_height, pooled_width, spatial_scale,\n num_samples, num_orientations, clockwise);\n}\n\nvoid riroi_align_rotated_forward(Tensor features, Tensor rois, Tensor output,\n int pooled_height, int pooled_width,\n float spatial_scale, int num_samples,\n int num_orientations, bool clockwise) {\n riroi_align_rotated_forward_impl(features, rois, output, pooled_height,\n pooled_width, spatial_scale, num_samples,\n num_orientations, clockwise);\n}\n\nvoid riroi_align_rotated_backward(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int pooled_height,\n int pooled_width, float spatial_scale,\n int num_samples, int num_orientations,\n bool clockwise) {\n riroi_align_rotated_backward_impl(top_grad, rois, bottom_grad, pooled_height,\n pooled_width, spatial_scale, num_samples,\n num_orientations, clockwise);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/roi_align.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roi_align_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n DISPATCH_DEVICE_IMPL(roi_align_forward_impl, input, rois, output, argmax_y,\n argmax_x, aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n int pool_mode, bool aligned) {\n DISPATCH_DEVICE_IMPL(roi_align_backward_impl, grad_output, rois, argmax_y,\n argmax_x, grad_input, aligned_height, aligned_width,\n spatial_scale, sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_forward(Tensor input, Tensor rois, Tensor output,\n Tensor argmax_y, Tensor argmax_x, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned) {\n roi_align_forward_impl(input, rois, output, argmax_y, argmax_x,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n\nvoid roi_align_backward(Tensor grad_output, Tensor rois, Tensor argmax_y,\n Tensor argmax_x, Tensor grad_input, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned) {\n roi_align_backward_impl(grad_output, rois, argmax_y, argmax_x, grad_input,\n aligned_height, aligned_width, spatial_scale,\n sampling_ratio, pool_mode, aligned);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/roi_align_rotated.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roi_align_rotated_forward_impl(Tensor features, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sample_ratio,\n bool aligned, bool clockwise) {\n DISPATCH_DEVICE_IMPL(roi_align_rotated_forward_impl, features, rois, output,\n aligned_height, aligned_width, spatial_scale,\n sample_ratio, aligned, clockwise);\n}\n\nvoid roi_align_rotated_backward_impl(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sample_ratio, bool aligned,\n bool clockwise) {\n DISPATCH_DEVICE_IMPL(roi_align_rotated_backward_impl, top_grad, rois,\n bottom_grad, aligned_height, aligned_width,\n spatial_scale, sample_ratio, aligned, clockwise);\n}\n\nvoid roi_align_rotated_forward(Tensor input, Tensor rois, Tensor output,\n int aligned_height, int aligned_width,\n float spatial_scale, int sampling_ratio,\n bool aligned, bool clockwise) {\n roi_align_rotated_forward_impl(input, rois, output, aligned_height,\n aligned_width, spatial_scale, sampling_ratio,\n aligned, clockwise);\n}\n\nvoid roi_align_rotated_backward(Tensor top_grad, Tensor rois,\n Tensor bottom_grad, int aligned_height,\n int aligned_width, float spatial_scale,\n int sampling_ratio, bool aligned,\n bool clockwise) {\n roi_align_rotated_backward_impl(top_grad, rois, bottom_grad, aligned_height,\n aligned_width, spatial_scale, sampling_ratio,\n aligned, clockwise);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/roi_pool.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roi_pool_forward_impl(Tensor input, Tensor rois, Tensor output,\n Tensor argmax, int pooled_height, int pooled_width,\n float spatial_scale) {\n DISPATCH_DEVICE_IMPL(roi_pool_forward_impl, input, rois, output, argmax,\n pooled_height, pooled_width, spatial_scale);\n}\n\nvoid roi_pool_backward_impl(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height,\n int pooled_width, float spatial_scale) {\n DISPATCH_DEVICE_IMPL(roi_pool_backward_impl, grad_output, rois, argmax,\n grad_input, pooled_height, pooled_width, spatial_scale);\n}\n\nvoid roi_pool_forward(Tensor input, Tensor rois, Tensor output, Tensor argmax,\n int pooled_height, int pooled_width,\n float spatial_scale) {\n roi_pool_forward_impl(input, rois, output, argmax, pooled_height,\n pooled_width, spatial_scale);\n}\n\nvoid roi_pool_backward(Tensor grad_output, Tensor rois, Tensor argmax,\n Tensor grad_input, int pooled_height, int pooled_width,\n float spatial_scale) {\n roi_pool_backward_impl(grad_output, rois, argmax, grad_input, pooled_height,\n pooled_width, spatial_scale);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/roiaware_pool3d.cpp", "content": "#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roiaware_pool3d_forward_impl(int boxes_num, int pts_num, int channels,\n int max_pts_each_voxel, int out_x, int out_y,\n int out_z, const Tensor rois,\n const Tensor pts, const Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method) {\n DISPATCH_DEVICE_IMPL(roiaware_pool3d_forward_impl, boxes_num, pts_num,\n channels, max_pts_each_voxel, out_x, out_y, out_z, rois,\n pts, pts_feature, argmax, pts_idx_of_voxels,\n pooled_features, pool_method);\n}\n\nvoid roiaware_pool3d_backward_impl(int boxes_num, int out_x, int out_y,\n int out_z, int channels,\n int max_pts_each_voxel,\n const Tensor pts_idx_of_voxels,\n const Tensor argmax, const Tensor grad_out,\n Tensor grad_in, int pool_method) {\n DISPATCH_DEVICE_IMPL(roiaware_pool3d_backward_impl, boxes_num, out_x, out_y,\n out_z, channels, max_pts_each_voxel, pts_idx_of_voxels,\n argmax, grad_out, grad_in, pool_method);\n}\n\nvoid roiaware_pool3d_forward(Tensor rois, Tensor pts, Tensor pts_feature,\n Tensor argmax, Tensor pts_idx_of_voxels,\n Tensor pooled_features, int pool_method) {\n // params rois: (N, 7) [x, y, z, x_size, y_size, z_size, ry] in LiDAR\n // coordinate\n // params pts: (npoints, 3) [x, y, z] in LiDAR coordinate\n // params pts_feature: (npoints, C)\n // params argmax: (N, out_x, out_y, out_z, C)\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel)\n // params pooled_features: (N, out_x, out_y, out_z, C)\n // params pool_method: 0: max_pool 1: avg_pool\n int boxes_num = rois.size(0);\n int pts_num = pts.size(0);\n int channels = pts_feature.size(1);\n int max_pts_each_voxel = pts_idx_of_voxels.size(4); // index 0 is the counter\n int out_x = pts_idx_of_voxels.size(1);\n int out_y = pts_idx_of_voxels.size(2);\n int out_z = pts_idx_of_voxels.size(3);\n assert((out_x < 256) && (out_y < 256) &&\n (out_z < 256)); // we encode index with 8bit\n\n roiaware_pool3d_forward_impl(boxes_num, pts_num, channels, max_pts_each_voxel,\n out_x, out_y, out_z, rois, pts, pts_feature,\n argmax, pts_idx_of_voxels, pooled_features,\n pool_method);\n}\n\nvoid roiaware_pool3d_backward(Tensor pts_idx_of_voxels, Tensor argmax,\n Tensor grad_out, Tensor grad_in,\n int pool_method) {\n // params pts_idx_of_voxels: (N, out_x, out_y, out_z, max_pts_each_voxel)\n // params argmax: (N, out_x, out_y, out_z, C)\n // params grad_out: (N, out_x, out_y, out_z, C)\n // params grad_in: (npoints, C), return value\n // params pool_method: 0: max_pool 1: avg_pool\n int boxes_num = pts_idx_of_voxels.size(0);\n int out_x = pts_idx_of_voxels.size(1);\n int out_y = pts_idx_of_voxels.size(2);\n int out_z = pts_idx_of_voxels.size(3);\n int max_pts_each_voxel = pts_idx_of_voxels.size(4); // index 0 is the counter\n int channels = grad_out.size(4);\n\n roiaware_pool3d_backward_impl(boxes_num, out_x, out_y, out_z, channels,\n max_pts_each_voxel, pts_idx_of_voxels, argmax,\n grad_out, grad_in, pool_method);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/roipoint_pool3d.cpp", "content": "/*\nModified from\nhttps://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/ops/roipoint_pool3d/src/roipoint_pool3d.cpp\nPoint cloud feature pooling\nWritten by Shaoshuai Shi\nAll Rights Reserved 2018.\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid roipoint_pool3d_forward_impl(int batch_size, int pts_num, int boxes_num,\n int feature_in_len, int sampled_pts_num,\n const Tensor xyz, const Tensor boxes3d,\n const Tensor pts_feature,\n Tensor pooled_features,\n Tensor pooled_empty_flag) {\n DISPATCH_DEVICE_IMPL(roipoint_pool3d_forward_impl, batch_size, pts_num,\n boxes_num, feature_in_len, sampled_pts_num, xyz, boxes3d,\n pts_feature, pooled_features, pooled_empty_flag);\n}\n\nvoid roipoint_pool3d_forward(Tensor xyz, Tensor boxes3d, Tensor pts_feature,\n Tensor pooled_features, Tensor pooled_empty_flag) {\n // params xyz: (B, N, 3)\n // params boxes3d: (B, M, 7)\n // params pts_feature: (B, N, C)\n // params pooled_features: (B, M, 512, 3+C)\n // params pooled_empty_flag: (B, M)\n int batch_size = xyz.size(0);\n int pts_num = xyz.size(1);\n int boxes_num = boxes3d.size(1);\n int feature_in_len = pts_feature.size(2);\n int sampled_pts_num = pooled_features.size(2);\n\n roipoint_pool3d_forward_impl(batch_size, pts_num, boxes_num, feature_in_len,\n sampled_pts_num, xyz, boxes3d, pts_feature,\n pooled_features, pooled_empty_flag);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/rotated_feature_align.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n// Modified from\n// https://github.com/SJTU-Thinklab-Det/r3det-on-mmdetection/blob/master/mmdet/ops/fr/src/feature_refine_cuda.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid rotated_feature_align_forward_impl(const Tensor features,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor output) {\n DISPATCH_DEVICE_IMPL(rotated_feature_align_forward_impl, features,\n best_bboxes, spatial_scale, points, output);\n}\n\nvoid rotated_feature_align_backward_impl(const Tensor top_grad,\n const Tensor best_bboxes,\n const float spatial_scale,\n const int points, Tensor bottom_grad) {\n DISPATCH_DEVICE_IMPL(rotated_feature_align_backward_impl, top_grad,\n best_bboxes, spatial_scale, points, bottom_grad);\n}\n\nvoid rotated_feature_align_forward(const Tensor features,\n const Tensor best_bboxes, Tensor output,\n const float spatial_scale,\n const int points) {\n rotated_feature_align_forward_impl(features, best_bboxes, spatial_scale,\n points, output);\n}\n\nvoid rotated_feature_align_backward(const Tensor top_grad,\n const Tensor best_bboxes,\n Tensor bottom_grad,\n const float spatial_scale,\n const int points) {\n rotated_feature_align_backward_impl(top_grad, best_bboxes, spatial_scale,\n points, bottom_grad);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/scatter_points.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntypedef enum { SUM = 0, MEAN = 1, MAX = 2 } reduce_t;\n\nstd::vector dynamic_point_to_voxel_forward_impl(\n const torch::Tensor &feats, const torch::Tensor &coors,\n const reduce_t reduce_type) {\n return DISPATCH_DEVICE_IMPL(dynamic_point_to_voxel_forward_impl, feats, coors,\n reduce_type);\n}\n\nvoid dynamic_point_to_voxel_backward_impl(\n torch::Tensor &grad_feats, const torch::Tensor &grad_reduced_feats,\n const torch::Tensor &feats, const torch::Tensor &reduced_feats,\n const torch::Tensor &coors_idx, const torch::Tensor &reduce_count,\n const reduce_t reduce_type) {\n DISPATCH_DEVICE_IMPL(dynamic_point_to_voxel_backward_impl, grad_feats,\n grad_reduced_feats, feats, reduced_feats, coors_idx,\n reduce_count, reduce_type);\n}\n\ninline reduce_t convert_reduce_type(const std::string &reduce_type) {\n if (reduce_type == \"max\")\n return reduce_t::MAX;\n else if (reduce_type == \"sum\")\n return reduce_t::SUM;\n else if (reduce_type == \"mean\")\n return reduce_t::MEAN;\n else\n TORCH_CHECK(false, \"do not support reduce type \" + reduce_type)\n return reduce_t::SUM;\n}\n\nstd::vector dynamic_point_to_voxel_forward(\n const torch::Tensor &feats, const torch::Tensor &coors,\n const std::string &reduce_type) {\n return dynamic_point_to_voxel_forward_impl(feats, coors,\n convert_reduce_type(reduce_type));\n}\n\nvoid dynamic_point_to_voxel_backward(torch::Tensor &grad_feats,\n const torch::Tensor &grad_reduced_feats,\n const torch::Tensor &feats,\n const torch::Tensor &reduced_feats,\n const torch::Tensor &coors_idx,\n const torch::Tensor &reduce_count,\n const std::string &reduce_type) {\n dynamic_point_to_voxel_backward_impl(grad_feats, grad_reduced_feats, feats,\n reduced_feats, coors_idx, reduce_count,\n convert_reduce_type(reduce_type));\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/sync_bn.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid sync_bn_forward_mean_impl(const Tensor input, Tensor mean) {\n DISPATCH_DEVICE_IMPL(sync_bn_forward_mean_impl, input, mean);\n}\n\nvoid sync_bn_forward_var_impl(const Tensor input, const Tensor mean,\n Tensor var) {\n DISPATCH_DEVICE_IMPL(sync_bn_forward_var_impl, input, mean, var);\n}\n\nvoid sync_bn_forward_output_impl(const Tensor input, const Tensor mean,\n const Tensor var, Tensor running_mean,\n Tensor running_var, const Tensor weight,\n const Tensor bias, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size) {\n DISPATCH_DEVICE_IMPL(sync_bn_forward_output_impl, input, mean, var,\n running_mean, running_var, weight, bias, norm, std,\n output, eps, momentum, group_size);\n}\n\nvoid sync_bn_backward_param_impl(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias) {\n DISPATCH_DEVICE_IMPL(sync_bn_backward_param_impl, grad_output, norm,\n grad_weight, grad_bias);\n}\n\nvoid sync_bn_backward_data_impl(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight,\n const Tensor grad_bias, const Tensor norm,\n const Tensor std, Tensor grad_input) {\n DISPATCH_DEVICE_IMPL(sync_bn_backward_data_impl, grad_output, weight,\n grad_weight, grad_bias, norm, std, grad_input);\n}\n\nvoid sync_bn_forward_mean(const Tensor input, Tensor mean) {\n sync_bn_forward_mean_impl(input, mean);\n}\n\nvoid sync_bn_forward_var(const Tensor input, const Tensor mean, Tensor var) {\n sync_bn_forward_var_impl(input, mean, var);\n}\n\nvoid sync_bn_forward_output(const Tensor input, const Tensor mean,\n const Tensor var, const Tensor weight,\n const Tensor bias, Tensor running_mean,\n Tensor running_var, Tensor norm, Tensor std,\n Tensor output, float eps, float momentum,\n int group_size) {\n sync_bn_forward_output_impl(input, mean, var, running_mean, running_var,\n weight, bias, norm, std, output, eps, momentum,\n group_size);\n}\n\nvoid sync_bn_backward_param(const Tensor grad_output, const Tensor norm,\n Tensor grad_weight, Tensor grad_bias) {\n sync_bn_backward_param_impl(grad_output, norm, grad_weight, grad_bias);\n}\n\nvoid sync_bn_backward_data(const Tensor grad_output, const Tensor weight,\n const Tensor grad_weight, const Tensor grad_bias,\n const Tensor norm, const Tensor std,\n Tensor grad_input) {\n sync_bn_backward_data_impl(grad_output, weight, grad_weight, grad_bias, norm,\n std, grad_input);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/three_interpolate.cpp", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/interpolate.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid three_interpolate_forward_impl(int b, int c, int m, int n,\n const Tensor points, const Tensor idx,\n const Tensor weight, Tensor out) {\n DISPATCH_DEVICE_IMPL(three_interpolate_forward_impl, b, c, m, n, points, idx,\n weight, out);\n}\n\nvoid three_interpolate_backward_impl(int b, int c, int n, int m,\n const Tensor grad_out, const Tensor idx,\n const Tensor weight, Tensor grad_points) {\n DISPATCH_DEVICE_IMPL(three_interpolate_backward_impl, b, c, n, m, grad_out,\n idx, weight, grad_points);\n}\n\nvoid three_interpolate_forward(Tensor points_tensor, Tensor idx_tensor,\n Tensor weight_tensor, Tensor out_tensor, int b,\n int c, int m, int n) {\n three_interpolate_forward_impl(b, c, m, n, points_tensor, idx_tensor,\n weight_tensor, out_tensor);\n}\n\nvoid three_interpolate_backward(Tensor grad_out_tensor, Tensor idx_tensor,\n Tensor weight_tensor, Tensor grad_points_tensor,\n int b, int c, int n, int m) {\n three_interpolate_backward_impl(b, c, n, m, grad_out_tensor, idx_tensor,\n weight_tensor, grad_points_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/three_nn.cpp", "content": "// Modified from\n// https://github.com/sshaoshuai/Pointnet2.PyTorch/tree/master/pointnet2/src/interpolate.cpp\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid three_nn_forward_impl(int b, int n, int m, const Tensor unknown,\n const Tensor known, Tensor dist2, Tensor idx) {\n DISPATCH_DEVICE_IMPL(three_nn_forward_impl, b, n, m, unknown, known, dist2,\n idx);\n}\n\nvoid three_nn_forward(Tensor unknown_tensor, Tensor known_tensor,\n Tensor dist2_tensor, Tensor idx_tensor, int b, int n,\n int m) {\n three_nn_forward_impl(b, n, m, unknown_tensor, known_tensor, dist2_tensor,\n idx_tensor);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/tin_shift.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nvoid tin_shift_forward_impl(Tensor input, Tensor shift, Tensor output) {\n DISPATCH_DEVICE_IMPL(tin_shift_forward_impl, input, shift, output);\n}\n\nvoid tin_shift_backward_impl(Tensor grad_output, Tensor shift,\n Tensor grad_input) {\n DISPATCH_DEVICE_IMPL(tin_shift_backward_impl, grad_output, shift, grad_input);\n}\n\nvoid tin_shift_forward(Tensor input, Tensor shift, Tensor output) {\n tin_shift_forward_impl(input, shift, output);\n}\n\nvoid tin_shift_backward(Tensor grad_output, Tensor shift, Tensor grad_input) {\n tin_shift_backward_impl(grad_output, shift, grad_input);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/upfirdn2d.cpp", "content": "// Modified from\n// https://github.com/rosinality/stylegan2-pytorch/blob/master/op/upfirdn2d.cpp\n\n/*\nCopyright (c) 2021, NVIDIA Corporation. All rights reserved.\n\nNVIDIA Source Code License for StyleGAN2 with Adaptive Discriminator\nAugmentation (ADA)\n=======================================================================\n\n1. Definitions\n\n\"Licensor\" means any person or entity that distributes its Work.\n\n\"Software\" means the original work of authorship made available under\nthis License.\n\n\"Work\" means the Software and any additions to or derivative works of\nthe Software that are made available under this License.\n\nThe terms \"reproduce,\" \"reproduction,\" \"derivative works,\" and\n\"distribution\" have the meaning as provided under U.S. copyright law;\nprovided, however, that for the purposes of this License, derivative\nworks shall not include works that remain separable from, or merely\nlink (or bind by name) to the interfaces of, the Work.\n\nWorks, including the Software, are \"made available\" under this License\nby including in or with the Work either (a) a copyright notice\nreferencing the applicability of this License to the Work, or (b) a\ncopy of this License.\n\n2. License Grants\n\n 2.1 Copyright Grant. Subject to the terms and conditions of this\n License, each Licensor grants to you a perpetual, worldwide,\n non-exclusive, royalty-free, copyright license to reproduce,\n prepare derivative works of, publicly display, publicly perform,\n sublicense and distribute its Work and any resulting derivative\n works in any form.\n\n3. Limitations\n\n 3.1 Redistribution. You may reproduce or distribute the Work only\n if (a) you do so under this License, (b) you include a complete\n copy of this License with your distribution, and (c) you retain\n without modification any copyright, patent, trademark, or\n attribution notices that are present in the Work.\n\n 3.2 Derivative Works. You may specify that additional or different\n terms apply to the use, reproduction, and distribution of your\n derivative works of the Work (\"Your Terms\") only if (a) Your Terms\n provide that the use limitation in Section 3.3 applies to your\n derivative works, and (b) you identify the specific derivative\n works that are subject to Your Terms. Notwithstanding Your Terms,\n this License (including the redistribution requirements in Section\n 3.1) will continue to apply to the Work itself.\n\n 3.3 Use Limitation. The Work and any derivative works thereof only\n may be used or intended for use non-commercially. Notwithstanding\n the foregoing, NVIDIA and its affiliates may use the Work and any\n derivative works commercially. As used herein, \"non-commercially\"\n means for research or evaluation purposes only.\n\n 3.4 Patent Claims. If you bring or threaten to bring a patent claim\n against any Licensor (including any claim, cross-claim or\n counterclaim in a lawsuit) to enforce any patents that you allege\n are infringed by any Work, then your rights under this License from\n such Licensor (including the grant in Section 2.1) will terminate\n immediately.\n\n 3.5 Trademarks. This License does not grant any rights to use any\n Licensor’s or its affiliates’ names, logos, or trademarks, except\n as necessary to reproduce the notices described in this License.\n\n 3.6 Termination. If you violate any term of this License, then your\n rights under this License (including the grant in Section 2.1) will\n terminate immediately.\n\n4. Disclaimer of Warranty.\n\nTHE WORK IS PROVIDED \"AS IS\" WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR\nNON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER\nTHIS LICENSE.\n\n5. Limitation of Liability.\n\nEXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL\nTHEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE\nSHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT,\nINDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF\nOR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK\n(INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION,\nLOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER\nCOMMERCIAL DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF\nTHE POSSIBILITY OF SUCH DAMAGES.\n\n=======================================================================\n*/\n\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\ntorch::Tensor upfirdn2d_op_impl(const torch::Tensor& input,\n const torch::Tensor& kernel, int up_x, int up_y,\n int down_x, int down_y, int pad_x0, int pad_x1,\n int pad_y0, int pad_y1) {\n return DISPATCH_DEVICE_IMPL(upfirdn2d_op_impl, input, kernel, up_x, up_y,\n down_x, down_y, pad_x0, pad_x1, pad_y0, pad_y1);\n}\n\ntorch::Tensor upfirdn2d(const torch::Tensor& input, const torch::Tensor& kernel,\n int up_x, int up_y, int down_x, int down_y, int pad_x0,\n int pad_x1, int pad_y0, int pad_y1) {\n return upfirdn2d_op_impl(input, kernel, up_x, up_y, down_x, down_y, pad_x0,\n pad_x1, pad_y0, pad_y1);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/pytorch/voxelization.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved.\n#include \"pytorch_cpp_helper.hpp\"\n#include \"pytorch_device_registry.hpp\"\n\nint hard_voxelize_forward_impl(const at::Tensor &points, at::Tensor &voxels,\n at::Tensor &coors,\n at::Tensor &num_points_per_voxel,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int max_points, const int max_voxels,\n const int NDim = 3) {\n return DISPATCH_DEVICE_IMPL(hard_voxelize_forward_impl, points, voxels, coors,\n num_points_per_voxel, voxel_size, coors_range,\n max_points, max_voxels, NDim);\n}\n\nvoid dynamic_voxelize_forward_impl(const at::Tensor &points, at::Tensor &coors,\n const std::vector voxel_size,\n const std::vector coors_range,\n const int NDim = 3) {\n DISPATCH_DEVICE_IMPL(dynamic_voxelize_forward_impl, points, coors, voxel_size,\n coors_range, NDim);\n}\n\nvoid hard_voxelize_forward(const at::Tensor &points,\n const at::Tensor &voxel_size,\n const at::Tensor &coors_range, at::Tensor &voxels,\n at::Tensor &coors, at::Tensor &num_points_per_voxel,\n at::Tensor &voxel_num, const int max_points,\n const int max_voxels, const int NDim = 3) {\n int64_t *voxel_num_data = voxel_num.data_ptr();\n std::vector voxel_size_v(\n voxel_size.data_ptr(),\n voxel_size.data_ptr() + voxel_size.numel());\n std::vector coors_range_v(\n coors_range.data_ptr(),\n coors_range.data_ptr() + coors_range.numel());\n\n *voxel_num_data = hard_voxelize_forward_impl(\n points, voxels, coors, num_points_per_voxel, voxel_size_v, coors_range_v,\n max_points, max_voxels, NDim);\n}\n\nvoid dynamic_voxelize_forward(const at::Tensor &points,\n const at::Tensor &voxel_size,\n const at::Tensor &coors_range, at::Tensor &coors,\n const int NDim = 3) {\n std::vector voxel_size_v(\n voxel_size.data_ptr(),\n voxel_size.data_ptr() + voxel_size.numel());\n std::vector coors_range_v(\n coors_range.data_ptr(),\n coors_range.data_ptr() + coors_range.numel());\n dynamic_voxelize_forward_impl(points, coors, voxel_size_v, coors_range_v,\n NDim);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_corner_pool.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_corner_pool.hpp\"\n\n#include \n\n#include \"trt_serialize.hpp\"\n\nvoid CornerPoolForwardLauncher_float(const float *input, float *output,\n const int batch_size, const int channels,\n const int height, const int width,\n const int pool_type, cudaStream_t stream);\n\nnamespace {\nstatic const char *PLUGIN_VERSION{\"1\"};\nstatic const char *CORNER_POOL_PLUGIN_NAME{\"MMCVCornerPool\"};\n} // namespace\n\nCornerPoolPluginDynamic::CornerPoolPluginDynamic(const std::string &name,\n TRT_CORNER_POOL_TYPE poolType)\n : mLayerName(name), mPoolType(poolType) {}\n\nCornerPoolPluginDynamic::CornerPoolPluginDynamic(const std::string name,\n const void *data,\n size_t length)\n : mLayerName(name) {\n deserialize_value(&data, &length, &mPoolType);\n}\n\nCornerPoolPluginDynamic::~CornerPoolPluginDynamic() {}\n\nnvinfer1::IPluginV2DynamicExt *CornerPoolPluginDynamic::clone() const {\n CornerPoolPluginDynamic *plugin =\n new CornerPoolPluginDynamic(mLayerName, mPoolType);\n plugin->setPluginNamespace(getPluginNamespace());\n\n return plugin;\n}\n\nnvinfer1::DimsExprs CornerPoolPluginDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) {\n return inputs[0];\n}\n\nbool CornerPoolPluginDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc *inOut, int nbInputs,\n int nbOutputs) {\n switch (pos) {\n // input[0]\n case 0:\n return inOut[pos].type == nvinfer1::DataType::kFLOAT &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR;\n // output[0]\n case 1:\n return inOut[pos].type == inOut[0].type &&\n inOut[pos].format == inOut[0].format;\n default:\n return false;\n }\n}\n\nvoid CornerPoolPluginDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *outputs, int nbOutputs) {}\n\nsize_t CornerPoolPluginDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs, int nbOutputs) const {\n int sizeof_dtype = mmcv::getElementSize(outputs[0].type);\n}\n\nint CornerPoolPluginDynamic::enqueue(\n const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc, const void *const *inputs,\n void *const *outputs, void *workSpace, cudaStream_t stream) {\n const void *input = inputs[0];\n void *output_value = outputs[0];\n\n const int batch_size = inputDesc[0].dims.d[0];\n const int channels = inputDesc[0].dims.d[1];\n const int height = inputDesc[0].dims.d[2];\n const int width = inputDesc[0].dims.d[3];\n\n CornerPoolForwardLauncher_float((float *)input, (float *)output_value,\n batch_size, channels, height, width,\n int(mPoolType), stream);\n\n return 0;\n}\n\nnvinfer1::DataType CornerPoolPluginDynamic::getOutputDataType(\n int index, const nvinfer1::DataType *inputTypes, int nbInputs) const {\n return inputTypes[0];\n}\n\n// IPluginV2 Methods\nconst char *CornerPoolPluginDynamic::getPluginType() const {\n switch (mPoolType) {\n case TRT_CORNER_POOL_TYPE::TRT_TOP_POOL:\n case TRT_CORNER_POOL_TYPE::TRT_BOTTOM_POOL:\n case TRT_CORNER_POOL_TYPE::TRT_LEFT_POOL:\n case TRT_CORNER_POOL_TYPE::TRT_RIGHT_POOL:\n return CORNER_POOL_PLUGIN_NAME;\n\n default:\n return \"UnknownpoolType\";\n }\n}\n\nconst char *CornerPoolPluginDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nint CornerPoolPluginDynamic::getNbOutputs() const { return 1; }\n\nint CornerPoolPluginDynamic::initialize() { return 0; }\n\nvoid CornerPoolPluginDynamic::terminate() {}\n\nsize_t CornerPoolPluginDynamic::getSerializationSize() const {\n return sizeof(mPoolType);\n}\n\nvoid CornerPoolPluginDynamic::serialize(void *buffer) const {\n serialize_value(&buffer, mPoolType);\n}\n\nvoid CornerPoolPluginDynamic::destroy() {\n // This gets called when the network containing plugin is destroyed\n delete this;\n}\n\nvoid CornerPoolPluginDynamic::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *CornerPoolPluginDynamic::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\nCornerPoolPluginDynamicCreator::CornerPoolPluginDynamicCreator() {\n mPluginAttributes.clear();\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"mode\"));\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char *CornerPoolPluginDynamicCreator::getPluginName() const {\n return CORNER_POOL_PLUGIN_NAME;\n}\n\nconst char *CornerPoolPluginDynamicCreator::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nconst nvinfer1::PluginFieldCollection *\nCornerPoolPluginDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nnvinfer1::IPluginV2 *CornerPoolPluginDynamicCreator::createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) {\n TRT_CORNER_POOL_TYPE poolType;\n int poolMode = -1;\n\n for (int i = 0; i < fc->nbFields; i++) {\n if (fc->fields[i].data == nullptr) {\n continue;\n }\n std::string field_name(fc->fields[i].name);\n\n if (field_name.compare(\"mode\") == 0) {\n poolMode = static_cast(fc->fields[i].data)[0];\n }\n }\n\n assert(poolMode >= 0 && poolMode <= 3);\n switch (poolMode) {\n case 0:\n poolType = TRT_CORNER_POOL_TYPE::TRT_TOP_POOL;\n break;\n case 1:\n poolType = TRT_CORNER_POOL_TYPE::TRT_BOTTOM_POOL;\n break;\n case 2:\n poolType = TRT_CORNER_POOL_TYPE::TRT_LEFT_POOL;\n break;\n case 3:\n poolType = TRT_CORNER_POOL_TYPE::TRT_RIGHT_POOL;\n break;\n\n default:\n break;\n }\n\n CornerPoolPluginDynamic *plugin = new CornerPoolPluginDynamic(name, poolType);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nnvinfer1::IPluginV2 *CornerPoolPluginDynamicCreator::deserializePlugin(\n const char *name, const void *serialData, size_t serialLength) {\n // This object will be deleted when the network is destroyed, which will\n // call FCPluginDynamic::destroy()\n auto plugin = new CornerPoolPluginDynamic(name, serialData, serialLength);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nvoid CornerPoolPluginDynamicCreator::setPluginNamespace(\n const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *CornerPoolPluginDynamicCreator::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_corner_pool_kernel.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"common_cuda_helper.hpp\"\n#include \"trt_cuda_helper.cuh\"\n#include \"trt_plugin_helper.hpp\"\n\ntemplate \n__global__ void top_bottom_pool_kernel(const scalar_t *input, scalar_t *output,\n const int batch_size, const int channels,\n const int height, const int width,\n const int pool_type) {\n const int nthreads = batch_size * channels * width;\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int n_idx = index / (channels * width); // batch\n int w_idx = index % width; // width\n int c_idx = (index / width) % channels; // channels\n int offset_n = n_idx * channels * width * height;\n int offset_n_c = offset_n + c_idx * width * height;\n int direction = -1; // in [-1, 1], default for TopPool\n int index_start = height - 2; // default for TopPool\n // pool_type in [0, 1]\n if (pool_type == 0) {\n // TopPool\n // directly copy the most bottom value from input to output\n output[offset_n_c + (height - 1) * width + w_idx] =\n input[offset_n_c + (height - 1) * width + w_idx];\n } else {\n // BottomPool\n // directly copy the most top value from input to output\n output[offset_n_c + w_idx] = input[offset_n_c + w_idx];\n index_start = 1;\n direction = 1;\n }\n // do pool\n for (int h = index_start; h >= 0 && h < height; h += direction) {\n output[offset_n_c + h * width + w_idx] =\n max(output[offset_n_c + (h - direction) * width + w_idx],\n input[offset_n_c + h * width + w_idx]);\n }\n }\n}\n\ntemplate \n__global__ void left_right_pool_kernel(const scalar_t *input, scalar_t *output,\n const int batch_size, const int channels,\n const int height, const int width,\n const int pool_type) {\n const int nthreads = batch_size * channels * height;\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n int n_idx = index / (channels * height); // batch\n int h_idx = index % height; // height\n int c_idx = (index / height) % channels; // channels\n int offset_n = n_idx * channels * width * height;\n int offset_n_c = offset_n + c_idx * width * height;\n int offset_n_c_h = offset_n_c + h_idx * width;\n int direction = -1; // in [-1, 1], default for LeftPool\n int index_start = width - 2; // default for LeftPool\n // pool_type in [2, 3]\n if (pool_type == 2) {\n // LeftPool\n // directly copy the most right value from input to output\n output[offset_n_c_h + width - 1] = input[offset_n_c_h + width - 1];\n } else {\n // RightPool\n // directly copy the most left value from input to output\n output[offset_n_c_h] = input[offset_n_c_h];\n index_start = 1;\n direction = 1;\n }\n // do pool\n for (int w = index_start; w >= 0 && w < width; w += direction) {\n output[offset_n_c_h + w] =\n max(output[offset_n_c_h + w - direction], input[offset_n_c_h + w]);\n }\n }\n}\n\ntemplate \nvoid CornerPoolForwardLauncher(const scalar_t *input, scalar_t *output,\n const int batch_size, const int channels,\n const int height, const int width,\n const int pool_type, cudaStream_t stream) {\n int nthreads = -1, col_block = -1;\n\n switch (pool_type) {\n case 0:\n case 1:\n nthreads = batch_size * channels * width;\n col_block = GET_BLOCKS(nthreads, THREADS_PER_BLOCK);\n top_bottom_pool_kernel\n <<>>(\n input, output, batch_size, channels, height, width, pool_type);\n break;\n case 2:\n case 3:\n nthreads = batch_size * channels * height;\n col_block = GET_BLOCKS(nthreads, THREADS_PER_BLOCK);\n left_right_pool_kernel\n <<>>(\n input, output, batch_size, channels, height, width, pool_type);\n break;\n }\n}\n\nvoid CornerPoolForwardLauncher_float(const float *input, float *output,\n const int batch_size, const int channels,\n const int height, const int width,\n const int pool_type, cudaStream_t stream) {\n CornerPoolForwardLauncher(input, output, batch_size, channels, height,\n width, pool_type, stream);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_cuda_helper.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n\n#include \"common_cuda_helper.hpp\"\n#include \"trt_cuda_helper.cuh\"\n#include \"trt_plugin_helper.hpp\"\n\nusing mmcv::TensorDesc;\n\ntemplate \n__global__ void copy_permute_kernel(scalar_t *dst, const scalar_t *src, int n,\n TensorDesc ts_src_stride,\n TensorDesc ts_dst_stride,\n TensorDesc ts_permute) {\n const int src_dim = ts_src_stride.dim;\n int *src_stride = &(ts_src_stride.stride[0]);\n int *dst_stride = &(ts_dst_stride.stride[0]);\n int *permute = &(ts_permute.shape[0]);\n CUDA_1D_KERNEL_LOOP(index, n) {\n size_t dst_index = index;\n size_t src_index = 0;\n for (int i = 0; i < src_dim; ++i) {\n int dim_index = dst_index / dst_stride[i];\n dst_index = dst_index % dst_stride[i];\n src_index += dim_index * src_stride[permute[i]];\n }\n dst[index] = src[src_index];\n }\n}\n\ntemplate \nvoid memcpyPermute(scalar_t *dst, const scalar_t *src, int *src_size,\n int *permute, int src_dim, cudaStream_t stream) {\n size_t copy_size = 1;\n TensorDesc ts_permute;\n memcpy(&(ts_permute.shape[0]), permute, src_dim * sizeof(int));\n\n TensorDesc ts_src_stride;\n TensorDesc ts_dst_stride;\n ts_src_stride.dim = src_dim;\n ts_dst_stride.dim = src_dim;\n int *src_stride = &(ts_src_stride.stride[0]);\n int *dst_stride = &(ts_dst_stride.stride[0]);\n int *dst_size = &(ts_dst_stride.shape[0]);\n src_stride[src_dim - 1] = 1;\n dst_stride[src_dim - 1] = 1;\n\n for (int i = src_dim - 1; i >= 0; --i) {\n dst_size[i] = src_size[permute[i]];\n if (i < src_dim - 1) {\n src_stride[i] = src_stride[i + 1] * src_size[i + 1];\n }\n }\n\n for (int i = src_dim - 1; i >= 0; --i) {\n copy_size *= dst_size[i];\n if (i < src_dim - 1) {\n dst_stride[i] = dst_stride[i + 1] * dst_size[i + 1];\n }\n }\n\n copy_permute_kernel\n <<>>(\n dst, src, copy_size, ts_src_stride, ts_dst_stride, ts_permute);\n}\n\ntemplate void memcpyPermute(float *dst, const float *src, int *src_size,\n int *permute, int src_dim,\n cudaStream_t stream);\n\ntemplate <>\ncublasStatus_t cublasGemmWrap(cublasHandle_t handle,\n cublasOperation_t transa,\n cublasOperation_t transb, int m, int n,\n int k, const float *alpha, const float *A,\n int lda, const float *B, int ldb,\n const float *beta, float *C, int ldc) {\n return cublasSgemm(handle, transa, transb, m, n, k, alpha, A, lda, B, ldb,\n beta, C, ldc);\n}\n\ntemplate <>\ncublasStatus_t cublasGemmWrap(cublasHandle_t handle,\n cublasOperation_t transa,\n cublasOperation_t transb, int m, int n,\n int k, const half *alpha, const half *A,\n int lda, const half *B, int ldb,\n const half *beta, half *C, int ldc) {\n return cublasHgemm(handle, transa, transb, m, n, k, alpha, A, lda, B, ldb,\n beta, C, ldc);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_cummaxmin.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_cummaxmin.hpp\"\n\n#include \n\n#include \"trt_serialize.hpp\"\n\nvoid CumMaxMinForwardLauncher_float(const float *input, float *output_value,\n int *output_index, const int *dims,\n int nbDims, int cum_dim, int cum_type,\n cudaStream_t stream);\n\nvoid CumMaxMinForwardLauncher_int32(const int *input, int *output_value,\n int *output_index, const int *dims,\n int nbDims, int cum_dim, int cum_type,\n cudaStream_t stream);\n\nnamespace {\nstatic const char *PLUGIN_VERSION{\"1\"};\nstatic const char *CUMMAXMIN_PLUGIN_NAME{\"cummaxmin\"};\nstatic const char *CUMMAX_PLUGIN_NAME{\"cummax\"};\nstatic const char *CUMMIN_PLUGIN_NAME{\"cummin\"};\n} // namespace\n\nCumMaxMinPluginDynamic::CumMaxMinPluginDynamic(const std::string &name, int dim,\n TRT_CUMCMPTYPE cumType)\n : mLayerName(name), mDim(dim), mCumType(cumType) {}\n\nCumMaxMinPluginDynamic::CumMaxMinPluginDynamic(const std::string name,\n const void *data, size_t length)\n : mLayerName(name) {\n deserialize_value(&data, &length, &mDim);\n deserialize_value(&data, &length, &mCumType);\n}\n\nCumMaxMinPluginDynamic::~CumMaxMinPluginDynamic() {}\n\nnvinfer1::IPluginV2DynamicExt *CumMaxMinPluginDynamic::clone() const {\n CumMaxMinPluginDynamic *plugin =\n new CumMaxMinPluginDynamic(mLayerName, mDim, mCumType);\n plugin->setPluginNamespace(getPluginNamespace());\n\n return plugin;\n}\n\nnvinfer1::DimsExprs CumMaxMinPluginDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) {\n return inputs[0];\n}\n\nbool CumMaxMinPluginDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc *inOut, int nbInputs,\n int nbOutputs) {\n switch (pos) {\n // input[0]\n case 0:\n return (inOut[pos].type == nvinfer1::DataType::kFLOAT ||\n inOut[pos].type == nvinfer1::DataType::kINT32) &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR;\n // output[0]\n case 1:\n return inOut[pos].type == inOut[0].type &&\n inOut[pos].format == inOut[0].format;\n // output[1]\n case 2:\n return inOut[pos].type == nvinfer1::DataType::kINT32 &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR;\n default:\n return false;\n }\n}\n\nvoid CumMaxMinPluginDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *outputs, int nbOutputs) {}\n\nsize_t CumMaxMinPluginDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs, int nbOutputs) const {\n int sizeof_dtype = mmcv::getElementSize(outputs[0].type);\n}\n\nint CumMaxMinPluginDynamic::enqueue(\n const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc, const void *const *inputs,\n void *const *outputs, void *workSpace, cudaStream_t stream) {\n const void *input = inputs[0];\n void *output_value = outputs[0];\n int *output_index = (int *)outputs[1];\n\n const int *dims = &(inputDesc[0].dims.d[0]);\n int nbDims = inputDesc[0].dims.nbDims;\n\n switch (inputDesc[0].type) {\n case nvinfer1::DataType::kFLOAT:\n CumMaxMinForwardLauncher_float((float *)input, (float *)output_value,\n output_index, dims, nbDims, mDim,\n int(mCumType), stream);\n break;\n case nvinfer1::DataType::kINT32:\n CumMaxMinForwardLauncher_int32((int *)input, (int *)output_value,\n output_index, dims, nbDims, mDim,\n int(mCumType), stream);\n break;\n default:\n break;\n }\n\n return 0;\n}\n\nnvinfer1::DataType CumMaxMinPluginDynamic::getOutputDataType(\n int index, const nvinfer1::DataType *inputTypes, int nbInputs) const {\n switch (index) {\n case 0:\n return inputTypes[0];\n case 1:\n return nvinfer1::DataType::kINT32;\n default:\n break;\n }\n}\n\n// IPluginV2 Methods\nconst char *CumMaxMinPluginDynamic::getPluginType() const {\n switch (mCumType) {\n case TRT_CUMCMPTYPE::TRT_CUMMAX:\n return CUMMAX_PLUGIN_NAME;\n case TRT_CUMCMPTYPE::TRT_CUMMIN:\n return CUMMIN_PLUGIN_NAME;\n default:\n return \"UnknownCumType\";\n }\n}\n\nconst char *CumMaxMinPluginDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nint CumMaxMinPluginDynamic::getNbOutputs() const { return 2; }\n\nint CumMaxMinPluginDynamic::initialize() { return 0; }\n\nvoid CumMaxMinPluginDynamic::terminate() {}\n\nsize_t CumMaxMinPluginDynamic::getSerializationSize() const {\n return sizeof(mDim) + sizeof(mCumType);\n}\n\nvoid CumMaxMinPluginDynamic::serialize(void *buffer) const {\n serialize_value(&buffer, mDim);\n serialize_value(&buffer, mCumType);\n}\n\nvoid CumMaxMinPluginDynamic::destroy() {\n // This gets called when the network containing plugin is destroyed\n delete this;\n}\n\nvoid CumMaxMinPluginDynamic::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *CumMaxMinPluginDynamic::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\nCumMaxMinPluginDynamicCreator::CumMaxMinPluginDynamicCreator(\n TRT_CUMCMPTYPE cumType)\n : mCumType(cumType) {\n mPluginAttributes.clear();\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"dim\"));\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char *CumMaxMinPluginDynamicCreator::getPluginName() const {\n return CUMMAXMIN_PLUGIN_NAME;\n}\n\nconst char *CumMaxMinPluginDynamicCreator::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nconst nvinfer1::PluginFieldCollection *\nCumMaxMinPluginDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nnvinfer1::IPluginV2 *CumMaxMinPluginDynamicCreator::createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) {\n int dim = 0;\n\n for (int i = 0; i < fc->nbFields; i++) {\n if (fc->fields[i].data == nullptr) {\n continue;\n }\n std::string field_name(fc->fields[i].name);\n\n if (field_name.compare(\"dim\") == 0) {\n dim = static_cast(fc->fields[i].data)[0];\n }\n }\n\n CumMaxMinPluginDynamic *plugin =\n new CumMaxMinPluginDynamic(name, dim, mCumType);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nnvinfer1::IPluginV2 *CumMaxMinPluginDynamicCreator::deserializePlugin(\n const char *name, const void *serialData, size_t serialLength) {\n // This object will be deleted when the network is destroyed, which will\n // call FCPluginDynamic::destroy()\n auto plugin = new CumMaxMinPluginDynamic(name, serialData, serialLength);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nvoid CumMaxMinPluginDynamicCreator::setPluginNamespace(\n const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *CumMaxMinPluginDynamicCreator::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\nCumMaxPluginDynamicCreator::CumMaxPluginDynamicCreator()\n : CumMaxMinPluginDynamicCreator(TRT_CUMCMPTYPE::TRT_CUMMAX) {}\n\nconst char *CumMaxPluginDynamicCreator::getPluginName() const {\n return CUMMAX_PLUGIN_NAME;\n}\n\nCumMinPluginDynamicCreator::CumMinPluginDynamicCreator()\n : CumMaxMinPluginDynamicCreator(TRT_CUMCMPTYPE::TRT_CUMMIN) {}\n\nconst char *CumMinPluginDynamicCreator::getPluginName() const {\n return CUMMIN_PLUGIN_NAME;\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_cummaxmin_kernel.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n\n#include \"common_cuda_helper.hpp\"\n#include \"trt_cuda_helper.cuh\"\n#include \"trt_plugin_helper.hpp\"\n\nusing mmcv::TensorDesc;\n\ntemplate \n__global__ void cummaxmin_kernel(const scalar_t *input, scalar_t *output_value,\n int *output_index, TensorDesc tensor_desc,\n int cum_dim, int cum_type) {\n const size_t cum_size = tensor_desc.shape[cum_dim];\n const size_t cum_stride = tensor_desc.stride[cum_dim];\n const size_t data_size =\n tensor_desc.stride[0] * tensor_desc.shape[0] / cum_size;\n CUDA_1D_KERNEL_LOOP(index, data_size) {\n size_t cum_offset =\n index / cum_stride * (cum_size * cum_stride) + index % cum_stride;\n int cum_index = 0;\n auto cum_value = input[cum_offset];\n output_value[cum_offset] = cum_value;\n output_index[cum_offset] = cum_index;\n\n for (size_t cum_index_current = 1; cum_index_current < cum_size;\n ++cum_index_current) {\n cum_offset += cum_stride;\n const auto cum_value_current = input[cum_offset];\n switch (cum_type) {\n case 0: // max\n if (cum_value_current > cum_value) {\n cum_value = cum_value_current;\n cum_index = cum_index_current;\n }\n break;\n case 1: // min\n if (cum_value_current < cum_value) {\n cum_value = cum_value_current;\n cum_index = cum_index_current;\n }\n break;\n }\n output_value[cum_offset] = cum_value;\n output_index[cum_offset] = cum_index;\n }\n }\n}\n\ntemplate \nvoid CumMaxMinForwardLauncher(const scalar_t *input, scalar_t *output_value,\n int *output_index, const int *dims, int nbDims,\n int cum_dim, int cum_type, cudaStream_t stream) {\n // fill tensordesc and initial\n TensorDesc tensor_desc;\n memset((void *)&tensor_desc, 0, sizeof(TensorDesc));\n tensor_desc.dim = nbDims;\n tensor_desc.shape[nbDims - 1] = dims[nbDims - 1];\n tensor_desc.stride[nbDims - 1] = 1;\n for (int i = nbDims - 2; i >= 0; --i) {\n tensor_desc.shape[i] = dims[i];\n tensor_desc.stride[i] = dims[i + 1] * tensor_desc.stride[i + 1];\n }\n\n // cum dim should be larger than 0\n cum_dim = cum_dim >= 0 ? cum_dim : (nbDims + cum_dim);\n\n const int data_size =\n tensor_desc.stride[0] * tensor_desc.shape[0] / tensor_desc.shape[cum_dim];\n\n const int col_block = GET_BLOCKS(data_size, THREADS_PER_BLOCK);\n\n cummaxmin_kernel<<>>(\n input, output_value, output_index, tensor_desc, cum_dim, cum_type);\n}\n\nvoid CumMaxMinForwardLauncher_float(const float *input, float *output_value,\n int *output_index, const int *dims,\n int nbDims, int cum_dim, int cum_type,\n cudaStream_t stream) {\n CumMaxMinForwardLauncher(input, output_value, output_index, dims,\n nbDims, cum_dim, cum_type, stream);\n}\n\nvoid CumMaxMinForwardLauncher_int32(const int *input, int *output_value,\n int *output_index, const int *dims,\n int nbDims, int cum_dim, int cum_type,\n cudaStream_t stream) {\n CumMaxMinForwardLauncher(input, output_value, output_index, dims, nbDims,\n cum_dim, cum_type, stream);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_deform_conv.hpp\"\n\n#include \n\n#include \n\n#include \"trt_serialize.hpp\"\n\nvoid DeformConvForwardCUDAKernelLauncher_float(\n const float *input, const float *weight, const float *offset, float *output,\n void *workspace, int batchSize, int nInputPlane, int inputHeight,\n int inputWidth, int nOutputPlane, int kW, int kH, int dW, int dH, int padW,\n int padH, int dilationW, int dilationH, int group, int deformable_group,\n int im2col_step, cublasHandle_t cublas_handle, cudaStream_t stream);\n\nnamespace {\nstatic const char *PLUGIN_VERSION{\"1\"};\nstatic const char *PLUGIN_NAME{\"MMCVDeformConv2d\"};\n} // namespace\n\nnvinfer1::PluginFieldCollection DeformableConvPluginDynamicCreator::mFC{};\nstd::vector\n DeformableConvPluginDynamicCreator::mPluginAttributes;\n\nDeformableConvPluginDynamic::DeformableConvPluginDynamic(\n const std::string &name, const nvinfer1::Dims &stride,\n const nvinfer1::Dims &padding, const nvinfer1::Dims &dilation,\n const int deformableGroup, const int group, int im2colStep)\n : mLayerName(name),\n mStride(stride),\n mPadding(padding),\n mDilation(dilation),\n mDeformableGroup(deformableGroup),\n mGroup(group),\n mIm2colStep(im2colStep) {}\n\nDeformableConvPluginDynamic::DeformableConvPluginDynamic(const std::string name,\n const void *data,\n size_t length)\n : mLayerName(name) {\n deserialize_value(&data, &length, &mStride);\n deserialize_value(&data, &length, &mPadding);\n deserialize_value(&data, &length, &mDilation);\n deserialize_value(&data, &length, &mDeformableGroup);\n deserialize_value(&data, &length, &mGroup);\n deserialize_value(&data, &length, &mIm2colStep);\n}\nDeformableConvPluginDynamic::~DeformableConvPluginDynamic() {}\n\nnvinfer1::IPluginV2DynamicExt *DeformableConvPluginDynamic::clone() const {\n DeformableConvPluginDynamic *plugin =\n new DeformableConvPluginDynamic(mLayerName, mStride, mPadding, mDilation,\n mDeformableGroup, mGroup, mIm2colStep);\n plugin->setPluginNamespace(getPluginNamespace());\n\n return plugin;\n}\n\nnvinfer1::DimsExprs DeformableConvPluginDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) {\n nvinfer1::DimsExprs ret;\n ret.nbDims = 4;\n ret.d[0] = inputs[0].d[0];\n ret.d[1] = inputs[2].d[0];\n\n ret.d[2] = inputs[1].d[2];\n ret.d[3] = inputs[1].d[3];\n\n return ret;\n}\n\nbool DeformableConvPluginDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc *inOut, int nbInputs,\n int nbOutputs) {\n if (pos == 0) {\n return (inOut[pos].type == nvinfer1::DataType::kFLOAT &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR);\n\n } else {\n return inOut[pos].type == inOut[0].type &&\n inOut[pos].format == inOut[0].format;\n }\n}\n\nvoid DeformableConvPluginDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *outputs, int nbOutputs) {}\n\nsize_t DeformableConvPluginDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs, int nbOutputs) const {\n int sizeof_dtype = mmcv::getElementSize(outputs[0].type);\n\n int batch_size = inputs[0].dims.d[0];\n int nInputPlane = inputs[0].dims.d[1];\n int inputHeight = inputs[0].dims.d[2];\n int inputWidth = inputs[0].dims.d[3];\n\n int nOutputPlane = outputs[0].dims.d[1];\n int outputHeight = outputs[0].dims.d[2];\n int outputWidth = outputs[0].dims.d[3];\n\n int kW = inputs[2].dims.d[2];\n int kH = inputs[2].dims.d[3];\n int im2col_step = std::min(batch_size, mIm2colStep);\n\n size_t col_size =\n mmcv::getAlignedSize(nInputPlane * kW * kH * im2col_step * outputHeight *\n outputWidth * sizeof_dtype);\n\n size_t out_size = 0;\n if (im2col_step != 1)\n out_size = mmcv::getAlignedSize(batch_size * nOutputPlane * outputHeight *\n outputWidth * sizeof_dtype);\n\n return col_size + out_size;\n}\n\nint DeformableConvPluginDynamic::enqueue(\n const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc, const void *const *inputs,\n void *const *outputs, void *workSpace, cudaStream_t stream) {\n int batch_size = inputDesc[0].dims.d[0];\n int inputChannel = inputDesc[0].dims.d[1];\n int inputHeight = inputDesc[0].dims.d[2];\n int inputWidth = inputDesc[0].dims.d[3];\n int outputChannel = outputDesc[0].dims.d[1];\n int kernelHeight = inputDesc[2].dims.d[2];\n int kernelWidth = inputDesc[2].dims.d[3];\n\n const void *x = inputs[0];\n const void *offset = inputs[1];\n const void *weight = inputs[2];\n void *output = outputs[0];\n int im2col_step = std::min(batch_size, mIm2colStep);\n\n // TODO: add fp16 support\n auto data_type = inputDesc[0].type;\n switch (data_type) {\n case nvinfer1::DataType::kFLOAT:\n DeformConvForwardCUDAKernelLauncher_float(\n (float *)x, (float *)weight, (float *)offset, (float *)output,\n workSpace, batch_size, inputChannel, inputHeight, inputWidth,\n outputChannel, kernelWidth, kernelHeight, mStride.d[0], mStride.d[1],\n mPadding.d[0], mPadding.d[1], mDilation.d[0], mDilation.d[1], mGroup,\n mDeformableGroup, im2col_step, m_cublas_handle, stream);\n break;\n default:\n return 1;\n break;\n }\n\n return 0;\n}\n\nnvinfer1::DataType DeformableConvPluginDynamic::getOutputDataType(\n int index, const nvinfer1::DataType *inputTypes, int nbInputs) const {\n return inputTypes[0];\n}\n\n// IPluginV2 Methods\nconst char *DeformableConvPluginDynamic::getPluginType() const {\n return PLUGIN_NAME;\n}\n\nconst char *DeformableConvPluginDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nint DeformableConvPluginDynamic::getNbOutputs() const { return 1; }\n\nint DeformableConvPluginDynamic::initialize() { return 0; }\n\nvoid DeformableConvPluginDynamic::terminate() {}\n\nsize_t DeformableConvPluginDynamic::getSerializationSize() const {\n return sizeof(mStride) + sizeof(mPadding) + sizeof(mDilation) +\n sizeof(mDeformableGroup) + sizeof(mGroup) + sizeof(mIm2colStep);\n}\n\nvoid DeformableConvPluginDynamic::serialize(void *buffer) const {\n serialize_value(&buffer, mStride);\n serialize_value(&buffer, mPadding);\n serialize_value(&buffer, mDilation);\n serialize_value(&buffer, mDeformableGroup);\n serialize_value(&buffer, mGroup);\n serialize_value(&buffer, mIm2colStep);\n}\n\nvoid DeformableConvPluginDynamic::destroy() {\n // This gets called when the network containing plugin is destroyed\n delete this;\n}\n\nvoid DeformableConvPluginDynamic::attachToContext(\n cudnnContext *cudnnContext, cublasContext *cublasContext,\n nvinfer1::IGpuAllocator *gpuAllocator) {\n m_cublas_handle = cublasContext;\n}\n\nvoid DeformableConvPluginDynamic::detachFromContext() {}\n\nvoid DeformableConvPluginDynamic::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *DeformableConvPluginDynamic::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\n////////////////////// creator /////////////////////////////\n\nDeformableConvPluginDynamicCreator::DeformableConvPluginDynamicCreator() {\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"stride\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"padding\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"dilation\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"groups\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"deform_groups\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"bias\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"im2col_step\"));\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char *DeformableConvPluginDynamicCreator::getPluginName() const {\n return PLUGIN_NAME;\n}\n\nconst char *DeformableConvPluginDynamicCreator::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nconst nvinfer1::PluginFieldCollection *\nDeformableConvPluginDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nnvinfer1::IPluginV2 *DeformableConvPluginDynamicCreator::createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) {\n nvinfer1::Dims stride{2, {1, 1}};\n nvinfer1::Dims padding{2, {0, 0}};\n nvinfer1::Dims dilation{2, {1, 1}};\n int deformableGroup = 1;\n int group = 1;\n int im2col_step = 32;\n\n for (int i = 0; i < fc->nbFields; i++) {\n if (fc->fields[i].data == nullptr) {\n continue;\n }\n std::string field_name(fc->fields[i].name);\n\n if (field_name.compare(\"stride\") == 0) {\n stride.nbDims = 2;\n stride.d[0] = static_cast(fc->fields[i].data)[0];\n if (fc->fields[i].length == 1) {\n stride.d[1] = stride.d[0];\n } else {\n stride.d[1] = static_cast(fc->fields[i].data)[1];\n }\n }\n\n if (field_name.compare(\"padding\") == 0) {\n padding.nbDims = 2;\n padding.d[0] = static_cast(fc->fields[i].data)[0];\n if (fc->fields[i].length == 1) {\n padding.d[1] = padding.d[0];\n } else {\n padding.d[1] = static_cast(fc->fields[i].data)[1];\n }\n }\n\n if (field_name.compare(\"dilation\") == 0) {\n dilation.nbDims = 2;\n dilation.d[0] = static_cast(fc->fields[i].data)[0];\n if (fc->fields[i].length == 1) {\n dilation.d[1] = dilation.d[0];\n } else {\n dilation.d[1] = static_cast(fc->fields[i].data)[1];\n }\n }\n\n if (field_name.compare(\"deformable_group\") == 0) {\n deformableGroup = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"group\") == 0) {\n group = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"im2col_step\") == 0) {\n im2col_step = static_cast(fc->fields[i].data)[0];\n }\n }\n\n DeformableConvPluginDynamic *plugin = new DeformableConvPluginDynamic(\n name, stride, padding, dilation, deformableGroup, group, im2col_step);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nnvinfer1::IPluginV2 *DeformableConvPluginDynamicCreator::deserializePlugin(\n const char *name, const void *serialData, size_t serialLength) {\n auto plugin = new DeformableConvPluginDynamic(name, serialData, serialLength);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nvoid DeformableConvPluginDynamicCreator::setPluginNamespace(\n const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *DeformableConvPluginDynamicCreator::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_deform_conv_kernel.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n\n#include \"common_cuda_helper.hpp\"\n#include \"deform_conv_cuda_kernel.cuh\"\n#include \"trt_cuda_helper.cuh\"\n#include \"trt_plugin_helper.hpp\"\n\ntemplate \nvoid trt_deformable_im2col(const T* data_input, const T* data_offset,\n const int channels, const int height,\n const int width, const int ksize_h,\n const int ksize_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w,\n const int dilation_h, const int dilation_w,\n const int parallel_imgs, const int deformable_group,\n T* data_col, cudaStream_t stream) {\n int height_col =\n (height + 2 * pad_h - (dilation_h * (ksize_h - 1) + 1)) / stride_h + 1;\n int width_col =\n (width + 2 * pad_w - (dilation_w * (ksize_w - 1) + 1)) / stride_w + 1;\n int num_kernels = channels * height_col * width_col * parallel_imgs;\n int channel_per_deformable_group = channels / deformable_group;\n\n deformable_im2col_gpu_kernel\n <<>>(\n num_kernels, data_input, data_offset, height, width, ksize_h, ksize_w,\n pad_h, pad_w, stride_h, stride_w, dilation_h, dilation_w,\n channel_per_deformable_group, parallel_imgs, channels,\n deformable_group, height_col, width_col, data_col);\n\n cudaCheckError();\n}\n\ntemplate \nvoid DeformConvForwardCUDAKernelLauncher(\n const scalar_t* input, const scalar_t* weight, const scalar_t* offset,\n scalar_t* output, void* workspace, int batchSize, int nInputPlane,\n int inputHeight, int inputWidth, int nOutputPlane, int kW, int kH, int dW,\n int dH, int padW, int padH, int dilationW, int dilationH, int group,\n int deformable_group, int im2col_step, cublasHandle_t cublas_handle,\n cudaStream_t stream) {\n size_t word_size = sizeof(scalar_t);\n\n im2col_step = std::min(int(batchSize), im2col_step);\n long outputWidth =\n (inputWidth + 2 * padW - (dilationW * (kW - 1) + 1)) / dW + 1;\n long outputHeight =\n (inputHeight + 2 * padH - (dilationH * (kH - 1) + 1)) / dH + 1;\n\n long long columns_size =\n mmcv::getAlignedSize(nInputPlane * kW * kH * im2col_step * outputHeight *\n outputWidth * word_size);\n\n // column buffer for img2col\n scalar_t* columns = (scalar_t*)workspace;\n workspace = workspace + columns_size;\n\n scalar_t* output_buffer;\n long long output_buffer_size = 0;\n if (im2col_step == 1) {\n output_buffer = output;\n } else {\n // output need permute when im2col_step!=1\n output_buffer = (scalar_t*)workspace;\n output_buffer_size = batchSize * nOutputPlane * outputWidth * outputHeight;\n }\n\n long long input_elt_step =\n im2col_step * nInputPlane * inputHeight * inputWidth;\n long long offset_elt_step =\n im2col_step * deformable_group * 2 * kH * kW * outputHeight * outputWidth;\n long long out_buffer_step =\n nOutputPlane * im2col_step * outputHeight * outputWidth;\n long long col_g_step =\n nInputPlane * kW * kH / group * im2col_step * outputHeight * outputWidth;\n long long weight_g_step =\n nOutputPlane / group * nInputPlane / group * kH * kW;\n long long out_buffer_g_step =\n nOutputPlane / group * im2col_step * outputHeight * outputWidth;\n int m = nOutputPlane / group;\n int n = im2col_step * outputHeight * outputWidth;\n int k = nInputPlane / group * kH * kW;\n scalar_t alpha = 1.;\n scalar_t beta = 0.;\n\n for (int elt = 0; elt < batchSize / im2col_step; elt++) {\n const scalar_t* input_start = input + elt * input_elt_step;\n const scalar_t* offset_start = offset + elt * offset_elt_step;\n\n trt_deformable_im2col(input_start, offset_start, nInputPlane,\n inputHeight, inputWidth, kH, kW, padH, padW,\n dH, dW, dilationH, dilationW, im2col_step,\n deformable_group, columns, stream);\n\n for (int g = 0; g < group; ++g) {\n const scalar_t* weight_start = weight + g * weight_g_step;\n scalar_t* col_start = columns + g * col_g_step;\n scalar_t* out_buffer_start =\n output_buffer + elt * out_buffer_step + g * out_buffer_g_step;\n\n cublasGemmWrap(cublas_handle, CUBLAS_OP_N, CUBLAS_OP_N, n, m, k,\n &alpha, col_start, n, weight_start, k, &beta,\n out_buffer_start, n);\n cudaCheckError();\n }\n }\n\n if (im2col_step != 1) {\n int output_buffer_shape[5] = {batchSize / im2col_step, nOutputPlane,\n im2col_step, outputHeight, outputWidth};\n int output_buffer_permute[5] = {0, 2, 1, 3, 4};\n memcpyPermute(output, output_buffer, &output_buffer_shape[0],\n &output_buffer_permute[0], 5, stream);\n }\n}\n\nvoid DeformConvForwardCUDAKernelLauncher_float(\n const float* input, const float* weight, const float* offset, float* output,\n void* workspace, int batchSize, int nInputPlane, int inputHeight,\n int inputWidth, int nOutputPlane, int kW, int kH, int dW, int dH, int padW,\n int padH, int dilationW, int dilationH, int group, int deformable_group,\n int im2col_step, cublasHandle_t cublas_handle, cudaStream_t stream) {\n DeformConvForwardCUDAKernelLauncher(\n input, weight, offset, output, workspace, batchSize, nInputPlane,\n inputHeight, inputWidth, nOutputPlane, kW, kH, dW, dH, padW, padH,\n dilationW, dilationH, group, deformable_group, im2col_step, cublas_handle,\n stream);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_grid_sampler.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_grid_sampler.hpp\"\n\n#include \n#include \n\n#include \n\n#include \"trt_serialize.hpp\"\n\nusing mmcv::GridSamplerInterpolation;\nusing mmcv::GridSamplerPadding;\n\nvoid grid_sample_float(float *output, const float *input, const float *grid,\n int *output_dims, int *input_dims, int *grid_dims,\n int nb_dims, GridSamplerInterpolation interp,\n GridSamplerPadding padding, bool align_corners,\n cudaStream_t stream);\n\nnamespace {\nstatic const char *PLUGIN_VERSION{\"1\"};\nstatic const char *PLUGIN_NAME{\"grid_sampler\"};\n} // namespace\n\nnvinfer1::PluginFieldCollection GridSamplerDynamicCreator::mFC{};\nstd::vector GridSamplerDynamicCreator::mPluginAttributes;\n\nGridSamplerDynamic::GridSamplerDynamic(const std::string &name, int mode,\n int paddingMode, bool alignCorners)\n : mLayerName(name),\n mMode(mode),\n mPaddingMode(paddingMode),\n mAlignCorners(alignCorners) {}\n\nGridSamplerDynamic::GridSamplerDynamic(const std::string name, const void *data,\n size_t length)\n : mLayerName(name) {\n deserialize_value(&data, &length, &mMode);\n deserialize_value(&data, &length, &mPaddingMode);\n deserialize_value(&data, &length, &mAlignCorners);\n}\n\nnvinfer1::IPluginV2DynamicExt *GridSamplerDynamic::clone() const {\n GridSamplerDynamic *plugin =\n new GridSamplerDynamic(mLayerName, mMode, mPaddingMode, mAlignCorners);\n plugin->setPluginNamespace(getPluginNamespace());\n\n return plugin;\n}\n\nnvinfer1::DimsExprs GridSamplerDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) {\n nvinfer1::DimsExprs ret;\n ret.nbDims = inputs[0].nbDims;\n ret.d[0] = inputs[0].d[0];\n ret.d[1] = inputs[0].d[1];\n for (int i = 2; i < ret.nbDims; ++i) {\n ret.d[i] = inputs[1].d[i - 1];\n }\n return ret;\n}\n\nbool GridSamplerDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc *inOut, int nbInputs,\n int nbOutputs) {\n if (pos == 0) {\n return (inOut[pos].type == nvinfer1::DataType::kFLOAT &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR);\n } else {\n return inOut[pos].type == inOut[0].type &&\n inOut[pos].format == inOut[0].format;\n }\n}\n\nvoid GridSamplerDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *outputs, int nbOutputs) {\n // Validate input arguments\n}\n\nsize_t GridSamplerDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs, int nbOutputs) const {\n return 0;\n}\n\nint GridSamplerDynamic::enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs,\n void *workSpace, cudaStream_t stream) {\n nvinfer1::Dims input_dims = inputDesc[0].dims;\n nvinfer1::Dims grid_dims = inputDesc[1].dims;\n nvinfer1::Dims output_dims = outputDesc[0].dims;\n\n using mmcv::GridSamplerInterpolation;\n using mmcv::GridSamplerPadding;\n\n GridSamplerInterpolation interp_mode = GridSamplerInterpolation::Bilinear;\n switch (mMode) {\n case 0:\n interp_mode = GridSamplerInterpolation::Bilinear;\n break;\n case 1:\n interp_mode = GridSamplerInterpolation::Nearest;\n break;\n default:\n break;\n }\n\n GridSamplerPadding padding_mode = GridSamplerPadding::Zeros;\n switch (mPaddingMode) {\n case 0:\n padding_mode = GridSamplerPadding::Zeros;\n break;\n\n case 1:\n padding_mode = GridSamplerPadding::Border;\n break;\n\n case 2:\n padding_mode = GridSamplerPadding::Reflection;\n break;\n default:\n break;\n }\n\n auto data_type = inputDesc[0].type;\n\n switch (data_type) {\n case nvinfer1::DataType::kFLOAT:\n grid_sample_float(\n (float *)outputs[0], (float *)inputs[0], (float *)inputs[1],\n &(output_dims.d[0]), &(input_dims.d[0]), &(grid_dims.d[0]),\n input_dims.nbDims, interp_mode, padding_mode, mAlignCorners, stream);\n break;\n default:\n return 1;\n break;\n }\n\n return 0;\n}\n\nnvinfer1::DataType GridSamplerDynamic::getOutputDataType(\n int index, const nvinfer1::DataType *inputTypes, int nbInputs) const {\n return inputTypes[0];\n}\n\n// IPluginV2 Methods\nconst char *GridSamplerDynamic::getPluginType() const { return PLUGIN_NAME; }\n\nconst char *GridSamplerDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nint GridSamplerDynamic::getNbOutputs() const { return 1; }\n\nint GridSamplerDynamic::initialize() { return 0; }\n\nvoid GridSamplerDynamic::terminate() {}\n\nsize_t GridSamplerDynamic::getSerializationSize() const {\n return sizeof(mMode) + sizeof(mPaddingMode) + sizeof(mAlignCorners);\n}\n\nvoid GridSamplerDynamic::serialize(void *buffer) const {\n serialize_value(&buffer, mMode);\n serialize_value(&buffer, mPaddingMode);\n serialize_value(&buffer, mAlignCorners);\n}\n\nvoid GridSamplerDynamic::destroy() {\n // This gets called when the network containing plugin is destroyed\n delete this;\n}\n\nvoid GridSamplerDynamic::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *GridSamplerDynamic::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\n////////////////////// creator /////////////////////////////\n\nGridSamplerDynamicCreator::GridSamplerDynamicCreator() {\n mPluginAttributes.clear();\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"interpolation_mode\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"padding_mode\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"align_corners\"));\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char *GridSamplerDynamicCreator::getPluginName() const {\n return PLUGIN_NAME;\n}\n\nconst char *GridSamplerDynamicCreator::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nconst nvinfer1::PluginFieldCollection *\nGridSamplerDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nnvinfer1::IPluginV2 *GridSamplerDynamicCreator::createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) {\n int mode = 0;\n int paddingMode = 0;\n bool alignCorners = false;\n\n for (int i = 0; i < fc->nbFields; i++) {\n if (fc->fields[i].data == nullptr) {\n continue;\n }\n std::string field_name(fc->fields[i].name);\n\n if (field_name.compare(\"interpolation_mode\") == 0) {\n mode = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"padding_mode\") == 0) {\n paddingMode = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"align_corners\") == 0) {\n alignCorners = (bool)(static_cast(fc->fields[i].data)[0]);\n }\n }\n\n GridSamplerDynamic *plugin =\n new GridSamplerDynamic(name, mode, paddingMode, alignCorners);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nnvinfer1::IPluginV2 *GridSamplerDynamicCreator::deserializePlugin(\n const char *name, const void *serialData, size_t serialLength) {\n // This object will be deleted when the network is destroyed, which will\n // call FCPluginDynamic::destroy()\n auto plugin = new GridSamplerDynamic(name, serialData, serialLength);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nvoid GridSamplerDynamicCreator::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *GridSamplerDynamicCreator::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_grid_sampler_kernel.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// modified from\n// https://github.com/pytorch/pytorch/blob/ec683299ebabf297a3504c76248d37be830e4342/aten/src/ATen/native/cuda/GridSampler.cuh\n// and\n// https://github.com/pytorch/pytorch/blob/ec683299ebabf297a3504c76248d37be830e4342/aten/src/ATen/native/cuda/GridSampler.cu\n\n#include \n#include \n\n#include \n#include \n#include \n\n#include \"common_cuda_helper.hpp\"\n#include \"trt_cuda_helper.cuh\"\n#include \"trt_grid_sampler.hpp\"\n#include \"trt_plugin_helper.hpp\"\n\nusing mmcv::GridSamplerInterpolation;\nusing mmcv::GridSamplerPadding;\nusing mmcv::TensorDesc;\n\n// Unnormalizes a coordinate from the -1 to +1 scale to its pixel index value,\n// where we view each pixel as an area between (idx - 0.5) and (idx + 0.5).\n// if align_corners: -1 and +1 get sent to the centers of the corner pixels\n// -1 --> 0\n// +1 --> (size - 1)\n// scale_factor = (size - 1) / 2\n// if not align_corners: -1 and +1 get sent to the image edges\n// -1 --> -0.5\n// +1 --> (size - 1) + 0.5 == size - 0.5\n// scale_factor = size / 2\ntemplate \nstatic __forceinline__ __device__ scalar_t\ngrid_sampler_unnormalize(scalar_t coord, int size, bool align_corners) {\n if (align_corners) {\n // unnormalize coord from [-1, 1] to [0, size - 1]\n return ((coord + 1.f) / 2) * (size - 1);\n } else {\n // unnormalize coord from [-1, 1] to [-0.5, size - 0.5]\n return ((coord + 1.f) * size - 1) / 2;\n }\n}\n\n// Clips coordinates to between 0 and clip_limit - 1\ntemplate \nstatic __forceinline__ __device__ scalar_t clip_coordinates(scalar_t in,\n int clip_limit) {\n return ::min(static_cast(clip_limit - 1),\n ::max(in, static_cast(0)));\n}\n\n// Reflects coordinates until they fall between low and high (inclusive).\n// The bounds are passed as twice their value so that half-integer values\n// can be represented as ints.\ntemplate \nstatic __forceinline__ __device__ scalar_t reflect_coordinates(scalar_t in,\n int twice_low,\n int twice_high) {\n if (twice_low == twice_high) {\n return static_cast(0);\n }\n scalar_t min = static_cast(twice_low) / 2;\n scalar_t span = static_cast(twice_high - twice_low) / 2;\n in = ::fabs(in - min);\n // `fmod` returns same sign as `in`, which is positive after the `fabs` above.\n scalar_t extra = ::fmod(in, span);\n int flips = static_cast(::floor(in / span));\n if (flips % 2 == 0) {\n return extra + min;\n } else {\n return span - extra + min;\n }\n}\n\ntemplate \nstatic __forceinline__ __device__ scalar_t\nsafe_downgrade_to_int_range(scalar_t x) {\n // -100.0 does not have special meaning. This is just to make sure\n // it's not within_bounds_2d or within_bounds_3d, and does not cause\n // undefined behavior. See #35506.\n if (x > INT_MAX - 1 || x < INT_MIN || !::isfinite(static_cast(x)))\n return static_cast(-100.0);\n return x;\n}\n\n// Computes the pixel source index value for a grid coordinate\ntemplate \nstatic __forceinline__ __device__ scalar_t grid_sampler_compute_source_index(\n scalar_t coord, int size, GridSamplerPadding padding_mode,\n bool align_corners) {\n coord = grid_sampler_unnormalize(coord, size, align_corners);\n if (padding_mode == GridSamplerPadding::Border) {\n // clip coordinates to image borders\n coord = clip_coordinates(coord, size);\n } else if (padding_mode == GridSamplerPadding::Reflection) {\n // reflect coordinates by image borders\n if (align_corners) {\n coord = reflect_coordinates(coord, 0, 2 * (size - 1));\n } else {\n coord = reflect_coordinates(coord, -1, 2 * size - 1);\n }\n // clip coordinates to image borders\n coord = clip_coordinates(coord, size);\n }\n\n coord = safe_downgrade_to_int_range(coord);\n return coord;\n}\n\nstatic __forceinline__ __device__ bool within_bounds_2d(int h, int w, int H,\n int W) {\n return h >= 0 && h < H && w >= 0 && w < W;\n}\n\nstatic __forceinline__ __device__ bool within_bounds_3d(int d, int h, int w,\n int D, int H, int W) {\n return d >= 0 && d < D && h >= 0 && h < H && w >= 0 && w < W;\n}\n\ntemplate \n__global__ void grid_sampler_2d_kernel(\n const int nthreads, const scalar_t *input, const scalar_t *grid,\n scalar_t *output, TensorDesc input_desc, TensorDesc grid_desc,\n TensorDesc output_desc, const GridSamplerInterpolation interpolation_mode,\n const GridSamplerPadding padding_mode, bool align_corners) {\n int C = input_desc.shape[1];\n int inp_H = input_desc.shape[2];\n int inp_W = input_desc.shape[3];\n int out_H = grid_desc.shape[1];\n int out_W = grid_desc.shape[2];\n int inp_sN = input_desc.stride[0];\n int inp_sC = input_desc.stride[1];\n int inp_sH = input_desc.stride[2];\n int inp_sW = input_desc.stride[3];\n int grid_sN = grid_desc.stride[0];\n int grid_sH = grid_desc.stride[1];\n int grid_sW = grid_desc.stride[2];\n int grid_sCoor = grid_desc.stride[3];\n int out_sN = output_desc.stride[0];\n int out_sC = output_desc.stride[1];\n int out_sH = output_desc.stride[2];\n int out_sW = output_desc.stride[3];\n\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n const int w = index % out_W;\n const int h = (index / out_W) % out_H;\n const int n = index / (out_H * out_W);\n const int grid_offset = n * grid_sN + h * grid_sH + w * grid_sW;\n\n // get the corresponding input x, y coordinates from grid\n scalar_t ix = grid[grid_offset];\n scalar_t iy = grid[grid_offset + grid_sCoor];\n\n ix = grid_sampler_compute_source_index(ix, inp_W, padding_mode,\n align_corners);\n iy = grid_sampler_compute_source_index(iy, inp_H, padding_mode,\n align_corners);\n\n if (interpolation_mode == GridSamplerInterpolation::Bilinear) {\n // get NE, NW, SE, SW pixel values from (x, y)\n int ix_nw = static_cast(::floor(ix));\n int iy_nw = static_cast(::floor(iy));\n int ix_ne = ix_nw + 1;\n int iy_ne = iy_nw;\n int ix_sw = ix_nw;\n int iy_sw = iy_nw + 1;\n int ix_se = ix_nw + 1;\n int iy_se = iy_nw + 1;\n\n // get surfaces to each neighbor:\n scalar_t nw = (ix_se - ix) * (iy_se - iy);\n scalar_t ne = (ix - ix_sw) * (iy_sw - iy);\n scalar_t sw = (ix_ne - ix) * (iy - iy_ne);\n scalar_t se = (ix - ix_nw) * (iy - iy_nw);\n\n // calculate bilinear weighted pixel value and set output pixel\n auto inp_ptr_NC = input + n * inp_sN;\n auto out_ptr_NCHW = output + n * out_sN + h * out_sH + w * out_sW;\n for (int c = 0; c < C;\n ++c, inp_ptr_NC += inp_sC, out_ptr_NCHW += out_sC) {\n *out_ptr_NCHW = static_cast(0);\n if (within_bounds_2d(iy_nw, ix_nw, inp_H, inp_W)) {\n *out_ptr_NCHW += inp_ptr_NC[iy_nw * inp_sH + ix_nw * inp_sW] * nw;\n }\n if (within_bounds_2d(iy_ne, ix_ne, inp_H, inp_W)) {\n *out_ptr_NCHW += inp_ptr_NC[iy_ne * inp_sH + ix_ne * inp_sW] * ne;\n }\n if (within_bounds_2d(iy_sw, ix_sw, inp_H, inp_W)) {\n *out_ptr_NCHW += inp_ptr_NC[iy_sw * inp_sH + ix_sw * inp_sW] * sw;\n }\n if (within_bounds_2d(iy_se, ix_se, inp_H, inp_W)) {\n *out_ptr_NCHW += inp_ptr_NC[iy_se * inp_sH + ix_se * inp_sW] * se;\n }\n }\n } else if (interpolation_mode == GridSamplerInterpolation::Nearest) {\n int ix_nearest = static_cast(::round(ix));\n int iy_nearest = static_cast(::round(iy));\n\n // assign nearest neighbor pixel value to output pixel\n auto inp_ptr_NC = input + n * inp_sN;\n auto out_ptr_NCHW = output + n * out_sN + h * out_sH + w * out_sW;\n for (int c = 0; c < C;\n ++c, inp_ptr_NC += inp_sC, out_ptr_NCHW += out_sC) {\n if (within_bounds_2d(iy_nearest, ix_nearest, inp_H, inp_W)) {\n *out_ptr_NCHW = inp_ptr_NC[iy_nearest * inp_sH + ix_nearest * inp_sW];\n } else {\n *out_ptr_NCHW = static_cast(0);\n }\n }\n }\n }\n}\n\ntemplate \n__global__ void grid_sampler_3d_kernel(\n const int nthreads, const scalar_t *input, const scalar_t *grid,\n scalar_t *output, TensorDesc input_desc, TensorDesc grid_desc,\n TensorDesc output_desc, const GridSamplerInterpolation interpolation_mode,\n const GridSamplerPadding padding_mode, bool align_corners) {\n int C = input_desc.shape[1];\n int inp_D = input_desc.shape[2];\n int inp_H = input_desc.shape[3];\n int inp_W = input_desc.shape[4];\n int out_D = grid_desc.shape[1];\n int out_H = grid_desc.shape[2];\n int out_W = grid_desc.shape[3];\n int inp_sN = input_desc.stride[0];\n int inp_sC = input_desc.stride[1];\n int inp_sD = input_desc.stride[2];\n int inp_sH = input_desc.stride[3];\n int inp_sW = input_desc.stride[4];\n int grid_sN = grid_desc.stride[0];\n int grid_sD = grid_desc.stride[1];\n int grid_sH = grid_desc.stride[2];\n int grid_sW = grid_desc.stride[3];\n int grid_sCoor = grid_desc.stride[4];\n int out_sN = output_desc.stride[0];\n int out_sC = output_desc.stride[1];\n int out_sD = output_desc.stride[2];\n int out_sH = output_desc.stride[3];\n int out_sW = output_desc.stride[4];\n\n CUDA_1D_KERNEL_LOOP(index, nthreads) {\n const int w = index % out_W;\n const int h = (index / out_W) % out_H;\n const int d = (index / (out_H * out_W)) % out_D;\n const int n = index / (out_D * out_H * out_W);\n const int grid_offset =\n n * grid_sN + d * grid_sD + h * grid_sH + w * grid_sW;\n\n // get the corresponding input x, y, z coordinates from grid\n scalar_t ix = grid[grid_offset];\n scalar_t iy = grid[grid_offset + grid_sCoor];\n scalar_t iz = grid[grid_offset + 2 * grid_sCoor];\n\n ix = grid_sampler_compute_source_index(ix, inp_W, padding_mode,\n align_corners);\n iy = grid_sampler_compute_source_index(iy, inp_H, padding_mode,\n align_corners);\n iz = grid_sampler_compute_source_index(iz, inp_D, padding_mode,\n align_corners);\n\n if (interpolation_mode == GridSamplerInterpolation::Bilinear) {\n // get corner pixel values from (x, y, z)\n // for 4d, we used north-east-south-west\n // for 5d, we add top-bottom\n int ix_tnw = static_cast(::floor(ix));\n int iy_tnw = static_cast(::floor(iy));\n int iz_tnw = static_cast(::floor(iz));\n\n int ix_tne = ix_tnw + 1;\n int iy_tne = iy_tnw;\n int iz_tne = iz_tnw;\n\n int ix_tsw = ix_tnw;\n int iy_tsw = iy_tnw + 1;\n int iz_tsw = iz_tnw;\n\n int ix_tse = ix_tnw + 1;\n int iy_tse = iy_tnw + 1;\n int iz_tse = iz_tnw;\n\n int ix_bnw = ix_tnw;\n int iy_bnw = iy_tnw;\n int iz_bnw = iz_tnw + 1;\n\n int ix_bne = ix_tnw + 1;\n int iy_bne = iy_tnw;\n int iz_bne = iz_tnw + 1;\n\n int ix_bsw = ix_tnw;\n int iy_bsw = iy_tnw + 1;\n int iz_bsw = iz_tnw + 1;\n\n int ix_bse = ix_tnw + 1;\n int iy_bse = iy_tnw + 1;\n int iz_bse = iz_tnw + 1;\n\n // get surfaces to each neighbor:\n scalar_t tnw = (ix_bse - ix) * (iy_bse - iy) * (iz_bse - iz);\n scalar_t tne = (ix - ix_bsw) * (iy_bsw - iy) * (iz_bsw - iz);\n scalar_t tsw = (ix_bne - ix) * (iy - iy_bne) * (iz_bne - iz);\n scalar_t tse = (ix - ix_bnw) * (iy - iy_bnw) * (iz_bnw - iz);\n scalar_t bnw = (ix_tse - ix) * (iy_tse - iy) * (iz - iz_tse);\n scalar_t bne = (ix - ix_tsw) * (iy_tsw - iy) * (iz - iz_tsw);\n scalar_t bsw = (ix_tne - ix) * (iy - iy_tne) * (iz - iz_tne);\n scalar_t bse = (ix - ix_tnw) * (iy - iy_tnw) * (iz - iz_tnw);\n\n auto inp_ptr_NC = input + n * inp_sN;\n auto out_ptr_NCDHW =\n output + n * out_sN + d * out_sD + h * out_sH + w * out_sW;\n for (int c = 0; c < C;\n ++c, inp_ptr_NC += inp_sC, out_ptr_NCDHW += out_sC) {\n // (c, iz_tnw, iy_tnw, ix_tnw) * tnw + (c, iz_tne, iy_tne, ix_tne) *\n // tne\n // + (c, iz_tsw, iy_tsw, ix_tsw) * tsw + (c, iz_tse, iy_tse, ix_tse) *\n // tse\n // + (c, iz_bnw, iy_bnw, ix_bnw) * bnw + (c, iz_bne, iy_bne, ix_bne) *\n // bne\n // + (c, iz_bsw, iy_bsw, ix_bsw) * bsw + (c, iz_bse, iy_bse, ix_bse) *\n // bse\n *out_ptr_NCDHW = static_cast(0);\n if (within_bounds_3d(iz_tnw, iy_tnw, ix_tnw, inp_D, inp_H, inp_W)) {\n *out_ptr_NCDHW +=\n inp_ptr_NC[iz_tnw * inp_sD + iy_tnw * inp_sH + ix_tnw * inp_sW] *\n tnw;\n }\n if (within_bounds_3d(iz_tne, iy_tne, ix_tne, inp_D, inp_H, inp_W)) {\n *out_ptr_NCDHW +=\n inp_ptr_NC[iz_tne * inp_sD + iy_tne * inp_sH + ix_tne * inp_sW] *\n tne;\n }\n if (within_bounds_3d(iz_tsw, iy_tsw, ix_tsw, inp_D, inp_H, inp_W)) {\n *out_ptr_NCDHW +=\n inp_ptr_NC[iz_tsw * inp_sD + iy_tsw * inp_sH + ix_tsw * inp_sW] *\n tsw;\n }\n if (within_bounds_3d(iz_tse, iy_tse, ix_tse, inp_D, inp_H, inp_W)) {\n *out_ptr_NCDHW +=\n inp_ptr_NC[iz_tse * inp_sD + iy_tse * inp_sH + ix_tse * inp_sW] *\n tse;\n }\n if (within_bounds_3d(iz_bnw, iy_bnw, ix_bnw, inp_D, inp_H, inp_W)) {\n *out_ptr_NCDHW +=\n inp_ptr_NC[iz_bnw * inp_sD + iy_bnw * inp_sH + ix_bnw * inp_sW] *\n bnw;\n }\n if (within_bounds_3d(iz_bne, iy_bne, ix_bne, inp_D, inp_H, inp_W)) {\n *out_ptr_NCDHW +=\n inp_ptr_NC[iz_bne * inp_sD + iy_bne * inp_sH + ix_bne * inp_sW] *\n bne;\n }\n if (within_bounds_3d(iz_bsw, iy_bsw, ix_bsw, inp_D, inp_H, inp_W)) {\n *out_ptr_NCDHW +=\n inp_ptr_NC[iz_bsw * inp_sD + iy_bsw * inp_sH + ix_bsw * inp_sW] *\n bsw;\n }\n if (within_bounds_3d(iz_bse, iy_bse, ix_bse, inp_D, inp_H, inp_W)) {\n *out_ptr_NCDHW +=\n inp_ptr_NC[iz_bse * inp_sD + iy_bse * inp_sH + ix_bse * inp_sW] *\n bse;\n }\n }\n } else if (interpolation_mode == GridSamplerInterpolation::Nearest) {\n int ix_nearest = static_cast(::round(ix));\n int iy_nearest = static_cast(::round(iy));\n int iz_nearest = static_cast(::round(iz));\n\n // assign nearest neighbor pixel value to output pixel\n auto inp_ptr_NC = input + n * inp_sN;\n auto out_ptr_NCDHW =\n output + n * out_sN + d * out_sD + h * out_sH + w * out_sW;\n for (int c = 0; c < C;\n ++c, inp_ptr_NC += inp_sC, out_ptr_NCDHW += out_sC) {\n if (within_bounds_3d(iz_nearest, iy_nearest, ix_nearest, inp_D, inp_H,\n inp_W)) {\n *out_ptr_NCDHW =\n inp_ptr_NC[iz_nearest * inp_sD + iy_nearest * inp_sH +\n ix_nearest * inp_sW];\n } else {\n *out_ptr_NCDHW = static_cast(0);\n }\n }\n }\n }\n}\n\nvoid create_desc(const int *dims, int nb_dims, TensorDesc &desc) {\n memcpy(&desc.shape[0], dims, sizeof(int) * nb_dims);\n desc.stride[nb_dims - 1] = 1;\n for (int i = nb_dims - 2; i >= 0; --i) {\n desc.stride[i] = desc.stride[i + 1] * desc.shape[i + 1];\n }\n}\n\ntemplate \nvoid grid_sample(T *output, const T *input, const T *grid, int *output_dims,\n int *input_dims, int *grid_dims, int nb_dims,\n GridSamplerInterpolation interp, GridSamplerPadding padding,\n bool align_corners, cudaStream_t stream) {\n TensorDesc input_desc;\n create_desc(input_dims, nb_dims, input_desc);\n\n TensorDesc output_desc;\n create_desc(output_dims, nb_dims, output_desc);\n\n TensorDesc grid_desc;\n create_desc(grid_dims, nb_dims, grid_desc);\n\n int count = 1;\n for (int i = 0; i < nb_dims; ++i) {\n if (i == 1) {\n continue;\n }\n count *= output_desc.shape[i];\n }\n\n if (nb_dims == 4) {\n grid_sampler_2d_kernel\n <<>>(\n count, input, grid, output, input_desc, grid_desc, output_desc,\n interp, padding, align_corners);\n } else if (nb_dims == 5) {\n grid_sampler_3d_kernel\n <<>>(\n count, input, grid, output, input_desc, grid_desc, output_desc,\n interp, padding, align_corners);\n } else {\n printf(\"input and grid dims should be 4 or 5\\n\");\n }\n}\n\nvoid grid_sample_float(float *output, const float *input, const float *grid,\n int *output_dims, int *input_dims, int *grid_dims,\n int nb_dims, GridSamplerInterpolation interp,\n GridSamplerPadding padding, bool align_corners,\n cudaStream_t stream) {\n grid_sample(output, input, grid, output_dims, input_dims, grid_dims,\n nb_dims, interp, padding, align_corners, stream);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_instance_norm.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n// Modified from:\n// https://github.com/NVIDIA/TensorRT/blob/master/plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.cpp\n\n#include \"trt_instance_norm.hpp\"\n\n#include \n\n#include \n\n#include \"trt_serialize.hpp\"\n\nusing namespace nvinfer1;\n\ncudnnStatus_t convert_trt2cudnn_dtype(nvinfer1::DataType trt_dtype,\n cudnnDataType_t* cudnn_dtype) {\n switch (trt_dtype) {\n case nvinfer1::DataType::kFLOAT:\n *cudnn_dtype = CUDNN_DATA_FLOAT;\n break;\n case nvinfer1::DataType::kHALF:\n *cudnn_dtype = CUDNN_DATA_HALF;\n break;\n default:\n return CUDNN_STATUS_BAD_PARAM;\n }\n return CUDNN_STATUS_SUCCESS;\n}\n\nnamespace {\nconstexpr const char* PLUGIN_VERSION{\"1\"};\nconstexpr const char* PLUGIN_NAME{\"MMCVInstanceNormalization\"};\n} // namespace\n\nPluginFieldCollection InstanceNormalizationDynamicCreator::mFC{};\nstd::vector InstanceNormalizationDynamicCreator::mPluginAttributes;\n\nInstanceNormalizationDynamic::InstanceNormalizationDynamic(\n const std::string& name, float epsilon)\n : mLayerName(name), mEpsilon(epsilon) {}\n\nInstanceNormalizationDynamic::InstanceNormalizationDynamic(\n const std::string& name, void const* serialData, size_t serialLength)\n : mLayerName(name) {\n deserialize_value(&serialData, &serialLength, &mEpsilon);\n}\n\nInstanceNormalizationDynamic::~InstanceNormalizationDynamic() {}\n\n// InstanceNormalizationDynamic returns one output.\nint InstanceNormalizationDynamic::getNbOutputs() const { return 1; }\n\nDimsExprs InstanceNormalizationDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs* inputs, int nbInputs,\n nvinfer1::IExprBuilder& exprBuilder) {\n nvinfer1::DimsExprs output(inputs[0]);\n return output;\n}\n\nint InstanceNormalizationDynamic::initialize() { return 0; }\n\nvoid InstanceNormalizationDynamic::terminate() {}\n\nsize_t InstanceNormalizationDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc* inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc* outputs, int nbOutputs) const {\n int n = inputs[0].dims.d[0];\n int c = inputs[0].dims.d[1];\n int elem_size = mmcv::getElementSize(inputs[1].type);\n return mmcv::getAlignedSize(n * c * elem_size) * 2;\n}\n\nint InstanceNormalizationDynamic::enqueue(\n const nvinfer1::PluginTensorDesc* inputDesc,\n const nvinfer1::PluginTensorDesc* outputDesc, const void* const* inputs,\n void* const* outputs, void* workspace, cudaStream_t stream) {\n nvinfer1::Dims input_dims = inputDesc[0].dims;\n int n = input_dims.d[0];\n int c = input_dims.d[1];\n int h = input_dims.d[2];\n int w = input_dims.nbDims > 3 ? input_dims.d[3] : 1;\n int elem_size = mmcv::getElementSize(inputDesc[1].type);\n\n void* n_scales = (void*)workspace;\n void* n_bias = (void*)(workspace + mmcv::getAlignedSize(n * c * elem_size));\n\n const void* scales = (const void*)inputs[1];\n const void* bias = (const void*)inputs[2];\n\n for (int i = 0; i < n; ++i) {\n cudaMemcpyAsync(n_scales + i * c * elem_size, scales, c * elem_size,\n cudaMemcpyDeviceToDevice, stream);\n cudaMemcpyAsync(n_bias + i * c * elem_size, bias, c * elem_size,\n cudaMemcpyDeviceToDevice, stream);\n }\n\n cudnnSetTensor4dDescriptor(_b_desc, CUDNN_TENSOR_NCHW, CUDNN_DATA_FLOAT, 1,\n n * c, 1, 1);\n cudnnDataType_t cudnn_dtype{};\n convert_trt2cudnn_dtype(inputDesc[0].type, &cudnn_dtype);\n cudnnSetTensor4dDescriptor(_x_desc, CUDNN_TENSOR_NCHW, cudnn_dtype, 1, n * c,\n h, w);\n cudnnSetTensor4dDescriptor(_y_desc, CUDNN_TENSOR_NCHW, cudnn_dtype, 1, n * c,\n h, w);\n float alpha = 1;\n float beta = 0;\n void const* x_ptr = inputs[0];\n void* y_ptr = outputs[0];\n cudnnSetStream(_cudnn_handle, stream);\n // Note: Use of CUDNN_BATCHNORM_SPATIAL_PERSISTENT can cause numerical\n // overflows (NaNs) for fp32 data in some circumstances. The lower-\n // performance CUDNN_BATCHNORM_SPATIAL should be used if this is not\n // acceptable.\n cudnnBatchNormalizationForwardTraining(\n _cudnn_handle, CUDNN_BATCHNORM_SPATIAL_PERSISTENT, &alpha, &beta, _x_desc,\n x_ptr, _y_desc, y_ptr, _b_desc, n_scales, n_bias, 1., nullptr, nullptr,\n mEpsilon, nullptr, nullptr);\n return 0;\n}\n\nsize_t InstanceNormalizationDynamic::getSerializationSize() const {\n return serialized_size(mEpsilon);\n}\n\nvoid InstanceNormalizationDynamic::serialize(void* buffer) const {\n serialize_value(&buffer, mEpsilon);\n}\n\nbool InstanceNormalizationDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc* inOut, int nbInputs,\n int nbOutputs) {\n return ((inOut[pos].type == nvinfer1::DataType::kFLOAT ||\n inOut[pos].type == nvinfer1::DataType::kHALF) &&\n inOut[pos].format == nvinfer1::PluginFormat::kLINEAR &&\n inOut[pos].type == inOut[0].type);\n}\n\nconst char* InstanceNormalizationDynamic::getPluginType() const {\n return PLUGIN_NAME;\n}\n\nconst char* InstanceNormalizationDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nvoid InstanceNormalizationDynamic::destroy() { delete this; }\n\nIPluginV2DynamicExt* InstanceNormalizationDynamic::clone() const {\n auto* plugin = new InstanceNormalizationDynamic{mLayerName, mEpsilon};\n plugin->setPluginNamespace(mPluginNamespace.c_str());\n return plugin;\n}\n\n// Set plugin namespace\nvoid InstanceNormalizationDynamic::setPluginNamespace(\n const char* pluginNamespace) {\n mPluginNamespace = pluginNamespace;\n}\n\nconst char* InstanceNormalizationDynamic::getPluginNamespace() const {\n return mPluginNamespace.c_str();\n}\n\nnvinfer1::DataType InstanceNormalizationDynamic::getOutputDataType(\n int index, const nvinfer1::DataType* inputTypes, int nbInputs) const {\n return inputTypes[0];\n}\n\n// Attach the plugin object to an execution context and grant the plugin the\n// access to some context resource.\nvoid InstanceNormalizationDynamic::attachToContext(\n cudnnContext* cudnnContext, cublasContext* cublasContext,\n IGpuAllocator* gpuAllocator) {\n _cudnn_handle = cudnnContext;\n cudnnCreateTensorDescriptor(&_b_desc);\n cudnnCreateTensorDescriptor(&_x_desc);\n cudnnCreateTensorDescriptor(&_y_desc);\n}\n\n// Detach the plugin object from its execution context.\nvoid InstanceNormalizationDynamic::detachFromContext() {\n cudnnDestroyTensorDescriptor(_y_desc);\n cudnnDestroyTensorDescriptor(_x_desc);\n cudnnDestroyTensorDescriptor(_b_desc);\n}\n\nvoid InstanceNormalizationDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc* in, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc* out, int nbOutputs) {}\n\n// InstanceNormalizationDynamicCreator methods\nInstanceNormalizationDynamicCreator::InstanceNormalizationDynamicCreator() {\n mPluginAttributes.clear();\n mPluginAttributes.emplace_back(\n PluginField(\"epsilon\", nullptr, PluginFieldType::kFLOAT32, 1));\n\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char* InstanceNormalizationDynamicCreator::getPluginName() const {\n return PLUGIN_NAME;\n}\n\nconst char* InstanceNormalizationDynamicCreator::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nconst PluginFieldCollection*\nInstanceNormalizationDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nIPluginV2DynamicExt* InstanceNormalizationDynamicCreator::createPlugin(\n const char* name, const nvinfer1::PluginFieldCollection* fc) {\n float epsilon = 1e-5;\n const PluginField* fields = fc->fields;\n for (int i = 0; i < fc->nbFields; ++i) {\n const char* attrName = fields[i].name;\n if (!strcmp(attrName, \"epsilon\")) {\n epsilon = *(static_cast(fields[i].data));\n }\n }\n\n InstanceNormalizationDynamic* obj =\n new InstanceNormalizationDynamic(name, epsilon);\n obj->setPluginNamespace(mNamespace.c_str());\n return obj;\n}\n\nIPluginV2DynamicExt* InstanceNormalizationDynamicCreator::deserializePlugin(\n const char* name, const void* serialData, size_t serialLength) {\n InstanceNormalizationDynamic* obj =\n new InstanceNormalizationDynamic{name, serialData, serialLength};\n obj->setPluginNamespace(mNamespace.c_str());\n return obj;\n}\n\nvoid InstanceNormalizationDynamicCreator::setPluginNamespace(\n const char* libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char* InstanceNormalizationDynamicCreator::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_modulated_deform_conv.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_modulated_deform_conv.hpp\"\n\n#include \n\n#include \n\n#include \"trt_serialize.hpp\"\n\nvoid ModulatedDeformConvForwardCUDAKernelLauncher_float(\n const float *input, const float *weight, const float *bias,\n const float *offset, const float *mask, float *output, void *workspace,\n int batch, int channels, int height, int width, int channels_out,\n int kernel_w, int kernel_h, int stride_w, int stride_h, int pad_w,\n int pad_h, int dilation_w, int dilation_h, int group, int deformable_group,\n int im2col_step, cublasHandle_t cublas_handle, cudaStream_t stream);\n\nnamespace {\nstatic const char *PLUGIN_VERSION{\"1\"};\nstatic const char *PLUGIN_NAME{\"MMCVModulatedDeformConv2d\"};\n} // namespace\n\nnvinfer1::PluginFieldCollection\n ModulatedDeformableConvPluginDynamicCreator::mFC{};\nstd::vector\n ModulatedDeformableConvPluginDynamicCreator::mPluginAttributes;\n\nModulatedDeformableConvPluginDynamic::ModulatedDeformableConvPluginDynamic(\n const std::string &name, const nvinfer1::Dims stride,\n const nvinfer1::Dims padding, const nvinfer1::Dims dilation,\n const int deformableGroup, const int group)\n : mLayerName(name),\n mStride(stride),\n mPadding(padding),\n mDilation(dilation),\n mDeformableGroup(deformableGroup),\n mGroup(group) {\n mWithBias = false;\n}\n\nModulatedDeformableConvPluginDynamic::ModulatedDeformableConvPluginDynamic(\n const std::string name, const void *data, size_t length)\n : mLayerName(name) {\n deserialize_value(&data, &length, &mStride);\n deserialize_value(&data, &length, &mPadding);\n deserialize_value(&data, &length, &mDilation);\n deserialize_value(&data, &length, &mDeformableGroup);\n deserialize_value(&data, &length, &mGroup);\n mWithBias = false;\n}\nModulatedDeformableConvPluginDynamic::~ModulatedDeformableConvPluginDynamic() {}\n\nnvinfer1::IPluginV2DynamicExt *ModulatedDeformableConvPluginDynamic::clone()\n const {\n ModulatedDeformableConvPluginDynamic *plugin =\n new ModulatedDeformableConvPluginDynamic(\n mLayerName, mStride, mPadding, mDilation, mDeformableGroup, mGroup);\n plugin->setPluginNamespace(getPluginNamespace());\n\n return plugin;\n}\n\nnvinfer1::DimsExprs ModulatedDeformableConvPluginDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) {\n nvinfer1::DimsExprs ret;\n ret.nbDims = 4;\n ret.d[0] = inputs[0].d[0];\n ret.d[1] = inputs[3].d[0];\n\n ret.d[2] = inputs[1].d[2];\n ret.d[3] = inputs[1].d[3];\n\n return ret;\n}\n\nbool ModulatedDeformableConvPluginDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc *inOut, int nbInputs,\n int nbOutputs) {\n if (pos == 0) {\n return (inOut[pos].type == nvinfer1::DataType::kFLOAT &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR);\n\n } else {\n return inOut[pos].type == inOut[0].type &&\n inOut[pos].format == inOut[0].format;\n }\n}\n\nvoid ModulatedDeformableConvPluginDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *outputs, int nbOutputs) {\n if (nbInputs == 5) {\n mWithBias = true;\n }\n}\n\nsize_t ModulatedDeformableConvPluginDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs, int nbOutputs) const {\n int sizeof_dtype = mmcv::getElementSize(outputs[0].type);\n\n int batch_size = inputs[0].dims.d[0];\n int nInputPlane = inputs[0].dims.d[1];\n int inputHeight = inputs[0].dims.d[2];\n int inputWidth = inputs[0].dims.d[3];\n\n int nOutputPlane = outputs[0].dims.d[1];\n int outputHeight = outputs[0].dims.d[2];\n int outputWidth = outputs[0].dims.d[3];\n\n int kW = inputs[3].dims.d[2];\n int kH = inputs[3].dims.d[3];\n int im2col_step = std::min(32, batch_size);\n\n size_t col_size = mmcv::getAlignedSize(nInputPlane * kW * kH * outputHeight *\n outputWidth * sizeof_dtype);\n\n return col_size;\n}\n\nint ModulatedDeformableConvPluginDynamic::enqueue(\n const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc, const void *const *inputs,\n void *const *outputs, void *workSpace, cudaStream_t stream) {\n int batch = inputDesc[0].dims.d[0];\n int channels = inputDesc[0].dims.d[1];\n int height = inputDesc[0].dims.d[2];\n int width = inputDesc[0].dims.d[3];\n int channels_out = outputDesc[0].dims.d[1];\n int kernel_h = inputDesc[3].dims.d[2];\n int kernel_w = inputDesc[3].dims.d[3];\n\n const void *x = inputs[0];\n const void *offset = inputs[1];\n const void *mask = inputs[2];\n const void *weight = inputs[3];\n const void *bias = mWithBias ? inputs[4] : nullptr;\n void *output = outputs[0];\n int im2col_step = std::min(batch, 32);\n\n // TODO: add fp16 support\n auto data_type = inputDesc[0].type;\n switch (data_type) {\n case nvinfer1::DataType::kFLOAT:\n ModulatedDeformConvForwardCUDAKernelLauncher_float(\n (float *)x, (float *)weight, (float *)bias, (float *)offset,\n (float *)mask, (float *)output, workSpace, batch, channels, height,\n width, channels_out, kernel_w, kernel_h, mStride.d[0], mStride.d[1],\n mPadding.d[0], mPadding.d[1], mDilation.d[0], mDilation.d[1], mGroup,\n mDeformableGroup, im2col_step, m_cublas_handle, stream);\n break;\n default:\n return 1;\n break;\n }\n\n return 0;\n}\n\nnvinfer1::DataType ModulatedDeformableConvPluginDynamic::getOutputDataType(\n int index, const nvinfer1::DataType *inputTypes, int nbInputs) const {\n return inputTypes[0];\n}\n\n// IPluginV2 Methods\nconst char *ModulatedDeformableConvPluginDynamic::getPluginType() const {\n return PLUGIN_NAME;\n}\n\nconst char *ModulatedDeformableConvPluginDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nint ModulatedDeformableConvPluginDynamic::getNbOutputs() const { return 1; }\n\nint ModulatedDeformableConvPluginDynamic::initialize() { return 0; }\n\nvoid ModulatedDeformableConvPluginDynamic::terminate() {}\n\nsize_t ModulatedDeformableConvPluginDynamic::getSerializationSize() const {\n return sizeof(mStride) + sizeof(mPadding) + sizeof(mDilation) +\n sizeof(mDeformableGroup) + sizeof(mGroup);\n}\n\nvoid ModulatedDeformableConvPluginDynamic::serialize(void *buffer) const {\n serialize_value(&buffer, mStride);\n serialize_value(&buffer, mPadding);\n serialize_value(&buffer, mDilation);\n serialize_value(&buffer, mDeformableGroup);\n serialize_value(&buffer, mGroup);\n}\n\nvoid ModulatedDeformableConvPluginDynamic::destroy() {\n // This gets called when the network containing plugin is destroyed\n delete this;\n}\n\nvoid ModulatedDeformableConvPluginDynamic::attachToContext(\n cudnnContext *cudnnContext, cublasContext *cublasContext,\n nvinfer1::IGpuAllocator *gpuAllocator) {\n m_cublas_handle = cublasContext;\n}\n\nvoid ModulatedDeformableConvPluginDynamic::detachFromContext() {}\n\nvoid ModulatedDeformableConvPluginDynamic::setPluginNamespace(\n const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *ModulatedDeformableConvPluginDynamic::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\n////////////////////// creator /////////////////////////////\n\nModulatedDeformableConvPluginDynamicCreator::\n ModulatedDeformableConvPluginDynamicCreator() {\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"stride\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"padding\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"dilation\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"groups\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"deform_groups\"));\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char *ModulatedDeformableConvPluginDynamicCreator::getPluginName() const {\n return PLUGIN_NAME;\n}\n\nconst char *ModulatedDeformableConvPluginDynamicCreator::getPluginVersion()\n const {\n return PLUGIN_VERSION;\n}\n\nconst nvinfer1::PluginFieldCollection *\nModulatedDeformableConvPluginDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nnvinfer1::IPluginV2 *ModulatedDeformableConvPluginDynamicCreator::createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) {\n nvinfer1::Dims stride{2, {1, 1}};\n nvinfer1::Dims padding{2, {0, 0}};\n nvinfer1::Dims dilation{2, {1, 1}};\n int deformableGroup = 1;\n int group = 1;\n\n for (int i = 0; i < fc->nbFields; i++) {\n if (fc->fields[i].data == nullptr) {\n continue;\n }\n std::string field_name(fc->fields[i].name);\n\n if (field_name.compare(\"deformable_group\") == 0) {\n deformableGroup = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"group\") == 0) {\n group = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"stride\") == 0) {\n stride.nbDims = 2;\n stride.d[0] = static_cast(fc->fields[i].data)[0];\n stride.d[1] = static_cast(fc->fields[i].data)[1];\n }\n\n if (field_name.compare(\"padding\") == 0) {\n padding.nbDims = 2;\n padding.d[0] = static_cast(fc->fields[i].data)[0];\n padding.d[1] = static_cast(fc->fields[i].data)[1];\n }\n\n if (field_name.compare(\"dilation\") == 0) {\n dilation.nbDims = 2;\n dilation.d[0] = static_cast(fc->fields[i].data)[0];\n dilation.d[1] = static_cast(fc->fields[i].data)[1];\n }\n }\n\n ModulatedDeformableConvPluginDynamic *plugin =\n new ModulatedDeformableConvPluginDynamic(name, stride, padding, dilation,\n deformableGroup, group);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nnvinfer1::IPluginV2 *\nModulatedDeformableConvPluginDynamicCreator::deserializePlugin(\n const char *name, const void *serialData, size_t serialLength) {\n auto plugin =\n new ModulatedDeformableConvPluginDynamic(name, serialData, serialLength);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nvoid ModulatedDeformableConvPluginDynamicCreator::setPluginNamespace(\n const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *ModulatedDeformableConvPluginDynamicCreator::getPluginNamespace()\n const {\n return mNamespace.c_str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_modulated_deform_conv_kernel.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n\n#include \"common_cuda_helper.hpp\"\n#include \"modulated_deform_conv_cuda_kernel.cuh\"\n#include \"trt_cuda_helper.cuh\"\n#include \"trt_plugin_helper.hpp\"\n\ntemplate \nvoid trt_modulated_deformable_im2col(\n const T* data_im_, const T* data_offset_, const T* data_mask_,\n const int batch_size, const int channels, const int height_im,\n const int width_im, const int height_col, const int width_col,\n const int kernel_h, const int kenerl_w, const int pad_h, const int pad_w,\n const int stride_h, const int stride_w, const int dilation_h,\n const int dilation_w, const int deformable_group, T* data_col_,\n cudaStream_t stream) {\n // num_axes should be smaller than block size\n const int channel_per_deformable_group = channels / deformable_group;\n const int num_kernels = channels * batch_size * height_col * width_col;\n\n modulated_deformable_im2col_gpu_kernel\n <<>>(\n num_kernels, data_im_, data_offset_, data_mask_, height_im, width_im,\n kernel_h, kenerl_w, pad_h, pad_w, stride_h, stride_w, dilation_h,\n dilation_w, channel_per_deformable_group, batch_size, channels,\n deformable_group, height_col, width_col, data_col_);\n\n cudaCheckError();\n}\n\ntemplate \n__global__ void output_add_bias_kernel(scalar_t* output, const scalar_t* bias,\n size_t step_batch, size_t step_channel,\n size_t n) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n output[index] += bias[(index % step_batch) / step_channel];\n }\n}\n\ntemplate \nstatic void output_add_bias(scalar_t* output, const scalar_t* bias,\n size_t batch, size_t channel, size_t height,\n size_t width, cudaStream_t stream) {\n size_t step_channel = height * width;\n size_t step_batch = step_channel * channel;\n size_t n = step_batch * batch;\n output_add_bias_kernel<<>>(\n output, bias, step_batch, step_channel, n);\n}\n\ntemplate \nvoid ModulatedDeformConvForwardCUDAKernelLauncher(\n const scalar_t* input, const scalar_t* weight, const scalar_t* bias,\n const scalar_t* offset, const scalar_t* mask, scalar_t* output,\n void* workspace, int batch, int channels, int height, int width,\n int channels_out, int kernel_w, int kernel_h, int stride_w, int stride_h,\n int pad_w, int pad_h, int dilation_w, int dilation_h, int group,\n int deformable_group, int im2col_step, cublasHandle_t cublas_handle,\n cudaStream_t stream) {\n size_t sizeof_dtype = sizeof(scalar_t);\n bool with_bias = (bias != nullptr);\n\n im2col_step = std::min(int(batch), im2col_step);\n assert(batch % im2col_step == 0);\n const int channels_kernel = channels / group;\n\n const int height_out =\n (height + 2 * pad_h - (dilation_h * (kernel_h - 1) + 1)) / stride_h + 1;\n const int width_out =\n (width + 2 * pad_w - (dilation_w * (kernel_w - 1) + 1)) / stride_w + 1;\n\n scalar_t* columns = (scalar_t*)workspace;\n\n const size_t input_step = channels * height * width;\n const size_t offset_step =\n deformable_group * kernel_h * kernel_w * 2 * height * width;\n const size_t mask_step =\n deformable_group * kernel_h * kernel_w * height * width;\n const size_t out_step = channels_out * height_out * width_out;\n const size_t out_group_step = out_step / group;\n const size_t col_g_step =\n channels * kernel_w * kernel_h / group * height_out * width_out;\n const size_t weight_g_step =\n channels_out / group * channels / group * kernel_h * kernel_w;\n\n const int m = channels_out / group;\n const int n = height_out * width_out;\n const int k = channels / group * kernel_h * kernel_w;\n scalar_t alpha = 1.;\n scalar_t beta = 0.;\n\n for (int b = 0; b < batch; b++) {\n const scalar_t* input_start = input + b * input_step;\n const scalar_t* offset_start = offset + b * offset_step;\n const scalar_t* mask_start = mask + b * mask_step;\n trt_modulated_deformable_im2col(\n input_start, offset_start, mask_start, 1, channels, height, width,\n height_out, width_out, kernel_h, kernel_w, pad_h, pad_w, stride_h,\n stride_w, dilation_h, dilation_w, deformable_group, columns, stream);\n\n for (int g = 0; g < group; g++) {\n const scalar_t* weight_start = weight + g * weight_g_step;\n scalar_t* col_start = columns + g * col_g_step;\n scalar_t* out_buffer_start = output + b * out_step + g * out_group_step;\n\n // cudaMemsetAsync(out_buffer_start, 0, 1, stream);\n cublasGemmWrap(cublas_handle, CUBLAS_OP_N, CUBLAS_OP_N, n, m, k,\n &alpha, col_start, n, weight_start, k, &beta,\n out_buffer_start, n);\n cudaCheckError();\n }\n }\n\n if (with_bias) {\n output_add_bias(output, bias, batch, channels_out, height_out,\n width_out, stream);\n }\n}\n\nvoid ModulatedDeformConvForwardCUDAKernelLauncher_float(\n const float* input, const float* weight, const float* bias,\n const float* offset, const float* mask, float* output, void* workspace,\n int batch, int channels, int height, int width, int channels_out,\n int kernel_w, int kernel_h, int stride_w, int stride_h, int pad_w,\n int pad_h, int dilation_w, int dilation_h, int group, int deformable_group,\n int im2col_step, cublasHandle_t cublas_handle, cudaStream_t stream) {\n ModulatedDeformConvForwardCUDAKernelLauncher(\n input, weight, bias, offset, mask, output, workspace, batch, channels,\n height, width, channels_out, kernel_w, kernel_h, stride_w, stride_h,\n pad_w, pad_h, dilation_w, dilation_h, group, deformable_group,\n im2col_step, cublas_handle, stream);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_nms.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_nms.hpp\"\n\n#include \n#include \n\n#include \n\n#include \"trt_serialize.hpp\"\n\nextern size_t get_onnxnms_workspace_size(\n size_t num_batches, size_t spatial_dimension, size_t num_classes,\n size_t boxes_word_size, int center_point_box, size_t output_length);\n\nextern void TRTNMSCUDAKernelLauncher_float(\n const float *boxes, const float *scores,\n const int max_output_boxes_per_class, const float iou_threshold,\n const float score_threshold, const int offset, int *output,\n int center_point_box, int num_batches, int spatial_dimension,\n int num_classes, size_t output_length, void *workspace,\n cudaStream_t stream);\n\nnamespace {\nstatic const char *PLUGIN_VERSION{\"1\"};\nstatic const char *PLUGIN_NAME{\"NonMaxSuppression\"};\n} // namespace\n\nnvinfer1::PluginFieldCollection NonMaxSuppressionDynamicCreator::mFC{};\nstd::vector\n NonMaxSuppressionDynamicCreator::mPluginAttributes;\n\nNonMaxSuppressionDynamic::NonMaxSuppressionDynamic(\n const std::string &name, int centerPointBox, int maxOutputBoxesPerClass,\n float iouThreshold, float scoreThreshold, int offset)\n : mLayerName(name),\n mCenterPointBox(centerPointBox),\n mMaxOutputBoxesPerClass(maxOutputBoxesPerClass),\n mIouThreshold(iouThreshold),\n mScoreThreshold(scoreThreshold),\n mOffset(offset) {}\n\nNonMaxSuppressionDynamic::NonMaxSuppressionDynamic(const std::string name,\n const void *data,\n size_t length)\n : mLayerName(name) {\n deserialize_value(&data, &length, &mCenterPointBox);\n deserialize_value(&data, &length, &mMaxOutputBoxesPerClass);\n deserialize_value(&data, &length, &mIouThreshold);\n deserialize_value(&data, &length, &mScoreThreshold);\n deserialize_value(&data, &length, &mOffset);\n}\n\nnvinfer1::IPluginV2DynamicExt *NonMaxSuppressionDynamic::clone() const {\n NonMaxSuppressionDynamic *plugin = new NonMaxSuppressionDynamic(\n mLayerName, mCenterPointBox, mMaxOutputBoxesPerClass, mIouThreshold,\n mScoreThreshold, mOffset);\n plugin->setPluginNamespace(getPluginNamespace());\n\n return plugin;\n}\n\nnvinfer1::DimsExprs NonMaxSuppressionDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) {\n nvinfer1::DimsExprs ret;\n ret.nbDims = 2;\n auto num_batches = inputs[0].d[0];\n auto spatial_dimension = inputs[0].d[1];\n if (mMaxOutputBoxesPerClass > 0) {\n spatial_dimension = exprBuilder.operation(\n nvinfer1::DimensionOperation::kMIN, *spatial_dimension,\n *exprBuilder.constant(mMaxOutputBoxesPerClass));\n }\n auto num_classes = inputs[1].d[1];\n ret.d[0] = exprBuilder.operation(\n nvinfer1::DimensionOperation::kPROD, *num_batches,\n *exprBuilder.operation(nvinfer1::DimensionOperation::kPROD,\n *spatial_dimension, *num_classes));\n ret.d[1] = exprBuilder.constant(3);\n\n return ret;\n}\n\nbool NonMaxSuppressionDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc *inOut, int nbInputs,\n int nbOutputs) {\n if (pos < nbInputs) {\n switch (pos) {\n case 0:\n // boxes\n return inOut[pos].type == nvinfer1::DataType::kFLOAT &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR;\n case 1:\n // scores\n return inOut[pos].type == nvinfer1::DataType::kFLOAT &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR;\n default:\n return true;\n }\n } else {\n switch (pos - nbInputs) {\n case 0:\n // selected_indices\n return inOut[pos].type == nvinfer1::DataType::kINT32 &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR;\n default:\n return true;\n }\n }\n return true;\n}\n\nvoid NonMaxSuppressionDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *outputs, int nbOutputs) {}\n\nsize_t NonMaxSuppressionDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs, int nbOutputs) const {\n size_t boxes_word_size = mmcv::getElementSize(inputs[0].type);\n size_t num_batches = inputs[0].dims.d[0];\n size_t spatial_dimension = inputs[0].dims.d[1];\n size_t num_classes = inputs[1].dims.d[1];\n size_t output_length = outputs[0].dims.d[0];\n\n return get_onnxnms_workspace_size(num_batches, spatial_dimension, num_classes,\n boxes_word_size, mCenterPointBox,\n output_length);\n}\n\nint NonMaxSuppressionDynamic::enqueue(\n const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc, const void *const *inputs,\n void *const *outputs, void *workSpace, cudaStream_t stream) {\n int num_batches = inputDesc[0].dims.d[0];\n int spatial_dimension = inputDesc[0].dims.d[1];\n int num_classes = inputDesc[1].dims.d[1];\n int output_length = outputDesc[0].dims.d[0];\n\n const float *boxes = (const float *)inputs[0];\n const float *scores = (const float *)inputs[1];\n int *output = (int *)outputs[0];\n TRTNMSCUDAKernelLauncher_float(\n boxes, scores, mMaxOutputBoxesPerClass, mIouThreshold, mScoreThreshold,\n mOffset, output, mCenterPointBox, num_batches, spatial_dimension,\n num_classes, output_length, workSpace, stream);\n\n return 0;\n}\n\nnvinfer1::DataType NonMaxSuppressionDynamic::getOutputDataType(\n int index, const nvinfer1::DataType *inputTypes, int nbInputs) const {\n return nvinfer1::DataType::kINT32;\n}\n\n// IPluginV2 Methods\nconst char *NonMaxSuppressionDynamic::getPluginType() const {\n return PLUGIN_NAME;\n}\n\nconst char *NonMaxSuppressionDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nint NonMaxSuppressionDynamic::getNbOutputs() const { return 1; }\n\nint NonMaxSuppressionDynamic::initialize() { return 0; }\n\nvoid NonMaxSuppressionDynamic::terminate() {}\n\nsize_t NonMaxSuppressionDynamic::getSerializationSize() const {\n return sizeof(mCenterPointBox) + sizeof(mMaxOutputBoxesPerClass) +\n sizeof(mIouThreshold) + sizeof(mScoreThreshold) + sizeof(mOffset);\n}\n\nvoid NonMaxSuppressionDynamic::serialize(void *buffer) const {\n serialize_value(&buffer, mCenterPointBox);\n serialize_value(&buffer, mMaxOutputBoxesPerClass);\n serialize_value(&buffer, mIouThreshold);\n serialize_value(&buffer, mScoreThreshold);\n serialize_value(&buffer, mOffset);\n}\n\nvoid NonMaxSuppressionDynamic::destroy() {\n // This gets called when the network containing plugin is destroyed\n delete this;\n}\n\nvoid NonMaxSuppressionDynamic::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *NonMaxSuppressionDynamic::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\n////////////////////// creator /////////////////////////////\n\nNonMaxSuppressionDynamicCreator::NonMaxSuppressionDynamicCreator() {\n mPluginAttributes.clear();\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"center_point_box\"));\n mPluginAttributes.emplace_back(\n nvinfer1::PluginField(\"max_output_boxes_per_class\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"iou_threshold\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"score_threshold\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"offset\"));\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char *NonMaxSuppressionDynamicCreator::getPluginName() const {\n return PLUGIN_NAME;\n}\n\nconst char *NonMaxSuppressionDynamicCreator::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nconst nvinfer1::PluginFieldCollection *\nNonMaxSuppressionDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nnvinfer1::IPluginV2 *NonMaxSuppressionDynamicCreator::createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) {\n int centerPointBox = 0;\n int maxOutputBoxesPerClass = 0;\n float iouThreshold = 0.0f;\n float scoreThreshold = 0.0f;\n int offset = 0;\n\n for (int i = 0; i < fc->nbFields; i++) {\n if (fc->fields[i].data == nullptr) {\n continue;\n }\n std::string field_name(fc->fields[i].name);\n\n if (field_name.compare(\"center_point_box\") == 0) {\n centerPointBox = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"max_output_boxes_per_class\") == 0) {\n maxOutputBoxesPerClass = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"iou_threshold\") == 0) {\n iouThreshold = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"score_threshold\") == 0) {\n scoreThreshold = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"offset\") == 0) {\n offset = static_cast(fc->fields[i].data)[0];\n }\n }\n NonMaxSuppressionDynamic *plugin =\n new NonMaxSuppressionDynamic(name, centerPointBox, maxOutputBoxesPerClass,\n iouThreshold, scoreThreshold, offset);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nnvinfer1::IPluginV2 *NonMaxSuppressionDynamicCreator::deserializePlugin(\n const char *name, const void *serialData, size_t serialLength) {\n auto plugin = new NonMaxSuppressionDynamic(name, serialData, serialLength);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nvoid NonMaxSuppressionDynamicCreator::setPluginNamespace(\n const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *NonMaxSuppressionDynamicCreator::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_nms_kernel.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n#include \n#include \n#include \n#include \n\n#include \n#include \n#include \n\n#include \"common_cuda_helper.hpp\"\n#include \"nms_cuda_kernel.cuh\"\n#include \"trt_cuda_helper.cuh\"\n#include \"trt_plugin_helper.hpp\"\n\nstruct NMSBox {\n float box[4];\n};\n\nstruct nms_centerwh2xyxy {\n __host__ __device__ NMSBox operator()(const NMSBox box) {\n NMSBox out;\n out.box[0] = box.box[0] - box.box[2] / 2.0f;\n out.box[1] = box.box[1] - box.box[3] / 2.0f;\n out.box[2] = box.box[0] + box.box[2] / 2.0f;\n out.box[3] = box.box[1] + box.box[3] / 2.0f;\n return out;\n }\n};\n\nstruct nms_sbox_idle {\n const float* idle_box_;\n __host__ __device__ nms_sbox_idle(const float* idle_box) {\n idle_box_ = idle_box;\n }\n\n __host__ __device__ NMSBox operator()(const NMSBox box) {\n return {idle_box_[0], idle_box_[1], idle_box_[2], idle_box_[3]};\n }\n};\n\nstruct nms_score_threshold {\n float score_threshold_;\n __host__ __device__ nms_score_threshold(const float score_threshold) {\n score_threshold_ = score_threshold;\n }\n\n __host__ __device__ bool operator()(const float score) {\n return score < score_threshold_;\n }\n};\n\n__global__ void nms_reindex_kernel(int n, int* output, int* index_cache) {\n CUDA_1D_KERNEL_LOOP(index, n) {\n const int old_index = output[index * 3 + 2];\n output[index * 3 + 2] = index_cache[old_index];\n }\n}\n\n__global__ void mask_to_output_kernel(const unsigned long long* dev_mask,\n const int* index, int* output,\n int* output_count, int batch_id,\n int cls_id, int spatial_dimension,\n int col_blocks,\n int max_output_boxes_per_class) {\n extern __shared__ unsigned long long remv[];\n\n // fill remv with 0\n CUDA_1D_KERNEL_LOOP(i, col_blocks) { remv[i] = 0; }\n __syncthreads();\n\n int start = *output_count;\n int out_per_class_count = 0;\n for (int i = 0; i < spatial_dimension; i++) {\n const int nblock = i / threadsPerBlock;\n const int inblock = i % threadsPerBlock;\n if (!(remv[nblock] & (1ULL << inblock))) {\n if (threadIdx.x == 0) {\n output[start * 3 + 0] = batch_id;\n output[start * 3 + 1] = cls_id;\n output[start * 3 + 2] = index[i];\n start += 1;\n }\n out_per_class_count += 1;\n if (out_per_class_count >= max_output_boxes_per_class) {\n break;\n }\n __syncthreads();\n // set every overlap box with bit 1 in remv\n const unsigned long long* p = dev_mask + i * col_blocks;\n CUDA_1D_KERNEL_LOOP(j, col_blocks) {\n if (j >= nblock) {\n remv[j] |= p[j];\n }\n } // j\n __syncthreads();\n }\n } // i\n if (threadIdx.x == 0) {\n *output_count = start;\n }\n}\n\nsize_t get_onnxnms_workspace_size(size_t num_batches, size_t spatial_dimension,\n size_t num_classes, size_t boxes_word_size,\n int center_point_box, size_t output_length) {\n size_t boxes_xyxy_workspace = 0;\n if (center_point_box == 1) {\n boxes_xyxy_workspace = mmcv::getAlignedSize(\n num_batches * spatial_dimension * 4 * boxes_word_size);\n }\n size_t scores_workspace =\n mmcv::getAlignedSize(spatial_dimension * boxes_word_size);\n size_t boxes_workspace =\n mmcv::getAlignedSize(spatial_dimension * 4 * boxes_word_size);\n const int col_blocks =\n (spatial_dimension + threadsPerBlock - 1) / threadsPerBlock;\n size_t mask_workspace = mmcv::getAlignedSize(spatial_dimension * col_blocks *\n sizeof(unsigned long long));\n size_t index_template_workspace =\n mmcv::getAlignedSize(spatial_dimension * sizeof(int));\n size_t index_workspace =\n mmcv::getAlignedSize(spatial_dimension * sizeof(int));\n size_t count_workspace = mmcv::getAlignedSize(sizeof(int));\n return scores_workspace + boxes_xyxy_workspace + boxes_workspace +\n mask_workspace + index_template_workspace + index_workspace +\n count_workspace;\n}\n\n/**\n * Launch the NonMaxSuppression kernel\n *\n * The NMS will be performed on each batch/class, share the kernel implement\n * `nms_cuda`. For each batch/class, the `boxes_sorted` and `index_cache` will\n * be sorted by scores, boxes_sorted will be used in `nms_cuda` kernel. After\n * that, the output would be generated by `mask_to_output_kernel` with\n * `dev_mask` and `sorted_cache`.\n *\n * @param[in] bboxes with shape [num_batch, spatial_dimension, 4], input boxes\n * @param[in] scores with shape [num_batch, num_classes, spatial_dimension],\n * input scores\n * @param[in] max_output_boxes_per_class max output boxes per class\n * @param[in] iou_threshold threshold of iou\n * @param[in] score_threshold threshold of scores\n * @param[in] offset box offset, only 0 or 1 is valid\n * @param[out] output with shape [output_length, 3], each row contain index\n * (batch_id, class_id, boxes_id), filling -1 if result is not valid.\n * @param[in] center_point_box 0 if boxes is [left, top, right, bottom] 1 if\n * boxes is [center_x, center_y, width, height]\n * @param[in] num_batches batch size of boxes and scores\n * @param[in] spatial_dimension boxes numbers each batch\n * @param[in] num_classes class numbers\n * @param[in] output_length the max output rows\n * @param[in] workspace memory for all temporary variables.\n * @param[in] stream cuda stream\n */\nvoid TRTNMSCUDAKernelLauncher_float(const float* boxes, const float* scores,\n const int max_output_boxes_per_class,\n const float iou_threshold,\n const float score_threshold,\n const int offset, int* output,\n int center_point_box, int num_batches,\n int spatial_dimension, int num_classes,\n size_t output_length, void* workspace,\n cudaStream_t stream) {\n const int col_blocks =\n (spatial_dimension + threadsPerBlock - 1) / threadsPerBlock;\n float* boxes_sorted = (float*)workspace;\n workspace = static_cast(workspace) +\n mmcv::getAlignedSize(spatial_dimension * 4 * sizeof(float));\n\n float* boxes_xyxy = nullptr;\n if (center_point_box == 1) {\n boxes_xyxy = (float*)workspace;\n workspace = static_cast(workspace) +\n mmcv::getAlignedSize(num_batches * spatial_dimension * 4 *\n sizeof(float));\n thrust::transform(thrust::cuda::par.on(stream), (NMSBox*)boxes,\n (NMSBox*)(boxes + num_batches * spatial_dimension * 4),\n (NMSBox*)boxes_xyxy, nms_centerwh2xyxy());\n cudaCheckError();\n }\n\n float* scores_sorted = (float*)workspace;\n workspace = static_cast(workspace) +\n mmcv::getAlignedSize(spatial_dimension * sizeof(float));\n\n unsigned long long* dev_mask = (unsigned long long*)workspace;\n workspace = static_cast(workspace) +\n mmcv::getAlignedSize(spatial_dimension * col_blocks *\n sizeof(unsigned long long));\n\n int* index_cache = (int*)workspace;\n workspace = static_cast(workspace) +\n mmcv::getAlignedSize(spatial_dimension * sizeof(int));\n\n // generate sequence [0,1,2,3,4 ....]\n int* index_template = (int*)workspace;\n workspace = static_cast(workspace) +\n mmcv::getAlignedSize(spatial_dimension * sizeof(int));\n thrust::sequence(thrust::cuda::par.on(stream), index_template,\n index_template + spatial_dimension, 0);\n\n int max_output_boxes_per_class_cpu = max_output_boxes_per_class;\n if (max_output_boxes_per_class_cpu <= 0) {\n max_output_boxes_per_class_cpu = spatial_dimension;\n }\n\n int* output_count = (int*)workspace;\n workspace = static_cast(workspace) + mmcv::getAlignedSize(sizeof(int));\n cudaMemsetAsync(output_count, 0, sizeof(int), stream);\n\n // fill output with -1\n thrust::fill(thrust::cuda::par.on(stream), output, output + output_length * 3,\n -1);\n cudaCheckError();\n\n dim3 blocks(col_blocks, col_blocks);\n dim3 threads(threadsPerBlock);\n\n for (int batch_id = 0; batch_id < num_batches; ++batch_id) {\n for (int cls_id = 0; cls_id < num_classes; ++cls_id) {\n const int batch_cls_id = batch_id * num_classes + cls_id;\n\n // sort boxes by score\n cudaMemcpyAsync(scores_sorted, scores + batch_cls_id * spatial_dimension,\n spatial_dimension * sizeof(float),\n cudaMemcpyDeviceToDevice, stream);\n cudaCheckError();\n\n cudaMemcpyAsync(index_cache, index_template,\n spatial_dimension * sizeof(int), cudaMemcpyDeviceToDevice,\n stream);\n cudaCheckError();\n\n thrust::sort_by_key(thrust::cuda::par.on(stream), scores_sorted,\n scores_sorted + spatial_dimension, index_cache,\n thrust::greater());\n\n if (center_point_box == 1) {\n thrust::gather(thrust::cuda::par.on(stream), index_cache,\n index_cache + spatial_dimension,\n (NMSBox*)(boxes_xyxy + batch_id * spatial_dimension * 4),\n (NMSBox*)boxes_sorted);\n } else {\n thrust::gather(thrust::cuda::par.on(stream), index_cache,\n index_cache + spatial_dimension,\n (NMSBox*)(boxes + batch_id * spatial_dimension * 4),\n (NMSBox*)boxes_sorted);\n }\n\n cudaCheckError();\n\n if (score_threshold > 0.0f) {\n thrust::transform_if(\n thrust::cuda::par.on(stream), (NMSBox*)boxes_sorted,\n (NMSBox*)(boxes_sorted + spatial_dimension * 4), scores_sorted,\n (NMSBox*)boxes_sorted, nms_sbox_idle(boxes_sorted),\n nms_score_threshold(score_threshold));\n }\n\n nms_cuda<<>>(spatial_dimension, iou_threshold,\n offset, boxes_sorted, dev_mask);\n\n // will be performed when dev_mask is full.\n mask_to_output_kernel<<<1, threadsPerBlock,\n col_blocks * sizeof(unsigned long long),\n stream>>>(\n dev_mask, index_cache, output, output_count, batch_id, cls_id,\n spatial_dimension, col_blocks, max_output_boxes_per_class_cpu);\n } // cls_id\n } // batch_id\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_plugin.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_plugin.hpp\"\n\n#include \"trt_corner_pool.hpp\"\n#include \"trt_cummaxmin.hpp\"\n#include \"trt_deform_conv.hpp\"\n#include \"trt_grid_sampler.hpp\"\n#include \"trt_instance_norm.hpp\"\n#include \"trt_modulated_deform_conv.hpp\"\n#include \"trt_nms.hpp\"\n#include \"trt_roi_align.hpp\"\n#include \"trt_scatternd.hpp\"\n\nREGISTER_TENSORRT_PLUGIN(CumMaxPluginDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(CumMinPluginDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(GridSamplerDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(DeformableConvPluginDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(ModulatedDeformableConvPluginDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(NonMaxSuppressionDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(RoIAlignPluginDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(ONNXScatterNDDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(InstanceNormalizationDynamicCreator);\nREGISTER_TENSORRT_PLUGIN(CornerPoolPluginDynamicCreator);\n\nextern \"C\" {\nbool initLibMMCVInferPlugins() { return true; }\n} // extern \"C\"\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_roi_align.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_roi_align.hpp\"\n\n#include \n\n#include \n\n#include \"trt_serialize.hpp\"\n\nextern void TRTRoIAlignForwardCUDAKernelLauncher_float(\n const float *input, const float *rois, float *output, float *argmax_y,\n float *argmax_x, int output_size, int channels, int height, int width,\n int aligned_height, int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned, cudaStream_t stream);\n\nnamespace {\nstatic const char *PLUGIN_VERSION{\"1\"};\nstatic const char *PLUGIN_NAME{\"MMCVRoiAlign\"};\n} // namespace\n\nnvinfer1::PluginFieldCollection RoIAlignPluginDynamicCreator::mFC{};\nstd::vector\n RoIAlignPluginDynamicCreator::mPluginAttributes;\n\nRoIAlignPluginDynamic::RoIAlignPluginDynamic(const std::string &name,\n int outWidth, int outHeight,\n float spatialScale,\n int sampleRatio, int poolMode,\n bool aligned)\n : mLayerName(name),\n mOutWidth(outWidth),\n mOutHeight(outHeight),\n mSpatialScale(spatialScale),\n mSampleRatio(sampleRatio),\n mPoolMode(poolMode),\n mAligned(aligned) {}\n\nRoIAlignPluginDynamic::RoIAlignPluginDynamic(const std::string name,\n const void *data, size_t length)\n : mLayerName(name) {\n deserialize_value(&data, &length, &mOutWidth);\n deserialize_value(&data, &length, &mOutHeight);\n deserialize_value(&data, &length, &mSpatialScale);\n deserialize_value(&data, &length, &mSampleRatio);\n deserialize_value(&data, &length, &mPoolMode);\n deserialize_value(&data, &length, &mAligned);\n}\n\nnvinfer1::IPluginV2DynamicExt *RoIAlignPluginDynamic::clone() const {\n RoIAlignPluginDynamic *plugin = new RoIAlignPluginDynamic(\n mLayerName, mOutWidth, mOutHeight, mSpatialScale, mSampleRatio, mPoolMode,\n mAligned);\n plugin->setPluginNamespace(getPluginNamespace());\n\n return plugin;\n}\n\nnvinfer1::DimsExprs RoIAlignPluginDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) {\n nvinfer1::DimsExprs ret;\n ret.nbDims = 4;\n ret.d[0] = inputs[1].d[0];\n ret.d[1] = inputs[0].d[1];\n ret.d[2] = exprBuilder.constant(mOutHeight);\n ret.d[3] = exprBuilder.constant(mOutWidth);\n\n return ret;\n}\n\nbool RoIAlignPluginDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc *inOut, int nbInputs,\n int nbOutputs) {\n return inOut[pos].type == nvinfer1::DataType::kFLOAT &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR;\n}\n\nvoid RoIAlignPluginDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *outputs, int nbOutputs) {}\n\nsize_t RoIAlignPluginDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs, int nbOutputs) const {\n size_t output_size = 0;\n size_t word_size = 0;\n switch (mPoolMode) {\n case 0: // max\n output_size = outputs[0].dims.d[0] * outputs[0].dims.d[1] *\n outputs[0].dims.d[2] * outputs[0].dims.d[3];\n word_size = mmcv::getElementSize(outputs[0].type);\n return output_size * word_size * 2;\n break;\n case 1:\n return 0;\n break;\n default:\n return 0;\n }\n return 0;\n}\n\nint RoIAlignPluginDynamic::enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs,\n void *const *outputs, void *workSpace,\n cudaStream_t stream) {\n int channels = inputDesc[0].dims.d[1];\n int height = inputDesc[0].dims.d[2];\n int width = inputDesc[0].dims.d[3];\n\n int output_size = outputDesc[0].dims.d[0] * outputDesc[0].dims.d[1] *\n outputDesc[0].dims.d[2] * outputDesc[0].dims.d[3];\n int word_size = mmcv::getElementSize(outputDesc[0].type);\n\n const void *feat = inputs[0];\n const void *rois = inputs[1];\n void *output = outputs[0];\n void *argmax_y = nullptr;\n void *argmax_x = nullptr;\n\n switch (mPoolMode) {\n case 0: // max\n argmax_y = workSpace;\n argmax_x = argmax_y + output_size * word_size;\n break;\n case 1: // avg\n break;\n }\n\n switch (outputDesc[0].type) {\n case nvinfer1::DataType::kFLOAT:\n TRTRoIAlignForwardCUDAKernelLauncher_float(\n (const float *)feat, (const float *)rois, (float *)output,\n (float *)argmax_y, (float *)argmax_x, output_size, channels, height,\n width, mOutHeight, mOutWidth, mSpatialScale, mSampleRatio, mPoolMode,\n mAligned, stream);\n break;\n\n default:\n break;\n }\n\n return 0;\n}\n\nnvinfer1::DataType RoIAlignPluginDynamic::getOutputDataType(\n int index, const nvinfer1::DataType *inputTypes, int nbInputs) const {\n return inputTypes[0];\n}\n\n// IPluginV2 Methods\nconst char *RoIAlignPluginDynamic::getPluginType() const { return PLUGIN_NAME; }\n\nconst char *RoIAlignPluginDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nint RoIAlignPluginDynamic::getNbOutputs() const { return 1; }\n\nint RoIAlignPluginDynamic::initialize() { return 0; }\n\nvoid RoIAlignPluginDynamic::terminate() {}\n\nsize_t RoIAlignPluginDynamic::getSerializationSize() const {\n return sizeof(mOutWidth) + sizeof(mOutHeight) + sizeof(mSpatialScale) +\n sizeof(mSampleRatio) + sizeof(mPoolMode) + sizeof(mAligned);\n}\n\nvoid RoIAlignPluginDynamic::serialize(void *buffer) const {\n serialize_value(&buffer, mOutWidth);\n serialize_value(&buffer, mOutHeight);\n serialize_value(&buffer, mSpatialScale);\n serialize_value(&buffer, mSampleRatio);\n serialize_value(&buffer, mPoolMode);\n serialize_value(&buffer, mAligned);\n}\n\nvoid RoIAlignPluginDynamic::destroy() {\n // This gets called when the network containing plugin is destroyed\n delete this;\n}\n\nvoid RoIAlignPluginDynamic::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *RoIAlignPluginDynamic::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\n////////////////////// creator /////////////////////////////\n\nRoIAlignPluginDynamicCreator::RoIAlignPluginDynamicCreator() {\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"output_height\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"output_width\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"spatial_scale\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"sampling_ratio\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"mode\"));\n mPluginAttributes.emplace_back(nvinfer1::PluginField(\"aligned\"));\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char *RoIAlignPluginDynamicCreator::getPluginName() const {\n return PLUGIN_NAME;\n}\n\nconst char *RoIAlignPluginDynamicCreator::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nconst nvinfer1::PluginFieldCollection *\nRoIAlignPluginDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nnvinfer1::IPluginV2 *RoIAlignPluginDynamicCreator::createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) {\n int outWidth = 7;\n int outHeight = 7;\n float spatialScale = 1.0;\n int sampleRatio = 0;\n int poolMode = -1;\n bool aligned = true;\n for (int i = 0; i < fc->nbFields; i++) {\n if (fc->fields[i].data == nullptr) {\n continue;\n }\n std::string field_name(fc->fields[i].name);\n\n if (field_name.compare(\"output_height\") == 0) {\n outHeight = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"output_width\") == 0) {\n outWidth = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"spatial_scale\") == 0) {\n spatialScale = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"sampling_ratio\") == 0) {\n sampleRatio = static_cast(fc->fields[i].data)[0];\n }\n\n if (field_name.compare(\"mode\") == 0) {\n int data_size = fc->fields[i].length;\n const char *data_start = static_cast(fc->fields[i].data);\n std::string poolModeStr(data_start, data_size);\n if (poolModeStr == \"avg\") {\n poolMode = 1;\n } else if (poolModeStr == \"max\") {\n poolMode = 0;\n } else {\n std::cout << \"Unknown pool mode \\\"\" << poolModeStr << \"\\\".\"\n << std::endl;\n }\n assert(poolMode >= 0);\n }\n\n if (field_name.compare(\"aligned\") == 0) {\n int aligned_int = static_cast(fc->fields[i].data)[0];\n aligned = aligned_int != 0;\n }\n }\n\n assert(outHeight > 0);\n assert(outWidth > 0);\n assert(spatialScale > 0.);\n assert(poolMode >= 0);\n\n RoIAlignPluginDynamic *plugin = new RoIAlignPluginDynamic(\n name, outWidth, outHeight, spatialScale, sampleRatio, poolMode, aligned);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nnvinfer1::IPluginV2 *RoIAlignPluginDynamicCreator::deserializePlugin(\n const char *name, const void *serialData, size_t serialLength) {\n auto plugin = new RoIAlignPluginDynamic(name, serialData, serialLength);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nvoid RoIAlignPluginDynamicCreator::setPluginNamespace(\n const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *RoIAlignPluginDynamicCreator::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_roi_align_kernel.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"common_cuda_helper.hpp\"\n#include \"roi_align_cuda_kernel.cuh\"\n\ntemplate \nvoid TRTRoIAlignForwardCUDAKernelLauncher(\n const scalar_t* input, const scalar_t* rois, scalar_t* output,\n scalar_t* argmax_y, scalar_t* argmax_x, int output_size, int channels,\n int height, int width, int aligned_height, int aligned_width,\n scalar_t spatial_scale, int sampling_ratio, int pool_mode, bool aligned,\n cudaStream_t stream) {\n roi_align_forward_cuda_kernel\n <<>>(\n output_size, input, rois, output, argmax_y, argmax_x, aligned_height,\n aligned_width, static_cast(spatial_scale), sampling_ratio,\n pool_mode, aligned, channels, height, width);\n}\n\nvoid TRTRoIAlignForwardCUDAKernelLauncher_float(\n const float* input, const float* rois, float* output, float* argmax_y,\n float* argmax_x, int output_size, int channels, int height, int width,\n int aligned_height, int aligned_width, float spatial_scale,\n int sampling_ratio, int pool_mode, bool aligned, cudaStream_t stream) {\n TRTRoIAlignForwardCUDAKernelLauncher(\n input, rois, output, argmax_y, argmax_x, output_size, channels, height,\n width, aligned_height, aligned_width, spatial_scale, sampling_ratio,\n pool_mode, aligned, stream);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_scatternd.cpp", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \"trt_scatternd.hpp\"\n\n#include \n#include \n\n#include \n\n#include \"trt_serialize.hpp\"\n\nextern void TRTONNXScatterNDKernelLauncher_float(\n const float *data, const int *indices, const float *update, const int *dims,\n int nbDims, const int *indices_dims, int indice_nbDims, float *output,\n cudaStream_t stream);\n\nextern void TRTONNXScatterNDKernelLauncher_int32(\n const int *data, const int *indices, const int *update, const int *dims,\n int nbDims, const int *indices_dims, int indice_nbDims, int *output,\n cudaStream_t stream);\n\nnamespace {\nstatic const char *PLUGIN_VERSION{\"1\"};\nstatic const char *PLUGIN_NAME{\"ScatterND\"};\n} // namespace\n\nnvinfer1::PluginFieldCollection ONNXScatterNDDynamicCreator::mFC{};\nstd::vector\n ONNXScatterNDDynamicCreator::mPluginAttributes;\n\nONNXScatterNDDynamic::ONNXScatterNDDynamic(const std::string &name)\n : mLayerName(name) {}\n\nONNXScatterNDDynamic::ONNXScatterNDDynamic(const std::string name,\n const void *data, size_t length)\n : mLayerName(name) {}\n\nnvinfer1::IPluginV2DynamicExt *ONNXScatterNDDynamic::clone() const {\n ONNXScatterNDDynamic *plugin = new ONNXScatterNDDynamic(mLayerName);\n plugin->setPluginNamespace(getPluginNamespace());\n\n return plugin;\n}\n\nnvinfer1::DimsExprs ONNXScatterNDDynamic::getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) {\n return inputs[0];\n}\n\nbool ONNXScatterNDDynamic::supportsFormatCombination(\n int pos, const nvinfer1::PluginTensorDesc *inOut, int nbInputs,\n int nbOutputs) {\n if (pos < nbInputs) {\n switch (pos) {\n case 0:\n // data\n return (inOut[pos].type == nvinfer1::DataType::kFLOAT &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR) ||\n (inOut[pos].type == nvinfer1::DataType::kINT32 &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR);\n case 1:\n // indices\n return inOut[pos].type == nvinfer1::DataType::kINT32 &&\n inOut[pos].format == nvinfer1::TensorFormat::kLINEAR;\n case 2:\n // updates\n return inOut[pos].type == inOut[0].type &&\n inOut[pos].format == inOut[0].format;\n default:\n return true;\n }\n } else {\n switch (pos - nbInputs) {\n case 0:\n // output\n return inOut[pos].type == inOut[0].type &&\n inOut[pos].format == inOut[0].format;\n default:\n return true;\n }\n }\n return true;\n}\n\nvoid ONNXScatterNDDynamic::configurePlugin(\n const nvinfer1::DynamicPluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *outputs, int nbOutputs) {}\n\nsize_t ONNXScatterNDDynamic::getWorkspaceSize(\n const nvinfer1::PluginTensorDesc *inputs, int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs, int nbOutputs) const {\n return 0;\n}\n\nint ONNXScatterNDDynamic::enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs,\n void *const *outputs, void *workSpace,\n cudaStream_t stream) {\n const int *dims = &(inputDesc[0].dims.d[0]);\n const int *indices_dims = &(inputDesc[1].dims.d[0]);\n int nbDims = inputDesc[0].dims.nbDims;\n int indice_nbDims = inputDesc[1].dims.nbDims;\n\n const void *data = inputs[0];\n const void *indices = inputs[1];\n const void *update = inputs[2];\n void *output = outputs[0];\n\n auto data_type = inputDesc[0].type;\n\n switch (data_type) {\n case nvinfer1::DataType::kFLOAT:\n TRTONNXScatterNDKernelLauncher_float(\n (float *)data, (int *)indices, (float *)update, dims, nbDims,\n indices_dims, indice_nbDims, (float *)output, stream);\n break;\n\n case nvinfer1::DataType::kINT32:\n TRTONNXScatterNDKernelLauncher_int32(\n (int *)data, (int *)indices, (int *)update, dims, nbDims,\n indices_dims, indice_nbDims, (int *)output, stream);\n break;\n default:\n break;\n }\n\n return 0;\n}\n\nnvinfer1::DataType ONNXScatterNDDynamic::getOutputDataType(\n int index, const nvinfer1::DataType *inputTypes, int nbInputs) const {\n return inputTypes[0];\n}\n\n// IPluginV2 Methods\nconst char *ONNXScatterNDDynamic::getPluginType() const { return PLUGIN_NAME; }\n\nconst char *ONNXScatterNDDynamic::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nint ONNXScatterNDDynamic::getNbOutputs() const { return 1; }\n\nint ONNXScatterNDDynamic::initialize() { return 0; }\n\nvoid ONNXScatterNDDynamic::terminate() {}\n\nsize_t ONNXScatterNDDynamic::getSerializationSize() const { return 0; }\n\nvoid ONNXScatterNDDynamic::serialize(void *buffer) const {}\n\nvoid ONNXScatterNDDynamic::destroy() {\n // This gets called when the network containing plugin is destroyed\n delete this;\n}\n\nvoid ONNXScatterNDDynamic::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *ONNXScatterNDDynamic::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n\n////////////////////// creator /////////////////////////////\n\nONNXScatterNDDynamicCreator::ONNXScatterNDDynamicCreator() {\n mPluginAttributes.clear();\n mFC.nbFields = mPluginAttributes.size();\n mFC.fields = mPluginAttributes.data();\n}\n\nconst char *ONNXScatterNDDynamicCreator::getPluginName() const {\n return PLUGIN_NAME;\n}\n\nconst char *ONNXScatterNDDynamicCreator::getPluginVersion() const {\n return PLUGIN_VERSION;\n}\n\nconst nvinfer1::PluginFieldCollection *\nONNXScatterNDDynamicCreator::getFieldNames() {\n return &mFC;\n}\n\nnvinfer1::IPluginV2 *ONNXScatterNDDynamicCreator::createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) {\n ONNXScatterNDDynamic *plugin = new ONNXScatterNDDynamic(name);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nnvinfer1::IPluginV2 *ONNXScatterNDDynamicCreator::deserializePlugin(\n const char *name, const void *serialData, size_t serialLength) {\n auto plugin = new ONNXScatterNDDynamic(name, serialData, serialLength);\n plugin->setPluginNamespace(getPluginNamespace());\n return plugin;\n}\n\nvoid ONNXScatterNDDynamicCreator::setPluginNamespace(const char *libNamespace) {\n mNamespace = libNamespace;\n}\n\nconst char *ONNXScatterNDDynamicCreator::getPluginNamespace() const {\n return mNamespace.c_str();\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/plugins/trt_scatternd_kernel.cu", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#include \n\n#include \n\n#include \"common_cuda_helper.hpp\"\n#include \"trt_cuda_helper.cuh\"\n#include \"trt_plugin_helper.hpp\"\n\nstatic int const threadsPerBlock = sizeof(unsigned long long int) * 8;\n\nusing mmcv::TensorDesc;\n\ntemplate \n__global__ void onnx_scatternd_kernel(const int n, const int* indices,\n const T* update, T* output,\n TensorDesc tensor_desc,\n TensorDesc indice_desc) {\n const int indice_cols = indice_desc.shape[indice_desc.dim - 1];\n const int copy_stride = tensor_desc.stride[indice_cols - 1];\n const int* stride = &(tensor_desc.stride[0]);\n CUDA_1D_KERNEL_LOOP(index, n) {\n int output_offset = 0;\n const int* indices_current = indices + index * indice_cols;\n for (int i = 0; i < indice_cols; ++i) {\n output_offset += stride[i] * indices_current[i];\n }\n memcpy(output + output_offset, update + index * copy_stride,\n copy_stride * sizeof(T));\n }\n}\n\ntemplate \nvoid TRTONNXScatterNDKernelLauncher(const T* data, const int* indices,\n const T* update, const int* dims,\n int nbDims, const int* indices_dims,\n int indice_nbDims, T* output,\n cudaStream_t stream) {\n // fill tensordesc and initial\n TensorDesc tensor_desc;\n memset((void*)&tensor_desc, 0, sizeof(TensorDesc));\n tensor_desc.dim = nbDims;\n tensor_desc.shape[nbDims - 1] = dims[nbDims - 1];\n tensor_desc.stride[nbDims - 1] = 1;\n for (int i = nbDims - 2; i >= 0; --i) {\n tensor_desc.shape[i] = dims[i];\n tensor_desc.stride[i] = dims[i + 1] * tensor_desc.stride[i + 1];\n }\n const int data_size = tensor_desc.stride[0] * tensor_desc.shape[0];\n\n TensorDesc indice_desc;\n memset((void*)&indice_desc, 0, sizeof(TensorDesc));\n indice_desc.dim = indice_nbDims;\n indice_desc.shape[indice_nbDims - 1] = indices_dims[indice_nbDims - 1];\n indice_desc.stride[indice_nbDims - 1] = 1;\n for (int i = indice_nbDims - 2; i >= 0; --i) {\n indice_desc.shape[i] = indices_dims[i];\n indice_desc.stride[i] = indices_dims[i + 1] * indice_desc.stride[i + 1];\n }\n\n // output = np.copy(data)\n cudaMemcpyAsync(output, data, data_size * sizeof(T),\n cudaMemcpyDeviceToDevice);\n\n int num_update_indice = 1;\n for (int i = 0; i < indice_nbDims - 1; ++i) {\n num_update_indice *= indice_desc.shape[i];\n }\n // scatter\n const int col_block = GET_BLOCKS(num_update_indice, threadsPerBlock);\n onnx_scatternd_kernel<<>>(\n num_update_indice, indices, update, output, tensor_desc, indice_desc);\n}\n\nvoid TRTONNXScatterNDKernelLauncher_float(const float* data, const int* indices,\n const float* update, const int* dims,\n int nbDims, const int* indices_dims,\n int indice_nbDims, float* output,\n cudaStream_t stream) {\n TRTONNXScatterNDKernelLauncher(data, indices, update, dims, nbDims,\n indices_dims, indice_nbDims, output,\n stream);\n}\n\nvoid TRTONNXScatterNDKernelLauncher_int32(const int* data, const int* indices,\n const int* update, const int* dims,\n int nbDims, const int* indices_dims,\n int indice_nbDims, int* output,\n cudaStream_t stream) {\n TRTONNXScatterNDKernelLauncher(data, indices, update, dims, nbDims,\n indices_dims, indice_nbDims, output,\n stream);\n}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_corner_pool.hpp", "content": "#ifndef TRT_CORNER_POOL_HPP\n#define TRT_CORNER_POOL_HPP\n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\nenum TRT_CORNER_POOL_TYPE {\n TRT_TOP_POOL = 0,\n TRT_BOTTOM_POOL = 1,\n TRT_LEFT_POOL = 2,\n TRT_RIGHT_POOL = 3\n};\n\n// implement of CornerPool\nclass CornerPoolPluginDynamic : public nvinfer1::IPluginV2DynamicExt {\n public:\n CornerPoolPluginDynamic(const std::string &name,\n TRT_CORNER_POOL_TYPE poolType);\n\n CornerPoolPluginDynamic(const std::string name, const void *data,\n size_t length);\n\n CornerPoolPluginDynamic() = delete;\n\n ~CornerPoolPluginDynamic();\n\n // IPluginV2DynamicExt Methods\n nvinfer1::IPluginV2DynamicExt *clone() const override;\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) override;\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc *inOut,\n int nbInputs, int nbOutputs) override;\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc *in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *out,\n int nbOutputs) override;\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc *inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs,\n int nbOutputs) const override;\n int enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs, void *workspace,\n cudaStream_t stream) override;\n\n // IPluginV2Ext Methods\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType *inputTypes,\n int nbInputs) const override;\n\n // IPluginV2 Methods\n const char *getPluginType() const override;\n const char *getPluginVersion() const override;\n int getNbOutputs() const override;\n int initialize() override;\n void terminate() override;\n size_t getSerializationSize() const override;\n void serialize(void *buffer) const override;\n void destroy() override;\n void setPluginNamespace(const char *pluginNamespace) override;\n const char *getPluginNamespace() const override;\n\n protected:\n const std::string mLayerName;\n std::string mNamespace;\n\n TRT_CORNER_POOL_TYPE mPoolType;\n\n protected:\n // To prevent compiler warnings.\n using nvinfer1::IPluginV2DynamicExt::canBroadcastInputAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::configurePlugin;\n using nvinfer1::IPluginV2DynamicExt::enqueue;\n using nvinfer1::IPluginV2DynamicExt::getOutputDimensions;\n using nvinfer1::IPluginV2DynamicExt::getWorkspaceSize;\n using nvinfer1::IPluginV2DynamicExt::isOutputBroadcastAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::supportsFormat;\n};\n\n// CornerPool creator\nclass CornerPoolPluginDynamicCreator : public nvinfer1::IPluginCreator {\n public:\n CornerPoolPluginDynamicCreator();\n\n const char *getPluginName() const override;\n\n const char *getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection *getFieldNames() override;\n\n nvinfer1::IPluginV2 *createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) override;\n\n nvinfer1::IPluginV2 *deserializePlugin(const char *name,\n const void *serialData,\n size_t serialLength) override;\n\n void setPluginNamespace(const char *pluginNamespace) override;\n\n const char *getPluginNamespace() const override;\n\n protected:\n nvinfer1::PluginFieldCollection mFC;\n std::vector mPluginAttributes;\n std::string mNamespace;\n};\n\n#endif TRT_CORNER_POOL_HPP // TRT_CORNER_POOL_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_cuda_helper.cuh", "content": "// Copyright (c) OpenMMLab. All rights reserved\n#ifndef TRT_CUDA_HELPER_HPP\n#define TRT_CUDA_HELPER_HPP\n#include \n\n#define cudaCheckError() \\\n { \\\n cudaError_t e = cudaGetLastError(); \\\n if (e != cudaSuccess) { \\\n printf(\"Cuda failure %s:%d: '%s'\\n\", __FILE__, __LINE__, \\\n cudaGetErrorString(e)); \\\n exit(0); \\\n } \\\n }\n\n/**\n * Returns a view of the original tensor with its dimensions permuted.\n *\n * @param[out] dst pointer to the destination tensor\n * @param[in] src pointer to the source tensor\n * @param[in] src_size shape of the src tensor\n * @param[in] permute The desired ordering of dimensions\n * @param[in] src_dim dim of src tensor\n * @param[in] stream cuda stream handle\n */\ntemplate \nvoid memcpyPermute(scalar_t* dst, const scalar_t* src, int* src_size,\n int* permute, int src_dim, cudaStream_t stream = 0);\n\ntemplate \ncublasStatus_t cublasGemmWrap(cublasHandle_t handle, cublasOperation_t transa,\n cublasOperation_t transb, int m, int n, int k,\n const scalar_t* alpha, const scalar_t* A, int lda,\n const scalar_t* B, int ldb, const scalar_t* beta,\n scalar_t* C, int ldc) {\n return CUBLAS_STATUS_INTERNAL_ERROR;\n}\n\n#endif // TRT_CUDA_HELPER_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_cummaxmin.hpp", "content": "#ifndef TRT_CUMMAXMIN_HPP\n#define TRT_CUMMAXMIN_HPP\n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\nenum TRT_CUMCMPTYPE { TRT_CUMMAX = 0, TRT_CUMMIN = 1 };\n\n// implement of cummax and cummin\nclass CumMaxMinPluginDynamic : public nvinfer1::IPluginV2DynamicExt {\n public:\n CumMaxMinPluginDynamic(const std::string &name, int dim,\n TRT_CUMCMPTYPE cumType);\n\n CumMaxMinPluginDynamic(const std::string name, const void *data,\n size_t length);\n\n CumMaxMinPluginDynamic() = delete;\n\n ~CumMaxMinPluginDynamic();\n\n // IPluginV2DynamicExt Methods\n nvinfer1::IPluginV2DynamicExt *clone() const override;\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) override;\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc *inOut,\n int nbInputs, int nbOutputs) override;\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc *in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *out,\n int nbOutputs) override;\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc *inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs,\n int nbOutputs) const override;\n int enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs, void *workspace,\n cudaStream_t stream) override;\n\n // IPluginV2Ext Methods\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType *inputTypes,\n int nbInputs) const override;\n\n // IPluginV2 Methods\n const char *getPluginType() const override;\n const char *getPluginVersion() const override;\n int getNbOutputs() const override;\n int initialize() override;\n void terminate() override;\n size_t getSerializationSize() const override;\n void serialize(void *buffer) const override;\n void destroy() override;\n void setPluginNamespace(const char *pluginNamespace) override;\n const char *getPluginNamespace() const override;\n\n protected:\n const std::string mLayerName;\n std::string mNamespace;\n\n int mDim;\n TRT_CUMCMPTYPE mCumType;\n\n protected:\n // To prevent compiler warnings.\n using nvinfer1::IPluginV2DynamicExt::canBroadcastInputAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::configurePlugin;\n using nvinfer1::IPluginV2DynamicExt::enqueue;\n using nvinfer1::IPluginV2DynamicExt::getOutputDimensions;\n using nvinfer1::IPluginV2DynamicExt::getWorkspaceSize;\n using nvinfer1::IPluginV2DynamicExt::isOutputBroadcastAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::supportsFormat;\n};\n\n// cummax and cummin creator\nclass CumMaxMinPluginDynamicCreator : public nvinfer1::IPluginCreator {\n public:\n CumMaxMinPluginDynamicCreator(TRT_CUMCMPTYPE cumType);\n\n const char *getPluginName() const override;\n\n const char *getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection *getFieldNames() override;\n\n nvinfer1::IPluginV2 *createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) override;\n\n nvinfer1::IPluginV2 *deserializePlugin(const char *name,\n const void *serialData,\n size_t serialLength) override;\n\n void setPluginNamespace(const char *pluginNamespace) override;\n\n const char *getPluginNamespace() const override;\n\n protected:\n TRT_CUMCMPTYPE mCumType;\n nvinfer1::PluginFieldCollection mFC;\n std::vector mPluginAttributes;\n std::string mNamespace;\n};\n\n// cummax creator\nclass CumMaxPluginDynamicCreator : public CumMaxMinPluginDynamicCreator {\n public:\n CumMaxPluginDynamicCreator();\n const char *getPluginName() const override;\n};\n\n// cummin creator\nclass CumMinPluginDynamicCreator : public CumMaxMinPluginDynamicCreator {\n public:\n CumMinPluginDynamicCreator();\n const char *getPluginName() const override;\n};\n\n#endif TRT_CUMMAXMIN_HPP // TRT_CUMMAXMIN_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_deform_conv.hpp", "content": "#ifndef TRT_DEFORM_CONV_HPP\n#define TRT_DEFORM_CONV_HPP\n#include \n\n#include \n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\nclass DeformableConvPluginDynamic : public nvinfer1::IPluginV2DynamicExt {\n public:\n DeformableConvPluginDynamic(const std::string &name,\n const nvinfer1::Dims &stride,\n const nvinfer1::Dims &padding,\n const nvinfer1::Dims &dilation,\n const int deformableGroup, const int group,\n int im2colStep);\n\n DeformableConvPluginDynamic(const std::string name, const void *data,\n size_t length);\n\n DeformableConvPluginDynamic() = delete;\n\n ~DeformableConvPluginDynamic();\n\n // IPluginV2DynamicExt Methods\n nvinfer1::IPluginV2DynamicExt *clone() const override;\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) override;\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc *inOut,\n int nbInputs, int nbOutputs) override;\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc *in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *out,\n int nbOutputs) override;\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc *inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs,\n int nbOutputs) const override;\n int enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs, void *workspace,\n cudaStream_t stream) override;\n void attachToContext(cudnnContext *cudnnContext, cublasContext *cublasContext,\n nvinfer1::IGpuAllocator *gpuAllocator) override;\n void detachFromContext() override;\n\n // IPluginV2Ext Methods\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType *inputTypes,\n int nbInputs) const override;\n\n // IPluginV2 Methods\n const char *getPluginType() const override;\n const char *getPluginVersion() const override;\n int getNbOutputs() const override;\n int initialize() override;\n void terminate() override;\n size_t getSerializationSize() const override;\n void serialize(void *buffer) const override;\n void destroy() override;\n void setPluginNamespace(const char *pluginNamespace) override;\n const char *getPluginNamespace() const override;\n\n private:\n const std::string mLayerName;\n std::string mNamespace;\n\n nvinfer1::Dims mStride;\n nvinfer1::Dims mPadding;\n nvinfer1::Dims mDilation;\n int mDeformableGroup;\n int mGroup;\n int mIm2colStep;\n\n cublasHandle_t m_cublas_handle;\n\n protected:\n // To prevent compiler warnings.\n using nvinfer1::IPluginV2DynamicExt::canBroadcastInputAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::configurePlugin;\n using nvinfer1::IPluginV2DynamicExt::enqueue;\n using nvinfer1::IPluginV2DynamicExt::getOutputDimensions;\n using nvinfer1::IPluginV2DynamicExt::getWorkspaceSize;\n using nvinfer1::IPluginV2DynamicExt::isOutputBroadcastAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::supportsFormat;\n};\n\nclass DeformableConvPluginDynamicCreator : public nvinfer1::IPluginCreator {\n public:\n DeformableConvPluginDynamicCreator();\n\n const char *getPluginName() const override;\n\n const char *getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection *getFieldNames() override;\n\n nvinfer1::IPluginV2 *createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) override;\n\n nvinfer1::IPluginV2 *deserializePlugin(const char *name,\n const void *serialData,\n size_t serialLength) override;\n\n void setPluginNamespace(const char *pluginNamespace) override;\n\n const char *getPluginNamespace() const override;\n\n private:\n static nvinfer1::PluginFieldCollection mFC;\n static std::vector mPluginAttributes;\n std::string mNamespace;\n};\n#endif // TRT_DEFORM_CONV_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_grid_sampler.hpp", "content": "#ifndef TRT_GRID_SAMPLER_HPP\n#define TRT_GRID_SAMPLER_HPP\n#include \n\n#include \n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\nnamespace mmcv {\nenum class GridSamplerInterpolation { Bilinear, Nearest };\nenum class GridSamplerPadding { Zeros, Border, Reflection };\n} // namespace mmcv\n\nclass GridSamplerDynamic : public nvinfer1::IPluginV2DynamicExt {\n public:\n GridSamplerDynamic(const std::string &name, int mode, int paddingMode,\n bool alignCorners);\n\n GridSamplerDynamic(const std::string name, const void *data, size_t length);\n\n GridSamplerDynamic() = delete;\n\n // IPluginV2DynamicExt Methods\n nvinfer1::IPluginV2DynamicExt *clone() const override;\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) override;\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc *inOut,\n int nbInputs, int nbOutputs) override;\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc *in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *out,\n int nbOutputs) override;\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc *inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs,\n int nbOutputs) const override;\n int enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs, void *workspace,\n cudaStream_t stream) override;\n\n // IPluginV2Ext Methods\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType *inputTypes,\n int nbInputs) const override;\n\n // IPluginV2 Methods\n const char *getPluginType() const override;\n const char *getPluginVersion() const override;\n int getNbOutputs() const override;\n int initialize() override;\n void terminate() override;\n size_t getSerializationSize() const override;\n void serialize(void *buffer) const override;\n void destroy() override;\n void setPluginNamespace(const char *pluginNamespace) override;\n const char *getPluginNamespace() const override;\n\n private:\n const std::string mLayerName;\n std::string mNamespace;\n\n int mMode;\n int mPaddingMode;\n bool mAlignCorners;\n\n protected:\n // To prevent compiler warnings.\n using nvinfer1::IPluginV2DynamicExt::canBroadcastInputAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::configurePlugin;\n using nvinfer1::IPluginV2DynamicExt::enqueue;\n using nvinfer1::IPluginV2DynamicExt::getOutputDimensions;\n using nvinfer1::IPluginV2DynamicExt::getWorkspaceSize;\n using nvinfer1::IPluginV2DynamicExt::isOutputBroadcastAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::supportsFormat;\n};\n\nclass GridSamplerDynamicCreator : public nvinfer1::IPluginCreator {\n public:\n GridSamplerDynamicCreator();\n\n const char *getPluginName() const override;\n\n const char *getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection *getFieldNames() override;\n\n nvinfer1::IPluginV2 *createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) override;\n\n nvinfer1::IPluginV2 *deserializePlugin(const char *name,\n const void *serialData,\n size_t serialLength) override;\n\n void setPluginNamespace(const char *pluginNamespace) override;\n\n const char *getPluginNamespace() const override;\n\n private:\n static nvinfer1::PluginFieldCollection mFC;\n static std::vector mPluginAttributes;\n std::string mNamespace;\n};\n#endif // TRT_GRID_SAMPLER_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_instance_norm.hpp", "content": "// Modified from:\n// https://github.com/NVIDIA/TensorRT/blob/master/plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.h\n\n#ifndef TRT_INSTANCE_NORMALIZATION_PLUGIN_H\n#define TRT_INSTANCE_NORMALIZATION_PLUGIN_H\n#include \n\n#include \n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\ntypedef unsigned short half_type;\n\nclass InstanceNormalizationDynamic final\n : public nvinfer1::IPluginV2DynamicExt {\n public:\n InstanceNormalizationDynamic(const std::string& name, float epsilon);\n\n InstanceNormalizationDynamic(const std::string& name, void const* serialData,\n size_t serialLength);\n\n InstanceNormalizationDynamic() = delete;\n\n ~InstanceNormalizationDynamic() override;\n\n int getNbOutputs() const override;\n\n // DynamicExt plugins returns DimsExprs class instead of Dims\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs* inputs, int nbInputs,\n nvinfer1::IExprBuilder& exprBuilder) override;\n\n int initialize() override;\n\n void terminate() override;\n\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc* inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc* outputs,\n int nbOutputs) const override;\n\n int enqueue(const nvinfer1::PluginTensorDesc* inputDesc,\n const nvinfer1::PluginTensorDesc* outputDesc,\n const void* const* inputs, void* const* outputs, void* workspace,\n cudaStream_t stream) override;\n\n size_t getSerializationSize() const override;\n\n void serialize(void* buffer) const override;\n\n // DynamicExt plugin supportsFormat update.\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc* inOut,\n int nbInputs, int nbOutputs) override;\n\n const char* getPluginType() const override;\n\n const char* getPluginVersion() const override;\n\n void destroy() override;\n\n nvinfer1::IPluginV2DynamicExt* clone() const override;\n\n void setPluginNamespace(const char* pluginNamespace) override;\n\n const char* getPluginNamespace() const override;\n\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType* inputTypes,\n int nbInputs) const override;\n\n void attachToContext(cudnnContext* cudnn, cublasContext* cublas,\n nvinfer1::IGpuAllocator* allocator) override;\n\n void detachFromContext() override;\n\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc* in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc* out,\n int nbOutputs) override;\n\n private:\n const std::string mLayerName;\n float mEpsilon{};\n cudnnHandle_t _cudnn_handle{};\n cudnnTensorDescriptor_t _x_desc{}, _y_desc{}, _b_desc{};\n std::string mPluginNamespace{};\n};\n\nclass InstanceNormalizationDynamicCreator : public nvinfer1::IPluginCreator {\n public:\n InstanceNormalizationDynamicCreator();\n\n ~InstanceNormalizationDynamicCreator() override = default;\n\n const char* getPluginName() const override;\n\n const char* getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection* getFieldNames() override;\n\n nvinfer1::IPluginV2DynamicExt* createPlugin(\n const char* name, const nvinfer1::PluginFieldCollection* fc) override;\n\n nvinfer1::IPluginV2DynamicExt* deserializePlugin(\n const char* name, const void* serialData, size_t serialLength) override;\n\n void setPluginNamespace(const char* pluginNamespace) override;\n\n const char* getPluginNamespace() const override;\n\n private:\n static nvinfer1::PluginFieldCollection mFC;\n static std::vector mPluginAttributes;\n std::string mNamespace;\n};\n\n#endif // TRT_INSTANCE_NORMALIZATION_PLUGIN_H\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_modulated_deform_conv.hpp", "content": "#ifndef TRT_MODULATED_DEFORM_CONV_HPP\n#define TRT_MODULATED_DEFORM_CONV_HPP\n#include \n\n#include \n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\nclass ModulatedDeformableConvPluginDynamic\n : public nvinfer1::IPluginV2DynamicExt {\n public:\n ModulatedDeformableConvPluginDynamic(const std::string &name,\n const nvinfer1::Dims stride,\n const nvinfer1::Dims padding,\n const nvinfer1::Dims dilation,\n const int deformableGroup,\n const int group);\n\n ModulatedDeformableConvPluginDynamic(const std::string name, const void *data,\n size_t length);\n\n ModulatedDeformableConvPluginDynamic() = delete;\n\n ~ModulatedDeformableConvPluginDynamic();\n\n // IPluginV2DynamicExt Methods\n nvinfer1::IPluginV2DynamicExt *clone() const override;\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) override;\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc *inOut,\n int nbInputs, int nbOutputs) override;\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc *in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *out,\n int nbOutputs) override;\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc *inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs,\n int nbOutputs) const override;\n int enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs, void *workspace,\n cudaStream_t stream) override;\n void attachToContext(cudnnContext *cudnnContext, cublasContext *cublasContext,\n nvinfer1::IGpuAllocator *gpuAllocator) override;\n void detachFromContext() override;\n\n // IPluginV2Ext Methods\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType *inputTypes,\n int nbInputs) const override;\n\n // IPluginV2 Methods\n const char *getPluginType() const override;\n const char *getPluginVersion() const override;\n int getNbOutputs() const override;\n int initialize() override;\n void terminate() override;\n size_t getSerializationSize() const override;\n void serialize(void *buffer) const override;\n void destroy() override;\n void setPluginNamespace(const char *pluginNamespace) override;\n const char *getPluginNamespace() const override;\n\n private:\n const std::string mLayerName;\n std::string mNamespace;\n\n nvinfer1::Dims mStride;\n nvinfer1::Dims mPadding;\n nvinfer1::Dims mDilation;\n int mDeformableGroup;\n int mGroup;\n bool mWithBias;\n\n cublasHandle_t m_cublas_handle;\n\n protected:\n // To prevent compiler warnings.\n using nvinfer1::IPluginV2DynamicExt::canBroadcastInputAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::configurePlugin;\n using nvinfer1::IPluginV2DynamicExt::enqueue;\n using nvinfer1::IPluginV2DynamicExt::getOutputDimensions;\n using nvinfer1::IPluginV2DynamicExt::getWorkspaceSize;\n using nvinfer1::IPluginV2DynamicExt::isOutputBroadcastAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::supportsFormat;\n};\n\nclass ModulatedDeformableConvPluginDynamicCreator\n : public nvinfer1::IPluginCreator {\n public:\n ModulatedDeformableConvPluginDynamicCreator();\n\n const char *getPluginName() const override;\n\n const char *getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection *getFieldNames() override;\n\n nvinfer1::IPluginV2 *createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) override;\n\n nvinfer1::IPluginV2 *deserializePlugin(const char *name,\n const void *serialData,\n size_t serialLength) override;\n\n void setPluginNamespace(const char *pluginNamespace) override;\n\n const char *getPluginNamespace() const override;\n\n private:\n static nvinfer1::PluginFieldCollection mFC;\n static std::vector mPluginAttributes;\n std::string mNamespace;\n};\n#endif // TRT_MODULATED_DEFORM_CONV_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_nms.hpp", "content": "#ifndef TRT_NMS_HPP\n#define TRT_NMS_HPP\n#include \n\n#include \n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\nclass NonMaxSuppressionDynamic : public nvinfer1::IPluginV2DynamicExt {\n public:\n NonMaxSuppressionDynamic(const std::string &name, int centerPointBox,\n int maxOutputBoxesPerClass, float iouThreshold,\n float scoreThreshold, int offset);\n\n NonMaxSuppressionDynamic(const std::string name, const void *data,\n size_t length);\n\n NonMaxSuppressionDynamic() = delete;\n\n // IPluginV2DynamicExt Methods\n nvinfer1::IPluginV2DynamicExt *clone() const override;\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) override;\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc *inOut,\n int nbInputs, int nbOutputs) override;\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc *in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *out,\n int nbOutputs) override;\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc *inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs,\n int nbOutputs) const override;\n int enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs, void *workspace,\n cudaStream_t stream) override;\n\n // IPluginV2Ext Methods\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType *inputTypes,\n int nbInputs) const override;\n\n // IPluginV2 Methods\n const char *getPluginType() const override;\n const char *getPluginVersion() const override;\n int getNbOutputs() const override;\n int initialize() override;\n void terminate() override;\n size_t getSerializationSize() const override;\n void serialize(void *buffer) const override;\n void destroy() override;\n void setPluginNamespace(const char *pluginNamespace) override;\n const char *getPluginNamespace() const override;\n\n private:\n const std::string mLayerName;\n std::string mNamespace;\n\n int mCenterPointBox;\n int mMaxOutputBoxesPerClass;\n float mIouThreshold;\n float mScoreThreshold;\n int mOffset;\n\n protected:\n // To prevent compiler warnings.\n using nvinfer1::IPluginV2DynamicExt::canBroadcastInputAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::configurePlugin;\n using nvinfer1::IPluginV2DynamicExt::enqueue;\n using nvinfer1::IPluginV2DynamicExt::getOutputDimensions;\n using nvinfer1::IPluginV2DynamicExt::getWorkspaceSize;\n using nvinfer1::IPluginV2DynamicExt::isOutputBroadcastAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::supportsFormat;\n};\n\nclass NonMaxSuppressionDynamicCreator : public nvinfer1::IPluginCreator {\n public:\n NonMaxSuppressionDynamicCreator();\n\n const char *getPluginName() const override;\n\n const char *getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection *getFieldNames() override;\n\n nvinfer1::IPluginV2 *createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) override;\n\n nvinfer1::IPluginV2 *deserializePlugin(const char *name,\n const void *serialData,\n size_t serialLength) override;\n\n void setPluginNamespace(const char *pluginNamespace) override;\n\n const char *getPluginNamespace() const override;\n\n private:\n static nvinfer1::PluginFieldCollection mFC;\n static std::vector mPluginAttributes;\n std::string mNamespace;\n};\n#endif // TRT_NMS_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_plugin.hpp", "content": "#ifndef TRT_PLUGIN_HPP\n#define TRT_PLUGIN_HPP\n\nextern \"C\" {\nbool initLibMMCVInferPlugins();\n} // extern \"C\"\n#endif // TRT_PLUGIN_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_plugin_helper.hpp", "content": "#ifndef TRT_PLUGIN_HELPER_HPP\n#define TRT_PLUGIN_HELPER_HPP\n#include \n\n#include \"NvInferPlugin.h\"\n\nnamespace mmcv {\n\nconst int MAXTENSORDIMS = 10;\n\nstruct TensorDesc {\n int shape[MAXTENSORDIMS];\n int stride[MAXTENSORDIMS];\n int dim;\n};\n\ninline unsigned int getElementSize(nvinfer1::DataType t) {\n switch (t) {\n case nvinfer1::DataType::kINT32:\n return 4;\n case nvinfer1::DataType::kFLOAT:\n return 4;\n case nvinfer1::DataType::kHALF:\n return 2;\n // case nvinfer1::DataType::kBOOL:\n case nvinfer1::DataType::kINT8:\n return 1;\n default:\n throw std::runtime_error(\"Invalid DataType.\");\n }\n throw std::runtime_error(\"Invalid DataType.\");\n return 0;\n}\n\ninline size_t getAlignedSize(size_t origin_size, size_t aligned_number = 16) {\n return size_t((origin_size + aligned_number - 1) / aligned_number) *\n aligned_number;\n}\n\n} // namespace mmcv\n#endif // TRT_PLUGIN_HELPER_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_roi_align.hpp", "content": "#ifndef TRT_ROI_ALIGN_HPP\n#define TRT_ROI_ALIGN_HPP\n#include \n\n#include \n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\nclass RoIAlignPluginDynamic : public nvinfer1::IPluginV2DynamicExt {\n public:\n RoIAlignPluginDynamic(const std::string &name, int outWidth, int outHeight,\n float spatialScale, int sampleRatio, int poolMode,\n bool aligned);\n\n RoIAlignPluginDynamic(const std::string name, const void *data,\n size_t length);\n\n RoIAlignPluginDynamic() = delete;\n\n // IPluginV2DynamicExt Methods\n nvinfer1::IPluginV2DynamicExt *clone() const override;\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) override;\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc *inOut,\n int nbInputs, int nbOutputs) override;\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc *in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *out,\n int nbOutputs) override;\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc *inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs,\n int nbOutputs) const override;\n int enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs, void *workspace,\n cudaStream_t stream) override;\n\n // IPluginV2Ext Methods\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType *inputTypes,\n int nbInputs) const override;\n\n // IPluginV2 Methods\n const char *getPluginType() const override;\n const char *getPluginVersion() const override;\n int getNbOutputs() const override;\n int initialize() override;\n void terminate() override;\n size_t getSerializationSize() const override;\n void serialize(void *buffer) const override;\n void destroy() override;\n void setPluginNamespace(const char *pluginNamespace) override;\n const char *getPluginNamespace() const override;\n\n private:\n const std::string mLayerName;\n std::string mNamespace;\n\n int mOutWidth;\n int mOutHeight;\n float mSpatialScale;\n int mSampleRatio;\n int mPoolMode; // 1:avg 0:max\n bool mAligned;\n\n protected:\n // To prevent compiler warnings.\n using nvinfer1::IPluginV2DynamicExt::canBroadcastInputAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::configurePlugin;\n using nvinfer1::IPluginV2DynamicExt::enqueue;\n using nvinfer1::IPluginV2DynamicExt::getOutputDimensions;\n using nvinfer1::IPluginV2DynamicExt::getWorkspaceSize;\n using nvinfer1::IPluginV2DynamicExt::isOutputBroadcastAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::supportsFormat;\n};\n\nclass RoIAlignPluginDynamicCreator : public nvinfer1::IPluginCreator {\n public:\n RoIAlignPluginDynamicCreator();\n\n const char *getPluginName() const override;\n\n const char *getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection *getFieldNames() override;\n\n nvinfer1::IPluginV2 *createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) override;\n\n nvinfer1::IPluginV2 *deserializePlugin(const char *name,\n const void *serialData,\n size_t serialLength) override;\n\n void setPluginNamespace(const char *pluginNamespace) override;\n\n const char *getPluginNamespace() const override;\n\n private:\n static nvinfer1::PluginFieldCollection mFC;\n static std::vector mPluginAttributes;\n std::string mNamespace;\n};\n#endif // TRT_ROI_ALIGN_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_scatternd.hpp", "content": "#ifndef TRT_SCATTERND_HPP\n#define TRT_SCATTERND_HPP\n#include \n\n#include \n#include \n#include \n\n#include \"trt_plugin_helper.hpp\"\n\nclass ONNXScatterNDDynamic : public nvinfer1::IPluginV2DynamicExt {\n public:\n ONNXScatterNDDynamic(const std::string &name);\n\n ONNXScatterNDDynamic(const std::string name, const void *data, size_t length);\n\n ONNXScatterNDDynamic() = delete;\n\n // IPluginV2DynamicExt Methods\n nvinfer1::IPluginV2DynamicExt *clone() const override;\n nvinfer1::DimsExprs getOutputDimensions(\n int outputIndex, const nvinfer1::DimsExprs *inputs, int nbInputs,\n nvinfer1::IExprBuilder &exprBuilder) override;\n bool supportsFormatCombination(int pos,\n const nvinfer1::PluginTensorDesc *inOut,\n int nbInputs, int nbOutputs) override;\n void configurePlugin(const nvinfer1::DynamicPluginTensorDesc *in,\n int nbInputs,\n const nvinfer1::DynamicPluginTensorDesc *out,\n int nbOutputs) override;\n size_t getWorkspaceSize(const nvinfer1::PluginTensorDesc *inputs,\n int nbInputs,\n const nvinfer1::PluginTensorDesc *outputs,\n int nbOutputs) const override;\n int enqueue(const nvinfer1::PluginTensorDesc *inputDesc,\n const nvinfer1::PluginTensorDesc *outputDesc,\n const void *const *inputs, void *const *outputs, void *workspace,\n cudaStream_t stream) override;\n\n // IPluginV2Ext Methods\n nvinfer1::DataType getOutputDataType(int index,\n const nvinfer1::DataType *inputTypes,\n int nbInputs) const override;\n\n // IPluginV2 Methods\n const char *getPluginType() const override;\n const char *getPluginVersion() const override;\n int getNbOutputs() const override;\n int initialize() override;\n void terminate() override;\n size_t getSerializationSize() const override;\n void serialize(void *buffer) const override;\n void destroy() override;\n void setPluginNamespace(const char *pluginNamespace) override;\n const char *getPluginNamespace() const override;\n\n private:\n const std::string mLayerName;\n std::string mNamespace;\n\n protected:\n // To prevent compiler warnings.\n using nvinfer1::IPluginV2DynamicExt::canBroadcastInputAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::configurePlugin;\n using nvinfer1::IPluginV2DynamicExt::enqueue;\n using nvinfer1::IPluginV2DynamicExt::getOutputDimensions;\n using nvinfer1::IPluginV2DynamicExt::getWorkspaceSize;\n using nvinfer1::IPluginV2DynamicExt::isOutputBroadcastAcrossBatch;\n using nvinfer1::IPluginV2DynamicExt::supportsFormat;\n};\n\nclass ONNXScatterNDDynamicCreator : public nvinfer1::IPluginCreator {\n public:\n ONNXScatterNDDynamicCreator();\n\n const char *getPluginName() const override;\n\n const char *getPluginVersion() const override;\n\n const nvinfer1::PluginFieldCollection *getFieldNames() override;\n\n nvinfer1::IPluginV2 *createPlugin(\n const char *name, const nvinfer1::PluginFieldCollection *fc) override;\n\n nvinfer1::IPluginV2 *deserializePlugin(const char *name,\n const void *serialData,\n size_t serialLength) override;\n\n void setPluginNamespace(const char *pluginNamespace) override;\n\n const char *getPluginNamespace() const override;\n\n private:\n static nvinfer1::PluginFieldCollection mFC;\n static std::vector mPluginAttributes;\n std::string mNamespace;\n};\n#endif // TRT_SCATTERND_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/csrc/tensorrt/trt_serialize.hpp", "content": "// Modified from:\n// https://github.com/NVIDIA/TensorRT/blob/master/plugin/common/serialize.hpp\n\n#ifndef TRT_SERIALIZE_HPP\n#define TRT_SERIALIZE_HPP\n#include \n#include \n#include \n#include \n#include \nusing std::cerr;\nusing std::cout;\nusing std::endl;\n\ntemplate \ninline void serialize_value(void** buffer, T const& value);\n\ntemplate \ninline void deserialize_value(void const** buffer, size_t* buffer_size,\n T* value);\n\nnamespace {\n\ntemplate \nstruct Serializer {};\n\ntemplate \nstruct Serializer::value ||\n std::is_enum::value ||\n std::is_pod::value>::type> {\n static size_t serialized_size(T const& value) { return sizeof(T); }\n static void serialize(void** buffer, T const& value) {\n ::memcpy(*buffer, &value, sizeof(T));\n reinterpret_cast(*buffer) += sizeof(T);\n }\n static void deserialize(void const** buffer, size_t* buffer_size, T* value) {\n assert(*buffer_size >= sizeof(T));\n ::memcpy(value, *buffer, sizeof(T));\n reinterpret_cast(*buffer) += sizeof(T);\n *buffer_size -= sizeof(T);\n }\n};\n\ntemplate <>\nstruct Serializer {\n static size_t serialized_size(const char* value) { return strlen(value) + 1; }\n static void serialize(void** buffer, const char* value) {\n ::strcpy(static_cast(*buffer), value);\n reinterpret_cast(*buffer) += strlen(value) + 1;\n }\n static void deserialize(void const** buffer, size_t* buffer_size,\n const char** value) {\n *value = static_cast(*buffer);\n size_t data_size = strnlen(*value, *buffer_size) + 1;\n assert(*buffer_size >= data_size);\n reinterpret_cast(*buffer) += data_size;\n *buffer_size -= data_size;\n }\n};\n\ntemplate \nstruct Serializer,\n typename std::enable_if::value ||\n std::is_enum::value ||\n std::is_pod::value>::type> {\n static size_t serialized_size(std::vector const& value) {\n return sizeof(value.size()) + value.size() * sizeof(T);\n }\n static void serialize(void** buffer, std::vector const& value) {\n serialize_value(buffer, value.size());\n size_t nbyte = value.size() * sizeof(T);\n ::memcpy(*buffer, value.data(), nbyte);\n reinterpret_cast(*buffer) += nbyte;\n }\n static void deserialize(void const** buffer, size_t* buffer_size,\n std::vector* value) {\n size_t size;\n deserialize_value(buffer, buffer_size, &size);\n value->resize(size);\n size_t nbyte = value->size() * sizeof(T);\n assert(*buffer_size >= nbyte);\n ::memcpy(value->data(), *buffer, nbyte);\n reinterpret_cast(*buffer) += nbyte;\n *buffer_size -= nbyte;\n }\n};\n\n} // namespace\n\ntemplate \ninline size_t serialized_size(T const& value) {\n return Serializer::serialized_size(value);\n}\n\ntemplate \ninline void serialize_value(void** buffer, T const& value) {\n return Serializer::serialize(buffer, value);\n}\n\ntemplate \ninline void deserialize_value(void const** buffer, size_t* buffer_size,\n T* value) {\n return Serializer::deserialize(buffer, buffer_size, value);\n}\n#endif // TRT_SERIALIZE_HPP\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/deform_conv.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom typing import Tuple, Union\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch import Tensor\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\nfrom torch.nn.modules.utils import _pair, _single\n\nfrom mmcv.utils import deprecated_api_warning\nfrom ..cnn import CONV_LAYERS\nfrom ..utils import ext_loader, print_log\n\next_module = ext_loader.load_ext('_ext', [\n 'deform_conv_forward', 'deform_conv_backward_input',\n 'deform_conv_backward_parameters'\n])\n\n\nclass DeformConv2dFunction(Function):\n\n @staticmethod\n def symbolic(g,\n input,\n offset,\n weight,\n stride,\n padding,\n dilation,\n groups,\n deform_groups,\n bias=False,\n im2col_step=32):\n return g.op(\n 'mmcv::MMCVDeformConv2d',\n input,\n offset,\n weight,\n stride_i=stride,\n padding_i=padding,\n dilation_i=dilation,\n groups_i=groups,\n deform_groups_i=deform_groups,\n bias_i=bias,\n im2col_step_i=im2col_step)\n\n @staticmethod\n def forward(ctx,\n input,\n offset,\n weight,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n deform_groups=1,\n bias=False,\n im2col_step=32):\n if input is not None and input.dim() != 4:\n raise ValueError(\n f'Expected 4D tensor as input, got {input.dim()}D tensor \\\n instead.')\n assert bias is False, 'Only support bias is False.'\n ctx.stride = _pair(stride)\n ctx.padding = _pair(padding)\n ctx.dilation = _pair(dilation)\n ctx.groups = groups\n ctx.deform_groups = deform_groups\n ctx.im2col_step = im2col_step\n\n # When pytorch version >= 1.6.0, amp is adopted for fp16 mode;\n # amp won't cast the type of model (float32), but \"offset\" is cast\n # to float16 by nn.Conv2d automatically, leading to the type\n # mismatch with input (when it is float32) or weight.\n # The flag for whether to use fp16 or amp is the type of \"offset\",\n # we cast weight and input to temporarily support fp16 and amp\n # whatever the pytorch version is.\n input = input.type_as(offset)\n weight = weight.type_as(input)\n ctx.save_for_backward(input, offset, weight)\n\n output = input.new_empty(\n DeformConv2dFunction._output_size(ctx, input, weight))\n\n ctx.bufs_ = [input.new_empty(0), input.new_empty(0)] # columns, ones\n\n cur_im2col_step = min(ctx.im2col_step, input.size(0))\n assert (input.size(0) % cur_im2col_step\n ) == 0, 'batch size must be divisible by im2col_step'\n ext_module.deform_conv_forward(\n input,\n weight,\n offset,\n output,\n ctx.bufs_[0],\n ctx.bufs_[1],\n kW=weight.size(3),\n kH=weight.size(2),\n dW=ctx.stride[1],\n dH=ctx.stride[0],\n padW=ctx.padding[1],\n padH=ctx.padding[0],\n dilationW=ctx.dilation[1],\n dilationH=ctx.dilation[0],\n group=ctx.groups,\n deformable_group=ctx.deform_groups,\n im2col_step=cur_im2col_step)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n input, offset, weight = ctx.saved_tensors\n\n grad_input = grad_offset = grad_weight = None\n\n cur_im2col_step = min(ctx.im2col_step, input.size(0))\n assert (input.size(0) % cur_im2col_step\n ) == 0, 'batch size must be divisible by im2col_step'\n\n grad_output = grad_output.contiguous()\n if ctx.needs_input_grad[0] or ctx.needs_input_grad[1]:\n grad_input = torch.zeros_like(input)\n grad_offset = torch.zeros_like(offset)\n ext_module.deform_conv_backward_input(\n input,\n offset,\n grad_output,\n grad_input,\n grad_offset,\n weight,\n ctx.bufs_[0],\n kW=weight.size(3),\n kH=weight.size(2),\n dW=ctx.stride[1],\n dH=ctx.stride[0],\n padW=ctx.padding[1],\n padH=ctx.padding[0],\n dilationW=ctx.dilation[1],\n dilationH=ctx.dilation[0],\n group=ctx.groups,\n deformable_group=ctx.deform_groups,\n im2col_step=cur_im2col_step)\n\n if ctx.needs_input_grad[2]:\n grad_weight = torch.zeros_like(weight)\n ext_module.deform_conv_backward_parameters(\n input,\n offset,\n grad_output,\n grad_weight,\n ctx.bufs_[0],\n ctx.bufs_[1],\n kW=weight.size(3),\n kH=weight.size(2),\n dW=ctx.stride[1],\n dH=ctx.stride[0],\n padW=ctx.padding[1],\n padH=ctx.padding[0],\n dilationW=ctx.dilation[1],\n dilationH=ctx.dilation[0],\n group=ctx.groups,\n deformable_group=ctx.deform_groups,\n scale=1,\n im2col_step=cur_im2col_step)\n\n return grad_input, grad_offset, grad_weight, \\\n None, None, None, None, None, None, None\n\n @staticmethod\n def _output_size(ctx, input, weight):\n channels = weight.size(0)\n output_size = (input.size(0), channels)\n for d in range(input.dim() - 2):\n in_size = input.size(d + 2)\n pad = ctx.padding[d]\n kernel = ctx.dilation[d] * (weight.size(d + 2) - 1) + 1\n stride_ = ctx.stride[d]\n output_size += ((in_size + (2 * pad) - kernel) // stride_ + 1, )\n if not all(map(lambda s: s > 0, output_size)):\n raise ValueError(\n 'convolution input is too small (output would be ' +\n 'x'.join(map(str, output_size)) + ')')\n return output_size\n\n\ndeform_conv2d = DeformConv2dFunction.apply\n\n\nclass DeformConv2d(nn.Module):\n r\"\"\"Deformable 2D convolution.\n\n Applies a deformable 2D convolution over an input signal composed of\n several input planes. DeformConv2d was described in the paper\n `Deformable Convolutional Networks\n `_\n\n Note:\n The argument ``im2col_step`` was added in version 1.3.17, which means\n number of samples processed by the ``im2col_cuda_kernel`` per call.\n It enables users to define ``batch_size`` and ``im2col_step`` more\n flexibly and solved `issue mmcv#1440\n `_.\n\n Args:\n in_channels (int): Number of channels in the input image.\n out_channels (int): Number of channels produced by the convolution.\n kernel_size(int, tuple): Size of the convolving kernel.\n stride(int, tuple): Stride of the convolution. Default: 1.\n padding (int or tuple): Zero-padding added to both sides of the input.\n Default: 0.\n dilation (int or tuple): Spacing between kernel elements. Default: 1.\n groups (int): Number of blocked connections from input.\n channels to output channels. Default: 1.\n deform_groups (int): Number of deformable group partitions.\n bias (bool): If True, adds a learnable bias to the output.\n Default: False.\n im2col_step (int): Number of samples processed by im2col_cuda_kernel\n per call. It will work when ``batch_size`` > ``im2col_step``, but\n ``batch_size`` must be divisible by ``im2col_step``. Default: 32.\n `New in version 1.3.17.`\n \"\"\"\n\n @deprecated_api_warning({'deformable_groups': 'deform_groups'},\n cls_name='DeformConv2d')\n def __init__(self,\n in_channels: int,\n out_channels: int,\n kernel_size: Union[int, Tuple[int, ...]],\n stride: Union[int, Tuple[int, ...]] = 1,\n padding: Union[int, Tuple[int, ...]] = 0,\n dilation: Union[int, Tuple[int, ...]] = 1,\n groups: int = 1,\n deform_groups: int = 1,\n bias: bool = False,\n im2col_step: int = 32) -> None:\n super(DeformConv2d, self).__init__()\n\n assert not bias, \\\n f'bias={bias} is not supported in DeformConv2d.'\n assert in_channels % groups == 0, \\\n f'in_channels {in_channels} cannot be divisible by groups {groups}'\n assert out_channels % groups == 0, \\\n f'out_channels {out_channels} cannot be divisible by groups \\\n {groups}'\n\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.kernel_size = _pair(kernel_size)\n self.stride = _pair(stride)\n self.padding = _pair(padding)\n self.dilation = _pair(dilation)\n self.groups = groups\n self.deform_groups = deform_groups\n self.im2col_step = im2col_step\n # enable compatibility with nn.Conv2d\n self.transposed = False\n self.output_padding = _single(0)\n\n # only weight, no bias\n self.weight = nn.Parameter(\n torch.Tensor(out_channels, in_channels // self.groups,\n *self.kernel_size))\n\n self.reset_parameters()\n\n def reset_parameters(self):\n # switch the initialization of `self.weight` to the standard kaiming\n # method described in `Delving deep into rectifiers: Surpassing\n # human-level performance on ImageNet classification` - He, K. et al.\n # (2015), using a uniform distribution\n nn.init.kaiming_uniform_(self.weight, nonlinearity='relu')\n\n def forward(self, x: Tensor, offset: Tensor) -> Tensor:\n \"\"\"Deformable Convolutional forward function.\n\n Args:\n x (Tensor): Input feature, shape (B, C_in, H_in, W_in)\n offset (Tensor): Offset for deformable convolution, shape\n (B, deform_groups*kernel_size[0]*kernel_size[1]*2,\n H_out, W_out), H_out, W_out are equal to the output's.\n\n An offset is like `[y0, x0, y1, x1, y2, x2, ..., y8, x8]`.\n The spatial arrangement is like:\n\n .. code:: text\n\n (x0, y0) (x1, y1) (x2, y2)\n (x3, y3) (x4, y4) (x5, y5)\n (x6, y6) (x7, y7) (x8, y8)\n\n Returns:\n Tensor: Output of the layer.\n \"\"\"\n # To fix an assert error in deform_conv_cuda.cpp:128\n # input image is smaller than kernel\n input_pad = (x.size(2) < self.kernel_size[0]) or (x.size(3) <\n self.kernel_size[1])\n if input_pad:\n pad_h = max(self.kernel_size[0] - x.size(2), 0)\n pad_w = max(self.kernel_size[1] - x.size(3), 0)\n x = F.pad(x, (0, pad_w, 0, pad_h), 'constant', 0).contiguous()\n offset = F.pad(offset, (0, pad_w, 0, pad_h), 'constant', 0)\n offset = offset.contiguous()\n out = deform_conv2d(x, offset, self.weight, self.stride, self.padding,\n self.dilation, self.groups, self.deform_groups,\n False, self.im2col_step)\n if input_pad:\n out = out[:, :, :out.size(2) - pad_h, :out.size(3) -\n pad_w].contiguous()\n return out\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'(in_channels={self.in_channels},\\n'\n s += f'out_channels={self.out_channels},\\n'\n s += f'kernel_size={self.kernel_size},\\n'\n s += f'stride={self.stride},\\n'\n s += f'padding={self.padding},\\n'\n s += f'dilation={self.dilation},\\n'\n s += f'groups={self.groups},\\n'\n s += f'deform_groups={self.deform_groups},\\n'\n # bias is not supported in DeformConv2d.\n s += 'bias=False)'\n return s\n\n\n@CONV_LAYERS.register_module('DCN')\nclass DeformConv2dPack(DeformConv2d):\n \"\"\"A Deformable Conv Encapsulation that acts as normal Conv layers.\n\n The offset tensor is like `[y0, x0, y1, x1, y2, x2, ..., y8, x8]`.\n The spatial arrangement is like:\n\n .. code:: text\n\n (x0, y0) (x1, y1) (x2, y2)\n (x3, y3) (x4, y4) (x5, y5)\n (x6, y6) (x7, y7) (x8, y8)\n\n Args:\n in_channels (int): Same as nn.Conv2d.\n out_channels (int): Same as nn.Conv2d.\n kernel_size (int or tuple[int]): Same as nn.Conv2d.\n stride (int or tuple[int]): Same as nn.Conv2d.\n padding (int or tuple[int]): Same as nn.Conv2d.\n dilation (int or tuple[int]): Same as nn.Conv2d.\n groups (int): Same as nn.Conv2d.\n bias (bool or str): If specified as `auto`, it will be decided by the\n norm_cfg. Bias will be set as True if norm_cfg is None, otherwise\n False.\n \"\"\"\n\n _version = 2\n\n def __init__(self, *args, **kwargs):\n super(DeformConv2dPack, self).__init__(*args, **kwargs)\n self.conv_offset = nn.Conv2d(\n self.in_channels,\n self.deform_groups * 2 * self.kernel_size[0] * self.kernel_size[1],\n kernel_size=self.kernel_size,\n stride=_pair(self.stride),\n padding=_pair(self.padding),\n dilation=_pair(self.dilation),\n bias=True)\n self.init_offset()\n\n def init_offset(self):\n self.conv_offset.weight.data.zero_()\n self.conv_offset.bias.data.zero_()\n\n def forward(self, x):\n offset = self.conv_offset(x)\n return deform_conv2d(x, offset, self.weight, self.stride, self.padding,\n self.dilation, self.groups, self.deform_groups,\n False, self.im2col_step)\n\n def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,\n missing_keys, unexpected_keys, error_msgs):\n version = local_metadata.get('version', None)\n\n if version is None or version < 2:\n # the key is different in early versions\n # In version < 2, DeformConvPack loads previous benchmark models.\n if (prefix + 'conv_offset.weight' not in state_dict\n and prefix[:-1] + '_offset.weight' in state_dict):\n state_dict[prefix + 'conv_offset.weight'] = state_dict.pop(\n prefix[:-1] + '_offset.weight')\n if (prefix + 'conv_offset.bias' not in state_dict\n and prefix[:-1] + '_offset.bias' in state_dict):\n state_dict[prefix +\n 'conv_offset.bias'] = state_dict.pop(prefix[:-1] +\n '_offset.bias')\n\n if version is not None and version > 1:\n print_log(\n f'DeformConv2dPack {prefix.rstrip(\".\")} is upgraded to '\n 'version 2.',\n logger='root')\n\n super()._load_from_state_dict(state_dict, prefix, local_metadata,\n strict, missing_keys, unexpected_keys,\n error_msgs)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/deform_roi_pool.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom torch import nn\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\nfrom torch.nn.modules.utils import _pair\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['deform_roi_pool_forward', 'deform_roi_pool_backward'])\n\n\nclass DeformRoIPoolFunction(Function):\n\n @staticmethod\n def symbolic(g, input, rois, offset, output_size, spatial_scale,\n sampling_ratio, gamma):\n return g.op(\n 'mmcv::MMCVDeformRoIPool',\n input,\n rois,\n offset,\n pooled_height_i=output_size[0],\n pooled_width_i=output_size[1],\n spatial_scale_f=spatial_scale,\n sampling_ratio_f=sampling_ratio,\n gamma_f=gamma)\n\n @staticmethod\n def forward(ctx,\n input,\n rois,\n offset,\n output_size,\n spatial_scale=1.0,\n sampling_ratio=0,\n gamma=0.1):\n if offset is None:\n offset = input.new_zeros(0)\n ctx.output_size = _pair(output_size)\n ctx.spatial_scale = float(spatial_scale)\n ctx.sampling_ratio = int(sampling_ratio)\n ctx.gamma = float(gamma)\n\n assert rois.size(1) == 5, 'RoI must be (idx, x1, y1, x2, y2)!'\n\n output_shape = (rois.size(0), input.size(1), ctx.output_size[0],\n ctx.output_size[1])\n output = input.new_zeros(output_shape)\n\n ext_module.deform_roi_pool_forward(\n input,\n rois,\n offset,\n output,\n pooled_height=ctx.output_size[0],\n pooled_width=ctx.output_size[1],\n spatial_scale=ctx.spatial_scale,\n sampling_ratio=ctx.sampling_ratio,\n gamma=ctx.gamma)\n\n ctx.save_for_backward(input, rois, offset)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n input, rois, offset = ctx.saved_tensors\n grad_input = grad_output.new_zeros(input.shape)\n grad_offset = grad_output.new_zeros(offset.shape)\n\n ext_module.deform_roi_pool_backward(\n grad_output,\n input,\n rois,\n offset,\n grad_input,\n grad_offset,\n pooled_height=ctx.output_size[0],\n pooled_width=ctx.output_size[1],\n spatial_scale=ctx.spatial_scale,\n sampling_ratio=ctx.sampling_ratio,\n gamma=ctx.gamma)\n if grad_offset.numel() == 0:\n grad_offset = None\n return grad_input, None, grad_offset, None, None, None, None\n\n\ndeform_roi_pool = DeformRoIPoolFunction.apply\n\n\nclass DeformRoIPool(nn.Module):\n\n def __init__(self,\n output_size,\n spatial_scale=1.0,\n sampling_ratio=0,\n gamma=0.1):\n super(DeformRoIPool, self).__init__()\n self.output_size = _pair(output_size)\n self.spatial_scale = float(spatial_scale)\n self.sampling_ratio = int(sampling_ratio)\n self.gamma = float(gamma)\n\n def forward(self, input, rois, offset=None):\n return deform_roi_pool(input, rois, offset, self.output_size,\n self.spatial_scale, self.sampling_ratio,\n self.gamma)\n\n\nclass DeformRoIPoolPack(DeformRoIPool):\n\n def __init__(self,\n output_size,\n output_channels,\n deform_fc_channels=1024,\n spatial_scale=1.0,\n sampling_ratio=0,\n gamma=0.1):\n super(DeformRoIPoolPack, self).__init__(output_size, spatial_scale,\n sampling_ratio, gamma)\n\n self.output_channels = output_channels\n self.deform_fc_channels = deform_fc_channels\n\n self.offset_fc = nn.Sequential(\n nn.Linear(\n self.output_size[0] * self.output_size[1] *\n self.output_channels, self.deform_fc_channels),\n nn.ReLU(inplace=True),\n nn.Linear(self.deform_fc_channels, self.deform_fc_channels),\n nn.ReLU(inplace=True),\n nn.Linear(self.deform_fc_channels,\n self.output_size[0] * self.output_size[1] * 2))\n self.offset_fc[-1].weight.data.zero_()\n self.offset_fc[-1].bias.data.zero_()\n\n def forward(self, input, rois):\n assert input.size(1) == self.output_channels\n x = deform_roi_pool(input, rois, None, self.output_size,\n self.spatial_scale, self.sampling_ratio,\n self.gamma)\n rois_num = rois.size(0)\n offset = self.offset_fc(x.view(rois_num, -1))\n offset = offset.view(rois_num, 2, self.output_size[0],\n self.output_size[1])\n return deform_roi_pool(input, rois, offset, self.output_size,\n self.spatial_scale, self.sampling_ratio,\n self.gamma)\n\n\nclass ModulatedDeformRoIPoolPack(DeformRoIPool):\n\n def __init__(self,\n output_size,\n output_channels,\n deform_fc_channels=1024,\n spatial_scale=1.0,\n sampling_ratio=0,\n gamma=0.1):\n super(ModulatedDeformRoIPoolPack,\n self).__init__(output_size, spatial_scale, sampling_ratio, gamma)\n\n self.output_channels = output_channels\n self.deform_fc_channels = deform_fc_channels\n\n self.offset_fc = nn.Sequential(\n nn.Linear(\n self.output_size[0] * self.output_size[1] *\n self.output_channels, self.deform_fc_channels),\n nn.ReLU(inplace=True),\n nn.Linear(self.deform_fc_channels, self.deform_fc_channels),\n nn.ReLU(inplace=True),\n nn.Linear(self.deform_fc_channels,\n self.output_size[0] * self.output_size[1] * 2))\n self.offset_fc[-1].weight.data.zero_()\n self.offset_fc[-1].bias.data.zero_()\n\n self.mask_fc = nn.Sequential(\n nn.Linear(\n self.output_size[0] * self.output_size[1] *\n self.output_channels, self.deform_fc_channels),\n nn.ReLU(inplace=True),\n nn.Linear(self.deform_fc_channels,\n self.output_size[0] * self.output_size[1] * 1),\n nn.Sigmoid())\n self.mask_fc[2].weight.data.zero_()\n self.mask_fc[2].bias.data.zero_()\n\n def forward(self, input, rois):\n assert input.size(1) == self.output_channels\n x = deform_roi_pool(input, rois, None, self.output_size,\n self.spatial_scale, self.sampling_ratio,\n self.gamma)\n rois_num = rois.size(0)\n offset = self.offset_fc(x.view(rois_num, -1))\n offset = offset.view(rois_num, 2, self.output_size[0],\n self.output_size[1])\n mask = self.mask_fc(x.view(rois_num, -1))\n mask = mask.view(rois_num, 1, self.output_size[0], self.output_size[1])\n d = deform_roi_pool(input, rois, offset, self.output_size,\n self.spatial_scale, self.sampling_ratio,\n self.gamma)\n return d * mask\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/deprecated_wrappers.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n# This file is for backward compatibility.\n# Module wrappers for empty tensor have been moved to mmcv.cnn.bricks.\nimport warnings\n\nfrom ..cnn.bricks.wrappers import Conv2d, ConvTranspose2d, Linear, MaxPool2d\n\n\nclass Conv2d_deprecated(Conv2d):\n\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n warnings.warn(\n 'Importing Conv2d wrapper from \"mmcv.ops\" will be deprecated in'\n ' the future. Please import them from \"mmcv.cnn\" instead',\n DeprecationWarning)\n\n\nclass ConvTranspose2d_deprecated(ConvTranspose2d):\n\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n warnings.warn(\n 'Importing ConvTranspose2d wrapper from \"mmcv.ops\" will be '\n 'deprecated in the future. Please import them from \"mmcv.cnn\" '\n 'instead', DeprecationWarning)\n\n\nclass MaxPool2d_deprecated(MaxPool2d):\n\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n warnings.warn(\n 'Importing MaxPool2d wrapper from \"mmcv.ops\" will be deprecated in'\n ' the future. Please import them from \"mmcv.cnn\" instead',\n DeprecationWarning)\n\n\nclass Linear_deprecated(Linear):\n\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n warnings.warn(\n 'Importing Linear wrapper from \"mmcv.ops\" will be deprecated in'\n ' the future. Please import them from \"mmcv.cnn\" instead',\n DeprecationWarning)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/focal_loss.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', [\n 'sigmoid_focal_loss_forward', 'sigmoid_focal_loss_backward',\n 'softmax_focal_loss_forward', 'softmax_focal_loss_backward'\n])\n\n\nclass SigmoidFocalLossFunction(Function):\n\n @staticmethod\n def symbolic(g, input, target, gamma, alpha, weight, reduction):\n return g.op(\n 'mmcv::MMCVSigmoidFocalLoss',\n input,\n target,\n gamma_f=gamma,\n alpha_f=alpha,\n weight_f=weight,\n reduction_s=reduction)\n\n @staticmethod\n def forward(ctx,\n input,\n target,\n gamma=2.0,\n alpha=0.25,\n weight=None,\n reduction='mean'):\n\n assert isinstance(target, (torch.LongTensor, torch.cuda.LongTensor))\n assert input.dim() == 2\n assert target.dim() == 1\n assert input.size(0) == target.size(0)\n if weight is None:\n weight = input.new_empty(0)\n else:\n assert weight.dim() == 1\n assert input.size(1) == weight.size(0)\n ctx.reduction_dict = {'none': 0, 'mean': 1, 'sum': 2}\n assert reduction in ctx.reduction_dict.keys()\n\n ctx.gamma = float(gamma)\n ctx.alpha = float(alpha)\n ctx.reduction = ctx.reduction_dict[reduction]\n\n output = input.new_zeros(input.size())\n\n ext_module.sigmoid_focal_loss_forward(\n input, target, weight, output, gamma=ctx.gamma, alpha=ctx.alpha)\n if ctx.reduction == ctx.reduction_dict['mean']:\n output = output.sum() / input.size(0)\n elif ctx.reduction == ctx.reduction_dict['sum']:\n output = output.sum()\n ctx.save_for_backward(input, target, weight)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n input, target, weight = ctx.saved_tensors\n\n grad_input = input.new_zeros(input.size())\n\n ext_module.sigmoid_focal_loss_backward(\n input,\n target,\n weight,\n grad_input,\n gamma=ctx.gamma,\n alpha=ctx.alpha)\n\n grad_input *= grad_output\n if ctx.reduction == ctx.reduction_dict['mean']:\n grad_input /= input.size(0)\n return grad_input, None, None, None, None, None\n\n\nsigmoid_focal_loss = SigmoidFocalLossFunction.apply\n\n\nclass SigmoidFocalLoss(nn.Module):\n\n def __init__(self, gamma, alpha, weight=None, reduction='mean'):\n super(SigmoidFocalLoss, self).__init__()\n self.gamma = gamma\n self.alpha = alpha\n self.register_buffer('weight', weight)\n self.reduction = reduction\n\n def forward(self, input, target):\n return sigmoid_focal_loss(input, target, self.gamma, self.alpha,\n self.weight, self.reduction)\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'(gamma={self.gamma}, '\n s += f'alpha={self.alpha}, '\n s += f'reduction={self.reduction})'\n return s\n\n\nclass SoftmaxFocalLossFunction(Function):\n\n @staticmethod\n def symbolic(g, input, target, gamma, alpha, weight, reduction):\n return g.op(\n 'mmcv::MMCVSoftmaxFocalLoss',\n input,\n target,\n gamma_f=gamma,\n alpha_f=alpha,\n weight_f=weight,\n reduction_s=reduction)\n\n @staticmethod\n def forward(ctx,\n input,\n target,\n gamma=2.0,\n alpha=0.25,\n weight=None,\n reduction='mean'):\n\n assert isinstance(target, (torch.LongTensor, torch.cuda.LongTensor))\n assert input.dim() == 2\n assert target.dim() == 1\n assert input.size(0) == target.size(0)\n if weight is None:\n weight = input.new_empty(0)\n else:\n assert weight.dim() == 1\n assert input.size(1) == weight.size(0)\n ctx.reduction_dict = {'none': 0, 'mean': 1, 'sum': 2}\n assert reduction in ctx.reduction_dict.keys()\n\n ctx.gamma = float(gamma)\n ctx.alpha = float(alpha)\n ctx.reduction = ctx.reduction_dict[reduction]\n\n channel_stats, _ = torch.max(input, dim=1)\n input_softmax = input - channel_stats.unsqueeze(1).expand_as(input)\n input_softmax.exp_()\n\n channel_stats = input_softmax.sum(dim=1)\n input_softmax /= channel_stats.unsqueeze(1).expand_as(input)\n\n output = input.new_zeros(input.size(0))\n ext_module.softmax_focal_loss_forward(\n input_softmax,\n target,\n weight,\n output,\n gamma=ctx.gamma,\n alpha=ctx.alpha)\n\n if ctx.reduction == ctx.reduction_dict['mean']:\n output = output.sum() / input.size(0)\n elif ctx.reduction == ctx.reduction_dict['sum']:\n output = output.sum()\n ctx.save_for_backward(input_softmax, target, weight)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n input_softmax, target, weight = ctx.saved_tensors\n buff = input_softmax.new_zeros(input_softmax.size(0))\n grad_input = input_softmax.new_zeros(input_softmax.size())\n\n ext_module.softmax_focal_loss_backward(\n input_softmax,\n target,\n weight,\n buff,\n grad_input,\n gamma=ctx.gamma,\n alpha=ctx.alpha)\n\n grad_input *= grad_output\n if ctx.reduction == ctx.reduction_dict['mean']:\n grad_input /= input_softmax.size(0)\n return grad_input, None, None, None, None, None\n\n\nsoftmax_focal_loss = SoftmaxFocalLossFunction.apply\n\n\nclass SoftmaxFocalLoss(nn.Module):\n\n def __init__(self, gamma, alpha, weight=None, reduction='mean'):\n super(SoftmaxFocalLoss, self).__init__()\n self.gamma = gamma\n self.alpha = alpha\n self.register_buffer('weight', weight)\n self.reduction = reduction\n\n def forward(self, input, target):\n return softmax_focal_loss(input, target, self.gamma, self.alpha,\n self.weight, self.reduction)\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'(gamma={self.gamma}, '\n s += f'alpha={self.alpha}, '\n s += f'reduction={self.reduction})'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/furthest_point_sample.py", "content": "import torch\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', [\n 'furthest_point_sampling_forward',\n 'furthest_point_sampling_with_dist_forward'\n])\n\n\nclass FurthestPointSampling(Function):\n \"\"\"Uses iterative furthest point sampling to select a set of features whose\n corresponding points have the furthest distance.\"\"\"\n\n @staticmethod\n def forward(ctx, points_xyz: torch.Tensor,\n num_points: int) -> torch.Tensor:\n \"\"\"\n Args:\n points_xyz (torch.Tensor): (B, N, 3) where N > num_points.\n num_points (int): Number of points in the sampled set.\n\n Returns:\n torch.Tensor: (B, num_points) indices of the sampled points.\n \"\"\"\n assert points_xyz.is_contiguous()\n\n B, N = points_xyz.size()[:2]\n output = torch.cuda.IntTensor(B, num_points)\n temp = torch.cuda.FloatTensor(B, N).fill_(1e10)\n\n ext_module.furthest_point_sampling_forward(\n points_xyz,\n temp,\n output,\n b=B,\n n=N,\n m=num_points,\n )\n if torch.__version__ != 'parrots':\n ctx.mark_non_differentiable(output)\n return output\n\n @staticmethod\n def backward(xyz, a=None):\n return None, None\n\n\nclass FurthestPointSamplingWithDist(Function):\n \"\"\"Uses iterative furthest point sampling to select a set of features whose\n corresponding points have the furthest distance.\"\"\"\n\n @staticmethod\n def forward(ctx, points_dist: torch.Tensor,\n num_points: int) -> torch.Tensor:\n \"\"\"\n Args:\n points_dist (torch.Tensor): (B, N, N) Distance between each point\n pair.\n num_points (int): Number of points in the sampled set.\n\n Returns:\n torch.Tensor: (B, num_points) indices of the sampled points.\n \"\"\"\n assert points_dist.is_contiguous()\n\n B, N, _ = points_dist.size()\n output = points_dist.new_zeros([B, num_points], dtype=torch.int32)\n temp = points_dist.new_zeros([B, N]).fill_(1e10)\n\n ext_module.furthest_point_sampling_with_dist_forward(\n points_dist, temp, output, b=B, n=N, m=num_points)\n if torch.__version__ != 'parrots':\n ctx.mark_non_differentiable(output)\n return output\n\n @staticmethod\n def backward(xyz, a=None):\n return None, None\n\n\nfurthest_point_sample = FurthestPointSampling.apply\nfurthest_point_sample_with_dist = FurthestPointSamplingWithDist.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/fused_bias_leakyrelu.py", "content": "# modified from https://github.com/rosinality/stylegan2-pytorch/blob/master/op/fused_act.py # noqa:E501\n\n# Copyright (c) 2021, NVIDIA Corporation. All rights reserved.\n# NVIDIA Source Code License for StyleGAN2 with Adaptive Discriminator\n# Augmentation (ADA)\n# =======================================================================\n\n# 1. Definitions\n\n# \"Licensor\" means any person or entity that distributes its Work.\n\n# \"Software\" means the original work of authorship made available under\n# this License.\n\n# \"Work\" means the Software and any additions to or derivative works of\n# the Software that are made available under this License.\n\n# The terms \"reproduce,\" \"reproduction,\" \"derivative works,\" and\n# \"distribution\" have the meaning as provided under U.S. copyright law;\n# provided, however, that for the purposes of this License, derivative\n# works shall not include works that remain separable from, or merely\n# link (or bind by name) to the interfaces of, the Work.\n\n# Works, including the Software, are \"made available\" under this License\n# by including in or with the Work either (a) a copyright notice\n# referencing the applicability of this License to the Work, or (b) a\n# copy of this License.\n\n# 2. License Grants\n\n# 2.1 Copyright Grant. Subject to the terms and conditions of this\n# License, each Licensor grants to you a perpetual, worldwide,\n# non-exclusive, royalty-free, copyright license to reproduce,\n# prepare derivative works of, publicly display, publicly perform,\n# sublicense and distribute its Work and any resulting derivative\n# works in any form.\n\n# 3. Limitations\n\n# 3.1 Redistribution. You may reproduce or distribute the Work only\n# if (a) you do so under this License, (b) you include a complete\n# copy of this License with your distribution, and (c) you retain\n# without modification any copyright, patent, trademark, or\n# attribution notices that are present in the Work.\n\n# 3.2 Derivative Works. You may specify that additional or different\n# terms apply to the use, reproduction, and distribution of your\n# derivative works of the Work (\"Your Terms\") only if (a) Your Terms\n# provide that the use limitation in Section 3.3 applies to your\n# derivative works, and (b) you identify the specific derivative\n# works that are subject to Your Terms. Notwithstanding Your Terms,\n# this License (including the redistribution requirements in Section\n# 3.1) will continue to apply to the Work itself.\n\n# 3.3 Use Limitation. The Work and any derivative works thereof only\n# may be used or intended for use non-commercially. Notwithstanding\n# the foregoing, NVIDIA and its affiliates may use the Work and any\n# derivative works commercially. As used herein, \"non-commercially\"\n# means for research or evaluation purposes only.\n\n# 3.4 Patent Claims. If you bring or threaten to bring a patent claim\n# against any Licensor (including any claim, cross-claim or\n# counterclaim in a lawsuit) to enforce any patents that you allege\n# are infringed by any Work, then your rights under this License from\n# such Licensor (including the grant in Section 2.1) will terminate\n# immediately.\n\n# 3.5 Trademarks. This License does not grant any rights to use any\n# Licensor’s or its affiliates’ names, logos, or trademarks, except\n# as necessary to reproduce the notices described in this License.\n\n# 3.6 Termination. If you violate any term of this License, then your\n# rights under this License (including the grant in Section 2.1) will\n# terminate immediately.\n\n# 4. Disclaimer of Warranty.\n\n# THE WORK IS PROVIDED \"AS IS\" WITHOUT WARRANTIES OR CONDITIONS OF ANY\n# KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF\n# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR\n# NON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER\n# THIS LICENSE.\n\n# 5. Limitation of Liability.\n\n# EXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL\n# THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE\n# SHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT,\n# INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF\n# OR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK\n# (INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION,\n# LOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER\n# COMMERCIAL DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF\n# THE POSSIBILITY OF SUCH DAMAGES.\n\n# =======================================================================\n\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['fused_bias_leakyrelu'])\n\n\nclass FusedBiasLeakyReLUFunctionBackward(Function):\n \"\"\"Calculate second order deviation.\n\n This function is to compute the second order deviation for the fused leaky\n relu operation.\n \"\"\"\n\n @staticmethod\n def forward(ctx, grad_output, out, negative_slope, scale):\n ctx.save_for_backward(out)\n ctx.negative_slope = negative_slope\n ctx.scale = scale\n\n empty = grad_output.new_empty(0)\n\n grad_input = ext_module.fused_bias_leakyrelu(\n grad_output,\n empty,\n out,\n act=3,\n grad=1,\n alpha=negative_slope,\n scale=scale)\n\n dim = [0]\n\n if grad_input.ndim > 2:\n dim += list(range(2, grad_input.ndim))\n\n grad_bias = grad_input.sum(dim).detach()\n\n return grad_input, grad_bias\n\n @staticmethod\n def backward(ctx, gradgrad_input, gradgrad_bias):\n out, = ctx.saved_tensors\n\n # The second order deviation, in fact, contains two parts, while the\n # the first part is zero. Thus, we direct consider the second part\n # which is similar with the first order deviation in implementation.\n gradgrad_out = ext_module.fused_bias_leakyrelu(\n gradgrad_input,\n gradgrad_bias.to(out.dtype),\n out,\n act=3,\n grad=1,\n alpha=ctx.negative_slope,\n scale=ctx.scale)\n\n return gradgrad_out, None, None, None\n\n\nclass FusedBiasLeakyReLUFunction(Function):\n\n @staticmethod\n def forward(ctx, input, bias, negative_slope, scale):\n empty = input.new_empty(0)\n\n out = ext_module.fused_bias_leakyrelu(\n input,\n bias,\n empty,\n act=3,\n grad=0,\n alpha=negative_slope,\n scale=scale)\n ctx.save_for_backward(out)\n ctx.negative_slope = negative_slope\n ctx.scale = scale\n\n return out\n\n @staticmethod\n def backward(ctx, grad_output):\n out, = ctx.saved_tensors\n\n grad_input, grad_bias = FusedBiasLeakyReLUFunctionBackward.apply(\n grad_output, out, ctx.negative_slope, ctx.scale)\n\n return grad_input, grad_bias, None, None\n\n\nclass FusedBiasLeakyReLU(nn.Module):\n r\"\"\"Fused bias leaky ReLU.\n\n This function is introduced in the StyleGAN2:\n `Analyzing and Improving the Image Quality of StyleGAN\n `_\n\n The bias term comes from the convolution operation. In addition, to keep\n the variance of the feature map or gradients unchanged, they also adopt a\n scale similarly with Kaiming initialization. However, since the\n :math:`1+{alpha}^2` is too small, we can just ignore it. Therefore, the\n final scale is just :math:`\\sqrt{2}`. Of course, you may change it with\n your own scale.\n\n TODO: Implement the CPU version.\n\n Args:\n channel (int): The channel number of the feature map.\n negative_slope (float, optional): Same as nn.LeakyRelu.\n Defaults to 0.2.\n scale (float, optional): A scalar to adjust the variance of the feature\n map. Defaults to 2**0.5.\n \"\"\"\n\n def __init__(self, num_channels, negative_slope=0.2, scale=2**0.5):\n super(FusedBiasLeakyReLU, self).__init__()\n\n self.bias = nn.Parameter(torch.zeros(num_channels))\n self.negative_slope = negative_slope\n self.scale = scale\n\n def forward(self, input):\n return fused_bias_leakyrelu(input, self.bias, self.negative_slope,\n self.scale)\n\n\ndef fused_bias_leakyrelu(input, bias, negative_slope=0.2, scale=2**0.5):\n r\"\"\"Fused bias leaky ReLU function.\n\n This function is introduced in the StyleGAN2:\n `Analyzing and Improving the Image Quality of StyleGAN\n `_\n\n The bias term comes from the convolution operation. In addition, to keep\n the variance of the feature map or gradients unchanged, they also adopt a\n scale similarly with Kaiming initialization. However, since the\n :math:`1+{alpha}^2` is too small, we can just ignore it. Therefore, the\n final scale is just :math:`\\sqrt{2}`. Of course, you may change it with\n your own scale.\n\n Args:\n input (torch.Tensor): Input feature map.\n bias (nn.Parameter): The bias from convolution operation.\n negative_slope (float, optional): Same as nn.LeakyRelu.\n Defaults to 0.2.\n scale (float, optional): A scalar to adjust the variance of the feature\n map. Defaults to 2**0.5.\n\n Returns:\n torch.Tensor: Feature map after non-linear activation.\n \"\"\"\n\n if not input.is_cuda:\n return bias_leakyrelu_ref(input, bias, negative_slope, scale)\n\n return FusedBiasLeakyReLUFunction.apply(input, bias.to(input.dtype),\n negative_slope, scale)\n\n\ndef bias_leakyrelu_ref(x, bias, negative_slope=0.2, scale=2**0.5):\n\n if bias is not None:\n assert bias.ndim == 1\n assert bias.shape[0] == x.shape[1]\n x = x + bias.reshape([-1 if i == 1 else 1 for i in range(x.ndim)])\n\n x = F.leaky_relu(x, negative_slope)\n if scale != 1:\n x = x * scale\n\n return x\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/gather_points.py", "content": "import torch\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['gather_points_forward', 'gather_points_backward'])\n\n\nclass GatherPoints(Function):\n \"\"\"Gather points with given index.\"\"\"\n\n @staticmethod\n def forward(ctx, features: torch.Tensor,\n indices: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Args:\n features (torch.Tensor): (B, C, N) features to gather.\n indices (torch.Tensor): (B, M) where M is the number of points.\n\n Returns:\n torch.Tensor: (B, C, M) where M is the number of points.\n \"\"\"\n assert features.is_contiguous()\n assert indices.is_contiguous()\n\n B, npoint = indices.size()\n _, C, N = features.size()\n output = torch.cuda.FloatTensor(B, C, npoint)\n\n ext_module.gather_points_forward(\n features, indices, output, b=B, c=C, n=N, npoints=npoint)\n\n ctx.for_backwards = (indices, C, N)\n if torch.__version__ != 'parrots':\n ctx.mark_non_differentiable(indices)\n return output\n\n @staticmethod\n def backward(ctx, grad_out):\n idx, C, N = ctx.for_backwards\n B, npoint = idx.size()\n\n grad_features = torch.cuda.FloatTensor(B, C, N).zero_()\n grad_out_data = grad_out.data.contiguous()\n ext_module.gather_points_backward(\n grad_out_data,\n idx,\n grad_features.data,\n b=B,\n c=C,\n n=N,\n npoints=npoint)\n return grad_features, None\n\n\ngather_points = GatherPoints.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/group_points.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom typing import Tuple\n\nimport torch\nfrom torch import nn as nn\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\nfrom .ball_query import ball_query\nfrom .knn import knn\n\next_module = ext_loader.load_ext(\n '_ext', ['group_points_forward', 'group_points_backward'])\n\n\nclass QueryAndGroup(nn.Module):\n \"\"\"Groups points with a ball query of radius.\n\n Args:\n max_radius (float): The maximum radius of the balls.\n If None is given, we will use kNN sampling instead of ball query.\n sample_num (int): Maximum number of features to gather in the ball.\n min_radius (float, optional): The minimum radius of the balls.\n Default: 0.\n use_xyz (bool, optional): Whether to use xyz.\n Default: True.\n return_grouped_xyz (bool, optional): Whether to return grouped xyz.\n Default: False.\n normalize_xyz (bool, optional): Whether to normalize xyz.\n Default: False.\n uniform_sample (bool, optional): Whether to sample uniformly.\n Default: False\n return_unique_cnt (bool, optional): Whether to return the count of\n unique samples. Default: False.\n return_grouped_idx (bool, optional): Whether to return grouped idx.\n Default: False.\n \"\"\"\n\n def __init__(self,\n max_radius,\n sample_num,\n min_radius=0,\n use_xyz=True,\n return_grouped_xyz=False,\n normalize_xyz=False,\n uniform_sample=False,\n return_unique_cnt=False,\n return_grouped_idx=False):\n super().__init__()\n self.max_radius = max_radius\n self.min_radius = min_radius\n self.sample_num = sample_num\n self.use_xyz = use_xyz\n self.return_grouped_xyz = return_grouped_xyz\n self.normalize_xyz = normalize_xyz\n self.uniform_sample = uniform_sample\n self.return_unique_cnt = return_unique_cnt\n self.return_grouped_idx = return_grouped_idx\n if self.return_unique_cnt:\n assert self.uniform_sample, \\\n 'uniform_sample should be True when ' \\\n 'returning the count of unique samples'\n if self.max_radius is None:\n assert not self.normalize_xyz, \\\n 'can not normalize grouped xyz when max_radius is None'\n\n def forward(self, points_xyz, center_xyz, features=None):\n \"\"\"\n Args:\n points_xyz (torch.Tensor): (B, N, 3) xyz coordinates of the\n points.\n center_xyz (torch.Tensor): (B, npoint, 3) coordinates of the\n centriods.\n features (torch.Tensor): (B, C, N) The features of grouped\n points.\n\n Returns:\n torch.Tensor: (B, 3 + C, npoint, sample_num) Grouped\n concatenated coordinates and features of points.\n \"\"\"\n # if self.max_radius is None, we will perform kNN instead of ball query\n # idx is of shape [B, npoint, sample_num]\n if self.max_radius is None:\n idx = knn(self.sample_num, points_xyz, center_xyz, False)\n idx = idx.transpose(1, 2).contiguous()\n else:\n idx = ball_query(self.min_radius, self.max_radius, self.sample_num,\n points_xyz, center_xyz)\n\n if self.uniform_sample:\n unique_cnt = torch.zeros((idx.shape[0], idx.shape[1]))\n for i_batch in range(idx.shape[0]):\n for i_region in range(idx.shape[1]):\n unique_ind = torch.unique(idx[i_batch, i_region, :])\n num_unique = unique_ind.shape[0]\n unique_cnt[i_batch, i_region] = num_unique\n sample_ind = torch.randint(\n 0,\n num_unique, (self.sample_num - num_unique, ),\n dtype=torch.long)\n all_ind = torch.cat((unique_ind, unique_ind[sample_ind]))\n idx[i_batch, i_region, :] = all_ind\n\n xyz_trans = points_xyz.transpose(1, 2).contiguous()\n # (B, 3, npoint, sample_num)\n grouped_xyz = grouping_operation(xyz_trans, idx)\n grouped_xyz_diff = grouped_xyz - \\\n center_xyz.transpose(1, 2).unsqueeze(-1) # relative offsets\n if self.normalize_xyz:\n grouped_xyz_diff /= self.max_radius\n\n if features is not None:\n grouped_features = grouping_operation(features, idx)\n if self.use_xyz:\n # (B, C + 3, npoint, sample_num)\n new_features = torch.cat([grouped_xyz_diff, grouped_features],\n dim=1)\n else:\n new_features = grouped_features\n else:\n assert (self.use_xyz\n ), 'Cannot have not features and not use xyz as a feature!'\n new_features = grouped_xyz_diff\n\n ret = [new_features]\n if self.return_grouped_xyz:\n ret.append(grouped_xyz)\n if self.return_unique_cnt:\n ret.append(unique_cnt)\n if self.return_grouped_idx:\n ret.append(idx)\n if len(ret) == 1:\n return ret[0]\n else:\n return tuple(ret)\n\n\nclass GroupAll(nn.Module):\n \"\"\"Group xyz with feature.\n\n Args:\n use_xyz (bool): Whether to use xyz.\n \"\"\"\n\n def __init__(self, use_xyz: bool = True):\n super().__init__()\n self.use_xyz = use_xyz\n\n def forward(self,\n xyz: torch.Tensor,\n new_xyz: torch.Tensor,\n features: torch.Tensor = None):\n \"\"\"\n Args:\n xyz (Tensor): (B, N, 3) xyz coordinates of the features.\n new_xyz (Tensor): new xyz coordinates of the features.\n features (Tensor): (B, C, N) features to group.\n\n Returns:\n Tensor: (B, C + 3, 1, N) Grouped feature.\n \"\"\"\n grouped_xyz = xyz.transpose(1, 2).unsqueeze(2)\n if features is not None:\n grouped_features = features.unsqueeze(2)\n if self.use_xyz:\n # (B, 3 + C, 1, N)\n new_features = torch.cat([grouped_xyz, grouped_features],\n dim=1)\n else:\n new_features = grouped_features\n else:\n new_features = grouped_xyz\n\n return new_features\n\n\nclass GroupingOperation(Function):\n \"\"\"Group feature with given index.\"\"\"\n\n @staticmethod\n def forward(ctx, features: torch.Tensor,\n indices: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Args:\n features (Tensor): (B, C, N) tensor of features to group.\n indices (Tensor): (B, npoint, nsample) the indices of\n features to group with.\n\n Returns:\n Tensor: (B, C, npoint, nsample) Grouped features.\n \"\"\"\n features = features.contiguous()\n indices = indices.contiguous()\n\n B, nfeatures, nsample = indices.size()\n _, C, N = features.size()\n output = torch.cuda.FloatTensor(B, C, nfeatures, nsample)\n\n ext_module.group_points_forward(\n features,\n indices,\n output,\n b=B,\n c=C,\n n=N,\n npoints=nfeatures,\n nsample=nsample)\n\n ctx.for_backwards = (indices, N)\n return output\n\n @staticmethod\n def backward(ctx,\n grad_out: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\n Args:\n grad_out (Tensor): (B, C, npoint, nsample) tensor of the gradients\n of the output from forward.\n\n Returns:\n Tensor: (B, C, N) gradient of the features.\n \"\"\"\n idx, N = ctx.for_backwards\n\n B, C, npoint, nsample = grad_out.size()\n grad_features = torch.cuda.FloatTensor(B, C, N).zero_()\n\n grad_out_data = grad_out.data.contiguous()\n ext_module.group_points_backward(\n grad_out_data,\n idx,\n grad_features.data,\n b=B,\n c=C,\n n=N,\n npoints=npoint,\n nsample=nsample)\n return grad_features, None\n\n\ngrouping_operation = GroupingOperation.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/info.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport glob\nimport os\n\nimport torch\n\nif torch.__version__ == 'parrots':\n import parrots\n\n def get_compiler_version():\n return 'GCC ' + parrots.version.compiler\n\n def get_compiling_cuda_version():\n return parrots.version.cuda\nelse:\n from ..utils import ext_loader\n ext_module = ext_loader.load_ext(\n '_ext', ['get_compiler_version', 'get_compiling_cuda_version'])\n\n def get_compiler_version():\n return ext_module.get_compiler_version()\n\n def get_compiling_cuda_version():\n return ext_module.get_compiling_cuda_version()\n\n\ndef get_onnxruntime_op_path():\n wildcard = os.path.join(\n os.path.abspath(os.path.dirname(os.path.dirname(__file__))),\n '_ext_ort.*.so')\n\n paths = glob.glob(wildcard)\n if len(paths) > 0:\n return paths[0]\n else:\n return ''\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/iou3d.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', [\n 'iou3d_boxes_iou_bev_forward', 'iou3d_nms_forward',\n 'iou3d_nms_normal_forward'\n])\n\n\ndef boxes_iou_bev(boxes_a, boxes_b):\n \"\"\"Calculate boxes IoU in the Bird's Eye View.\n\n Args:\n boxes_a (torch.Tensor): Input boxes a with shape (M, 5).\n boxes_b (torch.Tensor): Input boxes b with shape (N, 5).\n\n Returns:\n torch.Tensor: IoU result with shape (M, N).\n \"\"\"\n ans_iou = boxes_a.new_zeros(\n torch.Size((boxes_a.shape[0], boxes_b.shape[0])))\n\n ext_module.iou3d_boxes_iou_bev_forward(boxes_a.contiguous(),\n boxes_b.contiguous(), ans_iou)\n\n return ans_iou\n\n\ndef nms_bev(boxes, scores, thresh, pre_max_size=None, post_max_size=None):\n \"\"\"NMS function GPU implementation (for BEV boxes). The overlap of two\n boxes for IoU calculation is defined as the exact overlapping area of the\n two boxes. In this function, one can also set ``pre_max_size`` and\n ``post_max_size``.\n\n Args:\n boxes (torch.Tensor): Input boxes with the shape of [N, 5]\n ([x1, y1, x2, y2, ry]).\n scores (torch.Tensor): Scores of boxes with the shape of [N].\n thresh (float): Overlap threshold of NMS.\n pre_max_size (int, optional): Max size of boxes before NMS.\n Default: None.\n post_max_size (int, optional): Max size of boxes after NMS.\n Default: None.\n\n Returns:\n torch.Tensor: Indexes after NMS.\n \"\"\"\n assert boxes.size(1) == 5, 'Input boxes shape should be [N, 5]'\n order = scores.sort(0, descending=True)[1]\n\n if pre_max_size is not None:\n order = order[:pre_max_size]\n boxes = boxes[order].contiguous()\n\n keep = torch.zeros(boxes.size(0), dtype=torch.long)\n num_out = torch.zeros(size=(), dtype=torch.long)\n ext_module.iou3d_nms_forward(\n boxes, keep, num_out, nms_overlap_thresh=thresh)\n keep = order[keep[:num_out].cuda(boxes.device)].contiguous()\n if post_max_size is not None:\n keep = keep[:post_max_size]\n return keep\n\n\ndef nms_normal_bev(boxes, scores, thresh):\n \"\"\"Normal NMS function GPU implementation (for BEV boxes). The overlap of\n two boxes for IoU calculation is defined as the exact overlapping area of\n the two boxes WITH their yaw angle set to 0.\n\n Args:\n boxes (torch.Tensor): Input boxes with shape (N, 5).\n scores (torch.Tensor): Scores of predicted boxes with shape (N).\n thresh (float): Overlap threshold of NMS.\n\n Returns:\n torch.Tensor: Remaining indices with scores in descending order.\n \"\"\"\n assert boxes.shape[1] == 5, 'Input boxes shape should be [N, 5]'\n order = scores.sort(0, descending=True)[1]\n\n boxes = boxes[order].contiguous()\n\n keep = torch.zeros(boxes.size(0), dtype=torch.long)\n num_out = torch.zeros(size=(), dtype=torch.long)\n ext_module.iou3d_nms_normal_forward(\n boxes, keep, num_out, nms_overlap_thresh=thresh)\n return order[keep[:num_out].cuda(boxes.device)].contiguous()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/knn.py", "content": "import torch\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['knn_forward'])\n\n\nclass KNN(Function):\n r\"\"\"KNN (CUDA) based on heap data structure.\n\n Modified from `PAConv `_.\n\n Find k-nearest points.\n \"\"\"\n\n @staticmethod\n def forward(ctx,\n k: int,\n xyz: torch.Tensor,\n center_xyz: torch.Tensor = None,\n transposed: bool = False) -> torch.Tensor:\n \"\"\"\n Args:\n k (int): number of nearest neighbors.\n xyz (torch.Tensor): (B, N, 3) if transposed == False, else\n (B, 3, N). xyz coordinates of the features.\n center_xyz (torch.Tensor, optional): (B, npoint, 3) if transposed\n is False, else (B, 3, npoint). centers of the knn query.\n Default: None.\n transposed (bool, optional): whether the input tensors are\n transposed. Should not explicitly use this keyword when\n calling knn (=KNN.apply), just add the fourth param.\n Default: False.\n\n Returns:\n torch.Tensor: (B, k, npoint) tensor with the indices of the\n features that form k-nearest neighbours.\n \"\"\"\n assert (k > 0) & (k < 100), 'k should be in range(0, 100)'\n\n if center_xyz is None:\n center_xyz = xyz\n\n if transposed:\n xyz = xyz.transpose(2, 1).contiguous()\n center_xyz = center_xyz.transpose(2, 1).contiguous()\n\n assert xyz.is_contiguous() # [B, N, 3]\n assert center_xyz.is_contiguous() # [B, npoint, 3]\n\n center_xyz_device = center_xyz.get_device()\n assert center_xyz_device == xyz.get_device(), \\\n 'center_xyz and xyz should be put on the same device'\n if torch.cuda.current_device() != center_xyz_device:\n torch.cuda.set_device(center_xyz_device)\n\n B, npoint, _ = center_xyz.shape\n N = xyz.shape[1]\n\n idx = center_xyz.new_zeros((B, npoint, k)).int()\n dist2 = center_xyz.new_zeros((B, npoint, k)).float()\n\n ext_module.knn_forward(\n xyz, center_xyz, idx, dist2, b=B, n=N, m=npoint, nsample=k)\n # idx shape to [B, k, npoint]\n idx = idx.transpose(2, 1).contiguous()\n if torch.__version__ != 'parrots':\n ctx.mark_non_differentiable(idx)\n return idx\n\n @staticmethod\n def backward(ctx, a=None):\n return None, None, None\n\n\nknn = KNN.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/masked_conv.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport math\n\nimport torch\nimport torch.nn as nn\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\nfrom torch.nn.modules.utils import _pair\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['masked_im2col_forward', 'masked_col2im_forward'])\n\n\nclass MaskedConv2dFunction(Function):\n\n @staticmethod\n def symbolic(g, features, mask, weight, bias, padding, stride):\n return g.op(\n 'mmcv::MMCVMaskedConv2d',\n features,\n mask,\n weight,\n bias,\n padding_i=padding,\n stride_i=stride)\n\n @staticmethod\n def forward(ctx, features, mask, weight, bias, padding=0, stride=1):\n assert mask.dim() == 3 and mask.size(0) == 1\n assert features.dim() == 4 and features.size(0) == 1\n assert features.size()[2:] == mask.size()[1:]\n pad_h, pad_w = _pair(padding)\n stride_h, stride_w = _pair(stride)\n if stride_h != 1 or stride_w != 1:\n raise ValueError(\n 'Stride could not only be 1 in masked_conv2d currently.')\n out_channel, in_channel, kernel_h, kernel_w = weight.size()\n\n batch_size = features.size(0)\n out_h = int(\n math.floor((features.size(2) + 2 * pad_h -\n (kernel_h - 1) - 1) / stride_h + 1))\n out_w = int(\n math.floor((features.size(3) + 2 * pad_w -\n (kernel_h - 1) - 1) / stride_w + 1))\n mask_inds = torch.nonzero(mask[0] > 0, as_tuple=False)\n output = features.new_zeros(batch_size, out_channel, out_h, out_w)\n if mask_inds.numel() > 0:\n mask_h_idx = mask_inds[:, 0].contiguous()\n mask_w_idx = mask_inds[:, 1].contiguous()\n data_col = features.new_zeros(in_channel * kernel_h * kernel_w,\n mask_inds.size(0))\n ext_module.masked_im2col_forward(\n features,\n mask_h_idx,\n mask_w_idx,\n data_col,\n kernel_h=kernel_h,\n kernel_w=kernel_w,\n pad_h=pad_h,\n pad_w=pad_w)\n masked_output = torch.addmm(1, bias[:, None], 1,\n weight.view(out_channel, -1), data_col)\n ext_module.masked_col2im_forward(\n masked_output,\n mask_h_idx,\n mask_w_idx,\n output,\n height=out_h,\n width=out_w,\n channels=out_channel)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n return (None, ) * 5\n\n\nmasked_conv2d = MaskedConv2dFunction.apply\n\n\nclass MaskedConv2d(nn.Conv2d):\n \"\"\"A MaskedConv2d which inherits the official Conv2d.\n\n The masked forward doesn't implement the backward function and only\n supports the stride parameter to be 1 currently.\n \"\"\"\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n bias=True):\n super(MaskedConv2d,\n self).__init__(in_channels, out_channels, kernel_size, stride,\n padding, dilation, groups, bias)\n\n def forward(self, input, mask=None):\n if mask is None: # fallback to the normal Conv2d\n return super(MaskedConv2d, self).forward(input)\n else:\n return masked_conv2d(input, mask, self.weight, self.bias,\n self.padding)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/merge_cells.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom abc import abstractmethod\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom ..cnn import ConvModule\n\n\nclass BaseMergeCell(nn.Module):\n \"\"\"The basic class for cells used in NAS-FPN and NAS-FCOS.\n\n BaseMergeCell takes 2 inputs. After applying convolution\n on them, they are resized to the target size. Then,\n they go through binary_op, which depends on the type of cell.\n If with_out_conv is True, the result of output will go through\n another convolution layer.\n\n Args:\n in_channels (int): number of input channels in out_conv layer.\n out_channels (int): number of output channels in out_conv layer.\n with_out_conv (bool): Whether to use out_conv layer\n out_conv_cfg (dict): Config dict for convolution layer, which should\n contain \"groups\", \"kernel_size\", \"padding\", \"bias\" to build\n out_conv layer.\n out_norm_cfg (dict): Config dict for normalization layer in out_conv.\n out_conv_order (tuple): The order of conv/norm/activation layers in\n out_conv.\n with_input1_conv (bool): Whether to use convolution on input1.\n with_input2_conv (bool): Whether to use convolution on input2.\n input_conv_cfg (dict): Config dict for building input1_conv layer and\n input2_conv layer, which is expected to contain the type of\n convolution.\n Default: None, which means using conv2d.\n input_norm_cfg (dict): Config dict for normalization layer in\n input1_conv and input2_conv layer. Default: None.\n upsample_mode (str): Interpolation method used to resize the output\n of input1_conv and input2_conv to target size. Currently, we\n support ['nearest', 'bilinear']. Default: 'nearest'.\n \"\"\"\n\n def __init__(self,\n fused_channels=256,\n out_channels=256,\n with_out_conv=True,\n out_conv_cfg=dict(\n groups=1, kernel_size=3, padding=1, bias=True),\n out_norm_cfg=None,\n out_conv_order=('act', 'conv', 'norm'),\n with_input1_conv=False,\n with_input2_conv=False,\n input_conv_cfg=None,\n input_norm_cfg=None,\n upsample_mode='nearest'):\n super(BaseMergeCell, self).__init__()\n assert upsample_mode in ['nearest', 'bilinear']\n self.with_out_conv = with_out_conv\n self.with_input1_conv = with_input1_conv\n self.with_input2_conv = with_input2_conv\n self.upsample_mode = upsample_mode\n\n if self.with_out_conv:\n self.out_conv = ConvModule(\n fused_channels,\n out_channels,\n **out_conv_cfg,\n norm_cfg=out_norm_cfg,\n order=out_conv_order)\n\n self.input1_conv = self._build_input_conv(\n out_channels, input_conv_cfg,\n input_norm_cfg) if with_input1_conv else nn.Sequential()\n self.input2_conv = self._build_input_conv(\n out_channels, input_conv_cfg,\n input_norm_cfg) if with_input2_conv else nn.Sequential()\n\n def _build_input_conv(self, channel, conv_cfg, norm_cfg):\n return ConvModule(\n channel,\n channel,\n 3,\n padding=1,\n conv_cfg=conv_cfg,\n norm_cfg=norm_cfg,\n bias=True)\n\n @abstractmethod\n def _binary_op(self, x1, x2):\n pass\n\n def _resize(self, x, size):\n if x.shape[-2:] == size:\n return x\n elif x.shape[-2:] < size:\n return F.interpolate(x, size=size, mode=self.upsample_mode)\n else:\n assert x.shape[-2] % size[-2] == 0 and x.shape[-1] % size[-1] == 0\n kernel_size = x.shape[-1] // size[-1]\n x = F.max_pool2d(x, kernel_size=kernel_size, stride=kernel_size)\n return x\n\n def forward(self, x1, x2, out_size=None):\n assert x1.shape[:2] == x2.shape[:2]\n assert out_size is None or len(out_size) == 2\n if out_size is None: # resize to larger one\n out_size = max(x1.size()[2:], x2.size()[2:])\n\n x1 = self.input1_conv(x1)\n x2 = self.input2_conv(x2)\n\n x1 = self._resize(x1, out_size)\n x2 = self._resize(x2, out_size)\n\n x = self._binary_op(x1, x2)\n if self.with_out_conv:\n x = self.out_conv(x)\n return x\n\n\nclass SumCell(BaseMergeCell):\n\n def __init__(self, in_channels, out_channels, **kwargs):\n super(SumCell, self).__init__(in_channels, out_channels, **kwargs)\n\n def _binary_op(self, x1, x2):\n return x1 + x2\n\n\nclass ConcatCell(BaseMergeCell):\n\n def __init__(self, in_channels, out_channels, **kwargs):\n super(ConcatCell, self).__init__(in_channels * 2, out_channels,\n **kwargs)\n\n def _binary_op(self, x1, x2):\n ret = torch.cat([x1, x2], dim=1)\n return ret\n\n\nclass GlobalPoolingCell(BaseMergeCell):\n\n def __init__(self, in_channels=None, out_channels=None, **kwargs):\n super().__init__(in_channels, out_channels, **kwargs)\n self.global_pool = nn.AdaptiveAvgPool2d((1, 1))\n\n def _binary_op(self, x1, x2):\n x2_att = self.global_pool(x2).sigmoid()\n return x2 + x2_att * x1\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/min_area_polygons.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['min_area_polygons'])\n\n\ndef min_area_polygons(pointsets):\n \"\"\"Find the smallest polygons that surrounds all points in the point sets.\n\n Args:\n pointsets (Tensor): point sets with shape (N, 18).\n\n Returns:\n torch.Tensor: Return the smallest polygons with shape (N, 8).\n \"\"\"\n polygons = pointsets.new_zeros((pointsets.size(0), 8))\n ext_module.min_area_polygons(pointsets, polygons)\n return polygons\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/modulated_deform_conv.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport math\n\nimport torch\nimport torch.nn as nn\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\nfrom torch.nn.modules.utils import _pair, _single\n\nfrom mmcv.utils import deprecated_api_warning\nfrom ..cnn import CONV_LAYERS\nfrom ..utils import ext_loader, print_log\n\next_module = ext_loader.load_ext(\n '_ext',\n ['modulated_deform_conv_forward', 'modulated_deform_conv_backward'])\n\n\nclass ModulatedDeformConv2dFunction(Function):\n\n @staticmethod\n def symbolic(g, input, offset, mask, weight, bias, stride, padding,\n dilation, groups, deform_groups):\n input_tensors = [input, offset, mask, weight]\n if bias is not None:\n input_tensors.append(bias)\n return g.op(\n 'mmcv::MMCVModulatedDeformConv2d',\n *input_tensors,\n stride_i=stride,\n padding_i=padding,\n dilation_i=dilation,\n groups_i=groups,\n deform_groups_i=deform_groups)\n\n @staticmethod\n def forward(ctx,\n input,\n offset,\n mask,\n weight,\n bias=None,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n deform_groups=1):\n if input is not None and input.dim() != 4:\n raise ValueError(\n f'Expected 4D tensor as input, got {input.dim()}D tensor \\\n instead.')\n ctx.stride = _pair(stride)\n ctx.padding = _pair(padding)\n ctx.dilation = _pair(dilation)\n ctx.groups = groups\n ctx.deform_groups = deform_groups\n ctx.with_bias = bias is not None\n if not ctx.with_bias:\n bias = input.new_empty(0) # fake tensor\n # When pytorch version >= 1.6.0, amp is adopted for fp16 mode;\n # amp won't cast the type of model (float32), but \"offset\" is cast\n # to float16 by nn.Conv2d automatically, leading to the type\n # mismatch with input (when it is float32) or weight.\n # The flag for whether to use fp16 or amp is the type of \"offset\",\n # we cast weight and input to temporarily support fp16 and amp\n # whatever the pytorch version is.\n input = input.type_as(offset)\n weight = weight.type_as(input)\n bias = bias.type_as(input)\n ctx.save_for_backward(input, offset, mask, weight, bias)\n output = input.new_empty(\n ModulatedDeformConv2dFunction._output_size(ctx, input, weight))\n ctx._bufs = [input.new_empty(0), input.new_empty(0)]\n ext_module.modulated_deform_conv_forward(\n input,\n weight,\n bias,\n ctx._bufs[0],\n offset,\n mask,\n output,\n ctx._bufs[1],\n kernel_h=weight.size(2),\n kernel_w=weight.size(3),\n stride_h=ctx.stride[0],\n stride_w=ctx.stride[1],\n pad_h=ctx.padding[0],\n pad_w=ctx.padding[1],\n dilation_h=ctx.dilation[0],\n dilation_w=ctx.dilation[1],\n group=ctx.groups,\n deformable_group=ctx.deform_groups,\n with_bias=ctx.with_bias)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n input, offset, mask, weight, bias = ctx.saved_tensors\n grad_input = torch.zeros_like(input)\n grad_offset = torch.zeros_like(offset)\n grad_mask = torch.zeros_like(mask)\n grad_weight = torch.zeros_like(weight)\n grad_bias = torch.zeros_like(bias)\n grad_output = grad_output.contiguous()\n ext_module.modulated_deform_conv_backward(\n input,\n weight,\n bias,\n ctx._bufs[0],\n offset,\n mask,\n ctx._bufs[1],\n grad_input,\n grad_weight,\n grad_bias,\n grad_offset,\n grad_mask,\n grad_output,\n kernel_h=weight.size(2),\n kernel_w=weight.size(3),\n stride_h=ctx.stride[0],\n stride_w=ctx.stride[1],\n pad_h=ctx.padding[0],\n pad_w=ctx.padding[1],\n dilation_h=ctx.dilation[0],\n dilation_w=ctx.dilation[1],\n group=ctx.groups,\n deformable_group=ctx.deform_groups,\n with_bias=ctx.with_bias)\n if not ctx.with_bias:\n grad_bias = None\n\n return (grad_input, grad_offset, grad_mask, grad_weight, grad_bias,\n None, None, None, None, None)\n\n @staticmethod\n def _output_size(ctx, input, weight):\n channels = weight.size(0)\n output_size = (input.size(0), channels)\n for d in range(input.dim() - 2):\n in_size = input.size(d + 2)\n pad = ctx.padding[d]\n kernel = ctx.dilation[d] * (weight.size(d + 2) - 1) + 1\n stride_ = ctx.stride[d]\n output_size += ((in_size + (2 * pad) - kernel) // stride_ + 1, )\n if not all(map(lambda s: s > 0, output_size)):\n raise ValueError(\n 'convolution input is too small (output would be ' +\n 'x'.join(map(str, output_size)) + ')')\n return output_size\n\n\nmodulated_deform_conv2d = ModulatedDeformConv2dFunction.apply\n\n\nclass ModulatedDeformConv2d(nn.Module):\n\n @deprecated_api_warning({'deformable_groups': 'deform_groups'},\n cls_name='ModulatedDeformConv2d')\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n deform_groups=1,\n bias=True):\n super(ModulatedDeformConv2d, self).__init__()\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.kernel_size = _pair(kernel_size)\n self.stride = _pair(stride)\n self.padding = _pair(padding)\n self.dilation = _pair(dilation)\n self.groups = groups\n self.deform_groups = deform_groups\n # enable compatibility with nn.Conv2d\n self.transposed = False\n self.output_padding = _single(0)\n\n self.weight = nn.Parameter(\n torch.Tensor(out_channels, in_channels // groups,\n *self.kernel_size))\n if bias:\n self.bias = nn.Parameter(torch.Tensor(out_channels))\n else:\n self.register_parameter('bias', None)\n self.init_weights()\n\n def init_weights(self):\n n = self.in_channels\n for k in self.kernel_size:\n n *= k\n stdv = 1. / math.sqrt(n)\n self.weight.data.uniform_(-stdv, stdv)\n if self.bias is not None:\n self.bias.data.zero_()\n\n def forward(self, x, offset, mask):\n return modulated_deform_conv2d(x, offset, mask, self.weight, self.bias,\n self.stride, self.padding,\n self.dilation, self.groups,\n self.deform_groups)\n\n\n@CONV_LAYERS.register_module('DCNv2')\nclass ModulatedDeformConv2dPack(ModulatedDeformConv2d):\n \"\"\"A ModulatedDeformable Conv Encapsulation that acts as normal Conv\n layers.\n\n Args:\n in_channels (int): Same as nn.Conv2d.\n out_channels (int): Same as nn.Conv2d.\n kernel_size (int or tuple[int]): Same as nn.Conv2d.\n stride (int): Same as nn.Conv2d, while tuple is not supported.\n padding (int): Same as nn.Conv2d, while tuple is not supported.\n dilation (int): Same as nn.Conv2d, while tuple is not supported.\n groups (int): Same as nn.Conv2d.\n bias (bool or str): If specified as `auto`, it will be decided by the\n norm_cfg. Bias will be set as True if norm_cfg is None, otherwise\n False.\n \"\"\"\n\n _version = 2\n\n def __init__(self, *args, **kwargs):\n super(ModulatedDeformConv2dPack, self).__init__(*args, **kwargs)\n self.conv_offset = nn.Conv2d(\n self.in_channels,\n self.deform_groups * 3 * self.kernel_size[0] * self.kernel_size[1],\n kernel_size=self.kernel_size,\n stride=self.stride,\n padding=self.padding,\n dilation=self.dilation,\n bias=True)\n self.init_weights()\n\n def init_weights(self):\n super(ModulatedDeformConv2dPack, self).init_weights()\n if hasattr(self, 'conv_offset'):\n self.conv_offset.weight.data.zero_()\n self.conv_offset.bias.data.zero_()\n\n def forward(self, x):\n out = self.conv_offset(x)\n o1, o2, mask = torch.chunk(out, 3, dim=1)\n offset = torch.cat((o1, o2), dim=1)\n mask = torch.sigmoid(mask)\n return modulated_deform_conv2d(x, offset, mask, self.weight, self.bias,\n self.stride, self.padding,\n self.dilation, self.groups,\n self.deform_groups)\n\n def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,\n missing_keys, unexpected_keys, error_msgs):\n version = local_metadata.get('version', None)\n\n if version is None or version < 2:\n # the key is different in early versions\n # In version < 2, ModulatedDeformConvPack\n # loads previous benchmark models.\n if (prefix + 'conv_offset.weight' not in state_dict\n and prefix[:-1] + '_offset.weight' in state_dict):\n state_dict[prefix + 'conv_offset.weight'] = state_dict.pop(\n prefix[:-1] + '_offset.weight')\n if (prefix + 'conv_offset.bias' not in state_dict\n and prefix[:-1] + '_offset.bias' in state_dict):\n state_dict[prefix +\n 'conv_offset.bias'] = state_dict.pop(prefix[:-1] +\n '_offset.bias')\n\n if version is not None and version > 1:\n print_log(\n f'ModulatedDeformConvPack {prefix.rstrip(\".\")} is upgraded to '\n 'version 2.',\n logger='root')\n\n super()._load_from_state_dict(state_dict, prefix, local_metadata,\n strict, missing_keys, unexpected_keys,\n error_msgs)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/multi_scale_deform_attn.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport math\nimport warnings\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.autograd.function import Function, once_differentiable\n\nfrom mmcv import deprecated_api_warning\nfrom mmcv.cnn import constant_init, xavier_init\nfrom mmcv.cnn.bricks.registry import ATTENTION\nfrom mmcv.runner import BaseModule\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['ms_deform_attn_backward', 'ms_deform_attn_forward'])\n\n\nclass MultiScaleDeformableAttnFunction(Function):\n\n @staticmethod\n def forward(ctx, value, value_spatial_shapes, value_level_start_index,\n sampling_locations, attention_weights, im2col_step):\n \"\"\"GPU version of multi-scale deformable attention.\n\n Args:\n value (torch.Tensor): The value has shape\n (bs, num_keys, mum_heads, embed_dims//num_heads)\n value_spatial_shapes (torch.Tensor): Spatial shape of\n each feature map, has shape (num_levels, 2),\n last dimension 2 represent (h, w)\n sampling_locations (torch.Tensor): The location of sampling points,\n has shape\n (bs ,num_queries, num_heads, num_levels, num_points, 2),\n the last dimension 2 represent (x, y).\n attention_weights (torch.Tensor): The weight of sampling points\n used when calculate the attention, has shape\n (bs ,num_queries, num_heads, num_levels, num_points),\n im2col_step (Tensor): The step used in image to column.\n\n Returns:\n torch.Tensor: has shape (bs, num_queries, embed_dims)\n \"\"\"\n\n ctx.im2col_step = im2col_step\n output = ext_module.ms_deform_attn_forward(\n value,\n value_spatial_shapes,\n value_level_start_index,\n sampling_locations,\n attention_weights,\n im2col_step=ctx.im2col_step)\n ctx.save_for_backward(value, value_spatial_shapes,\n value_level_start_index, sampling_locations,\n attention_weights)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n \"\"\"GPU version of backward function.\n\n Args:\n grad_output (torch.Tensor): Gradient of output tensor of forward.\n\n Returns:\n tuple[Tensor]: Gradient of input tensors in forward.\n \"\"\"\n value, value_spatial_shapes, value_level_start_index,\\\n sampling_locations, attention_weights = ctx.saved_tensors\n grad_value = torch.zeros_like(value)\n grad_sampling_loc = torch.zeros_like(sampling_locations)\n grad_attn_weight = torch.zeros_like(attention_weights)\n\n ext_module.ms_deform_attn_backward(\n value,\n value_spatial_shapes,\n value_level_start_index,\n sampling_locations,\n attention_weights,\n grad_output.contiguous(),\n grad_value,\n grad_sampling_loc,\n grad_attn_weight,\n im2col_step=ctx.im2col_step)\n\n return grad_value, None, None, \\\n grad_sampling_loc, grad_attn_weight, None\n\n\ndef multi_scale_deformable_attn_pytorch(value, value_spatial_shapes,\n sampling_locations, attention_weights):\n \"\"\"CPU version of multi-scale deformable attention.\n\n Args:\n value (torch.Tensor): The value has shape\n (bs, num_keys, mum_heads, embed_dims//num_heads)\n value_spatial_shapes (torch.Tensor): Spatial shape of\n each feature map, has shape (num_levels, 2),\n last dimension 2 represent (h, w)\n sampling_locations (torch.Tensor): The location of sampling points,\n has shape\n (bs ,num_queries, num_heads, num_levels, num_points, 2),\n the last dimension 2 represent (x, y).\n attention_weights (torch.Tensor): The weight of sampling points used\n when calculate the attention, has shape\n (bs ,num_queries, num_heads, num_levels, num_points),\n\n Returns:\n torch.Tensor: has shape (bs, num_queries, embed_dims)\n \"\"\"\n\n bs, _, num_heads, embed_dims = value.shape\n _, num_queries, num_heads, num_levels, num_points, _ =\\\n sampling_locations.shape\n value_list = value.split([H_ * W_ for H_, W_ in value_spatial_shapes],\n dim=1)\n sampling_grids = 2 * sampling_locations - 1\n sampling_value_list = []\n for level, (H_, W_) in enumerate(value_spatial_shapes):\n # bs, H_*W_, num_heads, embed_dims ->\n # bs, H_*W_, num_heads*embed_dims ->\n # bs, num_heads*embed_dims, H_*W_ ->\n # bs*num_heads, embed_dims, H_, W_\n value_l_ = value_list[level].flatten(2).transpose(1, 2).reshape(\n bs * num_heads, embed_dims, H_, W_)\n # bs, num_queries, num_heads, num_points, 2 ->\n # bs, num_heads, num_queries, num_points, 2 ->\n # bs*num_heads, num_queries, num_points, 2\n sampling_grid_l_ = sampling_grids[:, :, :,\n level].transpose(1, 2).flatten(0, 1)\n # bs*num_heads, embed_dims, num_queries, num_points\n sampling_value_l_ = F.grid_sample(\n value_l_,\n sampling_grid_l_,\n mode='bilinear',\n padding_mode='zeros',\n align_corners=False)\n sampling_value_list.append(sampling_value_l_)\n # (bs, num_queries, num_heads, num_levels, num_points) ->\n # (bs, num_heads, num_queries, num_levels, num_points) ->\n # (bs, num_heads, 1, num_queries, num_levels*num_points)\n attention_weights = attention_weights.transpose(1, 2).reshape(\n bs * num_heads, 1, num_queries, num_levels * num_points)\n output = (torch.stack(sampling_value_list, dim=-2).flatten(-2) *\n attention_weights).sum(-1).view(bs, num_heads * embed_dims,\n num_queries)\n return output.transpose(1, 2).contiguous()\n\n\n@ATTENTION.register_module()\nclass MultiScaleDeformableAttention(BaseModule):\n \"\"\"An attention module used in Deformable-Detr.\n\n `Deformable DETR: Deformable Transformers for End-to-End Object Detection.\n `_.\n\n Args:\n embed_dims (int): The embedding dimension of Attention.\n Default: 256.\n num_heads (int): Parallel attention heads. Default: 64.\n num_levels (int): The number of feature map used in\n Attention. Default: 4.\n num_points (int): The number of sampling points for\n each query in each head. Default: 4.\n im2col_step (int): The step used in image_to_column.\n Default: 64.\n dropout (float): A Dropout layer on `inp_identity`.\n Default: 0.1.\n batch_first (bool): Key, Query and Value are shape of\n (batch, n, embed_dim)\n or (n, batch, embed_dim). Default to False.\n norm_cfg (dict): Config dict for normalization layer.\n Default: None.\n init_cfg (obj:`mmcv.ConfigDict`): The Config for initialization.\n Default: None.\n \"\"\"\n\n def __init__(self,\n embed_dims=256,\n num_heads=8,\n num_levels=4,\n num_points=4,\n im2col_step=64,\n dropout=0.1,\n batch_first=False,\n norm_cfg=None,\n init_cfg=None):\n super().__init__(init_cfg)\n if embed_dims % num_heads != 0:\n raise ValueError(f'embed_dims must be divisible by num_heads, '\n f'but got {embed_dims} and {num_heads}')\n dim_per_head = embed_dims // num_heads\n self.norm_cfg = norm_cfg\n self.dropout = nn.Dropout(dropout)\n self.batch_first = batch_first\n\n # you'd better set dim_per_head to a power of 2\n # which is more efficient in the CUDA implementation\n def _is_power_of_2(n):\n if (not isinstance(n, int)) or (n < 0):\n raise ValueError(\n 'invalid input for _is_power_of_2: {} (type: {})'.format(\n n, type(n)))\n return (n & (n - 1) == 0) and n != 0\n\n if not _is_power_of_2(dim_per_head):\n warnings.warn(\n \"You'd better set embed_dims in \"\n 'MultiScaleDeformAttention to make '\n 'the dimension of each attention head a power of 2 '\n 'which is more efficient in our CUDA implementation.')\n\n self.im2col_step = im2col_step\n self.embed_dims = embed_dims\n self.num_levels = num_levels\n self.num_heads = num_heads\n self.num_points = num_points\n self.sampling_offsets = nn.Linear(\n embed_dims, num_heads * num_levels * num_points * 2)\n self.attention_weights = nn.Linear(embed_dims,\n num_heads * num_levels * num_points)\n self.value_proj = nn.Linear(embed_dims, embed_dims)\n self.output_proj = nn.Linear(embed_dims, embed_dims)\n self.init_weights()\n\n def init_weights(self):\n \"\"\"Default initialization for Parameters of Module.\"\"\"\n constant_init(self.sampling_offsets, 0.)\n thetas = torch.arange(\n self.num_heads,\n dtype=torch.float32) * (2.0 * math.pi / self.num_heads)\n grid_init = torch.stack([thetas.cos(), thetas.sin()], -1)\n grid_init = (grid_init /\n grid_init.abs().max(-1, keepdim=True)[0]).view(\n self.num_heads, 1, 1,\n 2).repeat(1, self.num_levels, self.num_points, 1)\n for i in range(self.num_points):\n grid_init[:, :, i, :] *= i + 1\n\n self.sampling_offsets.bias.data = grid_init.view(-1)\n constant_init(self.attention_weights, val=0., bias=0.)\n xavier_init(self.value_proj, distribution='uniform', bias=0.)\n xavier_init(self.output_proj, distribution='uniform', bias=0.)\n self._is_init = True\n\n @deprecated_api_warning({'residual': 'identity'},\n cls_name='MultiScaleDeformableAttention')\n def forward(self,\n query,\n key=None,\n value=None,\n identity=None,\n query_pos=None,\n key_padding_mask=None,\n reference_points=None,\n spatial_shapes=None,\n level_start_index=None,\n **kwargs):\n \"\"\"Forward Function of MultiScaleDeformAttention.\n\n Args:\n query (torch.Tensor): Query of Transformer with shape\n (num_query, bs, embed_dims).\n key (torch.Tensor): The key tensor with shape\n `(num_key, bs, embed_dims)`.\n value (torch.Tensor): The value tensor with shape\n `(num_key, bs, embed_dims)`.\n identity (torch.Tensor): The tensor used for addition, with the\n same shape as `query`. Default None. If None,\n `query` will be used.\n query_pos (torch.Tensor): The positional encoding for `query`.\n Default: None.\n key_pos (torch.Tensor): The positional encoding for `key`. Default\n None.\n reference_points (torch.Tensor): The normalized reference\n points with shape (bs, num_query, num_levels, 2),\n all elements is range in [0, 1], top-left (0,0),\n bottom-right (1, 1), including padding area.\n or (N, Length_{query}, num_levels, 4), add\n additional two dimensions is (w, h) to\n form reference boxes.\n key_padding_mask (torch.Tensor): ByteTensor for `query`, with\n shape [bs, num_key].\n spatial_shapes (torch.Tensor): Spatial shape of features in\n different levels. With shape (num_levels, 2),\n last dimension represents (h, w).\n level_start_index (torch.Tensor): The start index of each level.\n A tensor has shape ``(num_levels, )`` and can be represented\n as [0, h_0*w_0, h_0*w_0+h_1*w_1, ...].\n\n Returns:\n torch.Tensor: forwarded results with shape\n [num_query, bs, embed_dims].\n \"\"\"\n\n if value is None:\n value = query\n\n if identity is None:\n identity = query\n if query_pos is not None:\n query = query + query_pos\n if not self.batch_first:\n # change to (bs, num_query ,embed_dims)\n query = query.permute(1, 0, 2)\n value = value.permute(1, 0, 2)\n\n bs, num_query, _ = query.shape\n bs, num_value, _ = value.shape\n assert (spatial_shapes[:, 0] * spatial_shapes[:, 1]).sum() == num_value\n\n value = self.value_proj(value)\n if key_padding_mask is not None:\n value = value.masked_fill(key_padding_mask[..., None], 0.0)\n value = value.view(bs, num_value, self.num_heads, -1)\n sampling_offsets = self.sampling_offsets(query).view(\n bs, num_query, self.num_heads, self.num_levels, self.num_points, 2)\n attention_weights = self.attention_weights(query).view(\n bs, num_query, self.num_heads, self.num_levels * self.num_points)\n attention_weights = attention_weights.softmax(-1)\n\n attention_weights = attention_weights.view(bs, num_query,\n self.num_heads,\n self.num_levels,\n self.num_points)\n if reference_points.shape[-1] == 2:\n offset_normalizer = torch.stack(\n [spatial_shapes[..., 1], spatial_shapes[..., 0]], -1)\n sampling_locations = reference_points[:, :, None, :, None, :] \\\n + sampling_offsets \\\n / offset_normalizer[None, None, None, :, None, :]\n elif reference_points.shape[-1] == 4:\n sampling_locations = reference_points[:, :, None, :, None, :2] \\\n + sampling_offsets / self.num_points \\\n * reference_points[:, :, None, :, None, 2:] \\\n * 0.5\n else:\n raise ValueError(\n f'Last dim of reference_points must be'\n f' 2 or 4, but get {reference_points.shape[-1]} instead.')\n if torch.cuda.is_available() and value.is_cuda:\n output = MultiScaleDeformableAttnFunction.apply(\n value, spatial_shapes, level_start_index, sampling_locations,\n attention_weights, self.im2col_step)\n else:\n output = multi_scale_deformable_attn_pytorch(\n value, spatial_shapes, sampling_locations, attention_weights)\n\n output = self.output_proj(output)\n\n if not self.batch_first:\n # (num_query, bs ,embed_dims)\n output = output.permute(1, 0, 2)\n\n return self.dropout(output) + identity\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/nms.py", "content": "import os\n\nimport numpy as np\nimport torch\n\nfrom mmcv.utils import deprecated_api_warning\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['nms', 'softnms', 'nms_match', 'nms_rotated'])\n\n\n# This function is modified from: https://github.com/pytorch/vision/\nclass NMSop(torch.autograd.Function):\n\n @staticmethod\n def forward(ctx, bboxes, scores, iou_threshold, offset, score_threshold,\n max_num):\n is_filtering_by_score = score_threshold > 0\n if is_filtering_by_score:\n valid_mask = scores > score_threshold\n bboxes, scores = bboxes[valid_mask], scores[valid_mask]\n valid_inds = torch.nonzero(\n valid_mask, as_tuple=False).squeeze(dim=1)\n\n inds = ext_module.nms(\n bboxes, scores, iou_threshold=float(iou_threshold), offset=offset)\n\n if max_num > 0:\n inds = inds[:max_num]\n if is_filtering_by_score:\n inds = valid_inds[inds]\n return inds\n\n @staticmethod\n def symbolic(g, bboxes, scores, iou_threshold, offset, score_threshold,\n max_num):\n from ..onnx import is_custom_op_loaded\n has_custom_op = is_custom_op_loaded()\n # TensorRT nms plugin is aligned with original nms in ONNXRuntime\n is_trt_backend = os.environ.get('ONNX_BACKEND') == 'MMCVTensorRT'\n if has_custom_op and (not is_trt_backend):\n return g.op(\n 'mmcv::NonMaxSuppression',\n bboxes,\n scores,\n iou_threshold_f=float(iou_threshold),\n offset_i=int(offset))\n else:\n from torch.onnx.symbolic_opset9 import select, squeeze, unsqueeze\n from ..onnx.onnx_utils.symbolic_helper import _size_helper\n\n boxes = unsqueeze(g, bboxes, 0)\n scores = unsqueeze(g, unsqueeze(g, scores, 0), 0)\n\n if max_num > 0:\n max_num = g.op(\n 'Constant',\n value_t=torch.tensor(max_num, dtype=torch.long))\n else:\n dim = g.op('Constant', value_t=torch.tensor(0))\n max_num = _size_helper(g, bboxes, dim)\n max_output_per_class = max_num\n iou_threshold = g.op(\n 'Constant',\n value_t=torch.tensor([iou_threshold], dtype=torch.float))\n score_threshold = g.op(\n 'Constant',\n value_t=torch.tensor([score_threshold], dtype=torch.float))\n nms_out = g.op('NonMaxSuppression', boxes, scores,\n max_output_per_class, iou_threshold,\n score_threshold)\n return squeeze(\n g,\n select(\n g, nms_out, 1,\n g.op(\n 'Constant',\n value_t=torch.tensor([2], dtype=torch.long))), 1)\n\n\nclass SoftNMSop(torch.autograd.Function):\n\n @staticmethod\n def forward(ctx, boxes, scores, iou_threshold, sigma, min_score, method,\n offset):\n dets = boxes.new_empty((boxes.size(0), 5), device='cpu')\n inds = ext_module.softnms(\n boxes.cpu(),\n scores.cpu(),\n dets.cpu(),\n iou_threshold=float(iou_threshold),\n sigma=float(sigma),\n min_score=float(min_score),\n method=int(method),\n offset=int(offset))\n return dets, inds\n\n @staticmethod\n def symbolic(g, boxes, scores, iou_threshold, sigma, min_score, method,\n offset):\n from packaging import version\n assert version.parse(torch.__version__) >= version.parse('1.7.0')\n nms_out = g.op(\n 'mmcv::SoftNonMaxSuppression',\n boxes,\n scores,\n iou_threshold_f=float(iou_threshold),\n sigma_f=float(sigma),\n min_score_f=float(min_score),\n method_i=int(method),\n offset_i=int(offset),\n outputs=2)\n return nms_out\n\n\n@deprecated_api_warning({'iou_thr': 'iou_threshold'})\ndef nms(boxes, scores, iou_threshold, offset=0, score_threshold=0, max_num=-1):\n \"\"\"Dispatch to either CPU or GPU NMS implementations.\n\n The input can be either torch tensor or numpy array. GPU NMS will be used\n if the input is gpu tensor, otherwise CPU NMS\n will be used. The returned type will always be the same as inputs.\n\n Arguments:\n boxes (torch.Tensor or np.ndarray): boxes in shape (N, 4).\n scores (torch.Tensor or np.ndarray): scores in shape (N, ).\n iou_threshold (float): IoU threshold for NMS.\n offset (int, 0 or 1): boxes' width or height is (x2 - x1 + offset).\n score_threshold (float): score threshold for NMS.\n max_num (int): maximum number of boxes after NMS.\n\n Returns:\n tuple: kept dets (boxes and scores) and indice, which always have\n the same data type as the input.\n\n Example:\n >>> boxes = np.array([[49.1, 32.4, 51.0, 35.9],\n >>> [49.3, 32.9, 51.0, 35.3],\n >>> [49.2, 31.8, 51.0, 35.4],\n >>> [35.1, 11.5, 39.1, 15.7],\n >>> [35.6, 11.8, 39.3, 14.2],\n >>> [35.3, 11.5, 39.9, 14.5],\n >>> [35.2, 11.7, 39.7, 15.7]], dtype=np.float32)\n >>> scores = np.array([0.9, 0.9, 0.5, 0.5, 0.5, 0.4, 0.3],\\\n dtype=np.float32)\n >>> iou_threshold = 0.6\n >>> dets, inds = nms(boxes, scores, iou_threshold)\n >>> assert len(inds) == len(dets) == 3\n \"\"\"\n assert isinstance(boxes, (torch.Tensor, np.ndarray))\n assert isinstance(scores, (torch.Tensor, np.ndarray))\n is_numpy = False\n if isinstance(boxes, np.ndarray):\n is_numpy = True\n boxes = torch.from_numpy(boxes)\n if isinstance(scores, np.ndarray):\n scores = torch.from_numpy(scores)\n assert boxes.size(1) == 4\n assert boxes.size(0) == scores.size(0)\n assert offset in (0, 1)\n\n if torch.__version__ == 'parrots':\n indata_list = [boxes, scores]\n indata_dict = {\n 'iou_threshold': float(iou_threshold),\n 'offset': int(offset)\n }\n inds = ext_module.nms(*indata_list, **indata_dict)\n else:\n inds = NMSop.apply(boxes, scores, iou_threshold, offset,\n score_threshold, max_num)\n dets = torch.cat((boxes[inds], scores[inds].reshape(-1, 1)), dim=1)\n if is_numpy:\n dets = dets.cpu().numpy()\n inds = inds.cpu().numpy()\n return dets, inds\n\n\n@deprecated_api_warning({'iou_thr': 'iou_threshold'})\ndef soft_nms(boxes,\n scores,\n iou_threshold=0.3,\n sigma=0.5,\n min_score=1e-3,\n method='linear',\n offset=0):\n \"\"\"Dispatch to only CPU Soft NMS implementations.\n\n The input can be either a torch tensor or numpy array.\n The returned type will always be the same as inputs.\n\n Args:\n boxes (torch.Tensor or np.ndarray): boxes in shape (N, 4).\n scores (torch.Tensor or np.ndarray): scores in shape (N, ).\n iou_threshold (float): IoU threshold for NMS.\n sigma (float): hyperparameter for gaussian method\n min_score (float): score filter threshold\n method (str): either 'linear' or 'gaussian'\n offset (int, 0 or 1): boxes' width or height is (x2 - x1 + offset).\n\n Returns:\n tuple: kept dets (boxes and scores) and indice, which always have\n the same data type as the input.\n\n Example:\n >>> boxes = np.array([[4., 3., 5., 3.],\n >>> [4., 3., 5., 4.],\n >>> [3., 1., 3., 1.],\n >>> [3., 1., 3., 1.],\n >>> [3., 1., 3., 1.],\n >>> [3., 1., 3., 1.]], dtype=np.float32)\n >>> scores = np.array([0.9, 0.9, 0.5, 0.5, 0.4, 0.0], dtype=np.float32)\n >>> iou_threshold = 0.6\n >>> dets, inds = soft_nms(boxes, scores, iou_threshold, sigma=0.5)\n >>> assert len(inds) == len(dets) == 5\n \"\"\"\n\n assert isinstance(boxes, (torch.Tensor, np.ndarray))\n assert isinstance(scores, (torch.Tensor, np.ndarray))\n is_numpy = False\n if isinstance(boxes, np.ndarray):\n is_numpy = True\n boxes = torch.from_numpy(boxes)\n if isinstance(scores, np.ndarray):\n scores = torch.from_numpy(scores)\n assert boxes.size(1) == 4\n assert boxes.size(0) == scores.size(0)\n assert offset in (0, 1)\n method_dict = {'naive': 0, 'linear': 1, 'gaussian': 2}\n assert method in method_dict.keys()\n\n if torch.__version__ == 'parrots':\n dets = boxes.new_empty((boxes.size(0), 5), device='cpu')\n indata_list = [boxes.cpu(), scores.cpu(), dets.cpu()]\n indata_dict = {\n 'iou_threshold': float(iou_threshold),\n 'sigma': float(sigma),\n 'min_score': min_score,\n 'method': method_dict[method],\n 'offset': int(offset)\n }\n inds = ext_module.softnms(*indata_list, **indata_dict)\n else:\n dets, inds = SoftNMSop.apply(boxes.cpu(), scores.cpu(),\n float(iou_threshold), float(sigma),\n float(min_score), method_dict[method],\n int(offset))\n\n dets = dets[:inds.size(0)]\n\n if is_numpy:\n dets = dets.cpu().numpy()\n inds = inds.cpu().numpy()\n return dets, inds\n else:\n return dets.to(device=boxes.device), inds.to(device=boxes.device)\n\n\ndef batched_nms(boxes, scores, idxs, nms_cfg, class_agnostic=False):\n r\"\"\"Performs non-maximum suppression in a batched fashion.\n\n Modified from `torchvision/ops/boxes.py#L39\n `_.\n In order to perform NMS independently per class, we add an offset to all\n the boxes. The offset is dependent only on the class idx, and is large\n enough so that boxes from different classes do not overlap.\n\n Note:\n In v1.4.1 and later, ``batched_nms`` supports skipping the NMS and\n returns sorted raw results when `nms_cfg` is None.\n\n Args:\n boxes (torch.Tensor): boxes in shape (N, 4).\n scores (torch.Tensor): scores in shape (N, ).\n idxs (torch.Tensor): each index value correspond to a bbox cluster,\n and NMS will not be applied between elements of different idxs,\n shape (N, ).\n nms_cfg (dict | None): Supports skipping the nms when `nms_cfg`\n is None, otherwise it should specify nms type and other\n parameters like `iou_thr`. Possible keys includes the following.\n\n - iou_thr (float): IoU threshold used for NMS.\n - split_thr (float): threshold number of boxes. In some cases the\n number of boxes is large (e.g., 200k). To avoid OOM during\n training, the users could set `split_thr` to a small value.\n If the number of boxes is greater than the threshold, it will\n perform NMS on each group of boxes separately and sequentially.\n Defaults to 10000.\n class_agnostic (bool): if true, nms is class agnostic,\n i.e. IoU thresholding happens over all boxes,\n regardless of the predicted class.\n\n Returns:\n tuple: kept dets and indice.\n\n - boxes (Tensor): Bboxes with score after nms, has shape\n (num_bboxes, 5). last dimension 5 arrange as\n (x1, y1, x2, y2, score)\n - keep (Tensor): The indices of remaining boxes in input\n boxes.\n \"\"\"\n # skip nms when nms_cfg is None\n if nms_cfg is None:\n scores, inds = scores.sort(descending=True)\n boxes = boxes[inds]\n return torch.cat([boxes, scores[:, None]], -1), inds\n\n nms_cfg_ = nms_cfg.copy()\n class_agnostic = nms_cfg_.pop('class_agnostic', class_agnostic)\n if class_agnostic:\n boxes_for_nms = boxes\n else:\n max_coordinate = boxes.max()\n offsets = idxs.to(boxes) * (max_coordinate + torch.tensor(1).to(boxes))\n boxes_for_nms = boxes + offsets[:, None]\n\n nms_type = nms_cfg_.pop('type', 'nms')\n nms_op = eval(nms_type)\n\n split_thr = nms_cfg_.pop('split_thr', 10000)\n # Won't split to multiple nms nodes when exporting to onnx\n if boxes_for_nms.shape[0] < split_thr or torch.onnx.is_in_onnx_export():\n dets, keep = nms_op(boxes_for_nms, scores, **nms_cfg_)\n boxes = boxes[keep]\n # -1 indexing works abnormal in TensorRT\n # This assumes `dets` has 5 dimensions where\n # the last dimension is score.\n # TODO: more elegant way to handle the dimension issue.\n # Some type of nms would reweight the score, such as SoftNMS\n scores = dets[:, 4]\n else:\n max_num = nms_cfg_.pop('max_num', -1)\n total_mask = scores.new_zeros(scores.size(), dtype=torch.bool)\n # Some type of nms would reweight the score, such as SoftNMS\n scores_after_nms = scores.new_zeros(scores.size())\n for id in torch.unique(idxs):\n mask = (idxs == id).nonzero(as_tuple=False).view(-1)\n dets, keep = nms_op(boxes_for_nms[mask], scores[mask], **nms_cfg_)\n total_mask[mask[keep]] = True\n scores_after_nms[mask[keep]] = dets[:, -1]\n keep = total_mask.nonzero(as_tuple=False).view(-1)\n\n scores, inds = scores_after_nms[keep].sort(descending=True)\n keep = keep[inds]\n boxes = boxes[keep]\n\n if max_num > 0:\n keep = keep[:max_num]\n boxes = boxes[:max_num]\n scores = scores[:max_num]\n\n boxes = torch.cat([boxes, scores[:, None]], -1)\n return boxes, keep\n\n\ndef nms_match(dets, iou_threshold):\n \"\"\"Matched dets into different groups by NMS.\n\n NMS match is Similar to NMS but when a bbox is suppressed, nms match will\n record the indice of suppressed bbox and form a group with the indice of\n kept bbox. In each group, indice is sorted as score order.\n\n Args:\n dets (torch.Tensor | np.ndarray): Det boxes with scores, shape (N, 5).\n iou_thr (float): IoU thresh for NMS.\n\n Returns:\n list[torch.Tensor | np.ndarray]: The outer list corresponds different\n matched group, the inner Tensor corresponds the indices for a group\n in score order.\n \"\"\"\n if dets.shape[0] == 0:\n matched = []\n else:\n assert dets.shape[-1] == 5, 'inputs dets.shape should be (N, 5), ' \\\n f'but get {dets.shape}'\n if isinstance(dets, torch.Tensor):\n dets_t = dets.detach().cpu()\n else:\n dets_t = torch.from_numpy(dets)\n indata_list = [dets_t]\n indata_dict = {'iou_threshold': float(iou_threshold)}\n matched = ext_module.nms_match(*indata_list, **indata_dict)\n if torch.__version__ == 'parrots':\n matched = matched.tolist()\n\n if isinstance(dets, torch.Tensor):\n return [dets.new_tensor(m, dtype=torch.long) for m in matched]\n else:\n return [np.array(m, dtype=int) for m in matched]\n\n\ndef nms_rotated(dets, scores, iou_threshold, labels=None, clockwise=True):\n \"\"\"Performs non-maximum suppression (NMS) on the rotated boxes according to\n their intersection-over-union (IoU).\n\n Rotated NMS iteratively removes lower scoring rotated boxes which have an\n IoU greater than iou_threshold with another (higher scoring) rotated box.\n\n Args:\n dets (Tensor): Rotated boxes in shape (N, 5). They are expected to\n be in (x_ctr, y_ctr, width, height, angle_radian) format.\n scores (Tensor): scores in shape (N, ).\n iou_threshold (float): IoU thresh for NMS.\n labels (Tensor): boxes' label in shape (N,).\n clockwise (bool): flag indicating whether the positive angular\n orientation is clockwise. default True.\n `New in version 1.4.3.`\n\n Returns:\n tuple: kept dets(boxes and scores) and indice, which is always the\n same data type as the input.\n \"\"\"\n if dets.shape[0] == 0:\n return dets, None\n if not clockwise:\n flip_mat = dets.new_ones(dets.shape[-1])\n flip_mat[-1] = -1\n dets_cw = dets * flip_mat\n else:\n dets_cw = dets\n multi_label = labels is not None\n if multi_label:\n dets_wl = torch.cat((dets_cw, labels.unsqueeze(1)), 1)\n else:\n dets_wl = dets_cw\n _, order = scores.sort(0, descending=True)\n dets_sorted = dets_wl.index_select(0, order)\n\n if torch.__version__ == 'parrots':\n keep_inds = ext_module.nms_rotated(\n dets_wl,\n scores,\n order,\n dets_sorted,\n iou_threshold=iou_threshold,\n multi_label=multi_label)\n else:\n keep_inds = ext_module.nms_rotated(dets_wl, scores, order, dets_sorted,\n iou_threshold, multi_label)\n dets = torch.cat((dets[keep_inds], scores[keep_inds].reshape(-1, 1)),\n dim=1)\n return dets, keep_inds\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/pixel_group.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numpy as np\nimport torch\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['pixel_group'])\n\n\ndef pixel_group(score, mask, embedding, kernel_label, kernel_contour,\n kernel_region_num, distance_threshold):\n \"\"\"Group pixels into text instances, which is widely used text detection\n methods.\n\n Arguments:\n score (np.array or torch.Tensor): The foreground score with size hxw.\n mask (np.array or Tensor): The foreground mask with size hxw.\n embedding (np.array or torch.Tensor): The embedding with size hxwxc to\n distinguish instances.\n kernel_label (np.array or torch.Tensor): The instance kernel index with\n size hxw.\n kernel_contour (np.array or torch.Tensor): The kernel contour with\n size hxw.\n kernel_region_num (int): The instance kernel region number.\n distance_threshold (float): The embedding distance threshold between\n kernel and pixel in one instance.\n\n Returns:\n list[list[float]]: The instance coordinates and attributes list. Each\n element consists of averaged confidence, pixel number, and coordinates\n (x_i, y_i for all pixels) in order.\n \"\"\"\n assert isinstance(score, (torch.Tensor, np.ndarray))\n assert isinstance(mask, (torch.Tensor, np.ndarray))\n assert isinstance(embedding, (torch.Tensor, np.ndarray))\n assert isinstance(kernel_label, (torch.Tensor, np.ndarray))\n assert isinstance(kernel_contour, (torch.Tensor, np.ndarray))\n assert isinstance(kernel_region_num, int)\n assert isinstance(distance_threshold, float)\n\n if isinstance(score, np.ndarray):\n score = torch.from_numpy(score)\n if isinstance(mask, np.ndarray):\n mask = torch.from_numpy(mask)\n if isinstance(embedding, np.ndarray):\n embedding = torch.from_numpy(embedding)\n if isinstance(kernel_label, np.ndarray):\n kernel_label = torch.from_numpy(kernel_label)\n if isinstance(kernel_contour, np.ndarray):\n kernel_contour = torch.from_numpy(kernel_contour)\n\n if torch.__version__ == 'parrots':\n label = ext_module.pixel_group(\n score,\n mask,\n embedding,\n kernel_label,\n kernel_contour,\n kernel_region_num=kernel_region_num,\n distance_threshold=distance_threshold)\n label = label.tolist()\n label = label[0]\n list_index = kernel_region_num\n pixel_assignment = []\n for x in range(kernel_region_num):\n pixel_assignment.append(\n np.array(\n label[list_index:list_index + int(label[x])],\n dtype=np.float))\n list_index = list_index + int(label[x])\n else:\n pixel_assignment = ext_module.pixel_group(score, mask, embedding,\n kernel_label, kernel_contour,\n kernel_region_num,\n distance_threshold)\n return pixel_assignment\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/point_sample.py", "content": "# Modified from https://github.com/facebookresearch/detectron2/tree/master/projects/PointRend # noqa\n\nfrom os import path as osp\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.nn.modules.utils import _pair\nfrom torch.onnx.operators import shape_as_tensor\n\n\ndef bilinear_grid_sample(im, grid, align_corners=False):\n \"\"\"Given an input and a flow-field grid, computes the output using input\n values and pixel locations from grid. Supported only bilinear interpolation\n method to sample the input pixels.\n\n Args:\n im (torch.Tensor): Input feature map, shape (N, C, H, W)\n grid (torch.Tensor): Point coordinates, shape (N, Hg, Wg, 2)\n align_corners {bool}: If set to True, the extrema (-1 and 1) are\n considered as referring to the center points of the input’s\n corner pixels. If set to False, they are instead considered as\n referring to the corner points of the input’s corner pixels,\n making the sampling more resolution agnostic.\n\n Returns:\n torch.Tensor: A tensor with sampled points, shape (N, C, Hg, Wg)\n \"\"\"\n n, c, h, w = im.shape\n gn, gh, gw, _ = grid.shape\n assert n == gn\n\n x = grid[:, :, :, 0]\n y = grid[:, :, :, 1]\n\n if align_corners:\n x = ((x + 1) / 2) * (w - 1)\n y = ((y + 1) / 2) * (h - 1)\n else:\n x = ((x + 1) * w - 1) / 2\n y = ((y + 1) * h - 1) / 2\n\n x = x.view(n, -1)\n y = y.view(n, -1)\n\n x0 = torch.floor(x).long()\n y0 = torch.floor(y).long()\n x1 = x0 + 1\n y1 = y0 + 1\n\n wa = ((x1 - x) * (y1 - y)).unsqueeze(1)\n wb = ((x1 - x) * (y - y0)).unsqueeze(1)\n wc = ((x - x0) * (y1 - y)).unsqueeze(1)\n wd = ((x - x0) * (y - y0)).unsqueeze(1)\n\n # Apply default for grid_sample function zero padding\n im_padded = F.pad(im, pad=[1, 1, 1, 1], mode='constant', value=0)\n padded_h = h + 2\n padded_w = w + 2\n # save points positions after padding\n x0, x1, y0, y1 = x0 + 1, x1 + 1, y0 + 1, y1 + 1\n\n # Clip coordinates to padded image size\n x0 = torch.where(x0 < 0, torch.tensor(0), x0)\n x0 = torch.where(x0 > padded_w - 1, torch.tensor(padded_w - 1), x0)\n x1 = torch.where(x1 < 0, torch.tensor(0), x1)\n x1 = torch.where(x1 > padded_w - 1, torch.tensor(padded_w - 1), x1)\n y0 = torch.where(y0 < 0, torch.tensor(0), y0)\n y0 = torch.where(y0 > padded_h - 1, torch.tensor(padded_h - 1), y0)\n y1 = torch.where(y1 < 0, torch.tensor(0), y1)\n y1 = torch.where(y1 > padded_h - 1, torch.tensor(padded_h - 1), y1)\n\n im_padded = im_padded.view(n, c, -1)\n\n x0_y0 = (x0 + y0 * padded_w).unsqueeze(1).expand(-1, c, -1)\n x0_y1 = (x0 + y1 * padded_w).unsqueeze(1).expand(-1, c, -1)\n x1_y0 = (x1 + y0 * padded_w).unsqueeze(1).expand(-1, c, -1)\n x1_y1 = (x1 + y1 * padded_w).unsqueeze(1).expand(-1, c, -1)\n\n Ia = torch.gather(im_padded, 2, x0_y0)\n Ib = torch.gather(im_padded, 2, x0_y1)\n Ic = torch.gather(im_padded, 2, x1_y0)\n Id = torch.gather(im_padded, 2, x1_y1)\n\n return (Ia * wa + Ib * wb + Ic * wc + Id * wd).reshape(n, c, gh, gw)\n\n\ndef is_in_onnx_export_without_custom_ops():\n from mmcv.ops import get_onnxruntime_op_path\n ort_custom_op_path = get_onnxruntime_op_path()\n return torch.onnx.is_in_onnx_export(\n ) and not osp.exists(ort_custom_op_path)\n\n\ndef normalize(grid):\n \"\"\"Normalize input grid from [-1, 1] to [0, 1]\n\n Args:\n grid (torch.Tensor): The grid to be normalize, range [-1, 1].\n\n Returns:\n torch.Tensor: Normalized grid, range [0, 1].\n \"\"\"\n\n return (grid + 1.0) / 2.0\n\n\ndef denormalize(grid):\n \"\"\"Denormalize input grid from range [0, 1] to [-1, 1]\n\n Args:\n grid (torch.Tensor): The grid to be denormalize, range [0, 1].\n\n Returns:\n torch.Tensor: Denormalized grid, range [-1, 1].\n \"\"\"\n\n return grid * 2.0 - 1.0\n\n\ndef generate_grid(num_grid, size, device):\n \"\"\"Generate regular square grid of points in [0, 1] x [0, 1] coordinate\n space.\n\n Args:\n num_grid (int): The number of grids to sample, one for each region.\n size (tuple[int, int]): The side size of the regular grid.\n device (torch.device): Desired device of returned tensor.\n\n Returns:\n torch.Tensor: A tensor of shape (num_grid, size[0]*size[1], 2) that\n contains coordinates for the regular grids.\n \"\"\"\n\n affine_trans = torch.tensor([[[1., 0., 0.], [0., 1., 0.]]], device=device)\n grid = F.affine_grid(\n affine_trans, torch.Size((1, 1, *size)), align_corners=False)\n grid = normalize(grid)\n return grid.view(1, -1, 2).expand(num_grid, -1, -1)\n\n\ndef rel_roi_point_to_abs_img_point(rois, rel_roi_points):\n \"\"\"Convert roi based relative point coordinates to image based absolute\n point coordinates.\n\n Args:\n rois (torch.Tensor): RoIs or BBoxes, shape (N, 4) or (N, 5)\n rel_roi_points (torch.Tensor): Point coordinates inside RoI, relative\n to RoI, location, range (0, 1), shape (N, P, 2)\n Returns:\n torch.Tensor: Image based absolute point coordinates, shape (N, P, 2)\n \"\"\"\n\n with torch.no_grad():\n assert rel_roi_points.size(0) == rois.size(0)\n assert rois.dim() == 2\n assert rel_roi_points.dim() == 3\n assert rel_roi_points.size(2) == 2\n # remove batch idx\n if rois.size(1) == 5:\n rois = rois[:, 1:]\n abs_img_points = rel_roi_points.clone()\n # To avoid an error during exporting to onnx use independent\n # variables instead inplace computation\n xs = abs_img_points[:, :, 0] * (rois[:, None, 2] - rois[:, None, 0])\n ys = abs_img_points[:, :, 1] * (rois[:, None, 3] - rois[:, None, 1])\n xs += rois[:, None, 0]\n ys += rois[:, None, 1]\n abs_img_points = torch.stack([xs, ys], dim=2)\n return abs_img_points\n\n\ndef get_shape_from_feature_map(x):\n \"\"\"Get spatial resolution of input feature map considering exporting to\n onnx mode.\n\n Args:\n x (torch.Tensor): Input tensor, shape (N, C, H, W)\n\n Returns:\n torch.Tensor: Spatial resolution (width, height), shape (1, 1, 2)\n \"\"\"\n if torch.onnx.is_in_onnx_export():\n img_shape = shape_as_tensor(x)[2:].flip(0).view(1, 1, 2).to(\n x.device).float()\n else:\n img_shape = torch.tensor(x.shape[2:]).flip(0).view(1, 1, 2).to(\n x.device).float()\n return img_shape\n\n\ndef abs_img_point_to_rel_img_point(abs_img_points, img, spatial_scale=1.):\n \"\"\"Convert image based absolute point coordinates to image based relative\n coordinates for sampling.\n\n Args:\n abs_img_points (torch.Tensor): Image based absolute point coordinates,\n shape (N, P, 2)\n img (tuple or torch.Tensor): (height, width) of image or feature map.\n spatial_scale (float, optional): Scale points by this factor.\n Default: 1.\n\n Returns:\n Tensor: Image based relative point coordinates for sampling, shape\n (N, P, 2).\n \"\"\"\n\n assert (isinstance(img, tuple) and len(img) == 2) or \\\n (isinstance(img, torch.Tensor) and len(img.shape) == 4)\n\n if isinstance(img, tuple):\n h, w = img\n scale = torch.tensor([w, h],\n dtype=torch.float,\n device=abs_img_points.device)\n scale = scale.view(1, 1, 2)\n else:\n scale = get_shape_from_feature_map(img)\n\n return abs_img_points / scale * spatial_scale\n\n\ndef rel_roi_point_to_rel_img_point(rois,\n rel_roi_points,\n img,\n spatial_scale=1.):\n \"\"\"Convert roi based relative point coordinates to image based absolute\n point coordinates.\n\n Args:\n rois (torch.Tensor): RoIs or BBoxes, shape (N, 4) or (N, 5)\n rel_roi_points (torch.Tensor): Point coordinates inside RoI, relative\n to RoI, location, range (0, 1), shape (N, P, 2)\n img (tuple or torch.Tensor): (height, width) of image or feature map.\n spatial_scale (float, optional): Scale points by this factor.\n Default: 1.\n\n Returns:\n torch.Tensor: Image based relative point coordinates for sampling,\n shape (N, P, 2).\n \"\"\"\n\n abs_img_point = rel_roi_point_to_abs_img_point(rois, rel_roi_points)\n rel_img_point = abs_img_point_to_rel_img_point(abs_img_point, img,\n spatial_scale)\n\n return rel_img_point\n\n\ndef point_sample(input, points, align_corners=False, **kwargs):\n \"\"\"A wrapper around :func:`grid_sample` to support 3D point_coords tensors\n Unlike :func:`torch.nn.functional.grid_sample` it assumes point_coords to\n lie inside ``[0, 1] x [0, 1]`` square.\n\n Args:\n input (torch.Tensor): Feature map, shape (N, C, H, W).\n points (torch.Tensor): Image based absolute point coordinates\n (normalized), range [0, 1] x [0, 1], shape (N, P, 2) or\n (N, Hgrid, Wgrid, 2).\n align_corners (bool, optional): Whether align_corners.\n Default: False\n\n Returns:\n torch.Tensor: Features of `point` on `input`, shape (N, C, P) or\n (N, C, Hgrid, Wgrid).\n \"\"\"\n\n add_dim = False\n if points.dim() == 3:\n add_dim = True\n points = points.unsqueeze(2)\n if is_in_onnx_export_without_custom_ops():\n # If custom ops for onnx runtime not compiled use python\n # implementation of grid_sample function to make onnx graph\n # with supported nodes\n output = bilinear_grid_sample(\n input, denormalize(points), align_corners=align_corners)\n else:\n output = F.grid_sample(\n input, denormalize(points), align_corners=align_corners, **kwargs)\n if add_dim:\n output = output.squeeze(3)\n return output\n\n\nclass SimpleRoIAlign(nn.Module):\n\n def __init__(self, output_size, spatial_scale, aligned=True):\n \"\"\"Simple RoI align in PointRend, faster than standard RoIAlign.\n\n Args:\n output_size (tuple[int]): h, w\n spatial_scale (float): scale the input boxes by this number\n aligned (bool): if False, use the legacy implementation in\n MMDetection, align_corners=True will be used in F.grid_sample.\n If True, align the results more perfectly.\n \"\"\"\n\n super(SimpleRoIAlign, self).__init__()\n self.output_size = _pair(output_size)\n self.spatial_scale = float(spatial_scale)\n # to be consistent with other RoI ops\n self.use_torchvision = False\n self.aligned = aligned\n\n def forward(self, features, rois):\n num_imgs = features.size(0)\n num_rois = rois.size(0)\n rel_roi_points = generate_grid(\n num_rois, self.output_size, device=rois.device)\n\n if torch.onnx.is_in_onnx_export():\n rel_img_points = rel_roi_point_to_rel_img_point(\n rois, rel_roi_points, features, self.spatial_scale)\n rel_img_points = rel_img_points.reshape(num_imgs, -1,\n *rel_img_points.shape[1:])\n point_feats = point_sample(\n features, rel_img_points, align_corners=not self.aligned)\n point_feats = point_feats.transpose(1, 2)\n else:\n point_feats = []\n for batch_ind in range(num_imgs):\n # unravel batch dim\n feat = features[batch_ind].unsqueeze(0)\n inds = (rois[:, 0].long() == batch_ind)\n if inds.any():\n rel_img_points = rel_roi_point_to_rel_img_point(\n rois[inds], rel_roi_points[inds], feat,\n self.spatial_scale).unsqueeze(0)\n point_feat = point_sample(\n feat, rel_img_points, align_corners=not self.aligned)\n point_feat = point_feat.squeeze(0).transpose(0, 1)\n point_feats.append(point_feat)\n\n point_feats = torch.cat(point_feats, dim=0)\n\n channels = features.size(1)\n roi_feats = point_feats.reshape(num_rois, channels, *self.output_size)\n\n return roi_feats\n\n def __repr__(self):\n format_str = self.__class__.__name__\n format_str += '(output_size={}, spatial_scale={}'.format(\n self.output_size, self.spatial_scale)\n return format_str\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/points_in_boxes.py", "content": "import torch\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', [\n 'points_in_boxes_part_forward', 'points_in_boxes_cpu_forward',\n 'points_in_boxes_all_forward'\n])\n\n\ndef points_in_boxes_part(points, boxes):\n \"\"\"Find the box in which each point is (CUDA).\n\n Args:\n points (torch.Tensor): [B, M, 3], [x, y, z] in LiDAR/DEPTH coordinate.\n boxes (torch.Tensor): [B, T, 7],\n num_valid_boxes <= T, [x, y, z, x_size, y_size, z_size, rz] in\n LiDAR/DEPTH coordinate, (x, y, z) is the bottom center.\n\n Returns:\n torch.Tensor: Return the box indices of points with the shape of\n (B, M). Default background = -1.\n \"\"\"\n assert points.shape[0] == boxes.shape[0], \\\n 'Points and boxes should have the same batch size, ' \\\n f'but got {points.shape[0]} and {boxes.shape[0]}'\n assert boxes.shape[2] == 7, \\\n 'boxes dimension should be 7, ' \\\n f'but got unexpected shape {boxes.shape[2]}'\n assert points.shape[2] == 3, \\\n 'points dimension should be 3, ' \\\n f'but got unexpected shape {points.shape[2]}'\n batch_size, num_points, _ = points.shape\n\n box_idxs_of_pts = points.new_zeros((batch_size, num_points),\n dtype=torch.int).fill_(-1)\n\n # If manually put the tensor 'points' or 'boxes' on a device\n # which is not the current device, some temporary variables\n # will be created on the current device in the cuda op,\n # and the output will be incorrect.\n # Therefore, we force the current device to be the same\n # as the device of the tensors if it was not.\n # Please refer to https://github.com/open-mmlab/mmdetection3d/issues/305\n # for the incorrect output before the fix.\n points_device = points.get_device()\n assert points_device == boxes.get_device(), \\\n 'Points and boxes should be put on the same device'\n if torch.cuda.current_device() != points_device:\n torch.cuda.set_device(points_device)\n\n ext_module.points_in_boxes_part_forward(boxes.contiguous(),\n points.contiguous(),\n box_idxs_of_pts)\n\n return box_idxs_of_pts\n\n\ndef points_in_boxes_cpu(points, boxes):\n \"\"\"Find all boxes in which each point is (CPU). The CPU version of\n :meth:`points_in_boxes_all`.\n\n Args:\n points (torch.Tensor): [B, M, 3], [x, y, z] in\n LiDAR/DEPTH coordinate\n boxes (torch.Tensor): [B, T, 7],\n num_valid_boxes <= T, [x, y, z, x_size, y_size, z_size, rz],\n (x, y, z) is the bottom center.\n\n Returns:\n torch.Tensor: Return the box indices of points with the shape of\n (B, M, T). Default background = 0.\n \"\"\"\n assert points.shape[0] == boxes.shape[0], \\\n 'Points and boxes should have the same batch size, ' \\\n f'but got {points.shape[0]} and {boxes.shape[0]}'\n assert boxes.shape[2] == 7, \\\n 'boxes dimension should be 7, ' \\\n f'but got unexpected shape {boxes.shape[2]}'\n assert points.shape[2] == 3, \\\n 'points dimension should be 3, ' \\\n f'but got unexpected shape {points.shape[2]}'\n batch_size, num_points, _ = points.shape\n num_boxes = boxes.shape[1]\n\n point_indices = points.new_zeros((batch_size, num_boxes, num_points),\n dtype=torch.int)\n for b in range(batch_size):\n ext_module.points_in_boxes_cpu_forward(boxes[b].float().contiguous(),\n points[b].float().contiguous(),\n point_indices[b])\n point_indices = point_indices.transpose(1, 2)\n\n return point_indices\n\n\ndef points_in_boxes_all(points, boxes):\n \"\"\"Find all boxes in which each point is (CUDA).\n\n Args:\n points (torch.Tensor): [B, M, 3], [x, y, z] in LiDAR/DEPTH coordinate\n boxes (torch.Tensor): [B, T, 7],\n num_valid_boxes <= T, [x, y, z, x_size, y_size, z_size, rz],\n (x, y, z) is the bottom center.\n\n Returns:\n torch.Tensor: Return the box indices of points with the shape of\n (B, M, T). Default background = 0.\n \"\"\"\n assert boxes.shape[0] == points.shape[0], \\\n 'Points and boxes should have the same batch size, ' \\\n f'but got {boxes.shape[0]} and {boxes.shape[0]}'\n assert boxes.shape[2] == 7, \\\n 'boxes dimension should be 7, ' \\\n f'but got unexpected shape {boxes.shape[2]}'\n assert points.shape[2] == 3, \\\n 'points dimension should be 3, ' \\\n f'but got unexpected shape {points.shape[2]}'\n batch_size, num_points, _ = points.shape\n num_boxes = boxes.shape[1]\n\n box_idxs_of_pts = points.new_zeros((batch_size, num_points, num_boxes),\n dtype=torch.int).fill_(0)\n\n # Same reason as line 25-32\n points_device = points.get_device()\n assert points_device == boxes.get_device(), \\\n 'Points and boxes should be put on the same device'\n if torch.cuda.current_device() != points_device:\n torch.cuda.set_device(points_device)\n\n ext_module.points_in_boxes_all_forward(boxes.contiguous(),\n points.contiguous(),\n box_idxs_of_pts)\n\n return box_idxs_of_pts\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/points_in_polygons.py", "content": "import torch\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['points_in_polygons_forward'])\n\n\ndef points_in_polygons(points, polygons):\n \"\"\"Judging whether points are inside polygons, which is used in the ATSS\n assignment for the rotated boxes.\n\n It should be noted that when the point is just at the polygon boundary, the\n judgment will be inaccurate, but the effect on assignment is limited.\n\n Args:\n points (torch.Tensor): It has shape (B, 2), indicating (x, y).\n M means the number of predicted points.\n polygons (torch.Tensor): It has shape (M, 8), indicating\n (x1, y1, x2, y2, x3, y3, x4, y4). M means the number of\n ground truth polygons.\n\n Returns:\n torch.Tensor: Return the result with the shape of (B, M),\n 1 indicates that the point is inside the polygon,\n 0 indicates that the point is outside the polygon.\n \"\"\"\n assert points.shape[1] == 2, \\\n 'points dimension should be 2, ' \\\n f'but got unexpected shape {points.shape[1]}'\n assert polygons.shape[1] == 8, \\\n 'polygons dimension should be 8, ' \\\n f'but got unexpected shape {polygons.shape[1]}'\n output = torch.full([points.shape[0], polygons.shape[0]],\n 0.).cuda().float()\n ext_module.points_in_polygons_forward(points.contiguous(),\n polygons.contiguous(), output)\n return output\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/points_sampler.py", "content": "from typing import List\n\nimport torch\nfrom torch import nn as nn\n\nfrom mmcv.runner import force_fp32\nfrom .furthest_point_sample import (furthest_point_sample,\n furthest_point_sample_with_dist)\n\n\ndef calc_square_dist(point_feat_a, point_feat_b, norm=True):\n \"\"\"Calculating square distance between a and b.\n\n Args:\n point_feat_a (torch.Tensor): (B, N, C) Feature vector of each point.\n point_feat_b (torch.Tensor): (B, M, C) Feature vector of each point.\n norm (bool, optional): Whether to normalize the distance.\n Default: True.\n\n Returns:\n torch.Tensor: (B, N, M) Square distance between each point pair.\n \"\"\"\n num_channel = point_feat_a.shape[-1]\n # [bs, n, 1]\n a_square = torch.sum(point_feat_a.unsqueeze(dim=2).pow(2), dim=-1)\n # [bs, 1, m]\n b_square = torch.sum(point_feat_b.unsqueeze(dim=1).pow(2), dim=-1)\n\n corr_matrix = torch.matmul(point_feat_a, point_feat_b.transpose(1, 2))\n\n dist = a_square + b_square - 2 * corr_matrix\n if norm:\n dist = torch.sqrt(dist) / num_channel\n return dist\n\n\ndef get_sampler_cls(sampler_type):\n \"\"\"Get the type and mode of points sampler.\n\n Args:\n sampler_type (str): The type of points sampler.\n The valid value are \"D-FPS\", \"F-FPS\", or \"FS\".\n\n Returns:\n class: Points sampler type.\n \"\"\"\n sampler_mappings = {\n 'D-FPS': DFPSSampler,\n 'F-FPS': FFPSSampler,\n 'FS': FSSampler,\n }\n try:\n return sampler_mappings[sampler_type]\n except KeyError:\n raise KeyError(\n f'Supported `sampler_type` are {sampler_mappings.keys()}, but got \\\n {sampler_type}')\n\n\nclass PointsSampler(nn.Module):\n \"\"\"Points sampling.\n\n Args:\n num_point (list[int]): Number of sample points.\n fps_mod_list (list[str], optional): Type of FPS method, valid mod\n ['F-FPS', 'D-FPS', 'FS'], Default: ['D-FPS'].\n F-FPS: using feature distances for FPS.\n D-FPS: using Euclidean distances of points for FPS.\n FS: using F-FPS and D-FPS simultaneously.\n fps_sample_range_list (list[int], optional):\n Range of points to apply FPS. Default: [-1].\n \"\"\"\n\n def __init__(self,\n num_point: List[int],\n fps_mod_list: List[str] = ['D-FPS'],\n fps_sample_range_list: List[int] = [-1]):\n super().__init__()\n # FPS would be applied to different fps_mod in the list,\n # so the length of the num_point should be equal to\n # fps_mod_list and fps_sample_range_list.\n assert len(num_point) == len(fps_mod_list) == len(\n fps_sample_range_list)\n self.num_point = num_point\n self.fps_sample_range_list = fps_sample_range_list\n self.samplers = nn.ModuleList()\n for fps_mod in fps_mod_list:\n self.samplers.append(get_sampler_cls(fps_mod)())\n self.fp16_enabled = False\n\n @force_fp32()\n def forward(self, points_xyz, features):\n \"\"\"\n Args:\n points_xyz (torch.Tensor): (B, N, 3) xyz coordinates of\n the points.\n features (torch.Tensor): (B, C, N) features of the points.\n\n Returns:\n torch.Tensor: (B, npoint, sample_num) Indices of sampled points.\n \"\"\"\n indices = []\n last_fps_end_index = 0\n\n for fps_sample_range, sampler, npoint in zip(\n self.fps_sample_range_list, self.samplers, self.num_point):\n assert fps_sample_range < points_xyz.shape[1]\n\n if fps_sample_range == -1:\n sample_points_xyz = points_xyz[:, last_fps_end_index:]\n if features is not None:\n sample_features = features[:, :, last_fps_end_index:]\n else:\n sample_features = None\n else:\n sample_points_xyz = \\\n points_xyz[:, last_fps_end_index:fps_sample_range]\n if features is not None:\n sample_features = features[:, :, last_fps_end_index:\n fps_sample_range]\n else:\n sample_features = None\n\n fps_idx = sampler(sample_points_xyz.contiguous(), sample_features,\n npoint)\n\n indices.append(fps_idx + last_fps_end_index)\n last_fps_end_index += fps_sample_range\n indices = torch.cat(indices, dim=1)\n\n return indices\n\n\nclass DFPSSampler(nn.Module):\n \"\"\"Using Euclidean distances of points for FPS.\"\"\"\n\n def __init__(self):\n super().__init__()\n\n def forward(self, points, features, npoint):\n \"\"\"Sampling points with D-FPS.\"\"\"\n fps_idx = furthest_point_sample(points.contiguous(), npoint)\n return fps_idx\n\n\nclass FFPSSampler(nn.Module):\n \"\"\"Using feature distances for FPS.\"\"\"\n\n def __init__(self):\n super().__init__()\n\n def forward(self, points, features, npoint):\n \"\"\"Sampling points with F-FPS.\"\"\"\n assert features is not None, \\\n 'feature input to FFPS_Sampler should not be None'\n features_for_fps = torch.cat([points, features.transpose(1, 2)], dim=2)\n features_dist = calc_square_dist(\n features_for_fps, features_for_fps, norm=False)\n fps_idx = furthest_point_sample_with_dist(features_dist, npoint)\n return fps_idx\n\n\nclass FSSampler(nn.Module):\n \"\"\"Using F-FPS and D-FPS simultaneously.\"\"\"\n\n def __init__(self):\n super().__init__()\n\n def forward(self, points, features, npoint):\n \"\"\"Sampling points with FS_Sampling.\"\"\"\n assert features is not None, \\\n 'feature input to FS_Sampler should not be None'\n ffps_sampler = FFPSSampler()\n dfps_sampler = DFPSSampler()\n fps_idx_ffps = ffps_sampler(points, features, npoint)\n fps_idx_dfps = dfps_sampler(points, features, npoint)\n fps_idx = torch.cat([fps_idx_ffps, fps_idx_dfps], dim=1)\n return fps_idx\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/psa_mask.py", "content": "# Modified from https://github.com/hszhao/semseg/blob/master/lib/psa\nfrom torch import nn\nfrom torch.autograd import Function\nfrom torch.nn.modules.utils import _pair\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext',\n ['psamask_forward', 'psamask_backward'])\n\n\nclass PSAMaskFunction(Function):\n\n @staticmethod\n def symbolic(g, input, psa_type, mask_size):\n return g.op(\n 'mmcv::MMCVPSAMask',\n input,\n psa_type_i=psa_type,\n mask_size_i=mask_size)\n\n @staticmethod\n def forward(ctx, input, psa_type, mask_size):\n ctx.psa_type = psa_type\n ctx.mask_size = _pair(mask_size)\n ctx.save_for_backward(input)\n\n h_mask, w_mask = ctx.mask_size\n batch_size, channels, h_feature, w_feature = input.size()\n assert channels == h_mask * w_mask\n output = input.new_zeros(\n (batch_size, h_feature * w_feature, h_feature, w_feature))\n\n ext_module.psamask_forward(\n input,\n output,\n psa_type=psa_type,\n num_=batch_size,\n h_feature=h_feature,\n w_feature=w_feature,\n h_mask=h_mask,\n w_mask=w_mask,\n half_h_mask=(h_mask - 1) // 2,\n half_w_mask=(w_mask - 1) // 2)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n input = ctx.saved_tensors[0]\n psa_type = ctx.psa_type\n h_mask, w_mask = ctx.mask_size\n batch_size, channels, h_feature, w_feature = input.size()\n grad_input = grad_output.new_zeros(\n (batch_size, channels, h_feature, w_feature))\n ext_module.psamask_backward(\n grad_output,\n grad_input,\n psa_type=psa_type,\n num_=batch_size,\n h_feature=h_feature,\n w_feature=w_feature,\n h_mask=h_mask,\n w_mask=w_mask,\n half_h_mask=(h_mask - 1) // 2,\n half_w_mask=(w_mask - 1) // 2)\n return grad_input, None, None, None\n\n\npsa_mask = PSAMaskFunction.apply\n\n\nclass PSAMask(nn.Module):\n\n def __init__(self, psa_type, mask_size=None):\n super(PSAMask, self).__init__()\n assert psa_type in ['collect', 'distribute']\n if psa_type == 'collect':\n psa_type_enum = 0\n else:\n psa_type_enum = 1\n self.psa_type_enum = psa_type_enum\n self.mask_size = mask_size\n self.psa_type = psa_type\n\n def forward(self, input):\n return psa_mask(input, self.psa_type_enum, self.mask_size)\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'(psa_type={self.psa_type}, '\n s += f'mask_size={self.mask_size})'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/readme.md", "content": "test\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/riroi_align_rotated.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch.nn as nn\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader, is_tuple_of\n\next_module = ext_loader.load_ext(\n '_ext', ['riroi_align_rotated_forward', 'riroi_align_rotated_backward'])\n\n\nclass RiRoIAlignRotatedFunction(Function):\n\n @staticmethod\n def forward(ctx,\n features,\n rois,\n out_size,\n spatial_scale,\n num_samples=0,\n num_orientations=8,\n clockwise=False):\n if isinstance(out_size, int):\n out_h = out_size\n out_w = out_size\n elif is_tuple_of(out_size, int):\n assert len(out_size) == 2\n out_h, out_w = out_size\n else:\n raise TypeError(\n f'\"out_size\" should be an integer or tuple of integers,'\n f' but got {out_size}')\n ctx.spatial_scale = spatial_scale\n ctx.num_samples = num_samples\n ctx.num_orientations = num_orientations\n ctx.clockwise = clockwise\n ctx.save_for_backward(rois)\n ctx.feature_size = features.size()\n\n batch_size, num_channels, _, _ = features.size()\n num_rois = rois.size(0)\n\n output = features.new_zeros(num_rois, num_channels, out_h, out_w)\n\n ext_module.riroi_align_rotated_forward(\n features,\n rois,\n output,\n pooled_height=out_h,\n pooled_width=out_w,\n spatial_scale=spatial_scale,\n num_samples=num_samples,\n num_orientations=num_orientations,\n clockwise=clockwise)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n feature_size = ctx.feature_size\n spatial_scale = ctx.spatial_scale\n num_orientations = ctx.num_orientations\n clockwise = ctx.clockwise\n num_samples = ctx.num_samples\n rois = ctx.saved_tensors[0]\n assert feature_size is not None\n batch_size, num_channels, feature_h, feature_w = feature_size\n\n out_w = grad_output.size(3)\n out_h = grad_output.size(2)\n\n grad_input = grad_rois = None\n\n if ctx.needs_input_grad[0]:\n grad_input = rois.new_zeros(batch_size, num_channels, feature_h,\n feature_w)\n ext_module.riroi_align_rotated_backward(\n grad_output.contiguous(),\n rois,\n grad_input,\n pooled_height=out_h,\n pooled_width=out_w,\n spatial_scale=spatial_scale,\n num_samples=num_samples,\n num_orientations=num_orientations,\n clockwise=clockwise)\n\n return grad_input, grad_rois, None, None, None, None, None\n\n\nriroi_align_rotated = RiRoIAlignRotatedFunction.apply\n\n\nclass RiRoIAlignRotated(nn.Module):\n \"\"\"Rotation-invariant RoI align pooling layer for rotated proposals.\n\n It accepts a feature map of shape (N, C, H, W) and rois with shape\n (n, 6) with each roi decoded as (batch_index, center_x, center_y,\n w, h, angle). The angle is in radian.\n\n The details are described in the paper `ReDet: A Rotation-equivariant\n Detector for Aerial Object Detection `_.\n\n Args:\n out_size (tuple): fixed dimensional RoI output with shape (h, w).\n spatial_scale (float): scale the input boxes by this number\n num_samples (int): number of inputs samples to take for each\n output sample. 0 to take samples densely for current models.\n num_orientations (int): number of oriented channels.\n clockwise (bool): If True, the angle in each proposal follows a\n clockwise fashion in image space, otherwise, the angle is\n counterclockwise. Default: False.\n \"\"\"\n\n def __init__(self,\n out_size,\n spatial_scale,\n num_samples=0,\n num_orientations=8,\n clockwise=False):\n super(RiRoIAlignRotated, self).__init__()\n\n self.out_size = out_size\n self.spatial_scale = float(spatial_scale)\n self.num_samples = int(num_samples)\n self.num_orientations = int(num_orientations)\n self.clockwise = clockwise\n\n def forward(self, features, rois):\n return RiRoIAlignRotatedFunction.apply(features, rois, self.out_size,\n self.spatial_scale,\n self.num_samples,\n self.num_orientations,\n self.clockwise)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/roi_align.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\nfrom torch.nn.modules.utils import _pair\n\nfrom ..utils import deprecated_api_warning, ext_loader\n\next_module = ext_loader.load_ext('_ext',\n ['roi_align_forward', 'roi_align_backward'])\n\n\nclass RoIAlignFunction(Function):\n\n @staticmethod\n def symbolic(g, input, rois, output_size, spatial_scale, sampling_ratio,\n pool_mode, aligned):\n from ..onnx import is_custom_op_loaded\n has_custom_op = is_custom_op_loaded()\n if has_custom_op:\n return g.op(\n 'mmcv::MMCVRoiAlign',\n input,\n rois,\n output_height_i=output_size[0],\n output_width_i=output_size[1],\n spatial_scale_f=spatial_scale,\n sampling_ratio_i=sampling_ratio,\n mode_s=pool_mode,\n aligned_i=aligned)\n else:\n from torch.onnx.symbolic_opset9 import sub, squeeze\n from torch.onnx.symbolic_helper import _slice_helper\n from torch.onnx import TensorProtoDataType\n # batch_indices = rois[:, 0].long()\n batch_indices = _slice_helper(\n g, rois, axes=[1], starts=[0], ends=[1])\n batch_indices = squeeze(g, batch_indices, 1)\n batch_indices = g.op(\n 'Cast', batch_indices, to_i=TensorProtoDataType.INT64)\n # rois = rois[:, 1:]\n rois = _slice_helper(g, rois, axes=[1], starts=[1], ends=[5])\n if aligned:\n # rois -= 0.5/spatial_scale\n aligned_offset = g.op(\n 'Constant',\n value_t=torch.tensor([0.5 / spatial_scale],\n dtype=torch.float32))\n rois = sub(g, rois, aligned_offset)\n # roi align\n return g.op(\n 'RoiAlign',\n input,\n rois,\n batch_indices,\n output_height_i=output_size[0],\n output_width_i=output_size[1],\n spatial_scale_f=spatial_scale,\n sampling_ratio_i=max(0, sampling_ratio),\n mode_s=pool_mode)\n\n @staticmethod\n def forward(ctx,\n input,\n rois,\n output_size,\n spatial_scale=1.0,\n sampling_ratio=0,\n pool_mode='avg',\n aligned=True):\n ctx.output_size = _pair(output_size)\n ctx.spatial_scale = spatial_scale\n ctx.sampling_ratio = sampling_ratio\n assert pool_mode in ('max', 'avg')\n ctx.pool_mode = 0 if pool_mode == 'max' else 1\n ctx.aligned = aligned\n ctx.input_shape = input.size()\n\n assert rois.size(1) == 5, 'RoI must be (idx, x1, y1, x2, y2)!'\n\n output_shape = (rois.size(0), input.size(1), ctx.output_size[0],\n ctx.output_size[1])\n output = input.new_zeros(output_shape)\n if ctx.pool_mode == 0:\n argmax_y = input.new_zeros(output_shape)\n argmax_x = input.new_zeros(output_shape)\n else:\n argmax_y = input.new_zeros(0)\n argmax_x = input.new_zeros(0)\n\n ext_module.roi_align_forward(\n input,\n rois,\n output,\n argmax_y,\n argmax_x,\n aligned_height=ctx.output_size[0],\n aligned_width=ctx.output_size[1],\n spatial_scale=ctx.spatial_scale,\n sampling_ratio=ctx.sampling_ratio,\n pool_mode=ctx.pool_mode,\n aligned=ctx.aligned)\n\n ctx.save_for_backward(rois, argmax_y, argmax_x)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n rois, argmax_y, argmax_x = ctx.saved_tensors\n grad_input = grad_output.new_zeros(ctx.input_shape)\n # complex head architecture may cause grad_output uncontiguous.\n grad_output = grad_output.contiguous()\n ext_module.roi_align_backward(\n grad_output,\n rois,\n argmax_y,\n argmax_x,\n grad_input,\n aligned_height=ctx.output_size[0],\n aligned_width=ctx.output_size[1],\n spatial_scale=ctx.spatial_scale,\n sampling_ratio=ctx.sampling_ratio,\n pool_mode=ctx.pool_mode,\n aligned=ctx.aligned)\n return grad_input, None, None, None, None, None, None\n\n\nroi_align = RoIAlignFunction.apply\n\n\nclass RoIAlign(nn.Module):\n \"\"\"RoI align pooling layer.\n\n Args:\n output_size (tuple): h, w\n spatial_scale (float): scale the input boxes by this number\n sampling_ratio (int): number of inputs samples to take for each\n output sample. 0 to take samples densely for current models.\n pool_mode (str, 'avg' or 'max'): pooling mode in each bin.\n aligned (bool): if False, use the legacy implementation in\n MMDetection. If True, align the results more perfectly.\n use_torchvision (bool): whether to use roi_align from torchvision.\n\n Note:\n The implementation of RoIAlign when aligned=True is modified from\n https://github.com/facebookresearch/detectron2/\n\n The meaning of aligned=True:\n\n Given a continuous coordinate c, its two neighboring pixel\n indices (in our pixel model) are computed by floor(c - 0.5) and\n ceil(c - 0.5). For example, c=1.3 has pixel neighbors with discrete\n indices [0] and [1] (which are sampled from the underlying signal\n at continuous coordinates 0.5 and 1.5). But the original roi_align\n (aligned=False) does not subtract the 0.5 when computing\n neighboring pixel indices and therefore it uses pixels with a\n slightly incorrect alignment (relative to our pixel model) when\n performing bilinear interpolation.\n\n With `aligned=True`,\n we first appropriately scale the ROI and then shift it by -0.5\n prior to calling roi_align. This produces the correct neighbors;\n\n The difference does not make a difference to the model's\n performance if ROIAlign is used together with conv layers.\n \"\"\"\n\n @deprecated_api_warning(\n {\n 'out_size': 'output_size',\n 'sample_num': 'sampling_ratio'\n },\n cls_name='RoIAlign')\n def __init__(self,\n output_size,\n spatial_scale=1.0,\n sampling_ratio=0,\n pool_mode='avg',\n aligned=True,\n use_torchvision=False):\n super(RoIAlign, self).__init__()\n\n self.output_size = _pair(output_size)\n self.spatial_scale = float(spatial_scale)\n self.sampling_ratio = int(sampling_ratio)\n self.pool_mode = pool_mode\n self.aligned = aligned\n self.use_torchvision = use_torchvision\n\n def forward(self, input, rois):\n \"\"\"\n Args:\n input: NCHW images\n rois: Bx5 boxes. First column is the index into N.\\\n The other 4 columns are xyxy.\n \"\"\"\n if self.use_torchvision:\n from torchvision.ops import roi_align as tv_roi_align\n if 'aligned' in tv_roi_align.__code__.co_varnames:\n return tv_roi_align(input, rois, self.output_size,\n self.spatial_scale, self.sampling_ratio,\n self.aligned)\n else:\n if self.aligned:\n rois -= rois.new_tensor([0.] +\n [0.5 / self.spatial_scale] * 4)\n return tv_roi_align(input, rois, self.output_size,\n self.spatial_scale, self.sampling_ratio)\n else:\n return roi_align(input, rois, self.output_size, self.spatial_scale,\n self.sampling_ratio, self.pool_mode, self.aligned)\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'(output_size={self.output_size}, '\n s += f'spatial_scale={self.spatial_scale}, '\n s += f'sampling_ratio={self.sampling_ratio}, '\n s += f'pool_mode={self.pool_mode}, '\n s += f'aligned={self.aligned}, '\n s += f'use_torchvision={self.use_torchvision})'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/roi_align_rotated.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch.nn as nn\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['roi_align_rotated_forward', 'roi_align_rotated_backward'])\n\n\nclass RoIAlignRotatedFunction(Function):\n\n @staticmethod\n def symbolic(g, features, rois, out_size, spatial_scale, sample_num,\n aligned, clockwise):\n if isinstance(out_size, int):\n out_h = out_size\n out_w = out_size\n elif isinstance(out_size, tuple):\n assert len(out_size) == 2\n assert isinstance(out_size[0], int)\n assert isinstance(out_size[1], int)\n out_h, out_w = out_size\n else:\n raise TypeError(\n '\"out_size\" must be an integer or tuple of integers')\n return g.op(\n 'mmcv::MMCVRoIAlignRotated',\n features,\n rois,\n output_height_i=out_h,\n output_width_i=out_h,\n spatial_scale_f=spatial_scale,\n sampling_ratio_i=sample_num,\n aligned_i=aligned,\n clockwise_i=clockwise)\n\n @staticmethod\n def forward(ctx,\n features,\n rois,\n out_size,\n spatial_scale,\n sample_num=0,\n aligned=True,\n clockwise=False):\n if isinstance(out_size, int):\n out_h = out_size\n out_w = out_size\n elif isinstance(out_size, tuple):\n assert len(out_size) == 2\n assert isinstance(out_size[0], int)\n assert isinstance(out_size[1], int)\n out_h, out_w = out_size\n else:\n raise TypeError(\n '\"out_size\" must be an integer or tuple of integers')\n ctx.spatial_scale = spatial_scale\n ctx.sample_num = sample_num\n ctx.aligned = aligned\n ctx.clockwise = clockwise\n ctx.save_for_backward(rois)\n ctx.feature_size = features.size()\n\n batch_size, num_channels, data_height, data_width = features.size()\n num_rois = rois.size(0)\n\n output = features.new_zeros(num_rois, num_channels, out_h, out_w)\n ext_module.roi_align_rotated_forward(\n features,\n rois,\n output,\n pooled_height=out_h,\n pooled_width=out_w,\n spatial_scale=spatial_scale,\n sample_num=sample_num,\n aligned=aligned,\n clockwise=clockwise)\n return output\n\n @staticmethod\n def backward(ctx, grad_output):\n feature_size = ctx.feature_size\n spatial_scale = ctx.spatial_scale\n aligned = ctx.aligned\n clockwise = ctx.clockwise\n sample_num = ctx.sample_num\n rois = ctx.saved_tensors[0]\n assert feature_size is not None\n batch_size, num_channels, data_height, data_width = feature_size\n\n out_w = grad_output.size(3)\n out_h = grad_output.size(2)\n\n grad_input = grad_rois = None\n\n if ctx.needs_input_grad[0]:\n grad_input = rois.new_zeros(batch_size, num_channels, data_height,\n data_width)\n ext_module.roi_align_rotated_backward(\n grad_output.contiguous(),\n rois,\n grad_input,\n pooled_height=out_h,\n pooled_width=out_w,\n spatial_scale=spatial_scale,\n sample_num=sample_num,\n aligned=aligned,\n clockwise=clockwise)\n return grad_input, grad_rois, None, None, None, None, None\n\n\nroi_align_rotated = RoIAlignRotatedFunction.apply\n\n\nclass RoIAlignRotated(nn.Module):\n \"\"\"RoI align pooling layer for rotated proposals.\n\n It accepts a feature map of shape (N, C, H, W) and rois with shape\n (n, 6) with each roi decoded as (batch_index, center_x, center_y,\n w, h, angle). The angle is in radian.\n\n Args:\n out_size (tuple): h, w\n spatial_scale (float): scale the input boxes by this number\n sample_num (int): number of inputs samples to take for each\n output sample. 0 to take samples densely for current models.\n aligned (bool): if False, use the legacy implementation in\n MMDetection. If True, align the results more perfectly.\n Default: True.\n clockwise (bool): If True, the angle in each proposal follows a\n clockwise fashion in image space, otherwise, the angle is\n counterclockwise. Default: False.\n\n Note:\n The implementation of RoIAlign when aligned=True is modified from\n https://github.com/facebookresearch/detectron2/\n\n The meaning of aligned=True:\n\n Given a continuous coordinate c, its two neighboring pixel\n indices (in our pixel model) are computed by floor(c - 0.5) and\n ceil(c - 0.5). For example, c=1.3 has pixel neighbors with discrete\n indices [0] and [1] (which are sampled from the underlying signal\n at continuous coordinates 0.5 and 1.5). But the original roi_align\n (aligned=False) does not subtract the 0.5 when computing\n neighboring pixel indices and therefore it uses pixels with a\n slightly incorrect alignment (relative to our pixel model) when\n performing bilinear interpolation.\n\n With `aligned=True`,\n we first appropriately scale the ROI and then shift it by -0.5\n prior to calling roi_align. This produces the correct neighbors;\n\n The difference does not make a difference to the model's\n performance if ROIAlign is used together with conv layers.\n \"\"\"\n\n def __init__(self,\n out_size,\n spatial_scale,\n sample_num=0,\n aligned=True,\n clockwise=False):\n super(RoIAlignRotated, self).__init__()\n\n self.out_size = out_size\n self.spatial_scale = float(spatial_scale)\n self.sample_num = int(sample_num)\n self.aligned = aligned\n self.clockwise = clockwise\n\n def forward(self, features, rois):\n return RoIAlignRotatedFunction.apply(features, rois, self.out_size,\n self.spatial_scale,\n self.sample_num, self.aligned,\n self.clockwise)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/roi_pool.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\nfrom torch.nn.modules.utils import _pair\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext',\n ['roi_pool_forward', 'roi_pool_backward'])\n\n\nclass RoIPoolFunction(Function):\n\n @staticmethod\n def symbolic(g, input, rois, output_size, spatial_scale):\n return g.op(\n 'MaxRoiPool',\n input,\n rois,\n pooled_shape_i=output_size,\n spatial_scale_f=spatial_scale)\n\n @staticmethod\n def forward(ctx, input, rois, output_size, spatial_scale=1.0):\n ctx.output_size = _pair(output_size)\n ctx.spatial_scale = spatial_scale\n ctx.input_shape = input.size()\n\n assert rois.size(1) == 5, 'RoI must be (idx, x1, y1, x2, y2)!'\n\n output_shape = (rois.size(0), input.size(1), ctx.output_size[0],\n ctx.output_size[1])\n output = input.new_zeros(output_shape)\n argmax = input.new_zeros(output_shape, dtype=torch.int)\n\n ext_module.roi_pool_forward(\n input,\n rois,\n output,\n argmax,\n pooled_height=ctx.output_size[0],\n pooled_width=ctx.output_size[1],\n spatial_scale=ctx.spatial_scale)\n\n ctx.save_for_backward(rois, argmax)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n rois, argmax = ctx.saved_tensors\n grad_input = grad_output.new_zeros(ctx.input_shape)\n\n ext_module.roi_pool_backward(\n grad_output,\n rois,\n argmax,\n grad_input,\n pooled_height=ctx.output_size[0],\n pooled_width=ctx.output_size[1],\n spatial_scale=ctx.spatial_scale)\n\n return grad_input, None, None, None\n\n\nroi_pool = RoIPoolFunction.apply\n\n\nclass RoIPool(nn.Module):\n\n def __init__(self, output_size, spatial_scale=1.0):\n super(RoIPool, self).__init__()\n\n self.output_size = _pair(output_size)\n self.spatial_scale = float(spatial_scale)\n\n def forward(self, input, rois):\n return roi_pool(input, rois, self.output_size, self.spatial_scale)\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'(output_size={self.output_size}, '\n s += f'spatial_scale={self.spatial_scale})'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/roiaware_pool3d.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch import nn as nn\nfrom torch.autograd import Function\n\nimport mmcv\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['roiaware_pool3d_forward', 'roiaware_pool3d_backward'])\n\n\nclass RoIAwarePool3d(nn.Module):\n \"\"\"Encode the geometry-specific features of each 3D proposal.\n\n Please refer to `PartA2 `_ for more\n details.\n\n Args:\n out_size (int or tuple): The size of output features. n or\n [n1, n2, n3].\n max_pts_per_voxel (int, optional): The maximum number of points per\n voxel. Default: 128.\n mode (str, optional): Pooling method of RoIAware, 'max' or 'avg'.\n Default: 'max'.\n \"\"\"\n\n def __init__(self, out_size, max_pts_per_voxel=128, mode='max'):\n super().__init__()\n\n self.out_size = out_size\n self.max_pts_per_voxel = max_pts_per_voxel\n assert mode in ['max', 'avg']\n pool_mapping = {'max': 0, 'avg': 1}\n self.mode = pool_mapping[mode]\n\n def forward(self, rois, pts, pts_feature):\n \"\"\"\n Args:\n rois (torch.Tensor): [N, 7], in LiDAR coordinate,\n (x, y, z) is the bottom center of rois.\n pts (torch.Tensor): [npoints, 3], coordinates of input points.\n pts_feature (torch.Tensor): [npoints, C], features of input points.\n\n Returns:\n torch.Tensor: Pooled features whose shape is\n [N, out_x, out_y, out_z, C].\n \"\"\"\n\n return RoIAwarePool3dFunction.apply(rois, pts, pts_feature,\n self.out_size,\n self.max_pts_per_voxel, self.mode)\n\n\nclass RoIAwarePool3dFunction(Function):\n\n @staticmethod\n def forward(ctx, rois, pts, pts_feature, out_size, max_pts_per_voxel,\n mode):\n \"\"\"\n Args:\n rois (torch.Tensor): [N, 7], in LiDAR coordinate,\n (x, y, z) is the bottom center of rois.\n pts (torch.Tensor): [npoints, 3], coordinates of input points.\n pts_feature (torch.Tensor): [npoints, C], features of input points.\n out_size (int or tuple): The size of output features. n or\n [n1, n2, n3].\n max_pts_per_voxel (int): The maximum number of points per voxel.\n Default: 128.\n mode (int): Pooling method of RoIAware, 0 (max pool) or 1 (average\n pool).\n\n Returns:\n torch.Tensor: Pooled features whose shape is\n [N, out_x, out_y, out_z, C].\n \"\"\"\n\n if isinstance(out_size, int):\n out_x = out_y = out_z = out_size\n else:\n assert len(out_size) == 3\n assert mmcv.is_tuple_of(out_size, int)\n out_x, out_y, out_z = out_size\n\n num_rois = rois.shape[0]\n num_channels = pts_feature.shape[-1]\n num_pts = pts.shape[0]\n\n pooled_features = pts_feature.new_zeros(\n (num_rois, out_x, out_y, out_z, num_channels))\n argmax = pts_feature.new_zeros(\n (num_rois, out_x, out_y, out_z, num_channels), dtype=torch.int)\n pts_idx_of_voxels = pts_feature.new_zeros(\n (num_rois, out_x, out_y, out_z, max_pts_per_voxel),\n dtype=torch.int)\n\n ext_module.roiaware_pool3d_forward(\n rois,\n pts,\n pts_feature,\n argmax,\n pts_idx_of_voxels,\n pooled_features,\n pool_method=mode)\n\n ctx.roiaware_pool3d_for_backward = (pts_idx_of_voxels, argmax, mode,\n num_pts, num_channels)\n return pooled_features\n\n @staticmethod\n def backward(ctx, grad_out):\n ret = ctx.roiaware_pool3d_for_backward\n pts_idx_of_voxels, argmax, mode, num_pts, num_channels = ret\n\n grad_in = grad_out.new_zeros((num_pts, num_channels))\n ext_module.roiaware_pool3d_backward(\n pts_idx_of_voxels,\n argmax,\n grad_out.contiguous(),\n grad_in,\n pool_method=mode)\n\n return None, None, grad_in, None, None, None\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/roipoint_pool3d.py", "content": "from torch import nn as nn\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['roipoint_pool3d_forward'])\n\n\nclass RoIPointPool3d(nn.Module):\n \"\"\"Encode the geometry-specific features of each 3D proposal.\n\n Please refer to `Paper of PartA2 `_\n for more details.\n\n Args:\n num_sampled_points (int, optional): Number of samples in each roi.\n Default: 512.\n \"\"\"\n\n def __init__(self, num_sampled_points=512):\n super().__init__()\n self.num_sampled_points = num_sampled_points\n\n def forward(self, points, point_features, boxes3d):\n \"\"\"\n Args:\n points (torch.Tensor): Input points whose shape is (B, N, C).\n point_features (torch.Tensor): Features of input points whose shape\n is (B, N, C).\n boxes3d (B, M, 7), Input bounding boxes whose shape is (B, M, 7).\n\n Returns:\n tuple[torch.Tensor]: A tuple contains two elements. The first one\n is the pooled features whose shape is (B, M, 512, 3 + C). The\n second is an empty flag whose shape is (B, M).\n \"\"\"\n return RoIPointPool3dFunction.apply(points, point_features, boxes3d,\n self.num_sampled_points)\n\n\nclass RoIPointPool3dFunction(Function):\n\n @staticmethod\n def forward(ctx, points, point_features, boxes3d, num_sampled_points=512):\n \"\"\"\n Args:\n points (torch.Tensor): Input points whose shape is (B, N, C).\n point_features (torch.Tensor): Features of input points whose shape\n is (B, N, C).\n boxes3d (B, M, 7), Input bounding boxes whose shape is (B, M, 7).\n num_sampled_points (int, optional): The num of sampled points.\n Default: 512.\n\n Returns:\n tuple[torch.Tensor]: A tuple contains two elements. The first one\n is the pooled features whose shape is (B, M, 512, 3 + C). The\n second is an empty flag whose shape is (B, M).\n \"\"\"\n assert len(points.shape) == 3 and points.shape[2] == 3\n batch_size, boxes_num, feature_len = points.shape[0], boxes3d.shape[\n 1], point_features.shape[2]\n pooled_boxes3d = boxes3d.view(batch_size, -1, 7)\n pooled_features = point_features.new_zeros(\n (batch_size, boxes_num, num_sampled_points, 3 + feature_len))\n pooled_empty_flag = point_features.new_zeros(\n (batch_size, boxes_num)).int()\n\n ext_module.roipoint_pool3d_forward(points.contiguous(),\n pooled_boxes3d.contiguous(),\n point_features.contiguous(),\n pooled_features, pooled_empty_flag)\n\n return pooled_features, pooled_empty_flag\n\n @staticmethod\n def backward(ctx, grad_out):\n raise NotImplementedError\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/rotated_feature_align.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext',\n ['rotated_feature_align_forward', 'rotated_feature_align_backward'])\n\n\nclass RotatedFeatureAlignFunction(Function):\n \"\"\"Using the feature interpolation to obtain the position information\n correspond to the refined rotate anchors and reconstruct the feature maps\n in pixel-wise manner to achieve feature alignment.\n\n The details are described in the paper\n `R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating\n Object `_.\n \"\"\"\n\n @staticmethod\n def forward(ctx, features, best_rbboxes, spatial_scale, points):\n \"\"\"\n Args:\n features (torch.Tensor): Input features with shape [N,C,H,W].\n best_rbboxes (torch.Tensor): Refined rotate anchors with\n shape [N,H,W,5]. Coordinate format (cx,cx,h,w,a).\n spatial_scale (float): The scale of feature map size and\n input image size.\n points (int, optional): The number of sample points.\n Only 1 and 5 are supported. Defaults to 1.\n\n Returns:\n torch.Tensor: Refined features with shape [N,C,H,W].\n \"\"\"\n ctx.spatial_scale = spatial_scale\n ctx.points = points\n ctx.save_for_backward(best_rbboxes)\n assert points in [1, 5]\n output = torch.zeros_like(features)\n ext_module.rotated_feature_align_forward(\n features,\n best_rbboxes,\n output,\n spatial_scale=spatial_scale,\n points=points)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(ctx, grad_output):\n \"\"\"\n Args:\n grad_output (torch.Tensor): The gradiant of output features\n with shape [N,C,H,W].\n\n Returns:\n torch.Tensor: The gradiant of input features with shape [N,C,H,W].\n \"\"\"\n best_rbboxes = ctx.saved_tensors[0]\n points = ctx.points\n spatial_scale = ctx.spatial_scale\n grad_input = None\n if ctx.needs_input_grad[0]:\n grad_input = torch.zeros_like(grad_output)\n ext_module.rotated_feature_align_backward(\n grad_output.contiguous(),\n best_rbboxes,\n grad_input,\n spatial_scale=spatial_scale,\n points=points)\n return grad_input, None, None, None\n\n\ndef rotated_feature_align(features,\n best_rbboxes,\n spatial_scale=1 / 8,\n points=1):\n return RotatedFeatureAlignFunction.apply(features, best_rbboxes,\n spatial_scale, points)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/saconv.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom mmcv.cnn import CONV_LAYERS, ConvAWS2d, constant_init\nfrom mmcv.ops.deform_conv import deform_conv2d\nfrom mmcv.utils import TORCH_VERSION, digit_version\n\n\n@CONV_LAYERS.register_module(name='SAC')\nclass SAConv2d(ConvAWS2d):\n \"\"\"SAC (Switchable Atrous Convolution)\n\n This is an implementation of `DetectoRS: Detecting Objects with Recursive\n Feature Pyramid and Switchable Atrous Convolution\n `_.\n\n Args:\n in_channels (int): Number of channels in the input image\n out_channels (int): Number of channels produced by the convolution\n kernel_size (int or tuple): Size of the convolving kernel\n stride (int or tuple, optional): Stride of the convolution. Default: 1\n padding (int or tuple, optional): Zero-padding added to both sides of\n the input. Default: 0\n padding_mode (string, optional): ``'zeros'``, ``'reflect'``,\n ``'replicate'`` or ``'circular'``. Default: ``'zeros'``\n dilation (int or tuple, optional): Spacing between kernel elements.\n Default: 1\n groups (int, optional): Number of blocked connections from input\n channels to output channels. Default: 1\n bias (bool, optional): If ``True``, adds a learnable bias to the\n output. Default: ``True``\n use_deform: If ``True``, replace convolution with deformable\n convolution. Default: ``False``.\n \"\"\"\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n bias=True,\n use_deform=False):\n super().__init__(\n in_channels,\n out_channels,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n groups=groups,\n bias=bias)\n self.use_deform = use_deform\n self.switch = nn.Conv2d(\n self.in_channels, 1, kernel_size=1, stride=stride, bias=True)\n self.weight_diff = nn.Parameter(torch.Tensor(self.weight.size()))\n self.pre_context = nn.Conv2d(\n self.in_channels, self.in_channels, kernel_size=1, bias=True)\n self.post_context = nn.Conv2d(\n self.out_channels, self.out_channels, kernel_size=1, bias=True)\n if self.use_deform:\n self.offset_s = nn.Conv2d(\n self.in_channels,\n 18,\n kernel_size=3,\n padding=1,\n stride=stride,\n bias=True)\n self.offset_l = nn.Conv2d(\n self.in_channels,\n 18,\n kernel_size=3,\n padding=1,\n stride=stride,\n bias=True)\n self.init_weights()\n\n def init_weights(self):\n constant_init(self.switch, 0, bias=1)\n self.weight_diff.data.zero_()\n constant_init(self.pre_context, 0)\n constant_init(self.post_context, 0)\n if self.use_deform:\n constant_init(self.offset_s, 0)\n constant_init(self.offset_l, 0)\n\n def forward(self, x):\n # pre-context\n avg_x = F.adaptive_avg_pool2d(x, output_size=1)\n avg_x = self.pre_context(avg_x)\n avg_x = avg_x.expand_as(x)\n x = x + avg_x\n # switch\n avg_x = F.pad(x, pad=(2, 2, 2, 2), mode='reflect')\n avg_x = F.avg_pool2d(avg_x, kernel_size=5, stride=1, padding=0)\n switch = self.switch(avg_x)\n # sac\n weight = self._get_weight(self.weight)\n zero_bias = torch.zeros(\n self.out_channels, device=weight.device, dtype=weight.dtype)\n\n if self.use_deform:\n offset = self.offset_s(avg_x)\n out_s = deform_conv2d(x, offset, weight, self.stride, self.padding,\n self.dilation, self.groups, 1)\n else:\n if (TORCH_VERSION == 'parrots'\n or digit_version(TORCH_VERSION) < digit_version('1.5.0')):\n out_s = super().conv2d_forward(x, weight)\n elif digit_version(TORCH_VERSION) >= digit_version('1.8.0'):\n # bias is a required argument of _conv_forward in torch 1.8.0\n out_s = super()._conv_forward(x, weight, zero_bias)\n else:\n out_s = super()._conv_forward(x, weight)\n ori_p = self.padding\n ori_d = self.dilation\n self.padding = tuple(3 * p for p in self.padding)\n self.dilation = tuple(3 * d for d in self.dilation)\n weight = weight + self.weight_diff\n if self.use_deform:\n offset = self.offset_l(avg_x)\n out_l = deform_conv2d(x, offset, weight, self.stride, self.padding,\n self.dilation, self.groups, 1)\n else:\n if (TORCH_VERSION == 'parrots'\n or digit_version(TORCH_VERSION) < digit_version('1.5.0')):\n out_l = super().conv2d_forward(x, weight)\n elif digit_version(TORCH_VERSION) >= digit_version('1.8.0'):\n # bias is a required argument of _conv_forward in torch 1.8.0\n out_l = super()._conv_forward(x, weight, zero_bias)\n else:\n out_l = super()._conv_forward(x, weight)\n\n out = switch * out_s + (1 - switch) * out_l\n self.padding = ori_p\n self.dilation = ori_d\n # post-context\n avg_x = F.adaptive_avg_pool2d(out, output_size=1)\n avg_x = self.post_context(avg_x)\n avg_x = avg_x.expand_as(out)\n out = out + avg_x\n return out\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/scatter_points.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext',\n ['dynamic_point_to_voxel_forward', 'dynamic_point_to_voxel_backward'])\n\n\nclass _DynamicScatter(Function):\n\n @staticmethod\n def forward(ctx, feats, coors, reduce_type='max'):\n \"\"\"convert kitti points(N, >=3) to voxels.\n\n Args:\n feats (torch.Tensor): [N, C]. Points features to be reduced\n into voxels.\n coors (torch.Tensor): [N, ndim]. Corresponding voxel coordinates\n (specifically multi-dim voxel index) of each points.\n reduce_type (str, optional): Reduce op. support 'max', 'sum' and\n 'mean'. Default: 'max'.\n\n Returns:\n tuple[torch.Tensor]: A tuple contains two elements. The first one\n is the voxel features with shape [M, C] which are respectively\n reduced from input features that share the same voxel coordinates.\n The second is voxel coordinates with shape [M, ndim].\n \"\"\"\n results = ext_module.dynamic_point_to_voxel_forward(\n feats, coors, reduce_type)\n (voxel_feats, voxel_coors, point2voxel_map,\n voxel_points_count) = results\n ctx.reduce_type = reduce_type\n ctx.save_for_backward(feats, voxel_feats, point2voxel_map,\n voxel_points_count)\n ctx.mark_non_differentiable(voxel_coors)\n return voxel_feats, voxel_coors\n\n @staticmethod\n def backward(ctx, grad_voxel_feats, grad_voxel_coors=None):\n (feats, voxel_feats, point2voxel_map,\n voxel_points_count) = ctx.saved_tensors\n grad_feats = torch.zeros_like(feats)\n # TODO: whether to use index put or use cuda_backward\n # To use index put, need point to voxel index\n ext_module.dynamic_point_to_voxel_backward(\n grad_feats, grad_voxel_feats.contiguous(), feats, voxel_feats,\n point2voxel_map, voxel_points_count, ctx.reduce_type)\n return grad_feats, None, None\n\n\ndynamic_scatter = _DynamicScatter.apply\n\n\nclass DynamicScatter(nn.Module):\n \"\"\"Scatters points into voxels, used in the voxel encoder with dynamic\n voxelization.\n\n Note:\n The CPU and GPU implementation get the same output, but have numerical\n difference after summation and division (e.g., 5e-7).\n\n Args:\n voxel_size (list): list [x, y, z] size of three dimension.\n point_cloud_range (list): The coordinate range of points, [x_min,\n y_min, z_min, x_max, y_max, z_max].\n average_points (bool): whether to use avg pooling to scatter points\n into voxel.\n \"\"\"\n\n def __init__(self, voxel_size, point_cloud_range, average_points: bool):\n super().__init__()\n\n self.voxel_size = voxel_size\n self.point_cloud_range = point_cloud_range\n self.average_points = average_points\n\n def forward_single(self, points, coors):\n \"\"\"Scatters points into voxels.\n\n Args:\n points (torch.Tensor): Points to be reduced into voxels.\n coors (torch.Tensor): Corresponding voxel coordinates (specifically\n multi-dim voxel index) of each points.\n\n Returns:\n tuple[torch.Tensor]: A tuple contains two elements. The first one\n is the voxel features with shape [M, C] which are respectively\n reduced from input features that share the same voxel coordinates.\n The second is voxel coordinates with shape [M, ndim].\n \"\"\"\n reduce = 'mean' if self.average_points else 'max'\n return dynamic_scatter(points.contiguous(), coors.contiguous(), reduce)\n\n def forward(self, points, coors):\n \"\"\"Scatters points/features into voxels.\n\n Args:\n points (torch.Tensor): Points to be reduced into voxels.\n coors (torch.Tensor): Corresponding voxel coordinates (specifically\n multi-dim voxel index) of each points.\n\n Returns:\n tuple[torch.Tensor]: A tuple contains two elements. The first one\n is the voxel features with shape [M, C] which are respectively\n reduced from input features that share the same voxel coordinates.\n The second is voxel coordinates with shape [M, ndim].\n \"\"\"\n if coors.size(-1) == 3:\n return self.forward_single(points, coors)\n else:\n batch_size = coors[-1, 0] + 1\n voxels, voxel_coors = [], []\n for i in range(batch_size):\n inds = torch.where(coors[:, 0] == i)\n voxel, voxel_coor = self.forward_single(\n points[inds], coors[inds][:, 1:])\n coor_pad = F.pad(voxel_coor, (1, 0), mode='constant', value=i)\n voxel_coors.append(coor_pad)\n voxels.append(voxel)\n features = torch.cat(voxels, dim=0)\n feature_coors = torch.cat(voxel_coors, dim=0)\n\n return features, feature_coors\n\n def __repr__(self):\n s = self.__class__.__name__ + '('\n s += 'voxel_size=' + str(self.voxel_size)\n s += ', point_cloud_range=' + str(self.point_cloud_range)\n s += ', average_points=' + str(self.average_points)\n s += ')'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/sync_bn.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.distributed as dist\nimport torch.nn.functional as F\nfrom torch.autograd import Function\nfrom torch.autograd.function import once_differentiable\nfrom torch.nn.modules.module import Module\nfrom torch.nn.parameter import Parameter\n\nfrom mmcv.cnn import NORM_LAYERS\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', [\n 'sync_bn_forward_mean', 'sync_bn_forward_var', 'sync_bn_forward_output',\n 'sync_bn_backward_param', 'sync_bn_backward_data'\n])\n\n\nclass SyncBatchNormFunction(Function):\n\n @staticmethod\n def symbolic(g, input, running_mean, running_var, weight, bias, momentum,\n eps, group, group_size, stats_mode):\n return g.op(\n 'mmcv::MMCVSyncBatchNorm',\n input,\n running_mean,\n running_var,\n weight,\n bias,\n momentum_f=momentum,\n eps_f=eps,\n group_i=group,\n group_size_i=group_size,\n stats_mode=stats_mode)\n\n @staticmethod\n def forward(self, input, running_mean, running_var, weight, bias, momentum,\n eps, group, group_size, stats_mode):\n self.momentum = momentum\n self.eps = eps\n self.group = group\n self.group_size = group_size\n self.stats_mode = stats_mode\n\n assert isinstance(\n input, (torch.HalfTensor, torch.FloatTensor,\n torch.cuda.HalfTensor, torch.cuda.FloatTensor)), \\\n f'only support Half or Float Tensor, but {input.type()}'\n output = torch.zeros_like(input)\n input3d = input.flatten(start_dim=2)\n output3d = output.view_as(input3d)\n num_channels = input3d.size(1)\n\n # ensure mean/var/norm/std are initialized as zeros\n # ``torch.empty()`` does not guarantee that\n mean = torch.zeros(\n num_channels, dtype=torch.float, device=input3d.device)\n var = torch.zeros(\n num_channels, dtype=torch.float, device=input3d.device)\n norm = torch.zeros_like(\n input3d, dtype=torch.float, device=input3d.device)\n std = torch.zeros(\n num_channels, dtype=torch.float, device=input3d.device)\n\n batch_size = input3d.size(0)\n if batch_size > 0:\n ext_module.sync_bn_forward_mean(input3d, mean)\n batch_flag = torch.ones([1], device=mean.device, dtype=mean.dtype)\n else:\n # skip updating mean and leave it as zeros when the input is empty\n batch_flag = torch.zeros([1], device=mean.device, dtype=mean.dtype)\n\n # synchronize mean and the batch flag\n vec = torch.cat([mean, batch_flag])\n if self.stats_mode == 'N':\n vec *= batch_size\n if self.group_size > 1:\n dist.all_reduce(vec, group=self.group)\n total_batch = vec[-1].detach()\n mean = vec[:num_channels]\n\n if self.stats_mode == 'default':\n mean = mean / self.group_size\n elif self.stats_mode == 'N':\n mean = mean / total_batch.clamp(min=1)\n else:\n raise NotImplementedError\n\n # leave var as zeros when the input is empty\n if batch_size > 0:\n ext_module.sync_bn_forward_var(input3d, mean, var)\n\n if self.stats_mode == 'N':\n var *= batch_size\n if self.group_size > 1:\n dist.all_reduce(var, group=self.group)\n\n if self.stats_mode == 'default':\n var /= self.group_size\n elif self.stats_mode == 'N':\n var /= total_batch.clamp(min=1)\n else:\n raise NotImplementedError\n\n # if the total batch size over all the ranks is zero,\n # we should not update the statistics in the current batch\n update_flag = total_batch.clamp(max=1)\n momentum = update_flag * self.momentum\n ext_module.sync_bn_forward_output(\n input3d,\n mean,\n var,\n weight,\n bias,\n running_mean,\n running_var,\n norm,\n std,\n output3d,\n eps=self.eps,\n momentum=momentum,\n group_size=self.group_size)\n self.save_for_backward(norm, std, weight)\n return output\n\n @staticmethod\n @once_differentiable\n def backward(self, grad_output):\n norm, std, weight = self.saved_tensors\n grad_weight = torch.zeros_like(weight)\n grad_bias = torch.zeros_like(weight)\n grad_input = torch.zeros_like(grad_output)\n grad_output3d = grad_output.flatten(start_dim=2)\n grad_input3d = grad_input.view_as(grad_output3d)\n\n batch_size = grad_input3d.size(0)\n if batch_size > 0:\n ext_module.sync_bn_backward_param(grad_output3d, norm, grad_weight,\n grad_bias)\n\n # all reduce\n if self.group_size > 1:\n dist.all_reduce(grad_weight, group=self.group)\n dist.all_reduce(grad_bias, group=self.group)\n grad_weight /= self.group_size\n grad_bias /= self.group_size\n\n if batch_size > 0:\n ext_module.sync_bn_backward_data(grad_output3d, weight,\n grad_weight, grad_bias, norm, std,\n grad_input3d)\n\n return grad_input, None, None, grad_weight, grad_bias, \\\n None, None, None, None, None\n\n\n@NORM_LAYERS.register_module(name='MMSyncBN')\nclass SyncBatchNorm(Module):\n \"\"\"Synchronized Batch Normalization.\n\n Args:\n num_features (int): number of features/chennels in input tensor\n eps (float, optional): a value added to the denominator for numerical\n stability. Defaults to 1e-5.\n momentum (float, optional): the value used for the running_mean and\n running_var computation. Defaults to 0.1.\n affine (bool, optional): whether to use learnable affine parameters.\n Defaults to True.\n track_running_stats (bool, optional): whether to track the running\n mean and variance during training. When set to False, this\n module does not track such statistics, and initializes statistics\n buffers ``running_mean`` and ``running_var`` as ``None``. When\n these buffers are ``None``, this module always uses batch\n statistics in both training and eval modes. Defaults to True.\n group (int, optional): synchronization of stats happen within\n each process group individually. By default it is synchronization\n across the whole world. Defaults to None.\n stats_mode (str, optional): The statistical mode. Available options\n includes ``'default'`` and ``'N'``. Defaults to 'default'.\n When ``stats_mode=='default'``, it computes the overall statistics\n using those from each worker with equal weight, i.e., the\n statistics are synchronized and simply divied by ``group``. This\n mode will produce inaccurate statistics when empty tensors occur.\n When ``stats_mode=='N'``, it compute the overall statistics using\n the total number of batches in each worker ignoring the number of\n group, i.e., the statistics are synchronized and then divied by\n the total batch ``N``. This mode is beneficial when empty tensors\n occur during training, as it average the total mean by the real\n number of batch.\n \"\"\"\n\n def __init__(self,\n num_features,\n eps=1e-5,\n momentum=0.1,\n affine=True,\n track_running_stats=True,\n group=None,\n stats_mode='default'):\n super(SyncBatchNorm, self).__init__()\n self.num_features = num_features\n self.eps = eps\n self.momentum = momentum\n self.affine = affine\n self.track_running_stats = track_running_stats\n group = dist.group.WORLD if group is None else group\n self.group = group\n self.group_size = dist.get_world_size(group)\n assert stats_mode in ['default', 'N'], \\\n f'\"stats_mode\" only accepts \"default\" and \"N\", got \"{stats_mode}\"'\n self.stats_mode = stats_mode\n if self.affine:\n self.weight = Parameter(torch.Tensor(num_features))\n self.bias = Parameter(torch.Tensor(num_features))\n else:\n self.register_parameter('weight', None)\n self.register_parameter('bias', None)\n if self.track_running_stats:\n self.register_buffer('running_mean', torch.zeros(num_features))\n self.register_buffer('running_var', torch.ones(num_features))\n self.register_buffer('num_batches_tracked',\n torch.tensor(0, dtype=torch.long))\n else:\n self.register_buffer('running_mean', None)\n self.register_buffer('running_var', None)\n self.register_buffer('num_batches_tracked', None)\n self.reset_parameters()\n\n def reset_running_stats(self):\n if self.track_running_stats:\n self.running_mean.zero_()\n self.running_var.fill_(1)\n self.num_batches_tracked.zero_()\n\n def reset_parameters(self):\n self.reset_running_stats()\n if self.affine:\n self.weight.data.uniform_() # pytorch use ones_()\n self.bias.data.zero_()\n\n def forward(self, input):\n if input.dim() < 2:\n raise ValueError(\n f'expected at least 2D input, got {input.dim()}D input')\n if self.momentum is None:\n exponential_average_factor = 0.0\n else:\n exponential_average_factor = self.momentum\n\n if self.training and self.track_running_stats:\n if self.num_batches_tracked is not None:\n self.num_batches_tracked += 1\n if self.momentum is None: # use cumulative moving average\n exponential_average_factor = 1.0 / float(\n self.num_batches_tracked)\n else: # use exponential moving average\n exponential_average_factor = self.momentum\n\n if self.training or not self.track_running_stats:\n return SyncBatchNormFunction.apply(\n input, self.running_mean, self.running_var, self.weight,\n self.bias, exponential_average_factor, self.eps, self.group,\n self.group_size, self.stats_mode)\n else:\n return F.batch_norm(input, self.running_mean, self.running_var,\n self.weight, self.bias, False,\n exponential_average_factor, self.eps)\n\n def __repr__(self):\n s = self.__class__.__name__\n s += f'({self.num_features}, '\n s += f'eps={self.eps}, '\n s += f'momentum={self.momentum}, '\n s += f'affine={self.affine}, '\n s += f'track_running_stats={self.track_running_stats}, '\n s += f'group_size={self.group_size},'\n s += f'stats_mode={self.stats_mode})'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/three_interpolate.py", "content": "from typing import Tuple\n\nimport torch\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['three_interpolate_forward', 'three_interpolate_backward'])\n\n\nclass ThreeInterpolate(Function):\n \"\"\"Performs weighted linear interpolation on 3 features.\n\n Please refer to `Paper of PointNet++ `_\n for more details.\n \"\"\"\n\n @staticmethod\n def forward(ctx, features: torch.Tensor, indices: torch.Tensor,\n weight: torch.Tensor) -> torch.Tensor:\n \"\"\"\n Args:\n features (torch.Tensor): (B, C, M) Features descriptors to be\n interpolated.\n indices (torch.Tensor): (B, n, 3) indices of three nearest\n neighbor features for the target features.\n weight (torch.Tensor): (B, n, 3) weights of three nearest\n neighbor features for the target features.\n\n Returns:\n torch.Tensor: (B, C, N) tensor of the interpolated features\n \"\"\"\n assert features.is_contiguous()\n assert indices.is_contiguous()\n assert weight.is_contiguous()\n\n B, c, m = features.size()\n n = indices.size(1)\n ctx.three_interpolate_for_backward = (indices, weight, m)\n output = torch.cuda.FloatTensor(B, c, n)\n\n ext_module.three_interpolate_forward(\n features, indices, weight, output, b=B, c=c, m=m, n=n)\n return output\n\n @staticmethod\n def backward(\n ctx, grad_out: torch.Tensor\n ) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]:\n \"\"\"\n Args:\n grad_out (torch.Tensor): (B, C, N) tensor with gradients of outputs\n\n Returns:\n torch.Tensor: (B, C, M) tensor with gradients of features\n \"\"\"\n idx, weight, m = ctx.three_interpolate_for_backward\n B, c, n = grad_out.size()\n\n grad_features = torch.cuda.FloatTensor(B, c, m).zero_()\n grad_out_data = grad_out.data.contiguous()\n\n ext_module.three_interpolate_backward(\n grad_out_data, idx, weight, grad_features.data, b=B, c=c, n=n, m=m)\n return grad_features, None, None\n\n\nthree_interpolate = ThreeInterpolate.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/three_nn.py", "content": "from typing import Tuple\n\nimport torch\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext', ['three_nn_forward'])\n\n\nclass ThreeNN(Function):\n \"\"\"Find the top-3 nearest neighbors of the target set from the source set.\n\n Please refer to `Paper of PointNet++ `_\n for more details.\n \"\"\"\n\n @staticmethod\n def forward(ctx, target: torch.Tensor,\n source: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\n Args:\n target (torch.Tensor): shape (B, N, 3), points set that needs to\n find the nearest neighbors.\n source (torch.Tensor): shape (B, M, 3), points set that is used\n to find the nearest neighbors of points in target set.\n\n Returns:\n torch.Tensor: shape (B, N, 3), L2 distance of each point in target\n set to their corresponding top three nearest neighbors.\n \"\"\"\n target = target.contiguous()\n source = source.contiguous()\n\n B, N, _ = target.size()\n m = source.size(1)\n dist2 = torch.cuda.FloatTensor(B, N, 3)\n idx = torch.cuda.IntTensor(B, N, 3)\n\n ext_module.three_nn_forward(target, source, dist2, idx, b=B, n=N, m=m)\n if torch.__version__ != 'parrots':\n ctx.mark_non_differentiable(idx)\n\n return torch.sqrt(dist2), idx\n\n @staticmethod\n def backward(ctx, a=None, b=None):\n return None, None\n\n\nthree_nn = ThreeNN.apply\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/tin_shift.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n# Code reference from \"Temporal Interlacing Network\"\n# https://github.com/deepcs233/TIN/blob/master/cuda_shift/rtc_wrap.py\n# Hao Shao, Shengju Qian, Yu Liu\n# shaoh19@mails.tsinghua.edu.cn, sjqian@cse.cuhk.edu.hk, yuliu@ee.cuhk.edu.hk\n\nimport torch\nimport torch.nn as nn\nfrom torch.autograd import Function\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext('_ext',\n ['tin_shift_forward', 'tin_shift_backward'])\n\n\nclass TINShiftFunction(Function):\n\n @staticmethod\n def forward(ctx, input, shift):\n C = input.size(2)\n num_segments = shift.size(1)\n if C // num_segments <= 0 or C % num_segments != 0:\n raise ValueError('C should be a multiple of num_segments, '\n f'but got C={C} and num_segments={num_segments}.')\n\n ctx.save_for_backward(shift)\n\n out = torch.zeros_like(input)\n ext_module.tin_shift_forward(input, shift, out)\n\n return out\n\n @staticmethod\n def backward(ctx, grad_output):\n\n shift = ctx.saved_tensors[0]\n data_grad_input = grad_output.new(*grad_output.size()).zero_()\n shift_grad_input = shift.new(*shift.size()).zero_()\n ext_module.tin_shift_backward(grad_output, shift, data_grad_input)\n\n return data_grad_input, shift_grad_input\n\n\ntin_shift = TINShiftFunction.apply\n\n\nclass TINShift(nn.Module):\n \"\"\"Temporal Interlace Shift.\n\n Temporal Interlace shift is a differentiable temporal-wise frame shifting\n which is proposed in \"Temporal Interlacing Network\"\n\n Please refer to `Temporal Interlacing Network\n `_ for more details.\n\n Code is modified from https://github.com/mit-han-lab/temporal-shift-module\n \"\"\"\n\n def forward(self, input, shift):\n \"\"\"Perform temporal interlace shift.\n\n Args:\n input (torch.Tensor): Feature map with shape\n [N, num_segments, C, H * W].\n shift (torch.Tensor): Shift tensor with shape [N, num_segments].\n\n Returns:\n Feature map after temporal interlace shift.\n \"\"\"\n return tin_shift(input, shift)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/upfirdn2d.py", "content": "# modified from https://github.com/rosinality/stylegan2-pytorch/blob/master/op/upfirdn2d.py # noqa:E501\n\n# Copyright (c) 2021, NVIDIA Corporation. All rights reserved.\n# NVIDIA Source Code License for StyleGAN2 with Adaptive Discriminator\n# Augmentation (ADA)\n# =======================================================================\n\n# 1. Definitions\n\n# \"Licensor\" means any person or entity that distributes its Work.\n\n# \"Software\" means the original work of authorship made available under\n# this License.\n\n# \"Work\" means the Software and any additions to or derivative works of\n# the Software that are made available under this License.\n\n# The terms \"reproduce,\" \"reproduction,\" \"derivative works,\" and\n# \"distribution\" have the meaning as provided under U.S. copyright law;\n# provided, however, that for the purposes of this License, derivative\n# works shall not include works that remain separable from, or merely\n# link (or bind by name) to the interfaces of, the Work.\n\n# Works, including the Software, are \"made available\" under this License\n# by including in or with the Work either (a) a copyright notice\n# referencing the applicability of this License to the Work, or (b) a\n# copy of this License.\n\n# 2. License Grants\n\n# 2.1 Copyright Grant. Subject to the terms and conditions of this\n# License, each Licensor grants to you a perpetual, worldwide,\n# non-exclusive, royalty-free, copyright license to reproduce,\n# prepare derivative works of, publicly display, publicly perform,\n# sublicense and distribute its Work and any resulting derivative\n# works in any form.\n\n# 3. Limitations\n\n# 3.1 Redistribution. You may reproduce or distribute the Work only\n# if (a) you do so under this License, (b) you include a complete\n# copy of this License with your distribution, and (c) you retain\n# without modification any copyright, patent, trademark, or\n# attribution notices that are present in the Work.\n\n# 3.2 Derivative Works. You may specify that additional or different\n# terms apply to the use, reproduction, and distribution of your\n# derivative works of the Work (\"Your Terms\") only if (a) Your Terms\n# provide that the use limitation in Section 3.3 applies to your\n# derivative works, and (b) you identify the specific derivative\n# works that are subject to Your Terms. Notwithstanding Your Terms,\n# this License (including the redistribution requirements in Section\n# 3.1) will continue to apply to the Work itself.\n\n# 3.3 Use Limitation. The Work and any derivative works thereof only\n# may be used or intended for use non-commercially. Notwithstanding\n# the foregoing, NVIDIA and its affiliates may use the Work and any\n# derivative works commercially. As used herein, \"non-commercially\"\n# means for research or evaluation purposes only.\n\n# 3.4 Patent Claims. If you bring or threaten to bring a patent claim\n# against any Licensor (including any claim, cross-claim or\n# counterclaim in a lawsuit) to enforce any patents that you allege\n# are infringed by any Work, then your rights under this License from\n# such Licensor (including the grant in Section 2.1) will terminate\n# immediately.\n\n# 3.5 Trademarks. This License does not grant any rights to use any\n# Licensor’s or its affiliates’ names, logos, or trademarks, except\n# as necessary to reproduce the notices described in this License.\n\n# 3.6 Termination. If you violate any term of this License, then your\n# rights under this License (including the grant in Section 2.1) will\n# terminate immediately.\n\n# 4. Disclaimer of Warranty.\n\n# THE WORK IS PROVIDED \"AS IS\" WITHOUT WARRANTIES OR CONDITIONS OF ANY\n# KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF\n# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR\n# NON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER\n# THIS LICENSE.\n\n# 5. Limitation of Liability.\n\n# EXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL\n# THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE\n# SHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT,\n# INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF\n# OR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK\n# (INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION,\n# LOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER\n# COMMERCIAL DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF\n# THE POSSIBILITY OF SUCH DAMAGES.\n\n# =======================================================================\n\nimport torch\nfrom torch.autograd import Function\nfrom torch.nn import functional as F\n\nfrom mmcv.utils import to_2tuple\nfrom ..utils import ext_loader\n\nupfirdn2d_ext = ext_loader.load_ext('_ext', ['upfirdn2d'])\n\n\nclass UpFirDn2dBackward(Function):\n\n @staticmethod\n def forward(ctx, grad_output, kernel, grad_kernel, up, down, pad, g_pad,\n in_size, out_size):\n\n up_x, up_y = up\n down_x, down_y = down\n g_pad_x0, g_pad_x1, g_pad_y0, g_pad_y1 = g_pad\n\n grad_output = grad_output.reshape(-1, out_size[0], out_size[1], 1)\n\n grad_input = upfirdn2d_ext.upfirdn2d(\n grad_output,\n grad_kernel,\n up_x=down_x,\n up_y=down_y,\n down_x=up_x,\n down_y=up_y,\n pad_x0=g_pad_x0,\n pad_x1=g_pad_x1,\n pad_y0=g_pad_y0,\n pad_y1=g_pad_y1)\n grad_input = grad_input.view(in_size[0], in_size[1], in_size[2],\n in_size[3])\n\n ctx.save_for_backward(kernel)\n\n pad_x0, pad_x1, pad_y0, pad_y1 = pad\n\n ctx.up_x = up_x\n ctx.up_y = up_y\n ctx.down_x = down_x\n ctx.down_y = down_y\n ctx.pad_x0 = pad_x0\n ctx.pad_x1 = pad_x1\n ctx.pad_y0 = pad_y0\n ctx.pad_y1 = pad_y1\n ctx.in_size = in_size\n ctx.out_size = out_size\n\n return grad_input\n\n @staticmethod\n def backward(ctx, gradgrad_input):\n kernel, = ctx.saved_tensors\n\n gradgrad_input = gradgrad_input.reshape(-1, ctx.in_size[2],\n ctx.in_size[3], 1)\n\n gradgrad_out = upfirdn2d_ext.upfirdn2d(\n gradgrad_input,\n kernel,\n up_x=ctx.up_x,\n up_y=ctx.up_y,\n down_x=ctx.down_x,\n down_y=ctx.down_y,\n pad_x0=ctx.pad_x0,\n pad_x1=ctx.pad_x1,\n pad_y0=ctx.pad_y0,\n pad_y1=ctx.pad_y1)\n # gradgrad_out = gradgrad_out.view(ctx.in_size[0], ctx.out_size[0],\n # ctx.out_size[1], ctx.in_size[3])\n gradgrad_out = gradgrad_out.view(ctx.in_size[0], ctx.in_size[1],\n ctx.out_size[0], ctx.out_size[1])\n\n return gradgrad_out, None, None, None, None, None, None, None, None\n\n\nclass UpFirDn2d(Function):\n\n @staticmethod\n def forward(ctx, input, kernel, up, down, pad):\n up_x, up_y = up\n down_x, down_y = down\n pad_x0, pad_x1, pad_y0, pad_y1 = pad\n\n kernel_h, kernel_w = kernel.shape\n batch, channel, in_h, in_w = input.shape\n ctx.in_size = input.shape\n\n input = input.reshape(-1, in_h, in_w, 1)\n\n ctx.save_for_backward(kernel, torch.flip(kernel, [0, 1]))\n\n out_h = (in_h * up_y + pad_y0 + pad_y1 - kernel_h) // down_y + 1\n out_w = (in_w * up_x + pad_x0 + pad_x1 - kernel_w) // down_x + 1\n ctx.out_size = (out_h, out_w)\n\n ctx.up = (up_x, up_y)\n ctx.down = (down_x, down_y)\n ctx.pad = (pad_x0, pad_x1, pad_y0, pad_y1)\n\n g_pad_x0 = kernel_w - pad_x0 - 1\n g_pad_y0 = kernel_h - pad_y0 - 1\n g_pad_x1 = in_w * up_x - out_w * down_x + pad_x0 - up_x + 1\n g_pad_y1 = in_h * up_y - out_h * down_y + pad_y0 - up_y + 1\n\n ctx.g_pad = (g_pad_x0, g_pad_x1, g_pad_y0, g_pad_y1)\n\n out = upfirdn2d_ext.upfirdn2d(\n input,\n kernel,\n up_x=up_x,\n up_y=up_y,\n down_x=down_x,\n down_y=down_y,\n pad_x0=pad_x0,\n pad_x1=pad_x1,\n pad_y0=pad_y0,\n pad_y1=pad_y1)\n # out = out.view(major, out_h, out_w, minor)\n out = out.view(-1, channel, out_h, out_w)\n\n return out\n\n @staticmethod\n def backward(ctx, grad_output):\n kernel, grad_kernel = ctx.saved_tensors\n\n grad_input = UpFirDn2dBackward.apply(\n grad_output,\n kernel,\n grad_kernel,\n ctx.up,\n ctx.down,\n ctx.pad,\n ctx.g_pad,\n ctx.in_size,\n ctx.out_size,\n )\n\n return grad_input, None, None, None, None\n\n\ndef upfirdn2d(input, kernel, up=1, down=1, pad=(0, 0)):\n \"\"\"UpFRIDn for 2d features.\n\n UpFIRDn is short for upsample, apply FIR filter and downsample. More\n details can be found in:\n https://www.mathworks.com/help/signal/ref/upfirdn.html\n\n Args:\n input (torch.Tensor): Tensor with shape of (n, c, h, w).\n kernel (torch.Tensor): Filter kernel.\n up (int | tuple[int], optional): Upsampling factor. If given a number,\n we will use this factor for the both height and width side.\n Defaults to 1.\n down (int | tuple[int], optional): Downsampling factor. If given a\n number, we will use this factor for the both height and width side.\n Defaults to 1.\n pad (tuple[int], optional): Padding for tensors, (x_pad, y_pad) or\n (x_pad_0, x_pad_1, y_pad_0, y_pad_1). Defaults to (0, 0).\n\n Returns:\n torch.Tensor: Tensor after UpFIRDn.\n \"\"\"\n if input.device.type == 'cpu':\n if len(pad) == 2:\n pad = (pad[0], pad[1], pad[0], pad[1])\n\n up = to_2tuple(up)\n\n down = to_2tuple(down)\n\n out = upfirdn2d_native(input, kernel, up[0], up[1], down[0], down[1],\n pad[0], pad[1], pad[2], pad[3])\n else:\n _up = to_2tuple(up)\n\n _down = to_2tuple(down)\n\n if len(pad) == 4:\n _pad = pad\n elif len(pad) == 2:\n _pad = (pad[0], pad[1], pad[0], pad[1])\n\n out = UpFirDn2d.apply(input, kernel, _up, _down, _pad)\n\n return out\n\n\ndef upfirdn2d_native(input, kernel, up_x, up_y, down_x, down_y, pad_x0, pad_x1,\n pad_y0, pad_y1):\n _, channel, in_h, in_w = input.shape\n input = input.reshape(-1, in_h, in_w, 1)\n\n _, in_h, in_w, minor = input.shape\n kernel_h, kernel_w = kernel.shape\n\n out = input.view(-1, in_h, 1, in_w, 1, minor)\n out = F.pad(out, [0, 0, 0, up_x - 1, 0, 0, 0, up_y - 1])\n out = out.view(-1, in_h * up_y, in_w * up_x, minor)\n\n out = F.pad(\n out,\n [0, 0,\n max(pad_x0, 0),\n max(pad_x1, 0),\n max(pad_y0, 0),\n max(pad_y1, 0)])\n out = out[:,\n max(-pad_y0, 0):out.shape[1] - max(-pad_y1, 0),\n max(-pad_x0, 0):out.shape[2] - max(-pad_x1, 0), :, ]\n\n out = out.permute(0, 3, 1, 2)\n out = out.reshape(\n [-1, 1, in_h * up_y + pad_y0 + pad_y1, in_w * up_x + pad_x0 + pad_x1])\n w = torch.flip(kernel, [0, 1]).view(1, 1, kernel_h, kernel_w)\n out = F.conv2d(out, w)\n out = out.reshape(\n -1,\n minor,\n in_h * up_y + pad_y0 + pad_y1 - kernel_h + 1,\n in_w * up_x + pad_x0 + pad_x1 - kernel_w + 1,\n )\n out = out.permute(0, 2, 3, 1)\n out = out[:, ::down_y, ::down_x, :]\n\n out_h = (in_h * up_y + pad_y0 + pad_y1 - kernel_h) // down_y + 1\n out_w = (in_w * up_x + pad_x0 + pad_x1 - kernel_w) // down_x + 1\n\n return out.view(-1, channel, out_h, out_w)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/ops/voxelize.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch import nn\nfrom torch.autograd import Function\nfrom torch.nn.modules.utils import _pair\n\nfrom ..utils import ext_loader\n\next_module = ext_loader.load_ext(\n '_ext', ['dynamic_voxelize_forward', 'hard_voxelize_forward'])\n\n\nclass _Voxelization(Function):\n\n @staticmethod\n def forward(ctx,\n points,\n voxel_size,\n coors_range,\n max_points=35,\n max_voxels=20000):\n \"\"\"Convert kitti points(N, >=3) to voxels.\n\n Args:\n points (torch.Tensor): [N, ndim]. Points[:, :3] contain xyz points\n and points[:, 3:] contain other information like reflectivity.\n voxel_size (tuple or float): The size of voxel with the shape of\n [3].\n coors_range (tuple or float): The coordinate range of voxel with\n the shape of [6].\n max_points (int, optional): maximum points contained in a voxel. if\n max_points=-1, it means using dynamic_voxelize. Default: 35.\n max_voxels (int, optional): maximum voxels this function create.\n for second, 20000 is a good choice. Users should shuffle points\n before call this function because max_voxels may drop points.\n Default: 20000.\n\n Returns:\n tuple[torch.Tensor]: tuple[torch.Tensor]: A tuple contains three\n elements. The first one is the output voxels with the shape of\n [M, max_points, n_dim], which only contain points and returned\n when max_points != -1. The second is the voxel coordinates with\n shape of [M, 3]. The last is number of point per voxel with the\n shape of [M], which only returned when max_points != -1.\n \"\"\"\n if max_points == -1 or max_voxels == -1:\n coors = points.new_zeros(size=(points.size(0), 3), dtype=torch.int)\n ext_module.dynamic_voxelize_forward(\n points,\n torch.tensor(voxel_size, dtype=torch.float),\n torch.tensor(coors_range, dtype=torch.float),\n coors,\n NDim=3)\n return coors\n else:\n voxels = points.new_zeros(\n size=(max_voxels, max_points, points.size(1)))\n coors = points.new_zeros(size=(max_voxels, 3), dtype=torch.int)\n num_points_per_voxel = points.new_zeros(\n size=(max_voxels, ), dtype=torch.int)\n voxel_num = torch.zeros(size=(), dtype=torch.long)\n ext_module.hard_voxelize_forward(\n points,\n torch.tensor(voxel_size, dtype=torch.float),\n torch.tensor(coors_range, dtype=torch.float),\n voxels,\n coors,\n num_points_per_voxel,\n voxel_num,\n max_points=max_points,\n max_voxels=max_voxels,\n NDim=3)\n # select the valid voxels\n voxels_out = voxels[:voxel_num]\n coors_out = coors[:voxel_num]\n num_points_per_voxel_out = num_points_per_voxel[:voxel_num]\n return voxels_out, coors_out, num_points_per_voxel_out\n\n\nvoxelization = _Voxelization.apply\n\n\nclass Voxelization(nn.Module):\n \"\"\"Convert kitti points(N, >=3) to voxels.\n\n Please refer to `Point-Voxel CNN for Efficient 3D Deep Learning\n `_ for more details.\n\n Args:\n voxel_size (tuple or float): The size of voxel with the shape of [3].\n point_cloud_range (tuple or float): The coordinate range of voxel with\n the shape of [6].\n max_num_points (int): maximum points contained in a voxel. if\n max_points=-1, it means using dynamic_voxelize.\n max_voxels (int, optional): maximum voxels this function create.\n for second, 20000 is a good choice. Users should shuffle points\n before call this function because max_voxels may drop points.\n Default: 20000.\n \"\"\"\n\n def __init__(self,\n voxel_size,\n point_cloud_range,\n max_num_points,\n max_voxels=20000):\n super().__init__()\n\n self.voxel_size = voxel_size\n self.point_cloud_range = point_cloud_range\n self.max_num_points = max_num_points\n if isinstance(max_voxels, tuple):\n self.max_voxels = max_voxels\n else:\n self.max_voxels = _pair(max_voxels)\n\n point_cloud_range = torch.tensor(\n point_cloud_range, dtype=torch.float32)\n voxel_size = torch.tensor(voxel_size, dtype=torch.float32)\n grid_size = (point_cloud_range[3:] -\n point_cloud_range[:3]) / voxel_size\n grid_size = torch.round(grid_size).long()\n input_feat_shape = grid_size[:2]\n self.grid_size = grid_size\n # the origin shape is as [x-len, y-len, z-len]\n # [w, h, d] -> [d, h, w]\n self.pcd_shape = [*input_feat_shape, 1][::-1]\n\n def forward(self, input):\n if self.training:\n max_voxels = self.max_voxels[0]\n else:\n max_voxels = self.max_voxels[1]\n\n return voxelization(input, self.voxel_size, self.point_cloud_range,\n self.max_num_points, max_voxels)\n\n def __repr__(self):\n s = self.__class__.__name__ + '('\n s += 'voxel_size=' + str(self.voxel_size)\n s += ', point_cloud_range=' + str(self.point_cloud_range)\n s += ', max_num_points=' + str(self.max_num_points)\n s += ', max_voxels=' + str(self.max_voxels)\n s += ')'\n return s\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .collate import collate\nfrom .data_container import DataContainer\nfrom .data_parallel import MMDataParallel\nfrom .distributed import MMDistributedDataParallel\nfrom .registry import MODULE_WRAPPERS\nfrom .scatter_gather import scatter, scatter_kwargs\nfrom .utils import is_module_wrapper\n\n__all__ = [\n 'collate', 'DataContainer', 'MMDataParallel', 'MMDistributedDataParallel',\n 'scatter', 'scatter_kwargs', 'is_module_wrapper', 'MODULE_WRAPPERS'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/_functions.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch.nn.parallel._functions import _get_stream\n\n\ndef scatter(input, devices, streams=None):\n \"\"\"Scatters tensor across multiple GPUs.\"\"\"\n if streams is None:\n streams = [None] * len(devices)\n\n if isinstance(input, list):\n chunk_size = (len(input) - 1) // len(devices) + 1\n outputs = [\n scatter(input[i], [devices[i // chunk_size]],\n [streams[i // chunk_size]]) for i in range(len(input))\n ]\n return outputs\n elif isinstance(input, torch.Tensor):\n output = input.contiguous()\n # TODO: copy to a pinned buffer first (if copying from CPU)\n stream = streams[0] if output.numel() > 0 else None\n if devices != [-1]:\n with torch.cuda.device(devices[0]), torch.cuda.stream(stream):\n output = output.cuda(devices[0], non_blocking=True)\n\n return output\n else:\n raise Exception(f'Unknown type {type(input)}.')\n\n\ndef synchronize_stream(output, devices, streams):\n if isinstance(output, list):\n chunk_size = len(output) // len(devices)\n for i in range(len(devices)):\n for j in range(chunk_size):\n synchronize_stream(output[i * chunk_size + j], [devices[i]],\n [streams[i]])\n elif isinstance(output, torch.Tensor):\n if output.numel() != 0:\n with torch.cuda.device(devices[0]):\n main_stream = torch.cuda.current_stream()\n main_stream.wait_stream(streams[0])\n output.record_stream(main_stream)\n else:\n raise Exception(f'Unknown type {type(output)}.')\n\n\ndef get_input_device(input):\n if isinstance(input, list):\n for item in input:\n input_device = get_input_device(item)\n if input_device != -1:\n return input_device\n return -1\n elif isinstance(input, torch.Tensor):\n return input.get_device() if input.is_cuda else -1\n else:\n raise Exception(f'Unknown type {type(input)}.')\n\n\nclass Scatter:\n\n @staticmethod\n def forward(target_gpus, input):\n input_device = get_input_device(input)\n streams = None\n if input_device == -1 and target_gpus != [-1]:\n # Perform CPU to GPU copies in a background stream\n streams = [_get_stream(device) for device in target_gpus]\n\n outputs = scatter(input, target_gpus, streams)\n # Synchronize with the copy stream\n if streams is not None:\n synchronize_stream(outputs, target_gpus, streams)\n\n return tuple(outputs) if isinstance(outputs, list) else (outputs, )\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/collate.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom collections.abc import Mapping, Sequence\n\nimport torch\nimport torch.nn.functional as F\nfrom torch.utils.data.dataloader import default_collate\n\nfrom .data_container import DataContainer\n\n\ndef collate(batch, samples_per_gpu=1):\n \"\"\"Puts each data field into a tensor/DataContainer with outer dimension\n batch size.\n\n Extend default_collate to add support for\n :type:`~mmcv.parallel.DataContainer`. There are 3 cases.\n\n 1. cpu_only = True, e.g., meta data\n 2. cpu_only = False, stack = True, e.g., images tensors\n 3. cpu_only = False, stack = False, e.g., gt bboxes\n \"\"\"\n\n if not isinstance(batch, Sequence):\n raise TypeError(f'{batch.dtype} is not supported.')\n\n if isinstance(batch[0], DataContainer):\n stacked = []\n if batch[0].cpu_only:\n for i in range(0, len(batch), samples_per_gpu):\n stacked.append(\n [sample.data for sample in batch[i:i + samples_per_gpu]])\n return DataContainer(\n stacked, batch[0].stack, batch[0].padding_value, cpu_only=True)\n elif batch[0].stack:\n for i in range(0, len(batch), samples_per_gpu):\n assert isinstance(batch[i].data, torch.Tensor)\n\n if batch[i].pad_dims is not None:\n ndim = batch[i].dim()\n assert ndim > batch[i].pad_dims\n max_shape = [0 for _ in range(batch[i].pad_dims)]\n for dim in range(1, batch[i].pad_dims + 1):\n max_shape[dim - 1] = batch[i].size(-dim)\n for sample in batch[i:i + samples_per_gpu]:\n for dim in range(0, ndim - batch[i].pad_dims):\n assert batch[i].size(dim) == sample.size(dim)\n for dim in range(1, batch[i].pad_dims + 1):\n max_shape[dim - 1] = max(max_shape[dim - 1],\n sample.size(-dim))\n padded_samples = []\n for sample in batch[i:i + samples_per_gpu]:\n pad = [0 for _ in range(batch[i].pad_dims * 2)]\n for dim in range(1, batch[i].pad_dims + 1):\n pad[2 * dim -\n 1] = max_shape[dim - 1] - sample.size(-dim)\n padded_samples.append(\n F.pad(\n sample.data, pad, value=sample.padding_value))\n stacked.append(default_collate(padded_samples))\n elif batch[i].pad_dims is None:\n stacked.append(\n default_collate([\n sample.data\n for sample in batch[i:i + samples_per_gpu]\n ]))\n else:\n raise ValueError(\n 'pad_dims should be either None or integers (1-3)')\n\n else:\n for i in range(0, len(batch), samples_per_gpu):\n stacked.append(\n [sample.data for sample in batch[i:i + samples_per_gpu]])\n return DataContainer(stacked, batch[0].stack, batch[0].padding_value)\n elif isinstance(batch[0], Sequence):\n transposed = zip(*batch)\n return [collate(samples, samples_per_gpu) for samples in transposed]\n elif isinstance(batch[0], Mapping):\n return {\n key: collate([d[key] for d in batch], samples_per_gpu)\n for key in batch[0]\n }\n else:\n return default_collate(batch)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/data_container.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport functools\n\nimport torch\n\n\ndef assert_tensor_type(func):\n\n @functools.wraps(func)\n def wrapper(*args, **kwargs):\n if not isinstance(args[0].data, torch.Tensor):\n raise AttributeError(\n f'{args[0].__class__.__name__} has no attribute '\n f'{func.__name__} for type {args[0].datatype}')\n return func(*args, **kwargs)\n\n return wrapper\n\n\nclass DataContainer:\n \"\"\"A container for any type of objects.\n\n Typically tensors will be stacked in the collate function and sliced along\n some dimension in the scatter function. This behavior has some limitations.\n 1. All tensors have to be the same size.\n 2. Types are limited (numpy array or Tensor).\n\n We design `DataContainer` and `MMDataParallel` to overcome these\n limitations. The behavior can be either of the following.\n\n - copy to GPU, pad all tensors to the same size and stack them\n - copy to GPU without stacking\n - leave the objects as is and pass it to the model\n - pad_dims specifies the number of last few dimensions to do padding\n \"\"\"\n\n def __init__(self,\n data,\n stack=False,\n padding_value=0,\n cpu_only=False,\n pad_dims=2):\n self._data = data\n self._cpu_only = cpu_only\n self._stack = stack\n self._padding_value = padding_value\n assert pad_dims in [None, 1, 2, 3]\n self._pad_dims = pad_dims\n\n def __repr__(self):\n return f'{self.__class__.__name__}({repr(self.data)})'\n\n def __len__(self):\n return len(self._data)\n\n @property\n def data(self):\n return self._data\n\n @property\n def datatype(self):\n if isinstance(self.data, torch.Tensor):\n return self.data.type()\n else:\n return type(self.data)\n\n @property\n def cpu_only(self):\n return self._cpu_only\n\n @property\n def stack(self):\n return self._stack\n\n @property\n def padding_value(self):\n return self._padding_value\n\n @property\n def pad_dims(self):\n return self._pad_dims\n\n @assert_tensor_type\n def size(self, *args, **kwargs):\n return self.data.size(*args, **kwargs)\n\n @assert_tensor_type\n def dim(self):\n return self.data.dim()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/data_parallel.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom itertools import chain\n\nfrom torch.nn.parallel import DataParallel\n\nfrom .scatter_gather import scatter_kwargs\n\n\nclass MMDataParallel(DataParallel):\n \"\"\"The DataParallel module that supports DataContainer.\n\n MMDataParallel has two main differences with PyTorch DataParallel:\n\n - It supports a custom type :class:`DataContainer` which allows more\n flexible control of input data during both GPU and CPU inference.\n - It implement two more APIs ``train_step()`` and ``val_step()``.\n\n .. warning::\n MMDataParallel only supports single GPU training, if you need to\n train with multiple GPUs, please use MMDistributedDataParallel\n instead. If you have multiple GPUs and you just want to use\n MMDataParallel, you can set the environment variable\n ``CUDA_VISIBLE_DEVICES=0`` or instantiate ``MMDataParallel`` with\n ``device_ids=[0]``.\n\n Args:\n module (:class:`nn.Module`): Module to be encapsulated.\n device_ids (list[int]): Device IDS of modules to be scattered to.\n Defaults to None when GPU is not available.\n output_device (str | int): Device ID for output. Defaults to None.\n dim (int): Dimension used to scatter the data. Defaults to 0.\n \"\"\"\n\n def __init__(self, *args, dim=0, **kwargs):\n super(MMDataParallel, self).__init__(*args, dim=dim, **kwargs)\n self.dim = dim\n if isinstance(self.module, dict) and len(self.device_ids) == 1:\n for name, m in self.module.items():\n self.module[name] = m.to(self.src_device_obj)\n\n def forward(self, *inputs, **kwargs):\n \"\"\"Override the original forward function.\n\n The main difference lies in the CPU inference where the data in\n :class:`DataContainers` will still be gathered.\n \"\"\"\n if not self.device_ids:\n # We add the following line thus the module could gather and\n # convert data containers as those in GPU inference\n inputs, kwargs = self.scatter(inputs, kwargs, [-1])\n return self.module(*inputs[0], **kwargs[0])\n else:\n return super().forward(*inputs, **kwargs)\n\n def scatter(self, inputs, kwargs, device_ids):\n return scatter_kwargs(inputs, kwargs, device_ids, dim=self.dim)\n\n def train_step(self, *inputs, **kwargs):\n if not self.device_ids:\n # We add the following line thus the module could gather and\n # convert data containers as those in GPU inference\n inputs, kwargs = self.scatter(inputs, kwargs, [-1])\n return self.module.train_step(*inputs[0], **kwargs[0])\n\n assert len(self.device_ids) == 1, \\\n ('MMDataParallel only supports single GPU training, if you need to'\n ' train with multiple GPUs, please use MMDistributedDataParallel'\n ' instead.')\n\n for t in chain(self.module.parameters(), self.module.buffers()):\n if t.device != self.src_device_obj:\n raise RuntimeError(\n 'module must have its parameters and buffers '\n f'on device {self.src_device_obj} (device_ids[0]) but '\n f'found one of them on device: {t.device}')\n\n inputs, kwargs = self.scatter(inputs, kwargs, self.device_ids)\n return self.module.train_step(*inputs[0], **kwargs[0])\n\n def val_step(self, *inputs, **kwargs):\n if not self.device_ids:\n # We add the following line thus the module could gather and\n # convert data containers as those in GPU inference\n inputs, kwargs = self.scatter(inputs, kwargs, [-1])\n return self.module.val_step(*inputs[0], **kwargs[0])\n\n assert len(self.device_ids) == 1, \\\n ('MMDataParallel only supports single GPU training, if you need to'\n ' train with multiple GPUs, please use MMDistributedDataParallel'\n ' instead.')\n\n for t in chain(self.module.parameters(), self.module.buffers()):\n if t.device != self.src_device_obj:\n raise RuntimeError(\n 'module must have its parameters and buffers '\n f'on device {self.src_device_obj} (device_ids[0]) but '\n f'found one of them on device: {t.device}')\n\n inputs, kwargs = self.scatter(inputs, kwargs, self.device_ids)\n return self.module.val_step(*inputs[0], **kwargs[0])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/distributed.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch.nn.parallel.distributed import (DistributedDataParallel,\n _find_tensors)\n\nfrom mmcv import print_log\nfrom mmcv.utils import TORCH_VERSION, digit_version\nfrom .scatter_gather import scatter_kwargs\n\n\nclass MMDistributedDataParallel(DistributedDataParallel):\n \"\"\"The DDP module that supports DataContainer.\n\n MMDDP has two main differences with PyTorch DDP:\n\n - It supports a custom type :class:`DataContainer` which allows more\n flexible control of input data.\n - It implement two APIs ``train_step()`` and ``val_step()``.\n \"\"\"\n\n def to_kwargs(self, inputs, kwargs, device_id):\n # Use `self.to_kwargs` instead of `self.scatter` in pytorch1.8\n # to move all tensors to device_id\n return scatter_kwargs(inputs, kwargs, [device_id], dim=self.dim)\n\n def scatter(self, inputs, kwargs, device_ids):\n return scatter_kwargs(inputs, kwargs, device_ids, dim=self.dim)\n\n def train_step(self, *inputs, **kwargs):\n \"\"\"train_step() API for module wrapped by DistributedDataParallel.\n\n This method is basically the same as\n ``DistributedDataParallel.forward()``, while replacing\n ``self.module.forward()`` with ``self.module.train_step()``.\n It is compatible with PyTorch 1.1 - 1.5.\n \"\"\"\n\n # In PyTorch >= 1.7, ``reducer._rebuild_buckets()`` is moved from the\n # end of backward to the beginning of forward.\n if ('parrots' not in TORCH_VERSION\n and digit_version(TORCH_VERSION) >= digit_version('1.7')\n and self.reducer._rebuild_buckets()):\n print_log(\n 'Reducer buckets have been rebuilt in this iteration.',\n logger='mmcv')\n\n if getattr(self, 'require_forward_param_sync', True):\n self._sync_params()\n if self.device_ids:\n inputs, kwargs = self.scatter(inputs, kwargs, self.device_ids)\n if len(self.device_ids) == 1:\n output = self.module.train_step(*inputs[0], **kwargs[0])\n else:\n outputs = self.parallel_apply(\n self._module_copies[:len(inputs)], inputs, kwargs)\n output = self.gather(outputs, self.output_device)\n else:\n output = self.module.train_step(*inputs, **kwargs)\n\n if torch.is_grad_enabled() and getattr(\n self, 'require_backward_grad_sync', True):\n if self.find_unused_parameters:\n self.reducer.prepare_for_backward(list(_find_tensors(output)))\n else:\n self.reducer.prepare_for_backward([])\n else:\n if ('parrots' not in TORCH_VERSION\n and digit_version(TORCH_VERSION) > digit_version('1.2')):\n self.require_forward_param_sync = False\n return output\n\n def val_step(self, *inputs, **kwargs):\n \"\"\"val_step() API for module wrapped by DistributedDataParallel.\n\n This method is basically the same as\n ``DistributedDataParallel.forward()``, while replacing\n ``self.module.forward()`` with ``self.module.val_step()``.\n It is compatible with PyTorch 1.1 - 1.5.\n \"\"\"\n # In PyTorch >= 1.7, ``reducer._rebuild_buckets()`` is moved from the\n # end of backward to the beginning of forward.\n if ('parrots' not in TORCH_VERSION\n and digit_version(TORCH_VERSION) >= digit_version('1.7')\n and self.reducer._rebuild_buckets()):\n print_log(\n 'Reducer buckets have been rebuilt in this iteration.',\n logger='mmcv')\n\n if getattr(self, 'require_forward_param_sync', True):\n self._sync_params()\n if self.device_ids:\n inputs, kwargs = self.scatter(inputs, kwargs, self.device_ids)\n if len(self.device_ids) == 1:\n output = self.module.val_step(*inputs[0], **kwargs[0])\n else:\n outputs = self.parallel_apply(\n self._module_copies[:len(inputs)], inputs, kwargs)\n output = self.gather(outputs, self.output_device)\n else:\n output = self.module.val_step(*inputs, **kwargs)\n\n if torch.is_grad_enabled() and getattr(\n self, 'require_backward_grad_sync', True):\n if self.find_unused_parameters:\n self.reducer.prepare_for_backward(list(_find_tensors(output)))\n else:\n self.reducer.prepare_for_backward([])\n else:\n if ('parrots' not in TORCH_VERSION\n and digit_version(TORCH_VERSION) > digit_version('1.2')):\n self.require_forward_param_sync = False\n return output\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/distributed_deprecated.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.distributed as dist\nimport torch.nn as nn\nfrom torch._utils import (_flatten_dense_tensors, _take_tensors,\n _unflatten_dense_tensors)\n\nfrom mmcv.utils import TORCH_VERSION, digit_version\nfrom .registry import MODULE_WRAPPERS\nfrom .scatter_gather import scatter_kwargs\n\n\n@MODULE_WRAPPERS.register_module()\nclass MMDistributedDataParallel(nn.Module):\n\n def __init__(self,\n module,\n dim=0,\n broadcast_buffers=True,\n bucket_cap_mb=25):\n super(MMDistributedDataParallel, self).__init__()\n self.module = module\n self.dim = dim\n self.broadcast_buffers = broadcast_buffers\n\n self.broadcast_bucket_size = bucket_cap_mb * 1024 * 1024\n self._sync_params()\n\n def _dist_broadcast_coalesced(self, tensors, buffer_size):\n for tensors in _take_tensors(tensors, buffer_size):\n flat_tensors = _flatten_dense_tensors(tensors)\n dist.broadcast(flat_tensors, 0)\n for tensor, synced in zip(\n tensors, _unflatten_dense_tensors(flat_tensors, tensors)):\n tensor.copy_(synced)\n\n def _sync_params(self):\n module_states = list(self.module.state_dict().values())\n if len(module_states) > 0:\n self._dist_broadcast_coalesced(module_states,\n self.broadcast_bucket_size)\n if self.broadcast_buffers:\n if (TORCH_VERSION != 'parrots'\n and digit_version(TORCH_VERSION) < digit_version('1.0')):\n buffers = [b.data for b in self.module._all_buffers()]\n else:\n buffers = [b.data for b in self.module.buffers()]\n if len(buffers) > 0:\n self._dist_broadcast_coalesced(buffers,\n self.broadcast_bucket_size)\n\n def scatter(self, inputs, kwargs, device_ids):\n return scatter_kwargs(inputs, kwargs, device_ids, dim=self.dim)\n\n def forward(self, *inputs, **kwargs):\n inputs, kwargs = self.scatter(inputs, kwargs,\n [torch.cuda.current_device()])\n return self.module(*inputs[0], **kwargs[0])\n\n def train_step(self, *inputs, **kwargs):\n inputs, kwargs = self.scatter(inputs, kwargs,\n [torch.cuda.current_device()])\n output = self.module.train_step(*inputs[0], **kwargs[0])\n return output\n\n def val_step(self, *inputs, **kwargs):\n inputs, kwargs = self.scatter(inputs, kwargs,\n [torch.cuda.current_device()])\n output = self.module.val_step(*inputs[0], **kwargs[0])\n return output\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/registry.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom torch.nn.parallel import DataParallel, DistributedDataParallel\n\nfrom mmcv.utils import Registry\n\nMODULE_WRAPPERS = Registry('module wrapper')\nMODULE_WRAPPERS.register_module(module=DataParallel)\nMODULE_WRAPPERS.register_module(module=DistributedDataParallel)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/scatter_gather.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nfrom torch.nn.parallel._functions import Scatter as OrigScatter\n\nfrom ._functions import Scatter\nfrom .data_container import DataContainer\n\n\ndef scatter(inputs, target_gpus, dim=0):\n \"\"\"Scatter inputs to target gpus.\n\n The only difference from original :func:`scatter` is to add support for\n :type:`~mmcv.parallel.DataContainer`.\n \"\"\"\n\n def scatter_map(obj):\n if isinstance(obj, torch.Tensor):\n if target_gpus != [-1]:\n return OrigScatter.apply(target_gpus, None, dim, obj)\n else:\n # for CPU inference we use self-implemented scatter\n return Scatter.forward(target_gpus, obj)\n if isinstance(obj, DataContainer):\n if obj.cpu_only:\n return obj.data\n else:\n return Scatter.forward(target_gpus, obj.data)\n if isinstance(obj, tuple) and len(obj) > 0:\n return list(zip(*map(scatter_map, obj)))\n if isinstance(obj, list) and len(obj) > 0:\n out = list(map(list, zip(*map(scatter_map, obj))))\n return out\n if isinstance(obj, dict) and len(obj) > 0:\n out = list(map(type(obj), zip(*map(scatter_map, obj.items()))))\n return out\n return [obj for targets in target_gpus]\n\n # After scatter_map is called, a scatter_map cell will exist. This cell\n # has a reference to the actual function scatter_map, which has references\n # to a closure that has a reference to the scatter_map cell (because the\n # fn is recursive). To avoid this reference cycle, we set the function to\n # None, clearing the cell\n try:\n return scatter_map(inputs)\n finally:\n scatter_map = None\n\n\ndef scatter_kwargs(inputs, kwargs, target_gpus, dim=0):\n \"\"\"Scatter with support for kwargs dictionary.\"\"\"\n inputs = scatter(inputs, target_gpus, dim) if inputs else []\n kwargs = scatter(kwargs, target_gpus, dim) if kwargs else []\n if len(inputs) < len(kwargs):\n inputs.extend([() for _ in range(len(kwargs) - len(inputs))])\n elif len(kwargs) < len(inputs):\n kwargs.extend([{} for _ in range(len(inputs) - len(kwargs))])\n inputs = tuple(inputs)\n kwargs = tuple(kwargs)\n return inputs, kwargs\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/parallel/utils.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .registry import MODULE_WRAPPERS\n\n\ndef is_module_wrapper(module):\n \"\"\"Check if a module is a module wrapper.\n\n The following 3 modules in MMCV (and their subclasses) are regarded as\n module wrappers: DataParallel, DistributedDataParallel,\n MMDistributedDataParallel (the deprecated version). You may add you own\n module wrapper by registering it to mmcv.parallel.MODULE_WRAPPERS.\n\n Args:\n module (nn.Module): The module to be checked.\n\n Returns:\n bool: True if the input module is a module wrapper.\n \"\"\"\n module_wrappers = tuple(MODULE_WRAPPERS.module_dict.values())\n return isinstance(module, module_wrappers)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/readme.md", "content": "test\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .base_module import BaseModel, BaseModule, ModuleDict, ModuleList, Sequential, BaseBackbone, BaseNecks, BaseLosses, BaseNecksV2\nfrom .base_runner import BaseRunner\nfrom .builder import RUNNERS, build_runner\nfrom .checkpoint import (CheckpointLoader, _load_checkpoint,\n _load_checkpoint_with_prefix, load_checkpoint,\n load_state_dict, save_checkpoint, weights_to_cpu)\nfrom .default_constructor import DefaultRunnerConstructor\nfrom .dist_utils import (allreduce_grads, allreduce_params, get_dist_info,\n init_dist, master_only)\nfrom .epoch_based_runner import EpochBasedRunner, Runner\nfrom .fp16_utils import LossScaler, auto_fp16, force_fp32, wrap_fp16_model\nfrom .hooks import (HOOKS, CheckpointHook, ClosureHook, DistEvalHook,\n DistSamplerSeedHook, DvcliveLoggerHook, EMAHook, EvalHook,\n Fp16OptimizerHook, GradientCumulativeFp16OptimizerHook,\n GradientCumulativeOptimizerHook, Hook, IterTimerHook,\n LoggerHook, MlflowLoggerHook, NeptuneLoggerHook,\n OptimizerHook, PaviLoggerHook, SyncBuffersHook,\n TensorboardLoggerHook, TextLoggerHook, WandbLoggerHook)\nfrom .hooks.lr_updater import StepLrUpdaterHook # noqa\nfrom .hooks.lr_updater import (CosineAnnealingLrUpdaterHook,\n CosineRestartLrUpdaterHook, CyclicLrUpdaterHook,\n ExpLrUpdaterHook, FixedLrUpdaterHook,\n FlatCosineAnnealingLrUpdaterHook,\n InvLrUpdaterHook, LrUpdaterHook,\n OneCycleLrUpdaterHook, PolyLrUpdaterHook)\nfrom .hooks.momentum_updater import (CosineAnnealingMomentumUpdaterHook,\n CyclicMomentumUpdaterHook,\n MomentumUpdaterHook,\n OneCycleMomentumUpdaterHook,\n StepMomentumUpdaterHook)\nfrom .iter_based_runner import IterBasedRunner, IterLoader\nfrom .log_buffer import LogBuffer\nfrom .optimizer import (OPTIMIZER_BUILDERS, OPTIMIZERS,\n DefaultOptimizerConstructor, build_optimizer,\n build_optimizer_constructor)\nfrom .priority import Priority, get_priority\nfrom .utils import get_host_info, get_time_str, obj_from_dict, set_random_seed\nfrom .record import MetricLogger\nfrom .hooks.nni_hook import NNIHook\nfrom .misc import find_latest_checkpoint\n\n__all__ = [\n 'BaseRunner', 'Runner', 'EpochBasedRunner', 'IterBasedRunner', 'LogBuffer',\n 'HOOKS', 'Hook', 'CheckpointHook', 'ClosureHook', 'LrUpdaterHook',\n 'FixedLrUpdaterHook', 'StepLrUpdaterHook', 'ExpLrUpdaterHook',\n 'PolyLrUpdaterHook', 'InvLrUpdaterHook', 'CosineAnnealingLrUpdaterHook',\n 'FlatCosineAnnealingLrUpdaterHook', 'CosineRestartLrUpdaterHook',\n 'CyclicLrUpdaterHook', 'OneCycleLrUpdaterHook', 'MomentumUpdaterHook',\n 'StepMomentumUpdaterHook', 'CosineAnnealingMomentumUpdaterHook',\n 'CyclicMomentumUpdaterHook', 'OneCycleMomentumUpdaterHook',\n 'OptimizerHook', 'IterTimerHook', 'DistSamplerSeedHook', 'LoggerHook',\n 'PaviLoggerHook', 'TextLoggerHook', 'TensorboardLoggerHook',\n 'NeptuneLoggerHook', 'WandbLoggerHook', 'MlflowLoggerHook',\n 'DvcliveLoggerHook', '_load_checkpoint', 'load_state_dict',\n 'load_checkpoint', 'weights_to_cpu', 'save_checkpoint', 'Priority',\n 'get_priority', 'get_host_info', 'get_time_str', 'obj_from_dict',\n 'init_dist', 'get_dist_info', 'master_only', 'OPTIMIZER_BUILDERS',\n 'OPTIMIZERS', 'DefaultOptimizerConstructor', 'build_optimizer',\n 'build_optimizer_constructor', 'IterLoader', 'set_random_seed',\n 'auto_fp16', 'force_fp32', 'wrap_fp16_model', 'Fp16OptimizerHook',\n 'SyncBuffersHook', 'EMAHook', 'build_runner', 'RUNNERS', 'allreduce_grads',\n 'allreduce_params', 'LossScaler', 'CheckpointLoader', 'BaseModule', 'BaseBackbone', 'BaseNecks', 'BaseLosses', 'BaseNecksV2',\n '_load_checkpoint_with_prefix', 'EvalHook', 'DistEvalHook', 'Sequential',\n 'ModuleDict', 'ModuleList', 'GradientCumulativeOptimizerHook',\n 'GradientCumulativeFp16OptimizerHook', 'DefaultRunnerConstructor', 'find_latest_checkpoint'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/base_module.py", "content": " # Copyright (c) OpenMMLab. All rights reserved.\nimport copy\nimport warnings\nfrom abc import ABCMeta\nfrom collections import defaultdict\nfrom logging import FileHandler\n\nimport torch.nn as nn\n\nfrom mmcv.runner.dist_utils import master_only\nfrom mmcv.utils.logging import print_log, get_logger, logger_initialized\n\nclass BaseModel(nn.Module):\n _task = {}\n\n def __init_subclass__(cls, name='', **kwargs):\n if name != '':\n # if name in cls._taskhead.keys():\n # raise ValueError(f'Got name={name} existed'\n # f'in{cls._taskhead.keys()}')\n # else:\n cls._task[name] = cls\n cls._name = name\n else:\n # if cls.__name__ in cls._taskhead.keys():\n # raise ValueError(f'Got cls.__name__={cls.__name__} existed '\n # f'in{cls._taskhead.keys()}')\n # else:\n # warnings.warn(f'Creating a subclass of MetaModel {cls.__name__} with no name.')\n cls._task[cls.__name__] = cls\n cls._name = cls.__name__\n\n @classmethod\n def build_model(cls, *args, **kwargs):\n\n # if cls is StreroSRModel:\n model = kwargs.pop('model')\n try:\n cls = cls._models[model]\n # print(cls)\n except KeyError:\n raise ValueError(f'Got model={model} but expected '\n f'one of {cls._models.keys()}')\n\n return cls(None, None)\n\n\n @classmethod\n def new(cls, *args, **kwargs):\n task = kwargs.pop('task')\n try:\n cls = cls._task[task]\n except KeyError:\n raise ValueError(f'Got task={task} but expected '\n f'one of {cls._task.keys()}')\n\n return cls(*args, **kwargs)\n\nclass BaseModule(BaseModel, name='BaseModule'):#nn.Module, metaclass=ABCMeta\n \"\"\"Base module for all modules in openmmlab.\n\n ``BaseModule`` is a wrapper of ``torch.nn.Module`` with additional\n functionality of parameter initialization. Compared with\n ``torch.nn.Module``, ``BaseModule`` mainly adds three attributes.\n\n - ``init_cfg``: the config to control the initialization.\n - ``init_weights``: The function of parameter initialization and recording\n initialization information.\n - ``_params_init_info``: Used to track the parameter initialization\n information. This attribute only exists during executing the\n ``init_weights``.\n\n Args:\n init_cfg (dict, optional): Initialization config dict.\n \"\"\"\n # _task = {}\n #\n # def __init_subclass__(cls, name='', **kwargs):\n # if name != '':\n # # if name in cls._taskhead.keys():\n # # raise ValueError(f'Got name={name} existed'\n # # f'in{cls._taskhead.keys()}')\n # # else:\n # cls._task[name] = cls\n # cls._name = name\n # else:\n # # if cls.__name__ in cls._taskhead.keys():\n # # raise ValueError(f'Got cls.__name__={cls.__name__} existed '\n # # f'in{cls._taskhead.keys()}')\n # # else:\n # # warnings.warn(f'Creating a subclass of MetaModel {cls.__name__} with no name.')\n # cls._task[cls.__name__] = cls\n # cls._name = cls.__name__\n #\n #\n # @classmethod\n # def new(cls, *args, **kwargs):\n # task = kwargs.pop('task')\n # try:\n # cls = cls._task[task]\n # except KeyError:\n # raise ValueError(f'Got task={task} but expected '\n # f'one of {cls._task.keys()}')\n #\n # return cls\n\n def __init__(self, init_cfg=None):\n \"\"\"Initialize BaseModule, inherited from `torch.nn.Module`\"\"\"\n\n # NOTE init_cfg can be defined in different levels, but init_cfg\n # in low levels has a higher priority.\n\n super(BaseModule, self).__init__()\n # define default value of init_cfg instead of hard code\n # in init_weights() function\n self._is_init = False\n\n self.init_cfg = copy.deepcopy(init_cfg)\n\n # Backward compatibility in derived classes\n # if pretrained is not None:\n # warnings.warn('DeprecationWarning: pretrained is a deprecated \\\n # key, please consider using init_cfg')\n # self.init_cfg = dict(type='Pretrained', checkpoint=pretrained)\n\n @property\n def is_init(self):\n return self._is_init\n\n def init_weights(self):\n \"\"\"Initialize the weights.\"\"\"\n\n is_top_level_module = False\n # check if it is top-level module\n if not hasattr(self, '_params_init_info'):\n # The `_params_init_info` is used to record the initialization\n # information of the parameters\n # the key should be the obj:`nn.Parameter` of model and the value\n # should be a dict containing\n # - init_info (str): The string that describes the initialization.\n # - tmp_mean_value (FloatTensor): The mean of the parameter,\n # which indicates whether the parameter has been modified.\n # this attribute would be deleted after all parameters\n # is initialized.\n self._params_init_info = defaultdict(dict)\n is_top_level_module = True\n\n # Initialize the `_params_init_info`,\n # When detecting the `tmp_mean_value` of\n # the corresponding parameter is changed, update related\n # initialization information\n for name, param in self.named_parameters():\n self._params_init_info[param][\n 'init_info'] = f'The value is the same before and ' \\\n f'after calling `init_weights` ' \\\n f'of {self.__class__.__name__} '\n self._params_init_info[param][\n 'tmp_mean_value'] = param.data.mean()\n\n # pass `params_init_info` to all submodules\n # All submodules share the same `params_init_info`,\n # so it will be updated when parameters are\n # modified at any level of the model.\n for sub_module in self.modules():\n sub_module._params_init_info = self._params_init_info\n\n # Get the initialized logger, if not exist,\n # create a logger named `mmcv`\n logger_names = list(logger_initialized.keys())\n logger_name = logger_names[0] if logger_names else 'mmcv'\n\n from ..cnn import initialize\n from ..cnn.utils.weight_init import update_init_info\n module_name = self.__class__.__name__\n if not self._is_init:\n if self.init_cfg:\n print_log(\n f'initialize {module_name} with init_cfg {self.init_cfg}',\n logger=logger_name)\n initialize(self, self.init_cfg)\n if isinstance(self.init_cfg, dict):\n # prevent the parameters of\n # the pre-trained model\n # from being overwritten by\n # the `init_weights`\n if self.init_cfg['type'] == 'Pretrained':\n return\n\n for m in self.children():\n if hasattr(m, 'init_weights'):\n m.init_weights()\n # users may overload the `init_weights`\n update_init_info(\n m,\n init_info=f'Initialized by '\n f'user-defined `init_weights`'\n f' in {m.__class__.__name__} ')\n\n self._is_init = True\n else:\n warnings.warn(f'init_weights of {self.__class__.__name__} has '\n f'been called more than once.')\n\n if is_top_level_module:\n self._dump_init_info(logger_name)\n\n for sub_module in self.modules():\n del sub_module._params_init_info\n\n @master_only\n def _dump_init_info(self, logger_name):\n \"\"\"Dump the initialization information to a file named\n `initialization.log.json` in workdir.\n\n Args:\n logger_name (str): The name of logger.\n \"\"\"\n\n logger = get_logger(logger_name)\n\n with_file_handler = False\n # dump the information to the logger file if there is a `FileHandler`\n for handler in logger.handlers:\n if isinstance(handler, FileHandler):\n handler.stream.write(\n 'Name of parameter - Initialization information\\n')\n for name, param in self.named_parameters():\n handler.stream.write(\n f'\\n{name} - {param.shape}: '\n f\"\\n{self._params_init_info[param]['init_info']} \\n\")\n handler.stream.flush()\n with_file_handler = True\n if not with_file_handler:\n for name, param in self.named_parameters():\n print_log(\n f'\\n{name} - {param.shape}: '\n f\"\\n{self._params_init_info[param]['init_info']} \\n \",\n logger=logger_name)\n\n def __repr__(self):\n s = super().__repr__()\n if self.init_cfg:\n s += f'\\ninit_cfg={self.init_cfg}'\n return s\n\n\nclass Sequential(BaseModule, nn.Sequential, name='Sequential'):\n \"\"\"Sequential module in openmmlab.\n\n Args:\n init_cfg (dict, optional): Initialization config dict.\n \"\"\"\n\n def __init__(self, *args, init_cfg=None):\n BaseModule.__init__(self, init_cfg)\n nn.Sequential.__init__(self, *args)\n\n\nclass ModuleList(BaseModule, nn.ModuleList, name='ModuleList'):\n \"\"\"ModuleList in openmmlab.\n\n Args:\n modules (iterable, optional): an iterable of modules to add.\n init_cfg (dict, optional): Initialization config dict.\n \"\"\"\n\n def __init__(self, modules=None, init_cfg=None):\n BaseModule.__init__(self, init_cfg)\n nn.ModuleList.__init__(self, modules)\n\n\nclass ModuleDict(BaseModule, nn.ModuleDict, name='ModuleDict'):\n \"\"\"ModuleDict in openmmlab.\n\n Args:\n modules (dict, optional): a mapping (dictionary) of (string: module)\n or an iterable of key-value pairs of type (string, module).\n init_cfg (dict, optional): Initialization config dict.\n \"\"\"\n\n def __init__(self, modules=None, init_cfg=None):\n BaseModule.__init__(self, init_cfg)\n nn.ModuleDict.__init__(self, modules)\n\n\nclass BaseBackbone(BaseModule, name='BaseBackbone'):\n _models = {}\n\n def __init_subclass__(cls, name='', **kwargs):\n if name != '':\n # if name in cls._models.keys():\n # raise ValueError(f'Got name={name} existed'\n # f'in{cls._models.keys()}')\n # else:\n cls._models[name] = cls\n cls._name = name\n else:\n # if cls.__name__ in cls._models.keys():\n # raise ValueError(f'Got cls.__name__={cls.__name__} existed'\n # f'in{cls._models.keys()}')\n # else:\n # warnings.warn(f'Creating a subclass of MetaModel {cls.__name__} with no name.')\n cls._models[cls.__name__] = cls\n cls._name = cls.__name__\n\n\n @classmethod\n def build_model(cls, *args, **kwargs):\n model = kwargs.pop('model')\n try:\n cls = cls._models[model]\n except KeyError:\n raise ValueError(f'Got models={model} but expected '\n f'one of {cls._models.keys()}')\n\n return cls\n\nclass BaseLosses(nn.Module):\n _models = {}\n\n def __init_subclass__(cls, name='', **kwargs):\n\n # print(name, cls)\n if name != '':\n # if name in cls._models.keys():\n # raise ValueError(f'Got name={name} existed'\n # f'in{cls._models.keys()}')\n # else:\n cls._models[name] = cls\n cls._name = name\n else:\n # if cls.__name__ in cls._models.keys():\n # raise ValueError(f'Got cls.__name__={cls.__name__} existed'\n # f'in{cls._models.keys()}')\n # else:\n # warnings.warn(f'Creating a subclass of MetaModel {cls.__name__} with no name.')\n cls._models[cls.__name__] = cls\n cls._name = cls.__name__\n\n @classmethod\n def build_model(cls, *args, **kwargs):\n model = kwargs.pop('model')\n try:\n cls = cls._models[model]\n except KeyError:\n raise ValueError(f'Got models={model} but expected '\n f'one of {cls._models.keys()}')\n\n return cls\n\nclass BaseNecks(nn.Module):\n _models = {}\n\n def __init_subclass__(cls, name='', **kwargs):\n # print(name, cls)\n if name != '':\n # if name in cls._models.keys():\n # raise ValueError(f'Got name={name} existed'\n # f'in{cls._models.keys()}')\n # else:\n cls._models[name] = cls\n cls._name = name\n else:\n # if cls.__name__ in cls._models.keys():\n # raise ValueError(f'Got cls.__name__={cls.__name__} existed'\n # f'in{cls._models.keys()}')\n # else:\n # warnings.warn(f'Creating a subclass of MetaModel {cls.__name__} with no name.')\n cls._models[cls.__name__] = cls\n cls._name = cls.__name__\n\n @classmethod\n def build_model(cls, *args, **kwargs):\n model = kwargs.pop('model')\n try:\n cls = cls._models[model]\n except KeyError:\n raise ValueError(f'Got models={model} but expected '\n f'one of {cls._models.keys()}')\n\n return cls\n\nclass BaseNecksV2(BaseModule, BaseNecks, name='BaseNecksV2'):\n '''\n 父类的_models, __init_subclass__都会被继承\n '''\n ...\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/base_runner.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport copy\nimport logging\nimport os.path as osp\nimport warnings\nfrom abc import ABCMeta, abstractmethod\n\nimport torch\nfrom torch.optim import Optimizer\n\nimport mmcv\nfrom ..parallel import is_module_wrapper\nfrom .checkpoint import load_checkpoint\nfrom .dist_utils import get_dist_info\nfrom .hooks import HOOKS, Hook\nfrom .log_buffer import LogBuffer\nfrom .priority import Priority, get_priority\nfrom .utils import get_time_str\nfrom .record import MetricLogger\n\n\nclass BaseRunner(metaclass=ABCMeta):\n \"\"\"The base class of Runner, a training helper for PyTorch.\n\n All subclasses should implement the following APIs:\n\n - ``run()``\n - ``train()``\n - ``val()``\n - ``save_checkpoint()``\n\n Args:\n model (:obj:`torch.nn.Module`): The model to be run.\n batch_processor (callable): A callable method that process a data\n batch. The interface of this method should be\n `batch_processor(model, data, train_mode) -> dict`\n optimizer (dict or :obj:`torch.optim.Optimizer`): It can be either an\n optimizer (in most cases) or a dict of optimizers (in models that\n requires more than one optimizer, e.g., GAN).\n work_dir (str, optional): The working directory to save checkpoints\n and logs. Defaults to None.\n logger (:obj:`logging.Logger`): Logger used during training.\n Defaults to None. (The default value is just for backward\n compatibility)\n meta (dict | None): A dict records some import information such as\n environment info and seed, which will be logged in logger hook.\n Defaults to None.\n max_epochs (int, optional): Total training epochs.\n max_iters (int, optional): Total training iterations.\n \"\"\"\n\n def __init__(self,\n model,\n batch_processor=None,\n optimizer=None,\n work_dir=None,\n logger=None,\n meta=None,\n max_iters=None,\n max_epochs=None,\n opt_cfg=None):\n if batch_processor is not None:\n if not callable(batch_processor):\n raise TypeError('batch_processor must be callable, '\n f'but got {type(batch_processor)}')\n warnings.warn(\n 'batch_processor is deprecated, please implement '\n 'train_step() and val_step() in the model instead.',\n DeprecationWarning)\n # raise an error is `batch_processor` is not None and\n # `model.train_step()` exists.\n if is_module_wrapper(model):\n _model = model.module\n else:\n _model = model\n if hasattr(_model, 'train_step') or hasattr(_model, 'val_step'):\n raise RuntimeError(\n 'batch_processor and model.train_step()/model.val_step() '\n 'cannot be both available.')\n # else:\n # assert hasattr(model, 'train_step')\n\n # check the type of `optimizer`\n if isinstance(optimizer, dict):\n for name, optim in optimizer.items():\n if not isinstance(optim, Optimizer):\n raise TypeError(\n f'optimizer must be a dict of torch.optim.Optimizers, '\n f'but optimizer[\"{name}\"] is a {type(optim)}')\n elif not isinstance(optimizer, Optimizer) and optimizer is not None:\n raise TypeError(\n f'optimizer must be a torch.optim.Optimizer object '\n f'or dict or None, but got {type(optimizer)}')\n\n # check the type of `logger`\n if not isinstance(logger, logging.Logger):\n raise TypeError(f'logger must be a logging.Logger object, '\n f'but got {type(logger)}')\n\n # check the type of `meta`\n if meta is not None and not isinstance(meta, dict):\n raise TypeError(\n f'meta must be a dict or None, but got {type(meta)}')\n\n self.model = model\n self.batch_processor = batch_processor\n self.optimizer = optimizer\n self.logger = logger\n self.meta = meta\n self.opt_cfg = opt_cfg\n self.earlyStop = False\n # create work_dir\n save_dir = opt_cfg['save_dir']\n if mmcv.is_str(work_dir):\n self.work_dir = osp.abspath(work_dir)\n self.save_dir = osp.abspath(save_dir)\n mmcv.mkdir_or_exist(self.save_dir)\n mmcv.mkdir_or_exist(self.work_dir)\n elif work_dir is None:\n self.work_dir = None\n self.save_dir = None\n else:\n raise TypeError(f'\"work_dir: {work_dir}\" must be a str or None')\n\n\n\n\n # get model name from the model class\n if hasattr(self.model, 'module'):\n self._model_name = self.model.module.__class__.__name__\n else:\n self._model_name = self.model.__class__.__name__\n\n self._rank, self._world_size = get_dist_info()\n self.timestamp = get_time_str()\n self.mode = None\n self._hooks = []\n self._epoch = 0\n self._iter = 0\n self._inner_iter = 0\n self.outputs = {}\n\n if max_epochs is not None and max_iters is not None:\n raise ValueError(\n 'Only one of `max_epochs` or `max_iters` can be set.')\n\n self._max_epochs = max_epochs\n self._max_iters = max_iters\n # TODO: Redesign LogBuffer, it is not flexible and elegant enough\n self.log_buffer = MetricLogger(logger=logger, delimiter=\" \") # LogBuffer()\n\n @property\n def model_name(self):\n \"\"\"str: Name of the model, usually the module class name.\"\"\"\n return self._model_name\n\n @property\n def rank(self):\n \"\"\"int: Rank of current process. (distributed training)\"\"\"\n return self._rank\n\n @property\n def world_size(self):\n \"\"\"int: Number of processes participating in the job.\n (distributed training)\"\"\"\n return self._world_size\n\n @property\n def hooks(self):\n \"\"\"list[:obj:`Hook`]: A list of registered hooks.\"\"\"\n return self._hooks\n\n @property\n def epoch(self):\n \"\"\"int: Current epoch.\"\"\"\n return self._epoch\n\n @property\n def iter(self):\n \"\"\"int: Current iteration.\"\"\"\n return self._iter\n\n @property\n def inner_iter(self):\n \"\"\"int: Iteration in an epoch.\"\"\"\n return self._inner_iter\n\n @property\n def max_epochs(self):\n \"\"\"int: Maximum training epochs.\"\"\"\n return self._max_epochs\n\n @property\n def max_iters(self):\n \"\"\"int: Maximum training iterations.\"\"\"\n return self._max_iters\n\n @abstractmethod\n def train(self):\n pass\n\n @abstractmethod\n def val(self):\n pass\n\n @abstractmethod\n def run(self, data_loaders, workflow, **kwargs):\n pass\n\n @abstractmethod\n def save_checkpoint(self,\n out_dir,\n filename_tmpl,\n save_optimizer=True,\n meta=None,\n create_symlink=True):\n pass\n\n def current_lr(self):\n \"\"\"Get current learning rates.\n\n Returns:\n list[float] | dict[str, list[float]]: Current learning rates of all\n param groups. If the runner has a dict of optimizers, this method\n will return a dict.\n \"\"\"\n if isinstance(self.optimizer, torch.optim.Optimizer):\n lr = [group['lr'] for group in self.optimizer.param_groups]\n elif isinstance(self.optimizer, dict):\n lr = dict()\n for name, optim in self.optimizer.items():\n lr[name] = [group['lr'] for group in optim.param_groups]\n else:\n raise RuntimeError(\n 'lr is not applicable because optimizer does not exist.')\n return lr\n\n def current_momentum(self):\n \"\"\"Get current momentums.\n\n Returns:\n list[float] | dict[str, list[float]]: Current momentums of all\n param groups. If the runner has a dict of optimizers, this method\n will return a dict.\n \"\"\"\n\n def _get_momentum(optimizer):\n momentums = []\n for group in optimizer.param_groups:\n if 'momentum' in group.keys():\n momentums.append(group['momentum'])\n elif 'betas' in group.keys():\n momentums.append(group['betas'][0])\n else:\n momentums.append(0)\n return momentums\n\n if self.optimizer is None:\n raise RuntimeError(\n 'momentum is not applicable because optimizer does not exist.')\n elif isinstance(self.optimizer, torch.optim.Optimizer):\n momentums = _get_momentum(self.optimizer)\n elif isinstance(self.optimizer, dict):\n momentums = dict()\n for name, optim in self.optimizer.items():\n momentums[name] = _get_momentum(optim)\n return momentums\n\n def register_hook(self, hook, priority='NORMAL'):\n \"\"\"Register a hook into the hook list.\n\n The hook will be inserted into a priority queue, with the specified\n priority (See :class:`Priority` for details of priorities).\n For hooks with the same priority, they will be triggered in the same\n order as they are registered.\n\n Args:\n hook (:obj:`Hook`): The hook to be registered.\n priority (int or str or :obj:`Priority`): Hook priority.\n Lower value means higher priority.\n \"\"\"\n assert isinstance(hook, Hook)\n if hasattr(hook, 'priority'):\n raise ValueError('\"priority\" is a reserved attribute for hooks')\n priority = get_priority(priority)\n hook.priority = priority\n # insert the hook to a sorted list\n inserted = False\n for i in range(len(self._hooks) - 1, -1, -1):\n if priority >= self._hooks[i].priority:\n self._hooks.insert(i + 1, hook)\n inserted = True\n break\n if not inserted:\n self._hooks.insert(0, hook)\n\n def register_hook_from_cfg(self, hook_cfg):\n \"\"\"Register a hook from its cfg.\n\n Args:\n hook_cfg (dict): Hook config. It should have at least keys 'type'\n and 'priority' indicating its type and priority.\n\n Note:\n The specific hook class to register should not use 'type' and\n 'priority' arguments during initialization.\n \"\"\"\n hook_cfg = hook_cfg.copy()\n priority = hook_cfg.pop('priority', 'NORMAL')\n hook = mmcv.build_from_cfg(hook_cfg, HOOKS)\n self.register_hook(hook, priority=priority)\n\n def call_hook(self, fn_name):\n \"\"\"Call all hooks.\n\n Args:\n fn_name (str): The function name in each hook to be called, such as\n \"before_train_epoch\".\n \"\"\"\n for hook in self._hooks:\n getattr(hook, fn_name)(self)\n\n def get_hook_info(self):\n # Get hooks info in each stage\n stage_hook_map = {stage: [] for stage in Hook.stages}\n for hook in self.hooks:\n try:\n priority = Priority(hook.priority).name\n except ValueError:\n priority = hook.priority\n classname = hook.__class__.__name__\n hook_info = f'({priority:<12}) {classname:<35}'\n for trigger_stage in hook.get_triggered_stages():\n stage_hook_map[trigger_stage].append(hook_info)\n\n stage_hook_infos = []\n for stage in Hook.stages:\n hook_infos = stage_hook_map[stage]\n if len(hook_infos) > 0:\n info = f'{stage}:\\n'\n info += '\\n'.join(hook_infos)\n info += '\\n -------------------- '\n stage_hook_infos.append(info)\n return '\\n'.join(stage_hook_infos)\n\n def load_checkpoint(self,\n filename,\n resume_mode,\n map_location='cpu',\n strict=False,\n revise_keys=[(r'^module.', '')]):\n\n return load_checkpoint(\n resume_mode,\n self.work_dir,\n self.model,\n filename,\n map_location,\n strict,\n self.logger,\n revise_keys=revise_keys)\n\n def resume(self,\n resume, resume_mode,\n reset_lr, lr,\n resume_optimizer=True,\n map_location='default'):\n\n if map_location == 'default':\n if torch.cuda.is_available():\n device_id = torch.cuda.current_device()\n checkpoint = self.load_checkpoint(\n resume, resume_mode,\n map_location=lambda storage, loc: storage.cuda(device_id))\n else:\n checkpoint = self.load_checkpoint(resume, resume_mode)\n else:\n checkpoint = self.load_checkpoint(\n resume, resume_mode, map_location=map_location)\n\n self._epoch = checkpoint['meta']['epoch']\n if self.opt_cfg['eval']:\n self._max_epochs = self._epoch\n self._iter = checkpoint['meta']['iter']\n if self.meta is None:\n self.meta = {}\n self.meta.setdefault('hook_msgs', {})\n # load `last_ckpt`, `best_score`, `best_ckpt`, etc. for hook messages\n self.meta['hook_msgs'].update(checkpoint['meta'].get('hook_msgs', {}))\n\n # Re-calculate the number of iterations when resuming\n # models with different number of GPUs\n if 'config' in checkpoint['meta']:\n config = mmcv.Config.fromstring(\n checkpoint['meta']['config'], file_format='.py')\n previous_gpu_ids = config.get('gpu_ids', None)\n if previous_gpu_ids and len(previous_gpu_ids) > 0 and len(\n previous_gpu_ids) != self.world_size:\n self._iter = int(self._iter * len(previous_gpu_ids) /\n self.world_size)\n self.logger.info('the iteration number is changed due to '\n 'change of GPU number')\n\n # resume meta information meta\n self.meta = checkpoint['meta']\n # if optimizer is not None:\n # if checkpoint.get('optimizer') is not None:\n # optimizer.load_state_dict(checkpoint['optimizer'])\n #\n # if lr > 0 and reset_lr:\n # for param_group in optimizer.param_groups:\n # param_group['lr'] = lr\n # print_log(\"loaded checkpoint.optimizer\")\n if 'optimizer' in checkpoint and resume_optimizer:\n if isinstance(self.optimizer, Optimizer):\n self.optimizer.load_state_dict(checkpoint['optimizer'])\n if lr > 0 and reset_lr:\n for param_group in self.optimizer.param_groups:\n param_group['lr'] = lr\n self.logger.info(\"loaded checkpoint.optimizer\")\n elif isinstance(self.optimizer, dict):\n for k in self.optimizer.keys():\n self.optimizer[k].load_state_dict(\n checkpoint['optimizer'][k])\n if lr > 0 and reset_lr:\n for param_group in self.optimizer[k].param_groups:\n param_group['lr'] = lr\n self.logger.info(\"loaded checkpoint.optimizer\")\n else:\n raise TypeError(\n 'Optimizer should be dict or torch.optim.Optimizer '\n f'but got {type(self.optimizer)}')\n\n self.logger.info('resumed epoch %d, iter %d', self.epoch, self.iter)\n\n def register_lr_hook(self, lr_config):\n if lr_config is None:\n return\n elif isinstance(lr_config, dict):\n assert 'policy' in lr_config\n policy_type = lr_config.pop('policy')\n # If the type of policy is all in lower case, e.g., 'cyclic',\n # then its first letter will be capitalized, e.g., to be 'Cyclic'.\n # This is for the convenient usage of Lr updater.\n # Since this is not applicable for `\n # CosineAnnealingLrUpdater`,\n # the string will not be changed if it contains capital letters.\n if policy_type == policy_type.lower():\n policy_type = policy_type.title()\n hook_type = policy_type + 'LrUpdaterHook'\n lr_config['type'] = hook_type\n hook = mmcv.build_from_cfg(lr_config, HOOKS)\n else:\n hook = lr_config\n self.register_hook(hook, priority='VERY_HIGH')\n\n def register_momentum_hook(self, momentum_config):\n if momentum_config is None:\n return\n if isinstance(momentum_config, dict):\n assert 'policy' in momentum_config\n policy_type = momentum_config.pop('policy')\n # If the type of policy is all in lower case, e.g., 'cyclic',\n # then its first letter will be capitalized, e.g., to be 'Cyclic'.\n # This is for the convenient usage of momentum updater.\n # Since this is not applicable for\n # `CosineAnnealingMomentumUpdater`,\n # the string will not be changed if it contains capital letters.\n if policy_type == policy_type.lower():\n policy_type = policy_type.title()\n hook_type = policy_type + 'MomentumUpdaterHook'\n momentum_config['type'] = hook_type\n hook = mmcv.build_from_cfg(momentum_config, HOOKS)\n else:\n hook = momentum_config\n self.register_hook(hook, priority='HIGH')\n\n def register_optimizer_hook(self, optimizer_config):\n if optimizer_config is None:\n return\n if isinstance(optimizer_config, dict):\n optimizer_config.setdefault('type', 'OptimizerHook')\n hook = mmcv.build_from_cfg(optimizer_config, HOOKS)\n else:\n hook = optimizer_config\n self.register_hook(hook, priority='ABOVE_NORMAL')\n\n def register_checkpoint_hook(self, checkpoint_config):\n if checkpoint_config is None:\n return\n if isinstance(checkpoint_config, dict):\n checkpoint_config.setdefault('type', 'CheckpointHook')\n hook = mmcv.build_from_cfg(checkpoint_config, HOOKS)\n else:\n hook = checkpoint_config\n self.register_hook(hook, priority='NORMAL')\n\n def register_logger_hooks(self, log_config):\n if log_config is None:\n return\n log_interval = log_config['interval']\n for info in log_config['hooks']:\n logger_hook = mmcv.build_from_cfg(\n info, HOOKS, default_args=dict(interval=log_interval))\n self.register_hook(logger_hook, priority='VERY_LOW')\n\n def register_timer_hook(self, timer_config):\n if timer_config is None:\n return\n if isinstance(timer_config, dict):\n timer_config_ = copy.deepcopy(timer_config)\n hook = mmcv.build_from_cfg(timer_config_, HOOKS)\n else:\n hook = timer_config\n self.register_hook(hook, priority='LOW')\n\n def register_custom_hooks(self, custom_config):\n if custom_config is None:\n return\n\n if not isinstance(custom_config, list):\n custom_config = [custom_config]\n\n for item in custom_config:\n if isinstance(item, dict):\n self.register_hook_from_cfg(item)\n else:\n self.register_hook(item, priority='NORMAL')\n\n def register_profiler_hook(self, profiler_config):\n if profiler_config is None:\n return\n if isinstance(profiler_config, dict):\n profiler_config.setdefault('type', 'ProfilerHook')\n hook = mmcv.build_from_cfg(profiler_config, HOOKS)\n else:\n hook = profiler_config\n self.register_hook(hook)\n\n def register_training_hooks(self,\n lr_config,\n optimizer_config=None,\n checkpoint_config=None,\n log_config=None,\n momentum_config=None,\n timer_config=dict(type='IterTimerHook'),\n custom_hooks_config=None):\n \"\"\"Register default and custom hooks for training.\n\n Default and custom hooks include:\n\n +----------------------+-------------------------+\n | Hooks | Priority |\n +======================+=========================+\n | LrUpdaterHook | VERY_HIGH (10) |\n +----------------------+-------------------------+\n | MomentumUpdaterHook | HIGH (30) |\n +----------------------+-------------------------+\n | OptimizerStepperHook | ABOVE_NORMAL (40) |\n +----------------------+-------------------------+\n | CheckpointSaverHook | NORMAL (50) |\n +----------------------+-------------------------+\n | IterTimerHook | LOW (70) |\n +----------------------+-------------------------+\n | LoggerHook(s) | VERY_LOW (90) |\n +----------------------+-------------------------+\n | CustomHook(s) | defaults to NORMAL (50) |\n +----------------------+-------------------------+\n\n If custom hooks have same priority with default hooks, custom hooks\n will be triggered after default hooks.\n \"\"\"\n self.register_lr_hook(lr_config)\n self.register_momentum_hook(momentum_config)\n self.register_optimizer_hook(optimizer_config)\n self.register_checkpoint_hook(checkpoint_config)\n self.register_timer_hook(timer_config)\n self.register_logger_hooks(log_config)\n self.register_custom_hooks(custom_hooks_config)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/builder.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport copy\n\nfrom ..utils import Registry\n\nRUNNERS = Registry('runner')\nRUNNER_BUILDERS = Registry('runner builder')\n\n\ndef build_runner_constructor(cfg):\n return RUNNER_BUILDERS.build(cfg)\n\n\ndef build_runner(cfg, default_args=None):\n runner_cfg = copy.deepcopy(cfg)\n constructor_type = runner_cfg.pop('constructor',\n 'DefaultRunnerConstructor')\n runner_constructor = build_runner_constructor(\n dict(\n type=constructor_type,\n runner_cfg=runner_cfg,\n default_args=default_args))\n runner = runner_constructor()\n return runner\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/checkpoint.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport io\nimport os\nimport os.path as osp\nimport pkgutil\nimport re\nimport time\nimport warnings\nfrom collections import OrderedDict\nfrom importlib import import_module\nfrom tempfile import TemporaryDirectory\nfrom glob import glob\nimport torch\nimport torchvision\nfrom torch.optim import Optimizer\n\nimport mmcv\nfrom ..fileio import FileClient\nfrom ..fileio import load as load_file\nfrom ..parallel import is_module_wrapper\nfrom ..utils import load_url, mkdir_or_exist, print_log\nfrom .dist_utils import get_dist_info\n\nENV_MMCV_HOME = 'MMCV_HOME'\nENV_XDG_CACHE_HOME = 'XDG_CACHE_HOME'\nDEFAULT_CACHE_DIR = '~/.cache'\n\n\ndef _get_mmcv_home():\n mmcv_home = os.path.expanduser(\n os.getenv(\n ENV_MMCV_HOME,\n os.path.join(\n os.getenv(ENV_XDG_CACHE_HOME, DEFAULT_CACHE_DIR), 'mmcv')))\n\n mkdir_or_exist(mmcv_home)\n return mmcv_home\n\n\ndef load_state_dict(module, state_dict, strict=False, logger=None):\n \"\"\"Load state_dict to a module.\n\n This method is modified from :meth:`torch.nn.Module.load_state_dict`.\n Default value for ``strict`` is set to ``False`` and the message for\n param mismatch will be shown even if strict is False.\n\n Args:\n module (Module): Module that receives the state_dict.\n state_dict (OrderedDict): Weights.\n strict (bool): whether to strictly enforce that the keys\n in :attr:`state_dict` match the keys returned by this module's\n :meth:`~torch.nn.Module.state_dict` function. Default: ``False``.\n logger (:obj:`logging.Logger`, optional): Logger to log the error\n message. If not specified, print function will be used.\n \"\"\"\n unexpected_keys = []\n all_missing_keys = []\n err_msg = []\n if hasattr(module, 'train'):\n metadata = getattr(state_dict, '_metadata', None)\n state_dict = state_dict.copy()\n if metadata is not None:\n state_dict._metadata = metadata\n else:\n for name in state_dict.keys():\n metadata = getattr(state_dict[name], '_metadata', None)\n state_dict[name] = state_dict[name].copy()\n if metadata is not None:\n state_dict[name]._metadata = metadata\n\n\n\n # use _load_from_state_dict to enable checkpoint version control\n def load(module, prefix=''):\n # recursively check parallel module in case that the model has a\n # complicated structure, e.g., nn.Module(nn.Module(DDP))\n if not hasattr(module, '_load_from_state_dict'):\n for name, m in module.model.items():\n if is_module_wrapper(m):\n m = m.module\n local_metadata = {} if metadata is None else metadata.get(\n prefix[:-1], {})\n m._load_from_state_dict(state_dict[name], prefix, local_metadata, True,\n all_missing_keys, unexpected_keys,\n err_msg)\n for name, child in m._modules.items():\n if child is not None:\n load(child, prefix + name + '.')\n else:\n if is_module_wrapper(module):\n module = module.module\n local_metadata = {} if metadata is None else metadata.get(\n prefix[:-1], {})\n module._load_from_state_dict(state_dict, prefix, local_metadata, True,\n all_missing_keys, unexpected_keys,\n err_msg)\n for name, child in module._modules.items():\n if child is not None:\n load(child, prefix + name + '.')\n\n load(module)\n load = None # break load->load reference cycle\n\n # ignore \"num_batches_tracked\" of BN layers\n missing_keys = [\n key for key in all_missing_keys if 'num_batches_tracked' not in key\n ]\n\n if unexpected_keys:\n err_msg.append('unexpected key in source '\n f'state_dict: {\", \".join(unexpected_keys)}\\n')\n if missing_keys:\n err_msg.append(\n f'missing keys in source state_dict: {\", \".join(missing_keys)}\\n')\n\n rank, _ = get_dist_info()\n if len(err_msg) > 0 and rank == 0:\n err_msg.insert(\n 0, 'The model and loaded state dict do not match exactly\\n')\n err_msg = '\\n'.join(err_msg)\n if strict:\n raise RuntimeError(err_msg)\n elif logger is not None:\n logger.warning(err_msg)\n else:\n print(err_msg)\n\n\ndef get_torchvision_models():\n model_urls = dict()\n for _, name, ispkg in pkgutil.walk_packages(torchvision.models.__path__):\n if ispkg:\n continue\n _zoo = import_module(f'torchvision.models.{name}')\n if hasattr(_zoo, 'model_urls'):\n _urls = getattr(_zoo, 'model_urls')\n model_urls.update(_urls)\n return model_urls\n\n\ndef get_external_models():\n mmcv_home = _get_mmcv_home()\n default_json_path = osp.join(mmcv.__path__[0], 'model_zoo/open_mmlab.json')\n default_urls = load_file(default_json_path)\n assert isinstance(default_urls, dict)\n external_json_path = osp.join(mmcv_home, 'open_mmlab.json')\n if osp.exists(external_json_path):\n external_urls = load_file(external_json_path)\n assert isinstance(external_urls, dict)\n default_urls.update(external_urls)\n\n return default_urls\n\n\ndef get_mmcls_models():\n mmcls_json_path = osp.join(mmcv.__path__[0], 'model_zoo/mmcls.json')\n mmcls_urls = load_file(mmcls_json_path)\n\n return mmcls_urls\n\n\ndef get_deprecated_model_names():\n deprecate_json_path = osp.join(mmcv.__path__[0],\n 'model_zoo/deprecated.json')\n deprecate_urls = load_file(deprecate_json_path)\n assert isinstance(deprecate_urls, dict)\n\n return deprecate_urls\n\n\ndef _process_mmcls_checkpoint(checkpoint):\n if 'state_dict' in checkpoint:\n state_dict = checkpoint['state_dict']\n else:\n # Some checkpoints converted from 3rd-party repo don't\n # have the \"state_dict\" key.\n state_dict = checkpoint\n new_state_dict = OrderedDict()\n for k, v in state_dict.items():\n if k.startswith('backbone.'):\n new_state_dict[k[9:]] = v\n new_checkpoint = dict(state_dict=new_state_dict)\n\n return new_checkpoint\n\n\nclass CheckpointLoader:\n \"\"\"A general checkpoint loader to manage all schemes.\"\"\"\n\n _schemes = {}\n\n @classmethod\n def _register_scheme(cls, prefixes, loader, force=False):\n if isinstance(prefixes, str):\n prefixes = [prefixes]\n else:\n assert isinstance(prefixes, (list, tuple))\n for prefix in prefixes:\n if (prefix not in cls._schemes) or force:\n cls._schemes[prefix] = loader\n else:\n raise KeyError(\n f'{prefix} is already registered as a loader backend, '\n 'add \"force=True\" if you want to override it')\n # sort, longer prefixes take priority\n cls._schemes = OrderedDict(\n sorted(cls._schemes.items(), key=lambda t: t[0], reverse=True))\n\n @classmethod\n def register_scheme(cls, prefixes, loader=None, force=False):\n \"\"\"Register a loader to CheckpointLoader.\n\n This method can be used as a normal class method or a decorator.\n\n Args:\n prefixes (str or list[str] or tuple[str]):\n The prefix of the registered loader.\n loader (function, optional): The loader function to be registered.\n When this method is used as a decorator, loader is None.\n Defaults to None.\n force (bool, optional): Whether to override the loader\n if the prefix has already been registered. Defaults to False.\n \"\"\"\n\n if loader is not None:\n cls._register_scheme(prefixes, loader, force=force)\n return\n\n def _register(loader_cls):\n cls._register_scheme(prefixes, loader_cls, force=force)\n return loader_cls\n\n return _register\n\n @classmethod\n def _get_checkpoint_loader(cls, path):\n \"\"\"Finds a loader that supports the given path. Falls back to the local\n loader if no other loader is found.\n\n Args:\n path (str): checkpoint path\n\n Returns:\n callable: checkpoint loader\n \"\"\"\n for p in cls._schemes:\n # use regular match to handle some cases that where the prefix of\n # loader has a prefix. For example, both 's3://path' and\n # 'open-mmlab:s3://path' should return `load_from_ceph`\n if re.match(p, path) is not None:\n return cls._schemes[p]\n\n @classmethod\n def load_checkpoint(cls, filename, map_location=None, logger=None):\n \"\"\"load checkpoint through URL scheme path.\n\n Args:\n filename (str): checkpoint file name with given prefix\n map_location (str, optional): Same as :func:`torch.load`.\n Default: None\n logger (:mod:`logging.Logger`, optional): The logger for message.\n Default: None\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n\n checkpoint_loader = cls._get_checkpoint_loader(filename)\n class_name = checkpoint_loader.__name__\n print_log(\n f'load checkpoint from {class_name[10:]} path: {filename}', logger=logger)\n return checkpoint_loader(filename, map_location)\n\n\n@CheckpointLoader.register_scheme(prefixes='')\ndef load_from_local(filename, map_location):\n \"\"\"load checkpoint by local file path.\n\n Args:\n filename (str): local checkpoint file path\n map_location (str, optional): Same as :func:`torch.load`.\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n filename = osp.expanduser(filename)\n if not osp.isfile(filename):\n raise FileNotFoundError(f'{filename} can not be found.')\n checkpoint = torch.load(filename, map_location=map_location)\n return checkpoint\n\n\n@CheckpointLoader.register_scheme(prefixes=('http://', 'https://'))\ndef load_from_http(filename, map_location=None, model_dir=None):\n \"\"\"load checkpoint through HTTP or HTTPS scheme path. In distributed\n setting, this function only download checkpoint at local rank 0.\n\n Args:\n filename (str): checkpoint file path with modelzoo or\n torchvision prefix\n map_location (str, optional): Same as :func:`torch.load`.\n model_dir (string, optional): directory in which to save the object,\n Default: None\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n rank, world_size = get_dist_info()\n if rank == 0:\n checkpoint = load_url(\n filename, model_dir=model_dir, map_location=map_location)\n if world_size > 1:\n torch.distributed.barrier()\n if rank > 0:\n checkpoint = load_url(\n filename, model_dir=model_dir, map_location=map_location)\n return checkpoint\n\n\n@CheckpointLoader.register_scheme(prefixes='pavi://')\ndef load_from_pavi(filename, map_location=None):\n \"\"\"load checkpoint through the file path prefixed with pavi. In distributed\n setting, this function download ckpt at all ranks to different temporary\n directories.\n\n Args:\n filename (str): checkpoint file path with pavi prefix\n map_location (str, optional): Same as :func:`torch.load`.\n Default: None\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n assert filename.startswith('pavi://'), \\\n f'Expected filename startswith `pavi://`, but get {filename}'\n model_path = filename[7:]\n\n try:\n from pavi import modelcloud\n except ImportError:\n raise ImportError(\n 'Please install pavi to load checkpoint from modelcloud.')\n\n model = modelcloud.get(model_path)\n with TemporaryDirectory() as tmp_dir:\n downloaded_file = osp.join(tmp_dir, model.name)\n model.download(downloaded_file)\n checkpoint = torch.load(downloaded_file, map_location=map_location)\n return checkpoint\n\n\n@CheckpointLoader.register_scheme(prefixes=r'(\\S+\\:)?s3://')\ndef load_from_ceph(filename, map_location=None, backend='petrel'):\n \"\"\"load checkpoint through the file path prefixed with s3. In distributed\n setting, this function download ckpt at all ranks to different temporary\n directories.\n\n Note:\n Since v1.4.1, the registered scheme prefixes have been enhanced to\n support bucket names in the path prefix, e.g. 's3://xx.xx/xx.path',\n 'bucket1:s3://xx.xx/xx.path'.\n\n Args:\n filename (str): checkpoint file path with s3 prefix\n map_location (str, optional): Same as :func:`torch.load`.\n backend (str, optional): The storage backend type. Options are 'ceph',\n 'petrel'. Default: 'petrel'.\n\n .. warning::\n :class:`mmcv.fileio.file_client.CephBackend` will be deprecated,\n please use :class:`mmcv.fileio.file_client.PetrelBackend` instead.\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n allowed_backends = ['ceph', 'petrel']\n if backend not in allowed_backends:\n raise ValueError(f'Load from Backend {backend} is not supported.')\n\n if backend == 'ceph':\n warnings.warn(\n 'CephBackend will be deprecated, please use PetrelBackend instead',\n DeprecationWarning)\n\n # CephClient and PetrelBackend have the same prefix 's3://' and the latter\n # will be chosen as default. If PetrelBackend can not be instantiated\n # successfully, the CephClient will be chosen.\n try:\n file_client = FileClient(backend=backend)\n except ImportError:\n allowed_backends.remove(backend)\n file_client = FileClient(backend=allowed_backends[0])\n\n with io.BytesIO(file_client.get(filename)) as buffer:\n checkpoint = torch.load(buffer, map_location=map_location)\n return checkpoint\n\n\n@CheckpointLoader.register_scheme(prefixes=('modelzoo://', 'torchvision://'))\ndef load_from_torchvision(filename, map_location=None):\n \"\"\"load checkpoint through the file path prefixed with modelzoo or\n torchvision.\n\n Args:\n filename (str): checkpoint file path with modelzoo or\n torchvision prefix\n map_location (str, optional): Same as :func:`torch.load`.\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n model_urls = get_torchvision_models()\n if filename.startswith('modelzoo://'):\n warnings.warn(\n 'The URL scheme of \"modelzoo://\" is deprecated, please '\n 'use \"torchvision://\" instead', DeprecationWarning)\n model_name = filename[11:]\n else:\n model_name = filename[14:]\n return load_from_http(model_urls[model_name], map_location=map_location)\n\n\n@CheckpointLoader.register_scheme(prefixes=('open-mmlab://', 'openmmlab://'))\ndef load_from_openmmlab(filename, map_location=None):\n \"\"\"load checkpoint through the file path prefixed with open-mmlab or\n openmmlab.\n\n Args:\n filename (str): checkpoint file path with open-mmlab or\n openmmlab prefix\n map_location (str, optional): Same as :func:`torch.load`.\n Default: None\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n\n model_urls = get_external_models()\n prefix_str = 'open-mmlab://'\n if filename.startswith(prefix_str):\n model_name = filename[13:]\n else:\n model_name = filename[12:]\n prefix_str = 'openmmlab://'\n\n deprecated_urls = get_deprecated_model_names()\n if model_name in deprecated_urls:\n warnings.warn(\n f'{prefix_str}{model_name} is deprecated in favor '\n f'of {prefix_str}{deprecated_urls[model_name]}',\n DeprecationWarning)\n model_name = deprecated_urls[model_name]\n model_url = model_urls[model_name]\n # check if is url\n if model_url.startswith(('http://', 'https://')):\n checkpoint = load_from_http(model_url, map_location=map_location)\n else:\n filename = osp.join(_get_mmcv_home(), model_url)\n if not osp.isfile(filename):\n raise FileNotFoundError(f'{filename} can not be found.')\n checkpoint = torch.load(filename, map_location=map_location)\n return checkpoint\n\n\n@CheckpointLoader.register_scheme(prefixes='mmcls://')\ndef load_from_mmcls(filename, map_location=None):\n \"\"\"load checkpoint through the file path prefixed with mmcls.\n\n Args:\n filename (str): checkpoint file path with mmcls prefix\n map_location (str, optional): Same as :func:`torch.load`.\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n\n model_urls = get_mmcls_models()\n model_name = filename[8:]\n checkpoint = load_from_http(\n model_urls[model_name], map_location=map_location)\n checkpoint = _process_mmcls_checkpoint(checkpoint)\n return checkpoint\n\n\ndef _load_checkpoint(filename, map_location=None, logger=None):\n \"\"\"Load checkpoint from somewhere (modelzoo, file, url).\n\n Args:\n filename (str): Accept local filepath, URL, ``torchvision://xxx``,\n ``open-mmlab://xxx``. Please refer to ``docs/model_zoo.md`` for\n details.\n map_location (str, optional): Same as :func:`torch.load`.\n Default: None.\n logger (:mod:`logging.Logger`, optional): The logger for error message.\n Default: None\n\n Returns:\n dict or OrderedDict: The loaded checkpoint. It can be either an\n OrderedDict storing model weights or a dict containing other\n information, which depends on the checkpoint.\n \"\"\"\n return CheckpointLoader.load_checkpoint(filename, map_location, logger)\n\n\ndef _load_checkpoint_with_prefix(prefix, filename, map_location=None):\n \"\"\"Load partial pretrained model with specific prefix.\n\n Args:\n prefix (str): The prefix of sub-module.\n filename (str): Accept local filepath, URL, ``torchvision://xxx``,\n ``open-mmlab://xxx``. Please refer to ``docs/model_zoo.md`` for\n details.\n map_location (str | None): Same as :func:`torch.load`. Default: None.\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n\n checkpoint = _load_checkpoint(filename, map_location=map_location)\n\n if 'state_dict' in checkpoint:\n state_dict = checkpoint['state_dict']\n else:\n state_dict = checkpoint\n if not prefix.endswith('.'):\n prefix += '.'\n prefix_len = len(prefix)\n\n state_dict = {\n k[prefix_len:]: v\n for k, v in state_dict.items() if k.startswith(prefix)\n }\n\n assert state_dict, f'{prefix} is not in the pretrained model'\n return state_dict\n\n\ndef get_checkpoint_dir(OUT_DIR):\n \"\"\"Retrieves the location for storing checkpoints.\"\"\"\n return os.path.join(OUT_DIR)\n\n\ndef get_last_checkpoint(OUT_DIR, NAME_PREFIX=\"amp_model_best\", logger=None):\n \"\"\"Retrieves the most recent checkpoint (highest epoch number).\"\"\"\n checkpoint_dir = get_checkpoint_dir(OUT_DIR)\n checkpoints = glob(checkpoint_dir + f\"/{NAME_PREFIX}*\")\n if len(checkpoints) > 0:\n last_checkpoint_name = sorted(checkpoints)[-1]\n last_checkpoint = os.path.join(checkpoint_dir, last_checkpoint_name)\n print_log(f\"loading last_checkpoint file: {last_checkpoint}\", logger=logger)\n return last_checkpoint\n else:\n return None\n\n\ndef get_best_k_model(OUT_DIR, indicator, _NAME_PREFIX=\"ckpt_ep_\"):\n best_k_models = []\n best_fname = []\n if os.path.isfile(OUT_DIR):\n with open(OUT_DIR, 'r') as f:\n # TODO: 通常checkpoint不会很大，如果太大open方法不合适，因为我们需要的是最后几行，open是从头遍历的\n stats2user = [line.strip('\\n') for line in f.readlines()]\n\n for line in stats2user:\n metric = {}\n line = line.split(',')\n for v in line[1:]:\n v = re.sub(r\"[{}'' ]\", \"\", v)\n k, v = v.split(':')\n metric[str(k)] = v\n # fname, v = line.split('-')\n epoch = line[0].replace('.pth.tar', '')\n epoch = epoch.replace('model_best_', '')\n # best_k_models[str(epoch)] = float(v)\n\n # 第一个line[0]用于save_top_k触发时, 删除多余的模型\n # 第二个用于加载best模型, 因为best_k_models是个[[]],索引不方便\n best_k_models.append([epoch, metric, line[0]])\n best_fname.append(line[0])\n # best_k_models['epoch'].append(epoch)\n # best_k_models[indicator].append(float(v))\n\n if len(best_k_models) == 0:\n msg = f\"checkpoint in directory {OUT_DIR} don't exist or is empty\"\n warnings.warn(msg)\n\n return best_k_models, best_fname\n\n\ndef load_checkpoint(resume_mode,\n work_dir,\n model,\n filename,\n map_location=None,\n strict=False,\n logger=None,\n revise_keys=[(r'^module\\.', '')]):\n \"\"\"Load checkpoint from a file or URI.\n\n Args:\n model (Module): Module to load checkpoint.\n filename (str): Accept local filepath, URL, ``torchvision://xxx``,\n ``open-mmlab://xxx``. Please refer to ``docs/model_zoo.md`` for\n details.\n map_location (str): Same as :func:`torch.load`.\n strict (bool): Whether to allow different params for the model and\n checkpoint.\n logger (:mod:`logging.Logger` or None): The logger for error message.\n revise_keys (list): A list of customized keywords to modify the\n state_dict in checkpoint. Each item is a (pattern, replacement)\n pair of the regular expression operations. Default: strip\n the prefix 'module.' by [(r'^module\\\\.', '')].\n\n Returns:\n dict or OrderedDict: The loaded checkpoint.\n \"\"\"\n ################\n # 只从work_dir里读ckpt，用于模型的继续训练\n resume_mode = resume_mode.lower()\n if resume_mode == 'best':\n _, best_k_fname = get_best_k_model(os.path.join(work_dir, \"checkpoint\"), None)\n if len(best_k_fname) > 0:\n best_k_model = sorted(best_k_fname)[-1]\n filename = os.path.join(work_dir, best_k_model)\n else:\n print_log(\"loading best model failed, maybe it's from scratch currently.\", logger=logger)\n\n elif resume_mode == 'auto':\n ckpt = get_last_checkpoint(work_dir)\n if ckpt is not None:\n filename = ckpt\n ################\n if not os.path.isfile(filename):\n print_log(f\"no checkpoint found at {filename}\", logger=logger)\n return {'meta': {'epoch': 1,\n 'iter': 1,\n 'best_epoch': 1,\n 'best_metric': None}}\n ################\n\n checkpoint = _load_checkpoint(filename, map_location, logger)\n # OrderedDict is a subclass of dict\n if not isinstance(checkpoint, dict):\n raise RuntimeError(\n f'No state_dict found in checkpoint file {filename}')\n\n if 'meta' not in checkpoint.keys():\n checkpoint['meta'] = {}\n if hasattr(model, 'train'):\n mod = {'model': model}\n checkpoint = {'model': checkpoint}\n else:\n mod = model.model\n if isinstance(mod, dict):\n for name, m in mod.items():\n if 'state_dict' in checkpoint[name]:\n state_dict = checkpoint[name]['state_dict']\n else:\n state_dict = checkpoint[name]\n\n # strip prefix of state_dict\n metadata = getattr(state_dict, '_metadata', OrderedDict())\n for p, r in revise_keys:\n state_dict = OrderedDict(\n {re.sub(p, r, k): v\n for k, v in state_dict.items()})\n # Keep metadata in state_dict\n state_dict._metadata = metadata\n load_state_dict(m, state_dict, strict, logger)\n else:\n if 'state_dict' in checkpoint:\n state_dict = checkpoint['state_dict']\n else:\n state_dict = checkpoint\n\n # strip prefix of state_dict\n metadata = getattr(state_dict, '_metadata', OrderedDict())\n for p, r in revise_keys:\n state_dict = OrderedDict(\n {re.sub(p, r, k): v\n for k, v in state_dict.items()})\n # Keep metadata in state_dict\n state_dict._metadata = metadata\n load_state_dict(mod, state_dict, strict, logger)\n # if optimizer is not None:\n # if checkpoint.get('optimizer') is not None:\n # optimizer.load_state_dict(checkpoint['optimizer'])\n #\n # if lr > 0 and reset_lr:\n # for param_group in optimizer.param_groups:\n # param_group['lr'] = lr\n # print_log(\"loaded checkpoint.optimizer\")\n\n # load state_dict\n\n checkpoint['meta'].setdefault('epoch', 1)\n checkpoint['meta'].setdefault('iter', 1)\n checkpoint['meta'].setdefault('best_epoch', 1)\n checkpoint['meta'].setdefault('best_metric', None)\n\n return checkpoint\n\n\ndef weights_to_cpu(state_dict):\n \"\"\"Copy a model state_dict to cpu.\n\n Args:\n state_dict (OrderedDict): Model weights on GPU.\n\n Returns:\n OrderedDict: Model weights on GPU.\n \"\"\"\n state_dict_cpu = OrderedDict()\n for key, val in state_dict.items():\n state_dict_cpu[key] = val.cpu()\n # Keep metadata in state_dict\n state_dict_cpu._metadata = getattr(state_dict, '_metadata', OrderedDict())\n return state_dict_cpu\n\n\ndef _save_to_state_dict(module, destination, prefix, keep_vars):\n \"\"\"Saves module state to `destination` dictionary.\n\n This method is modified from :meth:`torch.nn.Module._save_to_state_dict`.\n\n Args:\n module (nn.Module): The module to generate state_dict.\n destination (dict): A dict where state will be stored.\n prefix (str): The prefix for parameters and buffers used in this\n module.\n \"\"\"\n for name, param in module._parameters.items():\n if param is not None:\n destination[prefix + name] = param if keep_vars else param.detach()\n for name, buf in module._buffers.items():\n # remove check of _non_persistent_buffers_set to allow nn.BatchNorm2d\n if buf is not None:\n destination[prefix + name] = buf if keep_vars else buf.detach()\n\n\ndef get_state_dict(module, destination=None, prefix='', keep_vars=False):\n \"\"\"Returns a dictionary containing a whole state of the module.\n\n Both parameters and persistent buffers (e.g. running averages) are\n included. Keys are corresponding parameter and buffer names.\n\n This method is modified from :meth:`torch.nn.Module.state_dict` to\n recursively check parallel module in case that the model has a complicated\n structure, e.g., nn.Module(nn.Module(DDP)).\n\n Args:\n module (nn.Module): The module to generate state_dict.\n destination (OrderedDict): Returned dict for the state of the\n module.\n prefix (str): Prefix of the key.\n keep_vars (bool): Whether to keep the variable property of the\n parameters. Default: False.\n\n Returns:\n dict: A dictionary containing a whole state of the module.\n \"\"\"\n # recursively check parallel module in case that the model has a\n # complicated structure, e.g., nn.Module(nn.Module(DDP))\n if is_module_wrapper(module):\n module = module.module\n\n # below is the same as torch.nn.Module.state_dict()\n if destination is None:\n destination = OrderedDict()\n destination._metadata = OrderedDict()\n destination._metadata[prefix[:-1]] = local_metadata = dict(\n version=module._version)\n _save_to_state_dict(module, destination, prefix, keep_vars)\n for name, child in module._modules.items():\n if child is not None:\n get_state_dict(\n child, destination, prefix + name + '.', keep_vars=keep_vars)\n for hook in module._state_dict_hooks.values():\n hook_result = hook(module, destination, prefix, local_metadata)\n if hook_result is not None:\n destination = hook_result\n return destination\n\n\ndef save_checkpoint(#model,\n filename,\n #optimizer=None,\n meta=None,\n file_client_args=None):\n \"\"\"Save checkpoint to file.\n\n The checkpoint will have 3 fields: ``meta``, ``state_dict`` and\n ``optimizer``. By default ``meta`` will contain version and time info.\n\n Args:\n model (Module): Module whose params are to be saved.\n filename (str): Checkpoint filename.\n optimizer (:obj:`Optimizer`, optional): Optimizer to be saved.\n meta (dict, optional): Metadata to be saved in checkpoint.\n file_client_args (dict, optional): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n `New in version 1.3.16.`\n \"\"\"\n if meta is None:\n meta = {}\n elif not isinstance(meta, dict):\n raise TypeError(f'meta must be a dict or None, but got {type(meta)}')\n # meta.update(mmcv_version=mmcv.__version__, time=time.asctime())\n if 'model' not in meta:\n checkpoint = {}\n for name, sub_meta in meta.items():\n\n model = sub_meta.pop('model')\n optimizer = sub_meta.pop('optimizer')\n if is_module_wrapper(model):\n model = model.module\n\n if hasattr(model, 'CLASSES') and model.CLASSES is not None:\n # save class name to the meta\n sub_meta.update(CLASSES=model.CLASSES)\n checkpoint[name] = {\n 'meta': sub_meta,\n 'state_dict': weights_to_cpu(get_state_dict(model))\n }\n\n # save optimizer state dict in the checkpoint\n if isinstance(optimizer, Optimizer):\n checkpoint[name]['optimizer'] = optimizer.state_dict()\n\n file_client = FileClient.infer_client(file_client_args, filename)\n with io.BytesIO() as f:\n torch.save(checkpoint, f)\n file_client.put(f.getvalue(), filename)\n\n else:\n model = meta.pop('model')\n optimizer = meta.pop('optimizer')\n if is_module_wrapper(model):\n model = model.module\n\n if hasattr(model, 'CLASSES') and model.CLASSES is not None:\n # save class name to the meta\n meta.update(CLASSES=model.CLASSES)\n\n checkpoint = {\n 'meta': meta,\n 'state_dict': weights_to_cpu(get_state_dict(model.model))\n }\n # save optimizer state dict in the checkpoint\n if isinstance(optimizer, Optimizer):\n checkpoint['optimizer'] = optimizer.state_dict()\n elif isinstance(optimizer, dict):\n checkpoint['optimizer'] = {}\n for name, optim in optimizer.items():\n checkpoint['optimizer'][name] = optim.state_dict()\n\n if filename.startswith('pavi://'):\n if file_client_args is not None:\n raise ValueError(\n 'file_client_args should be \"None\" if filename starts with'\n f'\"pavi://\", but got {file_client_args}')\n try:\n from pavi import exception, modelcloud\n except ImportError:\n raise ImportError(\n 'Please install pavi to load checkpoint from modelcloud.')\n model_path = filename[7:]\n root = modelcloud.Folder()\n model_dir, model_name = osp.split(model_path)\n try:\n model = modelcloud.get(model_dir)\n except exception.NodeNotFoundError:\n model = root.create_training_model(model_dir)\n with TemporaryDirectory() as tmp_dir:\n checkpoint_file = osp.join(tmp_dir, model_name)\n with open(checkpoint_file, 'wb') as f:\n torch.save(checkpoint, f)\n f.flush()\n model.create_file(checkpoint_file, name=model_name)\n else:\n file_client = FileClient.infer_client(file_client_args, filename)\n with io.BytesIO() as f:\n torch.save(checkpoint, f)\n file_client.put(f.getvalue(), filename)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/default_constructor.py", "content": "from .builder import RUNNER_BUILDERS, RUNNERS\n\n\n@RUNNER_BUILDERS.register_module()\nclass DefaultRunnerConstructor:\n \"\"\"Default constructor for runners.\n\n Custom existing `Runner` like `EpocBasedRunner` though `RunnerConstructor`.\n For example, We can inject some new properties and functions for `Runner`.\n\n Example:\n >>> from mmcv.runner import RUNNER_BUILDERS, build_runner\n >>> # Define a new RunnerReconstructor\n >>> @RUNNER_BUILDERS.register_module()\n >>> class MyRunnerConstructor:\n ... def __init__(self, runner_cfg, default_args=None):\n ... if not isinstance(runner_cfg, dict):\n ... raise TypeError('runner_cfg should be a dict',\n ... f'but got {type(runner_cfg)}')\n ... self.runner_cfg = runner_cfg\n ... self.default_args = default_args\n ...\n ... def __call__(self):\n ... runner = RUNNERS.build(self.runner_cfg,\n ... default_args=self.default_args)\n ... # Add new properties for existing runner\n ... runner.my_name = 'my_runner'\n ... runner.my_function = lambda self: print(self.my_name)\n ... ...\n >>> # build your runner\n >>> runner_cfg = dict(type='EpochBasedRunner', max_epochs=40,\n ... constructor='MyRunnerConstructor')\n >>> runner = build_runner(runner_cfg)\n \"\"\"\n\n def __init__(self, runner_cfg, default_args=None):\n if not isinstance(runner_cfg, dict):\n raise TypeError('runner_cfg should be a dict',\n f'but got {type(runner_cfg)}')\n self.runner_cfg = runner_cfg\n self.default_args = default_args\n\n def __call__(self):\n return RUNNERS.build(self.runner_cfg, default_args=self.default_args)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/dist_utils.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport functools\nimport os\nimport subprocess\nfrom collections import OrderedDict\n\nimport torch\nimport torch.multiprocessing as mp\nfrom torch import distributed as dist\nfrom torch._utils import (_flatten_dense_tensors, _take_tensors,\n _unflatten_dense_tensors)\n\n\ndef init_dist(launcher, backend='nccl', **kwargs):\n if mp.get_start_method(allow_none=True) is None:\n mp.set_start_method('spawn')\n if launcher == 'pytorch':\n _init_dist_pytorch(backend, **kwargs)\n elif launcher == 'mpi':\n _init_dist_mpi(backend, **kwargs)\n elif launcher == 'slurm':\n _init_dist_slurm(backend, **kwargs)\n else:\n raise ValueError(f'Invalid launcher type: {launcher}')\n\n\ndef _init_dist_pytorch(backend, **kwargs):\n # TODO: use local_rank instead of rank % num_gpus\n rank = int(os.environ['RANK'])\n num_gpus = torch.cuda.device_count()\n torch.cuda.set_device(rank % num_gpus)\n dist.init_process_group(backend=backend, **kwargs)\n\n\ndef _init_dist_mpi(backend, **kwargs):\n # TODO: use local_rank instead of rank % num_gpus\n rank = int(os.environ['OMPI_COMM_WORLD_RANK'])\n num_gpus = torch.cuda.device_count()\n torch.cuda.set_device(rank % num_gpus)\n dist.init_process_group(backend=backend, **kwargs)\n\n\ndef _init_dist_slurm(backend, port=None):\n \"\"\"Initialize slurm distributed training environment.\n\n If argument ``port`` is not specified, then the master port will be system\n environment variable ``MASTER_PORT``. If ``MASTER_PORT`` is not in system\n environment variable, then a default port ``29500`` will be used.\n\n Args:\n backend (str): Backend of torch.distributed.\n port (int, optional): Master port. Defaults to None.\n \"\"\"\n proc_id = int(os.environ['SLURM_PROCID'])\n ntasks = int(os.environ['SLURM_NTASKS'])\n node_list = os.environ['SLURM_NODELIST']\n num_gpus = torch.cuda.device_count()\n torch.cuda.set_device(proc_id % num_gpus)\n addr = subprocess.getoutput(\n f'scontrol show hostname {node_list} | head -n1')\n # specify master port\n if port is not None:\n os.environ['MASTER_PORT'] = str(port)\n elif 'MASTER_PORT' in os.environ:\n pass # use MASTER_PORT in the environment variable\n else:\n # 29500 is torch.distributed default port\n os.environ['MASTER_PORT'] = '29500'\n # use MASTER_ADDR in the environment variable if it already exists\n if 'MASTER_ADDR' not in os.environ:\n os.environ['MASTER_ADDR'] = addr\n os.environ['WORLD_SIZE'] = str(ntasks)\n os.environ['LOCAL_RANK'] = str(proc_id % num_gpus)\n os.environ['RANK'] = str(proc_id)\n dist.init_process_group(backend=backend)\n\n\ndef get_dist_info():\n if dist.is_available() and dist.is_initialized():\n rank = dist.get_rank()\n world_size = dist.get_world_size()\n else:\n rank = 0\n world_size = 1\n return rank, world_size\n\n\ndef master_only(func):\n\n @functools.wraps(func)\n def wrapper(*args, **kwargs):\n rank, _ = get_dist_info()\n if rank == 0:\n return func(*args, **kwargs)\n\n return wrapper\n\n\ndef allreduce_params(params, coalesce=True, bucket_size_mb=-1):\n \"\"\"Allreduce parameters.\n\n Args:\n params (list[torch.Parameters]): List of parameters or buffers of a\n model.\n coalesce (bool, optional): Whether allreduce parameters as a whole.\n Defaults to True.\n bucket_size_mb (int, optional): Size of bucket, the unit is MB.\n Defaults to -1.\n \"\"\"\n _, world_size = get_dist_info()\n if world_size == 1:\n return\n params = [param.data for param in params]\n if coalesce:\n _allreduce_coalesced(params, world_size, bucket_size_mb)\n else:\n for tensor in params:\n dist.all_reduce(tensor.div_(world_size))\n\n\ndef allreduce_grads(params, coalesce=True, bucket_size_mb=-1):\n \"\"\"Allreduce gradients.\n\n Args:\n params (list[torch.Parameters]): List of parameters of a model\n coalesce (bool, optional): Whether allreduce parameters as a whole.\n Defaults to True.\n bucket_size_mb (int, optional): Size of bucket, the unit is MB.\n Defaults to -1.\n \"\"\"\n grads = [\n param.grad.data for param in params\n if param.requires_grad and param.grad is not None\n ]\n _, world_size = get_dist_info()\n if world_size == 1:\n return\n if coalesce:\n _allreduce_coalesced(grads, world_size, bucket_size_mb)\n else:\n for tensor in grads:\n dist.all_reduce(tensor.div_(world_size))\n\n\ndef _allreduce_coalesced(tensors, world_size, bucket_size_mb=-1):\n if bucket_size_mb > 0:\n bucket_size_bytes = bucket_size_mb * 1024 * 1024\n buckets = _take_tensors(tensors, bucket_size_bytes)\n else:\n buckets = OrderedDict()\n for tensor in tensors:\n tp = tensor.type()\n if tp not in buckets:\n buckets[tp] = []\n buckets[tp].append(tensor)\n buckets = buckets.values()\n\n for bucket in buckets:\n flat_tensors = _flatten_dense_tensors(bucket)\n dist.all_reduce(flat_tensors)\n flat_tensors.div_(world_size)\n for tensor, synced in zip(\n bucket, _unflatten_dense_tensors(flat_tensors, bucket)):\n tensor.copy_(synced)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/epoch_based_runner.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport platform\nimport shutil\nimport time\nimport warnings\nimport time\nimport datetime\nimport torch\nimport mmcv\nfrom .base_runner import BaseRunner\nfrom .builder import RUNNERS\nfrom .checkpoint import save_checkpoint\nfrom .utils import get_host_info\nfrom mmcv.utils.logging import print_log\nfrom .record import MetricLogger, get_grad_norm\n\n\n@RUNNERS.register_module()\nclass EpochBasedRunner(BaseRunner):\n \"\"\"Epoch-based Runner.\n\n This runner train models epoch by epoch.\n \"\"\"\n\n def run_iter(self, data_batch, train_mode, **kwargs):\n if self.batch_processor is not None:\n outputs = self.batch_processor(\n self.model, data_batch, train_mode=train_mode, **kwargs)\n elif train_mode:\n outputs = self.model.train_step(data_batch, self.optimizer,\n **kwargs)\n\n # if not isinstance(self.model, dict):\n # outputs = self.model.train_step(data_batch, self.optimizer,\n # **kwargs)\n # else:\n # outputs = {}\n # for name in self.model.keys():\n # outputs.update(self.model[name].train_step(data_batch, self.optimizer,\n # **kwargs))\n\n else:\n outputs = self.model.val_step(data_batch, self.optimizer, **kwargs)\n if not isinstance(outputs, dict):\n raise TypeError('\"batch_processor()\" or \"model.train_step()\"'\n 'and \"model.val_step()\" must return a dict')\n if 'log_vars' in outputs:\n self.log_buffer.update_dict(outputs['log_vars'])\n # {'loss': loss, 'log_vars': {'loss': loss, 'metric_1': ..., 'metric_2': ....} }\n self.outputs = outputs\n\n def train(self, data_loader, **kwargs):\n if hasattr(self.model, 'train'):\n self.model.train()\n elif isinstance(self.model.model, dict):\n for name in self.model.model.keys():\n self.model.model[name].train()\n else:\n self.model.model.train()\n # if not isinstance(self.model, dict):\n # self.model.train()\n # else:\n # for name in self.model.keys():\n # self.model[name].train()\n\n self.mode = 'train'\n self.data_loader = data_loader\n self._max_iters = self._max_epochs * len(self.data_loader)\n self.call_hook('before_train_epoch')\n time.sleep(2) # Prevent possible deadlock during epoch transition\n for i, data_batch in enumerate(self.data_loader):\n self._inner_iter = i\n self.call_hook('before_train_iter')\n self.run_iter(data_batch, train_mode=True, **kwargs)\n self.call_hook('after_train_iter')\n self._iter += 1\n\n self.metrics = {k: meter.avg for k, meter in self.log_buffer.meters.items()}\n self.call_hook('after_train_epoch')\n self._epoch += 1\n\n def simple_train(self, data_loader, **kwargs):\n optimizer = self.optimizer\n accumulated_step = self.opt_cfg.get('accumulated_step', 1)\n clip_max_norm = self.opt_cfg.get('clip_max_norm', 0)\n print_freq = self.opt_cfg.get('print_freq', 1)\n nni = self.opt_cfg.get('nni', None)\n self.model.train()\n self.mode = 'train'\n self.data_loader = data_loader\n self._max_iters = self._max_epochs * len(self.data_loader)\n # metric_logger = MetricLogger(delimiter=\" \", dist_print=0, logger=self.logger)\n header = 'Epoch: [{}]'.format(self._epoch)\n print_freq = len(data_loader) if print_freq <= 0 else print_freq\n metric_logger = self.log_buffer\n for data_batch, idx in metric_logger.log_every(data_loader, print_freq, header):\n self._inner_iter = idx\n self.run_iter(data_batch, train_mode=True, **kwargs)\n losses = self.outputs['loss'] / accumulated_step\n losses.backward()\n if clip_max_norm > 0:\n grad_norm = torch.nn.utils.clip_grad_norm_(self.model.parameters(), clip_max_norm)\n else:\n grad_norm = get_grad_norm(self.model.parameters())\n if idx % accumulated_step == 0:\n optimizer.step()\n optimizer.zero_grad()\n\n metric_logger.update(lr=optimizer.param_groups[0][\"lr\"])\n metric_logger.update(grad_norm=grad_norm)\n metric_logger.update_dict(self.outputs['log_vars'])\n self._iter += 1\n\n\n self.metrics = {k: meter.avg for k, meter in metric_logger.meters.items()}\n self.call_hook('after_train_epoch')\n metric_logger.clear()\n self._epoch += 1\n if nni is not None:\n nni.report_intermediate_result(\n {name: value for name, value in self.metrics.items() if self.opt_cfg.metrics in name})\n\n @torch.no_grad()\n def simple_val(self, data_loader, **kwargs):\n # 用IterBasedRunner是否会更统一？\n # 如果要更进一步整合，应该变成eval_hook,但这是一个simple case\n self.model.eval()\n self.mode = 'val'\n opt_cfg = self.opt_cfg\n save_fmt = opt_cfg['save_fmt']\n # metric_logger = MetricLogger(dist_print=0, delimiter=\" \", logger=self.logger)\n metric_logger = self.log_buffer\n header = 'TestEpoch: [{0}]'.format(self.epoch - 1)\n save_dir = os.path.join(self.work_dir, f\"{opt_cfg['dataset']}\")\n if save_fmt and self._epoch == 1:\n os.makedirs(save_dir, exist_ok=True)\n for batch, idx in metric_logger.log_every(data_loader, 1, header):\n metrics = self.model.val_step(batch, save_dir,\n idx=idx, save_fmt=save_fmt, filename=batch.get('filename', None))\n # self.run_iter()\n metric_logger.update_dict(metrics)\n stats = {k: meter.avg for k, meter in metric_logger.meters.items()}\n if opt_cfg['mode'] == 'nni':\n self.nni.report_final_result({name: value for name, value in stats.items() if opt_cfg['metrics'] in name})\n # 仅进行验证时触发，结束while\n metric_logger.clear()\n if not self.flag:\n self._epoch += 1\n\n @torch.no_grad()\n def val(self, data_loader, **kwargs):\n if hasattr(self.model, 'eval'):\n self.model.eval()\n elif isinstance(self.model.model, dict):\n for name in self.model.model.keys():\n self.model.model[name].eval()\n else:\n self.model.model.eval()\n self.mode = 'val'\n self.data_loader = data_loader\n self.call_hook('before_val_epoch')\n time.sleep(2) # Prevent possible deadlock during epoch transition\n tic = time.time()\n for i, data_batch in enumerate(self.data_loader):\n self._inner_iter = i\n self.call_hook('before_val_iter')\n self.run_iter(data_batch, train_mode=False, idx=i,\n img_range=self.opt_cfg['img_range'],\n save_fmt=self.opt_cfg['save_fmt'], filename=data_batch.get('filename', [None])[0], save_dir=self.save_dir)\n self.call_hook('after_val_iter')\n print(\"test time:\", time.time() - tic)\n self.call_hook('after_val_epoch')\n if self.opt_cfg['eval']:\n self._epoch += 1\n\n def run(self, data_loaders, workflow, max_epochs=None, **kwargs):\n \"\"\"Start running.\n\n Args:\n data_loaders (list[:obj:`DataLoader`]): Dataloaders for training\n and validation.\n workflow (list[tuple]): A list of (phase, epochs) to specify the\n running order and epochs. E.g, [('train', 2), ('val', 1)] means\n running 2 epochs for training and 1 epoch for validation,\n iteratively.\n \"\"\"\n assert isinstance(data_loaders, dict)\n assert mmcv.is_list_of(workflow, tuple)\n assert len(data_loaders) == len(workflow), print_log(f\"{len(data_loaders)} == {len(workflow)}\")\n if max_epochs is not None:\n warnings.warn(\n 'setting max_epochs in run is deprecated, '\n 'please set max_epochs in runner_config', DeprecationWarning)\n self._max_epochs = max_epochs\n\n assert self._max_epochs is not None, (\n 'max_epochs must be specified during instantiation')\n self.flag = any('train' in mode for mode, _ in workflow)\n self.workflow = workflow\n self.data_length = 1\n for i, flow in enumerate(workflow):\n mode, epochs = flow\n if mode == 'train':\n self._max_iters = self._max_epochs * len(data_loaders[mode])\n self.data_length = len(data_loaders[mode])\n break\n\n\n work_dir = self.work_dir if self.work_dir is not None else 'NONE'\n print_log(f'Start running, host: {get_host_info()}, work_dir: {work_dir}',\n logger=self.logger)\n print_log(f'Hooks will be executed in the following order:\\n{self.get_hook_info()}',\n logger=self.logger)\n print_log(f'workflow: {workflow}, max: {self._max_epochs} epochs',\n logger=self.logger)\n self.call_hook('before_run')\n tic = time.time()\n print_freq = self.opt_cfg.get('print_freq', 1)\n # from 1 to self._max_epochs, not from 0\n while self.epoch <= self._max_epochs:\n for i, flow in enumerate(workflow):\n mode, epochs = flow\n if isinstance(mode, str): # self.train()\n if not hasattr(self, mode):\n raise ValueError(\n f'runner has no method named \"{mode}\" to run an '\n 'epoch')\n epoch_runner = getattr(self, mode)\n else:\n raise TypeError(\n 'mode in workflow must be a str, but got {}'.format(\n type(mode)))\n\n for epoch in range(epochs):\n if mode == 'train' and self.epoch >= self._max_epochs:\n break\n epoch_runner(data_loaders[mode], **kwargs)\n if self.earlyStop:\n print_log(\"model train has diverged, python will stop training\", logger=self.logger)\n break\n time.sleep(1) # wait for some hooks like loggers to finish\n self.call_hook('after_run')\n total_time = time.time() - tic\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n print_log('Training time {}'.format(total_time_str), logger=self.logger)\n\n def save_checkpoint(self,\n out_dir,\n filename_tmpl='epoch_{}.pth',\n save_optimizer=True,\n meta=None,\n create_symlink=True):\n \"\"\"Save the checkpoint.\n\n Args:\n out_dir (str): The directory that checkpoints are saved.\n filename_tmpl (str, optional): The checkpoint filename template,\n which contains a placeholder for the epoch number.\n Defaults to 'epoch_{}.pth'.\n save_optimizer (bool, optional): Whether to save the optimizer to\n the checkpoint. Defaults to True.\n meta (dict, optional): The meta information to be saved in the\n checkpoint. Defaults to None.\n create_symlink (bool, optional): Whether to create a symlink\n \"latest.pth\" to point to the latest checkpoint.\n Defaults to True.\n \"\"\"\n if meta is None:\n meta = {}\n elif not isinstance(meta, dict):\n raise TypeError(\n f'meta should be a dict or None, but got {type(meta)}')\n if self.meta is not None:\n meta.update(self.meta)\n # Note: meta.update(self.meta) should be done before\n # meta.update(epoch=self.epoch + 1, iter=self.iter) otherwise\n # there will be problems with resumed checkpoints.\n # More details in https://github.com/open-mmlab/mmcv/pull/1108\n meta.update(epoch=self.epoch + 1, iter=self.iter)\n\n filename = filename_tmpl.format(self.epoch + 1)\n filepath = os.path.join(out_dir, filename)\n optimizer = self.optimizer if save_optimizer else None\n save_checkpoint(self.model, filepath, optimizer=optimizer, meta=meta)\n # in some environments, `os.symlink` is not supported, you may need to\n # set `create_symlink` to False\n if create_symlink:\n dst_file = os.path.join(out_dir, 'latest.pth')\n if platform.system() != 'Windows':\n mmcv.symlink(filename, dst_file)\n else:\n shutil.copy(filepath, dst_file)\n\n\n@RUNNERS.register_module()\nclass Runner(EpochBasedRunner):\n \"\"\"Deprecated name of EpochBasedRunner.\"\"\"\n\n def __init__(self, *args, **kwargs):\n warnings.warn(\n 'Runner was deprecated, please use EpochBasedRunner instead',\n DeprecationWarning)\n super().__init__(*args, **kwargs)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/fp16_utils.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport functools\nimport warnings\nfrom collections import abc\nfrom inspect import getfullargspec\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.utils import TORCH_VERSION, digit_version\nfrom .dist_utils import allreduce_grads as _allreduce_grads\n\ntry:\n # If PyTorch version >= 1.6.0, torch.cuda.amp.autocast would be imported\n # and used; otherwise, auto fp16 will adopt mmcv's implementation.\n # Note that when PyTorch >= 1.6.0, we still cast tensor types to fp16\n # manually, so the behavior may not be consistent with real amp.\n from torch.cuda.amp import autocast\nexcept ImportError:\n pass\n\n\ndef cast_tensor_type(inputs, src_type, dst_type):\n \"\"\"Recursively convert Tensor in inputs from src_type to dst_type.\n\n Note:\n In v1.4.4 and later, ``cast_tersor_type`` will only convert the\n torch.Tensor which is consistent with ``src_type`` to the ``dst_type``.\n Before v1.4.4, it ignores the ``src_type`` argument, leading to some\n potential problems. For example,\n ``cast_tensor_type(inputs, torch.float, torch.half)`` will convert all\n tensors in inputs to ``torch.half`` including those originally in\n ``torch.Int`` or other types, which is not expected.\n\n Args:\n inputs: Inputs that to be casted.\n src_type (torch.dtype): Source type..\n dst_type (torch.dtype): Destination type.\n\n Returns:\n The same type with inputs, but all contained Tensors have been cast.\n \"\"\"\n if isinstance(inputs, nn.Module):\n return inputs\n elif isinstance(inputs, torch.Tensor):\n # we need to ensure that the type of inputs to be casted are the same\n # as the argument `src_type`.\n return inputs.to(dst_type) if inputs.dtype == src_type else inputs\n elif isinstance(inputs, str):\n return inputs\n elif isinstance(inputs, np.ndarray):\n return inputs\n elif isinstance(inputs, abc.Mapping):\n return type(inputs)({\n k: cast_tensor_type(v, src_type, dst_type)\n for k, v in inputs.items()\n })\n elif isinstance(inputs, abc.Iterable):\n return type(inputs)(\n cast_tensor_type(item, src_type, dst_type) for item in inputs)\n else:\n return inputs\n\n\ndef auto_fp16(apply_to=None, out_fp32=False):\n \"\"\"Decorator to enable fp16 training automatically.\n\n This decorator is useful when you write custom modules and want to support\n mixed precision training. If inputs arguments are fp32 tensors, they will\n be converted to fp16 automatically. Arguments other than fp32 tensors are\n ignored. If you are using PyTorch >= 1.6, torch.cuda.amp is used as the\n backend, otherwise, original mmcv implementation will be adopted.\n\n Args:\n apply_to (Iterable, optional): The argument names to be converted.\n `None` indicates all arguments.\n out_fp32 (bool): Whether to convert the output back to fp32.\n\n Example:\n\n >>> import torch.nn as nn\n >>> class MyModule1(nn.Module):\n >>>\n >>> # Convert x and y to fp16\n >>> @auto_fp16()\n >>> def forward(self, x, y):\n >>> pass\n\n >>> import torch.nn as nn\n >>> class MyModule2(nn.Module):\n >>>\n >>> # convert pred to fp16\n >>> @auto_fp16(apply_to=('pred', ))\n >>> def do_something(self, pred, others):\n >>> pass\n \"\"\"\n\n def auto_fp16_wrapper(old_func):\n\n @functools.wraps(old_func)\n def new_func(*args, **kwargs):\n # check if the module has set the attribute `fp16_enabled`, if not,\n # just fallback to the original method.\n if not isinstance(args[0], torch.nn.Module):\n raise TypeError('@auto_fp16 can only be used to decorate the '\n 'method of nn.Module')\n if not (hasattr(args[0], 'fp16_enabled') and args[0].fp16_enabled):\n return old_func(*args, **kwargs)\n\n # get the arg spec of the decorated method\n args_info = getfullargspec(old_func)\n # get the argument names to be casted\n args_to_cast = args_info.args if apply_to is None else apply_to\n # convert the args that need to be processed\n new_args = []\n # NOTE: default args are not taken into consideration\n if args:\n arg_names = args_info.args[:len(args)]\n for i, arg_name in enumerate(arg_names):\n if arg_name in args_to_cast:\n new_args.append(\n cast_tensor_type(args[i], torch.float, torch.half))\n else:\n new_args.append(args[i])\n # convert the kwargs that need to be processed\n new_kwargs = {}\n if kwargs:\n for arg_name, arg_value in kwargs.items():\n if arg_name in args_to_cast:\n new_kwargs[arg_name] = cast_tensor_type(\n arg_value, torch.float, torch.half)\n else:\n new_kwargs[arg_name] = arg_value\n # apply converted arguments to the decorated method\n if (TORCH_VERSION != 'parrots' and\n digit_version(TORCH_VERSION) >= digit_version('1.6.0')):\n with autocast(enabled=True):\n output = old_func(*new_args, **new_kwargs)\n else:\n output = old_func(*new_args, **new_kwargs)\n # cast the results back to fp32 if necessary\n if out_fp32:\n output = cast_tensor_type(output, torch.half, torch.float)\n return output\n\n return new_func\n\n return auto_fp16_wrapper\n\n\ndef force_fp32(apply_to=None, out_fp16=False):\n \"\"\"Decorator to convert input arguments to fp32 in force.\n\n This decorator is useful when you write custom modules and want to support\n mixed precision training. If there are some inputs that must be processed\n in fp32 mode, then this decorator can handle it. If inputs arguments are\n fp16 tensors, they will be converted to fp32 automatically. Arguments other\n than fp16 tensors are ignored. If you are using PyTorch >= 1.6,\n torch.cuda.amp is used as the backend, otherwise, original mmcv\n implementation will be adopted.\n\n Args:\n apply_to (Iterable, optional): The argument names to be converted.\n `None` indicates all arguments.\n out_fp16 (bool): Whether to convert the output back to fp16.\n\n Example:\n\n >>> import torch.nn as nn\n >>> class MyModule1(nn.Module):\n >>>\n >>> # Convert x and y to fp32\n >>> @force_fp32()\n >>> def loss(self, x, y):\n >>> pass\n\n >>> import torch.nn as nn\n >>> class MyModule2(nn.Module):\n >>>\n >>> # convert pred to fp32\n >>> @force_fp32(apply_to=('pred', ))\n >>> def post_process(self, pred, others):\n >>> pass\n \"\"\"\n\n def force_fp32_wrapper(old_func):\n\n @functools.wraps(old_func)\n def new_func(*args, **kwargs):\n # check if the module has set the attribute `fp16_enabled`, if not,\n # just fallback to the original method.\n if not isinstance(args[0], torch.nn.Module):\n raise TypeError('@force_fp32 can only be used to decorate the '\n 'method of nn.Module')\n if not (hasattr(args[0], 'fp16_enabled') and args[0].fp16_enabled):\n return old_func(*args, **kwargs)\n # get the arg spec of the decorated method\n args_info = getfullargspec(old_func)\n # get the argument names to be casted\n args_to_cast = args_info.args if apply_to is None else apply_to\n # convert the args that need to be processed\n new_args = []\n if args:\n arg_names = args_info.args[:len(args)]\n for i, arg_name in enumerate(arg_names):\n if arg_name in args_to_cast:\n new_args.append(\n cast_tensor_type(args[i], torch.half, torch.float))\n else:\n new_args.append(args[i])\n # convert the kwargs that need to be processed\n new_kwargs = dict()\n if kwargs:\n for arg_name, arg_value in kwargs.items():\n if arg_name in args_to_cast:\n new_kwargs[arg_name] = cast_tensor_type(\n arg_value, torch.half, torch.float)\n else:\n new_kwargs[arg_name] = arg_value\n # apply converted arguments to the decorated method\n if (TORCH_VERSION != 'parrots' and\n digit_version(TORCH_VERSION) >= digit_version('1.6.0')):\n with autocast(enabled=False):\n output = old_func(*new_args, **new_kwargs)\n else:\n output = old_func(*new_args, **new_kwargs)\n # cast the results back to fp32 if necessary\n if out_fp16:\n output = cast_tensor_type(output, torch.float, torch.half)\n return output\n\n return new_func\n\n return force_fp32_wrapper\n\n\ndef allreduce_grads(params, coalesce=True, bucket_size_mb=-1):\n warnings.warning(\n '\"mmcv.runner.fp16_utils.allreduce_grads\" is deprecated, and will be '\n 'removed in v2.8. Please switch to \"mmcv.runner.allreduce_grads',\n DeprecationWarning)\n _allreduce_grads(params, coalesce=coalesce, bucket_size_mb=bucket_size_mb)\n\n\ndef wrap_fp16_model(model):\n \"\"\"Wrap the FP32 model to FP16.\n\n If you are using PyTorch >= 1.6, torch.cuda.amp is used as the\n backend, otherwise, original mmcv implementation will be adopted.\n\n For PyTorch >= 1.6, this function will\n 1. Set fp16 flag inside the model to True.\n\n Otherwise:\n 1. Convert FP32 model to FP16.\n 2. Remain some necessary layers to be FP32, e.g., normalization layers.\n 3. Set `fp16_enabled` flag inside the model to True.\n\n Args:\n model (nn.Module): Model in FP32.\n \"\"\"\n if (TORCH_VERSION == 'parrots'\n or digit_version(TORCH_VERSION) < digit_version('1.6.0')):\n # convert model to fp16\n model.half()\n # patch the normalization layers to make it work in fp32 mode\n patch_norm_fp32(model)\n # set `fp16_enabled` flag\n for m in model.modules():\n if hasattr(m, 'fp16_enabled'):\n m.fp16_enabled = True\n\n\ndef patch_norm_fp32(module):\n \"\"\"Recursively convert normalization layers from FP16 to FP32.\n\n Args:\n module (nn.Module): The modules to be converted in FP16.\n\n Returns:\n nn.Module: The converted module, the normalization layers have been\n converted to FP32.\n \"\"\"\n if isinstance(module, (nn.modules.batchnorm._BatchNorm, nn.GroupNorm)):\n module.float()\n if isinstance(module, nn.GroupNorm) or torch.__version__ < '1.3':\n module.forward = patch_forward_method(module.forward, torch.half,\n torch.float)\n for child in module.children():\n patch_norm_fp32(child)\n return module\n\n\ndef patch_forward_method(func, src_type, dst_type, convert_output=True):\n \"\"\"Patch the forward method of a module.\n\n Args:\n func (callable): The original forward method.\n src_type (torch.dtype): Type of input arguments to be converted from.\n dst_type (torch.dtype): Type of input arguments to be converted to.\n convert_output (bool): Whether to convert the output back to src_type.\n\n Returns:\n callable: The patched forward method.\n \"\"\"\n\n def new_forward(*args, **kwargs):\n output = func(*cast_tensor_type(args, src_type, dst_type),\n **cast_tensor_type(kwargs, src_type, dst_type))\n if convert_output:\n output = cast_tensor_type(output, dst_type, src_type)\n return output\n\n return new_forward\n\n\nclass LossScaler:\n \"\"\"Class that manages loss scaling in mixed precision training which\n supports both dynamic or static mode.\n\n The implementation refers to\n https://github.com/NVIDIA/apex/blob/master/apex/fp16_utils/loss_scaler.py.\n Indirectly, by supplying ``mode='dynamic'`` for dynamic loss scaling.\n It's important to understand how :class:`LossScaler` operates.\n Loss scaling is designed to combat the problem of underflowing\n gradients encountered at long times when training fp16 networks.\n Dynamic loss scaling begins by attempting a very high loss\n scale. Ironically, this may result in OVERflowing gradients.\n If overflowing gradients are encountered, :class:`FP16_Optimizer` then\n skips the update step for this particular iteration/minibatch,\n and :class:`LossScaler` adjusts the loss scale to a lower value.\n If a certain number of iterations occur without overflowing gradients\n detected,:class:`LossScaler` increases the loss scale once more.\n In this way :class:`LossScaler` attempts to \"ride the edge\" of always\n using the highest loss scale possible without incurring overflow.\n\n Args:\n init_scale (float): Initial loss scale value, default: 2**32.\n scale_factor (float): Factor used when adjusting the loss scale.\n Default: 2.\n mode (str): Loss scaling mode. 'dynamic' or 'static'\n scale_window (int): Number of consecutive iterations without an\n overflow to wait before increasing the loss scale. Default: 1000.\n \"\"\"\n\n def __init__(self,\n init_scale=2**32,\n mode='dynamic',\n scale_factor=2.,\n scale_window=1000):\n self.cur_scale = init_scale\n self.cur_iter = 0\n assert mode in ('dynamic',\n 'static'), 'mode can only be dynamic or static'\n self.mode = mode\n self.last_overflow_iter = -1\n self.scale_factor = scale_factor\n self.scale_window = scale_window\n\n def has_overflow(self, params):\n \"\"\"Check if params contain overflow.\"\"\"\n if self.mode != 'dynamic':\n return False\n for p in params:\n if p.grad is not None and LossScaler._has_inf_or_nan(p.grad.data):\n return True\n return False\n\n def _has_inf_or_nan(x):\n \"\"\"Check if params contain NaN.\"\"\"\n try:\n cpu_sum = float(x.float().sum())\n except RuntimeError as instance:\n if 'value cannot be converted' not in instance.args[0]:\n raise\n return True\n else:\n if cpu_sum == float('inf') or cpu_sum == -float('inf') \\\n or cpu_sum != cpu_sum:\n return True\n return False\n\n def update_scale(self, overflow):\n \"\"\"update the current loss scale value when overflow happens.\"\"\"\n if self.mode != 'dynamic':\n return\n if overflow:\n self.cur_scale = max(self.cur_scale / self.scale_factor, 1)\n self.last_overflow_iter = self.cur_iter\n else:\n if (self.cur_iter - self.last_overflow_iter) % \\\n self.scale_window == 0:\n self.cur_scale *= self.scale_factor\n self.cur_iter += 1\n\n def state_dict(self):\n \"\"\"Returns the state of the scaler as a :class:`dict`.\"\"\"\n return dict(\n cur_scale=self.cur_scale,\n cur_iter=self.cur_iter,\n mode=self.mode,\n last_overflow_iter=self.last_overflow_iter,\n scale_factor=self.scale_factor,\n scale_window=self.scale_window)\n\n def load_state_dict(self, state_dict):\n \"\"\"Loads the loss_scaler state dict.\n\n Args:\n state_dict (dict): scaler state.\n \"\"\"\n self.cur_scale = state_dict['cur_scale']\n self.cur_iter = state_dict['cur_iter']\n self.mode = state_dict['mode']\n self.last_overflow_iter = state_dict['last_overflow_iter']\n self.scale_factor = state_dict['scale_factor']\n self.scale_window = state_dict['scale_window']\n\n @property\n def loss_scale(self):\n return self.cur_scale\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .checkpoint import CheckpointHook\nfrom .closure import ClosureHook\nfrom .ema import EMAHook\nfrom .evaluation import DistEvalHook, EvalHook\nfrom .hook import HOOKS, Hook\nfrom .iter_timer import IterTimerHook\nfrom .logger import (DvcliveLoggerHook, LoggerHook, MlflowLoggerHook,\n NeptuneLoggerHook, PaviLoggerHook, TensorboardLoggerHook,\n TextLoggerHook, WandbLoggerHook)\nfrom .lr_updater import (CosineAnnealingLrUpdaterHook,\n CosineRestartLrUpdaterHook, CyclicLrUpdaterHook,\n ExpLrUpdaterHook, FixedLrUpdaterHook,\n FlatCosineAnnealingLrUpdaterHook, InvLrUpdaterHook,\n LrUpdaterHook, OneCycleLrUpdaterHook,\n PolyLrUpdaterHook, StepLrUpdaterHook)\nfrom .memory import EmptyCacheHook\nfrom .momentum_updater import (CosineAnnealingMomentumUpdaterHook,\n CyclicMomentumUpdaterHook, MomentumUpdaterHook,\n OneCycleMomentumUpdaterHook,\n StepMomentumUpdaterHook)\nfrom .optimizer import (Fp16OptimizerHook, GradientCumulativeFp16OptimizerHook,\n GradientCumulativeOptimizerHook, OptimizerHook)\nfrom .profiler import ProfilerHook\nfrom .sampler_seed import DistSamplerSeedHook\nfrom .sync_buffer import SyncBuffersHook\n\n__all__ = [\n 'HOOKS', 'Hook', 'CheckpointHook', 'ClosureHook', 'LrUpdaterHook',\n 'FixedLrUpdaterHook', 'StepLrUpdaterHook', 'ExpLrUpdaterHook',\n 'PolyLrUpdaterHook', 'InvLrUpdaterHook', 'CosineAnnealingLrUpdaterHook',\n 'FlatCosineAnnealingLrUpdaterHook', 'CosineRestartLrUpdaterHook',\n 'CyclicLrUpdaterHook', 'OneCycleLrUpdaterHook', 'OptimizerHook',\n 'Fp16OptimizerHook', 'IterTimerHook', 'DistSamplerSeedHook',\n 'EmptyCacheHook', 'LoggerHook', 'MlflowLoggerHook', 'PaviLoggerHook',\n 'TextLoggerHook', 'TensorboardLoggerHook', 'NeptuneLoggerHook',\n 'WandbLoggerHook', 'DvcliveLoggerHook', 'MomentumUpdaterHook',\n 'StepMomentumUpdaterHook', 'CosineAnnealingMomentumUpdaterHook',\n 'CyclicMomentumUpdaterHook', 'OneCycleMomentumUpdaterHook',\n 'SyncBuffersHook', 'EMAHook', 'EvalHook', 'DistEvalHook', 'ProfilerHook',\n 'GradientCumulativeOptimizerHook', 'GradientCumulativeFp16OptimizerHook'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/checkpoint.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\nimport warnings\nfrom mmcv.utils.logging import print_log\nfrom mmcv.fileio import FileClient\nfrom ..dist_utils import allreduce_params, master_only\nfrom .hook import HOOKS, Hook\nfrom math import inf\nimport os\nimport re\nfrom ..checkpoint import save_checkpoint, get_best_k_model\nimport platform\nimport mmcv\nimport shutil\n\n@HOOKS.register_module()\nclass CheckpointHook(Hook):\n \"\"\"Save checkpoints periodically.\n\n Args:\n interval (int): The saving period. If ``by_epoch=True``, interval\n indicates epochs, otherwise it indicates iterations.\n Default: -1, which means \"never\".\n by_epoch (bool): Saving checkpoints by epoch or by iteration.\n Default: True.\n save_optimizer (bool): Whether to save optimizer state_dict in the\n checkpoint. It is usually used for resuming experiments.\n Default: True.\n out_dir (str, optional): The root directory to save checkpoints. If not\n specified, ``runner.work_dir`` will be used by default. If\n specified, the ``out_dir`` will be the concatenation of ``out_dir``\n and the last level directory of ``runner.work_dir``.\n `Changed in version 1.3.16.`\n max_keep_ckpts (int, optional): The maximum checkpoints to keep.\n In some cases we want only the latest few checkpoints and would\n like to delete old ones to save the disk space.\n Default: -1, which means unlimited.\n save_last (bool, optional): Whether to force the last checkpoint to be\n saved regardless of interval. Default: True.\n sync_buffer (bool, optional): Whether to synchronize buffers in\n different gpus. Default: False.\n file_client_args (dict, optional): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n `New in version 1.3.16.`\n\n .. warning::\n Before v1.3.16, the ``out_dir`` argument indicates the path where the\n checkpoint is stored. However, since v1.3.16, ``out_dir`` indicates the\n root directory and the final path to save checkpoint is the\n concatenation of ``out_dir`` and the last level directory of\n ``runner.work_dir``. Suppose the value of ``out_dir`` is \"/path/of/A\"\n and the value of ``runner.work_dir`` is \"/path/of/B\", then the final\n path will be \"/path/of/A/B\".\n \"\"\"\n\n def __init__(self,\n interval=-1,\n by_epoch=True,\n save_optimizer=True,\n out_dir=None,\n max_keep_ckpts=-1,\n save_last=True,\n sync_buffer=False,\n file_client_args=None,\n **kwargs):\n self.interval = interval\n self.by_epoch = by_epoch\n self.save_optimizer = save_optimizer\n self.out_dir = out_dir\n self.max_keep_ckpts = max_keep_ckpts\n self.save_last = save_last\n self.args = kwargs\n self.sync_buffer = sync_buffer\n self.file_client_args = file_client_args\n\n def before_run(self, runner):\n if not self.out_dir:\n self.out_dir = runner.work_dir\n\n self.file_client = FileClient.infer_client(self.file_client_args,\n self.out_dir)\n\n # if `self.out_dir` is not equal to `runner.work_dir`, it means that\n # `self.out_dir` is set so the final `self.out_dir` is the\n # concatenation of `self.out_dir` and the last level directory of\n # `runner.work_dir`\n if self.out_dir != runner.work_dir:\n basename = osp.basename(runner.work_dir.rstrip(osp.sep))\n self.out_dir = self.file_client.join_path(self.out_dir, basename)\n\n print_log((f'Checkpoints will be saved to {self.out_dir} by '\n f'{self.file_client.name}.'), logger=runner.logger)\n\n # disable the create_symlink option because some file backends do not\n # allow to create a symlink\n if 'create_symlink' in self.args:\n if self.args[\n 'create_symlink'] and not self.file_client.allow_symlink:\n self.args['create_symlink'] = False\n warnings.warn(\n ('create_symlink is set as True by the user but is changed'\n 'to be False because creating symbolic link is not '\n f'allowed in {self.file_client.name}'))\n else:\n self.args['create_symlink'] = self.file_client.allow_symlink\n\n def after_train_epoch(self, runner):\n if not self.by_epoch:\n return\n\n # save checkpoint for following cases:\n # 1. every ``self.interval`` epochs\n # 2. reach the last epoch of training\n if self.every_n_epochs(\n runner, self.interval) or (self.save_last\n and self.is_last_epoch(runner)):\n print_log(\n f'Saving checkpoint at {runner.epoch + 1} epochs', logger=runner.logger)\n if self.sync_buffer:\n allreduce_params(runner.model.buffers())\n self._save_checkpoint(runner)\n\n @master_only\n def _save_checkpoint(self, runner):\n \"\"\"Save the current checkpoint and delete unwanted checkpoint.\"\"\"\n runner.save_checkpoint(\n self.out_dir, save_optimizer=self.save_optimizer, **self.args)\n if runner.meta is not None:\n if self.by_epoch:\n cur_ckpt_filename = self.args.get(\n 'filename_tmpl', 'epoch_{}.pth').format(runner.epoch + 1)\n else:\n cur_ckpt_filename = self.args.get(\n 'filename_tmpl', 'iter_{}.pth').format(runner.iter + 1)\n runner.meta.setdefault('hook_msgs', dict())\n runner.meta['hook_msgs']['last_ckpt'] = self.file_client.join_path(\n self.out_dir, cur_ckpt_filename)\n # remove other checkpoints\n if self.max_keep_ckpts > 0:\n if self.by_epoch:\n name = 'epoch_{}.pth'\n current_ckpt = runner.epoch + 1\n else:\n name = 'iter_{}.pth'\n current_ckpt = runner.iter + 1\n redundant_ckpts = range(\n current_ckpt - self.max_keep_ckpts * self.interval, 0,\n -self.interval)\n filename_tmpl = self.args.get('filename_tmpl', name)\n for _step in redundant_ckpts:\n ckpt_path = self.file_client.join_path(\n self.out_dir, filename_tmpl.format(_step))\n if self.file_client.isfile(ckpt_path):\n self.file_client.remove(ckpt_path)\n else:\n break\n\n def after_train_iter(self, runner):\n if self.by_epoch:\n return\n\n # save checkpoint for following cases:\n # 1. every ``self.interval`` iterations\n # 2. reach the last iteration of training\n if self.every_n_iters(\n runner, self.interval) or (self.save_last\n and self.is_last_iter(runner)):\n print_log(\n f'Saving checkpoint at {runner.iter + 1} iterations', logger=runner.logger)\n if self.sync_buffer:\n allreduce_params(runner.model.buffers())\n self._save_checkpoint(runner)\n\n\n\n@HOOKS.register_module()\nclass ModelCheckpoint(Hook):\n\n rule_map = {'greater': lambda x, y: x >= y, 'less': lambda x, y: x <= y}\n indicator_rule_map = {'greater': lambda x, y: max(x, y), 'less': lambda x, y: min(x, y)}\n _default_greater_keys = [\n 'acc', 'top', 'AR@', 'auc', 'precision', 'mAP', 'mDice', 'mIoU',\n 'mAcc', 'aAcc', 'psnr', 'ssim', 'q'\n ]\n _default_best_prec1 = {'greater': -inf, 'less': inf}\n _default_less_keys = ['loss', 'sam', 'ergas']\n\n def __init__(self, indicator: str, formatter_filename=\"model_best_{epoch},{best_metric}\", print_freq=1, save_top_k: int=1,\n greater_keys=None, less_keys=None, best_prec1=None, best_epoch=0, sync_buffer=False):\n '''\n Args:\n save_interval:\n save_top_k: ``save_top_k == k``,\n if ``save_top_k == 0``, no models are saved.\n if ``save_top_k == -1``, all models are saved.\n Please note that the monitors are checked every ``every_n_epochs`` epochs.\n Returns:\n '''\n self.best_epoch = best_epoch\n self.print_freq = print_freq\n self.save_top_k = save_top_k\n self.sync_buffer = sync_buffer\n self.indicator = 'top-1' if indicator == 'top' else indicator\n self.formatter_filename = formatter_filename\n\n # indicator_lc = indicator.lower()\n\n if greater_keys is None:\n greater_keys = ModelCheckpoint._default_greater_keys\n else:\n if not isinstance(greater_keys, (list, tuple)):\n greater_keys = (greater_keys,)\n # assert is_seq_of(greater_keys, str)\n greater_keys = [key.lower() for key in greater_keys]\n\n if less_keys is None:\n less_keys = self._default_less_keys\n else:\n if not isinstance(less_keys, (list, tuple)):\n less_keys = (less_keys,)\n # assert is_seq_of(less_keys, str)\n less_keys = [key.lower() for key in less_keys]\n\n if indicator in greater_keys:\n rule = 'greater'\n elif indicator in less_keys:\n rule = 'less'\n elif any(key in indicator for key in greater_keys):\n rule = 'greater'\n elif any(key in indicator for key in less_keys):\n rule = 'less'\n else:\n raise ValueError(f'Cannot infer the rule for key '\n f'{indicator}, thus a specific rule '\n f'must be specified.')\n self.best_prec1 = self._default_best_prec1[rule] if best_prec1 is None else best_prec1\n self.compare_func = self.rule_map[rule]\n self.indicator_func = self.indicator_rule_map[rule]\n self.rule = rule\n\n def before_run(self, runner):\n self.save_model_path = runner.work_dir\n self.ckpt = os.path.join(self.save_model_path, 'checkpoint')\n os.makedirs(self.save_model_path, exist_ok=True)\n print_log(f'Checkpoints will be saved to {self.save_model_path}', logger=runner.logger)\n\n\n def earlyStopping(self, avg_grad_norm):\n\n if avg_grad_norm > 100:\n return True\n\n\n def after_train_epoch(self, runner):\n if self.sync_buffer:\n allreduce_params(runner.model.buffers())\n metrics = runner.metrics# metrics = {k: meter.avg for k, meter in runner.log_buffer.meters.items()}\n runner.earlyStop = self.earlyStopping(metrics.get('grad_norm', 0))\n self.save_checkpoint(runner, metrics)\n\n # print_log(' * Best training metrics so far@ {best_metric} in epoch {best_epoch}'.format(\n # best_metric=metrics['best_metric'], best_epoch=metrics['best_epoch']), logger=runner.logger)\n\n def _save_checkpoint(self, meta, out_dir, filename, is_best, create_symlink=True):\n if meta is None:\n meta = {}\n elif not isinstance(meta, dict):\n raise TypeError(\n f'meta should be a dict or None, but got {type(meta)}')\n # meta.update(epoch=meta.pop('epoch') + 1, iter=meta.pop('iter'))\n filepath = os.path.join(out_dir, filename)\n # save_checkpoint(meta.pop('model'), filepath, optimizer=meta.pop('optimizer'), meta=meta)\n save_checkpoint(filepath, meta=meta)\n if create_symlink or is_best:\n dst_file = os.path.join(out_dir, 'model_best_.pth')\n if platform.system() != 'Windows':\n mmcv.symlink(filename, dst_file)\n else:\n shutil.copy(filepath, dst_file)\n\n @master_only\n def save_checkpoint(self, runner, metrics):\n flag = False\n if not hasattr(runner.model, 'train') and isinstance(runner.model.model, dict):\n flag = True\n stats = {}\n for k, m in runner.model.model.items():\n stats[k] = {\n 'epoch': runner.epoch,\n 'iter': runner.iter,\n 'model': m,\n 'best_metric': {name: value for name, value in metrics.items() if\n name not in ['grad_norm', 'lr', 'time', 'data_time']},\n # 保存多个metric的数值, 实际比较的时候还是只有一个\n 'loss': metrics['loss'],\n 'best_epoch': runner._epoch,\n 'optimizer': runner.optimizer[k]\n }\n runner.metrics.update(\n {'best_metric': {k: stats[k]['best_metric']}, 'best_epoch': {k: stats[k]['best_epoch']}})\n else:\n stats = {\n 'epoch': runner.epoch,\n 'iter': runner.iter,\n 'model': runner.model,\n 'best_metric': {name: value for name, value in metrics.items() if\n name not in ['grad_norm', 'lr', 'time', 'data_time']},\n # 保存多个metric的数值, 实际比较的时候还是只有一个\n 'loss': metrics['loss'],\n 'best_epoch': runner._epoch,\n 'optimizer': runner.optimizer\n }\n runner.metrics.update(best_metric=stats['best_metric'], best_epoch=stats['best_epoch'])\n\n new_best_k_model_flag = []\n indicator = self.indicator\n save_top_k = self.save_top_k\n # stats 应当是{epoch: X, score: Y} -> [epoch, score]\n assert isinstance(stats, dict), print(f\"stats in model_checkpoint should be dict but be {type(stats)}\")\n # stats = list(stats.values())\n best_k_model, _ = get_best_k_model(self.save_model_path + \"/checkpoint\", indicator)\n\n # print(best_k_model)\n if save_top_k < 0:\n raise ValueError(f\"Invalid value for save_top_k={save_top_k}. Must be >= 0\")\n if save_top_k == 0:\n stats['best_metric'] = self._default_best_prec1[self.rule]\n stats['best_epoch'] = 0\n self._save_checkpoint(stats, self.save_model_path, is_best=False, filename=f\"{stats['epoch']}.pth.tar\")\n\n if save_top_k >= 1:\n # self.best_prec1 = self.indicator_func(self.best_prec1, stats[self.indicator])\n if len(best_k_model) >= save_top_k:\n # reverse=True, 降序, default： False\n # 使用索引去对best_k_model进行排序,best_k_model应是列表，才能返回索引\n best_k_model.append([stats['epoch'], stats['best_metric'], None])\n sortedIndex_best_k_model = sorted(range(len(best_k_model)),\n key=lambda k: float(best_k_model[k][1][indicator]),\n reverse=self.rule == \"less\")\n # print(sortedIndex_best_k_model)\n new_best_k_model_flag = [\n not self.compare_func(float(query_score[indicator]), stats['best_metric'][indicator]) for\n _, query_score, _ in best_k_model]\n # print(new_best_k_model_flag)\n\n # ckpt_stats = [] # {}\n # key会冲突导致popitem出错\n # ckpt_stats[str(stats['epoch'])] = stats[indicator]\n ckpt_stats = [stats['epoch'], stats['best_metric']]\n\n for index in sortedIndex_best_k_model:\n if new_best_k_model_flag[index]:\n # top_k_count += 1\n # best_k_model[indicator][index] = stats[indicator]\n # best_k_model['epoch'][index] = stats['epoch']\n # best_k_model.pop(str(index))\n # best_k_model.update(ckpt_stats)\n # best_k_model[index] = list(ckpt_stats.popitem())\n fname = self.save_model_path + \"/\" + best_k_model[index][2]\n ckpt_stats.append(None)\n best_k_model[index] = ckpt_stats\n\n if os.path.isfile(fname):\n os.remove(fname)\n break\n stats['best_epoch'], stats['best_metric'] = best_k_model[sortedIndex_best_k_model[-1]][:2]\n best_k_model = best_k_model[:-1]\n # best_k_model = [{'epoch': k, 'score': v} for k, v in best_k_model.items()]\n best_k_model = [{'epoch': epoch, 'best_metric': score} for (epoch, score, _) in best_k_model]\n with open(self.ckpt, 'w') as f:\n outs = [self.formatter_filename.format(**line) + \"\\n\" for line in best_k_model]\n f.writelines(outs)\n else:\n if not flag:\n with open(self.ckpt, 'a') as f:\n outs = self.formatter_filename.format(**stats) + \"\\n\"\n f.writelines(outs)\n # 训练初期，不满topk时候, 模型是否保存下来\n # if save_top_k == 1:\n # if len(best_k_model) < save_top_k:\n # new_best_k_model_flag = [True]\n\n is_best = any(new_best_k_model_flag)\n if runner.epoch % self.print_freq == 0 or is_best:\n self._save_checkpoint(\n stats, out_dir=self.save_model_path, is_best=is_best, filename=f\"{runner.epoch}.pth.tar\")\n\n if not flag:\n print_log(' * Best training metrics so far@ {best_metric} in epoch {best_epoch}'.format(\n best_metric=stats['best_metric'], best_epoch=stats['best_epoch']), logger=runner.logger\n )\n\n return stats\n\n def after_train_iter(self, runner):\n if hasattr(runner.model, 'train'):\n if type(runner.model.module.model).__name__ == 'INN':\n runner.model.module.model.free()\n else:\n if isinstance(runner.model.model, dict):\n runner.model.model['PAN2MS'].module.free()\n # raise NotImplementedError(\"after_train_iter is not implemented by ModelCheckpoint (customed)\")\n\n\n\n\n\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/closure.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .hook import HOOKS, Hook\n\n\n@HOOKS.register_module()\nclass ClosureHook(Hook):\n\n def __init__(self, fn_name, fn):\n assert hasattr(self, fn_name)\n assert callable(fn)\n setattr(self, fn_name, fn)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/ema.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom ...parallel import is_module_wrapper\nfrom ..hooks.hook import HOOKS, Hook\n\n\n@HOOKS.register_module()\nclass EMAHook(Hook):\n r\"\"\"Exponential Moving Average Hook.\n\n Use Exponential Moving Average on all parameters of model in training\n process. All parameters have a ema backup, which update by the formula\n as below. EMAHook takes priority over EvalHook and CheckpointSaverHook.\n\n .. math::\n\n Xema\\_{t+1} = (1 - \\text{momentum}) \\times\n Xema\\_{t} + \\text{momentum} \\times X_t\n\n Args:\n momentum (float): The momentum used for updating ema parameter.\n Defaults to 0.0002.\n interval (int): Update ema parameter every interval iteration.\n Defaults to 1.\n warm_up (int): During first warm_up steps, we may use smaller momentum\n to update ema parameters more slowly. Defaults to 100.\n resume_from (str): The checkpoint path. Defaults to None.\n \"\"\"\n\n def __init__(self,\n momentum=0.0002,\n interval=1,\n warm_up=100,\n resume_from=None):\n assert isinstance(interval, int) and interval > 0\n self.warm_up = warm_up\n self.interval = interval\n assert momentum > 0 and momentum < 1\n self.momentum = momentum**interval\n self.checkpoint = resume_from\n\n def before_run(self, runner):\n \"\"\"To resume model with it's ema parameters more friendly.\n\n Register ema parameter as ``named_buffer`` to model\n \"\"\"\n model = runner.model\n if is_module_wrapper(model):\n model = model.module\n self.param_ema_buffer = {}\n self.model_parameters = dict(model.named_parameters(recurse=True))\n for name, value in self.model_parameters.items():\n # \".\" is not allowed in module's buffer name\n buffer_name = f\"ema_{name.replace('.', '_')}\"\n self.param_ema_buffer[name] = buffer_name\n model.register_buffer(buffer_name, value.data.clone())\n self.model_buffers = dict(model.named_buffers(recurse=True))\n if self.checkpoint is not None:\n runner.resume(self.checkpoint)\n\n def after_train_iter(self, runner):\n \"\"\"Update ema parameter every self.interval iterations.\"\"\"\n curr_step = runner.iter\n # We warm up the momentum considering the instability at beginning\n momentum = min(self.momentum,\n (1 + curr_step) / (self.warm_up + curr_step))\n if curr_step % self.interval != 0:\n return\n for name, parameter in self.model_parameters.items():\n buffer_name = self.param_ema_buffer[name]\n buffer_parameter = self.model_buffers[buffer_name]\n buffer_parameter.mul_(1 - momentum).add_(momentum, parameter.data)\n\n def after_train_epoch(self, runner):\n \"\"\"We load parameter values from ema backup to model before the\n EvalHook.\"\"\"\n self._swap_ema_parameters()\n\n def before_train_epoch(self, runner):\n \"\"\"We recover model's parameter from ema backup after last epoch's\n EvalHook.\"\"\"\n self._swap_ema_parameters()\n\n def _swap_ema_parameters(self):\n \"\"\"Swap the parameter of model with parameter in ema_buffer.\"\"\"\n for name, value in self.model_parameters.items():\n temp = value.data.clone()\n ema_buffer = self.model_buffers[self.param_ema_buffer[name]]\n value.data.copy_(ema_buffer.data)\n ema_buffer.data.copy_(temp)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/evaluation.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\nimport warnings\nfrom math import inf\n\nimport torch.distributed as dist\nfrom torch.nn.modules.batchnorm import _BatchNorm\nfrom torch.utils.data import DataLoader\n\nfrom mmcv.fileio import FileClient\nfrom mmcv.utils import is_seq_of\nfrom .hook import Hook\nfrom .logger import LoggerHook\n\n\nclass EvalHook(Hook):\n \"\"\"Non-Distributed evaluation hook.\n\n This hook will regularly perform evaluation in a given interval when\n performing in non-distributed environment.\n\n Args:\n dataloader (DataLoader): A PyTorch dataloader, whose dataset has\n implemented ``evaluate`` function.\n start (int | None, optional): Evaluation starting epoch. It enables\n evaluation before the training starts if ``start`` <= the resuming\n epoch. If None, whether to evaluate is merely decided by\n ``interval``. Default: None.\n interval (int): Evaluation interval. Default: 1.\n by_epoch (bool): Determine perform evaluation by epoch or by iteration.\n If set to True, it will perform by epoch. Otherwise, by iteration.\n Default: True.\n save_best (str, optional): If a metric is specified, it would measure\n the best checkpoint during evaluation. The information about best\n checkpoint would be saved in ``runner.meta['hook_msgs']`` to keep\n best score value and best checkpoint path, which will be also\n loaded when resume checkpoint. Options are the evaluation metrics\n on the test dataset. e.g., ``bbox_mAP``, ``segm_mAP`` for bbox\n detection and instance segmentation. ``AR@100`` for proposal\n recall. If ``save_best`` is ``auto``, the first key of the returned\n ``OrderedDict`` result will be used. Default: None.\n rule (str | None, optional): Comparison rule for best score. If set to\n None, it will infer a reasonable rule. Keys such as 'acc', 'top'\n .etc will be inferred by 'greater' rule. Keys contain 'loss' will\n be inferred by 'less' rule. Options are 'greater', 'less', None.\n Default: None.\n test_fn (callable, optional): test a model with samples from a\n dataloader, and return the test results. If ``None``, the default\n test function ``mmcv.engine.single_gpu_test`` will be used.\n (default: ``None``)\n greater_keys (List[str] | None, optional): Metric keys that will be\n inferred by 'greater' comparison rule. If ``None``,\n _default_greater_keys will be used. (default: ``None``)\n less_keys (List[str] | None, optional): Metric keys that will be\n inferred by 'less' comparison rule. If ``None``, _default_less_keys\n will be used. (default: ``None``)\n out_dir (str, optional): The root directory to save checkpoints. If not\n specified, `runner.work_dir` will be used by default. If specified,\n the `out_dir` will be the concatenation of `out_dir` and the last\n level directory of `runner.work_dir`.\n `New in version 1.3.16.`\n file_client_args (dict): Arguments to instantiate a FileClient.\n See :class:`mmcv.fileio.FileClient` for details. Default: None.\n `New in version 1.3.16.`\n **eval_kwargs: Evaluation arguments fed into the evaluate function of\n the dataset.\n\n Note:\n If new arguments are added for EvalHook, tools/test.py,\n tools/eval_metric.py may be affected.\n \"\"\"\n\n # Since the key for determine greater or less is related to the downstream\n # tasks, downstream repos may need to overwrite the following inner\n # variable accordingly.\n\n rule_map = {'greater': lambda x, y: x > y, 'less': lambda x, y: x < y}\n init_value_map = {'greater': -inf, 'less': inf}\n _default_greater_keys = [\n 'acc', 'top', 'AR@', 'auc', 'precision', 'mAP', 'mDice', 'mIoU',\n 'mAcc', 'aAcc'\n ]\n _default_less_keys = ['loss']\n\n def __init__(self,\n dataloader,\n start=None,\n interval=1,\n by_epoch=True,\n save_best=None,\n rule=None,\n test_fn=None,\n greater_keys=None,\n less_keys=None,\n out_dir=None,\n file_client_args=None,\n **eval_kwargs):\n if not isinstance(dataloader, DataLoader):\n raise TypeError(f'dataloader must be a pytorch DataLoader, '\n f'but got {type(dataloader)}')\n\n if interval <= 0:\n raise ValueError(f'interval must be a positive number, '\n f'but got {interval}')\n\n assert isinstance(by_epoch, bool), '``by_epoch`` should be a boolean'\n\n if start is not None and start < 0:\n raise ValueError(f'The evaluation start epoch {start} is smaller '\n f'than 0')\n\n self.dataloader = dataloader\n self.interval = interval\n self.start = start\n self.by_epoch = by_epoch\n\n assert isinstance(save_best, str) or save_best is None, \\\n '\"\"save_best\"\" should be a str or None ' \\\n f'rather than {type(save_best)}'\n self.save_best = save_best\n self.eval_kwargs = eval_kwargs\n self.initial_flag = True\n\n if test_fn is None:\n from mmcv.engine import single_gpu_test\n self.test_fn = single_gpu_test\n else:\n self.test_fn = test_fn\n\n if greater_keys is None:\n self.greater_keys = self._default_greater_keys\n else:\n if not isinstance(greater_keys, (list, tuple)):\n greater_keys = (greater_keys, )\n assert is_seq_of(greater_keys, str)\n self.greater_keys = greater_keys\n\n if less_keys is None:\n self.less_keys = self._default_less_keys\n else:\n if not isinstance(less_keys, (list, tuple)):\n less_keys = (less_keys, )\n assert is_seq_of(less_keys, str)\n self.less_keys = less_keys\n\n if self.save_best is not None:\n self.best_ckpt_path = None\n self._init_rule(rule, self.save_best)\n\n self.out_dir = out_dir\n self.file_client_args = file_client_args\n\n def _init_rule(self, rule, key_indicator):\n \"\"\"Initialize rule, key_indicator, comparison_func, and best score.\n\n Here is the rule to determine which rule is used for key indicator\n when the rule is not specific (note that the key indicator matching\n is case-insensitive):\n 1. If the key indicator is in ``self.greater_keys``, the rule will be\n specified as 'greater'.\n 2. Or if the key indicator is in ``self.less_keys``, the rule will be\n specified as 'less'.\n 3. Or if the key indicator is equal to the substring in any one item\n in ``self.greater_keys``, the rule will be specified as 'greater'.\n 4. Or if the key indicator is equal to the substring in any one item\n in ``self.less_keys``, the rule will be specified as 'less'.\n\n Args:\n rule (str | None): Comparison rule for best score.\n key_indicator (str | None): Key indicator to determine the\n comparison rule.\n \"\"\"\n if rule not in self.rule_map and rule is not None:\n raise KeyError(f'rule must be greater, less or None, '\n f'but got {rule}.')\n\n if rule is None:\n if key_indicator != 'auto':\n # `_lc` here means we use the lower case of keys for\n # case-insensitive matching\n key_indicator_lc = key_indicator.lower()\n greater_keys = [key.lower() for key in self.greater_keys]\n less_keys = [key.lower() for key in self.less_keys]\n\n if key_indicator_lc in greater_keys:\n rule = 'greater'\n elif key_indicator_lc in less_keys:\n rule = 'less'\n elif any(key in key_indicator_lc for key in greater_keys):\n rule = 'greater'\n elif any(key in key_indicator_lc for key in less_keys):\n rule = 'less'\n else:\n raise ValueError(f'Cannot infer the rule for key '\n f'{key_indicator}, thus a specific rule '\n f'must be specified.')\n self.rule = rule\n self.key_indicator = key_indicator\n if self.rule is not None:\n self.compare_func = self.rule_map[self.rule]\n\n def before_run(self, runner):\n if not self.out_dir:\n self.out_dir = runner.work_dir\n\n self.file_client = FileClient.infer_client(self.file_client_args,\n self.out_dir)\n\n # if `self.out_dir` is not equal to `runner.work_dir`, it means that\n # `self.out_dir` is set so the final `self.out_dir` is the\n # concatenation of `self.out_dir` and the last level directory of\n # `runner.work_dir`\n if self.out_dir != runner.work_dir:\n basename = osp.basename(runner.work_dir.rstrip(osp.sep))\n self.out_dir = self.file_client.join_path(self.out_dir, basename)\n runner.logger.info(\n (f'The best checkpoint will be saved to {self.out_dir} by '\n f'{self.file_client.name}'))\n\n if self.save_best is not None:\n if runner.meta is None:\n warnings.warn('runner.meta is None. Creating an empty one.')\n runner.meta = dict()\n runner.meta.setdefault('hook_msgs', dict())\n self.best_ckpt_path = runner.meta['hook_msgs'].get(\n 'best_ckpt', None)\n\n def before_train_iter(self, runner):\n \"\"\"Evaluate the model only at the start of training by iteration.\"\"\"\n if self.by_epoch or not self.initial_flag:\n return\n if self.start is not None and runner.iter >= self.start:\n self.after_train_iter(runner)\n self.initial_flag = False\n\n def before_train_epoch(self, runner):\n \"\"\"Evaluate the model only at the start of training by epoch.\"\"\"\n if not (self.by_epoch and self.initial_flag):\n return\n if self.start is not None and runner.epoch >= self.start:\n self.after_train_epoch(runner)\n self.initial_flag = False\n\n def after_train_iter(self, runner):\n \"\"\"Called after every training iter to evaluate the results.\"\"\"\n if not self.by_epoch and self._should_evaluate(runner):\n # Because the priority of EvalHook is higher than LoggerHook, the\n # training log and the evaluating log are mixed. Therefore,\n # we need to dump the training log and clear it before evaluating\n # log is generated. In addition, this problem will only appear in\n # `IterBasedRunner` whose `self.by_epoch` is False, because\n # `EpochBasedRunner` whose `self.by_epoch` is True calls\n # `_do_evaluate` in `after_train_epoch` stage, and at this stage\n # the training log has been printed, so it will not cause any\n # problem. more details at\n # https://github.com/open-mmlab/mmsegmentation/issues/694\n for hook in runner._hooks:\n if isinstance(hook, LoggerHook):\n hook.after_train_iter(runner)\n runner.log_buffer.clear()\n\n self._do_evaluate(runner)\n\n def after_train_epoch(self, runner):\n \"\"\"Called after every training epoch to evaluate the results.\"\"\"\n # if self.by_epoch and self._should_evaluate(runner):\n # self._do_evaluate(runner)\n ...\n\n def _do_evaluate(self, runner):\n \"\"\"perform evaluation and save ckpt.\"\"\"\n results = self.test_fn(runner.model, self.dataloader)\n runner.log_buffer.output['eval_iter_num'] = len(self.dataloader)\n key_score = self.evaluate(runner, results)\n # the key_score may be `None` so it needs to skip the action to save\n # the best checkpoint\n if self.save_best and key_score:\n self._save_ckpt(runner, key_score)\n\n def _should_evaluate(self, runner):\n \"\"\"Judge whether to perform evaluation.\n\n Here is the rule to judge whether to perform evaluation:\n 1. It will not perform evaluation during the epoch/iteration interval,\n which is determined by ``self.interval``.\n 2. It will not perform evaluation if the start time is larger than\n current time.\n 3. It will not perform evaluation when current time is larger than\n the start time but during epoch/iteration interval.\n\n Returns:\n bool: The flag indicating whether to perform evaluation.\n \"\"\"\n if self.by_epoch:\n current = runner.epoch\n check_time = self.every_n_epochs\n else:\n current = runner.iter\n check_time = self.every_n_iters\n\n if self.start is None:\n if not check_time(runner, self.interval):\n # No evaluation during the interval.\n return False\n elif (current + 1) < self.start:\n # No evaluation if start is larger than the current time.\n return False\n else:\n # Evaluation only at epochs/iters 3, 5, 7...\n # if start==3 and interval==2\n if (current + 1 - self.start) % self.interval:\n return False\n return True\n\n def _save_ckpt(self, runner, key_score):\n \"\"\"Save the best checkpoint.\n\n It will compare the score according to the compare function, write\n related information (best score, best checkpoint path) and save the\n best checkpoint into ``work_dir``.\n \"\"\"\n if self.by_epoch:\n current = f'epoch_{runner.epoch + 1}'\n cur_type, cur_time = 'epoch', runner.epoch + 1\n else:\n current = f'iter_{runner.iter + 1}'\n cur_type, cur_time = 'iter', runner.iter + 1\n\n best_score = runner.meta['hook_msgs'].get(\n 'best_score', self.init_value_map[self.rule])\n if self.compare_func(key_score, best_score):\n best_score = key_score\n runner.meta['hook_msgs']['best_score'] = best_score\n\n if self.best_ckpt_path and self.file_client.isfile(\n self.best_ckpt_path):\n self.file_client.remove(self.best_ckpt_path)\n runner.logger.info(\n (f'The previous best checkpoint {self.best_ckpt_path} was '\n 'removed'))\n\n best_ckpt_name = f'best_{self.key_indicator}_{current}.pth'\n self.best_ckpt_path = self.file_client.join_path(\n self.out_dir, best_ckpt_name)\n runner.meta['hook_msgs']['best_ckpt'] = self.best_ckpt_path\n\n runner.save_checkpoint(\n self.out_dir, best_ckpt_name, create_symlink=False)\n runner.logger.info(\n f'Now best checkpoint is saved as {best_ckpt_name}.')\n runner.logger.info(\n f'Best {self.key_indicator} is {best_score:0.4f} '\n f'at {cur_time} {cur_type}.')\n\n def evaluate(self, runner, results):\n \"\"\"Evaluate the results.\n\n Args:\n runner (:obj:`mmcv.Runner`): The underlined training runner.\n results (list): Output results.\n \"\"\"\n eval_res = self.dataloader.dataset.evaluate(\n results, logger=runner.logger, **self.eval_kwargs)\n\n for name, val in eval_res.items():\n runner.log_buffer.output[name] = val\n runner.log_buffer.ready = True\n\n if self.save_best is not None:\n # If the performance of model is pool, the `eval_res` may be an\n # empty dict and it will raise exception when `self.save_best` is\n # not None. More details at\n # https://github.com/open-mmlab/mmdetection/issues/6265.\n if not eval_res:\n warnings.warn(\n 'Since `eval_res` is an empty dict, the behavior to save '\n 'the best checkpoint will be skipped in this evaluation.')\n return None\n\n if self.key_indicator == 'auto':\n # infer from eval_results\n self._init_rule(self.rule, list(eval_res.keys())[0])\n return eval_res[self.key_indicator]\n\n return None\n\n\nclass DistEvalHook(EvalHook):\n \"\"\"Distributed evaluation hook.\n\n This hook will regularly perform evaluation in a given interval when\n performing in distributed environment.\n\n Args:\n dataloader (DataLoader): A PyTorch dataloader, whose dataset has\n implemented ``evaluate`` function.\n start (int | None, optional): Evaluation starting epoch. It enables\n evaluation before the training starts if ``start`` <= the resuming\n epoch. If None, whether to evaluate is merely decided by\n ``interval``. Default: None.\n interval (int): Evaluation interval. Default: 1.\n by_epoch (bool): Determine perform evaluation by epoch or by iteration.\n If set to True, it will perform by epoch. Otherwise, by iteration.\n default: True.\n save_best (str, optional): If a metric is specified, it would measure\n the best checkpoint during evaluation. The information about best\n checkpoint would be saved in ``runner.meta['hook_msgs']`` to keep\n best score value and best checkpoint path, which will be also\n loaded when resume checkpoint. Options are the evaluation metrics\n on the test dataset. e.g., ``bbox_mAP``, ``segm_mAP`` for bbox\n detection and instance segmentation. ``AR@100`` for proposal\n recall. If ``save_best`` is ``auto``, the first key of the returned\n ``OrderedDict`` result will be used. Default: None.\n rule (str | None, optional): Comparison rule for best score. If set to\n None, it will infer a reasonable rule. Keys such as 'acc', 'top'\n .etc will be inferred by 'greater' rule. Keys contain 'loss' will\n be inferred by 'less' rule. Options are 'greater', 'less', None.\n Default: None.\n test_fn (callable, optional): test a model with samples from a\n dataloader in a multi-gpu manner, and return the test results. If\n ``None``, the default test function ``mmcv.engine.multi_gpu_test``\n will be used. (default: ``None``)\n tmpdir (str | None): Temporary directory to save the results of all\n processes. Default: None.\n gpu_collect (bool): Whether to use gpu or cpu to collect results.\n Default: False.\n broadcast_bn_buffer (bool): Whether to broadcast the\n buffer(running_mean and running_var) of rank 0 to other rank\n before evaluation. Default: True.\n out_dir (str, optional): The root directory to save checkpoints. If not\n specified, `runner.work_dir` will be used by default. If specified,\n the `out_dir` will be the concatenation of `out_dir` and the last\n level directory of `runner.work_dir`.\n file_client_args (dict): Arguments to instantiate a FileClient.\n See :class:`mmcv.fileio.FileClient` for details. Default: None.\n **eval_kwargs: Evaluation arguments fed into the evaluate function of\n the dataset.\n \"\"\"\n\n def __init__(self,\n dataloader,\n start=None,\n interval=1,\n by_epoch=True,\n save_best=None,\n rule=None,\n test_fn=None,\n greater_keys=None,\n less_keys=None,\n broadcast_bn_buffer=True,\n tmpdir=None,\n gpu_collect=False,\n out_dir=None,\n file_client_args=None,\n **eval_kwargs):\n\n if test_fn is None:\n from mmcv.engine import multi_gpu_test\n test_fn = multi_gpu_test\n\n super().__init__(\n dataloader,\n start=start,\n interval=interval,\n by_epoch=by_epoch,\n save_best=save_best,\n rule=rule,\n test_fn=test_fn,\n greater_keys=greater_keys,\n less_keys=less_keys,\n out_dir=out_dir,\n file_client_args=file_client_args,\n **eval_kwargs)\n\n self.broadcast_bn_buffer = broadcast_bn_buffer\n self.tmpdir = tmpdir\n self.gpu_collect = gpu_collect\n\n def _do_evaluate(self, runner):\n \"\"\"perform evaluation and save ckpt.\"\"\"\n # Synchronization of BatchNorm's buffer (running_mean\n # and running_var) is not supported in the DDP of pytorch,\n # which may cause the inconsistent performance of models in\n # different ranks, so we broadcast BatchNorm's buffers\n # of rank 0 to other ranks to avoid this.\n if self.broadcast_bn_buffer:\n model = runner.model\n for name, module in model.named_modules():\n if isinstance(module,\n _BatchNorm) and module.track_running_stats:\n dist.broadcast(module.running_var, 0)\n dist.broadcast(module.running_mean, 0)\n\n tmpdir = self.tmpdir\n if tmpdir is None:\n tmpdir = osp.join(runner.work_dir, '.eval_hook')\n\n results = self.test_fn(\n runner.model,\n self.dataloader,\n tmpdir=tmpdir,\n gpu_collect=self.gpu_collect)\n if runner.rank == 0:\n print('\\n')\n runner.log_buffer.output['eval_iter_num'] = len(self.dataloader)\n key_score = self.evaluate(runner, results)\n # the key_score may be `None` so it needs to skip the action to\n # save the best checkpoint\n if self.save_best and key_score:\n self._save_ckpt(runner, key_score)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/hook.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom mmcv.utils import Registry, is_method_overridden\n\nHOOKS = Registry('hook')\n\n\nclass Hook:\n stages = ('before_run', 'before_train_epoch', 'before_train_iter',\n 'after_train_iter', 'after_train_epoch', 'before_val_epoch',\n 'before_val_iter', 'after_val_iter', 'after_val_epoch',\n 'after_run')\n\n def before_run(self, runner):\n pass\n\n def after_run(self, runner):\n pass\n\n def before_epoch(self, runner):\n pass\n\n def after_epoch(self, runner):\n pass\n\n def before_iter(self, runner):\n pass\n\n def after_iter(self, runner):\n pass\n\n def before_train_epoch(self, runner):\n self.before_epoch(runner)\n\n def before_val_epoch(self, runner):\n self.before_epoch(runner)\n\n def after_train_epoch(self, runner):\n self.after_epoch(runner)\n\n def after_val_epoch(self, runner):\n self.after_epoch(runner)\n\n def before_train_iter(self, runner):\n self.before_iter(runner)\n\n def before_val_iter(self, runner):\n self.before_iter(runner)\n\n def after_train_iter(self, runner):\n self.after_iter(runner)\n\n def after_val_iter(self, runner):\n self.after_iter(runner)\n\n def every_n_epochs(self, runner, n):\n return (runner.epoch + 1) % n == 0 if n > 0 else False\n\n def every_n_inner_iters(self, runner, n):\n return (runner.inner_iter + 1) % n == 0 if n > 0 else False\n\n def every_n_iters(self, runner, n):\n return (runner.iter + 1) % n == 0 if n > 0 else False\n\n def end_of_epoch(self, runner):\n return runner.inner_iter + 1 == len(runner.data_loader)\n\n def is_last_epoch(self, runner):\n return runner.epoch + 1 == runner._max_epochs\n\n def is_last_iter(self, runner):\n return runner.iter + 1 == runner._max_iters\n\n def get_triggered_stages(self):\n trigger_stages = set()\n for stage in Hook.stages:\n if is_method_overridden(stage, Hook, self):\n trigger_stages.add(stage)\n\n # some methods will be triggered in multi stages\n # use this dict to map method to stages.\n method_stages_map = {\n 'before_epoch': ['before_train_epoch', 'before_val_epoch'],\n 'after_epoch': ['after_train_epoch', 'after_val_epoch'],\n 'before_iter': ['before_train_iter', 'before_val_iter'],\n 'after_iter': ['after_train_iter', 'after_val_iter'],\n }\n\n for method, map_stages in method_stages_map.items():\n if is_method_overridden(method, Hook, self):\n trigger_stages.update(map_stages)\n\n return [stage for stage in Hook.stages if stage in trigger_stages]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/iter_timer.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport time\n\nfrom .hook import HOOKS, Hook\n\n\n@HOOKS.register_module()\nclass IterTimerHook(Hook):\n\n def before_epoch(self, runner):\n self.t = time.time()\n\n def before_iter(self, runner):\n runner.log_buffer.update(data_time=time.time() - self.t)#{'data_time': time.time() - self.t}\n\n def after_iter(self, runner):\n runner.log_buffer.update(time=time.time() - self.t)#{'time': time.time() - self.t}\n self.t = time.time()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .base import LoggerHook\nfrom .dvclive import DvcliveLoggerHook\nfrom .mlflow import MlflowLoggerHook\nfrom .neptune import NeptuneLoggerHook\nfrom .pavi import PaviLoggerHook\nfrom .tensorboard import TensorboardLoggerHook\nfrom .text import TextLoggerHook\nfrom .wandb import WandbLoggerHook\n\n__all__ = [\n 'LoggerHook', 'MlflowLoggerHook', 'PaviLoggerHook',\n 'TensorboardLoggerHook', 'TextLoggerHook', 'WandbLoggerHook',\n 'NeptuneLoggerHook', 'DvcliveLoggerHook'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/base.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numbers\nfrom abc import ABCMeta, abstractmethod\n\nimport numpy as np\nimport torch\n\nfrom ..hook import Hook\n\n\nclass LoggerHook(Hook):\n \"\"\"Base class for logger hooks.\n\n Args:\n interval (int): Logging interval (every k iterations). Default 10.\n ignore_last (bool): Ignore the log of last iterations in each epoch\n if less than `interval`. Default True.\n reset_flag (bool): Whether to clear the output buffer after logging.\n Default False.\n by_epoch (bool): Whether EpochBasedRunner is used. Default True.\n \"\"\"\n\n __metaclass__ = ABCMeta\n\n def __init__(self,\n interval=10,\n ignore_last=True,\n reset_flag=False,\n by_epoch=True):\n self.interval = interval\n self.ignore_last = ignore_last\n self.reset_flag = reset_flag\n self.by_epoch = by_epoch\n\n @abstractmethod\n def log(self, runner):\n pass\n\n @staticmethod\n def is_scalar(val, include_np=True, include_torch=True):\n \"\"\"Tell the input variable is a scalar or not.\n\n Args:\n val: Input variable.\n include_np (bool): Whether include 0-d np.ndarray as a scalar.\n include_torch (bool): Whether include 0-d torch.Tensor as a scalar.\n\n Returns:\n bool: True or False.\n \"\"\"\n if isinstance(val, numbers.Number):\n return True\n elif include_np and isinstance(val, np.ndarray) and val.ndim == 0:\n return True\n elif include_torch and isinstance(val, torch.Tensor) and len(val) == 1:\n return True\n else:\n return False\n\n def get_mode(self, runner):\n if runner.mode == 'train':\n if 'time' in runner.log_buffer.meters: #output\n mode = 'train'\n else:\n mode = 'val'\n elif runner.mode == 'val':\n mode = 'val'\n else:\n raise ValueError(f\"runner mode should be 'train' or 'val', \"\n f'but got {runner.mode}')\n return mode\n\n def get_epoch(self, runner):\n if runner.mode == 'train':\n epoch = runner.epoch# + 1\n elif runner.mode == 'val':\n # normal val mode\n # runner.epoch += 1 has been done before val workflow\n epoch = runner.epoch\n else:\n raise ValueError(f\"runner mode should be 'train' or 'val', \"\n f'but got {runner.mode}')\n return epoch\n\n def get_iter(self, runner, inner_iter=False):\n \"\"\"Get the current training iteration step.\"\"\"\n if self.by_epoch and inner_iter:\n current_iter = runner.inner_iter + 1\n else:\n current_iter = runner.iter + 1\n return current_iter\n\n def get_lr_tags(self, runner):\n tags = {}\n lrs = runner.current_lr()\n if isinstance(lrs, dict):\n for name, value in lrs.items():\n tags[f'learning_rate/{name}'] = value[0]\n else:\n tags['learning_rate'] = lrs[0]\n return tags\n\n def get_momentum_tags(self, runner):\n tags = {}\n momentums = runner.current_momentum()\n if isinstance(momentums, dict):\n for name, value in momentums.items():\n tags[f'momentum/{name}'] = value[0]\n else:\n tags['momentum'] = momentums[0]\n return tags\n\n def get_loggable_tags(self,\n runner,\n allow_scalar=True,\n allow_text=False,\n add_mode=True,\n tags_to_skip=('time', 'data_time', 'learning_rate', 'pan2ms', 'grad_norm', 'lr', 'memory')):\n tags = {}\n for var, val in runner.metrics.items():#log_buffer.output\n if var in tags_to_skip:\n continue\n if self.is_scalar(val) and not allow_scalar:\n continue\n if isinstance(val, str) and not allow_text:\n continue\n if add_mode:\n var = f'{self.get_mode(runner)}/{var}'\n tags[var] = val\n tags.update(self.get_lr_tags(runner))\n tags.update(self.get_momentum_tags(runner))\n return tags\n\n def before_run(self, runner):\n for hook in runner.hooks[::-1]:\n if isinstance(hook, LoggerHook):\n hook.reset_flag = True\n break\n\n def before_epoch(self, runner):\n runner.log_buffer.clear() # clear logs of last epoch\n\n def after_train_iter(self, runner):\n # if self.by_epoch and self.every_n_inner_iters(runner, self.interval):\n # runner.log_buffer.average(self.interval)\n # elif not self.by_epoch and self.every_n_iters(runner, self.interval):\n # runner.log_buffer.average(self.interval)\n # elif self.end_of_epoch(runner) and not self.ignore_last:\n # # not precise but more stable\n # runner.log_buffer.average(self.interval)\n\n # if runner.log_buffer.ready:\n # self.log(runner)\n # if self.reset_flag:\n # runner.log_buffer.clear_output()\n\n if self.by_epoch and self.every_n_inner_iters(runner, self.interval):\n # runner.log_buffer.ready = True\n self.log(runner)\n # if self.reset_flag:\n # runner.log_buffer.clear_output()\n\n\n def after_train_epoch(self, runner):\n # if runner.log_buffer.ready:\n if self.every_n_epochs(runner, self.interval):\n self.log(runner)\n if self.reset_flag:\n runner.log_buffer.clear_output()\n\n def after_val_epoch(self, runner):\n # runner.log_buffer.average()\n self.log(runner)\n if self.reset_flag:\n runner.log_buffer.clear_output()\n\n def after_val_iter(self, runner):\n self.log(runner)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/dvclive.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom pathlib import Path\n\nfrom ...dist_utils import master_only\nfrom ..hook import HOOKS\nfrom .base import LoggerHook\n\n\n@HOOKS.register_module()\nclass DvcliveLoggerHook(LoggerHook):\n \"\"\"Class to log metrics with dvclive.\n\n It requires `dvclive`_ to be installed.\n\n Args:\n model_file (str): Default None. If not None, after each epoch the\n model will be saved to {model_file}.\n interval (int): Logging interval (every k iterations). Default 10.\n ignore_last (bool): Ignore the log of last iterations in each epoch\n if less than `interval`. Default: True.\n reset_flag (bool): Whether to clear the output buffer after logging.\n Default: False.\n by_epoch (bool): Whether EpochBasedRunner is used. Default: True.\n kwargs: Arguments for instantiating `Live`_.\n\n .. _dvclive:\n https://dvc.org/doc/dvclive\n\n .. _Live:\n https://dvc.org/doc/dvclive/api-reference/live#parameters\n \"\"\"\n\n def __init__(self,\n model_file=None,\n interval=10,\n ignore_last=True,\n reset_flag=False,\n by_epoch=True,\n **kwargs):\n super().__init__(interval, ignore_last, reset_flag, by_epoch)\n self.model_file = model_file\n self.import_dvclive(**kwargs)\n\n def import_dvclive(self, **kwargs):\n try:\n from dvclive import Live\n except ImportError:\n raise ImportError(\n 'Please run \"pip install dvclive\" to install dvclive')\n self.dvclive = Live(**kwargs)\n\n @master_only\n def log(self, runner):\n tags = self.get_loggable_tags(runner)\n if tags:\n self.dvclive.set_step(self.get_iter(runner))\n for k, v in tags.items():\n self.dvclive.log(k, v)\n\n @master_only\n def after_train_epoch(self, runner):\n super().after_train_epoch(runner)\n if self.model_file is not None:\n runner.save_checkpoint(\n Path(self.model_file).parent,\n filename_tmpl=Path(self.model_file).name,\n create_symlink=False,\n )\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/mlflow.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom mmcv.utils import TORCH_VERSION\nfrom ...dist_utils import master_only\nfrom ..hook import HOOKS\nfrom .base import LoggerHook\n\n\n@HOOKS.register_module()\nclass MlflowLoggerHook(LoggerHook):\n \"\"\"Class to log metrics and (optionally) a trained model to MLflow.\n\n It requires `MLflow`_ to be installed.\n\n Args:\n exp_name (str, optional): Name of the experiment to be used.\n Default None. If not None, set the active experiment.\n If experiment does not exist, an experiment with provided name\n will be created.\n tags (Dict[str], optional): Tags for the current run.\n Default None. If not None, set tags for the current run.\n log_model (bool, optional): Whether to log an MLflow artifact.\n Default True. If True, log runner.model as an MLflow artifact\n for the current run.\n interval (int): Logging interval (every k iterations). Default: 10.\n ignore_last (bool): Ignore the log of last iterations in each epoch\n if less than `interval`. Default: True.\n reset_flag (bool): Whether to clear the output buffer after logging.\n Default: False.\n by_epoch (bool): Whether EpochBasedRunner is used. Default: True.\n\n .. _MLflow:\n https://www.mlflow.org/docs/latest/index.html\n \"\"\"\n\n def __init__(self,\n exp_name=None,\n tags=None,\n log_model=True,\n interval=10,\n ignore_last=True,\n reset_flag=False,\n by_epoch=True):\n super(MlflowLoggerHook, self).__init__(interval, ignore_last,\n reset_flag, by_epoch)\n self.import_mlflow()\n self.exp_name = exp_name\n self.tags = tags\n self.log_model = log_model\n\n def import_mlflow(self):\n try:\n import mlflow\n import mlflow.pytorch as mlflow_pytorch\n except ImportError:\n raise ImportError(\n 'Please run \"pip install mlflow\" to install mlflow')\n self.mlflow = mlflow\n self.mlflow_pytorch = mlflow_pytorch\n\n @master_only\n def before_run(self, runner):\n super(MlflowLoggerHook, self).before_run(runner)\n if self.exp_name is not None:\n self.mlflow.set_experiment(self.exp_name)\n if self.tags is not None:\n self.mlflow.set_tags(self.tags)\n\n @master_only\n def log(self, runner):\n tags = self.get_loggable_tags(runner)\n if tags:\n self.mlflow.log_metrics(tags, step=self.get_iter(runner))\n\n @master_only\n def after_run(self, runner):\n if self.log_model:\n self.mlflow_pytorch.log_model(\n runner.model,\n 'models',\n pip_requirements=[f'torch=={TORCH_VERSION}'])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/neptune.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom ...dist_utils import master_only\nfrom ..hook import HOOKS\nfrom .base import LoggerHook\n\n\n@HOOKS.register_module()\nclass NeptuneLoggerHook(LoggerHook):\n \"\"\"Class to log metrics to NeptuneAI.\n\n It requires `Neptune`_ to be installed.\n\n Args:\n init_kwargs (dict): a dict contains the initialization keys as below:\n\n - project (str): Name of a project in a form of\n namespace/project_name. If None, the value of NEPTUNE_PROJECT\n environment variable will be taken.\n - api_token (str): User’s API token. If None, the value of\n NEPTUNE_API_TOKEN environment variable will be taken. Note: It is\n strongly recommended to use NEPTUNE_API_TOKEN environment\n variable rather than placing your API token in plain text in your\n source code.\n - name (str, optional, default is 'Untitled'): Editable name of the\n run. Name is displayed in the run's Details and in Runs table as\n a column.\n\n Check https://docs.neptune.ai/api-reference/neptune#init for more\n init arguments.\n interval (int): Logging interval (every k iterations). Default: 10.\n ignore_last (bool): Ignore the log of last iterations in each epoch\n if less than ``interval``. Default: True.\n reset_flag (bool): Whether to clear the output buffer after logging.\n Default: True.\n with_step (bool): If True, the step will be logged from\n ``self.get_iters``. Otherwise, step will not be logged.\n Default: True.\n by_epoch (bool): Whether EpochBasedRunner is used. Default: True.\n\n .. _Neptune:\n https://docs.neptune.ai\n \"\"\"\n\n def __init__(self,\n init_kwargs=None,\n interval=10,\n ignore_last=True,\n reset_flag=True,\n with_step=True,\n by_epoch=True):\n\n super(NeptuneLoggerHook, self).__init__(interval, ignore_last,\n reset_flag, by_epoch)\n self.import_neptune()\n self.init_kwargs = init_kwargs\n self.with_step = with_step\n\n def import_neptune(self):\n try:\n import neptune.new as neptune\n except ImportError:\n raise ImportError(\n 'Please run \"pip install neptune-client\" to install neptune')\n self.neptune = neptune\n self.run = None\n\n @master_only\n def before_run(self, runner):\n if self.init_kwargs:\n self.run = self.neptune.init(**self.init_kwargs)\n else:\n self.run = self.neptune.init()\n\n @master_only\n def log(self, runner):\n tags = self.get_loggable_tags(runner)\n if tags:\n for tag_name, tag_value in tags.items():\n if self.with_step:\n self.run[tag_name].log(\n tag_value, step=self.get_iter(runner))\n else:\n tags['global_step'] = self.get_iter(runner)\n self.run[tag_name].log(tags)\n\n @master_only\n def after_run(self, runner):\n self.run.stop()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/pavi.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport json\nimport os\nimport os.path as osp\n\nimport torch\nimport yaml\n\nimport mmcv\nfrom ....parallel.utils import is_module_wrapper\nfrom ...dist_utils import master_only\nfrom ..hook import HOOKS\nfrom .base import LoggerHook\n\n\n@HOOKS.register_module()\nclass PaviLoggerHook(LoggerHook):\n \"\"\"Class to visual model, log metrics (for internal use).\n\n Args:\n init_kwargs (dict): A dict contains the initialization keys.\n add_graph (bool): Whether to visual model. Default: False.\n add_last_ckpt (bool): Whether to save checkpoint after run.\n Default: False.\n interval (int): Logging interval (every k iterations). Default: True.\n ignore_last (bool): Ignore the log of last iterations in each epoch\n if less than `interval`. Default: True.\n reset_flag (bool): Whether to clear the output buffer after logging.\n Default: False.\n by_epoch (bool): Whether EpochBasedRunner is used. Default: True.\n img_key (string): Get image data from Dataset. Default: 'img_info'.\n \"\"\"\n\n def __init__(self,\n init_kwargs=None,\n add_graph=False,\n add_last_ckpt=False,\n interval=10,\n ignore_last=True,\n reset_flag=False,\n by_epoch=True,\n img_key='img_info'):\n super(PaviLoggerHook, self).__init__(interval, ignore_last, reset_flag,\n by_epoch)\n self.init_kwargs = init_kwargs\n self.add_graph = add_graph\n self.add_last_ckpt = add_last_ckpt\n self.img_key = img_key\n\n @master_only\n def before_run(self, runner):\n super(PaviLoggerHook, self).before_run(runner)\n try:\n from pavi import SummaryWriter\n except ImportError:\n raise ImportError('Please run \"pip install pavi\" to install pavi.')\n\n self.run_name = runner.work_dir.split('/')[-1]\n\n if not self.init_kwargs:\n self.init_kwargs = dict()\n self.init_kwargs['name'] = self.run_name\n self.init_kwargs['model'] = runner._model_name\n if runner.meta is not None:\n if 'config_dict' in runner.meta:\n config_dict = runner.meta['config_dict']\n assert isinstance(\n config_dict,\n dict), ('meta[\"config_dict\"] has to be of a dict, '\n f'but got {type(config_dict)}')\n elif 'config_file' in runner.meta:\n config_file = runner.meta['config_file']\n config_dict = dict(mmcv.Config.fromfile(config_file))\n else:\n config_dict = None\n if config_dict is not None:\n # 'max_.*iter' is parsed in pavi sdk as the maximum iterations\n # to properly set up the progress bar.\n config_dict = config_dict.copy()\n config_dict.setdefault('max_iter', runner.max_iters)\n # non-serializable values are first converted in\n # mmcv.dump to json\n config_dict = json.loads(\n mmcv.dump(config_dict, file_format='json'))\n session_text = yaml.dump(config_dict)\n self.init_kwargs['session_text'] = session_text\n self.writer = SummaryWriter(**self.init_kwargs)\n\n def get_step(self, runner):\n \"\"\"Get the total training step/epoch.\"\"\"\n if self.get_mode(runner) == 'val' and self.by_epoch:\n return self.get_epoch(runner)\n else:\n return self.get_iter(runner)\n\n @master_only\n def log(self, runner):\n tags = self.get_loggable_tags(runner, add_mode=False)\n if tags:\n self.writer.add_scalars(\n self.get_mode(runner), tags, self.get_step(runner))\n\n @master_only\n def after_run(self, runner):\n if self.add_last_ckpt:\n ckpt_path = osp.join(runner.work_dir, 'latest.pth')\n if osp.islink(ckpt_path):\n ckpt_path = osp.join(runner.work_dir, os.readlink(ckpt_path))\n\n if osp.isfile(ckpt_path):\n # runner.epoch += 1 has been done before `after_run`.\n iteration = runner.epoch if self.by_epoch else runner.iter\n return self.writer.add_snapshot_file(\n tag=self.run_name,\n snapshot_file_path=ckpt_path,\n iteration=iteration)\n\n # flush the buffer and send a task ending signal to Pavi\n self.writer.close()\n\n @master_only\n def before_epoch(self, runner):\n if runner.epoch == 0 and self.add_graph:\n if is_module_wrapper(runner.model):\n _model = runner.model.module\n else:\n _model = runner.model\n device = next(_model.parameters()).device\n data = next(iter(runner.data_loader))\n image = data[self.img_key][0:1].to(device)\n with torch.no_grad():\n self.writer.add_graph(_model, image)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/tensorboard.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\n\nfrom mmcv.utils import TORCH_VERSION, digit_version\nfrom ...dist_utils import master_only\nfrom ..hook import HOOKS\nfrom .base import LoggerHook\n\n\n@HOOKS.register_module()\nclass TensorboardLoggerHook(LoggerHook):\n \"\"\"Class to log metrics to Tensorboard.\n\n Args:\n log_dir (string): Save directory location. Default: None. If default\n values are used, directory location is ``runner.work_dir``/tf_logs.\n interval (int): Logging interval (every k iterations). Default: True.\n ignore_last (bool): Ignore the log of last iterations in each epoch\n if less than `interval`. Default: True.\n reset_flag (bool): Whether to clear the output buffer after logging.\n Default: False.\n by_epoch (bool): Whether EpochBasedRunner is used. Default: True.\n \"\"\"\n\n def __init__(self,\n log_dir=None,\n interval=10,\n ignore_last=True,\n reset_flag=False,\n by_epoch=True):\n super(TensorboardLoggerHook, self).__init__(interval, ignore_last,\n reset_flag, by_epoch)\n self.log_dir = log_dir\n\n @master_only\n def before_run(self, runner):\n super(TensorboardLoggerHook, self).before_run(runner)\n if (TORCH_VERSION == 'parrots'\n or digit_version(TORCH_VERSION) < digit_version('1.1')):\n try:\n from tensorboardX import SummaryWriter\n except ImportError:\n raise ImportError('Please install tensorboardX to use '\n 'TensorboardLoggerHook.')\n else:\n try:\n from torch.utils.tensorboard import SummaryWriter\n except ImportError:\n raise ImportError(\n 'Please run \"pip install future tensorboard\" to install '\n 'the dependencies to use torch.utils.tensorboard '\n '(applicable to PyTorch 1.1 or higher)')\n\n if self.log_dir is None:\n self.log_dir = osp.join(runner.work_dir, 'tf_logs')\n self.writer = SummaryWriter(self.log_dir)\n\n @master_only\n def log(self, runner):\n tags = self.get_loggable_tags(runner, allow_text=True)\n for tag, val in tags.items():\n if isinstance(val, str):\n self.writer.add_text(tag, val, self.get_iter(runner))\n else:\n self.writer.add_scalar(tag, val, self.get_iter(runner))\n\n @master_only\n def after_run(self, runner):\n self.writer.close()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/text.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport datetime\nimport os\nimport os.path as osp\nfrom collections import OrderedDict\n\nimport torch\nimport torch.distributed as dist\n\nimport mmcv\nfrom mmcv.fileio.file_client import FileClient\nfrom mmcv.utils import is_tuple_of, scandir\nfrom ..hook import HOOKS\nfrom .base import LoggerHook\nfrom mmcv.utils.logging import print_log\n\n\n@HOOKS.register_module()\nclass TextLoggerHook(LoggerHook):\n \"\"\"Logger hook in text.\n\n In this logger hook, the information will be printed on terminal and\n saved in json file.\n\n Args:\n by_epoch (bool, optional): Whether EpochBasedRunner is used.\n Default: True.\n interval (int, optional): Logging interval (every k iterations).\n Default: 10.\n ignore_last (bool, optional): Ignore the log of last iterations in each\n epoch if less than :attr:`interval`. Default: True.\n reset_flag (bool, optional): Whether to clear the output buffer after\n logging. Default: False.\n interval_exp_name (int, optional): Logging interval for experiment\n name. This feature is to help users conveniently get the experiment\n information from screen or log file. Default: 1000.\n out_dir (str, optional): Logs are saved in ``runner.work_dir`` default.\n If ``out_dir`` is specified, logs will be copied to a new directory\n which is the concatenation of ``out_dir`` and the last level\n directory of ``runner.work_dir``. Default: None.\n `New in version 1.3.16.`\n out_suffix (str or tuple[str], optional): Those filenames ending with\n ``out_suffix`` will be copied to ``out_dir``.\n Default: ('.log.json', '.log', '.py').\n `New in version 1.3.16.`\n keep_local (bool, optional): Whether to keep local log when\n :attr:`out_dir` is specified. If False, the local log will be\n removed. Default: True.\n `New in version 1.3.16.`\n file_client_args (dict, optional): Arguments to instantiate a\n FileClient. See :class:`mmcv.fileio.FileClient` for details.\n Default: None.\n `New in version 1.3.16.`\n \"\"\"\n\n def __init__(self,\n by_epoch=True,\n interval=10,\n ignore_last=True,\n reset_flag=False,\n interval_exp_name=1000,\n out_dir=None,\n out_suffix=('.log.json', '.log', '.py'),\n keep_local=True,\n file_client_args=None):\n super(TextLoggerHook, self).__init__(interval, ignore_last, reset_flag,\n by_epoch)\n self.by_epoch = by_epoch\n self.time_sec_tot = 0\n self.interval_exp_name = interval_exp_name\n\n if out_dir is None and file_client_args is not None:\n raise ValueError(\n 'file_client_args should be \"None\" when `out_dir` is not'\n 'specified.')\n self.out_dir = out_dir\n\n if not (out_dir is None or isinstance(out_dir, str)\n or is_tuple_of(out_dir, str)):\n raise TypeError('out_dir should be \"None\" or string or tuple of '\n 'string, but got {out_dir}')\n self.out_suffix = out_suffix\n\n self.keep_local = keep_local\n self.file_client_args = file_client_args\n if self.out_dir is not None:\n self.file_client = FileClient.infer_client(file_client_args,\n self.out_dir)\n\n def before_run(self, runner):\n super(TextLoggerHook, self).before_run(runner)\n\n if self.out_dir is not None:\n self.file_client = FileClient.infer_client(self.file_client_args,\n self.out_dir)\n # The final `self.out_dir` is the concatenation of `self.out_dir`\n # and the last level directory of `runner.work_dir`\n basename = osp.basename(runner.work_dir.rstrip(osp.sep))\n self.out_dir = self.file_client.join_path(self.out_dir, basename)\n print_log(\n (f'Text logs will be saved to {self.out_dir} by '\n f'{self.file_client.name} after the training process.'), logger=runner.logger)\n\n self.start_iter = runner.iter\n self.data_length = runner.data_length\n self.max_epochs = runner.max_epochs\n self.json_log_path = osp.join(runner.work_dir,\n f'{runner.timestamp}.log.json')\n if runner.meta is not None:\n self._dump_log(runner.meta, runner)\n\n def _get_max_memory(self, runner):\n device = getattr(runner.model, 'output_device', None)\n mem = torch.cuda.max_memory_allocated(device=device)\n mem_mb = torch.tensor([mem / (1024 * 1024)],\n dtype=torch.int,\n device=device)\n if runner.world_size > 1:\n dist.reduce(mem_mb, 0, op=dist.ReduceOp.MAX)\n return mem_mb.item()\n\n def _log_info(self, log_dict, runner):\n # print exp name for users to distinguish experiments\n # at every ``interval_exp_name`` iterations and the end of each epoch\n if runner.meta is not None and 'exp_name' in runner.meta:\n if (self.every_n_iters(runner, self.interval_exp_name)) or (\n self.by_epoch and self.end_of_epoch(runner)):\n exp_info = f'Exp name: {runner.meta[\"exp_name\"]}'\n print_log(exp_info, logger=runner.logger)\n\n if log_dict['mode'] == 'train':\n if isinstance(log_dict['lr'], dict):\n lr_str = []\n for k, val in log_dict['lr'].items():\n lr_str.append(f'lr_{k}: {val:.3e}')\n lr_str = ' '.join(lr_str)\n else:\n lr_str = f'lr: {log_dict[\"lr\"]:.3e}'\n\n # by epoch: Epoch [4][100/1000]\n # by iter: Iter [100/100000]\n if self.by_epoch:\n log_str = f'Epoch [{log_dict[\"epoch\"]}]/[{self.max_epochs}]' \\\n f'[{log_dict[\"iter\"]}/{self.data_length}]\\t'\n else:\n log_str = f'Iter [{log_dict[\"iter\"]}/{runner.max_iters}]\\t'\n log_str += f'{lr_str}, '\n\n if 'time' in log_dict.keys():\n self.time_sec_tot += (log_dict['time'] * self.interval)\n time_sec_avg = self.time_sec_tot / (\n runner.iter - self.start_iter + 1) #\n eta_sec = time_sec_avg * (runner.max_iters - runner.iter - 1)\n eta_str = str(datetime.timedelta(seconds=int(eta_sec)))\n log_str += f'eta: {eta_str}, '\n log_str += f'time: {log_dict[\"time\"]:.3f}, ' \\\n f'data_time: {log_dict[\"data_time\"]:.3f}, '\n # statistic memory\n if torch.cuda.is_available():\n log_str += f'memory: {log_dict[\"memory\"]}MB, '\n else:\n # val/test time\n # here 1000 is the length of the val dataloader\n # by epoch: Epoch[val] [4][1000]\n # by iter: Iter[val] [1000]\n if self.by_epoch:\n log_str = f'Epoch({log_dict[\"mode\"]}) ' \\\n f'[{log_dict[\"epoch\"]}][{log_dict[\"iter\"]}]\\t'\n else:\n log_str = f'Iter({log_dict[\"mode\"]}) [{log_dict[\"iter\"]}]\\t'\n\n log_items = []\n for name, val in log_dict.items():\n # TODO: resolve this hack\n # these items have been in log_str\n if name in [\n 'mode', 'Epoch', 'iter', 'lr', 'time', 'data_time',\n 'memory', 'epoch'\n ]:\n continue\n if isinstance(val, float):\n val = f'{val:.5f}'\n log_items.append(f'{name}: {val}')\n log_str += ', '.join(log_items)\n print_log(log_str, logger=runner.logger)\n\n def _dump_log(self, log_dict, runner):\n # dump log in json format\n json_log = OrderedDict()\n for k, v in log_dict.items():\n json_log[k] = self._round_float(v)\n # only append log at last line\n if runner.rank == 0:\n with open(self.json_log_path, 'a+') as f:\n mmcv.dump(json_log, f, file_format='json')\n f.write('\\n')\n\n def _round_float(self, items):\n if isinstance(items, list):\n return [self._round_float(item) for item in items]\n elif isinstance(items, float):\n return round(items, 5)\n else:\n return items\n\n def log(self, runner):\n if 'eval_iter_num' in runner.log_buffer.meters: #output\n # this doesn't modify runner.iter and is regardless of by_epoch\n cur_iter = runner.log_buffer.meters.pop('eval_iter_num') #output\n else:\n cur_iter = self.get_iter(runner, inner_iter=True)\n\n log_dict = OrderedDict(\n mode=self.get_mode(runner),\n epoch=self.get_epoch(runner),\n iter=cur_iter)\n\n # only record lr of the first param group\n cur_lr = runner.current_lr()\n if isinstance(cur_lr, list):\n log_dict['lr'] = cur_lr[0]\n else:\n assert isinstance(cur_lr, dict)\n log_dict['lr'] = {}\n for k, lr_ in cur_lr.items():\n assert isinstance(lr_, list)\n log_dict['lr'].update({k: lr_[0]})\n\n if 'time' in runner.log_buffer.meters:#output\n # statistic memory\n if torch.cuda.is_available():\n log_dict['memory'] = self._get_max_memory(runner)\n\n runner.metrics = {k: meter.avg for k, meter in runner.log_buffer.meters.items()}\n log_dict = dict(log_dict, **runner.metrics) #output\n\n self._log_info(log_dict, runner)\n self._dump_log(log_dict, runner)\n return log_dict\n\n def after_run(self, runner):\n # copy or upload logs to self.out_dir\n if self.out_dir is not None:\n for filename in scandir(runner.work_dir, self.out_suffix, True):\n local_filepath = osp.join(runner.work_dir, filename)\n out_filepath = self.file_client.join_path(\n self.out_dir, filename)\n with open(local_filepath, 'r') as f:\n self.file_client.put_text(f.read(), out_filepath)\n\n print_log(\n (f'The file {local_filepath} has been uploaded to '\n f'{out_filepath}.'), logger=runner.logger)\n\n if not self.keep_local:\n os.remove(local_filepath)\n print_log(\n (f'{local_filepath} was removed due to the '\n '`self.keep_local=False`'), logger=runner.logger)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/logger/wandb.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\n\nfrom mmcv.utils import scandir\nfrom ...dist_utils import master_only\nfrom ..hook import HOOKS\nfrom .base import LoggerHook\n\n\n@HOOKS.register_module()\nclass WandbLoggerHook(LoggerHook):\n \"\"\"Class to log metrics with wandb.\n\n It requires `wandb`_ to be installed.\n\n\n Args:\n init_kwargs (dict): A dict contains the initialization keys. Check\n https://docs.wandb.ai/ref/python/init for more init arguments.\n interval (int): Logging interval (every k iterations).\n Default 10.\n ignore_last (bool): Ignore the log of last iterations in each epoch\n if less than `interval`.\n Default: True.\n reset_flag (bool): Whether to clear the output buffer after logging.\n Default: False.\n commit (bool): Save the metrics dict to the wandb server and increment\n the step. If false ``wandb.log`` just updates the current metrics\n dict with the row argument and metrics won't be saved until\n ``wandb.log`` is called with ``commit=True``.\n Default: True.\n by_epoch (bool): Whether EpochBasedRunner is used.\n Default: True.\n with_step (bool): If True, the step will be logged from\n ``self.get_iters``. Otherwise, step will not be logged.\n Default: True.\n log_artifact (bool): If True, artifacts in {work_dir} will be uploaded\n to wandb after training ends.\n Default: True\n `New in version 1.4.3.`\n out_suffix (str or tuple[str], optional): Those filenames ending with\n ``out_suffix`` will be uploaded to wandb.\n Default: ('.log.json', '.log', '.py').\n `New in version 1.4.3.`\n\n .. _wandb:\n https://docs.wandb.ai\n \"\"\"\n\n def __init__(self,\n init_kwargs=None,\n interval=10,\n ignore_last=True,\n reset_flag=False,\n commit=True,\n by_epoch=True,\n with_step=True,\n log_artifact=True,\n out_suffix=('.log.json', '.log', '.py')):\n super(WandbLoggerHook, self).__init__(interval, ignore_last,\n reset_flag, by_epoch)\n self.import_wandb()\n self.init_kwargs = init_kwargs\n self.commit = commit\n self.with_step = with_step\n self.log_artifact = log_artifact\n self.out_suffix = out_suffix\n\n def import_wandb(self):\n try:\n import wandb\n except ImportError:\n raise ImportError(\n 'Please run \"pip install wandb\" to install wandb')\n self.wandb = wandb\n\n @master_only\n def before_run(self, runner):\n super(WandbLoggerHook, self).before_run(runner)\n if self.wandb is None:\n self.import_wandb()\n if self.init_kwargs:\n self.wandb.init(**self.init_kwargs)\n else:\n self.wandb.init()\n\n @master_only\n def log(self, runner):\n tags = self.get_loggable_tags(runner)\n if tags:\n if self.with_step:\n self.wandb.log(\n tags, step=self.get_iter(runner), commit=self.commit)\n else:\n tags['global_step'] = self.get_iter(runner)\n self.wandb.log(tags, commit=self.commit)\n\n @master_only\n def after_run(self, runner):\n if self.log_artifact:\n wandb_artifact = self.wandb.Artifact(\n name='artifacts', type='model')\n for filename in scandir(runner.work_dir, self.out_suffix, True):\n local_filepath = osp.join(runner.work_dir, filename)\n wandb_artifact.add_file(local_filepath)\n self.wandb.log_artifact(wandb_artifact)\n self.wandb.join()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/lr_updater.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numbers\nfrom math import cos, pi\n\nimport mmcv\nfrom .hook import HOOKS, Hook\n\n\nclass LrUpdaterHook(Hook):\n \"\"\"LR Scheduler in MMCV.\n\n Args:\n by_epoch (bool): LR changes epoch by epoch\n warmup (string): Type of warmup used. It can be None(use no warmup),\n 'constant', 'linear' or 'exp'\n warmup_iters (int): The number of iterations or epochs that warmup\n lasts\n warmup_ratio (float): LR used at the beginning of warmup equals to\n warmup_ratio * initial_lr\n warmup_by_epoch (bool): When warmup_by_epoch == True, warmup_iters\n means the number of epochs that warmup lasts, otherwise means the\n number of iteration that warmup lasts\n \"\"\"\n\n def __init__(self,\n by_epoch=True,\n warmup=None,\n warmup_iters=0,\n warmup_ratio=0.1,\n warmup_by_epoch=False):\n # validate the \"warmup\" argument\n if warmup is not None:\n if warmup not in ['constant', 'linear', 'exp']:\n raise ValueError(\n f'\"{warmup}\" is not a supported type for warming up, valid'\n ' types are \"constant\" and \"linear\"')\n if warmup is not None:\n assert warmup_iters > 0, \\\n '\"warmup_iters\" must be a positive integer'\n assert 0 < warmup_ratio <= 1.0, \\\n '\"warmup_ratio\" must be in range (0,1]'\n\n self.by_epoch = by_epoch\n self.warmup = warmup\n self.warmup_iters = warmup_iters\n self.warmup_ratio = warmup_ratio\n self.warmup_by_epoch = warmup_by_epoch\n\n if self.warmup_by_epoch:\n self.warmup_epochs = self.warmup_iters\n self.warmup_iters = None\n else:\n self.warmup_epochs = None\n\n self.base_lr = [] # initial lr for all param groups\n self.regular_lr = [] # expected lr if no warming up is performed\n\n def _set_lr(self, runner, lr_groups):\n if isinstance(runner.optimizer, dict):\n for k, optim in runner.optimizer.items():\n for param_group, lr in zip(optim.param_groups, lr_groups[k]):\n param_group['lr'] = lr\n else:\n for param_group, lr in zip(runner.optimizer.param_groups,\n lr_groups):\n param_group['lr'] = lr\n\n def get_lr(self, runner, base_lr):\n raise NotImplementedError\n\n def get_regular_lr(self, runner):\n if isinstance(runner.optimizer, dict):\n lr_groups = {}\n for k in runner.optimizer.keys():\n _lr_group = [\n self.get_lr(runner, _base_lr)\n for _base_lr in self.base_lr[k]\n ]\n lr_groups.update({k: _lr_group})\n\n return lr_groups\n else:\n return [self.get_lr(runner, _base_lr) for _base_lr in self.base_lr]\n\n def get_warmup_lr(self, cur_iters):\n\n def _get_warmup_lr(cur_iters, regular_lr):\n if self.warmup == 'constant':\n warmup_lr = [_lr * self.warmup_ratio for _lr in regular_lr]\n elif self.warmup == 'linear':\n k = (1 - cur_iters / self.warmup_iters) * (1 -\n self.warmup_ratio)\n warmup_lr = [_lr * (1 - k) for _lr in regular_lr]\n elif self.warmup == 'exp':\n k = self.warmup_ratio**(1 - cur_iters / self.warmup_iters)\n warmup_lr = [_lr * k for _lr in regular_lr]\n return warmup_lr\n\n if isinstance(self.regular_lr, dict):\n lr_groups = {}\n for key, regular_lr in self.regular_lr.items():\n lr_groups[key] = _get_warmup_lr(cur_iters, regular_lr)\n return lr_groups\n else:\n return _get_warmup_lr(cur_iters, self.regular_lr)\n\n def before_run(self, runner):\n # NOTE: when resuming from a checkpoint, if 'initial_lr' is not saved,\n # it will be set according to the optimizer params\n if isinstance(runner.optimizer, dict):\n self.base_lr = {}\n for k, optim in runner.optimizer.items():\n for group in optim.param_groups:\n group.setdefault('initial_lr', group['lr'])\n _base_lr = [\n group['initial_lr'] for group in optim.param_groups\n ]\n self.base_lr.update({k: _base_lr})\n else:\n for group in runner.optimizer.param_groups:\n group.setdefault('initial_lr', group['lr'])\n self.base_lr = [\n group['initial_lr'] for group in runner.optimizer.param_groups\n ]\n\n def before_train_epoch(self, runner):\n if self.warmup_iters is None:\n epoch_len = len(runner.data_loader)\n self.warmup_iters = self.warmup_epochs * epoch_len\n\n if not self.by_epoch:\n return\n\n self.regular_lr = self.get_regular_lr(runner)\n self._set_lr(runner, self.regular_lr)\n\n def before_train_iter(self, runner):\n cur_iter = runner.iter\n if not self.by_epoch:\n self.regular_lr = self.get_regular_lr(runner)\n if self.warmup is None or cur_iter >= self.warmup_iters:\n self._set_lr(runner, self.regular_lr)\n else:\n warmup_lr = self.get_warmup_lr(cur_iter)\n self._set_lr(runner, warmup_lr)\n elif self.by_epoch:\n if self.warmup is None or cur_iter > self.warmup_iters:\n return\n elif cur_iter == self.warmup_iters:\n self._set_lr(runner, self.regular_lr)\n else:\n warmup_lr = self.get_warmup_lr(cur_iter)\n self._set_lr(runner, warmup_lr)\n\n\n@HOOKS.register_module()\nclass FixedLrUpdaterHook(LrUpdaterHook):\n\n def __init__(self, **kwargs):\n super(FixedLrUpdaterHook, self).__init__(**kwargs)\n\n def get_lr(self, runner, base_lr):\n return base_lr\n\n\n@HOOKS.register_module()\nclass StepLrUpdaterHook(LrUpdaterHook):\n \"\"\"Step LR scheduler with min_lr clipping.\n\n Args:\n step (int | list[int]): Step to decay the LR. If an int value is given,\n regard it as the decay interval. If a list is given, decay LR at\n these steps.\n gamma (float, optional): Decay LR ratio. Default: 0.1.\n min_lr (float, optional): Minimum LR value to keep. If LR after decay\n is lower than `min_lr`, it will be clipped to this value. If None\n is given, we don't perform lr clipping. Default: None.\n \"\"\"\n\n def __init__(self, step, gamma=0.1, min_lr=None, **kwargs):\n if isinstance(step, list):\n assert mmcv.is_list_of(step, int)\n assert all([s > 0 for s in step])\n elif isinstance(step, int):\n assert step > 0\n else:\n raise TypeError('\"step\" must be a list or integer')\n self.step = step\n self.gamma = gamma\n self.min_lr = min_lr\n super(StepLrUpdaterHook, self).__init__(**kwargs)\n\n def get_lr(self, runner, base_lr):\n progress = runner.epoch if self.by_epoch else runner.iter\n\n # calculate exponential term\n if isinstance(self.step, int):\n exp = progress // self.step\n else:\n exp = len(self.step)\n for i, s in enumerate(self.step):\n if progress < s:\n exp = i\n break\n\n lr = base_lr * (self.gamma**exp)\n if self.min_lr is not None:\n # clip to a minimum value\n lr = max(lr, self.min_lr)\n return lr\n\n\n@HOOKS.register_module()\nclass ExpLrUpdaterHook(LrUpdaterHook):\n\n def __init__(self, gamma, **kwargs):\n self.gamma = gamma\n super(ExpLrUpdaterHook, self).__init__(**kwargs)\n\n def get_lr(self, runner, base_lr):\n progress = runner.epoch if self.by_epoch else runner.iter\n return base_lr * self.gamma**progress\n\n\n@HOOKS.register_module()\nclass PolyLrUpdaterHook(LrUpdaterHook):\n\n def __init__(self, power=1., min_lr=0., **kwargs):\n self.power = power\n self.min_lr = min_lr\n super(PolyLrUpdaterHook, self).__init__(**kwargs)\n\n def get_lr(self, runner, base_lr):\n if self.by_epoch:\n progress = runner.epoch\n max_progress = runner.max_epochs\n else:\n progress = runner.iter\n max_progress = runner.max_iters\n coeff = (1 - progress / max_progress)**self.power\n return (base_lr - self.min_lr) * coeff + self.min_lr\n\n\n@HOOKS.register_module()\nclass InvLrUpdaterHook(LrUpdaterHook):\n\n def __init__(self, gamma, power=1., **kwargs):\n self.gamma = gamma\n self.power = power\n super(InvLrUpdaterHook, self).__init__(**kwargs)\n\n def get_lr(self, runner, base_lr):\n progress = runner.epoch if self.by_epoch else runner.iter\n return base_lr * (1 + self.gamma * progress)**(-self.power)\n\n\n@HOOKS.register_module()\nclass CosineAnnealingLrUpdaterHook(LrUpdaterHook):\n\n def __init__(self, min_lr=None, min_lr_ratio=None, **kwargs):\n assert (min_lr is None) ^ (min_lr_ratio is None)\n self.min_lr = min_lr\n self.min_lr_ratio = min_lr_ratio\n super(CosineAnnealingLrUpdaterHook, self).__init__(**kwargs)\n\n def get_lr(self, runner, base_lr):\n if self.by_epoch:\n progress = runner.epoch\n max_progress = runner.max_epochs\n else:\n progress = runner.iter\n max_progress = runner.max_iters\n\n if self.min_lr_ratio is not None:\n target_lr = base_lr * self.min_lr_ratio\n else:\n target_lr = self.min_lr\n return annealing_cos(base_lr, target_lr, progress / max_progress)\n\n\n@HOOKS.register_module()\nclass FlatCosineAnnealingLrUpdaterHook(LrUpdaterHook):\n \"\"\"Flat + Cosine lr schedule.\n\n Modified from https://github.com/fastai/fastai/blob/master/fastai/callback/schedule.py#L128 # noqa: E501\n\n Args:\n start_percent (float): When to start annealing the learning rate\n after the percentage of the total training steps.\n The value should be in range [0, 1).\n Default: 0.75\n min_lr (float, optional): The minimum lr. Default: None.\n min_lr_ratio (float, optional): The ratio of minimum lr to the base lr.\n Either `min_lr` or `min_lr_ratio` should be specified.\n Default: None.\n \"\"\"\n\n def __init__(self,\n start_percent=0.75,\n min_lr=None,\n min_lr_ratio=None,\n **kwargs):\n assert (min_lr is None) ^ (min_lr_ratio is None)\n if start_percent < 0 or start_percent > 1 or not isinstance(\n start_percent, float):\n raise ValueError(\n 'expected float between 0 and 1 start_percent, but '\n f'got {start_percent}')\n self.start_percent = start_percent\n self.min_lr = min_lr\n self.min_lr_ratio = min_lr_ratio\n super(FlatCosineAnnealingLrUpdaterHook, self).__init__(**kwargs)\n\n def get_lr(self, runner, base_lr):\n if self.by_epoch:\n start = round(runner.max_epochs * self.start_percent)\n progress = runner.epoch - start\n max_progress = runner.max_epochs - start\n else:\n start = round(runner.max_iters * self.start_percent)\n progress = runner.iter - start\n max_progress = runner.max_iters - start\n\n if self.min_lr_ratio is not None:\n target_lr = base_lr * self.min_lr_ratio\n else:\n target_lr = self.min_lr\n\n if progress < 0:\n return base_lr\n else:\n return annealing_cos(base_lr, target_lr, progress / max_progress)\n\n\n@HOOKS.register_module()\nclass CosineRestartLrUpdaterHook(LrUpdaterHook):\n \"\"\"Cosine annealing with restarts learning rate scheme.\n\n Args:\n periods (list[int]): Periods for each cosine anneling cycle.\n restart_weights (list[float], optional): Restart weights at each\n restart iteration. Default: [1].\n min_lr (float, optional): The minimum lr. Default: None.\n min_lr_ratio (float, optional): The ratio of minimum lr to the base lr.\n Either `min_lr` or `min_lr_ratio` should be specified.\n Default: None.\n \"\"\"\n\n def __init__(self,\n periods,\n restart_weights=[1],\n min_lr=None,\n min_lr_ratio=None,\n **kwargs):\n assert (min_lr is None) ^ (min_lr_ratio is None)\n self.periods = periods\n self.min_lr = min_lr\n self.min_lr_ratio = min_lr_ratio\n self.restart_weights = restart_weights\n assert (len(self.periods) == len(self.restart_weights)\n ), 'periods and restart_weights should have the same length.'\n super(CosineRestartLrUpdaterHook, self).__init__(**kwargs)\n\n self.cumulative_periods = [\n sum(self.periods[0:i + 1]) for i in range(0, len(self.periods))\n ]\n\n def get_lr(self, runner, base_lr):\n if self.by_epoch:\n progress = runner.epoch\n else:\n progress = runner.iter\n\n if self.min_lr_ratio is not None:\n target_lr = base_lr * self.min_lr_ratio\n else:\n target_lr = self.min_lr\n\n idx = get_position_from_periods(progress, self.cumulative_periods)\n current_weight = self.restart_weights[idx]\n nearest_restart = 0 if idx == 0 else self.cumulative_periods[idx - 1]\n current_periods = self.periods[idx]\n\n alpha = min((progress - nearest_restart) / current_periods, 1)\n return annealing_cos(base_lr, target_lr, alpha, current_weight)\n\n\ndef get_position_from_periods(iteration, cumulative_periods):\n \"\"\"Get the position from a period list.\n\n It will return the index of the right-closest number in the period list.\n For example, the cumulative_periods = [100, 200, 300, 400],\n if iteration == 50, return 0;\n if iteration == 210, return 2;\n if iteration == 300, return 3.\n\n Args:\n iteration (int): Current iteration.\n cumulative_periods (list[int]): Cumulative period list.\n\n Returns:\n int: The position of the right-closest number in the period list.\n \"\"\"\n for i, period in enumerate(cumulative_periods):\n if iteration < period:\n return i\n raise ValueError(f'Current iteration {iteration} exceeds '\n f'cumulative_periods {cumulative_periods}')\n\n\n@HOOKS.register_module()\nclass CyclicLrUpdaterHook(LrUpdaterHook):\n \"\"\"Cyclic LR Scheduler.\n\n Implement the cyclical learning rate policy (CLR) described in\n https://arxiv.org/pdf/1506.01186.pdf\n\n Different from the original paper, we use cosine annealing rather than\n triangular policy inside a cycle. This improves the performance in the\n 3D detection area.\n\n Args:\n by_epoch (bool, optional): Whether to update LR by epoch.\n target_ratio (tuple[float], optional): Relative ratio of the highest LR\n and the lowest LR to the initial LR.\n cyclic_times (int, optional): Number of cycles during training\n step_ratio_up (float, optional): The ratio of the increasing process of\n LR in the total cycle.\n anneal_strategy (str, optional): {'cos', 'linear'}\n Specifies the annealing strategy: 'cos' for cosine annealing,\n 'linear' for linear annealing. Default: 'cos'.\n gamma (float, optional): Cycle decay ratio. Default: 1.\n It takes values in the range (0, 1]. The difference between the\n maximum learning rate and the minimum learning rate decreases\n periodically when it is less than 1. `New in version 1.4.4.`\n \"\"\"\n\n def __init__(self,\n by_epoch=False,\n target_ratio=(10, 1e-4),\n cyclic_times=1,\n step_ratio_up=0.4,\n anneal_strategy='cos',\n gamma=1,\n **kwargs):\n if isinstance(target_ratio, float):\n target_ratio = (target_ratio, target_ratio / 1e5)\n elif isinstance(target_ratio, tuple):\n target_ratio = (target_ratio[0], target_ratio[0] / 1e5) \\\n if len(target_ratio) == 1 else target_ratio\n else:\n raise ValueError('target_ratio should be either float '\n f'or tuple, got {type(target_ratio)}')\n\n assert len(target_ratio) == 2, \\\n '\"target_ratio\" must be list or tuple of two floats'\n assert 0 <= step_ratio_up < 1.0, \\\n '\"step_ratio_up\" must be in range [0,1)'\n assert 0 < gamma <= 1, \\\n '\"gamma\" must be in range (0, 1]'\n\n self.target_ratio = target_ratio\n self.cyclic_times = cyclic_times\n self.step_ratio_up = step_ratio_up\n self.gamma = gamma\n self.max_iter_per_phase = None\n self.lr_phases = [] # init lr_phases\n # validate anneal_strategy\n if anneal_strategy not in ['cos', 'linear']:\n raise ValueError('anneal_strategy must be one of \"cos\" or '\n f'\"linear\", instead got {anneal_strategy}')\n elif anneal_strategy == 'cos':\n self.anneal_func = annealing_cos\n elif anneal_strategy == 'linear':\n self.anneal_func = annealing_linear\n\n assert not by_epoch, \\\n 'currently only support \"by_epoch\" = False'\n super(CyclicLrUpdaterHook, self).__init__(by_epoch, **kwargs)\n\n def before_run(self, runner):\n super(CyclicLrUpdaterHook, self).before_run(runner)\n # initiate lr_phases\n # total lr_phases are separated as up and down\n self.max_iter_per_phase = runner.max_iters // self.cyclic_times\n iter_up_phase = int(self.step_ratio_up * self.max_iter_per_phase)\n self.lr_phases.append([0, iter_up_phase, 1, self.target_ratio[0]])\n self.lr_phases.append([\n iter_up_phase, self.max_iter_per_phase, self.target_ratio[0],\n self.target_ratio[1]\n ])\n\n def get_lr(self, runner, base_lr):\n curr_iter = runner.iter % self.max_iter_per_phase\n curr_cycle = runner.iter // self.max_iter_per_phase\n # Update weight decay\n scale = self.gamma**curr_cycle\n\n for (start_iter, end_iter, start_ratio, end_ratio) in self.lr_phases:\n if start_iter <= curr_iter < end_iter:\n # Apply cycle scaling to gradually reduce the difference\n # between max_lr and base lr. The target end_ratio can be\n # expressed as:\n # end_ratio = (base_lr + scale * (max_lr - base_lr)) / base_lr\n # iteration: 0-iter_up_phase:\n if start_iter == 0:\n end_ratio = 1 - scale + end_ratio * scale\n # iteration: iter_up_phase-self.max_iter_per_phase\n else:\n start_ratio = 1 - scale + start_ratio * scale\n progress = curr_iter - start_iter\n return self.anneal_func(base_lr * start_ratio,\n base_lr * end_ratio,\n progress / (end_iter - start_iter))\n\n\n@HOOKS.register_module()\nclass OneCycleLrUpdaterHook(LrUpdaterHook):\n \"\"\"One Cycle LR Scheduler.\n\n The 1cycle learning rate policy changes the learning rate after every\n batch. The one cycle learning rate policy is described in\n https://arxiv.org/pdf/1708.07120.pdf\n\n Args:\n max_lr (float or list): Upper learning rate boundaries in the cycle\n for each parameter group.\n total_steps (int, optional): The total number of steps in the cycle.\n Note that if a value is not provided here, it will be the max_iter\n of runner. Default: None.\n pct_start (float): The percentage of the cycle (in number of steps)\n spent increasing the learning rate.\n Default: 0.3\n anneal_strategy (str): {'cos', 'linear'}\n Specifies the annealing strategy: 'cos' for cosine annealing,\n 'linear' for linear annealing.\n Default: 'cos'\n div_factor (float): Determines the initial learning rate via\n initial_lr = max_lr/div_factor\n Default: 25\n final_div_factor (float): Determines the minimum learning rate via\n min_lr = initial_lr/final_div_factor\n Default: 1e4\n three_phase (bool): If three_phase is True, use a third phase of the\n schedule to annihilate the learning rate according to\n final_div_factor instead of modifying the second phase (the first\n two phases will be symmetrical about the step indicated by\n pct_start).\n Default: False\n \"\"\"\n\n def __init__(self,\n max_lr,\n total_steps=None,\n pct_start=0.3,\n anneal_strategy='cos',\n div_factor=25,\n final_div_factor=1e4,\n three_phase=False,\n **kwargs):\n # validate by_epoch, currently only support by_epoch = False\n if 'by_epoch' not in kwargs:\n kwargs['by_epoch'] = False\n else:\n assert not kwargs['by_epoch'], \\\n 'currently only support \"by_epoch\" = False'\n if not isinstance(max_lr, (numbers.Number, list, dict)):\n raise ValueError('the type of max_lr must be the one of list or '\n f'dict, but got {type(max_lr)}')\n self._max_lr = max_lr\n if total_steps is not None:\n if not isinstance(total_steps, int):\n raise ValueError('the type of total_steps must be int, but'\n f'got {type(total_steps)}')\n self.total_steps = total_steps\n # validate pct_start\n if pct_start < 0 or pct_start > 1 or not isinstance(pct_start, float):\n raise ValueError('expected float between 0 and 1 pct_start, but '\n f'got {pct_start}')\n self.pct_start = pct_start\n # validate anneal_strategy\n if anneal_strategy not in ['cos', 'linear']:\n raise ValueError('anneal_strategy must be one of \"cos\" or '\n f'\"linear\", instead got {anneal_strategy}')\n elif anneal_strategy == 'cos':\n self.anneal_func = annealing_cos\n elif anneal_strategy == 'linear':\n self.anneal_func = annealing_linear\n self.div_factor = div_factor\n self.final_div_factor = final_div_factor\n self.three_phase = three_phase\n self.lr_phases = [] # init lr_phases\n super(OneCycleLrUpdaterHook, self).__init__(**kwargs)\n\n def before_run(self, runner):\n if hasattr(self, 'total_steps'):\n total_steps = self.total_steps\n else:\n total_steps = runner.max_iters\n if total_steps < runner.max_iters:\n raise ValueError(\n 'The total steps must be greater than or equal to max '\n f'iterations {runner.max_iters} of runner, but total steps '\n f'is {total_steps}.')\n\n if isinstance(runner.optimizer, dict):\n self.base_lr = {}\n for k, optim in runner.optimizer.items():\n _max_lr = format_param(k, optim, self._max_lr)\n self.base_lr[k] = [lr / self.div_factor for lr in _max_lr]\n for group, lr in zip(optim.param_groups, self.base_lr[k]):\n group.setdefault('initial_lr', lr)\n else:\n k = type(runner.optimizer).__name__\n _max_lr = format_param(k, runner.optimizer, self._max_lr)\n self.base_lr = [lr / self.div_factor for lr in _max_lr]\n for group, lr in zip(runner.optimizer.param_groups, self.base_lr):\n group.setdefault('initial_lr', lr)\n\n if self.three_phase:\n self.lr_phases.append(\n [float(self.pct_start * total_steps) - 1, 1, self.div_factor])\n self.lr_phases.append([\n float(2 * self.pct_start * total_steps) - 2, self.div_factor, 1\n ])\n self.lr_phases.append(\n [total_steps - 1, 1, 1 / self.final_div_factor])\n else:\n self.lr_phases.append(\n [float(self.pct_start * total_steps) - 1, 1, self.div_factor])\n self.lr_phases.append(\n [total_steps - 1, self.div_factor, 1 / self.final_div_factor])\n\n def get_lr(self, runner, base_lr):\n curr_iter = runner.iter\n start_iter = 0\n for i, (end_iter, start_lr, end_lr) in enumerate(self.lr_phases):\n if curr_iter <= end_iter:\n pct = (curr_iter - start_iter) / (end_iter - start_iter)\n lr = self.anneal_func(base_lr * start_lr, base_lr * end_lr,\n pct)\n break\n start_iter = end_iter\n return lr\n\n\ndef annealing_cos(start, end, factor, weight=1):\n \"\"\"Calculate annealing cos learning rate.\n\n Cosine anneal from `weight * start + (1 - weight) * end` to `end` as\n percentage goes from 0.0 to 1.0.\n\n Args:\n start (float): The starting learning rate of the cosine annealing.\n end (float): The ending learing rate of the cosine annealing.\n factor (float): The coefficient of `pi` when calculating the current\n percentage. Range from 0.0 to 1.0.\n weight (float, optional): The combination factor of `start` and `end`\n when calculating the actual starting learning rate. Default to 1.\n \"\"\"\n cos_out = cos(pi * factor) + 1\n return end + 0.5 * weight * (start - end) * cos_out\n\n\ndef annealing_linear(start, end, factor):\n \"\"\"Calculate annealing linear learning rate.\n\n Linear anneal from `start` to `end` as percentage goes from 0.0 to 1.0.\n\n Args:\n start (float): The starting learning rate of the linear annealing.\n end (float): The ending learing rate of the linear annealing.\n factor (float): The coefficient of `pi` when calculating the current\n percentage. Range from 0.0 to 1.0.\n \"\"\"\n return start + (end - start) * factor\n\n\ndef format_param(name, optim, param):\n if isinstance(param, numbers.Number):\n return [param] * len(optim.param_groups)\n elif isinstance(param, (list, tuple)): # multi param groups\n if len(param) != len(optim.param_groups):\n raise ValueError(f'expected {len(optim.param_groups)} '\n f'values for {name}, got {len(param)}')\n return param\n else: # multi optimizers\n if name not in param:\n raise KeyError(f'{name} is not found in {param.keys()}')\n return param[name]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/memory.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\n\nfrom .hook import HOOKS, Hook\n\n\n@HOOKS.register_module()\nclass EmptyCacheHook(Hook):\n\n def __init__(self, before_epoch=False, after_epoch=True, after_iter=False):\n self._before_epoch = before_epoch\n self._after_epoch = after_epoch\n self._after_iter = after_iter\n\n def after_iter(self, runner):\n if self._after_iter:\n torch.cuda.empty_cache()\n\n def before_epoch(self, runner):\n if self._before_epoch:\n torch.cuda.empty_cache()\n\n def after_epoch(self, runner):\n if self._after_epoch:\n torch.cuda.empty_cache()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/momentum_updater.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport mmcv\nfrom .hook import HOOKS, Hook\nfrom .lr_updater import annealing_cos, annealing_linear, format_param\n\n\nclass MomentumUpdaterHook(Hook):\n\n def __init__(self,\n by_epoch=True,\n warmup=None,\n warmup_iters=0,\n warmup_ratio=0.9):\n # validate the \"warmup\" argument\n if warmup is not None:\n if warmup not in ['constant', 'linear', 'exp']:\n raise ValueError(\n f'\"{warmup}\" is not a supported type for warming up, valid'\n ' types are \"constant\" and \"linear\"')\n if warmup is not None:\n assert warmup_iters > 0, \\\n '\"warmup_iters\" must be a positive integer'\n assert 0 < warmup_ratio <= 1.0, \\\n '\"warmup_momentum\" must be in range (0,1]'\n\n self.by_epoch = by_epoch\n self.warmup = warmup\n self.warmup_iters = warmup_iters\n self.warmup_ratio = warmup_ratio\n\n self.base_momentum = [] # initial momentum for all param groups\n self.regular_momentum = [\n ] # expected momentum if no warming up is performed\n\n def _set_momentum(self, runner, momentum_groups):\n if isinstance(runner.optimizer, dict):\n for k, optim in runner.optimizer.items():\n for param_group, mom in zip(optim.param_groups,\n momentum_groups[k]):\n if 'momentum' in param_group.keys():\n param_group['momentum'] = mom\n elif 'betas' in param_group.keys():\n param_group['betas'] = (mom, param_group['betas'][1])\n else:\n for param_group, mom in zip(runner.optimizer.param_groups,\n momentum_groups):\n if 'momentum' in param_group.keys():\n param_group['momentum'] = mom\n elif 'betas' in param_group.keys():\n param_group['betas'] = (mom, param_group['betas'][1])\n\n def get_momentum(self, runner, base_momentum):\n raise NotImplementedError\n\n def get_regular_momentum(self, runner):\n if isinstance(runner.optimizer, dict):\n momentum_groups = {}\n for k in runner.optimizer.keys():\n _momentum_group = [\n self.get_momentum(runner, _base_momentum)\n for _base_momentum in self.base_momentum[k]\n ]\n momentum_groups.update({k: _momentum_group})\n return momentum_groups\n else:\n return [\n self.get_momentum(runner, _base_momentum)\n for _base_momentum in self.base_momentum\n ]\n\n def get_warmup_momentum(self, cur_iters):\n\n def _get_warmup_momentum(cur_iters, regular_momentum):\n if self.warmup == 'constant':\n warmup_momentum = [\n _momentum / self.warmup_ratio\n for _momentum in regular_momentum\n ]\n elif self.warmup == 'linear':\n k = (1 - cur_iters / self.warmup_iters) * (1 -\n self.warmup_ratio)\n warmup_momentum = [\n _momentum / (1 - k) for _momentum in regular_momentum\n ]\n elif self.warmup == 'exp':\n k = self.warmup_ratio**(1 - cur_iters / self.warmup_iters)\n warmup_momentum = [\n _momentum / k for _momentum in regular_momentum\n ]\n return warmup_momentum\n\n if isinstance(self.regular_momentum, dict):\n momentum_groups = {}\n for key, regular_momentum in self.regular_momentum.items():\n momentum_groups[key] = _get_warmup_momentum(\n cur_iters, regular_momentum)\n return momentum_groups\n else:\n return _get_warmup_momentum(cur_iters, self.regular_momentum)\n\n def before_run(self, runner):\n # NOTE: when resuming from a checkpoint,\n # if 'initial_momentum' is not saved,\n # it will be set according to the optimizer params\n if isinstance(runner.optimizer, dict):\n self.base_momentum = {}\n for k, optim in runner.optimizer.items():\n for group in optim.param_groups:\n if 'momentum' in group.keys():\n group.setdefault('initial_momentum', group['momentum'])\n else:\n group.setdefault('initial_momentum', group['betas'][0])\n _base_momentum = [\n group['initial_momentum'] for group in optim.param_groups\n ]\n self.base_momentum.update({k: _base_momentum})\n else:\n for group in runner.optimizer.param_groups:\n if 'momentum' in group.keys():\n group.setdefault('initial_momentum', group['momentum'])\n else:\n group.setdefault('initial_momentum', group['betas'][0])\n self.base_momentum = [\n group['initial_momentum']\n for group in runner.optimizer.param_groups\n ]\n\n def before_train_epoch(self, runner):\n if not self.by_epoch:\n return\n self.regular_momentum = self.get_regular_momentum(runner)\n self._set_momentum(runner, self.regular_momentum)\n\n def before_train_iter(self, runner):\n cur_iter = runner.iter\n if not self.by_epoch:\n self.regular_momentum = self.get_regular_momentum(runner)\n if self.warmup is None or cur_iter >= self.warmup_iters:\n self._set_momentum(runner, self.regular_momentum)\n else:\n warmup_momentum = self.get_warmup_momentum(cur_iter)\n self._set_momentum(runner, warmup_momentum)\n elif self.by_epoch:\n if self.warmup is None or cur_iter > self.warmup_iters:\n return\n elif cur_iter == self.warmup_iters:\n self._set_momentum(runner, self.regular_momentum)\n else:\n warmup_momentum = self.get_warmup_momentum(cur_iter)\n self._set_momentum(runner, warmup_momentum)\n\n\n@HOOKS.register_module()\nclass StepMomentumUpdaterHook(MomentumUpdaterHook):\n \"\"\"Step momentum scheduler with min value clipping.\n\n Args:\n step (int | list[int]): Step to decay the momentum. If an int value is\n given, regard it as the decay interval. If a list is given, decay\n momentum at these steps.\n gamma (float, optional): Decay momentum ratio. Default: 0.5.\n min_momentum (float, optional): Minimum momentum value to keep. If\n momentum after decay is lower than this value, it will be clipped\n accordingly. If None is given, we don't perform lr clipping.\n Default: None.\n \"\"\"\n\n def __init__(self, step, gamma=0.5, min_momentum=None, **kwargs):\n if isinstance(step, list):\n assert mmcv.is_list_of(step, int)\n assert all([s > 0 for s in step])\n elif isinstance(step, int):\n assert step > 0\n else:\n raise TypeError('\"step\" must be a list or integer')\n self.step = step\n self.gamma = gamma\n self.min_momentum = min_momentum\n super(StepMomentumUpdaterHook, self).__init__(**kwargs)\n\n def get_momentum(self, runner, base_momentum):\n progress = runner.epoch if self.by_epoch else runner.iter\n\n # calculate exponential term\n if isinstance(self.step, int):\n exp = progress // self.step\n else:\n exp = len(self.step)\n for i, s in enumerate(self.step):\n if progress < s:\n exp = i\n break\n\n momentum = base_momentum * (self.gamma**exp)\n if self.min_momentum is not None:\n # clip to a minimum value\n momentum = max(momentum, self.min_momentum)\n return momentum\n\n\n@HOOKS.register_module()\nclass CosineAnnealingMomentumUpdaterHook(MomentumUpdaterHook):\n\n def __init__(self, min_momentum=None, min_momentum_ratio=None, **kwargs):\n assert (min_momentum is None) ^ (min_momentum_ratio is None)\n self.min_momentum = min_momentum\n self.min_momentum_ratio = min_momentum_ratio\n super(CosineAnnealingMomentumUpdaterHook, self).__init__(**kwargs)\n\n def get_momentum(self, runner, base_momentum):\n if self.by_epoch:\n progress = runner.epoch\n max_progress = runner.max_epochs\n else:\n progress = runner.iter\n max_progress = runner.max_iters\n if self.min_momentum_ratio is not None:\n target_momentum = base_momentum * self.min_momentum_ratio\n else:\n target_momentum = self.min_momentum\n return annealing_cos(base_momentum, target_momentum,\n progress / max_progress)\n\n\n@HOOKS.register_module()\nclass CyclicMomentumUpdaterHook(MomentumUpdaterHook):\n \"\"\"Cyclic momentum Scheduler.\n\n Implement the cyclical momentum scheduler policy described in\n https://arxiv.org/pdf/1708.07120.pdf\n\n This momentum scheduler usually used together with the CyclicLRUpdater\n to improve the performance in the 3D detection area.\n\n Args:\n target_ratio (tuple[float]): Relative ratio of the lowest momentum and\n the highest momentum to the initial momentum.\n cyclic_times (int): Number of cycles during training\n step_ratio_up (float): The ratio of the increasing process of momentum\n in the total cycle.\n by_epoch (bool): Whether to update momentum by epoch.\n anneal_strategy (str, optional): {'cos', 'linear'}\n Specifies the annealing strategy: 'cos' for cosine annealing,\n 'linear' for linear annealing. Default: 'cos'.\n gamma (float, optional): Cycle decay ratio. Default: 1.\n It takes values in the range (0, 1]. The difference between the\n maximum learning rate and the minimum learning rate decreases\n periodically when it is less than 1. `New in version 1.4.4.`\n \"\"\"\n\n def __init__(self,\n by_epoch=False,\n target_ratio=(0.85 / 0.95, 1),\n cyclic_times=1,\n step_ratio_up=0.4,\n anneal_strategy='cos',\n gamma=1,\n **kwargs):\n if isinstance(target_ratio, float):\n target_ratio = (target_ratio, target_ratio / 1e5)\n elif isinstance(target_ratio, tuple):\n target_ratio = (target_ratio[0], target_ratio[0] / 1e5) \\\n if len(target_ratio) == 1 else target_ratio\n else:\n raise ValueError('target_ratio should be either float '\n f'or tuple, got {type(target_ratio)}')\n\n assert len(target_ratio) == 2, \\\n '\"target_ratio\" must be list or tuple of two floats'\n assert 0 <= step_ratio_up < 1.0, \\\n '\"step_ratio_up\" must be in range [0,1)'\n\n self.target_ratio = target_ratio\n self.cyclic_times = cyclic_times\n self.step_ratio_up = step_ratio_up\n self.gamma = gamma\n self.momentum_phases = [] # init momentum_phases\n\n if anneal_strategy not in ['cos', 'linear']:\n raise ValueError('anneal_strategy must be one of \"cos\" or '\n f'\"linear\", instead got {anneal_strategy}')\n elif anneal_strategy == 'cos':\n self.anneal_func = annealing_cos\n elif anneal_strategy == 'linear':\n self.anneal_func = annealing_linear\n # currently only support by_epoch=False\n assert not by_epoch, \\\n 'currently only support \"by_epoch\" = False'\n super(CyclicMomentumUpdaterHook, self).__init__(by_epoch, **kwargs)\n\n def before_run(self, runner):\n super(CyclicMomentumUpdaterHook, self).before_run(runner)\n # initiate momentum_phases\n # total momentum_phases are separated as up and down\n max_iter_per_phase = runner.max_iters // self.cyclic_times\n iter_up_phase = int(self.step_ratio_up * max_iter_per_phase)\n self.max_iter_per_phase = max_iter_per_phase\n self.momentum_phases.append(\n [0, iter_up_phase, 1, self.target_ratio[0]])\n self.momentum_phases.append([\n iter_up_phase, max_iter_per_phase, self.target_ratio[0],\n self.target_ratio[1]\n ])\n\n def get_momentum(self, runner, base_momentum):\n curr_iter = runner.iter % self.max_iter_per_phase\n curr_cycle = runner.iter // self.max_iter_per_phase\n scale = self.gamma**curr_cycle\n for (start_iter, end_iter, start_ratio, end_ratio) \\\n in self.momentum_phases:\n if start_iter <= curr_iter < end_iter:\n # Apply cycle scaling to gradually reduce the difference\n # between max_momentum and base momentum. The target end_ratio\n # can be expressed as:\n # end_ratio = (base_momentum + scale * \\\n # (max_momentum - base_momentum)) / base_momentum\n # iteration: 0-iter_up_phase:\n if start_iter == 0:\n end_ratio = 1 - scale + end_ratio * scale\n # iteration: iter_up_phase-self.max_iter_per_phase\n else:\n start_ratio = 1 - scale + start_ratio * scale\n progress = curr_iter - start_iter\n return self.anneal_func(base_momentum * start_ratio,\n base_momentum * end_ratio,\n progress / (end_iter - start_iter))\n\n\n@HOOKS.register_module()\nclass OneCycleMomentumUpdaterHook(MomentumUpdaterHook):\n \"\"\"OneCycle momentum Scheduler.\n\n This momentum scheduler usually used together with the OneCycleLrUpdater\n to improve the performance.\n\n Args:\n base_momentum (float or list): Lower momentum boundaries in the cycle\n for each parameter group. Note that momentum is cycled inversely\n to learning rate; at the peak of a cycle, momentum is\n 'base_momentum' and learning rate is 'max_lr'.\n Default: 0.85\n max_momentum (float or list): Upper momentum boundaries in the cycle\n for each parameter group. Functionally,\n it defines the cycle amplitude (max_momentum - base_momentum).\n Note that momentum is cycled inversely\n to learning rate; at the start of a cycle, momentum is\n 'max_momentum' and learning rate is 'base_lr'\n Default: 0.95\n pct_start (float): The percentage of the cycle (in number of steps)\n spent increasing the learning rate.\n Default: 0.3\n anneal_strategy (str): {'cos', 'linear'}\n Specifies the annealing strategy: 'cos' for cosine annealing,\n 'linear' for linear annealing.\n Default: 'cos'\n three_phase (bool): If three_phase is True, use a third phase of the\n schedule to annihilate the learning rate according to\n final_div_factor instead of modifying the second phase (the first\n two phases will be symmetrical about the step indicated by\n pct_start).\n Default: False\n \"\"\"\n\n def __init__(self,\n base_momentum=0.85,\n max_momentum=0.95,\n pct_start=0.3,\n anneal_strategy='cos',\n three_phase=False,\n **kwargs):\n # validate by_epoch, currently only support by_epoch=False\n if 'by_epoch' not in kwargs:\n kwargs['by_epoch'] = False\n else:\n assert not kwargs['by_epoch'], \\\n 'currently only support \"by_epoch\" = False'\n if not isinstance(base_momentum, (float, list, dict)):\n raise ValueError('base_momentum must be the type among of float,'\n 'list or dict.')\n self._base_momentum = base_momentum\n if not isinstance(max_momentum, (float, list, dict)):\n raise ValueError('max_momentum must be the type among of float,'\n 'list or dict.')\n self._max_momentum = max_momentum\n # validate pct_start\n if pct_start < 0 or pct_start > 1 or not isinstance(pct_start, float):\n raise ValueError('Expected float between 0 and 1 pct_start, but '\n f'got {pct_start}')\n self.pct_start = pct_start\n # validate anneal_strategy\n if anneal_strategy not in ['cos', 'linear']:\n raise ValueError('anneal_strategy must by one of \"cos\" or '\n f'\"linear\", instead got {anneal_strategy}')\n elif anneal_strategy == 'cos':\n self.anneal_func = annealing_cos\n elif anneal_strategy == 'linear':\n self.anneal_func = annealing_linear\n self.three_phase = three_phase\n self.momentum_phases = [] # init momentum_phases\n super(OneCycleMomentumUpdaterHook, self).__init__(**kwargs)\n\n def before_run(self, runner):\n if isinstance(runner.optimizer, dict):\n for k, optim in runner.optimizer.items():\n if ('momentum' not in optim.defaults\n and 'betas' not in optim.defaults):\n raise ValueError('optimizer must support momentum with'\n 'option enabled')\n self.use_beta1 = 'betas' in optim.defaults\n _base_momentum = format_param(k, optim, self._base_momentum)\n _max_momentum = format_param(k, optim, self._max_momentum)\n for group, b_momentum, m_momentum in zip(\n optim.param_groups, _base_momentum, _max_momentum):\n if self.use_beta1:\n _, beta2 = group['betas']\n group['betas'] = (m_momentum, beta2)\n else:\n group['momentum'] = m_momentum\n group['base_momentum'] = b_momentum\n group['max_momentum'] = m_momentum\n else:\n optim = runner.optimizer\n if ('momentum' not in optim.defaults\n and 'betas' not in optim.defaults):\n raise ValueError('optimizer must support momentum with'\n 'option enabled')\n self.use_beta1 = 'betas' in optim.defaults\n k = type(optim).__name__\n _base_momentum = format_param(k, optim, self._base_momentum)\n _max_momentum = format_param(k, optim, self._max_momentum)\n for group, b_momentum, m_momentum in zip(optim.param_groups,\n _base_momentum,\n _max_momentum):\n if self.use_beta1:\n _, beta2 = group['betas']\n group['betas'] = (m_momentum, beta2)\n else:\n group['momentum'] = m_momentum\n group['base_momentum'] = b_momentum\n group['max_momentum'] = m_momentum\n\n if self.three_phase:\n self.momentum_phases.append({\n 'end_iter':\n float(self.pct_start * runner.max_iters) - 1,\n 'start_momentum':\n 'max_momentum',\n 'end_momentum':\n 'base_momentum'\n })\n self.momentum_phases.append({\n 'end_iter':\n float(2 * self.pct_start * runner.max_iters) - 2,\n 'start_momentum':\n 'base_momentum',\n 'end_momentum':\n 'max_momentum'\n })\n self.momentum_phases.append({\n 'end_iter': runner.max_iters - 1,\n 'start_momentum': 'max_momentum',\n 'end_momentum': 'max_momentum'\n })\n else:\n self.momentum_phases.append({\n 'end_iter':\n float(self.pct_start * runner.max_iters) - 1,\n 'start_momentum':\n 'max_momentum',\n 'end_momentum':\n 'base_momentum'\n })\n self.momentum_phases.append({\n 'end_iter': runner.max_iters - 1,\n 'start_momentum': 'base_momentum',\n 'end_momentum': 'max_momentum'\n })\n\n def _set_momentum(self, runner, momentum_groups):\n if isinstance(runner.optimizer, dict):\n for k, optim in runner.optimizer.items():\n for param_group, mom in zip(optim.param_groups,\n momentum_groups[k]):\n if 'momentum' in param_group.keys():\n param_group['momentum'] = mom\n elif 'betas' in param_group.keys():\n param_group['betas'] = (mom, param_group['betas'][1])\n else:\n for param_group, mom in zip(runner.optimizer.param_groups,\n momentum_groups):\n if 'momentum' in param_group.keys():\n param_group['momentum'] = mom\n elif 'betas' in param_group.keys():\n param_group['betas'] = (mom, param_group['betas'][1])\n\n def get_momentum(self, runner, param_group):\n curr_iter = runner.iter\n start_iter = 0\n for i, phase in enumerate(self.momentum_phases):\n end_iter = phase['end_iter']\n if curr_iter <= end_iter or i == len(self.momentum_phases) - 1:\n pct = (curr_iter - start_iter) / (end_iter - start_iter)\n momentum = self.anneal_func(\n param_group[phase['start_momentum']],\n param_group[phase['end_momentum']], pct)\n break\n start_iter = end_iter\n return momentum\n\n def get_regular_momentum(self, runner):\n if isinstance(runner.optimizer, dict):\n momentum_groups = {}\n for k, optim in runner.optimizer.items():\n _momentum_group = [\n self.get_momentum(runner, param_group)\n for param_group in optim.param_groups\n ]\n momentum_groups.update({k: _momentum_group})\n return momentum_groups\n else:\n momentum_groups = []\n for param_group in runner.optimizer.param_groups:\n momentum_groups.append(self.get_momentum(runner, param_group))\n return momentum_groups\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/nni_hook.py", "content": "from .hook import HOOKS, Hook\n\n@HOOKS.register_module()\nclass NNIHook(Hook):\n\n def before_run(self, runner):\n if runner.opt_cfg['mode'] == \"nni\":\n import nni\n runner.logger = None\n self.nni = nni\n\n def after_train_epoch(self, runner):\n opt_cfg = runner.opt_cfg\n if opt_cfg['mode'] == 'nni':\n # stats = runner.outputs['log_vars']\n stats = runner.metrics\n if len(runner.workflow) == 1 and runner.epoch == runner.max_epochs:\n self.nni.report_final_result({name: value for name, value in stats.items() if opt_cfg['metrics'] in name})\n else:\n print(\"report_intermediate_result\")\n metrics = {name: value for name, value in stats.items() if opt_cfg['metrics'] in name}\n self.nni.report_intermediate_result(metrics['loss'])\n\n\n def after_train_iter(self, runner):\n ...\n\n def before_val_iter(self, runner):\n ...\n\n def after_val_iter(self, runner):\n ...\n\n def after_val_epoch(self, runner):\n opt_cfg = runner.opt_cfg\n if opt_cfg['mode'] == 'nni':\n stats = runner.outputs\n if len(runner.workflow) != 1 and runner.epoch == runner.max_epochs:\n self.nni.report_final_result({name: value for name, value in stats.items() if opt_cfg['metrics'] in name}['loss'])\n else:\n self.nni.report_intermediate_result(\n {name: value for name, value in stats.items() if opt_cfg['metrics'] in name})" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/optimizer.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport copy\nimport logging\nfrom collections import defaultdict\nfrom itertools import chain\n\nfrom torch.nn.utils import clip_grad\n\nfrom mmcv.utils import TORCH_VERSION, _BatchNorm, digit_version\nfrom ..dist_utils import allreduce_grads\nfrom ..fp16_utils import LossScaler, wrap_fp16_model\nfrom .hook import HOOKS, Hook\nfrom mmcv.runner.record import get_grad_norm\ntry:\n # If PyTorch version >= 1.6.0, torch.cuda.amp.GradScaler would be imported\n # and used; otherwise, auto fp16 will adopt mmcv's implementation.\n from torch.cuda.amp import GradScaler\nexcept ImportError:\n pass\n\n\n@HOOKS.register_module()\nclass OptimizerHook(Hook):\n \"\"\"A hook contains custom operations for the optimizer.\n\n Args:\n grad_clip (dict, optional): A float to control the clip_grad.\n Default: None. (not a config dict)\n detect_anomalous_params (bool): This option is only used for\n debugging which will slow down the training speed.\n Detect anomalous parameters that are not included in\n the computational graph with `loss` as the root.\n There are two cases\n\n - Parameters were not used during\n forward pass.\n - Parameters were not used to produce\n loss.\n Default: False.\n \"\"\"\n\n def __init__(self, grad_clip=None, detect_anomalous_params=False):\n self.grad_clip = grad_clip\n self.detect_anomalous_params = detect_anomalous_params\n\n def clip_grads(self, params):\n # params = list(\n # filter(lambda p: p.requires_grad and p.grad is not None, params))\n params, grad_norm = get_grad_norm(params)\n if len(params) > 0 and self.grad_clip:\n # **self.grad_clip\n return clip_grad.clip_grad_norm_(params, self.grad_clip)\n\n else:\n return grad_norm\n\n def after_train_iter(self, runner):\n if isinstance(runner.optimizer, dict):\n if isinstance(runner.model, dict):\n for name, m in runner.model.items():\n grad_norm = self.clip_grads(m.parameters())\n # if grad_norm is not None:\n # Add grad norm to the logger\n # runner.outputs['num_samples'] 用于计算平均的，MetricLogger改进了log_buffer的几个计算，这里不需要了\n runner.log_buffer.update_dict({f'{name}_grad_norm': float(grad_norm)})\n for name, optim in runner.optimizer.items():\n optim.zero_grad()\n if self.detect_anomalous_params:\n self.detect_anomalous_parameters(runner.outputs[f'{name}_loss'], runner)\n runner.outputs[f'{name}_loss'].backward()\n optim.step()\n\n else:\n runner.optimizer.zero_grad()\n if self.detect_anomalous_params:\n self.detect_anomalous_parameters(runner.outputs['loss'], runner)\n runner.outputs['loss'].backward()\n if not hasattr(runner.model, 'train'):\n grad_norm = self.clip_grads(runner.model.model.parameters())\n else:\n grad_norm = self.clip_grads(runner.model.parameters())\n\n # runner.outputs['num_samples'] 用于计算平均的，MetricLogger改进了log_buffer的几个计算，这里不需要了\n runner.log_buffer.update_dict({'grad_norm': float(grad_norm)})\n\n runner.optimizer.step()\n\n def detect_anomalous_parameters(self, loss, runner):\n logger = runner.logger\n parameters_in_graph = set()\n visited = set()\n\n def traverse(grad_fn):\n if grad_fn is None:\n return\n if grad_fn not in visited:\n visited.add(grad_fn)\n if hasattr(grad_fn, 'variable'):\n parameters_in_graph.add(grad_fn.variable)\n parents = grad_fn.next_functions\n if parents is not None:\n for parent in parents:\n grad_fn = parent[0]\n traverse(grad_fn)\n\n traverse(loss.grad_fn)\n for n, p in runner.model.named_parameters():\n if p not in parameters_in_graph and p.requires_grad:\n logger.log(\n level=logging.ERROR,\n msg=f'{n} with shape {p.size()} is not '\n f'in the computational graph \\n')\n\n\n@HOOKS.register_module()\nclass GradientCumulativeOptimizerHook(OptimizerHook):\n \"\"\"Optimizer Hook implements multi-iters gradient cumulating.\n\n Args:\n cumulative_iters (int, optional): Num of gradient cumulative iters.\n The optimizer will step every `cumulative_iters` iters.\n Defaults to 1.\n\n Examples:\n >>> # Use cumulative_iters to simulate a large batch size\n >>> # It is helpful when the hardware cannot handle a large batch size.\n >>> loader = DataLoader(data, batch_size=64)\n >>> optim_hook = GradientCumulativeOptimizerHook(cumulative_iters=4)\n >>> # almost equals to\n >>> loader = DataLoader(data, batch_size=256)\n >>> optim_hook = OptimizerHook()\n \"\"\"\n\n def __init__(self, cumulative_iters=1, **kwargs):\n super(GradientCumulativeOptimizerHook, self).__init__(**kwargs)\n\n assert isinstance(cumulative_iters, int) and cumulative_iters > 0, \\\n f'cumulative_iters only accepts positive int, but got ' \\\n f'{type(cumulative_iters)} instead.'\n\n self.cumulative_iters = cumulative_iters\n self.divisible_iters = 0\n self.remainder_iters = 0\n self.initialized = False\n\n def has_batch_norm(self, module):\n if isinstance(module, _BatchNorm):\n return True\n for m in module.children():\n if self.has_batch_norm(m):\n return True\n return False\n\n def _init(self, runner):\n if runner.iter % self.cumulative_iters != 0:\n runner.logger.warning(\n 'Resume iter number is not divisible by cumulative_iters in '\n 'GradientCumulativeOptimizerHook, which means the gradient of '\n 'some iters is lost and the result may be influenced slightly.'\n )\n\n if self.has_batch_norm(runner.model) and self.cumulative_iters > 1:\n runner.logger.warning(\n 'GradientCumulativeOptimizerHook may slightly decrease '\n 'performance if the model has BatchNorm layers.')\n\n residual_iters = runner.max_iters - runner.iter\n\n self.divisible_iters = (\n residual_iters // self.cumulative_iters * self.cumulative_iters)\n self.remainder_iters = residual_iters - self.divisible_iters\n\n self.initialized = True\n\n def after_train_iter(self, runner):\n if not self.initialized:\n self._init(runner)\n\n if runner.iter < self.divisible_iters:\n loss_factor = self.cumulative_iters\n else:\n loss_factor = self.remainder_iters\n loss = runner.outputs['loss']\n loss = loss / loss_factor\n loss.backward()\n\n if (self.every_n_iters(runner, self.cumulative_iters)\n or self.is_last_iter(runner)):\n\n if self.grad_clip is not None:\n grad_norm = self.clip_grads(runner.model.parameters())\n if grad_norm is not None:\n # Add grad norm to the logger\n runner.log_buffer.update({'grad_norm': float(grad_norm)},\n runner.outputs['num_samples'])\n runner.optimizer.step()\n runner.optimizer.zero_grad()\n\n\nif (TORCH_VERSION != 'parrots'\n and digit_version(TORCH_VERSION) >= digit_version('1.6.0')):\n\n @HOOKS.register_module()\n class Fp16OptimizerHook(OptimizerHook):\n \"\"\"FP16 optimizer hook (using PyTorch's implementation).\n\n If you are using PyTorch >= 1.6, torch.cuda.amp is used as the backend,\n to take care of the optimization procedure.\n\n Args:\n loss_scale (float | str | dict): Scale factor configuration.\n If loss_scale is a float, static loss scaling will be used with\n the specified scale. If loss_scale is a string, it must be\n 'dynamic', then dynamic loss scaling will be used.\n It can also be a dict containing arguments of GradScalar.\n Defaults to 512. For Pytorch >= 1.6, mmcv uses official\n implementation of GradScaler. If you use a dict version of\n loss_scale to create GradScaler, please refer to:\n https://pytorch.org/docs/stable/amp.html#torch.cuda.amp.GradScaler\n for the parameters.\n\n Examples:\n >>> loss_scale = dict(\n ... init_scale=65536.0,\n ... growth_factor=2.0,\n ... backoff_factor=0.5,\n ... growth_interval=2000\n ... )\n >>> optimizer_hook = Fp16OptimizerHook(loss_scale=loss_scale)\n \"\"\"\n\n def __init__(self,\n grad_clip=None,\n coalesce=True,\n bucket_size_mb=-1,\n loss_scale=512.,\n distributed=True):\n self.grad_clip = grad_clip\n self.coalesce = coalesce\n self.bucket_size_mb = bucket_size_mb\n self.distributed = distributed\n self._scale_update_param = None\n if loss_scale == 'dynamic':\n self.loss_scaler = GradScaler()\n elif isinstance(loss_scale, float):\n self._scale_update_param = loss_scale\n self.loss_scaler = GradScaler(init_scale=loss_scale)\n elif isinstance(loss_scale, dict):\n self.loss_scaler = GradScaler(**loss_scale)\n else:\n raise ValueError('loss_scale must be of type float, dict, or '\n f'\"dynamic\", got {loss_scale}')\n\n def before_run(self, runner):\n \"\"\"Preparing steps before Mixed Precision Training.\"\"\"\n # wrap model mode to fp16\n wrap_fp16_model(runner.model)\n # resume from state dict\n if 'fp16' in runner.meta and 'loss_scaler' in runner.meta['fp16']:\n scaler_state_dict = runner.meta['fp16']['loss_scaler']\n self.loss_scaler.load_state_dict(scaler_state_dict)\n\n def copy_grads_to_fp32(self, fp16_net, fp32_weights):\n \"\"\"Copy gradients from fp16 model to fp32 weight copy.\"\"\"\n for fp32_param, fp16_param in zip(fp32_weights,\n fp16_net.parameters()):\n if fp16_param.grad is not None:\n if fp32_param.grad is None:\n fp32_param.grad = fp32_param.data.new(\n fp32_param.size())\n fp32_param.grad.copy_(fp16_param.grad)\n\n def copy_params_to_fp16(self, fp16_net, fp32_weights):\n \"\"\"Copy updated params from fp32 weight copy to fp16 model.\"\"\"\n for fp16_param, fp32_param in zip(fp16_net.parameters(),\n fp32_weights):\n fp16_param.data.copy_(fp32_param.data)\n\n def after_train_iter(self, runner):\n \"\"\"Backward optimization steps for Mixed Precision Training. For\n dynamic loss scaling, please refer to\n https://pytorch.org/docs/stable/amp.html#torch.cuda.amp.GradScaler.\n\n 1. Scale the loss by a scale factor.\n 2. Backward the loss to obtain the gradients.\n 3. Unscale the optimizer’s gradient tensors.\n 4. Call optimizer.step() and update scale factor.\n 5. Save loss_scaler state_dict for resume purpose.\n \"\"\"\n # clear grads of last iteration\n runner.model.zero_grad()\n runner.optimizer.zero_grad()\n\n self.loss_scaler.scale(runner.outputs['loss']).backward()\n self.loss_scaler.unscale_(runner.optimizer)\n # grad clip\n if self.grad_clip is not None:\n grad_norm = self.clip_grads(runner.model.parameters())\n if grad_norm is not None:\n # Add grad norm to the logger\n runner.log_buffer.update({'grad_norm': float(grad_norm)},\n runner.outputs['num_samples'])\n # backward and update scaler\n self.loss_scaler.step(runner.optimizer)\n self.loss_scaler.update(self._scale_update_param)\n\n # save state_dict of loss_scaler\n runner.meta.setdefault(\n 'fp16', {})['loss_scaler'] = self.loss_scaler.state_dict()\n\n @HOOKS.register_module()\n class GradientCumulativeFp16OptimizerHook(GradientCumulativeOptimizerHook,\n Fp16OptimizerHook):\n \"\"\"Fp16 optimizer Hook (using PyTorch's implementation) implements\n multi-iters gradient cumulating.\n\n If you are using PyTorch >= 1.6, torch.cuda.amp is used as the backend,\n to take care of the optimization procedure.\n \"\"\"\n\n def __init__(self, *args, **kwargs):\n super(GradientCumulativeFp16OptimizerHook,\n self).__init__(*args, **kwargs)\n\n def after_train_iter(self, runner):\n if not self.initialized:\n self._init(runner)\n\n if runner.iter < self.divisible_iters:\n loss_factor = self.cumulative_iters\n else:\n loss_factor = self.remainder_iters\n loss = runner.outputs['loss']\n loss = loss / loss_factor\n\n self.loss_scaler.scale(loss).backward()\n\n if (self.every_n_iters(runner, self.cumulative_iters)\n or self.is_last_iter(runner)):\n\n # copy fp16 grads in the model to fp32 params in the optimizer\n self.loss_scaler.unscale_(runner.optimizer)\n\n if self.grad_clip is not None:\n grad_norm = self.clip_grads(runner.model.parameters())\n if grad_norm is not None:\n # Add grad norm to the logger\n runner.log_buffer.update(\n {'grad_norm': float(grad_norm)},\n runner.outputs['num_samples'])\n\n # backward and update scaler\n self.loss_scaler.step(runner.optimizer)\n self.loss_scaler.update(self._scale_update_param)\n\n # save state_dict of loss_scaler\n runner.meta.setdefault(\n 'fp16', {})['loss_scaler'] = self.loss_scaler.state_dict()\n\n # clear grads\n runner.model.zero_grad()\n runner.optimizer.zero_grad()\n\nelse:\n\n @HOOKS.register_module()\n class Fp16OptimizerHook(OptimizerHook):\n \"\"\"FP16 optimizer hook (mmcv's implementation).\n\n The steps of fp16 optimizer is as follows.\n 1. Scale the loss value.\n 2. BP in the fp16 model.\n 2. Copy gradients from fp16 model to fp32 weights.\n 3. Update fp32 weights.\n 4. Copy updated parameters from fp32 weights to fp16 model.\n\n Refer to https://arxiv.org/abs/1710.03740 for more details.\n\n Args:\n loss_scale (float | str | dict): Scale factor configuration.\n If loss_scale is a float, static loss scaling will be used with\n the specified scale. If loss_scale is a string, it must be\n 'dynamic', then dynamic loss scaling will be used.\n It can also be a dict containing arguments of LossScaler.\n Defaults to 512.\n \"\"\"\n\n def __init__(self,\n grad_clip=None,\n coalesce=True,\n bucket_size_mb=-1,\n loss_scale=512.,\n distributed=True):\n self.grad_clip = grad_clip\n self.coalesce = coalesce\n self.bucket_size_mb = bucket_size_mb\n self.distributed = distributed\n if loss_scale == 'dynamic':\n self.loss_scaler = LossScaler(mode='dynamic')\n elif isinstance(loss_scale, float):\n self.loss_scaler = LossScaler(\n init_scale=loss_scale, mode='static')\n elif isinstance(loss_scale, dict):\n self.loss_scaler = LossScaler(**loss_scale)\n else:\n raise ValueError('loss_scale must be of type float, dict, or '\n f'\"dynamic\", got {loss_scale}')\n\n def before_run(self, runner):\n \"\"\"Preparing steps before Mixed Precision Training.\n\n 1. Make a master copy of fp32 weights for optimization.\n 2. Convert the main model from fp32 to fp16.\n \"\"\"\n # keep a copy of fp32 weights\n old_groups = runner.optimizer.param_groups\n runner.optimizer.param_groups = copy.deepcopy(\n runner.optimizer.param_groups)\n state = defaultdict(dict)\n p_map = {\n old_p: p\n for old_p, p in zip(\n chain(*(g['params'] for g in old_groups)),\n chain(*(g['params']\n for g in runner.optimizer.param_groups)))\n }\n for k, v in runner.optimizer.state.items():\n state[p_map[k]] = v\n runner.optimizer.state = state\n # convert model to fp16\n wrap_fp16_model(runner.model)\n # resume from state dict\n if 'fp16' in runner.meta and 'loss_scaler' in runner.meta['fp16']:\n scaler_state_dict = runner.meta['fp16']['loss_scaler']\n self.loss_scaler.load_state_dict(scaler_state_dict)\n\n def copy_grads_to_fp32(self, fp16_net, fp32_weights):\n \"\"\"Copy gradients from fp16 model to fp32 weight copy.\"\"\"\n for fp32_param, fp16_param in zip(fp32_weights,\n fp16_net.parameters()):\n if fp16_param.grad is not None:\n if fp32_param.grad is None:\n fp32_param.grad = fp32_param.data.new(\n fp32_param.size())\n fp32_param.grad.copy_(fp16_param.grad)\n\n def copy_params_to_fp16(self, fp16_net, fp32_weights):\n \"\"\"Copy updated params from fp32 weight copy to fp16 model.\"\"\"\n for fp16_param, fp32_param in zip(fp16_net.parameters(),\n fp32_weights):\n fp16_param.data.copy_(fp32_param.data)\n\n def after_train_iter(self, runner):\n \"\"\"Backward optimization steps for Mixed Precision Training. For\n dynamic loss scaling, please refer `loss_scalar.py`\n\n 1. Scale the loss by a scale factor.\n 2. Backward the loss to obtain the gradients (fp16).\n 3. Copy gradients from the model to the fp32 weight copy.\n 4. Scale the gradients back and update the fp32 weight copy.\n 5. Copy back the params from fp32 weight copy to the fp16 model.\n 6. Save loss_scaler state_dict for resume purpose.\n \"\"\"\n # clear grads of last iteration\n runner.model.zero_grad()\n runner.optimizer.zero_grad()\n # scale the loss value\n scaled_loss = runner.outputs['loss'] * self.loss_scaler.loss_scale\n scaled_loss.backward()\n # copy fp16 grads in the model to fp32 params in the optimizer\n\n fp32_weights = []\n for param_group in runner.optimizer.param_groups:\n fp32_weights += param_group['params']\n self.copy_grads_to_fp32(runner.model, fp32_weights)\n # allreduce grads\n if self.distributed:\n allreduce_grads(fp32_weights, self.coalesce,\n self.bucket_size_mb)\n\n has_overflow = self.loss_scaler.has_overflow(fp32_weights)\n # if has overflow, skip this iteration\n if not has_overflow:\n # scale the gradients back\n for param in fp32_weights:\n if param.grad is not None:\n param.grad.div_(self.loss_scaler.loss_scale)\n if self.grad_clip is not None:\n grad_norm = self.clip_grads(fp32_weights)\n if grad_norm is not None:\n # Add grad norm to the logger\n runner.log_buffer.update(\n {'grad_norm': float(grad_norm)},\n runner.outputs['num_samples'])\n # update fp32 params\n runner.optimizer.step()\n # copy fp32 params to the fp16 model\n self.copy_params_to_fp16(runner.model, fp32_weights)\n self.loss_scaler.update_scale(has_overflow)\n if has_overflow:\n runner.logger.warning('Check overflow, downscale loss scale '\n f'to {self.loss_scaler.cur_scale}')\n\n # save state_dict of loss_scaler\n runner.meta.setdefault(\n 'fp16', {})['loss_scaler'] = self.loss_scaler.state_dict()\n\n @HOOKS.register_module()\n class GradientCumulativeFp16OptimizerHook(GradientCumulativeOptimizerHook,\n Fp16OptimizerHook):\n \"\"\"Fp16 optimizer Hook (using mmcv implementation) implements multi-\n iters gradient cumulating.\"\"\"\n\n def __init__(self, *args, **kwargs):\n super(GradientCumulativeFp16OptimizerHook,\n self).__init__(*args, **kwargs)\n\n def after_train_iter(self, runner):\n if not self.initialized:\n self._init(runner)\n\n if runner.iter < self.divisible_iters:\n loss_factor = self.cumulative_iters\n else:\n loss_factor = self.remainder_iters\n\n loss = runner.outputs['loss']\n loss = loss / loss_factor\n\n # scale the loss value\n scaled_loss = loss * self.loss_scaler.loss_scale\n scaled_loss.backward()\n\n if (self.every_n_iters(runner, self.cumulative_iters)\n or self.is_last_iter(runner)):\n\n # copy fp16 grads in the model to fp32 params in the optimizer\n fp32_weights = []\n for param_group in runner.optimizer.param_groups:\n fp32_weights += param_group['params']\n self.copy_grads_to_fp32(runner.model, fp32_weights)\n # allreduce grads\n if self.distributed:\n allreduce_grads(fp32_weights, self.coalesce,\n self.bucket_size_mb)\n\n has_overflow = self.loss_scaler.has_overflow(fp32_weights)\n # if has overflow, skip this iteration\n if not has_overflow:\n # scale the gradients back\n for param in fp32_weights:\n if param.grad is not None:\n param.grad.div_(self.loss_scaler.loss_scale)\n if self.grad_clip is not None:\n grad_norm = self.clip_grads(fp32_weights)\n if grad_norm is not None:\n # Add grad norm to the logger\n runner.log_buffer.update(\n {'grad_norm': float(grad_norm)},\n runner.outputs['num_samples'])\n # update fp32 params\n runner.optimizer.step()\n # copy fp32 params to the fp16 model\n self.copy_params_to_fp16(runner.model, fp32_weights)\n else:\n runner.logger.warning(\n 'Check overflow, downscale loss scale '\n f'to {self.loss_scaler.cur_scale}')\n\n self.loss_scaler.update_scale(has_overflow)\n\n # save state_dict of loss_scaler\n runner.meta.setdefault(\n 'fp16', {})['loss_scaler'] = self.loss_scaler.state_dict()\n\n # clear grads\n runner.model.zero_grad()\n runner.optimizer.zero_grad()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/profiler.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport warnings\nfrom typing import Callable, List, Optional, Union\n\nimport torch\n\nfrom ..dist_utils import master_only\nfrom .hook import HOOKS, Hook\n\n\n@HOOKS.register_module()\nclass ProfilerHook(Hook):\n \"\"\"Profiler to analyze performance during training.\n\n PyTorch Profiler is a tool that allows the collection of the performance\n metrics during the training. More details on Profiler can be found at\n https://pytorch.org/docs/1.8.1/profiler.html#torch.profiler.profile\n\n Args:\n by_epoch (bool): Profile performance by epoch or by iteration.\n Default: True.\n profile_iters (int): Number of iterations for profiling.\n If ``by_epoch=True``, profile_iters indicates that they are the\n first profile_iters epochs at the beginning of the\n training, otherwise it indicates the first profile_iters\n iterations. Default: 1.\n activities (list[str]): List of activity groups (CPU, CUDA) to use in\n profiling. Default: ['cpu', 'cuda'].\n schedule (dict, optional): Config of generating the callable schedule.\n if schedule is None, profiler will not add step markers into the\n trace and table view. Default: None.\n on_trace_ready (callable, dict): Either a handler or a dict of generate\n handler. Default: None.\n record_shapes (bool): Save information about operator's input shapes.\n Default: False.\n profile_memory (bool): Track tensor memory allocation/deallocation.\n Default: False.\n with_stack (bool): Record source information (file and line number)\n for the ops. Default: False.\n with_flops (bool): Use formula to estimate the FLOPS of specific\n operators (matrix multiplication and 2D convolution).\n Default: False.\n json_trace_path (str, optional): Exports the collected trace in Chrome\n JSON format. Default: None.\n\n Example:\n >>> runner = ... # instantiate a Runner\n >>> # tensorboard trace\n >>> trace_config = dict(type='tb_trace', dir_name='work_dir')\n >>> profiler_config = dict(on_trace_ready=trace_config)\n >>> runner.register_profiler_hook(profiler_config)\n >>> runner.run(data_loaders=[trainloader], workflow=[('train', 1)])\n \"\"\"\n\n def __init__(self,\n by_epoch: bool = True,\n profile_iters: int = 1,\n activities: List[str] = ['cpu', 'cuda'],\n schedule: Optional[dict] = None,\n on_trace_ready: Optional[Union[Callable, dict]] = None,\n record_shapes: bool = False,\n profile_memory: bool = False,\n with_stack: bool = False,\n with_flops: bool = False,\n json_trace_path: Optional[str] = None) -> None:\n try:\n from torch import profiler # torch version >= 1.8.1\n except ImportError:\n raise ImportError('profiler is the new feature of torch1.8.1, '\n f'but your version is {torch.__version__}')\n\n assert isinstance(by_epoch, bool), '``by_epoch`` should be a boolean.'\n self.by_epoch = by_epoch\n\n if profile_iters < 1:\n raise ValueError('profile_iters should be greater than 0, but got '\n f'{profile_iters}')\n self.profile_iters = profile_iters\n\n if not isinstance(activities, list):\n raise ValueError(\n f'activities should be list, but got {type(activities)}')\n self.activities = []\n for activity in activities:\n activity = activity.lower()\n if activity == 'cpu':\n self.activities.append(profiler.ProfilerActivity.CPU)\n elif activity == 'cuda':\n self.activities.append(profiler.ProfilerActivity.CUDA)\n else:\n raise ValueError(\n f'activity should be \"cpu\" or \"cuda\", but got {activity}')\n\n if schedule is not None:\n self.schedule = profiler.schedule(**schedule)\n else:\n self.schedule = None\n\n self.on_trace_ready = on_trace_ready\n self.record_shapes = record_shapes\n self.profile_memory = profile_memory\n self.with_stack = with_stack\n self.with_flops = with_flops\n self.json_trace_path = json_trace_path\n\n @master_only\n def before_run(self, runner):\n if self.by_epoch and runner.max_epochs < self.profile_iters:\n raise ValueError('self.profile_iters should not be greater than '\n f'{runner.max_epochs}')\n\n if not self.by_epoch and runner.max_iters < self.profile_iters:\n raise ValueError('self.profile_iters should not be greater than '\n f'{runner.max_iters}')\n\n if callable(self.on_trace_ready): # handler\n _on_trace_ready = self.on_trace_ready\n elif isinstance(self.on_trace_ready, dict): # config of handler\n trace_cfg = self.on_trace_ready.copy()\n trace_type = trace_cfg.pop('type') # log_trace handler\n if trace_type == 'log_trace':\n\n def _log_handler(prof):\n print(prof.key_averages().table(**trace_cfg))\n\n _on_trace_ready = _log_handler\n elif trace_type == 'tb_trace': # tensorboard_trace handler\n try:\n import torch_tb_profiler # noqa: F401\n except ImportError:\n raise ImportError('please run \"pip install '\n 'torch-tb-profiler\" to install '\n 'torch_tb_profiler')\n _on_trace_ready = torch.profiler.tensorboard_trace_handler(\n **trace_cfg)\n else:\n raise ValueError('trace_type should be \"log_trace\" or '\n f'\"tb_trace\", but got {trace_type}')\n elif self.on_trace_ready is None:\n _on_trace_ready = None # type: ignore\n else:\n raise ValueError('on_trace_ready should be handler, dict or None, '\n f'but got {type(self.on_trace_ready)}')\n\n if runner.max_epochs > 1:\n warnings.warn(f'profiler will profile {runner.max_epochs} epochs '\n 'instead of 1 epoch. Since profiler will slow down '\n 'the training, it is recommended to train 1 epoch '\n 'with ProfilerHook and adjust your setting according'\n ' to the profiler summary. During normal training '\n '(epoch > 1), you may disable the ProfilerHook.')\n\n self.profiler = torch.profiler.profile(\n activities=self.activities,\n schedule=self.schedule,\n on_trace_ready=_on_trace_ready,\n record_shapes=self.record_shapes,\n profile_memory=self.profile_memory,\n with_stack=self.with_stack,\n with_flops=self.with_flops)\n\n self.profiler.__enter__()\n runner.logger.info('profiler is profiling...')\n\n @master_only\n def after_train_epoch(self, runner):\n if self.by_epoch and runner.epoch == self.profile_iters - 1:\n runner.logger.info('profiler may take a few minutes...')\n self.profiler.__exit__(None, None, None)\n if self.json_trace_path is not None:\n self.profiler.export_chrome_trace(self.json_trace_path)\n\n @master_only\n def after_train_iter(self, runner):\n self.profiler.step()\n if not self.by_epoch and runner.iter == self.profile_iters - 1:\n runner.logger.info('profiler may take a few minutes...')\n self.profiler.__exit__(None, None, None)\n if self.json_trace_path is not None:\n self.profiler.export_chrome_trace(self.json_trace_path)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/sampler_seed.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .hook import HOOKS, Hook\n\n\n@HOOKS.register_module()\nclass DistSamplerSeedHook(Hook):\n \"\"\"Data-loading sampler for distributed training.\n\n When distributed training, it is only useful in conjunction with\n :obj:`EpochBasedRunner`, while :obj:`IterBasedRunner` achieves the same\n purpose with :obj:`IterLoader`.\n \"\"\"\n\n def before_epoch(self, runner):\n if hasattr(runner.data_loader.sampler, 'set_epoch'):\n # in case the data loader uses `SequentialSampler` in Pytorch\n runner.data_loader.sampler.set_epoch(runner.epoch)\n elif hasattr(runner.data_loader.batch_sampler.sampler, 'set_epoch'):\n # batch sampler in pytorch warps the sampler as its attributes.\n runner.data_loader.batch_sampler.sampler.set_epoch(runner.epoch)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/hooks/sync_buffer.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom ..dist_utils import allreduce_params\nfrom .hook import HOOKS, Hook\n\n\n@HOOKS.register_module()\nclass SyncBuffersHook(Hook):\n \"\"\"Synchronize model buffers such as running_mean and running_var in BN at\n the end of each epoch.\n\n Args:\n distributed (bool): Whether distributed training is used. It is\n effective only for distributed training. Defaults to True.\n \"\"\"\n\n def __init__(self, distributed=True):\n self.distributed = distributed\n\n def after_epoch(self, runner):\n \"\"\"All-reduce model buffers at the end of each epoch.\"\"\"\n if self.distributed:\n allreduce_params(runner.model.buffers())\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/iter_based_runner.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\nimport platform\nimport shutil\nimport time\nimport warnings\n\nimport torch\nfrom torch.optim import Optimizer\n\nimport mmcv\nfrom .base_runner import BaseRunner\nfrom .builder import RUNNERS\nfrom .checkpoint import save_checkpoint\nfrom .hooks import IterTimerHook\nfrom .utils import get_host_info\n\n\nclass IterLoader:\n\n def __init__(self, dataloader):\n self._dataloader = dataloader\n self.iter_loader = iter(self._dataloader)\n self._epoch = 0\n\n @property\n def epoch(self):\n return self._epoch\n\n def __next__(self):\n try:\n data = next(self.iter_loader)\n except StopIteration:\n self._epoch += 1\n if hasattr(self._dataloader.sampler, 'set_epoch'):\n self._dataloader.sampler.set_epoch(self._epoch)\n time.sleep(2) # Prevent possible deadlock during epoch transition\n self.iter_loader = iter(self._dataloader)\n data = next(self.iter_loader)\n\n return data\n\n def __len__(self):\n return len(self._dataloader)\n\n\n@RUNNERS.register_module()\nclass IterBasedRunner(BaseRunner):\n \"\"\"Iteration-based Runner.\n\n This runner train models iteration by iteration.\n \"\"\"\n\n def train(self, data_loader, **kwargs):\n self.model.train()\n self.mode = 'train'\n self.data_loader = data_loader\n self._epoch = data_loader.epoch\n data_batch = next(data_loader)\n self.call_hook('before_train_iter')\n outputs = self.model.train_step(data_batch, self.optimizer, **kwargs)\n if not isinstance(outputs, dict):\n raise TypeError('model.train_step() must return a dict')\n if 'log_vars' in outputs:\n self.log_buffer.update(outputs['log_vars'], outputs['num_samples'])\n self.outputs = outputs\n self.call_hook('after_train_iter')\n self._inner_iter += 1\n self._iter += 1\n\n @torch.no_grad()\n def val(self, data_loader, **kwargs):\n self.model.eval()\n self.mode = 'val'\n self.data_loader = data_loader\n data_batch = next(data_loader)\n self.call_hook('before_val_iter')\n outputs = self.model.val_step(data_batch, **kwargs)\n if not isinstance(outputs, dict):\n raise TypeError('model.val_step() must return a dict')\n if 'log_vars' in outputs:\n self.log_buffer.update(outputs['log_vars'], outputs['num_samples'])\n self.outputs = outputs\n self.call_hook('after_val_iter')\n self._inner_iter += 1\n\n def run(self, data_loaders, workflow, max_iters=None, **kwargs):\n \"\"\"Start running.\n\n Args:\n data_loaders (list[:obj:`DataLoader`]): Dataloaders for training\n and validation.\n workflow (list[tuple]): A list of (phase, iters) to specify the\n running order and iterations. E.g, [('train', 10000),\n ('val', 1000)] means running 10000 iterations for training and\n 1000 iterations for validation, iteratively.\n \"\"\"\n assert isinstance(data_loaders, list)\n assert mmcv.is_list_of(workflow, tuple)\n assert len(data_loaders) == len(workflow)\n if max_iters is not None:\n warnings.warn(\n 'setting max_iters in run is deprecated, '\n 'please set max_iters in runner_config', DeprecationWarning)\n self._max_iters = max_iters\n assert self._max_iters is not None, (\n 'max_iters must be specified during instantiation')\n\n work_dir = self.work_dir if self.work_dir is not None else 'NONE'\n self.logger.info('Start running, host: %s, work_dir: %s',\n get_host_info(), work_dir)\n self.logger.info('Hooks will be executed in the following order:\\n%s',\n self.get_hook_info())\n self.logger.info('workflow: %s, max: %d iters', workflow,\n self._max_iters)\n self.call_hook('before_run')\n\n iter_loaders = [IterLoader(x) for x in data_loaders]\n\n self.call_hook('before_epoch')\n\n while self.iter < self._max_iters:\n for i, flow in enumerate(workflow):\n self._inner_iter = 0\n mode, iters = flow\n if not isinstance(mode, str) or not hasattr(self, mode):\n raise ValueError(\n 'runner has no method named \"{}\" to run a workflow'.\n format(mode))\n iter_runner = getattr(self, mode)\n for _ in range(iters):\n if mode == 'train' and self.iter >= self._max_iters:\n break\n iter_runner(iter_loaders[i], **kwargs) #>\n\n time.sleep(1) # wait for some hooks like loggers to finish\n self.call_hook('after_epoch')\n self.call_hook('after_run')\n\n def resume(self,\n checkpoint,\n resume_optimizer=True,\n map_location='default'):\n \"\"\"Resume model from checkpoint.\n\n Args:\n checkpoint (str): Checkpoint to resume from.\n resume_optimizer (bool, optional): Whether resume the optimizer(s)\n if the checkpoint file includes optimizer(s). Default to True.\n map_location (str, optional): Same as :func:`torch.load`.\n Default to 'default'.\n \"\"\"\n if map_location == 'default':\n device_id = torch.cuda.current_device()\n checkpoint = self.load_checkpoint(\n checkpoint,\n map_location=lambda storage, loc: storage.cuda(device_id))\n else:\n checkpoint = self.load_checkpoint(\n checkpoint, map_location=map_location)\n\n self._epoch = checkpoint['meta']['epoch']\n self._iter = checkpoint['meta']['iter']\n self._inner_iter = checkpoint['meta']['iter']\n if 'optimizer' in checkpoint and resume_optimizer:\n if isinstance(self.optimizer, Optimizer):\n self.optimizer.load_state_dict(checkpoint['optimizer'])\n elif isinstance(self.optimizer, dict):\n for k in self.optimizer.keys():\n self.optimizer[k].load_state_dict(\n checkpoint['optimizer'][k])\n else:\n raise TypeError(\n 'Optimizer should be dict or torch.optim.Optimizer '\n f'but got {type(self.optimizer)}')\n\n self.logger.info(f'resumed from epoch: {self.epoch}, iter {self.iter}')\n\n def save_checkpoint(self,\n out_dir,\n filename_tmpl='iter_{}.pth',\n meta=None,\n save_optimizer=True,\n create_symlink=True):\n \"\"\"Save checkpoint to file.\n\n Args:\n out_dir (str): Directory to save checkpoint files.\n filename_tmpl (str, optional): Checkpoint file template.\n Defaults to 'iter_{}.pth'.\n meta (dict, optional): Metadata to be saved in checkpoint.\n Defaults to None.\n save_optimizer (bool, optional): Whether save optimizer.\n Defaults to True.\n create_symlink (bool, optional): Whether create symlink to the\n latest checkpoint file. Defaults to True.\n \"\"\"\n if meta is None:\n meta = {}\n elif not isinstance(meta, dict):\n raise TypeError(\n f'meta should be a dict or None, but got {type(meta)}')\n if self.meta is not None:\n meta.update(self.meta)\n # Note: meta.update(self.meta) should be done before\n # meta.update(epoch=self.epoch + 1, iter=self.iter) otherwise\n # there will be problems with resumed checkpoints.\n # More details in https://github.com/open-mmlab/mmcv/pull/1108\n meta.update(epoch=self.epoch + 1, iter=self.iter)\n\n filename = filename_tmpl.format(self.iter + 1)\n filepath = osp.join(out_dir, filename)\n optimizer = self.optimizer if save_optimizer else None\n save_checkpoint(self.model, filepath, optimizer=optimizer, meta=meta)\n # in some environments, `os.symlink` is not supported, you may need to\n # set `create_symlink` to False\n if create_symlink:\n dst_file = osp.join(out_dir, 'latest.pth')\n if platform.system() != 'Windows':\n mmcv.symlink(filename, dst_file)\n else:\n shutil.copy(filepath, dst_file)\n\n def register_training_hooks(self,\n lr_config,\n optimizer_config=None,\n checkpoint_config=None,\n log_config=None,\n momentum_config=None,\n custom_hooks_config=None):\n \"\"\"Register default hooks for iter-based training.\n\n Checkpoint hook, optimizer stepper hook and logger hooks will be set to\n `by_epoch=False` by default.\n\n Default hooks include:\n\n +----------------------+-------------------------+\n | Hooks | Priority |\n +======================+=========================+\n | LrUpdaterHook | VERY_HIGH (10) |\n +----------------------+-------------------------+\n | MomentumUpdaterHook | HIGH (30) |\n +----------------------+-------------------------+\n | OptimizerStepperHook | ABOVE_NORMAL (40) |\n +----------------------+-------------------------+\n | CheckpointSaverHook | NORMAL (50) |\n +----------------------+-------------------------+\n | IterTimerHook | LOW (70) |\n +----------------------+-------------------------+\n | LoggerHook(s) | VERY_LOW (90) |\n +----------------------+-------------------------+\n | CustomHook(s) | defaults to NORMAL (50) |\n +----------------------+-------------------------+\n\n If custom hooks have same priority with default hooks, custom hooks\n will be triggered after default hooks.\n \"\"\"\n if checkpoint_config is not None:\n checkpoint_config.setdefault('by_epoch', False)\n if lr_config is not None:\n lr_config.setdefault('by_epoch', False)\n if log_config is not None:\n for info in log_config['hooks']:\n info.setdefault('by_epoch', False)\n super(IterBasedRunner, self).register_training_hooks(\n lr_config=lr_config,\n momentum_config=momentum_config,\n optimizer_config=optimizer_config,\n checkpoint_config=checkpoint_config,\n log_config=log_config,\n timer_config=IterTimerHook(),\n custom_hooks_config=custom_hooks_config)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/log_buffer.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom collections import OrderedDict\nimport torch\nimport numpy as np\n\n\nclass LogBuffer:\n\n def __init__(self):\n self.val_history = OrderedDict()\n self.n_history = OrderedDict()\n self.output = OrderedDict()\n self.ready = False\n\n def clear(self):\n self.val_history.clear()\n self.n_history.clear()\n self.clear_output()\n\n def clear_output(self):\n self.output.clear()\n self.ready = False\n\n # def update(self, vars, count=1):\n # assert isinstance(vars, dict)\n # for key, var in vars.items():\n # if key not in self.val_history:\n # self.val_history[key] = []\n # self.n_history[key] = []\n # self.val_history[key].append(var)\n # self.n_history[key].append(count)\n\n # {k:v}打印，对每个k都有val、avg、max、deque属性\n def update(self, vars, count=1):\n # dist.barrier()\n for k, v in vars.items():\n if k not in self.val_history:\n self.val_history[k] = []\n self.n_history[k] = []\n if isinstance(v, torch.Tensor):\n v = torch.mean(v)\n if hasattr(v, 'item'):\n v = v.item()\n assert isinstance(v, (float, int, str)), print(f\"{k} type: {type(v)}\")\n self.val_history[k].append(v)\n self.n_history[k].append(count)\n\n def average(self, n=0):\n \"\"\"Average latest n values or all values.\"\"\"\n assert n >= 0\n for key in self.val_history:\n values = np.array(self.val_history[key][-n:])\n nums = np.array(self.n_history[key][-n:])\n avg = np.sum(values * nums) / np.sum(nums)\n self.output[key] = avg\n self.ready = True\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/misc.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport glob\nimport os.path as osp\nimport warnings\n\n\ndef find_latest_checkpoint(path, suffix='pth'):\n \"\"\"Find the latest checkpoint from the working directory.\n\n Args:\n path(str): The path to find checkpoints.\n suffix(str): File extension.\n Defaults to pth.\n\n Returns:\n latest_path(str | None): File path of the latest checkpoint.\n References:\n .. [1] https://github.com/microsoft/SoftTeacher\n /blob/main/ssod/utils/patch.py\n \"\"\"\n if not osp.exists(path):\n warnings.warn('The path of checkpoints does not exist.')\n return None\n if osp.exists(osp.join(path, f'latest.{suffix}')):\n return osp.join(path, f'latest.{suffix}')\n\n checkpoints = glob.glob(osp.join(path, f'*.{suffix}'))\n if len(checkpoints) == 0:\n warnings.warn('There are no checkpoints in the path.')\n return None\n latest = -1\n latest_path = None\n for checkpoint in checkpoints:\n count = int(osp.basename(checkpoint).split('_')[-1].split('.')[0])\n if count > latest:\n latest = count\n latest_path = checkpoint\n return latest_path\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/optimizer/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .builder import (OPTIMIZER_BUILDERS, OPTIMIZERS, build_optimizer,\n build_optimizer_constructor)\nfrom .default_constructor import DefaultOptimizerConstructor\n\n__all__ = [\n 'OPTIMIZER_BUILDERS', 'OPTIMIZERS', 'DefaultOptimizerConstructor',\n 'build_optimizer', 'build_optimizer_constructor'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/optimizer/builder.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport copy\nimport inspect\n\nimport torch\n\nfrom ...utils import Registry, build_from_cfg\n\nOPTIMIZERS = Registry('optimizer')\nOPTIMIZER_BUILDERS = Registry('optimizer builder')\n\n\ndef register_torch_optimizers():\n torch_optimizers = []\n for module_name in dir(torch.optim):\n if module_name.startswith('__'):\n continue\n _optim = getattr(torch.optim, module_name)\n if inspect.isclass(_optim) and issubclass(_optim,\n torch.optim.Optimizer):\n OPTIMIZERS.register_module()(_optim)\n torch_optimizers.append(module_name)\n return torch_optimizers\n\n\nTORCH_OPTIMIZERS = register_torch_optimizers()\n\n\ndef build_optimizer_constructor(cfg):\n return build_from_cfg(cfg, OPTIMIZER_BUILDERS)\n\n\ndef build_optimizer(model, cfg):\n optimizer_cfg = copy.deepcopy(cfg)\n constructor_type = optimizer_cfg.pop('constructor',\n 'DefaultOptimizerConstructor')\n paramwise_cfg = optimizer_cfg.pop('paramwise_cfg', None)\n optim_constructor = build_optimizer_constructor(\n dict(\n type=constructor_type,\n optimizer_cfg=optimizer_cfg,\n paramwise_cfg=paramwise_cfg))\n optimizer = optim_constructor(model)\n return optimizer\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/optimizer/default_constructor.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport warnings\n\nimport torch\nfrom torch.nn import GroupNorm, LayerNorm\n\nfrom mmcv.utils import _BatchNorm, _InstanceNorm, build_from_cfg, is_list_of\nfrom mmcv.utils.ext_loader import check_ops_exist\nfrom .builder import OPTIMIZER_BUILDERS, OPTIMIZERS\n\n\n@OPTIMIZER_BUILDERS.register_module()\nclass DefaultOptimizerConstructor:\n \"\"\"Default constructor for optimizers.\n\n By default each parameter share the same optimizer settings, and we\n provide an argument ``paramwise_cfg`` to specify parameter-wise settings.\n It is a dict and may contain the following fields:\n\n - ``custom_keys`` (dict): Specified parameters-wise settings by keys. If\n one of the keys in ``custom_keys`` is a substring of the name of one\n parameter, then the setting of the parameter will be specified by\n ``custom_keys[key]`` and other setting like ``bias_lr_mult`` etc. will\n be ignored. It should be noted that the aforementioned ``key`` is the\n longest key that is a substring of the name of the parameter. If there\n are multiple matched keys with the same length, then the key with lower\n alphabet order will be chosen.\n ``custom_keys[key]`` should be a dict and may contain fields ``lr_mult``\n and ``decay_mult``. See Example 2 below.\n - ``bias_lr_mult`` (float): It will be multiplied to the learning\n rate for all bias parameters (except for those in normalization\n layers and offset layers of DCN).\n - ``bias_decay_mult`` (float): It will be multiplied to the weight\n decay for all bias parameters (except for those in\n normalization layers, depthwise conv layers, offset layers of DCN).\n - ``norm_decay_mult`` (float): It will be multiplied to the weight\n decay for all weight and bias parameters of normalization\n layers.\n - ``dwconv_decay_mult`` (float): It will be multiplied to the weight\n decay for all weight and bias parameters of depthwise conv\n layers.\n - ``dcn_offset_lr_mult`` (float): It will be multiplied to the learning\n rate for parameters of offset layer in the deformable convs\n of a model.\n - ``bypass_duplicate`` (bool): If true, the duplicate parameters\n would not be added into optimizer. Default: False.\n\n Note:\n\n 1. If the option ``dcn_offset_lr_mult`` is used, the constructor will\n override the effect of ``bias_lr_mult`` in the bias of offset layer.\n So be careful when using both ``bias_lr_mult`` and\n ``dcn_offset_lr_mult``. If you wish to apply both of them to the offset\n layer in deformable convs, set ``dcn_offset_lr_mult`` to the original\n ``dcn_offset_lr_mult`` * ``bias_lr_mult``.\n\n 2. If the option ``dcn_offset_lr_mult`` is used, the constructor will\n apply it to all the DCN layers in the model. So be careful when the\n model contains multiple DCN layers in places other than backbone.\n\n Args:\n model (:obj:`nn.Module`): The model with parameters to be optimized.\n optimizer_cfg (dict): The config dict of the optimizer.\n Positional fields are\n\n - `type`: class name of the optimizer.\n\n Optional fields are\n\n - any arguments of the corresponding optimizer type, e.g.,\n lr, weight_decay, momentum, etc.\n paramwise_cfg (dict, optional): Parameter-wise options.\n\n Example 1:\n >>> model = torch.nn.modules.Conv1d(1, 1, 1)\n >>> optimizer_cfg = dict(type='SGD', lr=0.01, momentum=0.9,\n >>> weight_decay=0.0001)\n >>> paramwise_cfg = dict(norm_decay_mult=0.)\n >>> optim_builder = DefaultOptimizerConstructor(\n >>> optimizer_cfg, paramwise_cfg)\n >>> optimizer = optim_builder(model)\n\n Example 2:\n >>> # assume model have attribute model.backbone and model.cls_head\n >>> optimizer_cfg = dict(type='SGD', lr=0.01, weight_decay=0.95)\n >>> paramwise_cfg = dict(custom_keys={\n '.backbone': dict(lr_mult=0.1, decay_mult=0.9)})\n >>> optim_builder = DefaultOptimizerConstructor(\n >>> optimizer_cfg, paramwise_cfg)\n >>> optimizer = optim_builder(model)\n >>> # Then the `lr` and `weight_decay` for model.backbone is\n >>> # (0.01 * 0.1, 0.95 * 0.9). `lr` and `weight_decay` for\n >>> # model.cls_head is (0.01, 0.95).\n \"\"\"\n\n def __init__(self, optimizer_cfg, paramwise_cfg=None):\n if not isinstance(optimizer_cfg, dict):\n raise TypeError('optimizer_cfg should be a dict',\n f'but got {type(optimizer_cfg)}')\n self.optimizer_cfg = optimizer_cfg\n self.paramwise_cfg = {} if paramwise_cfg is None else paramwise_cfg\n self.base_lr = optimizer_cfg.get('lr', None)\n self.base_wd = optimizer_cfg.get('weight_decay', None)\n self._validate_cfg()\n\n def _validate_cfg(self):\n if not isinstance(self.paramwise_cfg, dict):\n raise TypeError('paramwise_cfg should be None or a dict, '\n f'but got {type(self.paramwise_cfg)}')\n\n if 'custom_keys' in self.paramwise_cfg:\n if not isinstance(self.paramwise_cfg['custom_keys'], dict):\n raise TypeError(\n 'If specified, custom_keys must be a dict, '\n f'but got {type(self.paramwise_cfg[\"custom_keys\"])}')\n if self.base_wd is None:\n for key in self.paramwise_cfg['custom_keys']:\n if 'decay_mult' in self.paramwise_cfg['custom_keys'][key]:\n raise ValueError('base_wd should not be None')\n\n # get base lr and weight decay\n # weight_decay must be explicitly specified if mult is specified\n if ('bias_decay_mult' in self.paramwise_cfg\n or 'norm_decay_mult' in self.paramwise_cfg\n or 'dwconv_decay_mult' in self.paramwise_cfg):\n if self.base_wd is None:\n raise ValueError('base_wd should not be None')\n\n def _is_in(self, param_group, param_group_list):\n assert is_list_of(param_group_list, dict)\n param = set(param_group['params'])\n param_set = set()\n for group in param_group_list:\n param_set.update(set(group['params']))\n\n return not param.isdisjoint(param_set)\n\n def add_params(self, params, module, prefix='', is_dcn_module=None):\n \"\"\"Add all parameters of module to the params list.\n\n The parameters of the given module will be added to the list of param\n groups, with specific rules defined by paramwise_cfg.\n\n Args:\n params (list[dict]): A list of param groups, it will be modified\n in place.\n module (nn.Module): The module to be added.\n prefix (str): The prefix of the module\n is_dcn_module (int|float|None): If the current module is a\n submodule of DCN, `is_dcn_module` will be passed to\n control conv_offset layer's learning rate. Defaults to None.\n \"\"\"\n # get param-wise options\n custom_keys = self.paramwise_cfg.get('custom_keys', {})\n # first sort with alphabet order and then sort with reversed len of str\n sorted_keys = sorted(sorted(custom_keys.keys()), key=len, reverse=True)\n\n bias_lr_mult = self.paramwise_cfg.get('bias_lr_mult', 1.)\n bias_decay_mult = self.paramwise_cfg.get('bias_decay_mult', 1.)\n norm_decay_mult = self.paramwise_cfg.get('norm_decay_mult', 1.)\n dwconv_decay_mult = self.paramwise_cfg.get('dwconv_decay_mult', 1.)\n bypass_duplicate = self.paramwise_cfg.get('bypass_duplicate', False)\n dcn_offset_lr_mult = self.paramwise_cfg.get('dcn_offset_lr_mult', 1.)\n\n # special rules for norm layers and depth-wise conv layers\n is_norm = isinstance(module,\n (_BatchNorm, _InstanceNorm, GroupNorm, LayerNorm))\n is_dwconv = (\n isinstance(module, torch.nn.Conv2d)\n and module.in_channels == module.groups)\n\n for name, param in module.named_parameters(recurse=False):\n param_group = {'params': [param]}\n if not param.requires_grad:\n params.append(param_group)\n continue\n if bypass_duplicate and self._is_in(param_group, params):\n warnings.warn(f'{prefix} is duplicate. It is skipped since '\n f'bypass_duplicate={bypass_duplicate}')\n continue\n # if the parameter match one of the custom keys, ignore other rules\n is_custom = False\n for key in sorted_keys:\n if key in f'{prefix}.{name}':\n is_custom = True\n lr_mult = custom_keys[key].get('lr_mult', 1.)\n param_group['lr'] = self.base_lr * lr_mult\n if self.base_wd is not None:\n decay_mult = custom_keys[key].get('decay_mult', 1.)\n param_group['weight_decay'] = self.base_wd * decay_mult\n break\n\n if not is_custom:\n # bias_lr_mult affects all bias parameters\n # except for norm.bias dcn.conv_offset.bias\n if name == 'bias' and not (is_norm or is_dcn_module):\n param_group['lr'] = self.base_lr * bias_lr_mult\n\n if (prefix.find('conv_offset') != -1 and is_dcn_module\n and isinstance(module, torch.nn.Conv2d)):\n # deal with both dcn_offset's bias & weight\n param_group['lr'] = self.base_lr * dcn_offset_lr_mult\n\n # apply weight decay policies\n if self.base_wd is not None:\n # norm decay\n if is_norm:\n param_group[\n 'weight_decay'] = self.base_wd * norm_decay_mult\n # depth-wise conv\n elif is_dwconv:\n param_group[\n 'weight_decay'] = self.base_wd * dwconv_decay_mult\n # bias lr and decay\n elif name == 'bias' and not is_dcn_module:\n # TODO: current bias_decay_mult will have affect on DCN\n param_group[\n 'weight_decay'] = self.base_wd * bias_decay_mult\n params.append(param_group)\n\n if check_ops_exist():\n from mmcv.ops import DeformConv2d, ModulatedDeformConv2d\n is_dcn_module = isinstance(module,\n (DeformConv2d, ModulatedDeformConv2d))\n else:\n is_dcn_module = False\n for child_name, child_mod in module.named_children():\n child_prefix = f'{prefix}.{child_name}' if prefix else child_name\n self.add_params(\n params,\n child_mod,\n prefix=child_prefix,\n is_dcn_module=is_dcn_module)\n\n def __call__(self, model):\n if hasattr(model, 'module'):\n model = model.module\n\n optimizer_cfg = self.optimizer_cfg.copy()\n # if no paramwise option is specified, just use the global setting\n if not self.paramwise_cfg:\n optimizer_cfg['params'] = model.parameters()\n return build_from_cfg(optimizer_cfg, OPTIMIZERS)\n\n # set param-wise lr and weight decay recursively\n params = []\n self.add_params(params, model)\n optimizer_cfg['params'] = params\n\n return build_from_cfg(optimizer_cfg, OPTIMIZERS)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/priority.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom enum import Enum\n\n\nclass Priority(Enum):\n \"\"\"Hook priority levels.\n\n +--------------+------------+\n | Level | Value |\n +==============+============+\n | HIGHEST | 0 |\n +--------------+------------+\n | VERY_HIGH | 10 |\n +--------------+------------+\n | HIGH | 30 |\n +--------------+------------+\n | ABOVE_NORMAL | 40 |\n +--------------+------------+\n | NORMAL | 50 |\n +--------------+------------+\n | BELOW_NORMAL | 60 |\n +--------------+------------+\n | LOW | 70 |\n +--------------+------------+\n | VERY_LOW | 90 |\n +--------------+------------+\n | LOWEST | 100 |\n +--------------+------------+\n \"\"\"\n\n HIGHEST = 0\n VERY_HIGH = 10\n HIGH = 30\n ABOVE_NORMAL = 40\n NORMAL = 50\n BELOW_NORMAL = 60\n LOW = 70\n VERY_LOW = 90\n LOWEST = 100\n\n\ndef get_priority(priority):\n \"\"\"Get priority value.\n\n Args:\n priority (int or str or :obj:`Priority`): Priority.\n\n Returns:\n int: The priority value.\n \"\"\"\n if isinstance(priority, int):\n if priority < 0 or priority > 100:\n raise ValueError('priority must be between 0 and 100')\n return priority\n elif isinstance(priority, Priority):\n return priority.value\n elif isinstance(priority, str):\n return Priority[priority.upper()].value\n else:\n raise TypeError('priority must be an integer or Priority enum value')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/record.py", "content": "import os\nimport datetime\nimport torch\nimport psutil\nfrom collections import defaultdict, deque\nimport time\n# from UDL.AutoDL.logger import log_string\n# from logging import info as log_string\n# from .logger import get_root_logger\nimport numpy as np\nimport random\nimport torch.backends.cudnn as cudnn\nimport torch.distributed as dist\nfrom functools import partial\nfrom mmcv import print_log as log_string\n\ndef get_grad_norm(parameters, norm_type=2):\n if isinstance(parameters, torch.Tensor):\n parameters = [parameters]\n parameters = list(filter(lambda p: p.requires_grad and p.grad is not None, parameters))\n norm_type = float(norm_type)\n total_norm = 0\n for p in parameters:\n param_norm = p.grad.data.norm(norm_type)\n total_norm += param_norm.item() ** norm_type\n total_norm = total_norm ** (1. / norm_type)\n return parameters, total_norm\n\ndef set_random_seed(seed):\n np.random.seed(seed)\n random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.cuda.manual_seed_all(seed)\n cudnn.deterministic = True\n\n\ndef show_memory_info(hint):\n pid = os.getpid()\n p = psutil.Process(pid)\n\n info = p.memory_full_info()\n memory = info.uss / 1024. / 1024\n print('{} memory used: {} MB'.format(hint, memory))\n\n\n# class OrderedAverageMeter(object):\n# def __init__(self):\n\n\nclass AverageMeter(object):\n \"\"\"Computes and stores the average and current value\"\"\"\n\n def __init__(self, name=None, fmt=\":f\"):\n # self.name = name\n # self.fmt = fmt\n self.reset()\n\n def reset(self):\n self.val = 0\n self.avg = 0\n self.sum = 0\n self.count = 0\n\n def update(self, val, n=1):\n self.val = val\n self.sum += val * n\n self.count += n\n self.avg = self.sum / self.count\n\n # def __str__(self):\n # fmtstr = '{name} {val' + self.fmt + '} ({avg' + self.fmt + '})'\n # return fmtstr.format(**self.__dict__)\n\n\nclass ProgressMeter(object):\n def __init__(self, num_batches, meters, prefix=\"\"):\n self.batch_fmtstr = self._get_batch_fmtstr(num_batches)\n self.meters = meters\n self.prefix = prefix\n\n def display(self, batch):\n entries = [self.prefix + self.batch_fmtstr.format(batch)]\n entries += [str(meter) for meter in self.meters]\n print('\\t'.join(entries))\n\n def _get_batch_fmtstr(self, num_batches):\n num_digits = len(str(num_batches // 1))\n fmt = '{:' + str(num_digits) + 'd}'\n return '[' + fmt + '/' + fmt.format(num_batches) + ']'\n\n\ndef accuracy(output, target, topk=(1,)):\n \"\"\"Computes the precision@k for the specified values of k\"\"\"\n with torch.no_grad():\n maxk = max(topk)\n batch_size = target.size(0)\n\n _, pred = output.topk(maxk, 1, True, True)\n pred = pred.t()\n correct = pred.eq(target.view(1, -1).expand_as(pred))\n\n res = []\n for k in topk:\n correct_k = correct[:k].reshape(-1).float().sum(0, keepdim=True)\n res.append(correct_k.mul_(100.0 / batch_size))\n return res\n\n\n# class logger():\n# def __init__(self, obj, LOG_DIR, parser):\n# logname = 'log_train' + datetime.datetime.now().strftime('%Y_%m_%d-%H_%M_%S')+'.txt'\n# self.LOG_FOUT = open(os.path.join(LOG_DIR, logname), 'w')\n# self.LOG_FOUT.write(str(parser)+'\\n')\n# def __call__(self, out_str):\n# self.LOG_FOUT.write(out_str+'\\n')\n# self.LOG_FOUT.flush()\n# print(out_str)\n\ndef is_dist_avail_and_initialized():\n if not dist.is_available():\n return False\n if not dist.is_initialized():\n return False\n return True\n\n\nclass SmoothedValue(object):\n \"\"\"Track a series of values and provide access to smoothed values over a\n window or the global series average.\n \"\"\"\n\n def __init__(self, window_size=20, fmt=None, eval=False):\n if fmt is None:\n if not eval:\n fmt = \"{value:.7f} (avg:{avg:.7f})\"\n else:\n fmt = \"{value:.7f} (avg:{avg:.7f}, std:{std:.7f})\"\n self.reset(window_size)\n self.fmt = fmt\n\n def reset(self, window_size):\n self.deque = deque(maxlen=window_size)\n self.val = 0\n self.avg = 0\n self.total = 0\n self.count = 0\n\n def update(self, value, n=1):\n self.deque.append(value)\n self.val = value\n self.count += n\n self.total += value * n\n self.avg = self.total / self.count\n\n def synchronize_between_processes(self):\n \"\"\"\n Warning: does not synchronize the deque!\n \"\"\"\n if not is_dist_avail_and_initialized():\n return\n t = torch.tensor([self.val, self.count, self.total], dtype=torch.float64, device='cuda')\n dist.barrier()\n dist.all_reduce(t)\n t = t.tolist()\n self.val = t[0]\n self.count = int(t[1])\n self.total = t[2]\n self.avg = self.total / self.count\n\n @property\n def median(self):\n d = torch.tensor(list(self.deque))\n return d.median().item()\n\n @property\n def std(self):\n return torch.tensor(list(self.deque)).std().item()\n\n # @property\n # def avg(self):\n # d = torch.tensor(list(self.deque), dtype=torch.float32)\n # return d.mean().item()\n\n # @property\n # def global_avg(self):\n # return self.total / self.count\n\n @property\n def max(self):\n return max(self.deque)\n\n #\n # @property\n # def value(self):\n # return self.deque[-1]\n\n def __str__(self):\n # return self.fmt.format(\n # median=self.median,\n # avg=self.avg,\n # global_avg=self.global_avg,\n # max=self.max,\n # value=self.value)\n return self.fmt.format(\n median=self.median,\n avg=self.avg,\n max=self.max,\n value=self.val,\n std=self.std)\n\n\nclass MetricLogger(object):\n\n\n\n def __init__(self, logger=None, delimiter=\"\\t\", dist_print=0, window_size=20, eval=False):\n self.meters = defaultdict(partial(SmoothedValue, window_size=window_size, eval=eval))\n self.delimiter = delimiter\n self.dist_print = dist_print\n # self.log = get_root_logger(\"UDL\")\n self.logger = logger\n self.ready = False\n\n\n def clear(self):\n self.clear_output()\n\n def clear_output(self):\n self.meters.clear()\n self.ready = False\n\n # {k:v}打印，对每个k都有val、avg、max、deque属性\n def update(self, n=1, **kwargs):\n # dist.barrier()\n for k, v in kwargs.items():\n if isinstance(v, torch.Tensor):\n v = torch.mean(v)\n if hasattr(v, 'item'):\n v = v.item()\n assert isinstance(v, (float, int, str)), print(\"type: \", type(v))\n self.meters[k].update(v, n)\n\n # {k:v}打印，对每个k都有val、avg、max、deque属性\n def update_dict(self, kwargs: dict, n=1):\n # dist.barrier()\n for k, v in kwargs.items():\n if isinstance(v, torch.Tensor):\n v = torch.mean(v)\n if hasattr(v, 'item'):\n v = v.item()\n assert isinstance(v, (float, int, str)), print(\"type: \", type(v))\n self.meters[k].update(v, n)\n\n def __getattr__(self, attr):\n if attr in self.meters:\n return self.meters[attr]\n if attr in self.__dict__:\n return self.__dict__[attr]\n raise AttributeError(\"'{}' object has no attribute '{}'\".format(\n type(self).__name__, attr))\n\n def __str__(self):\n loss_str = []\n for name, meter in self.meters.items():\n loss_str.append(\n \"{}: {}\".format(name, str(meter))\n )\n return self.delimiter.join(loss_str)\n\n def synchronize_between_processes(self):\n for meter in self.meters.values():\n meter.synchronize_between_processes()\n\n def add_meter(self, name, meter):\n self.meters[name] = meter\n\n def log_every(self, iterable, print_freq, header=None):\n i = 1\n if not header:\n header = ''\n start_time = time.time()\n end = time.time()\n iter_time = SmoothedValue(fmt='{avg:.4f}')\n data_time = SmoothedValue(fmt='{avg:.4f}')\n space_fmt = ':' + str(len(str(len(iterable)))) + 'd'\n if torch.cuda.is_available():\n log_msg = self.delimiter.join([\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}',\n 'max mem: {memory:.0f}MB'\n ])\n else:\n log_msg = self.delimiter.join([\n header,\n '[{0' + space_fmt + '}/{1}]',\n 'eta: {eta}',\n '{meters}',\n 'time: {time}',\n 'data: {data}'\n ])\n MB = 1024.0 * 1024.0\n # log_string = self.logger.info\n for obj in iterable:\n data_time.update(time.time() - end)\n yield obj, i\n iter_time.update(time.time() - end)\n if i % print_freq == 0 or i == len(iterable):\n eta_seconds = iter_time.avg * (len(iterable) - i)\n eta_string = str(datetime.timedelta(seconds=int(eta_seconds)))\n if torch.cuda.is_available():\n if self.dist_print == 0:\n log_string(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time),\n memory=torch.cuda.max_memory_allocated() / MB), logger=self.logger)\n # self.logger.info(log_msg.format(\n # i, len(iterable), eta=eta_string,\n # meters=str(self),\n # time=str(iter_time), data=str(data_time),\n # memory=torch.cuda.max_memory_allocated() / MB))\n\n else:\n log_string(log_msg.format(\n i, len(iterable), eta=eta_string,\n meters=str(self),\n time=str(iter_time), data=str(data_time)), logger=self.logger)\n i += 1\n end = time.time()\n total_time = time.time() - start_time\n total_time_str = str(datetime.timedelta(seconds=int(total_time)))\n if self.dist_print == 0:\n log_string('{} Total time: {} ({:.4f} s / it)'.format(\n header, total_time_str, total_time / len(iterable)), logger=self.logger)\n # self.logger.info('{} Total time: {} ({:.4f} s / it)'.format(\n # header, total_time_str, total_time / len(iterable)))" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/runner/utils.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport random\nimport sys\nimport time\nimport warnings\nfrom getpass import getuser\nfrom socket import gethostname\n\nimport numpy as np\nimport torch\n\nimport mmcv\n\n\ndef get_host_info():\n \"\"\"Get hostname and username.\n\n Return empty string if exception raised, e.g. ``getpass.getuser()`` will\n lead to error in docker container\n \"\"\"\n host = ''\n try:\n host = f'{getuser()}@{gethostname()}'\n except Exception as e:\n warnings.warn(f'Host or user not found: {str(e)}')\n finally:\n return host\n\n\ndef get_time_str():\n return time.strftime('%Y%m%d_%H%M%S', time.localtime())\n\n\ndef obj_from_dict(info, parent=None, default_args=None):\n \"\"\"Initialize an object from dict.\n\n The dict must contain the key \"type\", which indicates the object type, it\n can be either a string or type, such as \"list\" or ``list``. Remaining\n fields are treated as the arguments for constructing the object.\n\n Args:\n info (dict): Object types and arguments.\n parent (:class:`module`): Module which may containing expected object\n classes.\n default_args (dict, optional): Default arguments for initializing the\n object.\n\n Returns:\n any type: Object built from the dict.\n \"\"\"\n assert isinstance(info, dict) and 'type' in info\n assert isinstance(default_args, dict) or default_args is None\n args = info.copy()\n obj_type = args.pop('type')\n if mmcv.is_str(obj_type):\n if parent is not None:\n obj_type = getattr(parent, obj_type)\n else:\n obj_type = sys.modules[obj_type]\n elif not isinstance(obj_type, type):\n raise TypeError('type must be a str or valid type, but '\n f'got {type(obj_type)}')\n if default_args is not None:\n for name, value in default_args.items():\n args.setdefault(name, value)\n return obj_type(**args)\n\n\ndef set_random_seed(seed, deterministic=False, use_rank_shift=False):\n \"\"\"Set random seed.\n\n Args:\n seed (int): Seed to be used.\n deterministic (bool): Whether to set the deterministic option for\n CUDNN backend, i.e., set `torch.backends.cudnn.deterministic`\n to True and `torch.backends.cudnn.benchmark` to False.\n Default: False.\n rank_shift (bool): Whether to add rank number to the random seed to\n have different random seed in different threads. Default: False.\n \"\"\"\n if use_rank_shift:\n rank, _ = mmcv.runner.get_dist_info()\n seed += rank\n random.seed(seed)\n np.random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.cuda.manual_seed_all(seed)\n os.environ['PYTHONHASHSEED'] = str(seed)\n if deterministic:\n torch.backends.cudnn.deterministic = True\n torch.backends.cudnn.benchmark = False\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/tensorrt/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n# flake8: noqa\nfrom .init_plugins import is_tensorrt_plugin_loaded, load_tensorrt_plugin\nfrom .preprocess import preprocess_onnx\n\n\ndef is_tensorrt_available():\n try:\n import tensorrt\n del tensorrt\n return True\n except ModuleNotFoundError:\n return False\n\n\n__all__ = []\n\nif is_tensorrt_available():\n from .tensorrt_utils import (TRTWraper, TRTWrapper, load_trt_engine,\n onnx2trt, save_trt_engine)\n\n # load tensorrt plugin lib\n load_tensorrt_plugin()\n\n __all__.append([\n 'onnx2trt', 'save_trt_engine', 'load_trt_engine', 'TRTWraper',\n 'TRTWrapper'\n ])\n\n__all__.append(['is_tensorrt_plugin_loaded', 'preprocess_onnx'])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/tensorrt/init_plugins.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport ctypes\nimport glob\nimport os\n\n\ndef get_tensorrt_op_path():\n \"\"\"Get TensorRT plugins library path.\"\"\"\n wildcard = os.path.join(\n os.path.abspath(os.path.dirname(os.path.dirname(__file__))),\n '_ext_trt.*.so')\n\n paths = glob.glob(wildcard)\n lib_path = paths[0] if len(paths) > 0 else ''\n return lib_path\n\n\nplugin_is_loaded = False\n\n\ndef is_tensorrt_plugin_loaded():\n \"\"\"Check if TensorRT plugins library is loaded or not.\n\n Returns:\n bool: plugin_is_loaded flag\n \"\"\"\n global plugin_is_loaded\n return plugin_is_loaded\n\n\ndef load_tensorrt_plugin():\n \"\"\"load TensorRT plugins library.\"\"\"\n global plugin_is_loaded\n lib_path = get_tensorrt_op_path()\n if (not plugin_is_loaded) and os.path.exists(lib_path):\n ctypes.CDLL(lib_path)\n plugin_is_loaded = True\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/tensorrt/preprocess.py", "content": "import numpy as np\nimport onnx\n\n\ndef preprocess_onnx(onnx_model):\n \"\"\"Modify onnx model to match with TensorRT plugins in mmcv.\n\n There are some conflict between onnx node definition and TensorRT limit.\n This function perform preprocess on the onnx model to solve the conflicts.\n For example, onnx `attribute` is loaded in TensorRT on host and onnx\n `input` is loaded on device. The shape inference is performed on host, so\n any `input` related to shape (such as `max_output_boxes_per_class` in\n NonMaxSuppression) should be transformed to `attribute` before conversion.\n\n Arguments:\n onnx_model (onnx.ModelProto): Input onnx model.\n\n Returns:\n onnx.ModelProto: Modified onnx model.\n \"\"\"\n graph = onnx_model.graph\n nodes = graph.node\n initializers = graph.initializer\n node_dict = {}\n for node in nodes:\n node_outputs = node.output\n for output in node_outputs:\n if len(output) > 0:\n node_dict[output] = node\n\n init_dict = {_.name: _ for _ in initializers}\n\n nodes_name_to_remove = set()\n\n def is_node_without_output(name):\n for node_name, node in node_dict.items():\n if node_name not in nodes_name_to_remove:\n if name in node.input:\n return False\n return True\n\n def mark_nodes_to_remove(name):\n node = node_dict[name]\n nodes_name_to_remove.add(name)\n for input_node_name in node.input:\n if is_node_without_output(input_node_name):\n mark_nodes_to_remove(input_node_name)\n\n def parse_data(name, typ, default_value=0):\n if name in node_dict:\n node = node_dict[name]\n if node.op_type == 'Constant':\n raw_data = node.attribute[0].t.raw_data\n else:\n mark_nodes_to_remove(name)\n return default_value\n elif name in init_dict:\n raw_data = init_dict[name].raw_data\n else:\n raise ValueError(f'{name} not found in node or initilizer.')\n return np.frombuffer(raw_data, typ).item()\n\n nrof_node = len(nodes)\n for idx in range(nrof_node):\n node = nodes[idx]\n node_attributes = node.attribute\n node_inputs = node.input\n node_outputs = node.output\n node_name = node.name\n # process NonMaxSuppression node\n if node.op_type == 'NonMaxSuppression':\n center_point_box = 0\n max_output_boxes_per_class = 1000000\n iou_threshold = 0.3\n score_threshold = 0.0\n offset = 0\n for attribute in node_attributes:\n if attribute.name == 'center_point_box':\n center_point_box = attribute.i\n elif attribute.name == 'offset':\n offset = attribute.i\n\n if len(node_inputs) >= 3:\n max_output_boxes_per_class = parse_data(\n node_inputs[2], np.int64, max_output_boxes_per_class)\n mark_nodes_to_remove(node_inputs[2])\n\n if len(node_inputs) >= 4:\n iou_threshold = parse_data(node_inputs[3], np.float32,\n iou_threshold)\n mark_nodes_to_remove(node_inputs[3])\n\n if len(node_inputs) >= 5:\n score_threshold = parse_data(node_inputs[4], np.float32)\n mark_nodes_to_remove(node_inputs[4])\n\n new_node = onnx.helper.make_node(\n 'NonMaxSuppression',\n node_inputs[:2],\n node_outputs,\n name=node_name,\n center_point_box=center_point_box,\n max_output_boxes_per_class=max_output_boxes_per_class,\n iou_threshold=iou_threshold,\n score_threshold=score_threshold,\n offset=offset)\n\n for output in node_outputs:\n if output in node_dict:\n node_dict[output] = new_node\n nodes.insert(idx, new_node)\n nodes.remove(node)\n elif node.op_type == 'InstanceNormalization':\n # directly change op name\n node.op_type = 'MMCVInstanceNormalization'\n\n for node_name in nodes_name_to_remove:\n nodes.remove(node_dict[node_name])\n\n return onnx_model\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/tensorrt/tensorrt_utils.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport warnings\n\nimport onnx\nimport tensorrt as trt\nimport torch\n\nfrom .preprocess import preprocess_onnx\n\n\ndef onnx2trt(onnx_model,\n opt_shape_dict,\n log_level=trt.Logger.ERROR,\n fp16_mode=False,\n max_workspace_size=0,\n device_id=0):\n \"\"\"Convert onnx model to tensorrt engine.\n\n Arguments:\n onnx_model (str or onnx.ModelProto): the onnx model to convert from\n opt_shape_dict (dict): the min/opt/max shape of each input\n log_level (TensorRT log level): the log level of TensorRT\n fp16_mode (bool): enable fp16 mode\n max_workspace_size (int): set max workspace size of TensorRT engine.\n some tactic and layers need large workspace.\n device_id (int): choice the device to create engine.\n\n Returns:\n tensorrt.ICudaEngine: the TensorRT engine created from onnx_model\n\n Example:\n >>> engine = onnx2trt(\n >>> \"onnx_model.onnx\",\n >>> {'input': [[1, 3, 160, 160],\n >>> [1, 3, 320, 320],\n >>> [1, 3, 640, 640]]},\n >>> log_level=trt.Logger.WARNING,\n >>> fp16_mode=True,\n >>> max_workspace_size=1 << 30,\n >>> device_id=0)\n >>> })\n \"\"\"\n device = torch.device('cuda:{}'.format(device_id))\n # create builder and network\n logger = trt.Logger(log_level)\n builder = trt.Builder(logger)\n EXPLICIT_BATCH = 1 << (int)(\n trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)\n network = builder.create_network(EXPLICIT_BATCH)\n\n # parse onnx\n parser = trt.OnnxParser(network, logger)\n\n if isinstance(onnx_model, str):\n onnx_model = onnx.load(onnx_model)\n\n onnx_model = preprocess_onnx(onnx_model)\n\n if not parser.parse(onnx_model.SerializeToString()):\n error_msgs = ''\n for error in range(parser.num_errors):\n error_msgs += f'{parser.get_error(error)}\\n'\n raise RuntimeError(f'parse onnx failed:\\n{error_msgs}')\n\n # config builder\n builder.max_workspace_size = max_workspace_size\n\n config = builder.create_builder_config()\n config.max_workspace_size = max_workspace_size\n profile = builder.create_optimization_profile()\n\n for input_name, param in opt_shape_dict.items():\n min_shape = tuple(param[0][:])\n opt_shape = tuple(param[1][:])\n max_shape = tuple(param[2][:])\n profile.set_shape(input_name, min_shape, opt_shape, max_shape)\n config.add_optimization_profile(profile)\n\n if fp16_mode:\n builder.fp16_mode = fp16_mode\n config.set_flag(trt.BuilderFlag.FP16)\n\n # create engine\n with torch.cuda.device(device):\n engine = builder.build_engine(network, config)\n\n return engine\n\n\ndef save_trt_engine(engine, path):\n \"\"\"Serialize TensorRT engine to disk.\n\n Arguments:\n engine (tensorrt.ICudaEngine): TensorRT engine to serialize\n path (str): disk path to write the engine\n \"\"\"\n with open(path, mode='wb') as f:\n f.write(bytearray(engine.serialize()))\n\n\ndef load_trt_engine(path):\n \"\"\"Deserialize TensorRT engine from disk.\n\n Arguments:\n path (str): disk path to read the engine\n\n Returns:\n tensorrt.ICudaEngine: the TensorRT engine loaded from disk\n \"\"\"\n with trt.Logger() as logger, trt.Runtime(logger) as runtime:\n with open(path, mode='rb') as f:\n engine_bytes = f.read()\n engine = runtime.deserialize_cuda_engine(engine_bytes)\n return engine\n\n\ndef torch_dtype_from_trt(dtype):\n \"\"\"Convert pytorch dtype to TensorRT dtype.\"\"\"\n if dtype == trt.bool:\n return torch.bool\n elif dtype == trt.int8:\n return torch.int8\n elif dtype == trt.int32:\n return torch.int32\n elif dtype == trt.float16:\n return torch.float16\n elif dtype == trt.float32:\n return torch.float32\n else:\n raise TypeError('%s is not supported by torch' % dtype)\n\n\ndef torch_device_from_trt(device):\n \"\"\"Convert pytorch device to TensorRT device.\"\"\"\n if device == trt.TensorLocation.DEVICE:\n return torch.device('cuda')\n elif device == trt.TensorLocation.HOST:\n return torch.device('cpu')\n else:\n return TypeError('%s is not supported by torch' % device)\n\n\nclass TRTWrapper(torch.nn.Module):\n \"\"\"TensorRT engine Wrapper.\n\n Arguments:\n engine (tensorrt.ICudaEngine): TensorRT engine to wrap\n input_names (list[str]): names of each inputs\n output_names (list[str]): names of each outputs\n\n Note:\n If the engine is converted from onnx model. The input_names and\n output_names should be the same as onnx model.\n \"\"\"\n\n def __init__(self, engine, input_names=None, output_names=None):\n super(TRTWrapper, self).__init__()\n self.engine = engine\n if isinstance(self.engine, str):\n self.engine = load_trt_engine(engine)\n\n if not isinstance(self.engine, trt.ICudaEngine):\n raise TypeError('engine should be str or trt.ICudaEngine')\n\n self._register_state_dict_hook(TRTWrapper._on_state_dict)\n self.context = self.engine.create_execution_context()\n\n # get input and output names from engine\n if input_names is None or output_names is None:\n names = [_ for _ in self.engine]\n input_names = list(filter(self.engine.binding_is_input, names))\n output_names = list(set(names) - set(input_names))\n self.input_names = input_names\n self.output_names = output_names\n\n def _on_state_dict(self, state_dict, prefix, local_metadata):\n state_dict[prefix + 'engine'] = bytearray(self.engine.serialize())\n state_dict[prefix + 'input_names'] = self.input_names\n state_dict[prefix + 'output_names'] = self.output_names\n\n def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,\n missing_keys, unexpected_keys, error_msgs):\n engine_bytes = state_dict[prefix + 'engine']\n\n with trt.Logger() as logger, trt.Runtime(logger) as runtime:\n self.engine = runtime.deserialize_cuda_engine(engine_bytes)\n self.context = self.engine.create_execution_context()\n\n self.input_names = state_dict[prefix + 'input_names']\n self.output_names = state_dict[prefix + 'output_names']\n\n def forward(self, inputs):\n \"\"\"\n Arguments:\n inputs (dict): dict of input name-tensors pair\n\n Return:\n dict: dict of output name-tensors pair\n \"\"\"\n assert self.input_names is not None\n assert self.output_names is not None\n bindings = [None] * (len(self.input_names) + len(self.output_names))\n\n for input_name, input_tensor in inputs.items():\n idx = self.engine.get_binding_index(input_name)\n\n if input_tensor.dtype == torch.long:\n input_tensor = input_tensor.int()\n self.context.set_binding_shape(idx, tuple(input_tensor.shape))\n bindings[idx] = input_tensor.contiguous().data_ptr()\n\n # create output tensors\n outputs = {}\n for i, output_name in enumerate(self.output_names):\n idx = self.engine.get_binding_index(output_name)\n dtype = torch_dtype_from_trt(self.engine.get_binding_dtype(idx))\n shape = tuple(self.context.get_binding_shape(idx))\n\n device = torch_device_from_trt(self.engine.get_location(idx))\n output = torch.empty(size=shape, dtype=dtype, device=device)\n outputs[output_name] = output\n bindings[idx] = output.data_ptr()\n\n self.context.execute_async_v2(bindings,\n torch.cuda.current_stream().cuda_stream)\n\n return outputs\n\n\nclass TRTWraper(TRTWrapper):\n\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n warnings.warn(\n 'TRTWraper will be deprecated in'\n ' future. Please use TRTWrapper instead', DeprecationWarning)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/__init__.py", "content": "# flake8: noqa\n# Copyright (c) OpenMMLab. All rights reserved.\nfrom .config import Config, ConfigDict, DictAction\nfrom .misc import (check_prerequisites, concat_list, deprecated_api_warning,\n has_method, import_modules_from_strings, is_list_of,\n is_method_overridden, is_seq_of, is_str, is_tuple_of,\n iter_cast, list_cast, requires_executable, requires_package,\n slice_list, to_1tuple, to_2tuple, to_3tuple, to_4tuple,\n to_ntuple, tuple_cast)\nfrom .path import (check_file_exist, fopen, is_filepath, mkdir_or_exist,\n scandir, symlink)\nfrom .progressbar import (ProgressBar, track_iter_progress,\n track_parallel_progress, track_progress)\nfrom .testing import (assert_attrs_equal, assert_dict_contains_subset,\n assert_dict_has_keys, assert_is_norm_layer,\n assert_keys_equal, assert_params_all_zeros,\n check_python_script)\nfrom .timer import Timer, TimerError, check_time\nfrom .version_utils import digit_version, get_git_hash\n\ntry:\n import torch\nexcept ImportError:\n __all__ = [\n 'Config', 'ConfigDict', 'DictAction', 'is_str', 'iter_cast',\n 'list_cast', 'tuple_cast', 'is_seq_of', 'is_list_of', 'is_tuple_of',\n 'slice_list', 'concat_list', 'check_prerequisites', 'requires_package',\n 'requires_executable', 'is_filepath', 'fopen', 'check_file_exist',\n 'mkdir_or_exist', 'symlink', 'scandir', 'ProgressBar',\n 'track_progress', 'track_iter_progress', 'track_parallel_progress',\n 'Timer', 'TimerError', 'check_time', 'deprecated_api_warning',\n 'digit_version', 'get_git_hash', 'import_modules_from_strings',\n 'assert_dict_contains_subset', 'assert_attrs_equal',\n 'assert_dict_has_keys', 'assert_keys_equal', 'check_python_script',\n 'to_1tuple', 'to_2tuple', 'to_3tuple', 'to_4tuple', 'to_ntuple',\n 'is_method_overridden', 'has_method'\n ]\nelse:\n from .env import collect_env\n from .logging import get_logger, print_log\n from .parrots_jit import jit, skip_no_elena\n from .parrots_wrapper import (\n TORCH_VERSION, BuildExtension, CppExtension, CUDAExtension, DataLoader,\n PoolDataLoader, SyncBatchNorm, _AdaptiveAvgPoolNd, _AdaptiveMaxPoolNd,\n _AvgPoolNd, _BatchNorm, _ConvNd, _ConvTransposeMixin, _InstanceNorm,\n _MaxPoolNd, get_build_config, is_rocm_pytorch, _get_cuda_home)\n from .registry import Registry, build_from_cfg\n from .trace import is_jit_tracing\n from .hub import load_url\n __all__ = [\n 'Config', 'ConfigDict', 'DictAction', 'collect_env', 'get_logger',\n 'print_log', 'is_str', 'iter_cast', 'list_cast', 'tuple_cast',\n 'is_seq_of', 'is_list_of', 'is_tuple_of', 'slice_list', 'concat_list',\n 'check_prerequisites', 'requires_package', 'requires_executable',\n 'is_filepath', 'fopen', 'check_file_exist', 'mkdir_or_exist',\n 'symlink', 'scandir', 'ProgressBar', 'track_progress',\n 'track_iter_progress', 'track_parallel_progress', 'Registry',\n 'build_from_cfg', 'Timer', 'TimerError', 'check_time', 'SyncBatchNorm',\n '_AdaptiveAvgPoolNd', '_AdaptiveMaxPoolNd', '_AvgPoolNd', '_BatchNorm',\n '_ConvNd', '_ConvTransposeMixin', '_InstanceNorm', '_MaxPoolNd',\n 'get_build_config', 'BuildExtension', 'CppExtension', 'CUDAExtension',\n 'DataLoader', 'PoolDataLoader', 'TORCH_VERSION',\n 'deprecated_api_warning', 'digit_version', 'get_git_hash',\n 'import_modules_from_strings', 'jit', 'skip_no_elena',\n 'assert_dict_contains_subset', 'assert_attrs_equal',\n 'assert_dict_has_keys', 'assert_keys_equal', 'assert_is_norm_layer',\n 'assert_params_all_zeros', 'check_python_script',\n 'is_method_overridden', 'is_jit_tracing', 'is_rocm_pytorch',\n '_get_cuda_home', 'load_url', 'has_method'\n ]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/config.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport ast\nimport copy\nimport os\nimport os.path as osp\nimport platform\nimport shutil\nimport sys\nimport tempfile\nimport uuid\nimport warnings\nfrom argparse import Action, ArgumentParser\nfrom collections import abc\nfrom importlib import import_module\n\nfrom addict import Dict\nfrom yapf.yapflib.yapf_api import FormatCode\n\nfrom .misc import import_modules_from_strings\nfrom .path import check_file_exist\n\nif platform.system() == 'Windows':\n import regex as re\nelse:\n import re\n\nBASE_KEY = '_base_'\nDELETE_KEY = '_delete_'\nDEPRECATION_KEY = '_deprecation_'\nRESERVED_KEYS = ['filename', 'text', 'pretty_text']\n\n\nclass ConfigDict(Dict):\n\n def __missing__(self, name):\n raise KeyError(name)\n\n def __getattr__(self, name):\n try:\n value = super(ConfigDict, self).__getattr__(name)\n except KeyError:\n ex = AttributeError(f\"'{self.__class__.__name__}' object has no \"\n f\"attribute '{name}'\")\n except Exception as e:\n ex = e\n else:\n return value\n raise ex\n\n\ndef add_args(parser, cfg, prefix=''):\n for k, v in cfg.items():\n if isinstance(v, str):\n parser.add_argument('--' + prefix + k)\n elif isinstance(v, int):\n parser.add_argument('--' + prefix + k, type=int)\n elif isinstance(v, float):\n parser.add_argument('--' + prefix + k, type=float)\n elif isinstance(v, bool):\n parser.add_argument('--' + prefix + k, action='store_true')\n elif isinstance(v, dict):\n add_args(parser, v, prefix + k + '.')\n elif isinstance(v, abc.Iterable):\n parser.add_argument('--' + prefix + k, type=type(v[0]), nargs='+')\n else:\n print(f'cannot parse key {prefix + k} of type {type(v)}')\n return parser\n\n\nclass Config:\n \"\"\"A facility for config and config files.\n\n It supports common file formats as configs: python/json/yaml. The interface\n is the same as a dict object and also allows access config values as\n attributes.\n\n Example:\n >>> cfg = Config(dict(a=1, b=dict(b1=[0, 1])))\n >>> cfg.a\n 1\n >>> cfg.b\n {'b1': [0, 1]}\n >>> cfg.b.b1\n [0, 1]\n >>> cfg = Config.fromfile('tests/data/config/a.py')\n >>> cfg.filename\n \"/home/kchen/projects/mmcv/tests/data/config/a.py\"\n >>> cfg.item4\n 'test'\n >>> cfg\n \"Config [path: /home/kchen/projects/mmcv/tests/data/config/a.py]: \"\n \"{'item1': [1, 2], 'item2': {'a': 0}, 'item3': True, 'item4': 'test'}\"\n \"\"\"\n\n @staticmethod\n def _validate_py_syntax(filename):\n with open(filename, 'r', encoding='utf-8') as f:\n # Setting encoding explicitly to resolve coding issue on windows\n content = f.read()\n try:\n ast.parse(content)\n except SyntaxError as e:\n raise SyntaxError('There are syntax errors in config '\n f'file {filename}: {e}')\n\n @staticmethod\n def _substitute_predefined_vars(filename, temp_config_name):\n file_dirname = osp.dirname(filename)\n file_basename = osp.basename(filename)\n file_basename_no_extension = osp.splitext(file_basename)[0]\n file_extname = osp.splitext(filename)[1]\n support_templates = dict(\n fileDirname=file_dirname,\n fileBasename=file_basename,\n fileBasenameNoExtension=file_basename_no_extension,\n fileExtname=file_extname)\n with open(filename, 'r', encoding='utf-8') as f:\n # Setting encoding explicitly to resolve coding issue on windows\n config_file = f.read()\n for key, value in support_templates.items():\n regexp = r'\\{\\{\\s*' + str(key) + r'\\s*\\}\\}'\n value = value.replace('\\\\', '/')\n config_file = re.sub(regexp, value, config_file)\n with open(temp_config_name, 'w', encoding='utf-8') as tmp_config_file:\n tmp_config_file.write(config_file)\n\n @staticmethod\n def _pre_substitute_base_vars(filename, temp_config_name):\n \"\"\"Substitute base variable placehoders to string, so that parsing\n would work.\"\"\"\n with open(filename, 'r', encoding='utf-8') as f:\n # Setting encoding explicitly to resolve coding issue on windows\n config_file = f.read()\n base_var_dict = {}\n regexp = r'\\{\\{\\s*' + BASE_KEY + r'\\.([\\w\\.]+)\\s*\\}\\}'\n base_vars = set(re.findall(regexp, config_file))\n for base_var in base_vars:\n randstr = f'_{base_var}_{uuid.uuid4().hex.lower()[:6]}'\n base_var_dict[randstr] = base_var\n regexp = r'\\{\\{\\s*' + BASE_KEY + r'\\.' + base_var + r'\\s*\\}\\}'\n config_file = re.sub(regexp, f'\"{randstr}\"', config_file)\n with open(temp_config_name, 'w', encoding='utf-8') as tmp_config_file:\n tmp_config_file.write(config_file)\n return base_var_dict\n\n @staticmethod\n def _substitute_base_vars(cfg, base_var_dict, base_cfg):\n \"\"\"Substitute variable strings to their actual values.\"\"\"\n cfg = copy.deepcopy(cfg)\n\n if isinstance(cfg, dict):\n for k, v in cfg.items():\n if isinstance(v, str) and v in base_var_dict:\n new_v = base_cfg\n for new_k in base_var_dict[v].split('.'):\n new_v = new_v[new_k]\n cfg[k] = new_v\n elif isinstance(v, (list, tuple, dict)):\n cfg[k] = Config._substitute_base_vars(\n v, base_var_dict, base_cfg)\n elif isinstance(cfg, tuple):\n cfg = tuple(\n Config._substitute_base_vars(c, base_var_dict, base_cfg)\n for c in cfg)\n elif isinstance(cfg, list):\n cfg = [\n Config._substitute_base_vars(c, base_var_dict, base_cfg)\n for c in cfg\n ]\n elif isinstance(cfg, str) and cfg in base_var_dict:\n new_v = base_cfg\n for new_k in base_var_dict[cfg].split('.'):\n new_v = new_v[new_k]\n cfg = new_v\n\n return cfg\n\n @staticmethod\n def _file2dict(filename, use_predefined_variables=True):\n filename = osp.abspath(osp.expanduser(filename))\n check_file_exist(filename)\n fileExtname = osp.splitext(filename)[1]\n if fileExtname not in ['.py', '.json', '.yaml', '.yml']:\n raise IOError('Only py/yml/yaml/json type are supported now!')\n\n with tempfile.TemporaryDirectory() as temp_config_dir:\n temp_config_file = tempfile.NamedTemporaryFile(\n dir=temp_config_dir, suffix=fileExtname)\n if platform.system() == 'Windows':\n temp_config_file.close()\n temp_config_name = osp.basename(temp_config_file.name)\n # Substitute predefined variables\n if use_predefined_variables:\n Config._substitute_predefined_vars(filename,\n temp_config_file.name)\n else:\n shutil.copyfile(filename, temp_config_file.name)\n # Substitute base variables from placeholders to strings\n base_var_dict = Config._pre_substitute_base_vars(\n temp_config_file.name, temp_config_file.name)\n\n if filename.endswith('.py'):\n temp_module_name = osp.splitext(temp_config_name)[0]\n sys.path.insert(0, temp_config_dir)\n Config._validate_py_syntax(filename)\n mod = import_module(temp_module_name)\n sys.path.pop(0)\n cfg_dict = {\n name: value\n for name, value in mod.__dict__.items()\n if not name.startswith('__')\n }\n # delete imported module\n del sys.modules[temp_module_name]\n elif filename.endswith(('.yml', '.yaml', '.json')):\n import mmcv\n cfg_dict = mmcv.load(temp_config_file.name)\n # close temp file\n temp_config_file.close()\n\n # check deprecation information\n if DEPRECATION_KEY in cfg_dict:\n deprecation_info = cfg_dict.pop(DEPRECATION_KEY)\n warning_msg = f'The config file {filename} will be deprecated ' \\\n 'in the future.'\n if 'expected' in deprecation_info:\n warning_msg += f' Please use {deprecation_info[\"expected\"]} ' \\\n 'instead.'\n if 'reference' in deprecation_info:\n warning_msg += ' More information can be found at ' \\\n f'{deprecation_info[\"reference\"]}'\n warnings.warn(warning_msg, DeprecationWarning)\n\n cfg_text = filename + '\\n'\n with open(filename, 'r', encoding='utf-8') as f:\n # Setting encoding explicitly to resolve coding issue on windows\n cfg_text += f.read()\n\n if BASE_KEY in cfg_dict:\n cfg_dir = osp.dirname(filename)\n base_filename = cfg_dict.pop(BASE_KEY)\n base_filename = base_filename if isinstance(\n base_filename, list) else [base_filename]\n\n cfg_dict_list = list()\n cfg_text_list = list()\n for f in base_filename:\n _cfg_dict, _cfg_text = Config._file2dict(osp.join(cfg_dir, f))\n cfg_dict_list.append(_cfg_dict)\n cfg_text_list.append(_cfg_text)\n\n base_cfg_dict = dict()\n for c in cfg_dict_list:\n duplicate_keys = base_cfg_dict.keys() & c.keys()\n if len(duplicate_keys) > 0:\n raise KeyError('Duplicate key is not allowed among bases. '\n f'Duplicate keys: {duplicate_keys}')\n base_cfg_dict.update(c)\n\n # Substitute base variables from strings to their actual values\n cfg_dict = Config._substitute_base_vars(cfg_dict, base_var_dict,\n base_cfg_dict)\n\n base_cfg_dict = Config._merge_a_into_b(cfg_dict, base_cfg_dict)\n cfg_dict = base_cfg_dict\n\n # merge cfg_text\n cfg_text_list.append(cfg_text)\n cfg_text = '\\n'.join(cfg_text_list)\n\n return cfg_dict, cfg_text\n\n @staticmethod\n def _merge_a_into_b(a, b, allow_list_keys=False):\n \"\"\"merge dict ``a`` into dict ``b`` (non-inplace).\n\n Values in ``a`` will overwrite ``b``. ``b`` is copied first to avoid\n in-place modifications.\n\n Args:\n a (dict): The source dict to be merged into ``b``.\n b (dict): The origin dict to be fetch keys from ``a``.\n allow_list_keys (bool): If True, int string keys (e.g. '0', '1')\n are allowed in source ``a`` and will replace the element of the\n corresponding index in b if b is a list. Default: False.\n\n Returns:\n dict: The modified dict of ``b`` using ``a``.\n\n Examples:\n # Normally merge a into b.\n >>> Config._merge_a_into_b(\n ... dict(obj=dict(a=2)), dict(obj=dict(a=1)))\n {'obj': {'a': 2}}\n\n # Delete b first and merge a into b.\n >>> Config._merge_a_into_b(\n ... dict(obj=dict(_delete_=True, a=2)), dict(obj=dict(a=1)))\n {'obj': {'a': 2}}\n\n # b is a list\n >>> Config._merge_a_into_b(\n ... {'0': dict(a=2)}, [dict(a=1), dict(b=2)], True)\n [{'a': 2}, {'b': 2}]\n \"\"\"\n b = b.copy()\n for k, v in a.items():\n if allow_list_keys and k.isdigit() and isinstance(b, list):\n k = int(k)\n if len(b) <= k:\n raise KeyError(f'Index {k} exceeds the length of list {b}')\n b[k] = Config._merge_a_into_b(v, b[k], allow_list_keys)\n elif isinstance(v, dict):\n if k in b and not v.pop(DELETE_KEY, False):\n allowed_types = (dict, list) if allow_list_keys else dict\n if not isinstance(b[k], allowed_types):\n raise TypeError(\n f'{k}={v} in child config cannot inherit from '\n f'base because {k} is a dict in the child config '\n f'but is of type {type(b[k])} in base config. '\n f'You may set `{DELETE_KEY}=True` to ignore the '\n f'base config.')\n b[k] = Config._merge_a_into_b(v, b[k], allow_list_keys)\n else:\n b[k] = ConfigDict(v)\n else:\n b[k] = v\n return b\n\n @staticmethod\n def fromfile(filename,\n use_predefined_variables=True,\n import_custom_modules=True):\n cfg_dict, cfg_text = Config._file2dict(filename,\n use_predefined_variables)\n if import_custom_modules and cfg_dict.get('custom_imports', None):\n import_modules_from_strings(**cfg_dict['custom_imports'])\n return Config(cfg_dict, cfg_text=cfg_text, filename=filename)\n\n @staticmethod\n def fromstring(cfg_str, file_format):\n \"\"\"Generate config from config str.\n\n Args:\n cfg_str (str): Config str.\n file_format (str): Config file format corresponding to the\n config str. Only py/yml/yaml/json type are supported now!\n\n Returns:\n :obj:`Config`: Config obj.\n \"\"\"\n if file_format not in ['.py', '.json', '.yaml', '.yml']:\n raise IOError('Only py/yml/yaml/json type are supported now!')\n if file_format != '.py' and 'dict(' in cfg_str:\n # check if users specify a wrong suffix for python\n warnings.warn(\n 'Please check \"file_format\", the file format may be .py')\n with tempfile.NamedTemporaryFile(\n 'w', encoding='utf-8', suffix=file_format,\n delete=False) as temp_file:\n temp_file.write(cfg_str)\n # on windows, previous implementation cause error\n # see PR 1077 for details\n cfg = Config.fromfile(temp_file.name)\n os.remove(temp_file.name)\n return cfg\n\n @staticmethod\n def auto_argparser(description=None):\n \"\"\"Generate argparser from config file automatically (experimental)\"\"\"\n partial_parser = ArgumentParser(description=description)\n partial_parser.add_argument('config', help='config file path')\n cfg_file = partial_parser.parse_known_args()[0].config\n cfg = Config.fromfile(cfg_file)\n parser = ArgumentParser(description=description)\n parser.add_argument('config', help='config file path')\n add_args(parser, cfg)\n return parser, cfg\n\n def __init__(self, cfg_dict=None, cfg_text=None, filename=None):\n if cfg_dict is None:\n cfg_dict = dict()\n elif not isinstance(cfg_dict, dict):\n raise TypeError('cfg_dict must be a dict, but '\n f'got {type(cfg_dict)}')\n for key in cfg_dict:\n if key in RESERVED_KEYS:\n raise KeyError(f'{key} is reserved for config file')\n\n super(Config, self).__setattr__('_cfg_dict', ConfigDict(cfg_dict))\n super(Config, self).__setattr__('_filename', filename)\n if cfg_text:\n text = cfg_text\n elif filename:\n with open(filename, 'r') as f:\n text = f.read()\n else:\n text = ''\n super(Config, self).__setattr__('_text', text)\n\n @property\n def filename(self):\n return self._filename\n\n @property\n def text(self):\n return self._text\n\n @property\n def pretty_text(self):\n\n indent = 4\n\n def _indent(s_, num_spaces):\n s = s_.split('\\n')\n if len(s) == 1:\n return s_\n first = s.pop(0)\n s = [(num_spaces * ' ') + line for line in s]\n s = '\\n'.join(s)\n s = first + '\\n' + s\n return s\n\n def _format_basic_types(k, v, use_mapping=False):\n if isinstance(v, str):\n v_str = f\"'{v}'\"\n else:\n v_str = str(v)\n\n if use_mapping:\n k_str = f\"'{k}'\" if isinstance(k, str) else str(k)\n attr_str = f'{k_str}: {v_str}'\n else:\n attr_str = f'{str(k)}={v_str}'\n attr_str = _indent(attr_str, indent)\n\n return attr_str\n\n def _format_list(k, v, use_mapping=False):\n # check if all items in the list are dict\n if all(isinstance(_, dict) for _ in v):\n v_str = '[\\n'\n v_str += '\\n'.join(\n f'dict({_indent(_format_dict(v_), indent)}),'\n for v_ in v).rstrip(',')\n if use_mapping:\n k_str = f\"'{k}'\" if isinstance(k, str) else str(k)\n attr_str = f'{k_str}: {v_str}'\n else:\n attr_str = f'{str(k)}={v_str}'\n attr_str = _indent(attr_str, indent) + ']'\n else:\n attr_str = _format_basic_types(k, v, use_mapping)\n return attr_str\n\n def _contain_invalid_identifier(dict_str):\n contain_invalid_identifier = False\n for key_name in dict_str:\n contain_invalid_identifier |= \\\n (not str(key_name).isidentifier())\n return contain_invalid_identifier\n\n def _format_dict(input_dict, outest_level=False):\n r = ''\n s = []\n\n use_mapping = _contain_invalid_identifier(input_dict)\n if use_mapping:\n r += '{'\n for idx, (k, v) in enumerate(input_dict.items()):\n is_last = idx >= len(input_dict) - 1\n end = '' if outest_level or is_last else ','\n if isinstance(v, dict):\n v_str = '\\n' + _format_dict(v)\n if use_mapping:\n k_str = f\"'{k}'\" if isinstance(k, str) else str(k)\n attr_str = f'{k_str}: dict({v_str}'\n else:\n attr_str = f'{str(k)}=dict({v_str}'\n attr_str = _indent(attr_str, indent) + ')' + end\n elif isinstance(v, list):\n attr_str = _format_list(k, v, use_mapping) + end\n else:\n attr_str = _format_basic_types(k, v, use_mapping) + end\n\n s.append(attr_str)\n r += '\\n'.join(s)\n if use_mapping:\n r += '}'\n return r\n\n cfg_dict = self._cfg_dict.to_dict()\n text = _format_dict(cfg_dict, outest_level=True)\n # copied from setup.cfg\n yapf_style = dict(\n based_on_style='pep8',\n blank_line_before_nested_class_or_def=True,\n split_before_expression_after_opening_paren=True)\n text, _ = FormatCode(text, style_config=yapf_style, verify=True)\n\n return text\n\n def __repr__(self):\n return f'Config (path: {self.filename}): {self._cfg_dict.__repr__()}'\n\n def __len__(self):\n return len(self._cfg_dict)\n\n def __getattr__(self, name):\n return getattr(self._cfg_dict, name)\n\n def __getitem__(self, name):\n return self._cfg_dict.__getitem__(name)\n\n def __setattr__(self, name, value):\n if isinstance(value, dict):\n value = ConfigDict(value)\n self._cfg_dict.__setattr__(name, value)\n\n def __setitem__(self, name, value):\n if isinstance(value, dict):\n value = ConfigDict(value)\n self._cfg_dict.__setitem__(name, value)\n\n def __iter__(self):\n return iter(self._cfg_dict)\n\n def __getstate__(self):\n return (self._cfg_dict, self._filename, self._text)\n\n def __setstate__(self, state):\n _cfg_dict, _filename, _text = state\n super(Config, self).__setattr__('_cfg_dict', _cfg_dict)\n super(Config, self).__setattr__('_filename', _filename)\n super(Config, self).__setattr__('_text', _text)\n\n def dump(self, file=None):\n cfg_dict = super(Config, self).__getattribute__('_cfg_dict').to_dict()\n if self.filename.endswith('.py'):\n if file is None:\n return self.pretty_text\n else:\n with open(file, 'w', encoding='utf-8') as f:\n f.write(self.pretty_text)\n else:\n import mmcv\n if file is None:\n file_format = self.filename.split('.')[-1]\n return mmcv.dump(cfg_dict, file_format=file_format)\n else:\n mmcv.dump(cfg_dict, file)\n\n def merge_from_dict(self, options, allow_list_keys=True):\n \"\"\"Merge list into cfg_dict.\n\n Merge the dict parsed by MultipleKVAction into this cfg.\n\n Examples:\n >>> options = {'model.backbone.depth': 50,\n ... 'model.backbone.with_cp':True}\n >>> cfg = Config(dict(model=dict(backbone=dict(type='ResNet'))))\n >>> cfg.merge_from_dict(options)\n >>> cfg_dict = super(Config, self).__getattribute__('_cfg_dict')\n >>> assert cfg_dict == dict(\n ... model=dict(backbone=dict(depth=50, with_cp=True)))\n\n >>> # Merge list element\n >>> cfg = Config(dict(pipeline=[\n ... dict(type='LoadImage'), dict(type='LoadAnnotations')]))\n >>> options = dict(pipeline={'0': dict(type='SelfLoadImage')})\n >>> cfg.merge_from_dict(options, allow_list_keys=True)\n >>> cfg_dict = super(Config, self).__getattribute__('_cfg_dict')\n >>> assert cfg_dict == dict(pipeline=[\n ... dict(type='SelfLoadImage'), dict(type='LoadAnnotations')])\n\n Args:\n options (dict): dict of configs to merge from.\n allow_list_keys (bool): If True, int string keys (e.g. '0', '1')\n are allowed in ``options`` and will replace the element of the\n corresponding index in the config if the config is a list.\n Default: True.\n \"\"\"\n option_cfg_dict = {}\n for full_key, v in options.items():\n d = option_cfg_dict\n key_list = full_key.split('.')\n for subkey in key_list[:-1]:\n d.setdefault(subkey, ConfigDict())\n d = d[subkey]\n subkey = key_list[-1]\n d[subkey] = v\n\n cfg_dict = super(Config, self).__getattribute__('_cfg_dict')\n super(Config, self).__setattr__(\n '_cfg_dict',\n Config._merge_a_into_b(\n option_cfg_dict, cfg_dict, allow_list_keys=allow_list_keys))\n\n\nclass DictAction(Action):\n \"\"\"\n argparse action to split an argument into KEY=VALUE form\n on the first = and append to a dictionary. List options can\n be passed as comma separated values, i.e 'KEY=V1,V2,V3', or with explicit\n brackets, i.e. 'KEY=[V1,V2,V3]'. It also support nested brackets to build\n list/tuple values. e.g. 'KEY=[(V1,V2),(V3,V4)]'\n \"\"\"\n\n @staticmethod\n def _parse_int_float_bool(val):\n try:\n return int(val)\n except ValueError:\n pass\n try:\n return float(val)\n except ValueError:\n pass\n if val.lower() in ['true', 'false']:\n return True if val.lower() == 'true' else False\n return val\n\n @staticmethod\n def _parse_iterable(val):\n \"\"\"Parse iterable values in the string.\n\n All elements inside '()' or '[]' are treated as iterable values.\n\n Args:\n val (str): Value string.\n\n Returns:\n list | tuple: The expanded list or tuple from the string.\n\n Examples:\n >>> DictAction._parse_iterable('1,2,3')\n [1, 2, 3]\n >>> DictAction._parse_iterable('[a, b, c]')\n ['a', 'b', 'c']\n >>> DictAction._parse_iterable('[(1, 2, 3), [a, b], c]')\n [(1, 2, 3), ['a', 'b'], 'c']\n \"\"\"\n\n def find_next_comma(string):\n \"\"\"Find the position of next comma in the string.\n\n If no ',' is found in the string, return the string length. All\n chars inside '()' and '[]' are treated as one element and thus ','\n inside these brackets are ignored.\n \"\"\"\n assert (string.count('(') == string.count(')')) and (\n string.count('[') == string.count(']')), \\\n f'Imbalanced brackets exist in {string}'\n end = len(string)\n for idx, char in enumerate(string):\n pre = string[:idx]\n # The string before this ',' is balanced\n if ((char == ',') and (pre.count('(') == pre.count(')'))\n and (pre.count('[') == pre.count(']'))):\n end = idx\n break\n return end\n\n # Strip ' and \" characters and replace whitespace.\n val = val.strip('\\'\\\"').replace(' ', '')\n is_tuple = False\n if val.startswith('(') and val.endswith(')'):\n is_tuple = True\n val = val[1:-1]\n elif val.startswith('[') and val.endswith(']'):\n val = val[1:-1]\n elif ',' not in val:\n # val is a single value\n return DictAction._parse_int_float_bool(val)\n\n values = []\n while len(val) > 0:\n comma_idx = find_next_comma(val)\n element = DictAction._parse_iterable(val[:comma_idx])\n values.append(element)\n val = val[comma_idx + 1:]\n if is_tuple:\n values = tuple(values)\n return values\n\n def __call__(self, parser, namespace, values, option_string=None):\n options = {}\n for kv in values:\n key, val = kv.split('=', maxsplit=1)\n options[key] = self._parse_iterable(val)\n setattr(namespace, self.dest, options)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/env.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n\"\"\"This file holding some environment constant for sharing by other files.\"\"\"\n\nimport os.path as osp\nimport subprocess\nimport sys\nfrom collections import defaultdict\n\nimport cv2\nimport torch\n\nimport mmcv\nfrom .parrots_wrapper import get_build_config\n\n\ndef collect_env():\n \"\"\"Collect the information of the running environments.\n\n Returns:\n dict: The environment information. The following fields are contained.\n\n - sys.platform: The variable of ``sys.platform``.\n - Python: Python version.\n - CUDA available: Bool, indicating if CUDA is available.\n - GPU devices: Device type of each GPU.\n - CUDA_HOME (optional): The env var ``CUDA_HOME``.\n - NVCC (optional): NVCC version.\n - GCC: GCC version, \"n/a\" if GCC is not installed.\n - PyTorch: PyTorch version.\n - PyTorch compiling details: The output of \\\n ``torch.__config__.show()``.\n - TorchVision (optional): TorchVision version.\n - OpenCV: OpenCV version.\n - MMCV: MMCV version.\n - MMCV Compiler: The GCC version for compiling MMCV ops.\n - MMCV CUDA Compiler: The CUDA version for compiling MMCV ops.\n \"\"\"\n env_info = {}\n env_info['sys.platform'] = sys.platform\n env_info['Python'] = sys.version.replace('\\n', '')\n\n cuda_available = torch.cuda.is_available()\n env_info['CUDA available'] = cuda_available\n\n if cuda_available:\n devices = defaultdict(list)\n for k in range(torch.cuda.device_count()):\n devices[torch.cuda.get_device_name(k)].append(str(k))\n for name, device_ids in devices.items():\n env_info['GPU ' + ','.join(device_ids)] = name\n\n from mmcv.utils.parrots_wrapper import _get_cuda_home\n CUDA_HOME = _get_cuda_home()\n env_info['CUDA_HOME'] = CUDA_HOME\n\n if CUDA_HOME is not None and osp.isdir(CUDA_HOME):\n try:\n nvcc = osp.join(CUDA_HOME, 'bin/nvcc')\n nvcc = subprocess.check_output(\n f'\"{nvcc}\" -V | tail -n1', shell=True)\n nvcc = nvcc.decode('utf-8').strip()\n except subprocess.SubprocessError:\n nvcc = 'Not Available'\n env_info['NVCC'] = nvcc\n\n try:\n gcc = subprocess.check_output('gcc --version | head -n1', shell=True)\n gcc = gcc.decode('utf-8').strip()\n env_info['GCC'] = gcc\n except subprocess.CalledProcessError: # gcc is unavailable\n env_info['GCC'] = 'n/a'\n\n env_info['PyTorch'] = torch.__version__\n env_info['PyTorch compiling details'] = get_build_config()\n\n try:\n import torchvision\n env_info['TorchVision'] = torchvision.__version__\n except ModuleNotFoundError:\n pass\n\n env_info['OpenCV'] = cv2.__version__\n\n env_info['MMCV'] = mmcv.__version__\n\n try:\n from mmcv.ops import get_compiler_version, get_compiling_cuda_version\n except ModuleNotFoundError:\n env_info['MMCV Compiler'] = 'n/a'\n env_info['MMCV CUDA Compiler'] = 'n/a'\n else:\n env_info['MMCV Compiler'] = get_compiler_version()\n env_info['MMCV CUDA Compiler'] = get_compiling_cuda_version()\n\n return env_info\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/ext_loader.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport importlib\nimport os\nimport pkgutil\nimport warnings\nfrom collections import namedtuple\n\nimport torch\n\nif torch.__version__ != 'parrots':\n\n def load_ext(name, funcs):\n ext = importlib.import_module('mmcv.' + name)\n for fun in funcs:\n assert hasattr(ext, fun), f'{fun} miss in module {name}'\n return ext\nelse:\n from parrots import extension\n from parrots.base import ParrotsException\n\n has_return_value_ops = [\n 'nms',\n 'softnms',\n 'nms_match',\n 'nms_rotated',\n 'top_pool_forward',\n 'top_pool_backward',\n 'bottom_pool_forward',\n 'bottom_pool_backward',\n 'left_pool_forward',\n 'left_pool_backward',\n 'right_pool_forward',\n 'right_pool_backward',\n 'fused_bias_leakyrelu',\n 'upfirdn2d',\n 'ms_deform_attn_forward',\n 'pixel_group',\n 'contour_expand',\n ]\n\n def get_fake_func(name, e):\n\n def fake_func(*args, **kwargs):\n warnings.warn(f'{name} is not supported in parrots now')\n raise e\n\n return fake_func\n\n def load_ext(name, funcs):\n ExtModule = namedtuple('ExtModule', funcs)\n ext_list = []\n lib_root = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))\n for fun in funcs:\n try:\n ext_fun = extension.load(fun, name, lib_dir=lib_root)\n except ParrotsException as e:\n if 'No element registered' not in e.message:\n warnings.warn(e.message)\n ext_fun = get_fake_func(fun, e)\n ext_list.append(ext_fun)\n else:\n if fun in has_return_value_ops:\n ext_list.append(ext_fun.op)\n else:\n ext_list.append(ext_fun.op_)\n return ExtModule(*ext_list)\n\n\ndef check_ops_exist():\n ext_loader = pkgutil.find_loader('mmcv._ext')\n return ext_loader is not None\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/hub.py", "content": "# The 1.6 release of PyTorch switched torch.save to use a new zipfile-based\n# file format. It will cause RuntimeError when a checkpoint was saved in\n# torch >= 1.6.0 but loaded in torch < 1.7.0.\n# More details at https://github.com/open-mmlab/mmpose/issues/904\nfrom .parrots_wrapper import TORCH_VERSION\nfrom .path import mkdir_or_exist\nfrom .version_utils import digit_version\n\nif TORCH_VERSION != 'parrots' and digit_version(TORCH_VERSION) < digit_version(\n '1.7.0'):\n # Modified from https://github.com/pytorch/pytorch/blob/master/torch/hub.py\n import os\n import torch\n import warnings\n from urllib.parse import urlparse\n import sys\n import zipfile\n from torch.hub import download_url_to_file, _get_torch_home, HASH_REGEX\n\n # Hub used to support automatically extracts from zipfile manually\n # compressed by users. The legacy zip format expects only one file from\n # torch.save() < 1.6 in the zip. We should remove this support since\n # zipfile is now default zipfile format for torch.save().\n def _is_legacy_zip_format(filename):\n if zipfile.is_zipfile(filename):\n infolist = zipfile.ZipFile(filename).infolist()\n return len(infolist) == 1 and not infolist[0].is_dir()\n return False\n\n def _legacy_zip_load(filename, model_dir, map_location):\n warnings.warn(\n 'Falling back to the old format < 1.6. This support will'\n ' be deprecated in favor of default zipfile format '\n 'introduced in 1.6. Please redo torch.save() to save it '\n 'in the new zipfile format.', DeprecationWarning)\n # Note: extractall() defaults to overwrite file if exists. No need to\n # clean up beforehand. We deliberately don't handle tarfile here\n # since our legacy serialization format was in tar.\n # E.g. resnet18-5c106cde.pth which is widely used.\n with zipfile.ZipFile(filename) as f:\n members = f.infolist()\n if len(members) != 1:\n raise RuntimeError(\n 'Only one file(not dir) is allowed in the zipfile')\n f.extractall(model_dir)\n extraced_name = members[0].filename\n extracted_file = os.path.join(model_dir, extraced_name)\n return torch.load(extracted_file, map_location=map_location)\n\n def load_url(url,\n model_dir=None,\n map_location=None,\n progress=True,\n check_hash=False,\n file_name=None):\n r\"\"\"Loads the Torch serialized object at the given URL.\n\n If downloaded file is a zip file, it will be automatically decompressed\n\n If the object is already present in `model_dir`, it's deserialized and\n returned.\n The default value of ``model_dir`` is ``/checkpoints`` where\n ``hub_dir`` is the directory returned by :func:`~torch.hub.get_dir`.\n\n Args:\n url (str): URL of the object to download\n model_dir (str, optional): directory in which to save the object\n map_location (optional): a function or a dict specifying how to\n remap storage locations (see torch.load)\n progress (bool, optional): whether or not to display a progress bar\n to stderr. Default: True\n check_hash(bool, optional): If True, the filename part of the URL\n should follow the naming convention ``filename-.ext``\n where ```` is the first eight or more digits of the\n SHA256 hash of the contents of the file. The hash is used to\n ensure unique names and to verify the contents of the file.\n Default: False\n file_name (str, optional): name for the downloaded file. Filename\n from ``url`` will be used if not set. Default: None.\n\n Example:\n >>> url = ('https://s3.amazonaws.com/pytorch/models/resnet18-5c106'\n ... 'cde.pth')\n >>> state_dict = torch.hub.load_state_dict_from_url(url)\n \"\"\"\n # Issue warning to move data if old env is set\n if os.getenv('TORCH_MODEL_ZOO'):\n warnings.warn(\n 'TORCH_MODEL_ZOO is deprecated, please use env '\n 'TORCH_HOME instead', DeprecationWarning)\n\n if model_dir is None:\n torch_home = _get_torch_home()\n model_dir = os.path.join(torch_home, 'checkpoints')\n\n mkdir_or_exist(model_dir)\n\n parts = urlparse(url)\n filename = os.path.basename(parts.path)\n if file_name is not None:\n filename = file_name\n cached_file = os.path.join(model_dir, filename)\n if not os.path.exists(cached_file):\n sys.stderr.write('Downloading: \"{}\" to {}\\n'.format(\n url, cached_file))\n hash_prefix = None\n if check_hash:\n r = HASH_REGEX.search(filename) # r is Optional[Match[str]]\n hash_prefix = r.group(1) if r else None\n download_url_to_file(\n url, cached_file, hash_prefix, progress=progress)\n\n if _is_legacy_zip_format(cached_file):\n return _legacy_zip_load(cached_file, model_dir, map_location)\n\n try:\n return torch.load(cached_file, map_location=map_location)\n except RuntimeError as error:\n if digit_version(TORCH_VERSION) < digit_version('1.5.0'):\n warnings.warn(\n f'If the error is the same as \"{cached_file} is a zip '\n 'archive (did you mean to use torch.jit.load()?)\", you can'\n ' upgrade your torch to 1.5.0 or higher (current torch '\n f'version is {TORCH_VERSION}). The error was raised '\n ' because the checkpoint was saved in torch>=1.6.0 but '\n 'loaded in torch<1.5.')\n raise error\nelse:\n from torch.utils.model_zoo import load_url # noqa: F401\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/logging.py", "content": "# # Copyright (c) OpenMMLab. All rights reserved.\n# import logging\n#\n# import torch.distributed as dist\n#\n# logger_initialized = {}\n#\n#\n# def get_logger(name, log_file=None, log_level=logging.INFO, file_mode='w'):\n# \"\"\"Initialize and get a logger by name.\n#\n# If the logger has not been initialized, this method will initialize the\n# logger by adding one or two handlers, otherwise the initialized logger will\n# be directly returned. During initialization, a StreamHandler will always be\n# added. If `log_file` is specified and the process rank is 0, a FileHandler\n# will also be added.\n#\n# Args:\n# name (str): Logger name.\n# log_file (str | None): The log filename. If specified, a FileHandler\n# will be added to the logger.\n# log_level (int): The logger level. Note that only the process of\n# rank 0 is affected, and other processes will set the level to\n# \"Error\" thus be silent most of the time.\n# file_mode (str): The file mode used in opening log file.\n# Defaults to 'w'.\n#\n# Returns:\n# logging.Logger: The expected logger.\n# \"\"\"\n# logger = logging.getLogger(name)\n# if name in logger_initialized:\n# return logger\n# # handle hierarchical names\n# # e.g., logger \"a\" is initialized, then logger \"a.b\" will skip the\n# # initialization since it is a child of \"a\".\n# for logger_name in logger_initialized:\n# if name.startswith(logger_name):\n# return logger\n#\n# # handle duplicate logs to the console\n# # Starting in 1.8.0, PyTorch DDP attaches a StreamHandler (NOTSET)\n# # to the root logger. As logger.propagate is True by default, this root\n# # level handler causes logging messages from rank>0 processes to\n# # unexpectedly show up on the console, creating much unwanted clutter.\n# # To fix this issue, we set the root logger's StreamHandler, if any, to log\n# # at the ERROR level.\n# for handler in logger.root.handlers:\n# if type(handler) is logging.StreamHandler:\n# handler.setLevel(logging.ERROR)\n#\n# stream_handler = logging.StreamHandler()\n# handlers = [stream_handler]\n#\n# if dist.is_available() and dist.is_initialized():\n# rank = dist.get_rank()\n# else:\n# rank = 0\n#\n# # only rank 0 will add a FileHandler\n# if rank == 0 and log_file is not None:\n# # Here, the default behaviour of the official logger is 'a'. Thus, we\n# # provide an interface to change the file mode to the default\n# # behaviour.\n# file_handler = logging.FileHandler(log_file, file_mode)\n# handlers.append(file_handler)\n#\n# formatter = logging.Formatter(\n# '%(asctime)s - %(name)s - %(levelname)s - %(message)s')\n# for handler in handlers:\n# handler.setFormatter(formatter)\n# handler.setLevel(log_level)\n# logger.addHandler(handler)\n#\n# if rank == 0:\n# logger.setLevel(log_level)\n# else:\n# logger.setLevel(logging.ERROR)\n#\n# logger_initialized[name] = True\n#\n# return logger\n#\n#\n# def print_log(msg, logger=None, level=logging.INFO):\n# \"\"\"Print a log message.\n#\n# Args:\n# msg (str): The message to be logged.\n# logger (logging.Logger | str | None): The logger to be used.\n# Some special loggers are:\n# - \"silent\": no message will be printed.\n# - other str: the logger obtained with `get_root_logger(logger)`.\n# - None: The `print()` method will be used to print log messages.\n# level (int): Logging level. Only available when `logger` is a Logger\n# object or \"root\".\n# \"\"\"\n# if logger is None:\n# print(msg)\n# elif isinstance(logger, logging.Logger):\n# logger.log(level, msg)\n# elif logger == 'silent':\n# pass\n# elif isinstance(logger, str):\n# _logger = get_logger(logger)\n# _logger.log(level, msg)\n# else:\n# raise TypeError(\n# 'logger should be either a logging.Logger object, str, '\n# f'\"silent\" or None, but got {type(logger)}')\n\n\n# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n#\n# @Time : 2022/1/24 11:03\n# @Author : Xiao Wu\n# @reference:\n# Copyright (c) OpenMMLab. All rights reserved.\nimport json\nfrom collections import defaultdict\nimport logging\nimport os\nimport functools\nimport torch.distributed as dist\nimport colorlog\nimport time\nfrom pathlib import Path\n\nlogger_initialized = {}\n\nlog_colors_config = {\n 'DEBUG': 'cyan',\n 'INFO': 'white',\n 'WARNING': 'yellow',\n 'ERROR': 'red',\n 'CRITICAL': 'red',\n}\n\n\n# def get_root_logger(name, log_file=None, log_level=logging.INFO):\n# return get_logger('mmcls', log_file, log_level)\ndef get_root_logger(name=None, cfg=None, cfg_name=None, log_level=logging.INFO):\n return get_logger(name, cfg, cfg_name, log_level)\n# TODO: Depre\n# the same as \"get_root_logger\"\ndef create_logger(cfg=None, cfg_name=None, dist_print=0, log_level=logging.INFO):\n return get_logger(None, cfg, cfg_name, log_level)\n\n@functools.lru_cache() # so that calling setup_logger multiple times won't add many handlers\ndef setup_logger(name, final_log_file, color=True):\n # LOG_DIR = cfg.log_dir\n # LOG_FOUT = open(final_log_file, 'w')\n # head = '%(asctime)-15s %(message)s'\n\n logging.basicConfig(filename=str(final_log_file).replace('\\\\', '/'), format='%(message)s', level=logging.INFO)\n # logger = logging.getLogger()\n # logger.setLevel(logging.INFO)\n # console = logging.StreamHandler()\n # logging.getLogger('').addHandler(console)\n\n logger = logging.getLogger(name)\n # if name in logger_initialized:\n # return logger\n\n for handler in logger.root.handlers:\n if type(handler) is logging.StreamHandler:\n handler.setLevel(logging.ERROR)\n\n # stream_handler = logging.StreamHandler()\n console = colorlog.StreamHandler()\n handlers = [console]\n\n # logger.setLevel(logging.INFO)\n # formatter = colorlog.ColoredFormatter(\n # '%(log_color)s[%(asctime)s] [%(filename)s:%(lineno)d] [%(module)s:%(funcName)s] [%(levelname)s]- %(message)s',\n # log_colors=log_colors_config) # 日志输出格式\n\n if dist.is_available() and dist.is_initialized():\n rank = dist.get_rank()\n else:\n rank = 0\n\n if rank == 0:\n # console = colorlog.StreamHandler()\n # console.setLevel(logging.DEBUG)\n handlers.append(console)\n # if color:\n # formatter = _ColorfulFormatter(\n # colored(\"%(message)s\", \"green\")\n # )\n # else:\n formatter = colorlog.ColoredFormatter(\n '%(log_color)s- %(message)s',\n log_colors=log_colors_config) # 日志输出格式\n\n # console.setFormatter(formatter)\n # logger.addHandler(console)\n for handler in handlers:\n handler.setFormatter(formatter)\n handler.setLevel(logging.INFO) # log_level\n logger.addHandler(handler)\n\n # if rank == 0:\n # logger.setLevel(logging.INFO) # log_level\n # else:\n # logger.setLevel(logging.ERROR)\n\n logger_initialized[name] = True\n\n return logger\n\n\ndef get_logger(name=None, cfg=None, cfg_name=None, phase='train', log_level=logging.INFO, file_mode='w'): # log_file=None,\n \"\"\"Initialize and get a logger by name.\n\n If the logger has not been initialized, this method will initialize the\n logger by adding one or two handlers, otherwise the initialized logger will\n be directly returned. During initialization, a StreamHandler will always be\n added. If `log_file` is specified and the process rank is 0, a FileHandler\n will also be added.\n\n Args:\n name (str): Logger name.\n log_file (str | None): The log filename. If specified, a FileHandler\n will be added to the logger.\n log_level (int): The logger level. Note that only the process of\n rank 0 is affected, and other processes will set the level to\n \"Error\" thus be silent most of the time.\n file_mode (str): The file mode used in opening log file.\n Defaults to 'w'.\n\n Returns:\n logging.Logger: The expected logger.\n \"\"\"\n if name in logger_initialized:\n if cfg is None: # cfg.use_log\n return logging.getLogger(name)\n else:\n return None\n # handle hierarchical names\n # e.g., logger \"a\" is initialized, then logger \"a.b\" will skip the\n # initialization since it is a child of \"a\".\n for logger_name in logger_initialized:\n if name.startswith(logger_name):\n if cfg.use_log:\n return logging.getLogger(name)\n else:\n return None\n\n logger = None\n tensorboard_log_dir = None\n root_output_dir = Path(cfg.out_dir)\n # set up logger in root_path\n if not root_output_dir.exists():\n # if not dist_print: #rank 0-N, 0 is False\n print('=> creating {}'.format(root_output_dir))\n root_output_dir.mkdir(parents=True, exist_ok=True)\n\n dataset = cfg.dataset\n assert isinstance(dataset, dict), print(f\"{dataset}'s type is {type(dataset)}, not a dict. \")\n dataset = dataset.get('train') if dataset.get('train', None) is not None else dataset.get('val')\n model = cfg.arch\n cfg_name = os.path.basename(cfg_name).split('.')[0]\n time_str = time.strftime('%Y-%m-%d-%H-%M-%S')\n\n # store all output except tb_log file\n final_output_dir = root_output_dir / dataset / model / cfg_name\n if cfg.eval:\n model_save_tmp = os.path.dirname(cfg.resume_from).split('/')[-1]\n else:\n model_save_tmp = \"model_{}\".format(time_str)\n\n model_save_dir = final_output_dir / model_save_tmp\n # if not dist_print:\n print_log('=> creating {}'.format(final_output_dir))\n final_output_dir.mkdir(parents=True, exist_ok=True)\n model_save_dir.mkdir(parents=True, exist_ok=True)\n\n\n if cfg.use_log:\n cfg_name = '{}_{}'.format(cfg_name, time_str)\n # a logger to save results\n log_file = '{}_{}.log'.format(cfg_name, phase)\n if cfg.eval:\n final_log_file = model_save_dir / log_file\n else:\n final_log_file = final_output_dir / log_file\n # tensorboard_log\n tensorboard_log_dir = root_output_dir / Path(cfg.log_dir) / dataset / model / cfg_name\n # if not dist_print:\n print_log('=> creating tfb logs {}'.format(tensorboard_log_dir))\n tensorboard_log_dir.mkdir(parents=True, exist_ok=True)\n logger = setup_logger(name, final_log_file)\n\n return logger, str(final_output_dir), str(model_save_dir), str(\n tensorboard_log_dir) # logger,\n\ndef print_log(msg, logger=None, level=logging.INFO):\n \"\"\"Print a log message.\n\n Args:\n msg (str): The message to be logged.\n logger (logging.Logger | str | None): The logger to be used.\n Some special loggers are:\n - \"silent\": no message will be printed.\n - other str: the logger obtained with `get_root_logger(logger)`.\n - None: The `print()` method will be used to print log messages.\n level (int): Logging level. Only available when `logger` is a Logger\n object or \"root\".\n \"\"\"\n if logger is None:\n print(msg)\n elif isinstance(logger, logging.Logger):\n logger.log(level, msg)\n elif logger == 'silent':\n pass\n elif isinstance(logger, str):\n _logger = get_logger(logger)\n _logger.log(level, msg)\n else:\n raise TypeError(\n 'logger should be either a logging.Logger object, str, '\n f'\"silent\" or None, but got {type(logger)}')\n\n\ndef load_json_log(json_log):\n \"\"\"load and convert json_logs to log_dicts.\n\n Args:\n json_log (str): The path of the json log file.\n\n Returns:\n dict[int, dict[str, list]]:\n Key is the epoch, value is a sub dict. The keys in each sub dict\n are different metrics, e.g. memory, bbox_mAP, and the value is a\n list of corresponding values in all iterations in this epoch.\n\n .. code-block:: python\n\n # An example output\n {\n 1: {'iter': [100, 200, 300], 'loss': [6.94, 6.73, 6.53]},\n 2: {'iter': [100, 200, 300], 'loss': [6.33, 6.20, 6.07]},\n ...\n }\n \"\"\"\n log_dict = dict()\n with open(json_log, 'r') as log_file:\n for line in log_file:\n log = json.loads(line.strip())\n # skip lines without `epoch` field\n if 'epoch' not in log:\n continue\n epoch = log.pop('epoch')\n if epoch not in log_dict:\n log_dict[epoch] = defaultdict(list)\n for k, v in log.items():\n log_dict[epoch][k].append(v)\n return log_dict\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/misc.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport collections.abc\nimport functools\nimport itertools\nimport subprocess\nimport warnings\nfrom collections import abc\nfrom importlib import import_module\nfrom inspect import getfullargspec\nfrom itertools import repeat\n\n\n# From PyTorch internals\ndef _ntuple(n):\n\n def parse(x):\n if isinstance(x, collections.abc.Iterable):\n return x\n return tuple(repeat(x, n))\n\n return parse\n\n\nto_1tuple = _ntuple(1)\nto_2tuple = _ntuple(2)\nto_3tuple = _ntuple(3)\nto_4tuple = _ntuple(4)\nto_ntuple = _ntuple\n\n\ndef is_str(x):\n \"\"\"Whether the input is an string instance.\n\n Note: This method is deprecated since python 2 is no longer supported.\n \"\"\"\n return isinstance(x, str)\n\n\ndef import_modules_from_strings(imports, allow_failed_imports=False):\n \"\"\"Import modules from the given list of strings.\n\n Args:\n imports (list | str | None): The given module names to be imported.\n allow_failed_imports (bool): If True, the failed imports will return\n None. Otherwise, an ImportError is raise. Default: False.\n\n Returns:\n list[module] | module | None: The imported modules.\n\n Examples:\n >>> osp, sys = import_modules_from_strings(\n ... ['os.path', 'sys'])\n >>> import os.path as osp_\n >>> import sys as sys_\n >>> assert osp == osp_\n >>> assert sys == sys_\n \"\"\"\n if not imports:\n return\n single_import = False\n if isinstance(imports, str):\n single_import = True\n imports = [imports]\n if not isinstance(imports, list):\n raise TypeError(\n f'custom_imports must be a list but got type {type(imports)}')\n imported = []\n for imp in imports:\n if not isinstance(imp, str):\n raise TypeError(\n f'{imp} is of type {type(imp)} and cannot be imported.')\n try:\n imported_tmp = import_module(imp)\n except ImportError:\n if allow_failed_imports:\n warnings.warn(f'{imp} failed to import and is ignored.',\n UserWarning)\n imported_tmp = None\n else:\n raise ImportError\n imported.append(imported_tmp)\n if single_import:\n imported = imported[0]\n return imported\n\n\ndef iter_cast(inputs, dst_type, return_type=None):\n \"\"\"Cast elements of an iterable object into some type.\n\n Args:\n inputs (Iterable): The input object.\n dst_type (type): Destination type.\n return_type (type, optional): If specified, the output object will be\n converted to this type, otherwise an iterator.\n\n Returns:\n iterator or specified type: The converted object.\n \"\"\"\n if not isinstance(inputs, abc.Iterable):\n raise TypeError('inputs must be an iterable object')\n if not isinstance(dst_type, type):\n raise TypeError('\"dst_type\" must be a valid type')\n\n out_iterable = map(dst_type, inputs)\n\n if return_type is None:\n return out_iterable\n else:\n return return_type(out_iterable)\n\n\ndef list_cast(inputs, dst_type):\n \"\"\"Cast elements of an iterable object into a list of some type.\n\n A partial method of :func:`iter_cast`.\n \"\"\"\n return iter_cast(inputs, dst_type, return_type=list)\n\n\ndef tuple_cast(inputs, dst_type):\n \"\"\"Cast elements of an iterable object into a tuple of some type.\n\n A partial method of :func:`iter_cast`.\n \"\"\"\n return iter_cast(inputs, dst_type, return_type=tuple)\n\n\ndef is_seq_of(seq, expected_type, seq_type=None):\n \"\"\"Check whether it is a sequence of some type.\n\n Args:\n seq (Sequence): The sequence to be checked.\n expected_type (type): Expected type of sequence items.\n seq_type (type, optional): Expected sequence type.\n\n Returns:\n bool: Whether the sequence is valid.\n \"\"\"\n if seq_type is None:\n exp_seq_type = abc.Sequence\n else:\n assert isinstance(seq_type, type)\n exp_seq_type = seq_type\n if not isinstance(seq, exp_seq_type):\n return False\n for item in seq:\n if not isinstance(item, expected_type):\n return False\n return True\n\n\ndef is_list_of(seq, expected_type):\n \"\"\"Check whether it is a list of some type.\n\n A partial method of :func:`is_seq_of`.\n \"\"\"\n return is_seq_of(seq, expected_type, seq_type=list)\n\n\ndef is_tuple_of(seq, expected_type):\n \"\"\"Check whether it is a tuple of some type.\n\n A partial method of :func:`is_seq_of`.\n \"\"\"\n return is_seq_of(seq, expected_type, seq_type=tuple)\n\n\ndef slice_list(in_list, lens):\n \"\"\"Slice a list into several sub lists by a list of given length.\n\n Args:\n in_list (list): The list to be sliced.\n lens(int or list): The expected length of each out list.\n\n Returns:\n list: A list of sliced list.\n \"\"\"\n if isinstance(lens, int):\n assert len(in_list) % lens == 0\n lens = [lens] * int(len(in_list) / lens)\n if not isinstance(lens, list):\n raise TypeError('\"indices\" must be an integer or a list of integers')\n elif sum(lens) != len(in_list):\n raise ValueError('sum of lens and list length does not '\n f'match: {sum(lens)} != {len(in_list)}')\n out_list = []\n idx = 0\n for i in range(len(lens)):\n out_list.append(in_list[idx:idx + lens[i]])\n idx += lens[i]\n return out_list\n\n\ndef concat_list(in_list):\n \"\"\"Concatenate a list of list into a single list.\n\n Args:\n in_list (list): The list of list to be merged.\n\n Returns:\n list: The concatenated flat list.\n \"\"\"\n return list(itertools.chain(*in_list))\n\n\ndef check_prerequisites(\n prerequisites,\n checker,\n msg_tmpl='Prerequisites \"{}\" are required in method \"{}\" but not '\n 'found, please install them first.'): # yapf: disable\n \"\"\"A decorator factory to check if prerequisites are satisfied.\n\n Args:\n prerequisites (str of list[str]): Prerequisites to be checked.\n checker (callable): The checker method that returns True if a\n prerequisite is meet, False otherwise.\n msg_tmpl (str): The message template with two variables.\n\n Returns:\n decorator: A specific decorator.\n \"\"\"\n\n def wrap(func):\n\n @functools.wraps(func)\n def wrapped_func(*args, **kwargs):\n requirements = [prerequisites] if isinstance(\n prerequisites, str) else prerequisites\n missing = []\n for item in requirements:\n if not checker(item):\n missing.append(item)\n if missing:\n print(msg_tmpl.format(', '.join(missing), func.__name__))\n raise RuntimeError('Prerequisites not meet.')\n else:\n return func(*args, **kwargs)\n\n return wrapped_func\n\n return wrap\n\n\ndef _check_py_package(package):\n try:\n import_module(package)\n except ImportError:\n return False\n else:\n return True\n\n\ndef _check_executable(cmd):\n if subprocess.call(f'which {cmd}', shell=True) != 0:\n return False\n else:\n return True\n\n\ndef requires_package(prerequisites):\n \"\"\"A decorator to check if some python packages are installed.\n\n Example:\n >>> @requires_package('numpy')\n >>> func(arg1, args):\n >>> return numpy.zeros(1)\n array([0.])\n >>> @requires_package(['numpy', 'non_package'])\n >>> func(arg1, args):\n >>> return numpy.zeros(1)\n ImportError\n \"\"\"\n return check_prerequisites(prerequisites, checker=_check_py_package)\n\n\ndef requires_executable(prerequisites):\n \"\"\"A decorator to check if some executable files are installed.\n\n Example:\n >>> @requires_executable('ffmpeg')\n >>> func(arg1, args):\n >>> print(1)\n 1\n \"\"\"\n return check_prerequisites(prerequisites, checker=_check_executable)\n\n\ndef deprecated_api_warning(name_dict, cls_name=None):\n \"\"\"A decorator to check if some arguments are deprecate and try to replace\n deprecate src_arg_name to dst_arg_name.\n\n Args:\n name_dict(dict):\n key (str): Deprecate argument names.\n val (str): Expected argument names.\n\n Returns:\n func: New function.\n \"\"\"\n\n def api_warning_wrapper(old_func):\n\n @functools.wraps(old_func)\n def new_func(*args, **kwargs):\n # get the arg spec of the decorated method\n args_info = getfullargspec(old_func)\n # get name of the function\n func_name = old_func.__name__\n if cls_name is not None:\n func_name = f'{cls_name}.{func_name}'\n if args:\n arg_names = args_info.args[:len(args)]\n for src_arg_name, dst_arg_name in name_dict.items():\n if src_arg_name in arg_names:\n warnings.warn(\n f'\"{src_arg_name}\" is deprecated in '\n f'`{func_name}`, please use \"{dst_arg_name}\" '\n 'instead', DeprecationWarning)\n arg_names[arg_names.index(src_arg_name)] = dst_arg_name\n if kwargs:\n for src_arg_name, dst_arg_name in name_dict.items():\n if src_arg_name in kwargs:\n\n assert dst_arg_name not in kwargs, (\n f'The expected behavior is to replace '\n f'the deprecated key `{src_arg_name}` to '\n f'new key `{dst_arg_name}`, but got them '\n f'in the arguments at the same time, which '\n f'is confusing. `{src_arg_name} will be '\n f'deprecated in the future, please '\n f'use `{dst_arg_name}` instead.')\n\n warnings.warn(\n f'\"{src_arg_name}\" is deprecated in '\n f'`{func_name}`, please use \"{dst_arg_name}\" '\n 'instead', DeprecationWarning)\n kwargs[dst_arg_name] = kwargs.pop(src_arg_name)\n\n # apply converted arguments to the decorated method\n output = old_func(*args, **kwargs)\n return output\n\n return new_func\n\n return api_warning_wrapper\n\n\ndef is_method_overridden(method, base_class, derived_class):\n \"\"\"Check if a method of base class is overridden in derived class.\n\n Args:\n method (str): the method name to check.\n base_class (type): the class of the base class.\n derived_class (type | Any): the class or instance of the derived class.\n \"\"\"\n assert isinstance(base_class, type), \\\n \"base_class doesn't accept instance, Please pass class instead.\"\n\n if not isinstance(derived_class, type):\n derived_class = derived_class.__class__\n\n base_method = getattr(base_class, method)\n derived_method = getattr(derived_class, method)\n return derived_method != base_method\n\n\ndef has_method(obj: object, method: str) -> bool:\n \"\"\"Check whether the object has a method.\n\n Args:\n method (str): The method name to check.\n obj (object): The object to check.\n\n Returns:\n bool: True if the object has the method else False.\n \"\"\"\n return hasattr(obj, method) and callable(getattr(obj, method))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/parrots_jit.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\n\nfrom .parrots_wrapper import TORCH_VERSION\n\nparrots_jit_option = os.getenv('PARROTS_JIT_OPTION')\n\nif TORCH_VERSION == 'parrots' and parrots_jit_option == 'ON':\n from parrots.jit import pat as jit\nelse:\n\n def jit(func=None,\n check_input=None,\n full_shape=True,\n derivate=False,\n coderize=False,\n optimize=False):\n\n def wrapper(func):\n\n def wrapper_inner(*args, **kargs):\n return func(*args, **kargs)\n\n return wrapper_inner\n\n if func is None:\n return wrapper\n else:\n return func\n\n\nif TORCH_VERSION == 'parrots':\n from parrots.utils.tester import skip_no_elena\nelse:\n\n def skip_no_elena(func):\n\n def wrapper(*args, **kargs):\n return func(*args, **kargs)\n\n return wrapper\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/parrots_wrapper.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom functools import partial\n\nimport torch\n\nTORCH_VERSION = torch.__version__\n\n\ndef is_rocm_pytorch() -> bool:\n is_rocm = False\n if TORCH_VERSION != 'parrots':\n try:\n from torch.utils.cpp_extension import ROCM_HOME\n is_rocm = True if ((torch.version.hip is not None) and\n (ROCM_HOME is not None)) else False\n except ImportError:\n pass\n return is_rocm\n\n\ndef _get_cuda_home():\n if TORCH_VERSION == 'parrots':\n from parrots.utils.build_extension import CUDA_HOME\n else:\n if is_rocm_pytorch():\n from torch.utils.cpp_extension import ROCM_HOME\n CUDA_HOME = ROCM_HOME\n else:\n from torch.utils.cpp_extension import CUDA_HOME\n return CUDA_HOME\n\n\ndef get_build_config():\n if TORCH_VERSION == 'parrots':\n from parrots.config import get_build_info\n return get_build_info()\n else:\n return torch.__config__.show()\n\n\ndef _get_conv():\n if TORCH_VERSION == 'parrots':\n from parrots.nn.modules.conv import _ConvNd, _ConvTransposeMixin\n else:\n from torch.nn.modules.conv import _ConvNd, _ConvTransposeMixin\n return _ConvNd, _ConvTransposeMixin\n\n\ndef _get_dataloader():\n if TORCH_VERSION == 'parrots':\n from torch.utils.data import DataLoader, PoolDataLoader\n else:\n from torch.utils.data import DataLoader\n PoolDataLoader = DataLoader\n return DataLoader, PoolDataLoader\n\n\ndef _get_extension():\n if TORCH_VERSION == 'parrots':\n from parrots.utils.build_extension import BuildExtension, Extension\n CppExtension = partial(Extension, cuda=False)\n CUDAExtension = partial(Extension, cuda=True)\n else:\n from torch.utils.cpp_extension import (BuildExtension, CppExtension,\n CUDAExtension)\n return BuildExtension, CppExtension, CUDAExtension\n\n\ndef _get_pool():\n if TORCH_VERSION == 'parrots':\n from parrots.nn.modules.pool import (_AdaptiveAvgPoolNd,\n _AdaptiveMaxPoolNd, _AvgPoolNd,\n _MaxPoolNd)\n else:\n from torch.nn.modules.pooling import (_AdaptiveAvgPoolNd,\n _AdaptiveMaxPoolNd, _AvgPoolNd,\n _MaxPoolNd)\n return _AdaptiveAvgPoolNd, _AdaptiveMaxPoolNd, _AvgPoolNd, _MaxPoolNd\n\n\ndef _get_norm():\n if TORCH_VERSION == 'parrots':\n from parrots.nn.modules.batchnorm import _BatchNorm, _InstanceNorm\n SyncBatchNorm_ = torch.nn.SyncBatchNorm2d\n else:\n from torch.nn.modules.instancenorm import _InstanceNorm\n from torch.nn.modules.batchnorm import _BatchNorm\n SyncBatchNorm_ = torch.nn.SyncBatchNorm\n return _BatchNorm, _InstanceNorm, SyncBatchNorm_\n\n\n_ConvNd, _ConvTransposeMixin = _get_conv()\nDataLoader, PoolDataLoader = _get_dataloader()\nBuildExtension, CppExtension, CUDAExtension = _get_extension()\n_BatchNorm, _InstanceNorm, SyncBatchNorm_ = _get_norm()\n_AdaptiveAvgPoolNd, _AdaptiveMaxPoolNd, _AvgPoolNd, _MaxPoolNd = _get_pool()\n\n\nclass SyncBatchNorm(SyncBatchNorm_):\n\n def _check_input_dim(self, input):\n if TORCH_VERSION == 'parrots':\n if input.dim() < 2:\n raise ValueError(\n f'expected at least 2D input (got {input.dim()}D input)')\n else:\n super()._check_input_dim(input)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/path.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport os.path as osp\nfrom pathlib import Path\n\nfrom .misc import is_str\n\n\ndef is_filepath(x):\n return is_str(x) or isinstance(x, Path)\n\n\ndef fopen(filepath, *args, **kwargs):\n if is_str(filepath):\n return open(filepath, *args, **kwargs)\n elif isinstance(filepath, Path):\n return filepath.open(*args, **kwargs)\n raise ValueError('`filepath` should be a string or a Path')\n\n\ndef check_file_exist(filename, msg_tmpl='file \"{}\" does not exist'):\n if not osp.isfile(filename):\n raise FileNotFoundError(msg_tmpl.format(filename))\n\n\ndef mkdir_or_exist(dir_name, mode=0o777):\n if dir_name == '':\n return\n dir_name = osp.expanduser(dir_name)\n os.makedirs(dir_name, mode=mode, exist_ok=True)\n\n\ndef symlink(src, dst, overwrite=True, **kwargs):\n if os.path.lexists(dst) and overwrite:\n os.remove(dst)\n os.symlink(src, dst, **kwargs)\n\n\ndef scandir(dir_path, suffix=None, recursive=False, case_sensitive=True):\n \"\"\"Scan a directory to find the interested files.\n\n Args:\n dir_path (str | :obj:`Path`): Path of the directory.\n suffix (str | tuple(str), optional): File suffix that we are\n interested in. Default: None.\n recursive (bool, optional): If set to True, recursively scan the\n directory. Default: False.\n case_sensitive (bool, optional) : If set to False, ignore the case of\n suffix. Default: True.\n\n Returns:\n A generator for all the interested files with relative paths.\n \"\"\"\n if isinstance(dir_path, (str, Path)):\n dir_path = str(dir_path)\n else:\n raise TypeError('\"dir_path\" must be a string or Path object')\n\n if (suffix is not None) and not isinstance(suffix, (str, tuple)):\n raise TypeError('\"suffix\" must be a string or tuple of strings')\n\n if suffix is not None and not case_sensitive:\n suffix = suffix.lower() if isinstance(suffix, str) else tuple(\n item.lower() for item in suffix)\n\n root = dir_path\n\n def _scandir(dir_path, suffix, recursive, case_sensitive):\n for entry in os.scandir(dir_path):\n if not entry.name.startswith('.') and entry.is_file():\n rel_path = osp.relpath(entry.path, root)\n _rel_path = rel_path if case_sensitive else rel_path.lower()\n if suffix is None or _rel_path.endswith(suffix):\n yield rel_path\n elif recursive and os.path.isdir(entry.path):\n # scan recursively if entry.path is a directory\n yield from _scandir(entry.path, suffix, recursive,\n case_sensitive)\n\n return _scandir(dir_path, suffix, recursive, case_sensitive)\n\n\ndef find_vcs_root(path, markers=('.git', )):\n \"\"\"Finds the root directory (including itself) of specified markers.\n\n Args:\n path (str): Path of directory or file.\n markers (list[str], optional): List of file or directory names.\n\n Returns:\n The directory contained one of the markers or None if not found.\n \"\"\"\n if osp.isfile(path):\n path = osp.dirname(path)\n\n prev, cur = None, osp.abspath(osp.expanduser(path))\n while cur != prev:\n if any(osp.exists(osp.join(cur, marker)) for marker in markers):\n return cur\n prev, cur = cur, osp.split(cur)[0]\n return None\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/progressbar.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport sys\nfrom collections.abc import Iterable\nfrom multiprocessing import Pool\nfrom shutil import get_terminal_size\n\nfrom .timer import Timer\n\n\nclass ProgressBar:\n \"\"\"A progress bar which can print the progress.\"\"\"\n\n def __init__(self, task_num=0, bar_width=50, start=True, file=sys.stdout):\n self.task_num = task_num\n self.bar_width = bar_width\n self.completed = 0\n self.file = file\n if start:\n self.start()\n\n @property\n def terminal_width(self):\n width, _ = get_terminal_size()\n return width\n\n def start(self):\n if self.task_num > 0:\n self.file.write(f'[{\" \" * self.bar_width}] 0/{self.task_num}, '\n 'elapsed: 0s, ETA:')\n else:\n self.file.write('completed: 0, elapsed: 0s')\n self.file.flush()\n self.timer = Timer()\n\n def update(self, num_tasks=1):\n assert num_tasks > 0\n self.completed += num_tasks\n elapsed = self.timer.since_start()\n if elapsed > 0:\n fps = self.completed / elapsed\n else:\n fps = float('inf')\n if self.task_num > 0:\n percentage = self.completed / float(self.task_num)\n eta = int(elapsed * (1 - percentage) / percentage + 0.5)\n msg = f'\\r[{{}}] {self.completed}/{self.task_num}, ' \\\n f'{fps:.1f} task/s, elapsed: {int(elapsed + 0.5)}s, ' \\\n f'ETA: {eta:5}s'\n\n bar_width = min(self.bar_width,\n int(self.terminal_width - len(msg)) + 2,\n int(self.terminal_width * 0.6))\n bar_width = max(2, bar_width)\n mark_width = int(bar_width * percentage)\n bar_chars = '>' * mark_width + ' ' * (bar_width - mark_width)\n self.file.write(msg.format(bar_chars))\n else:\n self.file.write(\n f'completed: {self.completed}, elapsed: {int(elapsed + 0.5)}s,'\n f' {fps:.1f} tasks/s')\n self.file.flush()\n\n\ndef track_progress(func, tasks, bar_width=50, file=sys.stdout, **kwargs):\n \"\"\"Track the progress of tasks execution with a progress bar.\n\n Tasks are done with a simple for-loop.\n\n Args:\n func (callable): The function to be applied to each task.\n tasks (list or tuple[Iterable, int]): A list of tasks or\n (tasks, total num).\n bar_width (int): Width of progress bar.\n\n Returns:\n list: The task results.\n \"\"\"\n if isinstance(tasks, tuple):\n assert len(tasks) == 2\n assert isinstance(tasks[0], Iterable)\n assert isinstance(tasks[1], int)\n task_num = tasks[1]\n tasks = tasks[0]\n elif isinstance(tasks, Iterable):\n task_num = len(tasks)\n else:\n raise TypeError(\n '\"tasks\" must be an iterable object or a (iterator, int) tuple')\n prog_bar = ProgressBar(task_num, bar_width, file=file)\n results = []\n for task in tasks:\n results.append(func(task, **kwargs))\n prog_bar.update()\n prog_bar.file.write('\\n')\n return results\n\n\ndef init_pool(process_num, initializer=None, initargs=None):\n if initializer is None:\n return Pool(process_num)\n elif initargs is None:\n return Pool(process_num, initializer)\n else:\n if not isinstance(initargs, tuple):\n raise TypeError('\"initargs\" must be a tuple')\n return Pool(process_num, initializer, initargs)\n\n\ndef track_parallel_progress(func,\n tasks,\n nproc,\n initializer=None,\n initargs=None,\n bar_width=50,\n chunksize=1,\n skip_first=False,\n keep_order=True,\n file=sys.stdout):\n \"\"\"Track the progress of parallel task execution with a progress bar.\n\n The built-in :mod:`multiprocessing` module is used for process pools and\n tasks are done with :func:`Pool.map` or :func:`Pool.imap_unordered`.\n\n Args:\n func (callable): The function to be applied to each task.\n tasks (list or tuple[Iterable, int]): A list of tasks or\n (tasks, total num).\n nproc (int): Process (worker) number.\n initializer (None or callable): Refer to :class:`multiprocessing.Pool`\n for details.\n initargs (None or tuple): Refer to :class:`multiprocessing.Pool` for\n details.\n chunksize (int): Refer to :class:`multiprocessing.Pool` for details.\n bar_width (int): Width of progress bar.\n skip_first (bool): Whether to skip the first sample for each worker\n when estimating fps, since the initialization step may takes\n longer.\n keep_order (bool): If True, :func:`Pool.imap` is used, otherwise\n :func:`Pool.imap_unordered` is used.\n\n Returns:\n list: The task results.\n \"\"\"\n if isinstance(tasks, tuple):\n assert len(tasks) == 2\n assert isinstance(tasks[0], Iterable)\n assert isinstance(tasks[1], int)\n task_num = tasks[1]\n tasks = tasks[0]\n elif isinstance(tasks, Iterable):\n task_num = len(tasks)\n else:\n raise TypeError(\n '\"tasks\" must be an iterable object or a (iterator, int) tuple')\n pool = init_pool(nproc, initializer, initargs)\n start = not skip_first\n task_num -= nproc * chunksize * int(skip_first)\n prog_bar = ProgressBar(task_num, bar_width, start, file=file)\n results = []\n if keep_order:\n gen = pool.imap(func, tasks, chunksize)\n else:\n gen = pool.imap_unordered(func, tasks, chunksize)\n for result in gen:\n results.append(result)\n if skip_first:\n if len(results) < nproc * chunksize:\n continue\n elif len(results) == nproc * chunksize:\n prog_bar.start()\n continue\n prog_bar.update()\n prog_bar.file.write('\\n')\n pool.close()\n pool.join()\n return results\n\n\ndef track_iter_progress(tasks, bar_width=50, file=sys.stdout):\n \"\"\"Track the progress of tasks iteration or enumeration with a progress\n bar.\n\n Tasks are yielded with a simple for-loop.\n\n Args:\n tasks (list or tuple[Iterable, int]): A list of tasks or\n (tasks, total num).\n bar_width (int): Width of progress bar.\n\n Yields:\n list: The task results.\n \"\"\"\n if isinstance(tasks, tuple):\n assert len(tasks) == 2\n assert isinstance(tasks[0], Iterable)\n assert isinstance(tasks[1], int)\n task_num = tasks[1]\n tasks = tasks[0]\n elif isinstance(tasks, Iterable):\n task_num = len(tasks)\n else:\n raise TypeError(\n '\"tasks\" must be an iterable object or a (iterator, int) tuple')\n prog_bar = ProgressBar(task_num, bar_width, file=file)\n for task in tasks:\n yield task\n prog_bar.update()\n prog_bar.file.write('\\n')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/registry.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport inspect\nimport warnings\nfrom functools import partial\n\nfrom .misc import is_seq_of\n\n\ndef build_from_cfg(cfg, registry, default_args=None):\n \"\"\"Build a module from config dict.\n\n Args:\n cfg (dict): Config dict. It should at least contain the key \"type\".\n registry (:obj:`Registry`): The registry to search the type from.\n default_args (dict, optional): Default initialization arguments.\n\n Returns:\n object: The constructed object.\n \"\"\"\n if not isinstance(cfg, dict):\n raise TypeError(f'cfg must be a dict, but got {type(cfg)}')\n if 'type' not in cfg:\n if default_args is None or 'type' not in default_args:\n raise KeyError(\n '`cfg` or `default_args` must contain the key \"type\", '\n f'but got {cfg}\\n{default_args}')\n if not isinstance(registry, Registry):\n raise TypeError('registry must be an mmcv.Registry object, '\n f'but got {type(registry)}')\n if not (isinstance(default_args, dict) or default_args is None):\n raise TypeError('default_args must be a dict or None, '\n f'but got {type(default_args)}')\n\n args = cfg.copy()\n\n if default_args is not None:\n for name, value in default_args.items():\n args.setdefault(name, value)\n\n obj_type = args.pop('type')\n if isinstance(obj_type, str):\n obj_cls = registry.get(obj_type)\n if obj_cls is None:\n raise KeyError(\n f'{obj_type} is not in the {registry.name} registry')\n elif inspect.isclass(obj_type):\n obj_cls = obj_type\n else:\n raise TypeError(\n f'type must be a str or valid type, but got {type(obj_type)}')\n try:\n return obj_cls(**args)\n except Exception as e:\n # Normal TypeError does not print class name.\n raise type(e)(f'{obj_cls.__name__}: {e}')\n\n\nclass Registry:\n \"\"\"A registry to map strings to classes.\n\n Registered object could be built from registry.\n\n Example:\n >>> MODELS = Registry('models')\n >>> @MODELS.register_module()\n >>> class ResNet:\n >>> pass\n >>> resnet = MODELS.build(dict(type='ResNet'))\n\n Please refer to\n https://mmcv.readthedocs.io/en/latest/understand_mmcv/registry.html for\n advanced usage.\n\n Args:\n name (str): Registry name.\n build_func(func, optional): Build function to construct instance from\n Registry, func:`build_from_cfg` is used if neither ``parent`` or\n ``build_func`` is specified. If ``parent`` is specified and\n ``build_func`` is not given, ``build_func`` will be inherited\n from ``parent``. Default: None.\n parent (Registry, optional): Parent registry. The class registered in\n children registry could be built from parent. Default: None.\n scope (str, optional): The scope of registry. It is the key to search\n for children registry. If not specified, scope will be the name of\n the package where class is defined, e.g. mmdet, mmcls, mmseg.\n Default: None.\n \"\"\"\n\n def __init__(self, name, build_func=None, parent=None, scope=None):\n self._name = name\n self._module_dict = dict()\n self._children = dict()\n self._scope = self.infer_scope() if scope is None else scope\n\n # self.build_func will be set with the following priority:\n # 1. build_func\n # 2. parent.build_func\n # 3. build_from_cfg\n if build_func is None:\n if parent is not None:\n self.build_func = parent.build_func\n else:\n self.build_func = build_from_cfg\n else:\n self.build_func = build_func\n if parent is not None:\n assert isinstance(parent, Registry)\n parent._add_children(self)\n self.parent = parent\n else:\n self.parent = None\n\n def __len__(self):\n return len(self._module_dict)\n\n def __contains__(self, key):\n return self.get(key) is not None\n\n def __repr__(self):\n format_str = self.__class__.__name__ + \\\n f'(name={self._name}, ' \\\n f'items={self._module_dict})'\n return format_str\n\n @staticmethod\n def infer_scope():\n \"\"\"Infer the scope of registry.\n\n The name of the package where registry is defined will be returned.\n\n Example:\n >>> # in mmdet/models/backbone/resnet.py\n >>> MODELS = Registry('models')\n >>> @MODELS.register_module()\n >>> class ResNet:\n >>> pass\n The scope of ``ResNet`` will be ``mmdet``.\n\n Returns:\n str: The inferred scope name.\n \"\"\"\n # inspect.stack() trace where this function is called, the index-2\n # indicates the frame where `infer_scope()` is called\n filename = inspect.getmodule(inspect.stack()[2][0]).__name__\n split_filename = filename.split('.')\n return split_filename[0]\n\n @staticmethod\n def split_scope_key(key):\n \"\"\"Split scope and key.\n\n The first scope will be split from key.\n\n Examples:\n >>> Registry.split_scope_key('mmdet.ResNet')\n 'mmdet', 'ResNet'\n >>> Registry.split_scope_key('ResNet')\n None, 'ResNet'\n\n Return:\n tuple[str | None, str]: The former element is the first scope of\n the key, which can be ``None``. The latter is the remaining key.\n \"\"\"\n split_index = key.find('.')\n if split_index != -1:\n return key[:split_index], key[split_index + 1:]\n else:\n return None, key\n\n @property\n def name(self):\n return self._name\n\n @property\n def scope(self):\n return self._scope\n\n @property\n def module_dict(self):\n return self._module_dict\n\n @property\n def children(self):\n return self._children\n\n def get(self, key):\n \"\"\"Get the registry record.\n\n Args:\n key (str): The class name in string format.\n\n Returns:\n class: The corresponding class.\n \"\"\"\n scope, real_key = self.split_scope_key(key)\n if scope is None or scope == self._scope:\n # get from self\n if real_key in self._module_dict:\n return self._module_dict[real_key]\n else:\n # get from self._children\n if scope in self._children:\n return self._children[scope].get(real_key)\n else:\n # goto root\n parent = self.parent\n while parent.parent is not None:\n parent = parent.parent\n return parent.get(key)\n\n def build(self, *args, **kwargs):\n return self.build_func(*args, **kwargs, registry=self)\n\n def _add_children(self, registry):\n \"\"\"Add children for a registry.\n\n The ``registry`` will be added as children based on its scope.\n The parent registry could build objects from children registry.\n\n Example:\n >>> models = Registry('models')\n >>> mmdet_models = Registry('models', parent=models)\n >>> @mmdet_models.register_module()\n >>> class ResNet:\n >>> pass\n >>> resnet = models.build(dict(type='mmdet.ResNet'))\n \"\"\"\n\n assert isinstance(registry, Registry)\n assert registry.scope is not None\n assert registry.scope not in self.children, \\\n f'scope {registry.scope} exists in {self.name} registry'\n self.children[registry.scope] = registry\n\n def _register_module(self, module_class, module_name=None, force=False):\n if not inspect.isclass(module_class):\n raise TypeError('module must be a class, '\n f'but got {type(module_class)}')\n if not force and module_name in self._module_dict.keys():\n return self._module_dict[module_name]\n if module_name is None:\n module_name = module_class.__name__\n if isinstance(module_name, str):\n module_name = [module_name]\n for name in module_name:\n if not force and name in self._module_dict:\n # print(isinstance(module_name, list) and len(module_name) == 1)\n raise KeyError(f'{name} is already registered '\n f'in {self.name}')\n # print(f'{name} is already registered in {self.name}')\n\n self._module_dict[name] = module_class\n\n def deprecated_register_module(self, cls=None, force=False):\n warnings.warn(\n 'The old API of register_module(module, force=False) '\n 'is deprecated and will be removed, please use the new API '\n 'register_module(name=None, force=False, module=None) instead.',\n DeprecationWarning)\n if cls is None:\n return partial(self.deprecated_register_module, force=force)\n self._register_module(cls, force=force)\n return cls\n\n def register_module(self, name=None, force=False, module=None):\n \"\"\"Register a module.\n\n A record will be added to `self._module_dict`, whose key is the class\n name or the specified name, and value is the class itself.\n It can be used as a decorator or a normal function.\n\n Example:\n >>> backbones = Registry('backbone')\n >>> @backbones.register_module()\n >>> class ResNet:\n >>> pass\n\n >>> backbones = Registry('backbone')\n >>> @backbones.register_module(name='mnet')\n >>> class MobileNet:\n >>> pass\n\n >>> backbones = Registry('backbone')\n >>> class ResNet:\n >>> pass\n >>> backbones.register_module(ResNet)\n\n Args:\n name (str | None): The module name to be registered. If not\n specified, the class name will be used.\n force (bool, optional): Whether to override an existing class with\n the same name. Default: False.\n module (type): Module class to be registered.\n \"\"\"\n if not isinstance(force, bool):\n raise TypeError(f'force must be a boolean, but got {type(force)}')\n # NOTE: This is a walkaround to be compatible with the old api,\n # while it may introduce unexpected bugs.\n if isinstance(name, type):\n return self.deprecated_register_module(name, force=force)\n\n # raise the error ahead of time\n if not (name is None or isinstance(name, str) or is_seq_of(name, str)):\n raise TypeError(\n 'name must be either of None, an instance of str or a sequence'\n f' of str, but got {type(name)}')\n\n # use it as a normal method: x.register_module(module=SomeClass)\n if module is not None:\n self._register_module(\n module_class=module, module_name=name, force=force)\n return module\n\n # use it as a decorator: @x.register_module()\n def _register(cls):\n self._register_module(\n module_class=cls, module_name=name, force=force)\n return cls\n\n return _register\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/testing.py", "content": "# Copyright (c) Open-MMLab.\nimport sys\nfrom collections.abc import Iterable\nfrom runpy import run_path\nfrom shlex import split\nfrom typing import Any, Dict, List\nfrom unittest.mock import patch\n\n\ndef check_python_script(cmd):\n \"\"\"Run the python cmd script with `__main__`. The difference between\n `os.system` is that, this function exectues code in the current process, so\n that it can be tracked by coverage tools. Currently it supports two forms:\n\n - ./tests/data/scripts/hello.py zz\n - python tests/data/scripts/hello.py zz\n \"\"\"\n args = split(cmd)\n if args[0] == 'python':\n args = args[1:]\n with patch.object(sys, 'argv', args):\n run_path(args[0], run_name='__main__')\n\n\ndef _any(judge_result):\n \"\"\"Since built-in ``any`` works only when the element of iterable is not\n iterable, implement the function.\"\"\"\n if not isinstance(judge_result, Iterable):\n return judge_result\n\n try:\n for element in judge_result:\n if _any(element):\n return True\n except TypeError:\n # Maybe encounter the case: torch.tensor(True) | torch.tensor(False)\n if judge_result:\n return True\n return False\n\n\ndef assert_dict_contains_subset(dict_obj: Dict[Any, Any],\n expected_subset: Dict[Any, Any]) -> bool:\n \"\"\"Check if the dict_obj contains the expected_subset.\n\n Args:\n dict_obj (Dict[Any, Any]): Dict object to be checked.\n expected_subset (Dict[Any, Any]): Subset expected to be contained in\n dict_obj.\n\n Returns:\n bool: Whether the dict_obj contains the expected_subset.\n \"\"\"\n\n for key, value in expected_subset.items():\n if key not in dict_obj.keys() or _any(dict_obj[key] != value):\n return False\n return True\n\n\ndef assert_attrs_equal(obj: Any, expected_attrs: Dict[str, Any]) -> bool:\n \"\"\"Check if attribute of class object is correct.\n\n Args:\n obj (object): Class object to be checked.\n expected_attrs (Dict[str, Any]): Dict of the expected attrs.\n\n Returns:\n bool: Whether the attribute of class object is correct.\n \"\"\"\n for attr, value in expected_attrs.items():\n if not hasattr(obj, attr) or _any(getattr(obj, attr) != value):\n return False\n return True\n\n\ndef assert_dict_has_keys(obj: Dict[str, Any],\n expected_keys: List[str]) -> bool:\n \"\"\"Check if the obj has all the expected_keys.\n\n Args:\n obj (Dict[str, Any]): Object to be checked.\n expected_keys (List[str]): Keys expected to contained in the keys of\n the obj.\n\n Returns:\n bool: Whether the obj has the expected keys.\n \"\"\"\n return set(expected_keys).issubset(set(obj.keys()))\n\n\ndef assert_keys_equal(result_keys: List[str], target_keys: List[str]) -> bool:\n \"\"\"Check if target_keys is equal to result_keys.\n\n Args:\n result_keys (List[str]): Result keys to be checked.\n target_keys (List[str]): Target keys to be checked.\n\n Returns:\n bool: Whether target_keys is equal to result_keys.\n \"\"\"\n return set(result_keys) == set(target_keys)\n\n\ndef assert_is_norm_layer(module) -> bool:\n \"\"\"Check if the module is a norm layer.\n\n Args:\n module (nn.Module): The module to be checked.\n\n Returns:\n bool: Whether the module is a norm layer.\n \"\"\"\n from .parrots_wrapper import _BatchNorm, _InstanceNorm\n from torch.nn import GroupNorm, LayerNorm\n norm_layer_candidates = (_BatchNorm, _InstanceNorm, GroupNorm, LayerNorm)\n return isinstance(module, norm_layer_candidates)\n\n\ndef assert_params_all_zeros(module) -> bool:\n \"\"\"Check if the parameters of the module is all zeros.\n\n Args:\n module (nn.Module): The module to be checked.\n\n Returns:\n bool: Whether the parameters of the module is all zeros.\n \"\"\"\n weight_data = module.weight.data\n is_weight_zero = weight_data.allclose(\n weight_data.new_zeros(weight_data.size()))\n\n if hasattr(module, 'bias') and module.bias is not None:\n bias_data = module.bias.data\n is_bias_zero = bias_data.allclose(\n bias_data.new_zeros(bias_data.size()))\n else:\n is_bias_zero = True\n\n return is_weight_zero and is_bias_zero\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/timer.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom time import time\n\n\nclass TimerError(Exception):\n\n def __init__(self, message):\n self.message = message\n super(TimerError, self).__init__(message)\n\n\nclass Timer:\n \"\"\"A flexible Timer class.\n\n Examples:\n >>> import time\n >>> import mmcv\n >>> with mmcv.Timer():\n >>> # simulate a code block that will run for 1s\n >>> time.sleep(1)\n 1.000\n >>> with mmcv.Timer(print_tmpl='it takes {:.1f} seconds'):\n >>> # simulate a code block that will run for 1s\n >>> time.sleep(1)\n it takes 1.0 seconds\n >>> timer = mmcv.Timer()\n >>> time.sleep(0.5)\n >>> print(timer.since_start())\n 0.500\n >>> time.sleep(0.5)\n >>> print(timer.since_last_check())\n 0.500\n >>> print(timer.since_start())\n 1.000\n \"\"\"\n\n def __init__(self, start=True, print_tmpl=None):\n self._is_running = False\n self.print_tmpl = print_tmpl if print_tmpl else '{:.3f}'\n if start:\n self.start()\n\n @property\n def is_running(self):\n \"\"\"bool: indicate whether the timer is running\"\"\"\n return self._is_running\n\n def __enter__(self):\n self.start()\n return self\n\n def __exit__(self, type, value, traceback):\n print(self.print_tmpl.format(self.since_last_check()))\n self._is_running = False\n\n def start(self):\n \"\"\"Start the timer.\"\"\"\n if not self._is_running:\n self._t_start = time()\n self._is_running = True\n self._t_last = time()\n\n def since_start(self):\n \"\"\"Total time since the timer is started.\n\n Returns:\n float: Time in seconds.\n \"\"\"\n if not self._is_running:\n raise TimerError('timer is not running')\n self._t_last = time()\n return self._t_last - self._t_start\n\n def since_last_check(self):\n \"\"\"Time since the last checking.\n\n Either :func:`since_start` or :func:`since_last_check` is a checking\n operation.\n\n Returns:\n float: Time in seconds.\n \"\"\"\n if not self._is_running:\n raise TimerError('timer is not running')\n dur = time() - self._t_last\n self._t_last = time()\n return dur\n\n\n_g_timers = {} # global timers\n\n\ndef check_time(timer_id):\n \"\"\"Add check points in a single line.\n\n This method is suitable for running a task on a list of items. A timer will\n be registered when the method is called for the first time.\n\n Examples:\n >>> import time\n >>> import mmcv\n >>> for i in range(1, 6):\n >>> # simulate a code block\n >>> time.sleep(i)\n >>> mmcv.check_time('task1')\n 2.000\n 3.000\n 4.000\n 5.000\n\n Args:\n str: Timer identifier.\n \"\"\"\n if timer_id not in _g_timers:\n _g_timers[timer_id] = Timer()\n return 0\n else:\n return _g_timers[timer_id].since_last_check()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/trace.py", "content": "import warnings\n\nimport torch\n\nfrom mmcv.utils import digit_version\n\n\ndef is_jit_tracing() -> bool:\n if (torch.__version__ != 'parrots'\n and digit_version(torch.__version__) >= digit_version('1.6.0')):\n on_trace = torch.jit.is_tracing()\n # In PyTorch 1.6, torch.jit.is_tracing has a bug.\n # Refers to https://github.com/pytorch/pytorch/issues/42448\n if isinstance(on_trace, bool):\n return on_trace\n else:\n return torch._C._is_tracing()\n else:\n warnings.warn(\n 'torch.jit.is_tracing is only supported after v1.6.0. '\n 'Therefore is_tracing returns False automatically. Please '\n 'set on_trace manually if you are using trace.', UserWarning)\n return False\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/utils/version_utils.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport subprocess\nimport warnings\n\nfrom packaging.version import parse\n\n\ndef digit_version(version_str: str, length: int = 4):\n \"\"\"Convert a version string into a tuple of integers.\n\n This method is usually used for comparing two versions. For pre-release\n versions: alpha < beta < rc.\n\n Args:\n version_str (str): The version string.\n length (int): The maximum number of version levels. Default: 4.\n\n Returns:\n tuple[int]: The version info in digits (integers).\n \"\"\"\n assert 'parrots' not in version_str\n version = parse(version_str)\n assert version.release, f'failed to parse version {version_str}'\n release = list(version.release)\n release = release[:length]\n if len(release) < length:\n release = release + [0] * (length - len(release))\n if version.is_prerelease:\n mapping = {'a': -3, 'b': -2, 'rc': -1}\n val = -4\n # version.pre can be None\n if version.pre:\n if version.pre[0] not in mapping:\n warnings.warn(f'unknown prerelease version {version.pre[0]}, '\n 'version checking may go wrong')\n else:\n val = mapping[version.pre[0]]\n release.extend([val, version.pre[-1]])\n else:\n release.extend([val, 0])\n\n elif version.is_postrelease:\n release.extend([1, version.post])\n else:\n release.extend([0, 0])\n return tuple(release)\n\n\ndef _minimal_ext_cmd(cmd):\n # construct minimal environment\n env = {}\n for k in ['SYSTEMROOT', 'PATH', 'HOME']:\n v = os.environ.get(k)\n if v is not None:\n env[k] = v\n # LANGUAGE is used on win32\n env['LANGUAGE'] = 'C'\n env['LANG'] = 'C'\n env['LC_ALL'] = 'C'\n out = subprocess.Popen(\n cmd, stdout=subprocess.PIPE, env=env).communicate()[0]\n return out\n\n\ndef get_git_hash(fallback='unknown', digits=None):\n \"\"\"Get the git hash of the current repo.\n\n Args:\n fallback (str, optional): The fallback string when git hash is\n unavailable. Defaults to 'unknown'.\n digits (int, optional): kept digits of the hash. Defaults to None,\n meaning all digits are kept.\n\n Returns:\n str: Git commit hash.\n \"\"\"\n\n if digits is not None and not isinstance(digits, int):\n raise TypeError('digits must be None or an integer')\n\n try:\n out = _minimal_ext_cmd(['git', 'rev-parse', 'HEAD'])\n sha = out.strip().decode('ascii')\n if digits is not None:\n sha = sha[:digits]\n except OSError:\n sha = fallback\n\n return sha\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/version.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\n__version__ = '1.4.4'\n\n\ndef parse_version_info(version_str: str, length: int = 4) -> tuple:\n \"\"\"Parse a version string into a tuple.\n\n Args:\n version_str (str): The version string.\n length (int): The maximum number of version levels. Default: 4.\n\n Returns:\n tuple[int | str]: The version info, e.g., \"1.3.0\" is parsed into\n (1, 3, 0, 0, 0, 0), and \"2.0.0rc1\" is parsed into\n (2, 0, 0, 0, 'rc', 1) (when length is set to 4).\n \"\"\"\n from packaging.version import parse\n version = parse(version_str)\n assert version.release, f'failed to parse version {version_str}'\n release = list(version.release)\n release = release[:length]\n if len(release) < length:\n release = release + [0] * (length - len(release))\n if version.is_prerelease:\n release.extend(list(version.pre))\n elif version.is_postrelease:\n release.extend(list(version.post))\n else:\n release.extend([0, 0])\n return tuple(release)\n\n\nversion_info = tuple(int(x) for x in __version__.split('.')[:3])\n\n__all__ = ['__version__', 'version_info', 'parse_version_info']\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/video/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .io import Cache, VideoReader, frames2video\nfrom .optflow import (dequantize_flow, flow_from_bytes, flow_warp, flowread,\n flowwrite, quantize_flow, sparse_flow_from_bytes)\nfrom .processing import concat_video, convert_video, cut_video, resize_video\n\n__all__ = [\n 'Cache', 'VideoReader', 'frames2video', 'convert_video', 'resize_video',\n 'cut_video', 'concat_video', 'flowread', 'flowwrite', 'quantize_flow',\n 'dequantize_flow', 'flow_warp', 'flow_from_bytes', 'sparse_flow_from_bytes'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/video/io.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\nfrom collections import OrderedDict\n\nimport cv2\nfrom cv2 import (CAP_PROP_FOURCC, CAP_PROP_FPS, CAP_PROP_FRAME_COUNT,\n CAP_PROP_FRAME_HEIGHT, CAP_PROP_FRAME_WIDTH,\n CAP_PROP_POS_FRAMES, VideoWriter_fourcc)\n\nfrom mmcv.utils import (check_file_exist, mkdir_or_exist, scandir,\n track_progress)\n\n\nclass Cache:\n\n def __init__(self, capacity):\n self._cache = OrderedDict()\n self._capacity = int(capacity)\n if capacity <= 0:\n raise ValueError('capacity must be a positive integer')\n\n @property\n def capacity(self):\n return self._capacity\n\n @property\n def size(self):\n return len(self._cache)\n\n def put(self, key, val):\n if key in self._cache:\n return\n if len(self._cache) >= self.capacity:\n self._cache.popitem(last=False)\n self._cache[key] = val\n\n def get(self, key, default=None):\n val = self._cache[key] if key in self._cache else default\n return val\n\n\nclass VideoReader:\n \"\"\"Video class with similar usage to a list object.\n\n This video warpper class provides convenient apis to access frames.\n There exists an issue of OpenCV's VideoCapture class that jumping to a\n certain frame may be inaccurate. It is fixed in this class by checking\n the position after jumping each time.\n Cache is used when decoding videos. So if the same frame is visited for\n the second time, there is no need to decode again if it is stored in the\n cache.\n\n Examples:\n >>> import mmcv\n >>> v = mmcv.VideoReader('sample.mp4')\n >>> len(v) # get the total frame number with `len()`\n 120\n >>> for img in v: # v is iterable\n >>> mmcv.imshow(img)\n >>> v[5] # get the 6th frame\n \"\"\"\n\n def __init__(self, filename, cache_capacity=10):\n # Check whether the video path is a url\n if not filename.startswith(('https://', 'http://')):\n check_file_exist(filename, 'Video file not found: ' + filename)\n self._vcap = cv2.VideoCapture(filename)\n assert cache_capacity > 0\n self._cache = Cache(cache_capacity)\n self._position = 0\n # get basic info\n self._width = int(self._vcap.get(CAP_PROP_FRAME_WIDTH))\n self._height = int(self._vcap.get(CAP_PROP_FRAME_HEIGHT))\n self._fps = self._vcap.get(CAP_PROP_FPS)\n self._frame_cnt = int(self._vcap.get(CAP_PROP_FRAME_COUNT))\n self._fourcc = self._vcap.get(CAP_PROP_FOURCC)\n\n @property\n def vcap(self):\n \"\"\":obj:`cv2.VideoCapture`: The raw VideoCapture object.\"\"\"\n return self._vcap\n\n @property\n def opened(self):\n \"\"\"bool: Indicate whether the video is opened.\"\"\"\n return self._vcap.isOpened()\n\n @property\n def width(self):\n \"\"\"int: Width of video frames.\"\"\"\n return self._width\n\n @property\n def height(self):\n \"\"\"int: Height of video frames.\"\"\"\n return self._height\n\n @property\n def resolution(self):\n \"\"\"tuple: Video resolution (width, height).\"\"\"\n return (self._width, self._height)\n\n @property\n def fps(self):\n \"\"\"float: FPS of the video.\"\"\"\n return self._fps\n\n @property\n def frame_cnt(self):\n \"\"\"int: Total frames of the video.\"\"\"\n return self._frame_cnt\n\n @property\n def fourcc(self):\n \"\"\"str: \"Four character code\" of the video.\"\"\"\n return self._fourcc\n\n @property\n def position(self):\n \"\"\"int: Current cursor position, indicating frame decoded.\"\"\"\n return self._position\n\n def _get_real_position(self):\n return int(round(self._vcap.get(CAP_PROP_POS_FRAMES)))\n\n def _set_real_position(self, frame_id):\n self._vcap.set(CAP_PROP_POS_FRAMES, frame_id)\n pos = self._get_real_position()\n for _ in range(frame_id - pos):\n self._vcap.read()\n self._position = frame_id\n\n def read(self):\n \"\"\"Read the next frame.\n\n If the next frame have been decoded before and in the cache, then\n return it directly, otherwise decode, cache and return it.\n\n Returns:\n ndarray or None: Return the frame if successful, otherwise None.\n \"\"\"\n # pos = self._position\n if self._cache:\n img = self._cache.get(self._position)\n if img is not None:\n ret = True\n else:\n if self._position != self._get_real_position():\n self._set_real_position(self._position)\n ret, img = self._vcap.read()\n if ret:\n self._cache.put(self._position, img)\n else:\n ret, img = self._vcap.read()\n if ret:\n self._position += 1\n return img\n\n def get_frame(self, frame_id):\n \"\"\"Get frame by index.\n\n Args:\n frame_id (int): Index of the expected frame, 0-based.\n\n Returns:\n ndarray or None: Return the frame if successful, otherwise None.\n \"\"\"\n if frame_id < 0 or frame_id >= self._frame_cnt:\n raise IndexError(\n f'\"frame_id\" must be between 0 and {self._frame_cnt - 1}')\n if frame_id == self._position:\n return self.read()\n if self._cache:\n img = self._cache.get(frame_id)\n if img is not None:\n self._position = frame_id + 1\n return img\n self._set_real_position(frame_id)\n ret, img = self._vcap.read()\n if ret:\n if self._cache:\n self._cache.put(self._position, img)\n self._position += 1\n return img\n\n def current_frame(self):\n \"\"\"Get the current frame (frame that is just visited).\n\n Returns:\n ndarray or None: If the video is fresh, return None, otherwise\n return the frame.\n \"\"\"\n if self._position == 0:\n return None\n return self._cache.get(self._position - 1)\n\n def cvt2frames(self,\n frame_dir,\n file_start=0,\n filename_tmpl='{:06d}.jpg',\n start=0,\n max_num=0,\n show_progress=True):\n \"\"\"Convert a video to frame images.\n\n Args:\n frame_dir (str): Output directory to store all the frame images.\n file_start (int): Filenames will start from the specified number.\n filename_tmpl (str): Filename template with the index as the\n placeholder.\n start (int): The starting frame index.\n max_num (int): Maximum number of frames to be written.\n show_progress (bool): Whether to show a progress bar.\n \"\"\"\n mkdir_or_exist(frame_dir)\n if max_num == 0:\n task_num = self.frame_cnt - start\n else:\n task_num = min(self.frame_cnt - start, max_num)\n if task_num <= 0:\n raise ValueError('start must be less than total frame number')\n if start > 0:\n self._set_real_position(start)\n\n def write_frame(file_idx):\n img = self.read()\n if img is None:\n return\n filename = osp.join(frame_dir, filename_tmpl.format(file_idx))\n cv2.imwrite(filename, img)\n\n if show_progress:\n track_progress(write_frame, range(file_start,\n file_start + task_num))\n else:\n for i in range(task_num):\n write_frame(file_start + i)\n\n def __len__(self):\n return self.frame_cnt\n\n def __getitem__(self, index):\n if isinstance(index, slice):\n return [\n self.get_frame(i)\n for i in range(*index.indices(self.frame_cnt))\n ]\n # support negative indexing\n if index < 0:\n index += self.frame_cnt\n if index < 0:\n raise IndexError('index out of range')\n return self.get_frame(index)\n\n def __iter__(self):\n self._set_real_position(0)\n return self\n\n def __next__(self):\n img = self.read()\n if img is not None:\n return img\n else:\n raise StopIteration\n\n next = __next__\n\n def __enter__(self):\n return self\n\n def __exit__(self, exc_type, exc_value, traceback):\n self._vcap.release()\n\n\ndef frames2video(frame_dir,\n video_file,\n fps=30,\n fourcc='XVID',\n filename_tmpl='{:06d}.jpg',\n start=0,\n end=0,\n show_progress=True):\n \"\"\"Read the frame images from a directory and join them as a video.\n\n Args:\n frame_dir (str): The directory containing video frames.\n video_file (str): Output filename.\n fps (float): FPS of the output video.\n fourcc (str): Fourcc of the output video, this should be compatible\n with the output file type.\n filename_tmpl (str): Filename template with the index as the variable.\n start (int): Starting frame index.\n end (int): Ending frame index.\n show_progress (bool): Whether to show a progress bar.\n \"\"\"\n if end == 0:\n ext = filename_tmpl.split('.')[-1]\n end = len([name for name in scandir(frame_dir, ext)])\n first_file = osp.join(frame_dir, filename_tmpl.format(start))\n check_file_exist(first_file, 'The start frame not found: ' + first_file)\n img = cv2.imread(first_file)\n height, width = img.shape[:2]\n resolution = (width, height)\n vwriter = cv2.VideoWriter(video_file, VideoWriter_fourcc(*fourcc), fps,\n resolution)\n\n def write_frame(file_idx):\n filename = osp.join(frame_dir, filename_tmpl.format(file_idx))\n img = cv2.imread(filename)\n vwriter.write(img)\n\n if show_progress:\n track_progress(write_frame, range(start, end))\n else:\n for i in range(start, end):\n write_frame(i)\n vwriter.release()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/video/optflow.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport warnings\n\nimport cv2\nimport numpy as np\n\nfrom mmcv.arraymisc import dequantize, quantize\nfrom mmcv.image import imread, imwrite\nfrom mmcv.utils import is_str\n\n\ndef flowread(flow_or_path, quantize=False, concat_axis=0, *args, **kwargs):\n \"\"\"Read an optical flow map.\n\n Args:\n flow_or_path (ndarray or str): A flow map or filepath.\n quantize (bool): whether to read quantized pair, if set to True,\n remaining args will be passed to :func:`dequantize_flow`.\n concat_axis (int): The axis that dx and dy are concatenated,\n can be either 0 or 1. Ignored if quantize is False.\n\n Returns:\n ndarray: Optical flow represented as a (h, w, 2) numpy array\n \"\"\"\n if isinstance(flow_or_path, np.ndarray):\n if (flow_or_path.ndim != 3) or (flow_or_path.shape[-1] != 2):\n raise ValueError(f'Invalid flow with shape {flow_or_path.shape}')\n return flow_or_path\n elif not is_str(flow_or_path):\n raise TypeError(f'\"flow_or_path\" must be a filename or numpy array, '\n f'not {type(flow_or_path)}')\n\n if not quantize:\n with open(flow_or_path, 'rb') as f:\n try:\n header = f.read(4).decode('utf-8')\n except Exception:\n raise IOError(f'Invalid flow file: {flow_or_path}')\n else:\n if header != 'PIEH':\n raise IOError(f'Invalid flow file: {flow_or_path}, '\n 'header does not contain PIEH')\n\n w = np.fromfile(f, np.int32, 1).squeeze()\n h = np.fromfile(f, np.int32, 1).squeeze()\n flow = np.fromfile(f, np.float32, w * h * 2).reshape((h, w, 2))\n else:\n assert concat_axis in [0, 1]\n cat_flow = imread(flow_or_path, flag='unchanged')\n if cat_flow.ndim != 2:\n raise IOError(\n f'{flow_or_path} is not a valid quantized flow file, '\n f'its dimension is {cat_flow.ndim}.')\n assert cat_flow.shape[concat_axis] % 2 == 0\n dx, dy = np.split(cat_flow, 2, axis=concat_axis)\n flow = dequantize_flow(dx, dy, *args, **kwargs)\n\n return flow.astype(np.float32)\n\n\ndef flowwrite(flow, filename, quantize=False, concat_axis=0, *args, **kwargs):\n \"\"\"Write optical flow to file.\n\n If the flow is not quantized, it will be saved as a .flo file losslessly,\n otherwise a jpeg image which is lossy but of much smaller size. (dx and dy\n will be concatenated horizontally into a single image if quantize is True.)\n\n Args:\n flow (ndarray): (h, w, 2) array of optical flow.\n filename (str): Output filepath.\n quantize (bool): Whether to quantize the flow and save it to 2 jpeg\n images. If set to True, remaining args will be passed to\n :func:`quantize_flow`.\n concat_axis (int): The axis that dx and dy are concatenated,\n can be either 0 or 1. Ignored if quantize is False.\n \"\"\"\n if not quantize:\n with open(filename, 'wb') as f:\n f.write('PIEH'.encode('utf-8'))\n np.array([flow.shape[1], flow.shape[0]], dtype=np.int32).tofile(f)\n flow = flow.astype(np.float32)\n flow.tofile(f)\n f.flush()\n else:\n assert concat_axis in [0, 1]\n dx, dy = quantize_flow(flow, *args, **kwargs)\n dxdy = np.concatenate((dx, dy), axis=concat_axis)\n imwrite(dxdy, filename)\n\n\ndef quantize_flow(flow, max_val=0.02, norm=True):\n \"\"\"Quantize flow to [0, 255].\n\n After this step, the size of flow will be much smaller, and can be\n dumped as jpeg images.\n\n Args:\n flow (ndarray): (h, w, 2) array of optical flow.\n max_val (float): Maximum value of flow, values beyond\n [-max_val, max_val] will be truncated.\n norm (bool): Whether to divide flow values by image width/height.\n\n Returns:\n tuple[ndarray]: Quantized dx and dy.\n \"\"\"\n h, w, _ = flow.shape\n dx = flow[..., 0]\n dy = flow[..., 1]\n if norm:\n dx = dx / w # avoid inplace operations\n dy = dy / h\n # use 255 levels instead of 256 to make sure 0 is 0 after dequantization.\n flow_comps = [\n quantize(d, -max_val, max_val, 255, np.uint8) for d in [dx, dy]\n ]\n return tuple(flow_comps)\n\n\ndef dequantize_flow(dx, dy, max_val=0.02, denorm=True):\n \"\"\"Recover from quantized flow.\n\n Args:\n dx (ndarray): Quantized dx.\n dy (ndarray): Quantized dy.\n max_val (float): Maximum value used when quantizing.\n denorm (bool): Whether to multiply flow values with width/height.\n\n Returns:\n ndarray: Dequantized flow.\n \"\"\"\n assert dx.shape == dy.shape\n assert dx.ndim == 2 or (dx.ndim == 3 and dx.shape[-1] == 1)\n\n dx, dy = [dequantize(d, -max_val, max_val, 255) for d in [dx, dy]]\n\n if denorm:\n dx *= dx.shape[1]\n dy *= dx.shape[0]\n flow = np.dstack((dx, dy))\n return flow\n\n\ndef flow_warp(img, flow, filling_value=0, interpolate_mode='nearest'):\n \"\"\"Use flow to warp img.\n\n Args:\n img (ndarray, float or uint8): Image to be warped.\n flow (ndarray, float): Optical Flow.\n filling_value (int): The missing pixels will be set with filling_value.\n interpolate_mode (str): bilinear -> Bilinear Interpolation;\n nearest -> Nearest Neighbor.\n\n Returns:\n ndarray: Warped image with the same shape of img\n \"\"\"\n warnings.warn('This function is just for prototyping and cannot '\n 'guarantee the computational efficiency.')\n assert flow.ndim == 3, 'Flow must be in 3D arrays.'\n height = flow.shape[0]\n width = flow.shape[1]\n channels = img.shape[2]\n\n output = np.ones(\n (height, width, channels), dtype=img.dtype) * filling_value\n\n grid = np.indices((height, width)).swapaxes(0, 1).swapaxes(1, 2)\n dx = grid[:, :, 0] + flow[:, :, 1]\n dy = grid[:, :, 1] + flow[:, :, 0]\n sx = np.floor(dx).astype(int)\n sy = np.floor(dy).astype(int)\n valid = (sx >= 0) & (sx < height - 1) & (sy >= 0) & (sy < width - 1)\n\n if interpolate_mode == 'nearest':\n output[valid, :] = img[dx[valid].round().astype(int),\n dy[valid].round().astype(int), :]\n elif interpolate_mode == 'bilinear':\n # dirty walkround for integer positions\n eps_ = 1e-6\n dx, dy = dx + eps_, dy + eps_\n left_top_ = img[np.floor(dx[valid]).astype(int),\n np.floor(dy[valid]).astype(int), :] * (\n np.ceil(dx[valid]) - dx[valid])[:, None] * (\n np.ceil(dy[valid]) - dy[valid])[:, None]\n left_down_ = img[np.ceil(dx[valid]).astype(int),\n np.floor(dy[valid]).astype(int), :] * (\n dx[valid] - np.floor(dx[valid]))[:, None] * (\n np.ceil(dy[valid]) - dy[valid])[:, None]\n right_top_ = img[np.floor(dx[valid]).astype(int),\n np.ceil(dy[valid]).astype(int), :] * (\n np.ceil(dx[valid]) - dx[valid])[:, None] * (\n dy[valid] - np.floor(dy[valid]))[:, None]\n right_down_ = img[np.ceil(dx[valid]).astype(int),\n np.ceil(dy[valid]).astype(int), :] * (\n dx[valid] - np.floor(dx[valid]))[:, None] * (\n dy[valid] - np.floor(dy[valid]))[:, None]\n output[valid, :] = left_top_ + left_down_ + right_top_ + right_down_\n else:\n raise NotImplementedError(\n 'We only support interpolation modes of nearest and bilinear, '\n f'but got {interpolate_mode}.')\n return output.astype(img.dtype)\n\n\ndef flow_from_bytes(content):\n \"\"\"Read dense optical flow from bytes.\n\n .. note::\n This load optical flow function works for FlyingChairs, FlyingThings3D,\n Sintel, FlyingChairsOcc datasets, but cannot load the data from\n ChairsSDHom.\n\n Args:\n content (bytes): Optical flow bytes got from files or other streams.\n\n Returns:\n ndarray: Loaded optical flow with the shape (H, W, 2).\n \"\"\"\n\n # header in first 4 bytes\n header = content[:4]\n if header.decode('utf-8') != 'PIEH':\n raise Exception('Flow file header does not contain PIEH')\n # width in second 4 bytes\n width = np.frombuffer(content[4:], np.int32, 1).squeeze()\n # height in third 4 bytes\n height = np.frombuffer(content[8:], np.int32, 1).squeeze()\n # after first 12 bytes, all bytes are flow\n flow = np.frombuffer(content[12:], np.float32, width * height * 2).reshape(\n (height, width, 2))\n\n return flow\n\n\ndef sparse_flow_from_bytes(content):\n \"\"\"Read the optical flow in KITTI datasets from bytes.\n\n This function is modified from RAFT load the `KITTI datasets\n `_.\n\n Args:\n content (bytes): Optical flow bytes got from files or other streams.\n\n Returns:\n Tuple(ndarray, ndarray): Loaded optical flow with the shape (H, W, 2)\n and flow valid mask with the shape (H, W).\n \"\"\" # nopa\n\n content = np.frombuffer(content, np.uint8)\n flow = cv2.imdecode(content, cv2.IMREAD_ANYDEPTH | cv2.IMREAD_COLOR)\n flow = flow[:, :, ::-1].astype(np.float32)\n # flow shape (H, W, 2) valid shape (H, W)\n flow, valid = flow[:, :, :2], flow[:, :, 2]\n flow = (flow - 2**15) / 64.0\n return flow, valid\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/video/processing.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport os.path as osp\nimport subprocess\nimport tempfile\n\nfrom mmcv.utils import requires_executable\n\n\n@requires_executable('ffmpeg')\ndef convert_video(in_file,\n out_file,\n print_cmd=False,\n pre_options='',\n **kwargs):\n \"\"\"Convert a video with ffmpeg.\n\n This provides a general api to ffmpeg, the executed command is::\n\n `ffmpeg -y -i `\n\n Options(kwargs) are mapped to ffmpeg commands with the following rules:\n\n - key=val: \"-key val\"\n - key=True: \"-key\"\n - key=False: \"\"\n\n Args:\n in_file (str): Input video filename.\n out_file (str): Output video filename.\n pre_options (str): Options appears before \"-i \".\n print_cmd (bool): Whether to print the final ffmpeg command.\n \"\"\"\n options = []\n for k, v in kwargs.items():\n if isinstance(v, bool):\n if v:\n options.append(f'-{k}')\n elif k == 'log_level':\n assert v in [\n 'quiet', 'panic', 'fatal', 'error', 'warning', 'info',\n 'verbose', 'debug', 'trace'\n ]\n options.append(f'-loglevel {v}')\n else:\n options.append(f'-{k} {v}')\n cmd = f'ffmpeg -y {pre_options} -i {in_file} {\" \".join(options)} ' \\\n f'{out_file}'\n if print_cmd:\n print(cmd)\n subprocess.call(cmd, shell=True)\n\n\n@requires_executable('ffmpeg')\ndef resize_video(in_file,\n out_file,\n size=None,\n ratio=None,\n keep_ar=False,\n log_level='info',\n print_cmd=False):\n \"\"\"Resize a video.\n\n Args:\n in_file (str): Input video filename.\n out_file (str): Output video filename.\n size (tuple): Expected size (w, h), eg, (320, 240) or (320, -1).\n ratio (tuple or float): Expected resize ratio, (2, 0.5) means\n (w*2, h*0.5).\n keep_ar (bool): Whether to keep original aspect ratio.\n log_level (str): Logging level of ffmpeg.\n print_cmd (bool): Whether to print the final ffmpeg command.\n \"\"\"\n if size is None and ratio is None:\n raise ValueError('expected size or ratio must be specified')\n if size is not None and ratio is not None:\n raise ValueError('size and ratio cannot be specified at the same time')\n options = {'log_level': log_level}\n if size:\n if not keep_ar:\n options['vf'] = f'scale={size[0]}:{size[1]}'\n else:\n options['vf'] = f'scale=w={size[0]}:h={size[1]}:' \\\n 'force_original_aspect_ratio=decrease'\n else:\n if not isinstance(ratio, tuple):\n ratio = (ratio, ratio)\n options['vf'] = f'scale=\"trunc(iw*{ratio[0]}):trunc(ih*{ratio[1]})\"'\n convert_video(in_file, out_file, print_cmd, **options)\n\n\n@requires_executable('ffmpeg')\ndef cut_video(in_file,\n out_file,\n start=None,\n end=None,\n vcodec=None,\n acodec=None,\n log_level='info',\n print_cmd=False):\n \"\"\"Cut a clip from a video.\n\n Args:\n in_file (str): Input video filename.\n out_file (str): Output video filename.\n start (None or float): Start time (in seconds).\n end (None or float): End time (in seconds).\n vcodec (None or str): Output video codec, None for unchanged.\n acodec (None or str): Output audio codec, None for unchanged.\n log_level (str): Logging level of ffmpeg.\n print_cmd (bool): Whether to print the final ffmpeg command.\n \"\"\"\n options = {'log_level': log_level}\n if vcodec is None:\n options['vcodec'] = 'copy'\n if acodec is None:\n options['acodec'] = 'copy'\n if start:\n options['ss'] = start\n else:\n start = 0\n if end:\n options['t'] = end - start\n convert_video(in_file, out_file, print_cmd, **options)\n\n\n@requires_executable('ffmpeg')\ndef concat_video(video_list,\n out_file,\n vcodec=None,\n acodec=None,\n log_level='info',\n print_cmd=False):\n \"\"\"Concatenate multiple videos into a single one.\n\n Args:\n video_list (list): A list of video filenames\n out_file (str): Output video filename\n vcodec (None or str): Output video codec, None for unchanged\n acodec (None or str): Output audio codec, None for unchanged\n log_level (str): Logging level of ffmpeg.\n print_cmd (bool): Whether to print the final ffmpeg command.\n \"\"\"\n tmp_filehandler, tmp_filename = tempfile.mkstemp(suffix='.txt', text=True)\n with open(tmp_filename, 'w') as f:\n for filename in video_list:\n f.write(f'file {osp.abspath(filename)}\\n')\n options = {'log_level': log_level}\n if vcodec is None:\n options['vcodec'] = 'copy'\n if acodec is None:\n options['acodec'] = 'copy'\n convert_video(\n tmp_filename,\n out_file,\n print_cmd,\n pre_options='-f concat -safe 0',\n **options)\n os.close(tmp_filehandler)\n os.remove(tmp_filename)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/visualization/__init__.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom .color import Color, color_val\nfrom .image import imshow, imshow_bboxes, imshow_det_bboxes\nfrom .optflow import flow2rgb, flowshow, make_color_wheel\n\n__all__ = [\n 'Color', 'color_val', 'imshow', 'imshow_bboxes', 'imshow_det_bboxes',\n 'flowshow', 'flow2rgb', 'make_color_wheel'\n]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/visualization/color.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom enum import Enum\n\nimport numpy as np\n\nfrom mmcv.utils import is_str\n\n\nclass Color(Enum):\n \"\"\"An enum that defines common colors.\n\n Contains red, green, blue, cyan, yellow, magenta, white and black.\n \"\"\"\n red = (0, 0, 255)\n green = (0, 255, 0)\n blue = (255, 0, 0)\n cyan = (255, 255, 0)\n yellow = (0, 255, 255)\n magenta = (255, 0, 255)\n white = (255, 255, 255)\n black = (0, 0, 0)\n\n\ndef color_val(color):\n \"\"\"Convert various input to color tuples.\n\n Args:\n color (:obj:`Color`/str/tuple/int/ndarray): Color inputs\n\n Returns:\n tuple[int]: A tuple of 3 integers indicating BGR channels.\n \"\"\"\n if is_str(color):\n return Color[color].value\n elif isinstance(color, Color):\n return color.value\n elif isinstance(color, tuple):\n assert len(color) == 3\n for channel in color:\n assert 0 <= channel <= 255\n return color\n elif isinstance(color, int):\n assert 0 <= color <= 255\n return color, color, color\n elif isinstance(color, np.ndarray):\n assert color.ndim == 1 and color.size == 3\n assert np.all((color >= 0) & (color <= 255))\n color = color.astype(np.uint8)\n return tuple(color)\n else:\n raise TypeError(f'Invalid type for color: {type(color)}')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/visualization/image.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport cv2\nimport numpy as np\n\nfrom mmcv.image import imread, imwrite\nfrom .color import color_val\n\n\ndef imshow(img, win_name='', wait_time=0):\n \"\"\"Show an image.\n\n Args:\n img (str or ndarray): The image to be displayed.\n win_name (str): The window name.\n wait_time (int): Value of waitKey param.\n \"\"\"\n cv2.imshow(win_name, imread(img))\n if wait_time == 0: # prevent from hanging if windows was closed\n while True:\n ret = cv2.waitKey(1)\n\n closed = cv2.getWindowProperty(win_name, cv2.WND_PROP_VISIBLE) < 1\n # if user closed window or if some key pressed\n if closed or ret != -1:\n break\n else:\n ret = cv2.waitKey(wait_time)\n\n\ndef imshow_bboxes(img,\n bboxes,\n colors='green',\n top_k=-1,\n thickness=1,\n show=True,\n win_name='',\n wait_time=0,\n out_file=None):\n \"\"\"Draw bboxes on an image.\n\n Args:\n img (str or ndarray): The image to be displayed.\n bboxes (list or ndarray): A list of ndarray of shape (k, 4).\n colors (list[str or tuple or Color]): A list of colors.\n top_k (int): Plot the first k bboxes only if set positive.\n thickness (int): Thickness of lines.\n show (bool): Whether to show the image.\n win_name (str): The window name.\n wait_time (int): Value of waitKey param.\n out_file (str, optional): The filename to write the image.\n\n Returns:\n ndarray: The image with bboxes drawn on it.\n \"\"\"\n img = imread(img)\n img = np.ascontiguousarray(img)\n\n if isinstance(bboxes, np.ndarray):\n bboxes = [bboxes]\n if not isinstance(colors, list):\n colors = [colors for _ in range(len(bboxes))]\n colors = [color_val(c) for c in colors]\n assert len(bboxes) == len(colors)\n\n for i, _bboxes in enumerate(bboxes):\n _bboxes = _bboxes.astype(np.int32)\n if top_k <= 0:\n _top_k = _bboxes.shape[0]\n else:\n _top_k = min(top_k, _bboxes.shape[0])\n for j in range(_top_k):\n left_top = (_bboxes[j, 0], _bboxes[j, 1])\n right_bottom = (_bboxes[j, 2], _bboxes[j, 3])\n cv2.rectangle(\n img, left_top, right_bottom, colors[i], thickness=thickness)\n\n if show:\n imshow(img, win_name, wait_time)\n if out_file is not None:\n imwrite(img, out_file)\n return img\n\n\ndef imshow_det_bboxes(img,\n bboxes,\n labels,\n class_names=None,\n score_thr=0,\n bbox_color='green',\n text_color='green',\n thickness=1,\n font_scale=0.5,\n show=True,\n win_name='',\n wait_time=0,\n out_file=None):\n \"\"\"Draw bboxes and class labels (with scores) on an image.\n\n Args:\n img (str or ndarray): The image to be displayed.\n bboxes (ndarray): Bounding boxes (with scores), shaped (n, 4) or\n (n, 5).\n labels (ndarray): Labels of bboxes.\n class_names (list[str]): Names of each classes.\n score_thr (float): Minimum score of bboxes to be shown.\n bbox_color (str or tuple or :obj:`Color`): Color of bbox lines.\n text_color (str or tuple or :obj:`Color`): Color of texts.\n thickness (int): Thickness of lines.\n font_scale (float): Font scales of texts.\n show (bool): Whether to show the image.\n win_name (str): The window name.\n wait_time (int): Value of waitKey param.\n out_file (str or None): The filename to write the image.\n\n Returns:\n ndarray: The image with bboxes drawn on it.\n \"\"\"\n assert bboxes.ndim == 2\n assert labels.ndim == 1\n assert bboxes.shape[0] == labels.shape[0]\n assert bboxes.shape[1] == 4 or bboxes.shape[1] == 5\n img = imread(img)\n img = np.ascontiguousarray(img)\n\n if score_thr > 0:\n assert bboxes.shape[1] == 5\n scores = bboxes[:, -1]\n inds = scores > score_thr\n bboxes = bboxes[inds, :]\n labels = labels[inds]\n\n bbox_color = color_val(bbox_color)\n text_color = color_val(text_color)\n\n for bbox, label in zip(bboxes, labels):\n bbox_int = bbox.astype(np.int32)\n left_top = (bbox_int[0], bbox_int[1])\n right_bottom = (bbox_int[2], bbox_int[3])\n cv2.rectangle(\n img, left_top, right_bottom, bbox_color, thickness=thickness)\n label_text = class_names[\n label] if class_names is not None else f'cls {label}'\n if len(bbox) > 4:\n label_text += f'|{bbox[-1]:.02f}'\n cv2.putText(img, label_text, (bbox_int[0], bbox_int[1] - 2),\n cv2.FONT_HERSHEY_COMPLEX, font_scale, text_color)\n\n if show:\n imshow(img, win_name, wait_time)\n if out_file is not None:\n imwrite(img, out_file)\n return img\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/mmcv/visualization/optflow.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom __future__ import division\n\nimport numpy as np\n\nfrom mmcv.image import rgb2bgr\nfrom mmcv.video import flowread\nfrom .image import imshow\n\n\ndef flowshow(flow, win_name='', wait_time=0):\n \"\"\"Show optical flow.\n\n Args:\n flow (ndarray or str): The optical flow to be displayed.\n win_name (str): The window name.\n wait_time (int): Value of waitKey param.\n \"\"\"\n flow = flowread(flow)\n flow_img = flow2rgb(flow)\n imshow(rgb2bgr(flow_img), win_name, wait_time)\n\n\ndef flow2rgb(flow, color_wheel=None, unknown_thr=1e6):\n \"\"\"Convert flow map to RGB image.\n\n Args:\n flow (ndarray): Array of optical flow.\n color_wheel (ndarray or None): Color wheel used to map flow field to\n RGB colorspace. Default color wheel will be used if not specified.\n unknown_thr (str): Values above this threshold will be marked as\n unknown and thus ignored.\n\n Returns:\n ndarray: RGB image that can be visualized.\n \"\"\"\n assert flow.ndim == 3 and flow.shape[-1] == 2\n if color_wheel is None:\n color_wheel = make_color_wheel()\n assert color_wheel.ndim == 2 and color_wheel.shape[1] == 3\n num_bins = color_wheel.shape[0]\n\n dx = flow[:, :, 0].copy()\n dy = flow[:, :, 1].copy()\n\n ignore_inds = (\n np.isnan(dx) | np.isnan(dy) | (np.abs(dx) > unknown_thr) |\n (np.abs(dy) > unknown_thr))\n dx[ignore_inds] = 0\n dy[ignore_inds] = 0\n\n rad = np.sqrt(dx**2 + dy**2)\n if np.any(rad > np.finfo(float).eps):\n max_rad = np.max(rad)\n dx /= max_rad\n dy /= max_rad\n\n rad = np.sqrt(dx**2 + dy**2)\n angle = np.arctan2(-dy, -dx) / np.pi\n\n bin_real = (angle + 1) / 2 * (num_bins - 1)\n bin_left = np.floor(bin_real).astype(int)\n bin_right = (bin_left + 1) % num_bins\n w = (bin_real - bin_left.astype(np.float32))[..., None]\n flow_img = (1 -\n w) * color_wheel[bin_left, :] + w * color_wheel[bin_right, :]\n small_ind = rad <= 1\n flow_img[small_ind] = 1 - rad[small_ind, None] * (1 - flow_img[small_ind])\n flow_img[np.logical_not(small_ind)] *= 0.75\n\n flow_img[ignore_inds, :] = 0\n\n return flow_img\n\n\ndef make_color_wheel(bins=None):\n \"\"\"Build a color wheel.\n\n Args:\n bins(list or tuple, optional): Specify the number of bins for each\n color range, corresponding to six ranges: red -> yellow,\n yellow -> green, green -> cyan, cyan -> blue, blue -> magenta,\n magenta -> red. [15, 6, 4, 11, 13, 6] is used for default\n (see Middlebury).\n\n Returns:\n ndarray: Color wheel of shape (total_bins, 3).\n \"\"\"\n if bins is None:\n bins = [15, 6, 4, 11, 13, 6]\n assert len(bins) == 6\n\n RY, YG, GC, CB, BM, MR = tuple(bins)\n\n ry = [1, np.arange(RY) / RY, 0]\n yg = [1 - np.arange(YG) / YG, 1, 0]\n gc = [0, 1, np.arange(GC) / GC]\n cb = [0, 1 - np.arange(CB) / CB, 1]\n bm = [np.arange(BM) / BM, 0, 1]\n mr = [1, 0, 1 - np.arange(MR) / MR]\n\n num_bins = RY + YG + GC + CB + BM + MR\n\n color_wheel = np.zeros((3, num_bins), dtype=np.float32)\n\n col = 0\n for i, color in enumerate([ry, yg, gc, cb, bm, mr]):\n for j in range(3):\n color_wheel[j, col:col + bins[i]] = color[j]\n col += bins[i]\n\n return color_wheel.T\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/readme.md", "content": "test\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/setup.cfg", "content": "[bdist_wheel]\nuniversal=1\n\n[aliases]\ntest=pytest\n\n[yapf]\nbased_on_style = pep8\nblank_line_before_nested_class_or_def = true\nsplit_before_expression_after_opening_paren = true\n\n[isort]\nline_length = 79\nmulti_line_output = 0\nknown_standard_library = pkg_resources,setuptools,logging,os,warnings,abc\nknown_first_party = mmcv\nknown_third_party = addict,cv2,numpy,onnx,onnxruntime,packaging,pytest,pytorch_sphinx_theme,scipy,sphinx,tensorrt,torch,torchvision,yaml,yapf\nno_lines_before = STDLIB,LOCALFOLDER\ndefault_section = THIRDPARTY\n\n[codespell]\nignore-words-list = inout,hist\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/setup.py", "content": "import glob\nimport os\nimport platform\nimport re\nfrom pkg_resources import DistributionNotFound, get_distribution\nfrom setuptools import find_packages, setup\n\nEXT_TYPE = ''\ntry:\n import torch\n if torch.__version__ == 'parrots':\n from parrots.utils.build_extension import BuildExtension\n EXT_TYPE = 'parrots'\n else:\n from torch.utils.cpp_extension import BuildExtension\n EXT_TYPE = 'pytorch'\n cmd_class = {'build_ext': BuildExtension}\nexcept ModuleNotFoundError:\n cmd_class = {}\n print('Skip building ext ops due to the absence of torch.')\n\n\ndef choose_requirement(primary, secondary):\n \"\"\"If some version of primary requirement installed, return primary, else\n return secondary.\"\"\"\n try:\n name = re.split(r'[!<>=]', primary)[0]\n get_distribution(name)\n except DistributionNotFound:\n return secondary\n\n return str(primary)\n\n\ndef get_version():\n version_file = 'mmcv/version.py'\n with open(version_file, 'r', encoding='utf-8') as f:\n exec(compile(f.read(), version_file, 'exec'))\n return locals()['__version__']\n\n\ndef parse_requirements(fname='requirements/runtime.txt', with_version=True):\n \"\"\"Parse the package dependencies listed in a requirements file but strips\n specific versioning information.\n\n Args:\n fname (str): path to requirements file\n with_version (bool, default=False): if True include version specs\n\n Returns:\n List[str]: list of requirements items\n\n CommandLine:\n python -c \"import setup; print(setup.parse_requirements())\"\n \"\"\"\n import sys\n from os.path import exists\n require_fpath = fname\n\n def parse_line(line):\n \"\"\"Parse information from a line in a requirements text file.\"\"\"\n if line.startswith('-r '):\n # Allow specifying requirements in other files\n target = line.split(' ')[1]\n for info in parse_require_file(target):\n yield info\n else:\n info = {'line': line}\n if line.startswith('-e '):\n info['package'] = line.split('#egg=')[1]\n else:\n # Remove versioning from the package\n pat = '(' + '|'.join(['>=', '==', '>']) + ')'\n parts = re.split(pat, line, maxsplit=1)\n parts = [p.strip() for p in parts]\n\n info['package'] = parts[0]\n if len(parts) > 1:\n op, rest = parts[1:]\n if ';' in rest:\n # Handle platform specific dependencies\n # http://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-platform-specific-dependencies\n version, platform_deps = map(str.strip,\n rest.split(';'))\n info['platform_deps'] = platform_deps\n else:\n version = rest # NOQA\n info['version'] = (op, version)\n yield info\n\n def parse_require_file(fpath):\n with open(fpath, 'r') as f:\n for line in f.readlines():\n line = line.strip()\n if line and not line.startswith('#'):\n for info in parse_line(line):\n yield info\n\n def gen_packages_items():\n if exists(require_fpath):\n for info in parse_require_file(require_fpath):\n parts = [info['package']]\n if with_version and 'version' in info:\n parts.extend(info['version'])\n if not sys.version.startswith('3.4'):\n # apparently package_deps are broken in 3.4\n platform_deps = info.get('platform_deps')\n if platform_deps is not None:\n parts.append(';' + platform_deps)\n item = ''.join(parts)\n yield item\n\n packages = list(gen_packages_items())\n return packages\n\n\ninstall_requires = parse_requirements()\n\ntry:\n # OpenCV installed via conda.\n import cv2 # NOQA: F401\n major, minor, *rest = cv2.__version__.split('.')\n if int(major) < 3:\n raise RuntimeError(\n f'OpenCV >=3 is required but {cv2.__version__} is installed')\nexcept ImportError:\n # If first not installed install second package\n CHOOSE_INSTALL_REQUIRES = [('opencv-python-headless>=3',\n 'opencv-python>=3')]\n for main, secondary in CHOOSE_INSTALL_REQUIRES:\n install_requires.append(choose_requirement(main, secondary))\n\n\ndef get_extensions():\n extensions = []\n\n if os.getenv('MMCV_WITH_TRT', '0') != '0':\n ext_name = 'mmcv._ext_trt'\n from torch.utils.cpp_extension import include_paths, library_paths\n library_dirs = []\n libraries = []\n include_dirs = []\n tensorrt_path = os.getenv('TENSORRT_DIR', '0')\n tensorrt_lib_path = glob.glob(\n os.path.join(tensorrt_path, 'targets', '*', 'lib'))[0]\n library_dirs += [tensorrt_lib_path]\n libraries += ['nvinfer', 'nvparsers', 'nvinfer_plugin']\n libraries += ['cudart']\n define_macros = []\n extra_compile_args = {'cxx': []}\n\n include_path = os.path.abspath('./mmcv/ops/csrc/common/cuda')\n include_trt_path = os.path.abspath('./mmcv/ops/csrc/tensorrt')\n include_dirs.append(include_path)\n include_dirs.append(include_trt_path)\n include_dirs.append(os.path.join(tensorrt_path, 'include'))\n include_dirs += include_paths(cuda=True)\n\n op_files = glob.glob('./mmcv/ops/csrc/tensorrt/plugins/*')\n define_macros += [('MMCV_WITH_CUDA', None)]\n define_macros += [('MMCV_WITH_TRT', None)]\n cuda_args = os.getenv('MMCV_CUDA_ARGS')\n extra_compile_args['nvcc'] = [cuda_args] if cuda_args else []\n # prevent cub/thrust conflict with other python library\n # More context See issues #1454\n extra_compile_args['nvcc'] += ['-Xcompiler=-fno-gnu-unique']\n library_dirs += library_paths(cuda=True)\n\n from setuptools import Extension\n ext_ops = Extension(\n name=ext_name,\n sources=op_files,\n include_dirs=include_dirs,\n define_macros=define_macros,\n extra_compile_args=extra_compile_args,\n language='c++',\n library_dirs=library_dirs,\n libraries=libraries)\n extensions.append(ext_ops)\n\n if os.getenv('MMCV_WITH_OPS', '0') == '0':\n return extensions\n\n if EXT_TYPE == 'parrots':\n ext_name = 'mmcv._ext'\n from parrots.utils.build_extension import Extension\n # new parrots op impl do not use MMCV_USE_PARROTS\n # define_macros = [('MMCV_USE_PARROTS', None)]\n define_macros = []\n include_dirs = []\n op_files = glob.glob('./mmcv/ops/csrc/pytorch/cuda/*.cu') +\\\n glob.glob('./mmcv/ops/csrc/pytorch/cpu/*.cpp') +\\\n glob.glob('./mmcv/ops/csrc/parrots/*.cpp')\n include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common'))\n include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common/cuda'))\n cuda_args = os.getenv('MMCV_CUDA_ARGS')\n extra_compile_args = {\n 'nvcc': [cuda_args, '-std=c++14'] if cuda_args else ['-std=c++14'],\n 'cxx': ['-std=c++14'],\n }\n if torch.cuda.is_available() or os.getenv('FORCE_CUDA', '0') == '1':\n define_macros += [('MMCV_WITH_CUDA', None)]\n extra_compile_args['nvcc'] += [\n '-D__CUDA_NO_HALF_OPERATORS__',\n '-D__CUDA_NO_HALF_CONVERSIONS__',\n '-D__CUDA_NO_HALF2_OPERATORS__',\n ]\n ext_ops = Extension(\n name=ext_name,\n sources=op_files,\n include_dirs=include_dirs,\n define_macros=define_macros,\n extra_compile_args=extra_compile_args,\n cuda=True,\n pytorch=True)\n extensions.append(ext_ops)\n elif EXT_TYPE == 'pytorch':\n ext_name = 'mmcv._ext'\n from torch.utils.cpp_extension import CppExtension, CUDAExtension\n\n # prevent ninja from using too many resources\n try:\n import psutil\n num_cpu = len(psutil.Process().cpu_affinity())\n cpu_use = max(4, num_cpu - 1)\n except (ModuleNotFoundError, AttributeError):\n cpu_use = 4\n\n os.environ.setdefault('MAX_JOBS', str(cpu_use))\n define_macros = []\n\n # Before PyTorch1.8.0, when compiling CUDA code, `cxx` is a\n # required key passed to PyTorch. Even if there is no flag passed\n # to cxx, users also need to pass an empty list to PyTorch.\n # Since PyTorch1.8.0, it has a default value so users do not need\n # to pass an empty list anymore.\n # More details at https://github.com/pytorch/pytorch/pull/45956\n extra_compile_args = {'cxx': []}\n\n # Since the PR (https://github.com/open-mmlab/mmcv/pull/1463) uses\n # c++14 features, the argument ['std=c++14'] must be added here.\n # However, in the windows environment, some standard libraries\n # will depend on c++17 or higher. In fact, for the windows\n # environment, the compiler will choose the appropriate compiler\n # to compile those cpp files, so there is no need to add the\n # argument\n if platform.system() != 'Windows':\n extra_compile_args['cxx'] = ['-std=c++14']\n\n include_dirs = []\n\n is_rocm_pytorch = False\n try:\n from torch.utils.cpp_extension import ROCM_HOME\n is_rocm_pytorch = True if ((torch.version.hip is not None) and\n (ROCM_HOME is not None)) else False\n except ImportError:\n pass\n\n project_dir = 'mmcv/ops/csrc/'\n if is_rocm_pytorch:\n from torch.utils.hipify import hipify_python\n\n hipify_python.hipify(\n project_directory=project_dir,\n output_directory=project_dir,\n includes='mmcv/ops/csrc/*',\n show_detailed=True,\n is_pytorch_extension=True,\n )\n define_macros += [('MMCV_WITH_CUDA', None)]\n define_macros += [('HIP_DIFF', None)]\n cuda_args = os.getenv('MMCV_CUDA_ARGS')\n extra_compile_args['nvcc'] = [cuda_args] if cuda_args else []\n op_files = glob.glob('./mmcv/ops/csrc/pytorch/hip/*') \\\n + glob.glob('./mmcv/ops/csrc/pytorch/cpu/hip/*')\n extension = CUDAExtension\n include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common/hip'))\n elif torch.cuda.is_available() or os.getenv('FORCE_CUDA', '0') == '1':\n define_macros += [('MMCV_WITH_CUDA', None)]\n cuda_args = os.getenv('MMCV_CUDA_ARGS')\n extra_compile_args['nvcc'] = [cuda_args] if cuda_args else []\n op_files = glob.glob('./mmcv/ops/csrc/pytorch/*.cpp') + \\\n glob.glob('./mmcv/ops/csrc/pytorch/cpu/*.cpp') + \\\n glob.glob('./mmcv/ops/csrc/pytorch/cuda/*.cu') + \\\n glob.glob('./mmcv/ops/csrc/pytorch/cuda/*.cpp')\n extension = CUDAExtension\n include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common'))\n include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common/cuda'))\n else:\n print(f'Compiling {ext_name} without CUDA')\n op_files = glob.glob('./mmcv/ops/csrc/pytorch/*.cpp') + \\\n glob.glob('./mmcv/ops/csrc/pytorch/cpu/*.cpp')\n extension = CppExtension\n include_dirs.append(os.path.abspath('./mmcv/ops/csrc/common'))\n\n # Since the PR (https://github.com/open-mmlab/mmcv/pull/1463) uses\n # c++14 features, the argument ['std=c++14'] must be added here.\n # However, in the windows environment, some standard libraries\n # will depend on c++17 or higher. In fact, for the windows\n # environment, the compiler will choose the appropriate compiler\n # to compile those cpp files, so there is no need to add the\n # argument\n if 'nvcc' in extra_compile_args and platform.system() != 'Windows':\n extra_compile_args['nvcc'] += ['-std=c++14']\n\n ext_ops = extension(\n name=ext_name,\n sources=op_files,\n include_dirs=include_dirs,\n define_macros=define_macros,\n extra_compile_args=extra_compile_args)\n extensions.append(ext_ops)\n\n if EXT_TYPE == 'pytorch' and os.getenv('MMCV_WITH_ORT', '0') != '0':\n ext_name = 'mmcv._ext_ort'\n from torch.utils.cpp_extension import library_paths, include_paths\n import onnxruntime\n library_dirs = []\n libraries = []\n include_dirs = []\n ort_path = os.getenv('ONNXRUNTIME_DIR', '0')\n library_dirs += [os.path.join(ort_path, 'lib')]\n libraries.append('onnxruntime')\n define_macros = []\n extra_compile_args = {'cxx': []}\n\n include_path = os.path.abspath('./mmcv/ops/csrc/onnxruntime')\n include_dirs.append(include_path)\n include_dirs.append(os.path.join(ort_path, 'include'))\n\n op_files = glob.glob('./mmcv/ops/csrc/onnxruntime/cpu/*')\n if onnxruntime.get_device() == 'GPU' or os.getenv('FORCE_CUDA',\n '0') == '1':\n define_macros += [('MMCV_WITH_CUDA', None)]\n cuda_args = os.getenv('MMCV_CUDA_ARGS')\n extra_compile_args['nvcc'] = [cuda_args] if cuda_args else []\n op_files += glob.glob('./mmcv/ops/csrc/onnxruntime/gpu/*')\n include_dirs += include_paths(cuda=True)\n library_dirs += library_paths(cuda=True)\n else:\n include_dirs += include_paths(cuda=False)\n library_dirs += library_paths(cuda=False)\n\n from setuptools import Extension\n ext_ops = Extension(\n name=ext_name,\n sources=op_files,\n include_dirs=include_dirs,\n define_macros=define_macros,\n extra_compile_args=extra_compile_args,\n language='c++',\n library_dirs=library_dirs,\n libraries=libraries)\n extensions.append(ext_ops)\n\n return extensions\n\n\nsetup(\n name='mmcv' if os.getenv('MMCV_WITH_OPS', '0') == '0' else 'mmcv-full',\n version=get_version(),\n description='OpenMMLab Computer Vision Foundation',\n keywords='computer vision',\n packages=find_packages(),\n include_package_data=True,\n classifiers=[\n 'Development Status :: 4 - Beta',\n 'License :: OSI Approved :: Apache Software License',\n 'Operating System :: OS Independent',\n 'Programming Language :: Python :: 3',\n 'Programming Language :: Python :: 3.6',\n 'Programming Language :: Python :: 3.7',\n 'Programming Language :: Python :: 3.8',\n 'Programming Language :: Python :: 3.9',\n 'Topic :: Utilities',\n ],\n url='https://github.com/open-mmlab/mmcv',\n author='MMCV Contributors',\n author_email='openmmlab@gmail.com',\n install_requires=install_requires,\n extras_require={\n 'all': parse_requirements('requirements.txt'),\n 'tests': parse_requirements('requirements/test.txt'),\n 'build': parse_requirements('requirements/build.txt'),\n 'optional': parse_requirements('requirements/optional.txt'),\n },\n ext_modules=get_extensions(),\n cmdclass=cmd_class,\n zip_safe=False)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_arraymisc.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nfrom __future__ import division\n\nimport numpy as np\nimport pytest\n\nimport mmcv\n\n\ndef test_quantize():\n arr = np.random.randn(10, 10)\n levels = 20\n\n qarr = mmcv.quantize(arr, -1, 1, levels)\n assert qarr.shape == arr.shape\n assert qarr.dtype == np.dtype('int64')\n for i in range(arr.shape[0]):\n for j in range(arr.shape[1]):\n ref = min(levels - 1,\n int(np.floor(10 * (1 + max(min(arr[i, j], 1), -1)))))\n assert qarr[i, j] == ref\n\n qarr = mmcv.quantize(arr, -1, 1, 20, dtype=np.uint8)\n assert qarr.shape == arr.shape\n assert qarr.dtype == np.dtype('uint8')\n\n with pytest.raises(ValueError):\n mmcv.quantize(arr, -1, 1, levels=0)\n with pytest.raises(ValueError):\n mmcv.quantize(arr, -1, 1, levels=10.0)\n with pytest.raises(ValueError):\n mmcv.quantize(arr, 2, 1, levels)\n\n\ndef test_dequantize():\n levels = 20\n qarr = np.random.randint(levels, size=(10, 10))\n\n arr = mmcv.dequantize(qarr, -1, 1, levels)\n assert arr.shape == qarr.shape\n assert arr.dtype == np.dtype('float64')\n for i in range(qarr.shape[0]):\n for j in range(qarr.shape[1]):\n assert arr[i, j] == (qarr[i, j] + 0.5) / 10 - 1\n\n arr = mmcv.dequantize(qarr, -1, 1, levels, dtype=np.float32)\n assert arr.shape == qarr.shape\n assert arr.dtype == np.dtype('float32')\n\n with pytest.raises(ValueError):\n mmcv.dequantize(arr, -1, 1, levels=0)\n with pytest.raises(ValueError):\n mmcv.dequantize(arr, -1, 1, levels=10.0)\n with pytest.raises(ValueError):\n mmcv.dequantize(arr, 2, 1, levels)\n\n\ndef test_joint():\n arr = np.random.randn(100, 100)\n levels = 1000\n qarr = mmcv.quantize(arr, -1, 1, levels)\n recover = mmcv.dequantize(qarr, -1, 1, levels)\n assert np.abs(recover[arr < -1] + 0.999).max() < 1e-6\n assert np.abs(recover[arr > 1] - 0.999).max() < 1e-6\n assert np.abs((recover - arr)[(arr >= -1) & (arr <= 1)]).max() <= 1e-3\n\n arr = np.clip(np.random.randn(100) / 1000, -0.01, 0.01)\n levels = 99\n qarr = mmcv.quantize(arr, -1, 1, levels)\n recover = mmcv.dequantize(qarr, -1, 1, levels)\n assert np.all(recover == 0)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_build_layers.py", "content": "import numpy as np\nimport pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.cnn.bricks import (ACTIVATION_LAYERS, CONV_LAYERS, NORM_LAYERS,\n PADDING_LAYERS, PLUGIN_LAYERS,\n build_activation_layer, build_conv_layer,\n build_norm_layer, build_padding_layer,\n build_plugin_layer, build_upsample_layer, is_norm)\nfrom mmcv.cnn.bricks.norm import infer_abbr as infer_norm_abbr\nfrom mmcv.cnn.bricks.plugin import infer_abbr as infer_plugin_abbr\nfrom mmcv.cnn.bricks.upsample import PixelShufflePack\nfrom mmcv.utils.parrots_wrapper import _BatchNorm\n\n\ndef test_build_conv_layer():\n with pytest.raises(TypeError):\n # cfg must be a dict\n cfg = 'Conv2d'\n build_conv_layer(cfg)\n\n with pytest.raises(KeyError):\n # `type` must be in cfg\n cfg = dict(kernel_size=3)\n build_conv_layer(cfg)\n\n with pytest.raises(KeyError):\n # unsupported conv type\n cfg = dict(type='FancyConv')\n build_conv_layer(cfg)\n\n kwargs = dict(\n in_channels=4, out_channels=8, kernel_size=3, groups=2, dilation=2)\n cfg = None\n layer = build_conv_layer(cfg, **kwargs)\n assert isinstance(layer, nn.Conv2d)\n assert layer.in_channels == kwargs['in_channels']\n assert layer.out_channels == kwargs['out_channels']\n assert layer.kernel_size == (kwargs['kernel_size'], kwargs['kernel_size'])\n assert layer.groups == kwargs['groups']\n assert layer.dilation == (kwargs['dilation'], kwargs['dilation'])\n\n cfg = dict(type='Conv')\n layer = build_conv_layer(cfg, **kwargs)\n assert isinstance(layer, nn.Conv2d)\n assert layer.in_channels == kwargs['in_channels']\n assert layer.out_channels == kwargs['out_channels']\n assert layer.kernel_size == (kwargs['kernel_size'], kwargs['kernel_size'])\n assert layer.groups == kwargs['groups']\n assert layer.dilation == (kwargs['dilation'], kwargs['dilation'])\n\n cfg = dict(type='deconv')\n layer = build_conv_layer(cfg, **kwargs)\n assert isinstance(layer, nn.ConvTranspose2d)\n assert layer.in_channels == kwargs['in_channels']\n assert layer.out_channels == kwargs['out_channels']\n assert layer.kernel_size == (kwargs['kernel_size'], kwargs['kernel_size'])\n assert layer.groups == kwargs['groups']\n assert layer.dilation == (kwargs['dilation'], kwargs['dilation'])\n\n for type_name, module in CONV_LAYERS.module_dict.items():\n cfg = dict(type=type_name)\n layer = build_conv_layer(cfg, **kwargs)\n assert isinstance(layer, module)\n assert layer.in_channels == kwargs['in_channels']\n assert layer.out_channels == kwargs['out_channels']\n\n\ndef test_infer_norm_abbr():\n with pytest.raises(TypeError):\n # class_type must be a class\n infer_norm_abbr(0)\n\n class MyNorm:\n\n _abbr_ = 'mn'\n\n assert infer_norm_abbr(MyNorm) == 'mn'\n\n class FancyBatchNorm:\n pass\n\n assert infer_norm_abbr(FancyBatchNorm) == 'bn'\n\n class FancyInstanceNorm:\n pass\n\n assert infer_norm_abbr(FancyInstanceNorm) == 'in'\n\n class FancyLayerNorm:\n pass\n\n assert infer_norm_abbr(FancyLayerNorm) == 'ln'\n\n class FancyGroupNorm:\n pass\n\n assert infer_norm_abbr(FancyGroupNorm) == 'gn'\n\n class FancyNorm:\n pass\n\n assert infer_norm_abbr(FancyNorm) == 'norm_layer'\n\n\ndef test_build_norm_layer():\n with pytest.raises(TypeError):\n # cfg must be a dict\n cfg = 'BN'\n build_norm_layer(cfg, 3)\n\n with pytest.raises(KeyError):\n # `type` must be in cfg\n cfg = dict()\n build_norm_layer(cfg, 3)\n\n with pytest.raises(KeyError):\n # unsupported norm type\n cfg = dict(type='FancyNorm')\n build_norm_layer(cfg, 3)\n\n with pytest.raises(AssertionError):\n # postfix must be int or str\n cfg = dict(type='BN')\n build_norm_layer(cfg, 3, postfix=[1, 2])\n\n with pytest.raises(AssertionError):\n # `num_groups` must be in cfg when using 'GN'\n cfg = dict(type='GN')\n build_norm_layer(cfg, 3)\n\n # test each type of norm layer in norm_cfg\n abbr_mapping = {\n 'BN': 'bn',\n 'BN1d': 'bn',\n 'BN2d': 'bn',\n 'BN3d': 'bn',\n 'SyncBN': 'bn',\n 'GN': 'gn',\n 'LN': 'ln',\n 'IN': 'in',\n 'IN1d': 'in',\n 'IN2d': 'in',\n 'IN3d': 'in',\n }\n for type_name, module in NORM_LAYERS.module_dict.items():\n if type_name == 'MMSyncBN': # skip MMSyncBN\n continue\n for postfix in ['_test', 1]:\n cfg = dict(type=type_name)\n if type_name == 'GN':\n cfg['num_groups'] = 2\n name, layer = build_norm_layer(cfg, 3, postfix=postfix)\n assert name == abbr_mapping[type_name] + str(postfix)\n assert isinstance(layer, module)\n if type_name == 'GN':\n assert layer.num_channels == 3\n assert layer.num_groups == cfg['num_groups']\n elif type_name != 'LN':\n assert layer.num_features == 3\n\n\ndef test_build_activation_layer():\n with pytest.raises(TypeError):\n # cfg must be a dict\n cfg = 'ReLU'\n build_activation_layer(cfg)\n\n with pytest.raises(KeyError):\n # `type` must be in cfg\n cfg = dict()\n build_activation_layer(cfg)\n\n with pytest.raises(KeyError):\n # unsupported activation type\n cfg = dict(type='FancyReLU')\n build_activation_layer(cfg)\n\n # test each type of activation layer in activation_cfg\n for type_name, module in ACTIVATION_LAYERS.module_dict.items():\n cfg['type'] = type_name\n layer = build_activation_layer(cfg)\n assert isinstance(layer, module)\n\n # sanity check for Clamp\n act = build_activation_layer(dict(type='Clamp'))\n x = torch.randn(10) * 1000\n y = act(x)\n assert np.logical_and((y >= -1).numpy(), (y <= 1).numpy()).all()\n act = build_activation_layer(dict(type='Clip', min=0))\n y = act(x)\n assert np.logical_and((y >= 0).numpy(), (y <= 1).numpy()).all()\n act = build_activation_layer(dict(type='Clamp', max=0))\n y = act(x)\n assert np.logical_and((y >= -1).numpy(), (y <= 0).numpy()).all()\n\n\ndef test_build_padding_layer():\n with pytest.raises(TypeError):\n # cfg must be a dict\n cfg = 'reflect'\n build_padding_layer(cfg)\n\n with pytest.raises(KeyError):\n # `type` must be in cfg\n cfg = dict()\n build_padding_layer(cfg)\n\n with pytest.raises(KeyError):\n # unsupported activation type\n cfg = dict(type='FancyPad')\n build_padding_layer(cfg)\n\n for type_name, module in PADDING_LAYERS.module_dict.items():\n cfg['type'] = type_name\n layer = build_padding_layer(cfg, 2)\n assert isinstance(layer, module)\n\n input_x = torch.randn(1, 2, 5, 5)\n cfg = dict(type='reflect')\n padding_layer = build_padding_layer(cfg, 2)\n res = padding_layer(input_x)\n assert res.shape == (1, 2, 9, 9)\n\n\ndef test_upsample_layer():\n with pytest.raises(TypeError):\n # cfg must be a dict\n cfg = 'bilinear'\n build_upsample_layer(cfg)\n\n with pytest.raises(KeyError):\n # `type` must be in cfg\n cfg = dict()\n build_upsample_layer(cfg)\n\n with pytest.raises(KeyError):\n # unsupported activation type\n cfg = dict(type='FancyUpsample')\n build_upsample_layer(cfg)\n\n for type_name in ['nearest', 'bilinear']:\n cfg['type'] = type_name\n layer = build_upsample_layer(cfg)\n assert isinstance(layer, nn.Upsample)\n assert layer.mode == type_name\n\n cfg = dict(\n type='deconv', in_channels=3, out_channels=3, kernel_size=3, stride=2)\n layer = build_upsample_layer(cfg)\n assert isinstance(layer, nn.ConvTranspose2d)\n\n cfg = dict(type='deconv')\n kwargs = dict(in_channels=3, out_channels=3, kernel_size=3, stride=2)\n layer = build_upsample_layer(cfg, **kwargs)\n assert isinstance(layer, nn.ConvTranspose2d)\n assert layer.in_channels == kwargs['in_channels']\n assert layer.out_channels == kwargs['out_channels']\n assert layer.kernel_size == (kwargs['kernel_size'], kwargs['kernel_size'])\n assert layer.stride == (kwargs['stride'], kwargs['stride'])\n\n layer = build_upsample_layer(cfg, 3, 3, 3, 2)\n assert isinstance(layer, nn.ConvTranspose2d)\n assert layer.in_channels == kwargs['in_channels']\n assert layer.out_channels == kwargs['out_channels']\n assert layer.kernel_size == (kwargs['kernel_size'], kwargs['kernel_size'])\n assert layer.stride == (kwargs['stride'], kwargs['stride'])\n\n cfg = dict(\n type='pixel_shuffle',\n in_channels=3,\n out_channels=3,\n scale_factor=2,\n upsample_kernel=3)\n layer = build_upsample_layer(cfg)\n\n assert isinstance(layer, PixelShufflePack)\n assert layer.scale_factor == 2\n assert layer.upsample_kernel == 3\n\n\ndef test_pixel_shuffle_pack():\n x_in = torch.rand(2, 3, 10, 10)\n pixel_shuffle = PixelShufflePack(3, 3, scale_factor=2, upsample_kernel=3)\n assert pixel_shuffle.upsample_conv.kernel_size == (3, 3)\n x_out = pixel_shuffle(x_in)\n assert x_out.shape == (2, 3, 20, 20)\n\n\ndef test_is_norm():\n norm_set1 = [\n nn.BatchNorm1d, nn.BatchNorm2d, nn.BatchNorm3d, nn.InstanceNorm1d,\n nn.InstanceNorm2d, nn.InstanceNorm3d, nn.LayerNorm\n ]\n norm_set2 = [nn.GroupNorm]\n for norm_type in norm_set1:\n layer = norm_type(3)\n assert is_norm(layer)\n assert not is_norm(layer, exclude=(norm_type, ))\n for norm_type in norm_set2:\n layer = norm_type(3, 6)\n assert is_norm(layer)\n assert not is_norm(layer, exclude=(norm_type, ))\n\n class MyNorm(nn.BatchNorm2d):\n pass\n\n layer = MyNorm(3)\n assert is_norm(layer)\n assert not is_norm(layer, exclude=_BatchNorm)\n assert not is_norm(layer, exclude=(_BatchNorm, ))\n\n layer = nn.Conv2d(3, 8, 1)\n assert not is_norm(layer)\n\n with pytest.raises(TypeError):\n layer = nn.BatchNorm1d(3)\n is_norm(layer, exclude='BN')\n\n with pytest.raises(TypeError):\n layer = nn.BatchNorm1d(3)\n is_norm(layer, exclude=('BN', ))\n\n\ndef test_infer_plugin_abbr():\n with pytest.raises(TypeError):\n # class_type must be a class\n infer_plugin_abbr(0)\n\n class MyPlugin:\n\n _abbr_ = 'mp'\n\n assert infer_plugin_abbr(MyPlugin) == 'mp'\n\n class FancyPlugin:\n pass\n\n assert infer_plugin_abbr(FancyPlugin) == 'fancy_plugin'\n\n\ndef test_build_plugin_layer():\n with pytest.raises(TypeError):\n # cfg must be a dict\n cfg = 'Plugin'\n build_plugin_layer(cfg)\n\n with pytest.raises(KeyError):\n # `type` must be in cfg\n cfg = dict()\n build_plugin_layer(cfg)\n\n with pytest.raises(KeyError):\n # unsupported plugin type\n cfg = dict(type='FancyPlugin')\n build_plugin_layer(cfg)\n\n with pytest.raises(AssertionError):\n # postfix must be int or str\n cfg = dict(type='ConvModule')\n build_plugin_layer(cfg, postfix=[1, 2])\n\n # test ContextBlock\n for postfix in ['', '_test', 1]:\n cfg = dict(type='ContextBlock')\n name, layer = build_plugin_layer(\n cfg, postfix=postfix, in_channels=16, ratio=1. / 4)\n assert name == 'context_block' + str(postfix)\n assert isinstance(layer, PLUGIN_LAYERS.module_dict['ContextBlock'])\n\n # test GeneralizedAttention\n for postfix in ['', '_test', 1]:\n cfg = dict(type='GeneralizedAttention')\n name, layer = build_plugin_layer(cfg, postfix=postfix, in_channels=16)\n assert name == 'gen_attention_block' + str(postfix)\n assert isinstance(layer,\n PLUGIN_LAYERS.module_dict['GeneralizedAttention'])\n\n # test NonLocal2d\n for postfix in ['', '_test', 1]:\n cfg = dict(type='NonLocal2d')\n name, layer = build_plugin_layer(cfg, postfix=postfix, in_channels=16)\n assert name == 'nonlocal_block' + str(postfix)\n assert isinstance(layer, PLUGIN_LAYERS.module_dict['NonLocal2d'])\n\n # test ConvModule\n for postfix in ['', '_test', 1]:\n cfg = dict(type='ConvModule')\n name, layer = build_plugin_layer(\n cfg,\n postfix=postfix,\n in_channels=16,\n out_channels=4,\n kernel_size=3)\n assert name == 'conv_block' + str(postfix)\n assert isinstance(layer, PLUGIN_LAYERS.module_dict['ConvModule'])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_context_block.py", "content": "import pytest\nimport torch\n\nfrom mmcv.cnn.bricks import ContextBlock\n\n\ndef test_context_block():\n with pytest.raises(AssertionError):\n # pooling_type should be in ['att', 'avg']\n ContextBlock(16, 1. / 4, pooling_type='unsupport_type')\n\n with pytest.raises(AssertionError):\n # fusion_types should be of type list or tuple\n ContextBlock(16, 1. / 4, fusion_types='unsupport_type')\n\n with pytest.raises(AssertionError):\n # fusion_types should be in ['channel_add', 'channel_mul']\n ContextBlock(16, 1. / 4, fusion_types=('unsupport_type', ))\n\n # test pooling_type='att'\n imgs = torch.randn(2, 16, 20, 20)\n context_block = ContextBlock(16, 1. / 4, pooling_type='att')\n out = context_block(imgs)\n assert context_block.conv_mask.in_channels == 16\n assert context_block.conv_mask.out_channels == 1\n assert out.shape == imgs.shape\n\n # test pooling_type='avg'\n imgs = torch.randn(2, 16, 20, 20)\n context_block = ContextBlock(16, 1. / 4, pooling_type='avg')\n out = context_block(imgs)\n assert hasattr(context_block, 'avg_pool')\n assert out.shape == imgs.shape\n\n # test fusion_types=('channel_add',)\n imgs = torch.randn(2, 16, 20, 20)\n context_block = ContextBlock(16, 1. / 4, fusion_types=('channel_add', ))\n out = context_block(imgs)\n assert context_block.channel_add_conv is not None\n assert context_block.channel_mul_conv is None\n assert out.shape == imgs.shape\n\n # test fusion_types=('channel_mul',)\n imgs = torch.randn(2, 16, 20, 20)\n context_block = ContextBlock(16, 1. / 4, fusion_types=('channel_mul', ))\n out = context_block(imgs)\n assert context_block.channel_add_conv is None\n assert context_block.channel_mul_conv is not None\n assert out.shape == imgs.shape\n\n # test fusion_types=('channel_add', 'channel_mul')\n imgs = torch.randn(2, 16, 20, 20)\n context_block = ContextBlock(\n 16, 1. / 4, fusion_types=('channel_add', 'channel_mul'))\n out = context_block(imgs)\n assert context_block.channel_add_conv is not None\n assert context_block.channel_mul_conv is not None\n assert out.shape == imgs.shape\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_conv2d_adaptive_padding.py", "content": "import torch\n\nfrom mmcv.cnn.bricks import Conv2dAdaptivePadding\n\n\ndef test_conv2d_samepadding():\n # test Conv2dAdaptivePadding with stride=1\n inputs = torch.rand((1, 3, 28, 28))\n conv = Conv2dAdaptivePadding(3, 3, kernel_size=3, stride=1)\n output = conv(inputs)\n assert output.shape == inputs.shape\n\n inputs = torch.rand((1, 3, 13, 13))\n conv = Conv2dAdaptivePadding(3, 3, kernel_size=3, stride=1)\n output = conv(inputs)\n assert output.shape == inputs.shape\n\n # test Conv2dAdaptivePadding with stride=2\n inputs = torch.rand((1, 3, 28, 28))\n conv = Conv2dAdaptivePadding(3, 3, kernel_size=3, stride=2)\n output = conv(inputs)\n assert output.shape == torch.Size([1, 3, 14, 14])\n\n inputs = torch.rand((1, 3, 13, 13))\n conv = Conv2dAdaptivePadding(3, 3, kernel_size=3, stride=2)\n output = conv(inputs)\n assert output.shape == torch.Size([1, 3, 7, 7])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_conv_module.py", "content": "import warnings\nfrom unittest.mock import patch\n\nimport pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.cnn.bricks import CONV_LAYERS, ConvModule, HSigmoid, HSwish\n\n\n@CONV_LAYERS.register_module()\nclass ExampleConv(nn.Module):\n\n def __init__(self,\n in_channels,\n out_channels,\n kernel_size,\n stride=1,\n padding=0,\n dilation=1,\n groups=1,\n bias=True,\n norm_cfg=None):\n super(ExampleConv, self).__init__()\n self.in_channels = in_channels\n self.out_channels = out_channels\n self.kernel_size = kernel_size\n self.stride = stride\n self.padding = padding\n self.dilation = dilation\n self.groups = groups\n self.bias = bias\n self.norm_cfg = norm_cfg\n self.output_padding = (0, 0, 0)\n self.transposed = False\n\n self.conv0 = nn.Conv2d(in_channels, out_channels, kernel_size)\n self.init_weights()\n\n def forward(self, x):\n x = self.conv0(x)\n return x\n\n def init_weights(self):\n nn.init.constant_(self.conv0.weight, 0)\n\n\ndef test_conv_module():\n with pytest.raises(AssertionError):\n # conv_cfg must be a dict or None\n conv_cfg = 'conv'\n ConvModule(3, 8, 2, conv_cfg=conv_cfg)\n\n with pytest.raises(AssertionError):\n # norm_cfg must be a dict or None\n norm_cfg = 'norm'\n ConvModule(3, 8, 2, norm_cfg=norm_cfg)\n\n with pytest.raises(KeyError):\n # softmax is not supported\n act_cfg = dict(type='softmax')\n ConvModule(3, 8, 2, act_cfg=act_cfg)\n\n # conv + norm + act\n conv = ConvModule(3, 8, 2, norm_cfg=dict(type='BN'))\n assert conv.with_activation\n assert hasattr(conv, 'activate')\n assert conv.with_norm\n assert hasattr(conv, 'norm')\n x = torch.rand(1, 3, 256, 256)\n output = conv(x)\n assert output.shape == (1, 8, 255, 255)\n\n # conv + act\n conv = ConvModule(3, 8, 2)\n assert conv.with_activation\n assert hasattr(conv, 'activate')\n assert not conv.with_norm\n assert conv.norm is None\n x = torch.rand(1, 3, 256, 256)\n output = conv(x)\n assert output.shape == (1, 8, 255, 255)\n\n # conv\n conv = ConvModule(3, 8, 2, act_cfg=None)\n assert not conv.with_norm\n assert conv.norm is None\n assert not conv.with_activation\n assert not hasattr(conv, 'activate')\n x = torch.rand(1, 3, 256, 256)\n output = conv(x)\n assert output.shape == (1, 8, 255, 255)\n\n # conv with its own `init_weights` method\n conv_module = ConvModule(\n 3, 8, 2, conv_cfg=dict(type='ExampleConv'), act_cfg=None)\n assert torch.equal(conv_module.conv.conv0.weight, torch.zeros(8, 3, 2, 2))\n\n # with_spectral_norm=True\n conv = ConvModule(3, 8, 3, padding=1, with_spectral_norm=True)\n assert hasattr(conv.conv, 'weight_orig')\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # padding_mode='reflect'\n conv = ConvModule(3, 8, 3, padding=1, padding_mode='reflect')\n assert isinstance(conv.padding_layer, nn.ReflectionPad2d)\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # non-existing padding mode\n with pytest.raises(KeyError):\n conv = ConvModule(3, 8, 3, padding=1, padding_mode='non_exists')\n\n # leaky relu\n conv = ConvModule(3, 8, 3, padding=1, act_cfg=dict(type='LeakyReLU'))\n assert isinstance(conv.activate, nn.LeakyReLU)\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # tanh\n conv = ConvModule(3, 8, 3, padding=1, act_cfg=dict(type='Tanh'))\n assert isinstance(conv.activate, nn.Tanh)\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # Sigmoid\n conv = ConvModule(3, 8, 3, padding=1, act_cfg=dict(type='Sigmoid'))\n assert isinstance(conv.activate, nn.Sigmoid)\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # PReLU\n conv = ConvModule(3, 8, 3, padding=1, act_cfg=dict(type='PReLU'))\n assert isinstance(conv.activate, nn.PReLU)\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # HSwish\n conv = ConvModule(3, 8, 3, padding=1, act_cfg=dict(type='HSwish'))\n assert isinstance(conv.activate, HSwish)\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # HSigmoid\n conv = ConvModule(3, 8, 3, padding=1, act_cfg=dict(type='HSigmoid'))\n assert isinstance(conv.activate, HSigmoid)\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n\ndef test_bias():\n # bias: auto, without norm\n conv = ConvModule(3, 8, 2)\n assert conv.conv.bias is not None\n\n # bias: auto, with norm\n conv = ConvModule(3, 8, 2, norm_cfg=dict(type='BN'))\n assert conv.conv.bias is None\n\n # bias: False, without norm\n conv = ConvModule(3, 8, 2, bias=False)\n assert conv.conv.bias is None\n\n # bias: True, with batch norm\n with pytest.warns(UserWarning) as record:\n ConvModule(3, 8, 2, bias=True, norm_cfg=dict(type='BN'))\n assert len(record) == 1\n assert record[0].message.args[\n 0] == 'Unnecessary conv bias before batch/instance norm'\n\n # bias: True, with instance norm\n with pytest.warns(UserWarning) as record:\n ConvModule(3, 8, 2, bias=True, norm_cfg=dict(type='IN'))\n assert len(record) == 1\n assert record[0].message.args[\n 0] == 'Unnecessary conv bias before batch/instance norm'\n\n # bias: True, with other norm\n with pytest.warns(UserWarning) as record:\n norm_cfg = dict(type='GN', num_groups=1)\n ConvModule(3, 8, 2, bias=True, norm_cfg=norm_cfg)\n warnings.warn('No warnings')\n assert len(record) == 1\n assert record[0].message.args[0] == 'No warnings'\n\n\ndef conv_forward(self, x):\n return x + '_conv'\n\n\ndef bn_forward(self, x):\n return x + '_bn'\n\n\ndef relu_forward(self, x):\n return x + '_relu'\n\n\n@patch('torch.nn.ReLU.forward', relu_forward)\n@patch('torch.nn.BatchNorm2d.forward', bn_forward)\n@patch('torch.nn.Conv2d.forward', conv_forward)\ndef test_order():\n\n with pytest.raises(AssertionError):\n # order must be a tuple\n order = ['conv', 'norm', 'act']\n ConvModule(3, 8, 2, order=order)\n\n with pytest.raises(AssertionError):\n # length of order must be 3\n order = ('conv', 'norm')\n ConvModule(3, 8, 2, order=order)\n\n with pytest.raises(AssertionError):\n # order must be an order of 'conv', 'norm', 'act'\n order = ('conv', 'norm', 'norm')\n ConvModule(3, 8, 2, order=order)\n\n with pytest.raises(AssertionError):\n # order must be an order of 'conv', 'norm', 'act'\n order = ('conv', 'norm', 'something')\n ConvModule(3, 8, 2, order=order)\n\n # ('conv', 'norm', 'act')\n conv = ConvModule(3, 8, 2, norm_cfg=dict(type='BN'))\n out = conv('input')\n assert out == 'input_conv_bn_relu'\n\n # ('norm', 'conv', 'act')\n conv = ConvModule(\n 3, 8, 2, norm_cfg=dict(type='BN'), order=('norm', 'conv', 'act'))\n out = conv('input')\n assert out == 'input_bn_conv_relu'\n\n # ('conv', 'norm', 'act'), activate=False\n conv = ConvModule(3, 8, 2, norm_cfg=dict(type='BN'))\n out = conv('input', activate=False)\n assert out == 'input_conv_bn'\n\n # ('conv', 'norm', 'act'), activate=False\n conv = ConvModule(3, 8, 2, norm_cfg=dict(type='BN'))\n out = conv('input', norm=False)\n assert out == 'input_conv_relu'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_depthwise_seperable_conv_module.py", "content": "import pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.cnn.bricks import DepthwiseSeparableConvModule\n\n\ndef test_depthwise_separable_conv():\n with pytest.raises(AssertionError):\n # conv_cfg must be a dict or None\n DepthwiseSeparableConvModule(4, 8, 2, groups=2)\n\n # test default config\n conv = DepthwiseSeparableConvModule(3, 8, 2)\n assert conv.depthwise_conv.conv.groups == 3\n assert conv.pointwise_conv.conv.kernel_size == (1, 1)\n assert not conv.depthwise_conv.with_norm\n assert not conv.pointwise_conv.with_norm\n assert conv.depthwise_conv.activate.__class__.__name__ == 'ReLU'\n assert conv.pointwise_conv.activate.__class__.__name__ == 'ReLU'\n x = torch.rand(1, 3, 256, 256)\n output = conv(x)\n assert output.shape == (1, 8, 255, 255)\n\n # test dw_norm_cfg\n conv = DepthwiseSeparableConvModule(3, 8, 2, dw_norm_cfg=dict(type='BN'))\n assert conv.depthwise_conv.norm_name == 'bn'\n assert not conv.pointwise_conv.with_norm\n x = torch.rand(1, 3, 256, 256)\n output = conv(x)\n assert output.shape == (1, 8, 255, 255)\n\n # test pw_norm_cfg\n conv = DepthwiseSeparableConvModule(3, 8, 2, pw_norm_cfg=dict(type='BN'))\n assert not conv.depthwise_conv.with_norm\n assert conv.pointwise_conv.norm_name == 'bn'\n x = torch.rand(1, 3, 256, 256)\n output = conv(x)\n assert output.shape == (1, 8, 255, 255)\n\n # test norm_cfg\n conv = DepthwiseSeparableConvModule(3, 8, 2, norm_cfg=dict(type='BN'))\n assert conv.depthwise_conv.norm_name == 'bn'\n assert conv.pointwise_conv.norm_name == 'bn'\n x = torch.rand(1, 3, 256, 256)\n output = conv(x)\n assert output.shape == (1, 8, 255, 255)\n\n # add test for ['norm', 'conv', 'act']\n conv = DepthwiseSeparableConvModule(3, 8, 2, order=('norm', 'conv', 'act'))\n x = torch.rand(1, 3, 256, 256)\n output = conv(x)\n assert output.shape == (1, 8, 255, 255)\n\n conv = DepthwiseSeparableConvModule(\n 3, 8, 3, padding=1, with_spectral_norm=True)\n assert hasattr(conv.depthwise_conv.conv, 'weight_orig')\n assert hasattr(conv.pointwise_conv.conv, 'weight_orig')\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n conv = DepthwiseSeparableConvModule(\n 3, 8, 3, padding=1, padding_mode='reflect')\n assert isinstance(conv.depthwise_conv.padding_layer, nn.ReflectionPad2d)\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # test dw_act_cfg\n conv = DepthwiseSeparableConvModule(\n 3, 8, 3, padding=1, dw_act_cfg=dict(type='LeakyReLU'))\n assert conv.depthwise_conv.activate.__class__.__name__ == 'LeakyReLU'\n assert conv.pointwise_conv.activate.__class__.__name__ == 'ReLU'\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # test pw_act_cfg\n conv = DepthwiseSeparableConvModule(\n 3, 8, 3, padding=1, pw_act_cfg=dict(type='LeakyReLU'))\n assert conv.depthwise_conv.activate.__class__.__name__ == 'ReLU'\n assert conv.pointwise_conv.activate.__class__.__name__ == 'LeakyReLU'\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n\n # test act_cfg\n conv = DepthwiseSeparableConvModule(\n 3, 8, 3, padding=1, act_cfg=dict(type='LeakyReLU'))\n assert conv.depthwise_conv.activate.__class__.__name__ == 'LeakyReLU'\n assert conv.pointwise_conv.activate.__class__.__name__ == 'LeakyReLU'\n output = conv(x)\n assert output.shape == (1, 8, 256, 256)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_flops_counter.py", "content": "import pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.cnn import get_model_complexity_info\nfrom mmcv.cnn.utils.flops_counter import flops_to_string, params_to_string\n\ntry:\n from StringIO import StringIO\nexcept ImportError:\n from io import StringIO\n\n# yapf: disable\ngt_results = [\n {'model': nn.Conv1d(3, 8, 3), 'input': (3, 16), 'flops': 1120.0, 'params': 80.0}, # noqa: E501\n {'model': nn.Conv2d(3, 8, 3), 'input': (3, 16, 16), 'flops': 43904.0, 'params': 224.0}, # noqa: E501\n {'model': nn.Conv3d(3, 8, 3), 'input': (3, 3, 16, 16), 'flops': 128576.0, 'params': 656.0}, # noqa: E501\n {'model': nn.ReLU(), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.PReLU(), 'input': (3, 16, 16), 'flops': 768.0, 'params': 1}, # noqa: E501\n {'model': nn.ELU(), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.LeakyReLU(), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.ReLU6(), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.MaxPool1d(2), 'input': (3, 16), 'flops': 48.0, 'params': 0}, # noqa: E501\n {'model': nn.MaxPool2d(2), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.MaxPool3d(2), 'input': (3, 3, 16, 16), 'flops': 2304.0, 'params': 0}, # noqa: E501\n {'model': nn.AvgPool1d(2), 'input': (3, 16), 'flops': 48.0, 'params': 0}, # noqa: E501\n {'model': nn.AvgPool2d(2), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.AvgPool3d(2), 'input': (3, 3, 16, 16), 'flops': 2304.0, 'params': 0}, # noqa: E501\n {'model': nn.AdaptiveMaxPool1d(2), 'input': (3, 16), 'flops': 48.0, 'params': 0}, # noqa: E501\n {'model': nn.AdaptiveMaxPool2d(2), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.AdaptiveMaxPool3d(2), 'input': (3, 3, 16, 16), 'flops': 2304.0, 'params': 0}, # noqa: E501\n {'model': nn.AdaptiveAvgPool1d(2), 'input': (3, 16), 'flops': 48.0, 'params': 0}, # noqa: E501\n {'model': nn.AdaptiveAvgPool2d(2), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.AdaptiveAvgPool3d(2), 'input': (3, 3, 16, 16), 'flops': 2304.0, 'params': 0}, # noqa: E501\n {'model': nn.BatchNorm1d(3), 'input': (3, 16), 'flops': 96.0, 'params': 6.0}, # noqa: E501\n {'model': nn.BatchNorm2d(3), 'input': (3, 16, 16), 'flops': 1536.0, 'params': 6.0}, # noqa: E501\n {'model': nn.BatchNorm3d(3), 'input': (3, 3, 16, 16), 'flops': 4608.0, 'params': 6.0}, # noqa: E501\n {'model': nn.GroupNorm(2, 6), 'input': (6, 16, 16), 'flops': 3072.0, 'params': 12.0}, # noqa: E501\n {'model': nn.InstanceNorm1d(3, affine=True), 'input': (3, 16), 'flops': 96.0, 'params': 6.0}, # noqa: E501\n {'model': nn.InstanceNorm2d(3, affine=True), 'input': (3, 16, 16), 'flops': 1536.0, 'params': 6.0}, # noqa: E501\n {'model': nn.InstanceNorm3d(3, affine=True), 'input': (3, 3, 16, 16), 'flops': 4608.0, 'params': 6.0}, # noqa: E501\n {'model': nn.LayerNorm((3, 16, 16)), 'input': (3, 16, 16), 'flops': 1536.0, 'params': 1536.0}, # noqa: E501\n {'model': nn.LayerNorm((3, 16, 16), elementwise_affine=False), 'input': (3, 16, 16), 'flops': 768.0, 'params': 0}, # noqa: E501\n {'model': nn.Linear(1024, 2), 'input': (1024, ), 'flops': 2048.0, 'params': 2050.0}, # noqa: E501\n {'model': nn.ConvTranspose2d(3, 8, 3), 'input': (3, 16, 16), 'flops': 57888, 'params': 224.0}, # noqa: E501\n {'model': nn.Upsample((32, 32)), 'input': (3, 16, 16), 'flops': 3072.0, 'params': 0} # noqa: E501\n]\n# yapf: enable\n\n\nclass ExampleModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.conv2d = nn.Conv2d(3, 8, 3)\n\n def forward(self, imgs):\n x = torch.randn((1, *imgs))\n return self.conv2d(x)\n\n\ndef input_constructor(x):\n return dict(imgs=x)\n\n\ndef test_flops_counter():\n with pytest.raises(AssertionError):\n # input_res should be a tuple\n model = nn.Conv2d(3, 8, 3)\n input_res = [1, 3, 16, 16]\n get_model_complexity_info(model, input_res)\n\n with pytest.raises(AssertionError):\n # len(input_res) >= 2\n model = nn.Conv2d(3, 8, 3)\n input_res = tuple()\n get_model_complexity_info(model, input_res)\n\n # test common layers\n for item in gt_results:\n model = item['model']\n input = item['input']\n flops, params = get_model_complexity_info(\n model, input, as_strings=False, print_per_layer_stat=False)\n assert flops == item['flops'] and params == item['params']\n\n # test input constructor\n model = ExampleModel()\n x = (3, 16, 16)\n flops, params = get_model_complexity_info(\n model,\n x,\n as_strings=False,\n print_per_layer_stat=False,\n input_constructor=input_constructor)\n assert flops == 43904.0 and params == 224.0\n\n # test output string\n model = nn.Conv3d(3, 8, 3)\n x = (3, 3, 512, 512)\n flops, params = get_model_complexity_info(\n model, x, print_per_layer_stat=False)\n assert flops == '0.17 GFLOPs' and params == str(656)\n\n # test print per layer status\n model = nn.Conv1d(3, 8, 3)\n x = (3, 16)\n out = StringIO()\n get_model_complexity_info(model, x, ost=out)\n assert out.getvalue() == \\\n 'Conv1d(0.0 M, 100.000% Params, 0.0 GFLOPs, 100.000% FLOPs, 3, 8, kernel_size=(3,), stride=(1,))\\n' # noqa: E501\n\n # test when model is not a common instance\n model = nn.Sequential(nn.Conv2d(3, 8, 3), nn.Flatten(), nn.Linear(1568, 2))\n x = (3, 16, 16)\n flops, params = get_model_complexity_info(\n model, x, as_strings=False, print_per_layer_stat=True)\n assert flops == 47040.0 and params == 3362\n\n\ndef test_flops_to_string():\n flops = 6.54321 * 10.**9\n assert flops_to_string(flops) == '6.54 GFLOPs'\n assert flops_to_string(flops, 'MFLOPs') == '6543.21 MFLOPs'\n assert flops_to_string(flops, 'KFLOPs') == '6543210.0 KFLOPs'\n assert flops_to_string(flops, 'FLOPs') == '6543210000.0 FLOPs'\n assert flops_to_string(flops, precision=4) == '6.5432 GFLOPs'\n\n flops = 6.54321 * 10.**9\n assert flops_to_string(flops, None) == '6.54 GFLOPs'\n flops = 3.21 * 10.**7\n assert flops_to_string(flops, None) == '32.1 MFLOPs'\n flops = 5.4 * 10.**3\n assert flops_to_string(flops, None) == '5.4 KFLOPs'\n flops = 987\n assert flops_to_string(flops, None) == '987 FLOPs'\n\n\ndef test_params_to_string():\n num_params = 3.21 * 10.**7\n assert params_to_string(num_params) == '32.1 M'\n num_params = 4.56 * 10.**5\n assert params_to_string(num_params) == '456.0 k'\n num_params = 7.89 * 10.**2\n assert params_to_string(num_params) == '789.0'\n\n num_params = 6.54321 * 10.**7\n assert params_to_string(num_params, 'M') == '65.43 M'\n assert params_to_string(num_params, 'K') == '65432.1 K'\n assert params_to_string(num_params, '') == '65432100.0'\n assert params_to_string(num_params, precision=4) == '65.4321 M'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_fuse_conv_bn.py", "content": "import torch\nimport torch.nn as nn\n\nfrom mmcv.cnn import ConvModule, fuse_conv_bn\n\n\ndef test_fuse_conv_bn():\n inputs = torch.rand((1, 3, 5, 5))\n modules = nn.ModuleList()\n modules.append(nn.BatchNorm2d(3))\n modules.append(ConvModule(3, 5, 3, norm_cfg=dict(type='BN')))\n modules.append(ConvModule(5, 5, 3, norm_cfg=dict(type='BN')))\n modules = nn.Sequential(*modules)\n fused_modules = fuse_conv_bn(modules)\n assert torch.equal(modules(inputs), fused_modules(inputs))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_generalized_attention.py", "content": "import torch\n\nfrom mmcv.cnn.bricks import GeneralizedAttention\n\n\ndef test_context_block():\n\n # test attention_type='1000'\n imgs = torch.randn(2, 16, 20, 20)\n gen_attention_block = GeneralizedAttention(16, attention_type='1000')\n assert gen_attention_block.query_conv.in_channels == 16\n assert gen_attention_block.key_conv.in_channels == 16\n assert gen_attention_block.key_conv.in_channels == 16\n out = gen_attention_block(imgs)\n assert out.shape == imgs.shape\n\n # test attention_type='0100'\n imgs = torch.randn(2, 16, 20, 20)\n gen_attention_block = GeneralizedAttention(16, attention_type='0100')\n assert gen_attention_block.query_conv.in_channels == 16\n assert gen_attention_block.appr_geom_fc_x.in_features == 8\n assert gen_attention_block.appr_geom_fc_y.in_features == 8\n out = gen_attention_block(imgs)\n assert out.shape == imgs.shape\n\n # test attention_type='0010'\n imgs = torch.randn(2, 16, 20, 20)\n gen_attention_block = GeneralizedAttention(16, attention_type='0010')\n assert gen_attention_block.key_conv.in_channels == 16\n assert hasattr(gen_attention_block, 'appr_bias')\n out = gen_attention_block(imgs)\n assert out.shape == imgs.shape\n\n # test attention_type='0001'\n imgs = torch.randn(2, 16, 20, 20)\n gen_attention_block = GeneralizedAttention(16, attention_type='0001')\n assert gen_attention_block.appr_geom_fc_x.in_features == 8\n assert gen_attention_block.appr_geom_fc_y.in_features == 8\n assert hasattr(gen_attention_block, 'geom_bias')\n out = gen_attention_block(imgs)\n assert out.shape == imgs.shape\n\n # test spatial_range >= 0\n imgs = torch.randn(2, 256, 20, 20)\n gen_attention_block = GeneralizedAttention(256, spatial_range=10)\n assert hasattr(gen_attention_block, 'local_constraint_map')\n out = gen_attention_block(imgs)\n assert out.shape == imgs.shape\n\n # test q_stride > 1\n imgs = torch.randn(2, 16, 20, 20)\n gen_attention_block = GeneralizedAttention(16, q_stride=2)\n assert gen_attention_block.q_downsample is not None\n out = gen_attention_block(imgs)\n assert out.shape == imgs.shape\n\n # test kv_stride > 1\n imgs = torch.randn(2, 16, 20, 20)\n gen_attention_block = GeneralizedAttention(16, kv_stride=2)\n assert gen_attention_block.kv_downsample is not None\n out = gen_attention_block(imgs)\n assert out.shape == imgs.shape\n\n # test fp16 with attention_type='1111'\n if torch.cuda.is_available():\n imgs = torch.randn(2, 16, 20, 20).cuda().to(torch.half)\n gen_attention_block = GeneralizedAttention(\n 16,\n spatial_range=-1,\n num_heads=8,\n attention_type='1111',\n kv_stride=2)\n gen_attention_block.cuda().type(torch.half)\n out = gen_attention_block(imgs)\n assert out.shape == imgs.shape\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_hsigmoid.py", "content": "import pytest\nimport torch\n\nfrom mmcv.cnn.bricks import HSigmoid\n\n\ndef test_hsigmoid():\n # test assertion divisor can not be zero\n with pytest.raises(AssertionError):\n HSigmoid(divisor=0)\n\n # test with default parameters\n act = HSigmoid()\n input_shape = torch.Size([1, 3, 64, 64])\n input = torch.randn(input_shape)\n output = act(input)\n expected_output = torch.min(\n torch.max((input + 3) / 6, torch.zeros(input_shape)),\n torch.ones(input_shape))\n # test output shape\n assert output.shape == expected_output.shape\n # test output value\n assert torch.equal(output, expected_output)\n\n # test with designated parameters\n act = HSigmoid(1, 2, 0, 1)\n input_shape = torch.Size([1, 3, 64, 64])\n input = torch.randn(input_shape)\n output = act(input)\n expected_output = torch.min(\n torch.max((input + 1) / 2, torch.zeros(input_shape)),\n torch.ones(input_shape))\n # test output shape\n assert output.shape == expected_output.shape\n # test output value\n assert torch.equal(output, expected_output)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_hswish.py", "content": "import torch\nfrom torch.nn.functional import relu6\n\nfrom mmcv.cnn.bricks import HSwish\n\n\ndef test_hswish():\n # test inplace\n act = HSwish(inplace=True)\n assert act.act.inplace\n act = HSwish()\n assert not act.act.inplace\n\n input = torch.randn(1, 3, 64, 64)\n expected_output = input * relu6(input + 3) / 6\n output = act(input)\n # test output shape\n assert output.shape == expected_output.shape\n # test output value\n assert torch.equal(output, expected_output)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_model_registry.py", "content": "import torch.nn as nn\n\nimport mmcv\nfrom mmcv.cnn import MODELS, build_model_from_cfg\n\n\ndef test_build_model_from_cfg():\n BACKBONES = mmcv.Registry('backbone', build_func=build_model_from_cfg)\n\n @BACKBONES.register_module()\n class ResNet(nn.Module):\n\n def __init__(self, depth, stages=4):\n super().__init__()\n self.depth = depth\n self.stages = stages\n\n def forward(self, x):\n return x\n\n @BACKBONES.register_module()\n class ResNeXt(nn.Module):\n\n def __init__(self, depth, stages=4):\n super().__init__()\n self.depth = depth\n self.stages = stages\n\n def forward(self, x):\n return x\n\n cfg = dict(type='ResNet', depth=50)\n model = BACKBONES.build(cfg)\n assert isinstance(model, ResNet)\n assert model.depth == 50 and model.stages == 4\n\n cfg = dict(type='ResNeXt', depth=50, stages=3)\n model = BACKBONES.build(cfg)\n assert isinstance(model, ResNeXt)\n assert model.depth == 50 and model.stages == 3\n\n cfg = [\n dict(type='ResNet', depth=50),\n dict(type='ResNeXt', depth=50, stages=3)\n ]\n model = BACKBONES.build(cfg)\n assert isinstance(model, nn.Sequential)\n assert isinstance(model[0], ResNet)\n assert model[0].depth == 50 and model[0].stages == 4\n assert isinstance(model[1], ResNeXt)\n assert model[1].depth == 50 and model[1].stages == 3\n\n # test inherit `build_func` from parent\n NEW_MODELS = mmcv.Registry('models', parent=MODELS, scope='new')\n assert NEW_MODELS.build_func is build_model_from_cfg\n\n # test specify `build_func`\n def pseudo_build(cfg):\n return cfg\n\n NEW_MODELS = mmcv.Registry(\n 'models', parent=MODELS, build_func=pseudo_build)\n assert NEW_MODELS.build_func is pseudo_build\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_non_local.py", "content": "import pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.cnn import NonLocal1d, NonLocal2d, NonLocal3d\nfrom mmcv.cnn.bricks.non_local import _NonLocalNd\n\n\ndef test_nonlocal():\n with pytest.raises(ValueError):\n # mode should be in ['embedded_gaussian', 'dot_product']\n _NonLocalNd(3, mode='unsupport_mode')\n\n # _NonLocalNd with zero initialization\n _NonLocalNd(3)\n _NonLocalNd(3, norm_cfg=dict(type='BN'))\n\n # _NonLocalNd without zero initialization\n _NonLocalNd(3, zeros_init=False)\n _NonLocalNd(3, norm_cfg=dict(type='BN'), zeros_init=False)\n\n\ndef test_nonlocal3d():\n # NonLocal3d with 'embedded_gaussian' mode\n imgs = torch.randn(2, 3, 10, 20, 20)\n nonlocal_3d = NonLocal3d(3)\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n # NonLocal is only implemented on gpu in parrots\n imgs = imgs.cuda()\n nonlocal_3d.cuda()\n out = nonlocal_3d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal3d with 'dot_product' mode\n nonlocal_3d = NonLocal3d(3, mode='dot_product')\n assert nonlocal_3d.mode == 'dot_product'\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_3d.cuda()\n out = nonlocal_3d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal3d with 'concatenation' mode\n nonlocal_3d = NonLocal3d(3, mode='concatenation')\n assert nonlocal_3d.mode == 'concatenation'\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_3d.cuda()\n out = nonlocal_3d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal3d with 'gaussian' mode\n nonlocal_3d = NonLocal3d(3, mode='gaussian')\n assert not hasattr(nonlocal_3d, 'phi')\n assert nonlocal_3d.mode == 'gaussian'\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_3d.cuda()\n out = nonlocal_3d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal3d with 'gaussian' mode and sub_sample\n nonlocal_3d = NonLocal3d(3, mode='gaussian', sub_sample=True)\n assert isinstance(nonlocal_3d.g, nn.Sequential) and len(nonlocal_3d.g) == 2\n assert isinstance(nonlocal_3d.g[1], nn.MaxPool3d)\n assert nonlocal_3d.g[1].kernel_size == (1, 2, 2)\n assert isinstance(nonlocal_3d.phi, nn.MaxPool3d)\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_3d.cuda()\n out = nonlocal_3d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal3d with 'dot_product' mode and sub_sample\n nonlocal_3d = NonLocal3d(3, mode='dot_product', sub_sample=True)\n for m in [nonlocal_3d.g, nonlocal_3d.phi]:\n assert isinstance(m, nn.Sequential) and len(m) == 2\n assert isinstance(m[1], nn.MaxPool3d)\n assert m[1].kernel_size == (1, 2, 2)\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_3d.cuda()\n out = nonlocal_3d(imgs)\n assert out.shape == imgs.shape\n\n\ndef test_nonlocal2d():\n # NonLocal2d with 'embedded_gaussian' mode\n imgs = torch.randn(2, 3, 20, 20)\n nonlocal_2d = NonLocal2d(3)\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n imgs = imgs.cuda()\n nonlocal_2d.cuda()\n out = nonlocal_2d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal2d with 'dot_product' mode\n imgs = torch.randn(2, 3, 20, 20)\n nonlocal_2d = NonLocal2d(3, mode='dot_product')\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n imgs = imgs.cuda()\n nonlocal_2d.cuda()\n out = nonlocal_2d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal2d with 'concatenation' mode\n imgs = torch.randn(2, 3, 20, 20)\n nonlocal_2d = NonLocal2d(3, mode='concatenation')\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n imgs = imgs.cuda()\n nonlocal_2d.cuda()\n out = nonlocal_2d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal2d with 'gaussian' mode\n imgs = torch.randn(2, 3, 20, 20)\n nonlocal_2d = NonLocal2d(3, mode='gaussian')\n assert not hasattr(nonlocal_2d, 'phi')\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n imgs = imgs.cuda()\n nonlocal_2d.cuda()\n out = nonlocal_2d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal2d with 'gaussian' mode and sub_sample\n nonlocal_2d = NonLocal2d(3, mode='gaussian', sub_sample=True)\n assert isinstance(nonlocal_2d.g, nn.Sequential) and len(nonlocal_2d.g) == 2\n assert isinstance(nonlocal_2d.g[1], nn.MaxPool2d)\n assert nonlocal_2d.g[1].kernel_size == (2, 2)\n assert isinstance(nonlocal_2d.phi, nn.MaxPool2d)\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_2d.cuda()\n out = nonlocal_2d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal2d with 'dot_product' mode and sub_sample\n nonlocal_2d = NonLocal2d(3, mode='dot_product', sub_sample=True)\n for m in [nonlocal_2d.g, nonlocal_2d.phi]:\n assert isinstance(m, nn.Sequential) and len(m) == 2\n assert isinstance(m[1], nn.MaxPool2d)\n assert m[1].kernel_size == (2, 2)\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_2d.cuda()\n out = nonlocal_2d(imgs)\n assert out.shape == imgs.shape\n\n\ndef test_nonlocal1d():\n # NonLocal1d with 'embedded_gaussian' mode\n imgs = torch.randn(2, 3, 20)\n nonlocal_1d = NonLocal1d(3)\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n imgs = imgs.cuda()\n nonlocal_1d.cuda()\n out = nonlocal_1d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal1d with 'dot_product' mode\n imgs = torch.randn(2, 3, 20)\n nonlocal_1d = NonLocal1d(3, mode='dot_product')\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n imgs = imgs.cuda()\n nonlocal_1d.cuda()\n out = nonlocal_1d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal1d with 'concatenation' mode\n imgs = torch.randn(2, 3, 20)\n nonlocal_1d = NonLocal1d(3, mode='concatenation')\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n imgs = imgs.cuda()\n nonlocal_1d.cuda()\n out = nonlocal_1d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal1d with 'gaussian' mode\n imgs = torch.randn(2, 3, 20)\n nonlocal_1d = NonLocal1d(3, mode='gaussian')\n assert not hasattr(nonlocal_1d, 'phi')\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n imgs = imgs.cuda()\n nonlocal_1d.cuda()\n out = nonlocal_1d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal1d with 'gaussian' mode and sub_sample\n nonlocal_1d = NonLocal1d(3, mode='gaussian', sub_sample=True)\n assert isinstance(nonlocal_1d.g, nn.Sequential) and len(nonlocal_1d.g) == 2\n assert isinstance(nonlocal_1d.g[1], nn.MaxPool1d)\n assert nonlocal_1d.g[1].kernel_size == 2\n assert isinstance(nonlocal_1d.phi, nn.MaxPool1d)\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_1d.cuda()\n out = nonlocal_1d(imgs)\n assert out.shape == imgs.shape\n\n # NonLocal1d with 'dot_product' mode and sub_sample\n nonlocal_1d = NonLocal1d(3, mode='dot_product', sub_sample=True)\n for m in [nonlocal_1d.g, nonlocal_1d.phi]:\n assert isinstance(m, nn.Sequential) and len(m) == 2\n assert isinstance(m[1], nn.MaxPool1d)\n assert m[1].kernel_size == 2\n if torch.__version__ == 'parrots':\n if torch.cuda.is_available():\n nonlocal_1d.cuda()\n out = nonlocal_1d(imgs)\n assert out.shape == imgs.shape\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_revert_syncbn.py", "content": "import os\nimport platform\n\nimport numpy as np\nimport pytest\nimport torch\nimport torch.distributed as dist\n\nfrom mmcv.cnn.bricks import ConvModule\nfrom mmcv.cnn.utils import revert_sync_batchnorm\n\nif platform.system() == 'Windows':\n import regex as re\nelse:\n import re\n\n\ndef test_revert_syncbn():\n conv = ConvModule(3, 8, 2, norm_cfg=dict(type='SyncBN'))\n x = torch.randn(1, 3, 10, 10)\n # Expect a ValueError prompting that SyncBN is not supported on CPU\n with pytest.raises(ValueError):\n y = conv(x)\n conv = revert_sync_batchnorm(conv)\n y = conv(x)\n assert y.shape == (1, 8, 9, 9)\n\n\ndef test_revert_mmsyncbn():\n if 'SLURM_NTASKS' not in os.environ or int(os.environ['SLURM_NTASKS']) < 2:\n print('Must run on slurm with more than 1 process!\\n'\n 'srun -p test --gres=gpu:2 -n2')\n return\n rank = int(os.environ['SLURM_PROCID'])\n world_size = int(os.environ['SLURM_NTASKS'])\n local_rank = int(os.environ['SLURM_LOCALID'])\n node_list = str(os.environ['SLURM_NODELIST'])\n\n node_parts = re.findall('[0-9]+', node_list)\n os.environ['MASTER_ADDR'] = (f'{node_parts[1]}.{node_parts[2]}' +\n f'.{node_parts[3]}.{node_parts[4]}')\n os.environ['MASTER_PORT'] = '12341'\n os.environ['WORLD_SIZE'] = str(world_size)\n os.environ['RANK'] = str(rank)\n\n dist.init_process_group('nccl')\n torch.cuda.set_device(local_rank)\n x = torch.randn(1, 3, 10, 10).cuda()\n dist.broadcast(x, src=0)\n conv = ConvModule(3, 8, 2, norm_cfg=dict(type='MMSyncBN')).cuda()\n conv.eval()\n y_mmsyncbn = conv(x).detach().cpu().numpy()\n conv = revert_sync_batchnorm(conv)\n y_bn = conv(x).detach().cpu().numpy()\n assert np.all(np.isclose(y_bn, y_mmsyncbn, 1e-3))\n conv, x = conv.to('cpu'), x.to('cpu')\n y_bn_cpu = conv(x).detach().numpy()\n assert np.all(np.isclose(y_bn, y_bn_cpu, 1e-3))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_scale.py", "content": "import torch\n\nfrom mmcv.cnn.bricks import Scale\n\n\ndef test_scale():\n # test default scale\n scale = Scale()\n assert scale.scale.data == 1.\n assert scale.scale.dtype == torch.float\n x = torch.rand(1, 3, 64, 64)\n output = scale(x)\n assert output.shape == (1, 3, 64, 64)\n\n # test given scale\n scale = Scale(10.)\n assert scale.scale.data == 10.\n assert scale.scale.dtype == torch.float\n x = torch.rand(1, 3, 64, 64)\n output = scale(x)\n assert output.shape == (1, 3, 64, 64)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_swish.py", "content": "import torch\nimport torch.nn.functional as F\n\nfrom mmcv.cnn.bricks import Swish\n\n\ndef test_swish():\n act = Swish()\n input = torch.randn(1, 3, 64, 64)\n expected_output = input * F.sigmoid(input)\n output = act(input)\n # test output shape\n assert output.shape == expected_output.shape\n # test output value\n assert torch.equal(output, expected_output)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_transformer.py", "content": "import copy\n\nimport pytest\nimport torch\n\nfrom mmcv.cnn.bricks.drop import DropPath\nfrom mmcv.cnn.bricks.transformer import (FFN, AdaptivePadding,\n BaseTransformerLayer,\n MultiheadAttention, PatchEmbed,\n PatchMerging,\n TransformerLayerSequence)\nfrom mmcv.runner import ModuleList\n\n\ndef test_adaptive_padding():\n\n for padding in ('same', 'corner'):\n kernel_size = 16\n stride = 16\n dilation = 1\n input = torch.rand(1, 1, 15, 17)\n adap_pad = AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=padding)\n out = adap_pad(input)\n # padding to divisible by 16\n assert (out.shape[2], out.shape[3]) == (16, 32)\n input = torch.rand(1, 1, 16, 17)\n out = adap_pad(input)\n # padding to divisible by 16\n assert (out.shape[2], out.shape[3]) == (16, 32)\n\n kernel_size = (2, 2)\n stride = (2, 2)\n dilation = (1, 1)\n\n adap_pad = AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=padding)\n input = torch.rand(1, 1, 11, 13)\n out = adap_pad(input)\n # padding to divisible by 2\n assert (out.shape[2], out.shape[3]) == (12, 14)\n\n kernel_size = (2, 2)\n stride = (10, 10)\n dilation = (1, 1)\n\n adap_pad = AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=padding)\n input = torch.rand(1, 1, 10, 13)\n out = adap_pad(input)\n # no padding\n assert (out.shape[2], out.shape[3]) == (10, 13)\n\n kernel_size = (11, 11)\n adap_pad = AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=padding)\n input = torch.rand(1, 1, 11, 13)\n out = adap_pad(input)\n # all padding\n assert (out.shape[2], out.shape[3]) == (21, 21)\n\n # test padding as kernel is (7,9)\n input = torch.rand(1, 1, 11, 13)\n stride = (3, 4)\n kernel_size = (4, 5)\n dilation = (2, 2)\n # actually (7, 9)\n adap_pad = AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=padding)\n dilation_out = adap_pad(input)\n assert (dilation_out.shape[2], dilation_out.shape[3]) == (16, 21)\n kernel_size = (7, 9)\n dilation = (1, 1)\n adap_pad = AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=padding)\n kernel79_out = adap_pad(input)\n assert (kernel79_out.shape[2], kernel79_out.shape[3]) == (16, 21)\n assert kernel79_out.shape == dilation_out.shape\n\n # assert only support \"same\" \"corner\"\n with pytest.raises(AssertionError):\n AdaptivePadding(\n kernel_size=kernel_size,\n stride=stride,\n dilation=dilation,\n padding=1)\n\n\ndef test_patch_embed():\n B = 2\n H = 3\n W = 4\n C = 3\n embed_dims = 10\n kernel_size = 3\n stride = 1\n dummy_input = torch.rand(B, C, H, W)\n patch_merge_1 = PatchEmbed(\n in_channels=C,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=0,\n dilation=1,\n norm_cfg=None)\n\n x1, shape = patch_merge_1(dummy_input)\n # test out shape\n assert x1.shape == (2, 2, 10)\n # test outsize is correct\n assert shape == (1, 2)\n # test L = out_h * out_w\n assert shape[0] * shape[1] == x1.shape[1]\n\n B = 2\n H = 10\n W = 10\n C = 3\n embed_dims = 10\n kernel_size = 5\n stride = 2\n dummy_input = torch.rand(B, C, H, W)\n # test dilation\n patch_merge_2 = PatchEmbed(\n in_channels=C,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=0,\n dilation=2,\n norm_cfg=None,\n )\n\n x2, shape = patch_merge_2(dummy_input)\n # test out shape\n assert x2.shape == (2, 1, 10)\n # test outsize is correct\n assert shape == (1, 1)\n # test L = out_h * out_w\n assert shape[0] * shape[1] == x2.shape[1]\n\n stride = 2\n input_size = (10, 10)\n\n dummy_input = torch.rand(B, C, H, W)\n # test stride and norm\n patch_merge_3 = PatchEmbed(\n in_channels=C,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=0,\n dilation=2,\n norm_cfg=dict(type='LN'),\n input_size=input_size)\n\n x3, shape = patch_merge_3(dummy_input)\n # test out shape\n assert x3.shape == (2, 1, 10)\n # test outsize is correct\n assert shape == (1, 1)\n # test L = out_h * out_w\n assert shape[0] * shape[1] == x3.shape[1]\n\n # test the init_out_size with nn.Unfold\n assert patch_merge_3.init_out_size[1] == (input_size[0] - 2 * 4 -\n 1) // 2 + 1\n assert patch_merge_3.init_out_size[0] == (input_size[0] - 2 * 4 -\n 1) // 2 + 1\n H = 11\n W = 12\n input_size = (H, W)\n dummy_input = torch.rand(B, C, H, W)\n # test stride and norm\n patch_merge_3 = PatchEmbed(\n in_channels=C,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=0,\n dilation=2,\n norm_cfg=dict(type='LN'),\n input_size=input_size)\n\n _, shape = patch_merge_3(dummy_input)\n # when input_size equal to real input\n # the out_size should be equal to `init_out_size`\n assert shape == patch_merge_3.init_out_size\n\n input_size = (H, W)\n dummy_input = torch.rand(B, C, H, W)\n # test stride and norm\n patch_merge_3 = PatchEmbed(\n in_channels=C,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=0,\n dilation=2,\n norm_cfg=dict(type='LN'),\n input_size=input_size)\n\n _, shape = patch_merge_3(dummy_input)\n # when input_size equal to real input\n # the out_size should be equal to `init_out_size`\n assert shape == patch_merge_3.init_out_size\n\n # test adap padding\n for padding in ('same', 'corner'):\n in_c = 2\n embed_dims = 3\n B = 2\n\n # test stride is 1\n input_size = (5, 5)\n kernel_size = (5, 5)\n stride = (1, 1)\n dilation = 1\n bias = False\n\n x = torch.rand(B, in_c, *input_size)\n patch_embed = PatchEmbed(\n in_channels=in_c,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n x_out, out_size = patch_embed(x)\n assert x_out.size() == (B, 25, 3)\n assert out_size == (5, 5)\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n # test kernel_size == stride\n input_size = (5, 5)\n kernel_size = (5, 5)\n stride = (5, 5)\n dilation = 1\n bias = False\n\n x = torch.rand(B, in_c, *input_size)\n patch_embed = PatchEmbed(\n in_channels=in_c,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n x_out, out_size = patch_embed(x)\n assert x_out.size() == (B, 1, 3)\n assert out_size == (1, 1)\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n # test kernel_size == stride\n input_size = (6, 5)\n kernel_size = (5, 5)\n stride = (5, 5)\n dilation = 1\n bias = False\n\n x = torch.rand(B, in_c, *input_size)\n patch_embed = PatchEmbed(\n in_channels=in_c,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n x_out, out_size = patch_embed(x)\n assert x_out.size() == (B, 2, 3)\n assert out_size == (2, 1)\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n # test different kernel_size with different stride\n input_size = (6, 5)\n kernel_size = (6, 2)\n stride = (6, 2)\n dilation = 1\n bias = False\n\n x = torch.rand(B, in_c, *input_size)\n patch_embed = PatchEmbed(\n in_channels=in_c,\n embed_dims=embed_dims,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n x_out, out_size = patch_embed(x)\n assert x_out.size() == (B, 3, 3)\n assert out_size == (1, 3)\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n\ndef test_patch_merging():\n\n # Test the model with int padding\n in_c = 3\n out_c = 4\n kernel_size = 3\n stride = 3\n padding = 1\n dilation = 1\n bias = False\n # test the case `pad_to_stride` is False\n patch_merge = PatchMerging(\n in_channels=in_c,\n out_channels=out_c,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n B, L, C = 1, 100, 3\n input_size = (10, 10)\n x = torch.rand(B, L, C)\n x_out, out_size = patch_merge(x, input_size)\n assert x_out.size() == (1, 16, 4)\n assert out_size == (4, 4)\n # assert out size is consistent with real output\n assert x_out.size(1) == out_size[0] * out_size[1]\n in_c = 4\n out_c = 5\n kernel_size = 6\n stride = 3\n padding = 2\n dilation = 2\n bias = False\n patch_merge = PatchMerging(\n in_channels=in_c,\n out_channels=out_c,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n B, L, C = 1, 100, 4\n input_size = (10, 10)\n x = torch.rand(B, L, C)\n x_out, out_size = patch_merge(x, input_size)\n assert x_out.size() == (1, 4, 5)\n assert out_size == (2, 2)\n # assert out size is consistent with real output\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n # Test with adaptive padding\n for padding in ('same', 'corner'):\n in_c = 2\n out_c = 3\n B = 2\n\n # test stride is 1\n input_size = (5, 5)\n kernel_size = (5, 5)\n stride = (1, 1)\n dilation = 1\n bias = False\n L = input_size[0] * input_size[1]\n\n x = torch.rand(B, L, in_c)\n patch_merge = PatchMerging(\n in_channels=in_c,\n out_channels=out_c,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n x_out, out_size = patch_merge(x, input_size)\n assert x_out.size() == (B, 25, 3)\n assert out_size == (5, 5)\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n # test kernel_size == stride\n input_size = (5, 5)\n kernel_size = (5, 5)\n stride = (5, 5)\n dilation = 1\n bias = False\n L = input_size[0] * input_size[1]\n\n x = torch.rand(B, L, in_c)\n patch_merge = PatchMerging(\n in_channels=in_c,\n out_channels=out_c,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n x_out, out_size = patch_merge(x, input_size)\n assert x_out.size() == (B, 1, 3)\n assert out_size == (1, 1)\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n # test kernel_size == stride\n input_size = (6, 5)\n kernel_size = (5, 5)\n stride = (5, 5)\n dilation = 1\n bias = False\n L = input_size[0] * input_size[1]\n\n x = torch.rand(B, L, in_c)\n patch_merge = PatchMerging(\n in_channels=in_c,\n out_channels=out_c,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n x_out, out_size = patch_merge(x, input_size)\n assert x_out.size() == (B, 2, 3)\n assert out_size == (2, 1)\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n # test different kernel_size with different stride\n input_size = (6, 5)\n kernel_size = (6, 2)\n stride = (6, 2)\n dilation = 1\n bias = False\n L = input_size[0] * input_size[1]\n\n x = torch.rand(B, L, in_c)\n patch_merge = PatchMerging(\n in_channels=in_c,\n out_channels=out_c,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=bias)\n\n x_out, out_size = patch_merge(x, input_size)\n assert x_out.size() == (B, 3, 3)\n assert out_size == (1, 3)\n assert x_out.size(1) == out_size[0] * out_size[1]\n\n\ndef test_multiheadattention():\n MultiheadAttention(\n embed_dims=5,\n num_heads=5,\n attn_drop=0,\n proj_drop=0,\n dropout_layer=dict(type='Dropout', drop_prob=0.),\n batch_first=True)\n batch_dim = 2\n embed_dim = 5\n num_query = 100\n attn_batch_first = MultiheadAttention(\n embed_dims=5,\n num_heads=5,\n attn_drop=0,\n proj_drop=0,\n dropout_layer=dict(type='DropPath', drop_prob=0.),\n batch_first=True)\n\n attn_query_first = MultiheadAttention(\n embed_dims=5,\n num_heads=5,\n attn_drop=0,\n proj_drop=0,\n dropout_layer=dict(type='DropPath', drop_prob=0.),\n batch_first=False)\n\n param_dict = dict(attn_query_first.named_parameters())\n for n, v in attn_batch_first.named_parameters():\n param_dict[n].data = v.data\n\n input_batch_first = torch.rand(batch_dim, num_query, embed_dim)\n input_query_first = input_batch_first.transpose(0, 1)\n\n assert torch.allclose(\n attn_query_first(input_query_first).sum(),\n attn_batch_first(input_batch_first).sum())\n\n key_batch_first = torch.rand(batch_dim, num_query, embed_dim)\n key_query_first = key_batch_first.transpose(0, 1)\n\n assert torch.allclose(\n attn_query_first(input_query_first, key_query_first).sum(),\n attn_batch_first(input_batch_first, key_batch_first).sum())\n\n identity = torch.ones_like(input_query_first)\n\n # check deprecated arguments can be used normally\n\n assert torch.allclose(\n attn_query_first(\n input_query_first, key_query_first, residual=identity).sum(),\n attn_batch_first(input_batch_first, key_batch_first).sum() +\n identity.sum() - input_batch_first.sum())\n\n assert torch.allclose(\n attn_query_first(\n input_query_first, key_query_first, identity=identity).sum(),\n attn_batch_first(input_batch_first, key_batch_first).sum() +\n identity.sum() - input_batch_first.sum())\n\n attn_query_first(\n input_query_first, key_query_first, identity=identity).sum(),\n\n\ndef test_ffn():\n with pytest.raises(AssertionError):\n # num_fcs should be no less than 2\n FFN(num_fcs=1)\n FFN(dropout=0, add_residual=True)\n ffn = FFN(dropout=0, add_identity=True)\n\n input_tensor = torch.rand(2, 20, 256)\n input_tensor_nbc = input_tensor.transpose(0, 1)\n assert torch.allclose(ffn(input_tensor).sum(), ffn(input_tensor_nbc).sum())\n residual = torch.rand_like(input_tensor)\n torch.allclose(\n ffn(input_tensor, residual=residual).sum(),\n ffn(input_tensor).sum() + residual.sum() - input_tensor.sum())\n\n torch.allclose(\n ffn(input_tensor, identity=residual).sum(),\n ffn(input_tensor).sum() + residual.sum() - input_tensor.sum())\n\n\n@pytest.mark.skipif(not torch.cuda.is_available(), reason='Cuda not available')\ndef test_basetransformerlayer_cuda():\n # To test if the BaseTransformerLayer's behaviour remains\n # consistent after being deepcopied\n operation_order = ('self_attn', 'ffn')\n baselayer = BaseTransformerLayer(\n operation_order=operation_order,\n batch_first=True,\n attn_cfgs=dict(\n type='MultiheadAttention',\n embed_dims=256,\n num_heads=8,\n ),\n )\n baselayers = ModuleList([copy.deepcopy(baselayer) for _ in range(2)])\n baselayers.to('cuda')\n x = torch.rand(2, 10, 256).cuda()\n for m in baselayers:\n x = m(x)\n assert x.shape == torch.Size([2, 10, 256])\n\n\n@pytest.mark.parametrize('embed_dims', [False, 256])\ndef test_basetransformerlayer(embed_dims):\n attn_cfgs = dict(type='MultiheadAttention', embed_dims=256, num_heads=8),\n if embed_dims:\n ffn_cfgs = dict(\n type='FFN',\n embed_dims=embed_dims,\n feedforward_channels=1024,\n num_fcs=2,\n ffn_drop=0.,\n act_cfg=dict(type='ReLU', inplace=True),\n )\n else:\n ffn_cfgs = dict(\n type='FFN',\n feedforward_channels=1024,\n num_fcs=2,\n ffn_drop=0.,\n act_cfg=dict(type='ReLU', inplace=True),\n )\n\n feedforward_channels = 2048\n ffn_dropout = 0.1\n operation_order = ('self_attn', 'norm', 'ffn', 'norm')\n\n # test deprecated_args\n baselayer = BaseTransformerLayer(\n attn_cfgs=attn_cfgs,\n ffn_cfgs=ffn_cfgs,\n feedforward_channels=feedforward_channels,\n ffn_dropout=ffn_dropout,\n operation_order=operation_order)\n assert baselayer.batch_first is False\n assert baselayer.ffns[0].feedforward_channels == feedforward_channels\n\n attn_cfgs = dict(type='MultiheadAttention', num_heads=8, embed_dims=256),\n feedforward_channels = 2048\n ffn_dropout = 0.1\n operation_order = ('self_attn', 'norm', 'ffn', 'norm')\n baselayer = BaseTransformerLayer(\n attn_cfgs=attn_cfgs,\n feedforward_channels=feedforward_channels,\n ffn_dropout=ffn_dropout,\n operation_order=operation_order,\n batch_first=True)\n assert baselayer.attentions[0].batch_first\n in_tensor = torch.rand(2, 10, 256)\n baselayer(in_tensor)\n\n\ndef test_transformerlayersequence():\n squeue = TransformerLayerSequence(\n num_layers=6,\n transformerlayers=dict(\n type='BaseTransformerLayer',\n attn_cfgs=[\n dict(\n type='MultiheadAttention',\n embed_dims=256,\n num_heads=8,\n dropout=0.1),\n dict(type='MultiheadAttention', embed_dims=256, num_heads=4)\n ],\n feedforward_channels=1024,\n ffn_dropout=0.1,\n operation_order=('self_attn', 'norm', 'cross_attn', 'norm', 'ffn',\n 'norm')))\n assert len(squeue.layers) == 6\n assert squeue.pre_norm is False\n with pytest.raises(AssertionError):\n # if transformerlayers is a list, len(transformerlayers)\n # should be equal to num_layers\n TransformerLayerSequence(\n num_layers=6,\n transformerlayers=[\n dict(\n type='BaseTransformerLayer',\n attn_cfgs=[\n dict(\n type='MultiheadAttention',\n embed_dims=256,\n num_heads=8,\n dropout=0.1),\n dict(type='MultiheadAttention', embed_dims=256)\n ],\n feedforward_channels=1024,\n ffn_dropout=0.1,\n operation_order=('self_attn', 'norm', 'cross_attn', 'norm',\n 'ffn', 'norm'))\n ])\n\n\ndef test_drop_path():\n drop_path = DropPath(drop_prob=0)\n test_in = torch.rand(2, 3, 4, 5)\n assert test_in is drop_path(test_in)\n\n drop_path = DropPath(drop_prob=0.1)\n drop_path.training = False\n test_in = torch.rand(2, 3, 4, 5)\n assert test_in is drop_path(test_in)\n drop_path.training = True\n assert test_in is not drop_path(test_in)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_weight_init.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport random\nfrom tempfile import TemporaryDirectory\n\nimport numpy as np\nimport pytest\nimport torch\nfrom scipy import stats\nfrom torch import nn\n\nfrom mmcv.cnn import (Caffe2XavierInit, ConstantInit, KaimingInit, NormalInit,\n PretrainedInit, TruncNormalInit, UniformInit, XavierInit,\n bias_init_with_prob, caffe2_xavier_init, constant_init,\n initialize, kaiming_init, normal_init, trunc_normal_init,\n uniform_init, xavier_init)\n\n\ndef test_constant_init():\n conv_module = nn.Conv2d(3, 16, 3)\n constant_init(conv_module, 0.1)\n assert conv_module.weight.allclose(\n torch.full_like(conv_module.weight, 0.1))\n assert conv_module.bias.allclose(torch.zeros_like(conv_module.bias))\n conv_module_no_bias = nn.Conv2d(3, 16, 3, bias=False)\n constant_init(conv_module_no_bias, 0.1)\n assert conv_module.weight.allclose(\n torch.full_like(conv_module.weight, 0.1))\n\n\ndef test_xavier_init():\n conv_module = nn.Conv2d(3, 16, 3)\n xavier_init(conv_module, bias=0.1)\n assert conv_module.bias.allclose(torch.full_like(conv_module.bias, 0.1))\n xavier_init(conv_module, distribution='uniform')\n # TODO: sanity check of weight distribution, e.g. mean, std\n with pytest.raises(AssertionError):\n xavier_init(conv_module, distribution='student-t')\n conv_module_no_bias = nn.Conv2d(3, 16, 3, bias=False)\n xavier_init(conv_module_no_bias)\n\n\ndef test_normal_init():\n conv_module = nn.Conv2d(3, 16, 3)\n normal_init(conv_module, bias=0.1)\n # TODO: sanity check of weight distribution, e.g. mean, std\n assert conv_module.bias.allclose(torch.full_like(conv_module.bias, 0.1))\n conv_module_no_bias = nn.Conv2d(3, 16, 3, bias=False)\n normal_init(conv_module_no_bias)\n # TODO: sanity check distribution, e.g. mean, std\n\n\ndef test_trunc_normal_init():\n\n def _random_float(a, b):\n return (b - a) * random.random() + a\n\n def _is_trunc_normal(tensor, mean, std, a, b):\n # scipy's trunc norm is suited for data drawn from N(0, 1),\n # so we need to transform our data to test it using scipy.\n z_samples = (tensor.view(-1) - mean) / std\n z_samples = z_samples.tolist()\n a0 = (a - mean) / std\n b0 = (b - mean) / std\n p_value = stats.kstest(z_samples, 'truncnorm', args=(a0, b0))[1]\n return p_value > 0.0001\n\n conv_module = nn.Conv2d(3, 16, 3)\n mean = _random_float(-3, 3)\n std = _random_float(.01, 1)\n a = _random_float(mean - 2 * std, mean)\n b = _random_float(mean, mean + 2 * std)\n trunc_normal_init(conv_module, mean, std, a, b, bias=0.1)\n assert _is_trunc_normal(conv_module.weight, mean, std, a, b)\n assert conv_module.bias.allclose(torch.full_like(conv_module.bias, 0.1))\n\n conv_module_no_bias = nn.Conv2d(3, 16, 3, bias=False)\n trunc_normal_init(conv_module_no_bias)\n # TODO: sanity check distribution, e.g. mean, std\n\n\ndef test_uniform_init():\n conv_module = nn.Conv2d(3, 16, 3)\n uniform_init(conv_module, bias=0.1)\n # TODO: sanity check of weight distribution, e.g. mean, std\n assert conv_module.bias.allclose(torch.full_like(conv_module.bias, 0.1))\n conv_module_no_bias = nn.Conv2d(3, 16, 3, bias=False)\n uniform_init(conv_module_no_bias)\n\n\ndef test_kaiming_init():\n conv_module = nn.Conv2d(3, 16, 3)\n kaiming_init(conv_module, bias=0.1)\n # TODO: sanity check of weight distribution, e.g. mean, std\n assert conv_module.bias.allclose(torch.full_like(conv_module.bias, 0.1))\n kaiming_init(conv_module, distribution='uniform')\n with pytest.raises(AssertionError):\n kaiming_init(conv_module, distribution='student-t')\n conv_module_no_bias = nn.Conv2d(3, 16, 3, bias=False)\n kaiming_init(conv_module_no_bias)\n\n\ndef test_caffe_xavier_init():\n conv_module = nn.Conv2d(3, 16, 3)\n caffe2_xavier_init(conv_module)\n\n\ndef test_bias_init_with_prob():\n conv_module = nn.Conv2d(3, 16, 3)\n prior_prob = 0.1\n normal_init(conv_module, bias=bias_init_with_prob(0.1))\n # TODO: sanity check of weight distribution, e.g. mean, std\n bias = float(-np.log((1 - prior_prob) / prior_prob))\n assert conv_module.bias.allclose(torch.full_like(conv_module.bias, bias))\n\n\ndef test_constaninit():\n \"\"\"test ConstantInit class.\"\"\"\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Linear(1, 2))\n func = ConstantInit(val=1, bias=2, layer='Conv2d')\n func(model)\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape, 1.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 2.))\n\n assert not torch.equal(model[2].weight,\n torch.full(model[2].weight.shape, 1.))\n assert not torch.equal(model[2].bias, torch.full(model[2].bias.shape, 2.))\n\n func = ConstantInit(val=3, bias_prob=0.01, layer='Linear')\n func(model)\n res = bias_init_with_prob(0.01)\n\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape, 1.))\n assert torch.equal(model[2].weight, torch.full(model[2].weight.shape, 3.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 2.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, res))\n\n # test layer key with base class name\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Conv1d(1, 2, 1))\n func = ConstantInit(val=4., bias=5., layer='_ConvNd')\n func(model)\n assert torch.all(model[0].weight == 4.)\n assert torch.all(model[2].weight == 4.)\n assert torch.all(model[0].bias == 5.)\n assert torch.all(model[2].bias == 5.)\n\n # test bias input type\n with pytest.raises(TypeError):\n func = ConstantInit(val=1, bias='1')\n # test bias_prob type\n with pytest.raises(TypeError):\n func = ConstantInit(val=1, bias_prob='1')\n # test layer input type\n with pytest.raises(TypeError):\n func = ConstantInit(val=1, layer=1)\n\n\ndef test_xavierinit():\n \"\"\"test XavierInit class.\"\"\"\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Linear(1, 2))\n func = XavierInit(bias=0.1, layer='Conv2d')\n func(model)\n assert model[0].bias.allclose(torch.full_like(model[2].bias, 0.1))\n assert not model[2].bias.allclose(torch.full_like(model[0].bias, 0.1))\n\n constant_func = ConstantInit(val=0, bias=0, layer=['Conv2d', 'Linear'])\n func = XavierInit(gain=100, bias_prob=0.01, layer=['Conv2d', 'Linear'])\n model.apply(constant_func)\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape, 0.))\n assert torch.equal(model[2].weight, torch.full(model[2].weight.shape, 0.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 0.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 0.))\n\n res = bias_init_with_prob(0.01)\n func(model)\n assert not torch.equal(model[0].weight,\n torch.full(model[0].weight.shape, 0.))\n assert not torch.equal(model[2].weight,\n torch.full(model[2].weight.shape, 0.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, res))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, res))\n\n # test layer key with base class name\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Conv1d(1, 2, 1))\n func = ConstantInit(val=4., bias=5., layer='_ConvNd')\n func(model)\n assert torch.all(model[0].weight == 4.)\n assert torch.all(model[2].weight == 4.)\n assert torch.all(model[0].bias == 5.)\n assert torch.all(model[2].bias == 5.)\n\n func = XavierInit(gain=100, bias_prob=0.01, layer='_ConvNd')\n func(model)\n assert not torch.all(model[0].weight == 4.)\n assert not torch.all(model[2].weight == 4.)\n assert torch.all(model[0].bias == res)\n assert torch.all(model[2].bias == res)\n\n # test bias input type\n with pytest.raises(TypeError):\n func = XavierInit(bias='0.1', layer='Conv2d')\n # test layer inpur type\n with pytest.raises(TypeError):\n func = XavierInit(bias=0.1, layer=1)\n\n\ndef test_normalinit():\n \"\"\"test Normalinit class.\"\"\"\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Linear(1, 2))\n\n func = NormalInit(mean=100, std=1e-5, bias=200, layer=['Conv2d', 'Linear'])\n func(model)\n assert model[0].weight.allclose(torch.tensor(100.))\n assert model[2].weight.allclose(torch.tensor(100.))\n assert model[0].bias.allclose(torch.tensor(200.))\n assert model[2].bias.allclose(torch.tensor(200.))\n\n func = NormalInit(\n mean=300, std=1e-5, bias_prob=0.01, layer=['Conv2d', 'Linear'])\n res = bias_init_with_prob(0.01)\n func(model)\n assert model[0].weight.allclose(torch.tensor(300.))\n assert model[2].weight.allclose(torch.tensor(300.))\n assert model[0].bias.allclose(torch.tensor(res))\n assert model[2].bias.allclose(torch.tensor(res))\n\n # test layer key with base class name\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Conv1d(1, 2, 1))\n\n func = NormalInit(mean=300, std=1e-5, bias_prob=0.01, layer='_ConvNd')\n func(model)\n assert model[0].weight.allclose(torch.tensor(300.))\n assert model[2].weight.allclose(torch.tensor(300.))\n assert torch.all(model[0].bias == res)\n assert torch.all(model[2].bias == res)\n\n\ndef test_truncnormalinit():\n \"\"\"test TruncNormalInit class.\"\"\"\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Linear(1, 2))\n\n func = TruncNormalInit(\n mean=100, std=1e-5, bias=200, a=0, b=200, layer=['Conv2d', 'Linear'])\n func(model)\n assert model[0].weight.allclose(torch.tensor(100.))\n assert model[2].weight.allclose(torch.tensor(100.))\n assert model[0].bias.allclose(torch.tensor(200.))\n assert model[2].bias.allclose(torch.tensor(200.))\n\n func = TruncNormalInit(\n mean=300,\n std=1e-5,\n a=100,\n b=400,\n bias_prob=0.01,\n layer=['Conv2d', 'Linear'])\n res = bias_init_with_prob(0.01)\n func(model)\n assert model[0].weight.allclose(torch.tensor(300.))\n assert model[2].weight.allclose(torch.tensor(300.))\n assert model[0].bias.allclose(torch.tensor(res))\n assert model[2].bias.allclose(torch.tensor(res))\n\n # test layer key with base class name\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Conv1d(1, 2, 1))\n\n func = TruncNormalInit(\n mean=300, std=1e-5, a=100, b=400, bias_prob=0.01, layer='_ConvNd')\n func(model)\n assert model[0].weight.allclose(torch.tensor(300.))\n assert model[2].weight.allclose(torch.tensor(300.))\n assert torch.all(model[0].bias == res)\n assert torch.all(model[2].bias == res)\n\n\ndef test_uniforminit():\n \"\"\"\"test UniformInit class.\"\"\"\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Linear(1, 2))\n func = UniformInit(a=1, b=1, bias=2, layer=['Conv2d', 'Linear'])\n func(model)\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape, 1.))\n assert torch.equal(model[2].weight, torch.full(model[2].weight.shape, 1.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 2.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 2.))\n\n func = UniformInit(a=100, b=100, layer=['Conv2d', 'Linear'], bias=10)\n func(model)\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape,\n 100.))\n assert torch.equal(model[2].weight, torch.full(model[2].weight.shape,\n 100.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 10.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 10.))\n\n # test layer key with base class name\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Conv1d(1, 2, 1))\n\n func = UniformInit(a=100, b=100, bias_prob=0.01, layer='_ConvNd')\n res = bias_init_with_prob(0.01)\n func(model)\n assert torch.all(model[0].weight == 100.)\n assert torch.all(model[2].weight == 100.)\n assert torch.all(model[0].bias == res)\n assert torch.all(model[2].bias == res)\n\n\ndef test_kaiminginit():\n \"\"\"test KaimingInit class.\"\"\"\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Linear(1, 2))\n func = KaimingInit(bias=0.1, layer='Conv2d')\n func(model)\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 0.1))\n assert not torch.equal(model[2].bias, torch.full(model[2].bias.shape, 0.1))\n\n func = KaimingInit(a=100, bias=10, layer=['Conv2d', 'Linear'])\n constant_func = ConstantInit(val=0, bias=0, layer=['Conv2d', 'Linear'])\n model.apply(constant_func)\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape, 0.))\n assert torch.equal(model[2].weight, torch.full(model[2].weight.shape, 0.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 0.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 0.))\n\n func(model)\n assert not torch.equal(model[0].weight,\n torch.full(model[0].weight.shape, 0.))\n assert not torch.equal(model[2].weight,\n torch.full(model[2].weight.shape, 0.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 10.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 10.))\n\n # test layer key with base class name\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Conv1d(1, 2, 1))\n func = KaimingInit(bias=0.1, layer='_ConvNd')\n func(model)\n assert torch.all(model[0].bias == 0.1)\n assert torch.all(model[2].bias == 0.1)\n\n func = KaimingInit(a=100, bias=10, layer='_ConvNd')\n constant_func = ConstantInit(val=0, bias=0, layer='_ConvNd')\n model.apply(constant_func)\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape, 0.))\n assert torch.equal(model[2].weight, torch.full(model[2].weight.shape, 0.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 0.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 0.))\n\n func(model)\n assert not torch.equal(model[0].weight,\n torch.full(model[0].weight.shape, 0.))\n assert not torch.equal(model[2].weight,\n torch.full(model[2].weight.shape, 0.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 10.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 10.))\n\n\ndef test_caffe2xavierinit():\n \"\"\"test Caffe2XavierInit.\"\"\"\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Linear(1, 2))\n func = Caffe2XavierInit(bias=0.1, layer='Conv2d')\n func(model)\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 0.1))\n assert not torch.equal(model[2].bias, torch.full(model[2].bias.shape, 0.1))\n\n\nclass FooModule(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.linear = nn.Linear(1, 2)\n self.conv2d = nn.Conv2d(3, 1, 3)\n self.conv2d_2 = nn.Conv2d(3, 2, 3)\n\n\ndef test_pretrainedinit():\n \"\"\"test PretrainedInit class.\"\"\"\n\n modelA = FooModule()\n constant_func = ConstantInit(val=1, bias=2, layer=['Conv2d', 'Linear'])\n modelA.apply(constant_func)\n modelB = FooModule()\n funcB = PretrainedInit(checkpoint='modelA.pth')\n modelC = nn.Linear(1, 2)\n funcC = PretrainedInit(checkpoint='modelA.pth', prefix='linear.')\n with TemporaryDirectory():\n torch.save(modelA.state_dict(), 'modelA.pth')\n funcB(modelB)\n assert torch.equal(modelB.linear.weight,\n torch.full(modelB.linear.weight.shape, 1.))\n assert torch.equal(modelB.linear.bias,\n torch.full(modelB.linear.bias.shape, 2.))\n assert torch.equal(modelB.conv2d.weight,\n torch.full(modelB.conv2d.weight.shape, 1.))\n assert torch.equal(modelB.conv2d.bias,\n torch.full(modelB.conv2d.bias.shape, 2.))\n assert torch.equal(modelB.conv2d_2.weight,\n torch.full(modelB.conv2d_2.weight.shape, 1.))\n assert torch.equal(modelB.conv2d_2.bias,\n torch.full(modelB.conv2d_2.bias.shape, 2.))\n\n funcC(modelC)\n assert torch.equal(modelC.weight, torch.full(modelC.weight.shape, 1.))\n assert torch.equal(modelC.bias, torch.full(modelC.bias.shape, 2.))\n\n\ndef test_initialize():\n model = nn.Sequential(nn.Conv2d(3, 1, 3), nn.ReLU(), nn.Linear(1, 2))\n foonet = FooModule()\n\n # test layer key\n init_cfg = dict(type='Constant', layer=['Conv2d', 'Linear'], val=1, bias=2)\n initialize(model, init_cfg)\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape, 1.))\n assert torch.equal(model[2].weight, torch.full(model[2].weight.shape, 1.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 2.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 2.))\n assert init_cfg == dict(\n type='Constant', layer=['Conv2d', 'Linear'], val=1, bias=2)\n\n # test init_cfg with list type\n init_cfg = [\n dict(type='Constant', layer='Conv2d', val=1, bias=2),\n dict(type='Constant', layer='Linear', val=3, bias=4)\n ]\n initialize(model, init_cfg)\n assert torch.equal(model[0].weight, torch.full(model[0].weight.shape, 1.))\n assert torch.equal(model[2].weight, torch.full(model[2].weight.shape, 3.))\n assert torch.equal(model[0].bias, torch.full(model[0].bias.shape, 2.))\n assert torch.equal(model[2].bias, torch.full(model[2].bias.shape, 4.))\n assert init_cfg == [\n dict(type='Constant', layer='Conv2d', val=1, bias=2),\n dict(type='Constant', layer='Linear', val=3, bias=4)\n ]\n\n # test layer key and override key\n init_cfg = dict(\n type='Constant',\n val=1,\n bias=2,\n layer=['Conv2d', 'Linear'],\n override=dict(type='Constant', name='conv2d_2', val=3, bias=4))\n initialize(foonet, init_cfg)\n assert torch.equal(foonet.linear.weight,\n torch.full(foonet.linear.weight.shape, 1.))\n assert torch.equal(foonet.linear.bias,\n torch.full(foonet.linear.bias.shape, 2.))\n assert torch.equal(foonet.conv2d.weight,\n torch.full(foonet.conv2d.weight.shape, 1.))\n assert torch.equal(foonet.conv2d.bias,\n torch.full(foonet.conv2d.bias.shape, 2.))\n assert torch.equal(foonet.conv2d_2.weight,\n torch.full(foonet.conv2d_2.weight.shape, 3.))\n assert torch.equal(foonet.conv2d_2.bias,\n torch.full(foonet.conv2d_2.bias.shape, 4.))\n assert init_cfg == dict(\n type='Constant',\n val=1,\n bias=2,\n layer=['Conv2d', 'Linear'],\n override=dict(type='Constant', name='conv2d_2', val=3, bias=4))\n\n # test override key\n init_cfg = dict(\n type='Constant', val=5, bias=6, override=dict(name='conv2d_2'))\n initialize(foonet, init_cfg)\n assert not torch.equal(foonet.linear.weight,\n torch.full(foonet.linear.weight.shape, 5.))\n assert not torch.equal(foonet.linear.bias,\n torch.full(foonet.linear.bias.shape, 6.))\n assert not torch.equal(foonet.conv2d.weight,\n torch.full(foonet.conv2d.weight.shape, 5.))\n assert not torch.equal(foonet.conv2d.bias,\n torch.full(foonet.conv2d.bias.shape, 6.))\n assert torch.equal(foonet.conv2d_2.weight,\n torch.full(foonet.conv2d_2.weight.shape, 5.))\n assert torch.equal(foonet.conv2d_2.bias,\n torch.full(foonet.conv2d_2.bias.shape, 6.))\n assert init_cfg == dict(\n type='Constant', val=5, bias=6, override=dict(name='conv2d_2'))\n\n init_cfg = dict(\n type='Pretrained',\n checkpoint='modelA.pth',\n override=dict(type='Constant', name='conv2d_2', val=3, bias=4))\n modelA = FooModule()\n constant_func = ConstantInit(val=1, bias=2, layer=['Conv2d', 'Linear'])\n modelA.apply(constant_func)\n with TemporaryDirectory():\n torch.save(modelA.state_dict(), 'modelA.pth')\n initialize(foonet, init_cfg)\n assert torch.equal(foonet.linear.weight,\n torch.full(foonet.linear.weight.shape, 1.))\n assert torch.equal(foonet.linear.bias,\n torch.full(foonet.linear.bias.shape, 2.))\n assert torch.equal(foonet.conv2d.weight,\n torch.full(foonet.conv2d.weight.shape, 1.))\n assert torch.equal(foonet.conv2d.bias,\n torch.full(foonet.conv2d.bias.shape, 2.))\n assert torch.equal(foonet.conv2d_2.weight,\n torch.full(foonet.conv2d_2.weight.shape, 3.))\n assert torch.equal(foonet.conv2d_2.bias,\n torch.full(foonet.conv2d_2.bias.shape, 4.))\n assert init_cfg == dict(\n type='Pretrained',\n checkpoint='modelA.pth',\n override=dict(type='Constant', name='conv2d_2', val=3, bias=4))\n\n # test init_cfg type\n with pytest.raises(TypeError):\n init_cfg = 'init_cfg'\n initialize(foonet, init_cfg)\n\n # test override value type\n with pytest.raises(TypeError):\n init_cfg = dict(\n type='Constant',\n val=1,\n bias=2,\n layer=['Conv2d', 'Linear'],\n override='conv')\n initialize(foonet, init_cfg)\n\n # test override name\n with pytest.raises(RuntimeError):\n init_cfg = dict(\n type='Constant',\n val=1,\n bias=2,\n layer=['Conv2d', 'Linear'],\n override=dict(type='Constant', name='conv2d_3', val=3, bias=4))\n initialize(foonet, init_cfg)\n\n # test list override name\n with pytest.raises(RuntimeError):\n init_cfg = dict(\n type='Constant',\n val=1,\n bias=2,\n layer=['Conv2d', 'Linear'],\n override=[\n dict(type='Constant', name='conv2d', val=3, bias=4),\n dict(type='Constant', name='conv2d_3', val=5, bias=6)\n ])\n initialize(foonet, init_cfg)\n\n # test override with args except type key\n with pytest.raises(ValueError):\n init_cfg = dict(\n type='Constant',\n val=1,\n bias=2,\n override=dict(name='conv2d_2', val=3, bias=4))\n initialize(foonet, init_cfg)\n\n # test override without name\n with pytest.raises(ValueError):\n init_cfg = dict(\n type='Constant',\n val=1,\n bias=2,\n override=dict(type='Constant', val=3, bias=4))\n initialize(foonet, init_cfg)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_cnn/test_wrappers.py", "content": "from unittest.mock import patch\n\nimport pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.cnn.bricks import (Conv2d, Conv3d, ConvTranspose2d, ConvTranspose3d,\n Linear, MaxPool2d, MaxPool3d)\n\nif torch.__version__ != 'parrots':\n torch_version = '1.1'\nelse:\n torch_version = 'parrots'\n\n\n@patch('torch.__version__', torch_version)\n@pytest.mark.parametrize(\n 'in_w,in_h,in_channel,out_channel,kernel_size,stride,padding,dilation',\n [(10, 10, 1, 1, 3, 1, 0, 1), (20, 20, 3, 3, 5, 2, 1, 2)])\ndef test_conv2d(in_w, in_h, in_channel, out_channel, kernel_size, stride,\n padding, dilation):\n \"\"\"\n CommandLine:\n xdoctest -m tests/test_wrappers.py test_conv2d\n \"\"\"\n # train mode\n # wrapper op with 0-dim input\n x_empty = torch.randn(0, in_channel, in_h, in_w)\n torch.manual_seed(0)\n wrapper = Conv2d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation)\n wrapper_out = wrapper(x_empty)\n\n # torch op with 3-dim input as shape reference\n x_normal = torch.randn(3, in_channel, in_h, in_w).requires_grad_(True)\n torch.manual_seed(0)\n ref = nn.Conv2d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation)\n ref_out = ref(x_normal)\n\n assert wrapper_out.shape[0] == 0\n assert wrapper_out.shape[1:] == ref_out.shape[1:]\n\n wrapper_out.sum().backward()\n assert wrapper.weight.grad is not None\n assert wrapper.weight.grad.shape == wrapper.weight.shape\n\n assert torch.equal(wrapper(x_normal), ref_out)\n\n # eval mode\n x_empty = torch.randn(0, in_channel, in_h, in_w)\n wrapper = Conv2d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation)\n wrapper.eval()\n wrapper(x_empty)\n\n\n@patch('torch.__version__', torch_version)\n@pytest.mark.parametrize(\n 'in_w,in_h,in_t,in_channel,out_channel,kernel_size,stride,padding,dilation', # noqa: E501\n [(10, 10, 10, 1, 1, 3, 1, 0, 1), (20, 20, 20, 3, 3, 5, 2, 1, 2)])\ndef test_conv3d(in_w, in_h, in_t, in_channel, out_channel, kernel_size, stride,\n padding, dilation):\n \"\"\"\n CommandLine:\n xdoctest -m tests/test_wrappers.py test_conv3d\n \"\"\"\n # train mode\n # wrapper op with 0-dim input\n x_empty = torch.randn(0, in_channel, in_t, in_h, in_w)\n torch.manual_seed(0)\n wrapper = Conv3d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation)\n wrapper_out = wrapper(x_empty)\n\n # torch op with 3-dim input as shape reference\n x_normal = torch.randn(3, in_channel, in_t, in_h,\n in_w).requires_grad_(True)\n torch.manual_seed(0)\n ref = nn.Conv3d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation)\n ref_out = ref(x_normal)\n\n assert wrapper_out.shape[0] == 0\n assert wrapper_out.shape[1:] == ref_out.shape[1:]\n\n wrapper_out.sum().backward()\n assert wrapper.weight.grad is not None\n assert wrapper.weight.grad.shape == wrapper.weight.shape\n\n assert torch.equal(wrapper(x_normal), ref_out)\n\n # eval mode\n x_empty = torch.randn(0, in_channel, in_t, in_h, in_w)\n wrapper = Conv3d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation)\n wrapper.eval()\n wrapper(x_empty)\n\n\n@patch('torch.__version__', torch_version)\n@pytest.mark.parametrize(\n 'in_w,in_h,in_channel,out_channel,kernel_size,stride,padding,dilation',\n [(10, 10, 1, 1, 3, 1, 0, 1), (20, 20, 3, 3, 5, 2, 1, 2)])\ndef test_conv_transposed_2d(in_w, in_h, in_channel, out_channel, kernel_size,\n stride, padding, dilation):\n # wrapper op with 0-dim input\n x_empty = torch.randn(0, in_channel, in_h, in_w, requires_grad=True)\n # out padding must be smaller than either stride or dilation\n op = min(stride, dilation) - 1\n if torch.__version__ == 'parrots':\n op = 0\n torch.manual_seed(0)\n wrapper = ConvTranspose2d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n output_padding=op)\n wrapper_out = wrapper(x_empty)\n\n # torch op with 3-dim input as shape reference\n x_normal = torch.randn(3, in_channel, in_h, in_w)\n torch.manual_seed(0)\n ref = nn.ConvTranspose2d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n output_padding=op)\n ref_out = ref(x_normal)\n\n assert wrapper_out.shape[0] == 0\n assert wrapper_out.shape[1:] == ref_out.shape[1:]\n\n wrapper_out.sum().backward()\n assert wrapper.weight.grad is not None\n assert wrapper.weight.grad.shape == wrapper.weight.shape\n\n assert torch.equal(wrapper(x_normal), ref_out)\n\n # eval mode\n x_empty = torch.randn(0, in_channel, in_h, in_w)\n wrapper = ConvTranspose2d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n output_padding=op)\n wrapper.eval()\n wrapper(x_empty)\n\n\n@patch('torch.__version__', torch_version)\n@pytest.mark.parametrize(\n 'in_w,in_h,in_t,in_channel,out_channel,kernel_size,stride,padding,dilation', # noqa: E501\n [(10, 10, 10, 1, 1, 3, 1, 0, 1), (20, 20, 20, 3, 3, 5, 2, 1, 2)])\ndef test_conv_transposed_3d(in_w, in_h, in_t, in_channel, out_channel,\n kernel_size, stride, padding, dilation):\n # wrapper op with 0-dim input\n x_empty = torch.randn(0, in_channel, in_t, in_h, in_w, requires_grad=True)\n # out padding must be smaller than either stride or dilation\n op = min(stride, dilation) - 1\n torch.manual_seed(0)\n wrapper = ConvTranspose3d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n output_padding=op)\n wrapper_out = wrapper(x_empty)\n\n # torch op with 3-dim input as shape reference\n x_normal = torch.randn(3, in_channel, in_t, in_h, in_w)\n torch.manual_seed(0)\n ref = nn.ConvTranspose3d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n output_padding=op)\n ref_out = ref(x_normal)\n\n assert wrapper_out.shape[0] == 0\n assert wrapper_out.shape[1:] == ref_out.shape[1:]\n\n wrapper_out.sum().backward()\n assert wrapper.weight.grad is not None\n assert wrapper.weight.grad.shape == wrapper.weight.shape\n\n assert torch.equal(wrapper(x_normal), ref_out)\n\n # eval mode\n x_empty = torch.randn(0, in_channel, in_t, in_h, in_w)\n wrapper = ConvTranspose3d(\n in_channel,\n out_channel,\n kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n output_padding=op)\n wrapper.eval()\n wrapper(x_empty)\n\n\n@patch('torch.__version__', torch_version)\n@pytest.mark.parametrize(\n 'in_w,in_h,in_channel,out_channel,kernel_size,stride,padding,dilation',\n [(10, 10, 1, 1, 3, 1, 0, 1), (20, 20, 3, 3, 5, 2, 1, 2)])\ndef test_max_pool_2d(in_w, in_h, in_channel, out_channel, kernel_size, stride,\n padding, dilation):\n # wrapper op with 0-dim input\n x_empty = torch.randn(0, in_channel, in_h, in_w, requires_grad=True)\n wrapper = MaxPool2d(\n kernel_size, stride=stride, padding=padding, dilation=dilation)\n wrapper_out = wrapper(x_empty)\n\n # torch op with 3-dim input as shape reference\n x_normal = torch.randn(3, in_channel, in_h, in_w)\n ref = nn.MaxPool2d(\n kernel_size, stride=stride, padding=padding, dilation=dilation)\n ref_out = ref(x_normal)\n\n assert wrapper_out.shape[0] == 0\n assert wrapper_out.shape[1:] == ref_out.shape[1:]\n\n assert torch.equal(wrapper(x_normal), ref_out)\n\n\n@patch('torch.__version__', torch_version)\n@pytest.mark.parametrize(\n 'in_w,in_h,in_t,in_channel,out_channel,kernel_size,stride,padding,dilation', # noqa: E501\n [(10, 10, 10, 1, 1, 3, 1, 0, 1), (20, 20, 20, 3, 3, 5, 2, 1, 2)])\n@pytest.mark.skipif(\n torch.__version__ == 'parrots' and not torch.cuda.is_available(),\n reason='parrots requires CUDA support')\ndef test_max_pool_3d(in_w, in_h, in_t, in_channel, out_channel, kernel_size,\n stride, padding, dilation):\n # wrapper op with 0-dim input\n x_empty = torch.randn(0, in_channel, in_t, in_h, in_w, requires_grad=True)\n wrapper = MaxPool3d(\n kernel_size, stride=stride, padding=padding, dilation=dilation)\n if torch.__version__ == 'parrots':\n x_empty = x_empty.cuda()\n wrapper_out = wrapper(x_empty)\n # torch op with 3-dim input as shape reference\n x_normal = torch.randn(3, in_channel, in_t, in_h, in_w)\n ref = nn.MaxPool3d(\n kernel_size, stride=stride, padding=padding, dilation=dilation)\n if torch.__version__ == 'parrots':\n x_normal = x_normal.cuda()\n ref_out = ref(x_normal)\n\n assert wrapper_out.shape[0] == 0\n assert wrapper_out.shape[1:] == ref_out.shape[1:]\n\n assert torch.equal(wrapper(x_normal), ref_out)\n\n\n@patch('torch.__version__', torch_version)\n@pytest.mark.parametrize('in_w,in_h,in_feature,out_feature', [(10, 10, 1, 1),\n (20, 20, 3, 3)])\ndef test_linear(in_w, in_h, in_feature, out_feature):\n # wrapper op with 0-dim input\n x_empty = torch.randn(0, in_feature, requires_grad=True)\n torch.manual_seed(0)\n wrapper = Linear(in_feature, out_feature)\n wrapper_out = wrapper(x_empty)\n\n # torch op with 3-dim input as shape reference\n x_normal = torch.randn(3, in_feature)\n torch.manual_seed(0)\n ref = nn.Linear(in_feature, out_feature)\n ref_out = ref(x_normal)\n\n assert wrapper_out.shape[0] == 0\n assert wrapper_out.shape[1:] == ref_out.shape[1:]\n\n wrapper_out.sum().backward()\n assert wrapper.weight.grad is not None\n assert wrapper.weight.grad.shape == wrapper.weight.shape\n\n assert torch.equal(wrapper(x_normal), ref_out)\n\n # eval mode\n x_empty = torch.randn(0, in_feature)\n wrapper = Linear(in_feature, out_feature)\n wrapper.eval()\n wrapper(x_empty)\n\n\n@patch('mmcv.cnn.bricks.wrappers.TORCH_VERSION', (1, 10))\ndef test_nn_op_forward_called():\n\n for m in ['Conv2d', 'ConvTranspose2d', 'MaxPool2d']:\n with patch(f'torch.nn.{m}.forward') as nn_module_forward:\n # randn input\n x_empty = torch.randn(0, 3, 10, 10)\n wrapper = eval(m)(3, 2, 1)\n wrapper(x_empty)\n nn_module_forward.assert_called_with(x_empty)\n\n # non-randn input\n x_normal = torch.randn(1, 3, 10, 10)\n wrapper = eval(m)(3, 2, 1)\n wrapper(x_normal)\n nn_module_forward.assert_called_with(x_normal)\n\n for m in ['Conv3d', 'ConvTranspose3d', 'MaxPool3d']:\n with patch(f'torch.nn.{m}.forward') as nn_module_forward:\n # randn input\n x_empty = torch.randn(0, 3, 10, 10, 10)\n wrapper = eval(m)(3, 2, 1)\n wrapper(x_empty)\n nn_module_forward.assert_called_with(x_empty)\n\n # non-randn input\n x_normal = torch.randn(1, 3, 10, 10, 10)\n wrapper = eval(m)(3, 2, 1)\n wrapper(x_normal)\n nn_module_forward.assert_called_with(x_normal)\n\n with patch('torch.nn.Linear.forward') as nn_module_forward:\n # randn input\n x_empty = torch.randn(0, 3)\n wrapper = Linear(3, 3)\n wrapper(x_empty)\n nn_module_forward.assert_called_with(x_empty)\n\n # non-randn input\n x_normal = torch.randn(1, 3)\n wrapper = Linear(3, 3)\n wrapper(x_normal)\n nn_module_forward.assert_called_with(x_normal)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_fileclient.py", "content": "import os\nimport os.path as osp\nimport sys\nimport tempfile\nfrom contextlib import contextmanager\nfrom copy import deepcopy\nfrom pathlib import Path\nfrom unittest.mock import MagicMock, patch\n\nimport pytest\n\nimport mmcv\nfrom mmcv import BaseStorageBackend, FileClient\nfrom mmcv.utils import has_method\n\nsys.modules['ceph'] = MagicMock()\nsys.modules['petrel_client'] = MagicMock()\nsys.modules['petrel_client.client'] = MagicMock()\nsys.modules['mc'] = MagicMock()\n\n\n@contextmanager\ndef build_temporary_directory():\n \"\"\"Build a temporary directory containing many files to test\n ``FileClient.list_dir_or_file``.\n\n . \\n\n | -- dir1 \\n\n | -- | -- text3.txt \\n\n | -- dir2 \\n\n | -- | -- dir3 \\n\n | -- | -- | -- text4.txt \\n\n | -- | -- img.jpg \\n\n | -- text1.txt \\n\n | -- text2.txt \\n\n \"\"\"\n with tempfile.TemporaryDirectory() as tmp_dir:\n text1 = Path(tmp_dir) / 'text1.txt'\n text1.open('w').write('text1')\n text2 = Path(tmp_dir) / 'text2.txt'\n text2.open('w').write('text2')\n dir1 = Path(tmp_dir) / 'dir1'\n dir1.mkdir()\n text3 = dir1 / 'text3.txt'\n text3.open('w').write('text3')\n dir2 = Path(tmp_dir) / 'dir2'\n dir2.mkdir()\n jpg1 = dir2 / 'img.jpg'\n jpg1.open('wb').write(b'img')\n dir3 = dir2 / 'dir3'\n dir3.mkdir()\n text4 = dir3 / 'text4.txt'\n text4.open('w').write('text4')\n yield tmp_dir\n\n\n@contextmanager\ndef delete_and_reset_method(obj, method):\n method_obj = deepcopy(getattr(type(obj), method))\n try:\n delattr(type(obj), method)\n yield\n finally:\n setattr(type(obj), method, method_obj)\n\n\nclass MockS3Client:\n\n def __init__(self, enable_mc=True):\n self.enable_mc = enable_mc\n\n def Get(self, filepath):\n with open(filepath, 'rb') as f:\n content = f.read()\n return content\n\n\nclass MockPetrelClient:\n\n def __init__(self, enable_mc=True, enable_multi_cluster=False):\n self.enable_mc = enable_mc\n self.enable_multi_cluster = enable_multi_cluster\n\n def Get(self, filepath):\n with open(filepath, 'rb') as f:\n content = f.read()\n return content\n\n def put(self):\n pass\n\n def delete(self):\n pass\n\n def contains(self):\n pass\n\n def isdir(self):\n pass\n\n def list(self, dir_path):\n for entry in os.scandir(dir_path):\n if not entry.name.startswith('.') and entry.is_file():\n yield entry.name\n elif osp.isdir(entry.path):\n yield entry.name + '/'\n\n\nclass MockMemcachedClient:\n\n def __init__(self, server_list_cfg, client_cfg):\n pass\n\n def Get(self, filepath, buffer):\n with open(filepath, 'rb') as f:\n buffer.content = f.read()\n\n\nclass TestFileClient:\n\n @classmethod\n def setup_class(cls):\n cls.test_data_dir = Path(__file__).parent / 'data'\n cls.img_path = cls.test_data_dir / 'color.jpg'\n cls.img_shape = (300, 400, 3)\n cls.text_path = cls.test_data_dir / 'filelist.txt'\n\n def test_error(self):\n with pytest.raises(ValueError):\n FileClient('hadoop')\n\n def test_disk_backend(self):\n disk_backend = FileClient('disk')\n\n # test `name` attribute\n assert disk_backend.name == 'HardDiskBackend'\n # test `allow_symlink` attribute\n assert disk_backend.allow_symlink\n # test `get`\n # input path is Path object\n img_bytes = disk_backend.get(self.img_path)\n img = mmcv.imfrombytes(img_bytes)\n assert self.img_path.open('rb').read() == img_bytes\n assert img.shape == self.img_shape\n # input path is str\n img_bytes = disk_backend.get(str(self.img_path))\n img = mmcv.imfrombytes(img_bytes)\n assert self.img_path.open('rb').read() == img_bytes\n assert img.shape == self.img_shape\n\n # test `get_text`\n # input path is Path object\n value_buf = disk_backend.get_text(self.text_path)\n assert self.text_path.open('r').read() == value_buf\n # input path is str\n value_buf = disk_backend.get_text(str(self.text_path))\n assert self.text_path.open('r').read() == value_buf\n\n with tempfile.TemporaryDirectory() as tmp_dir:\n # test `put`\n filepath1 = Path(tmp_dir) / 'test.jpg'\n disk_backend.put(b'disk', filepath1)\n assert filepath1.open('rb').read() == b'disk'\n # test the `mkdir_or_exist` behavior in `put`\n _filepath1 = Path(tmp_dir) / 'not_existed_dir1' / 'test.jpg'\n disk_backend.put(b'disk', _filepath1)\n assert _filepath1.open('rb').read() == b'disk'\n\n # test `put_text`\n filepath2 = Path(tmp_dir) / 'test.txt'\n disk_backend.put_text('disk', filepath2)\n assert filepath2.open('r').read() == 'disk'\n # test the `mkdir_or_exist` behavior in `put_text`\n _filepath2 = Path(tmp_dir) / 'not_existed_dir2' / 'test.txt'\n disk_backend.put_text('disk', _filepath2)\n assert _filepath2.open('r').read() == 'disk'\n\n # test `isfile`\n assert disk_backend.isfile(filepath2)\n assert not disk_backend.isfile(Path(tmp_dir) / 'not/existed/path')\n\n # test `remove`\n disk_backend.remove(filepath2)\n\n # test `exists`\n assert not disk_backend.exists(filepath2)\n\n # test `get_local_path`\n # if the backend is disk, `get_local_path` just return the input\n with disk_backend.get_local_path(filepath1) as path:\n assert str(filepath1) == path\n assert osp.isfile(filepath1)\n\n # test `join_path`\n disk_dir = '/path/of/your/directory'\n assert disk_backend.join_path(disk_dir, 'file') == \\\n osp.join(disk_dir, 'file')\n assert disk_backend.join_path(disk_dir, 'dir', 'file') == \\\n osp.join(disk_dir, 'dir', 'file')\n\n # test `list_dir_or_file`\n with build_temporary_directory() as tmp_dir:\n # 1. list directories and files\n assert set(disk_backend.list_dir_or_file(tmp_dir)) == set(\n ['dir1', 'dir2', 'text1.txt', 'text2.txt'])\n # 2. list directories and files recursively\n assert set(disk_backend.list_dir_or_file(\n tmp_dir, recursive=True)) == set([\n 'dir1',\n osp.join('dir1', 'text3.txt'), 'dir2',\n osp.join('dir2', 'dir3'),\n osp.join('dir2', 'dir3', 'text4.txt'),\n osp.join('dir2', 'img.jpg'), 'text1.txt', 'text2.txt'\n ])\n # 3. only list directories\n assert set(\n disk_backend.list_dir_or_file(\n tmp_dir, list_file=False)) == set(['dir1', 'dir2'])\n with pytest.raises(\n TypeError,\n match='`suffix` should be None when `list_dir` is True'):\n # Exception is raised among the `list_dir_or_file` of client,\n # so we need to invode the client to trigger the exception\n disk_backend.client.list_dir_or_file(\n tmp_dir, list_file=False, suffix='.txt')\n # 4. only list directories recursively\n assert set(\n disk_backend.list_dir_or_file(\n tmp_dir, list_file=False, recursive=True)) == set(\n ['dir1', 'dir2',\n osp.join('dir2', 'dir3')])\n # 5. only list files\n assert set(disk_backend.list_dir_or_file(\n tmp_dir, list_dir=False)) == set(['text1.txt', 'text2.txt'])\n # 6. only list files recursively\n assert set(\n disk_backend.list_dir_or_file(\n tmp_dir, list_dir=False, recursive=True)) == set([\n osp.join('dir1', 'text3.txt'),\n osp.join('dir2', 'dir3', 'text4.txt'),\n osp.join('dir2', 'img.jpg'), 'text1.txt', 'text2.txt'\n ])\n # 7. only list files ending with suffix\n assert set(\n disk_backend.list_dir_or_file(\n tmp_dir, list_dir=False,\n suffix='.txt')) == set(['text1.txt', 'text2.txt'])\n assert set(\n disk_backend.list_dir_or_file(\n tmp_dir, list_dir=False,\n suffix=('.txt',\n '.jpg'))) == set(['text1.txt', 'text2.txt'])\n with pytest.raises(\n TypeError,\n match='`suffix` must be a string or tuple of strings'):\n disk_backend.client.list_dir_or_file(\n tmp_dir, list_dir=False, suffix=['.txt', '.jpg'])\n # 8. only list files ending with suffix recursively\n assert set(\n disk_backend.list_dir_or_file(\n tmp_dir, list_dir=False, suffix='.txt',\n recursive=True)) == set([\n osp.join('dir1', 'text3.txt'),\n osp.join('dir2', 'dir3', 'text4.txt'), 'text1.txt',\n 'text2.txt'\n ])\n # 7. only list files ending with suffix\n assert set(\n disk_backend.list_dir_or_file(\n tmp_dir,\n list_dir=False,\n suffix=('.txt', '.jpg'),\n recursive=True)) == set([\n osp.join('dir1', 'text3.txt'),\n osp.join('dir2', 'dir3', 'text4.txt'),\n osp.join('dir2', 'img.jpg'), 'text1.txt', 'text2.txt'\n ])\n\n @patch('ceph.S3Client', MockS3Client)\n def test_ceph_backend(self):\n ceph_backend = FileClient('ceph')\n\n # test `allow_symlink` attribute\n assert not ceph_backend.allow_symlink\n\n # input path is Path object\n with pytest.raises(NotImplementedError):\n ceph_backend.get_text(self.text_path)\n # input path is str\n with pytest.raises(NotImplementedError):\n ceph_backend.get_text(str(self.text_path))\n\n # input path is Path object\n img_bytes = ceph_backend.get(self.img_path)\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == self.img_shape\n # input path is str\n img_bytes = ceph_backend.get(str(self.img_path))\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == self.img_shape\n\n # `path_mapping` is either None or dict\n with pytest.raises(AssertionError):\n FileClient('ceph', path_mapping=1)\n # test `path_mapping`\n ceph_path = 's3://user/data'\n ceph_backend = FileClient(\n 'ceph', path_mapping={str(self.test_data_dir): ceph_path})\n ceph_backend.client._client.Get = MagicMock(\n return_value=ceph_backend.client._client.Get(self.img_path))\n img_bytes = ceph_backend.get(self.img_path)\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == self.img_shape\n ceph_backend.client._client.Get.assert_called_with(\n str(self.img_path).replace(str(self.test_data_dir), ceph_path))\n\n @patch('petrel_client.client.Client', MockPetrelClient)\n @pytest.mark.parametrize('backend,prefix', [('petrel', None),\n (None, 's3')])\n def test_petrel_backend(self, backend, prefix):\n petrel_backend = FileClient(backend=backend, prefix=prefix)\n\n # test `allow_symlink` attribute\n assert not petrel_backend.allow_symlink\n\n # input path is Path object\n img_bytes = petrel_backend.get(self.img_path)\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == self.img_shape\n # input path is str\n img_bytes = petrel_backend.get(str(self.img_path))\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == self.img_shape\n\n # `path_mapping` is either None or dict\n with pytest.raises(AssertionError):\n FileClient('petrel', path_mapping=1)\n\n # test `_map_path`\n petrel_dir = 's3://user/data'\n petrel_backend = FileClient(\n 'petrel', path_mapping={str(self.test_data_dir): petrel_dir})\n assert petrel_backend.client._map_path(str(self.img_path)) == \\\n str(self.img_path).replace(str(self.test_data_dir), petrel_dir)\n\n petrel_path = f'{petrel_dir}/test.jpg'\n petrel_backend = FileClient('petrel')\n\n # test `_format_path`\n assert petrel_backend.client._format_path('s3://user\\\\data\\\\test.jpg')\\\n == petrel_path\n\n # test `get`\n with patch.object(\n petrel_backend.client._client, 'Get',\n return_value=b'petrel') as mock_get:\n assert petrel_backend.get(petrel_path) == b'petrel'\n mock_get.assert_called_once_with(petrel_path)\n\n # test `get_text`\n with patch.object(\n petrel_backend.client._client, 'Get',\n return_value=b'petrel') as mock_get:\n assert petrel_backend.get_text(petrel_path) == 'petrel'\n mock_get.assert_called_once_with(petrel_path)\n\n # test `put`\n with patch.object(petrel_backend.client._client, 'put') as mock_put:\n petrel_backend.put(b'petrel', petrel_path)\n mock_put.assert_called_once_with(petrel_path, b'petrel')\n\n # test `put_text`\n with patch.object(petrel_backend.client._client, 'put') as mock_put:\n petrel_backend.put_text('petrel', petrel_path)\n mock_put.assert_called_once_with(petrel_path, b'petrel')\n\n # test `remove`\n assert has_method(petrel_backend.client._client, 'delete')\n # raise Exception if `delete` is not implemented\n with delete_and_reset_method(petrel_backend.client._client, 'delete'):\n assert not has_method(petrel_backend.client._client, 'delete')\n with pytest.raises(NotImplementedError):\n petrel_backend.remove(petrel_path)\n\n with patch.object(petrel_backend.client._client,\n 'delete') as mock_delete:\n petrel_backend.remove(petrel_path)\n mock_delete.assert_called_once_with(petrel_path)\n\n # test `exists`\n assert has_method(petrel_backend.client._client, 'contains')\n assert has_method(petrel_backend.client._client, 'isdir')\n # raise Exception if `delete` is not implemented\n with delete_and_reset_method(petrel_backend.client._client,\n 'contains'), delete_and_reset_method(\n petrel_backend.client._client,\n 'isdir'):\n assert not has_method(petrel_backend.client._client, 'contains')\n assert not has_method(petrel_backend.client._client, 'isdir')\n with pytest.raises(NotImplementedError):\n petrel_backend.exists(petrel_path)\n\n with patch.object(\n petrel_backend.client._client, 'contains',\n return_value=True) as mock_contains:\n assert petrel_backend.exists(petrel_path)\n mock_contains.assert_called_once_with(petrel_path)\n\n # test `isdir`\n assert has_method(petrel_backend.client._client, 'isdir')\n with delete_and_reset_method(petrel_backend.client._client, 'isdir'):\n assert not has_method(petrel_backend.client._client, 'isdir')\n with pytest.raises(NotImplementedError):\n petrel_backend.isdir(petrel_path)\n\n with patch.object(\n petrel_backend.client._client, 'isdir',\n return_value=True) as mock_isdir:\n assert petrel_backend.isdir(petrel_dir)\n mock_isdir.assert_called_once_with(petrel_dir)\n\n # test `isfile`\n assert has_method(petrel_backend.client._client, 'contains')\n with delete_and_reset_method(petrel_backend.client._client,\n 'contains'):\n assert not has_method(petrel_backend.client._client, 'contains')\n with pytest.raises(NotImplementedError):\n petrel_backend.isfile(petrel_path)\n\n with patch.object(\n petrel_backend.client._client, 'contains',\n return_value=True) as mock_contains:\n assert petrel_backend.isfile(petrel_path)\n mock_contains.assert_called_once_with(petrel_path)\n\n # test `join_path`\n assert petrel_backend.join_path(petrel_dir, 'file') == \\\n f'{petrel_dir}/file'\n assert petrel_backend.join_path(f'{petrel_dir}/', 'file') == \\\n f'{petrel_dir}/file'\n assert petrel_backend.join_path(petrel_dir, 'dir', 'file') == \\\n f'{petrel_dir}/dir/file'\n\n # test `get_local_path`\n with patch.object(petrel_backend.client._client, 'Get',\n return_value=b'petrel') as mock_get, \\\n patch.object(petrel_backend.client._client, 'contains',\n return_value=True) as mock_contains:\n with petrel_backend.get_local_path(petrel_path) as path:\n assert Path(path).open('rb').read() == b'petrel'\n # exist the with block and path will be released\n assert not osp.isfile(path)\n mock_get.assert_called_once_with(petrel_path)\n mock_contains.assert_called_once_with(petrel_path)\n\n # test `list_dir_or_file`\n assert has_method(petrel_backend.client._client, 'list')\n with delete_and_reset_method(petrel_backend.client._client, 'list'):\n assert not has_method(petrel_backend.client._client, 'list')\n with pytest.raises(NotImplementedError):\n list(petrel_backend.list_dir_or_file(petrel_dir))\n\n with build_temporary_directory() as tmp_dir:\n # 1. list directories and files\n assert set(petrel_backend.list_dir_or_file(tmp_dir)) == set(\n ['dir1', 'dir2', 'text1.txt', 'text2.txt'])\n # 2. list directories and files recursively\n assert set(\n petrel_backend.list_dir_or_file(\n tmp_dir, recursive=True)) == set([\n 'dir1', '/'.join(('dir1', 'text3.txt')), 'dir2',\n '/'.join(('dir2', 'dir3')), '/'.join(\n ('dir2', 'dir3', 'text4.txt')), '/'.join(\n ('dir2', 'img.jpg')), 'text1.txt', 'text2.txt'\n ])\n # 3. only list directories\n assert set(\n petrel_backend.list_dir_or_file(\n tmp_dir, list_file=False)) == set(['dir1', 'dir2'])\n with pytest.raises(\n TypeError,\n match=('`list_dir` should be False when `suffix` is not '\n 'None')):\n # Exception is raised among the `list_dir_or_file` of client,\n # so we need to invode the client to trigger the exception\n petrel_backend.client.list_dir_or_file(\n tmp_dir, list_file=False, suffix='.txt')\n # 4. only list directories recursively\n assert set(\n petrel_backend.list_dir_or_file(\n tmp_dir, list_file=False, recursive=True)) == set(\n ['dir1', 'dir2', '/'.join(('dir2', 'dir3'))])\n # 5. only list files\n assert set(\n petrel_backend.list_dir_or_file(tmp_dir,\n list_dir=False)) == set(\n ['text1.txt', 'text2.txt'])\n # 6. only list files recursively\n assert set(\n petrel_backend.list_dir_or_file(\n tmp_dir, list_dir=False, recursive=True)) == set([\n '/'.join(('dir1', 'text3.txt')), '/'.join(\n ('dir2', 'dir3', 'text4.txt')), '/'.join(\n ('dir2', 'img.jpg')), 'text1.txt', 'text2.txt'\n ])\n # 7. only list files ending with suffix\n assert set(\n petrel_backend.list_dir_or_file(\n tmp_dir, list_dir=False,\n suffix='.txt')) == set(['text1.txt', 'text2.txt'])\n assert set(\n petrel_backend.list_dir_or_file(\n tmp_dir, list_dir=False,\n suffix=('.txt',\n '.jpg'))) == set(['text1.txt', 'text2.txt'])\n with pytest.raises(\n TypeError,\n match='`suffix` must be a string or tuple of strings'):\n petrel_backend.client.list_dir_or_file(\n tmp_dir, list_dir=False, suffix=['.txt', '.jpg'])\n # 8. only list files ending with suffix recursively\n assert set(\n petrel_backend.list_dir_or_file(\n tmp_dir, list_dir=False, suffix='.txt',\n recursive=True)) == set([\n '/'.join(('dir1', 'text3.txt')), '/'.join(\n ('dir2', 'dir3', 'text4.txt')), 'text1.txt',\n 'text2.txt'\n ])\n # 7. only list files ending with suffix\n assert set(\n petrel_backend.list_dir_or_file(\n tmp_dir,\n list_dir=False,\n suffix=('.txt', '.jpg'),\n recursive=True)) == set([\n '/'.join(('dir1', 'text3.txt')), '/'.join(\n ('dir2', 'dir3', 'text4.txt')), '/'.join(\n ('dir2', 'img.jpg')), 'text1.txt', 'text2.txt'\n ])\n\n @patch('mc.MemcachedClient.GetInstance', MockMemcachedClient)\n @patch('mc.pyvector', MagicMock)\n @patch('mc.ConvertBuffer', lambda x: x.content)\n def test_memcached_backend(self):\n mc_cfg = dict(server_list_cfg='', client_cfg='', sys_path=None)\n mc_backend = FileClient('memcached', **mc_cfg)\n\n # test `allow_symlink` attribute\n assert not mc_backend.allow_symlink\n\n # input path is Path object\n with pytest.raises(NotImplementedError):\n mc_backend.get_text(self.text_path)\n # input path is str\n with pytest.raises(NotImplementedError):\n mc_backend.get_text(str(self.text_path))\n\n # input path is Path object\n img_bytes = mc_backend.get(self.img_path)\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == self.img_shape\n # input path is str\n img_bytes = mc_backend.get(str(self.img_path))\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == self.img_shape\n\n def test_lmdb_backend(self):\n lmdb_path = self.test_data_dir / 'demo.lmdb'\n\n # db_path is Path object\n lmdb_backend = FileClient('lmdb', db_path=lmdb_path)\n\n # test `allow_symlink` attribute\n assert not lmdb_backend.allow_symlink\n\n with pytest.raises(NotImplementedError):\n lmdb_backend.get_text(self.text_path)\n\n img_bytes = lmdb_backend.get('baboon')\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == (120, 125, 3)\n\n # db_path is str\n lmdb_backend = FileClient('lmdb', db_path=str(lmdb_path))\n with pytest.raises(NotImplementedError):\n lmdb_backend.get_text(str(self.text_path))\n img_bytes = lmdb_backend.get('baboon')\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == (120, 125, 3)\n\n @pytest.mark.parametrize('backend,prefix', [('http', None),\n (None, 'http')])\n def test_http_backend(self, backend, prefix):\n http_backend = FileClient(backend=backend, prefix=prefix)\n img_url = 'https://raw.githubusercontent.com/open-mmlab/mmcv/' \\\n 'master/tests/data/color.jpg'\n text_url = 'https://raw.githubusercontent.com/open-mmlab/mmcv/' \\\n 'master/tests/data/filelist.txt'\n\n # test `allow_symlink` attribute\n assert not http_backend.allow_symlink\n\n # input is path or Path object\n with pytest.raises(Exception):\n http_backend.get(self.img_path)\n with pytest.raises(Exception):\n http_backend.get(str(self.img_path))\n with pytest.raises(Exception):\n http_backend.get_text(self.text_path)\n with pytest.raises(Exception):\n http_backend.get_text(str(self.text_path))\n\n # input url is http image\n img_bytes = http_backend.get(img_url)\n img = mmcv.imfrombytes(img_bytes)\n assert img.shape == self.img_shape\n\n # input url is http text\n value_buf = http_backend.get_text(text_url)\n assert self.text_path.open('r').read() == value_buf\n\n # test `_get_local_path`\n # exist the with block and path will be released\n with http_backend.get_local_path(img_url) as path:\n assert mmcv.imread(path).shape == self.img_shape\n assert not osp.isfile(path)\n\n def test_new_magic_method(self):\n\n class DummyBackend1(BaseStorageBackend):\n\n def get(self, filepath):\n return filepath\n\n def get_text(self, filepath, encoding='utf-8'):\n return filepath\n\n FileClient.register_backend('dummy_backend', DummyBackend1)\n client1 = FileClient(backend='dummy_backend')\n client2 = FileClient(backend='dummy_backend')\n assert client1 is client2\n\n # if a backend is overwrote, it will disable the singleton pattern for\n # the backend\n class DummyBackend2(BaseStorageBackend):\n\n def get(self, filepath):\n pass\n\n def get_text(self, filepath):\n pass\n\n FileClient.register_backend('dummy_backend', DummyBackend2, force=True)\n client3 = FileClient(backend='dummy_backend')\n client4 = FileClient(backend='dummy_backend')\n assert client3 is not client4\n\n def test_parse_uri_prefix(self):\n # input path is None\n with pytest.raises(AssertionError):\n FileClient.parse_uri_prefix(None)\n # input path is list\n with pytest.raises(AssertionError):\n FileClient.parse_uri_prefix([])\n\n # input path is Path object\n assert FileClient.parse_uri_prefix(self.img_path) is None\n # input path is str\n assert FileClient.parse_uri_prefix(str(self.img_path)) is None\n\n # input path starts with https\n img_url = 'https://raw.githubusercontent.com/open-mmlab/mmcv/' \\\n 'master/tests/data/color.jpg'\n assert FileClient.parse_uri_prefix(img_url) == 'https'\n\n # input path starts with s3\n img_url = 's3://your_bucket/img.png'\n assert FileClient.parse_uri_prefix(img_url) == 's3'\n\n # input path starts with clusterName:s3\n img_url = 'clusterName:s3://your_bucket/img.png'\n assert FileClient.parse_uri_prefix(img_url) == 's3'\n\n def test_infer_client(self):\n # HardDiskBackend\n file_client_args = {'backend': 'disk'}\n client = FileClient.infer_client(file_client_args)\n assert client.name == 'HardDiskBackend'\n client = FileClient.infer_client(uri=self.img_path)\n assert client.name == 'HardDiskBackend'\n\n # PetrelBackend\n file_client_args = {'backend': 'petrel'}\n client = FileClient.infer_client(file_client_args)\n assert client.name == 'PetrelBackend'\n uri = 's3://user_data'\n client = FileClient.infer_client(uri=uri)\n assert client.name == 'PetrelBackend'\n\n def test_register_backend(self):\n\n # name must be a string\n with pytest.raises(TypeError):\n\n class TestClass1:\n pass\n\n FileClient.register_backend(1, TestClass1)\n\n # module must be a class\n with pytest.raises(TypeError):\n FileClient.register_backend('int', 0)\n\n # module must be a subclass of BaseStorageBackend\n with pytest.raises(TypeError):\n\n class TestClass1:\n pass\n\n FileClient.register_backend('TestClass1', TestClass1)\n\n class ExampleBackend(BaseStorageBackend):\n\n def get(self, filepath):\n return filepath\n\n def get_text(self, filepath, encoding='utf-8'):\n return filepath\n\n FileClient.register_backend('example', ExampleBackend)\n example_backend = FileClient('example')\n assert example_backend.get(self.img_path) == self.img_path\n assert example_backend.get_text(self.text_path) == self.text_path\n assert 'example' in FileClient._backends\n\n class Example2Backend(BaseStorageBackend):\n\n def get(self, filepath):\n return b'bytes2'\n\n def get_text(self, filepath, encoding='utf-8'):\n return 'text2'\n\n # force=False\n with pytest.raises(KeyError):\n FileClient.register_backend('example', Example2Backend)\n\n FileClient.register_backend('example', Example2Backend, force=True)\n example_backend = FileClient('example')\n assert example_backend.get(self.img_path) == b'bytes2'\n assert example_backend.get_text(self.text_path) == 'text2'\n\n @FileClient.register_backend(name='example3')\n class Example3Backend(BaseStorageBackend):\n\n def get(self, filepath):\n return b'bytes3'\n\n def get_text(self, filepath, encoding='utf-8'):\n return 'text3'\n\n example_backend = FileClient('example3')\n assert example_backend.get(self.img_path) == b'bytes3'\n assert example_backend.get_text(self.text_path) == 'text3'\n assert 'example3' in FileClient._backends\n\n # force=False\n with pytest.raises(KeyError):\n\n @FileClient.register_backend(name='example3')\n class Example4Backend(BaseStorageBackend):\n\n def get(self, filepath):\n return b'bytes4'\n\n def get_text(self, filepath, encoding='utf-8'):\n return 'text4'\n\n @FileClient.register_backend(name='example3', force=True)\n class Example5Backend(BaseStorageBackend):\n\n def get(self, filepath):\n return b'bytes5'\n\n def get_text(self, filepath, encoding='utf-8'):\n return 'text5'\n\n example_backend = FileClient('example3')\n assert example_backend.get(self.img_path) == b'bytes5'\n assert example_backend.get_text(self.text_path) == 'text5'\n\n # prefixes is a str\n class Example6Backend(BaseStorageBackend):\n\n def get(self, filepath):\n return b'bytes6'\n\n def get_text(self, filepath, encoding='utf-8'):\n return 'text6'\n\n FileClient.register_backend(\n 'example4',\n Example6Backend,\n force=True,\n prefixes='example4_prefix')\n example_backend = FileClient('example4')\n assert example_backend.get(self.img_path) == b'bytes6'\n assert example_backend.get_text(self.text_path) == 'text6'\n example_backend = FileClient(prefix='example4_prefix')\n assert example_backend.get(self.img_path) == b'bytes6'\n assert example_backend.get_text(self.text_path) == 'text6'\n example_backend = FileClient('example4', prefix='example4_prefix')\n assert example_backend.get(self.img_path) == b'bytes6'\n assert example_backend.get_text(self.text_path) == 'text6'\n\n # prefixes is a list of str\n class Example7Backend(BaseStorageBackend):\n\n def get(self, filepath):\n return b'bytes7'\n\n def get_text(self, filepath, encoding='utf-8'):\n return 'text7'\n\n FileClient.register_backend(\n 'example5',\n Example7Backend,\n force=True,\n prefixes=['example5_prefix1', 'example5_prefix2'])\n example_backend = FileClient('example5')\n assert example_backend.get(self.img_path) == b'bytes7'\n assert example_backend.get_text(self.text_path) == 'text7'\n example_backend = FileClient(prefix='example5_prefix1')\n assert example_backend.get(self.img_path) == b'bytes7'\n assert example_backend.get_text(self.text_path) == 'text7'\n example_backend = FileClient(prefix='example5_prefix2')\n assert example_backend.get(self.img_path) == b'bytes7'\n assert example_backend.get_text(self.text_path) == 'text7'\n\n # backend has a higher priority than prefixes\n class Example8Backend(BaseStorageBackend):\n\n def get(self, filepath):\n return b'bytes8'\n\n def get_text(self, filepath, encoding='utf-8'):\n return 'text8'\n\n FileClient.register_backend(\n 'example6',\n Example8Backend,\n force=True,\n prefixes='example6_prefix')\n example_backend = FileClient('example6')\n assert example_backend.get(self.img_path) == b'bytes8'\n assert example_backend.get_text(self.text_path) == 'text8'\n example_backend = FileClient('example6', prefix='example4_prefix')\n assert example_backend.get(self.img_path) == b'bytes8'\n assert example_backend.get_text(self.text_path) == 'text8'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_fileio.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport os.path as osp\nimport sys\nimport tempfile\nfrom unittest.mock import MagicMock, patch\n\nimport pytest\n\nimport mmcv\nfrom mmcv.fileio.file_client import HTTPBackend, PetrelBackend\n\nsys.modules['petrel_client'] = MagicMock()\nsys.modules['petrel_client.client'] = MagicMock()\n\n\ndef _test_handler(file_format, test_obj, str_checker, mode='r+'):\n # dump to a string\n dump_str = mmcv.dump(test_obj, file_format=file_format)\n str_checker(dump_str)\n\n # load/dump with filenames from disk\n tmp_filename = osp.join(tempfile.gettempdir(), 'mmcv_test_dump')\n mmcv.dump(test_obj, tmp_filename, file_format=file_format)\n assert osp.isfile(tmp_filename)\n load_obj = mmcv.load(tmp_filename, file_format=file_format)\n assert load_obj == test_obj\n os.remove(tmp_filename)\n\n # load/dump with filename from petrel\n method = 'put' if 'b' in mode else 'put_text'\n with patch.object(PetrelBackend, method, return_value=None) as mock_method:\n filename = 's3://path/of/your/file'\n mmcv.dump(test_obj, filename, file_format=file_format)\n mock_method.assert_called()\n\n # json load/dump with a file-like object\n with tempfile.NamedTemporaryFile(mode, delete=False) as f:\n tmp_filename = f.name\n mmcv.dump(test_obj, f, file_format=file_format)\n assert osp.isfile(tmp_filename)\n with open(tmp_filename, mode) as f:\n load_obj = mmcv.load(f, file_format=file_format)\n assert load_obj == test_obj\n os.remove(tmp_filename)\n\n # automatically inference the file format from the given filename\n tmp_filename = osp.join(tempfile.gettempdir(),\n 'mmcv_test_dump.' + file_format)\n mmcv.dump(test_obj, tmp_filename)\n assert osp.isfile(tmp_filename)\n load_obj = mmcv.load(tmp_filename)\n assert load_obj == test_obj\n os.remove(tmp_filename)\n\n\nobj_for_test = [{'a': 'abc', 'b': 1}, 2, 'c']\n\n\ndef test_json():\n\n def json_checker(dump_str):\n assert dump_str in [\n '[{\"a\": \"abc\", \"b\": 1}, 2, \"c\"]', '[{\"b\": 1, \"a\": \"abc\"}, 2, \"c\"]'\n ]\n\n _test_handler('json', obj_for_test, json_checker)\n\n\ndef test_yaml():\n\n def yaml_checker(dump_str):\n assert dump_str in [\n '- {a: abc, b: 1}\\n- 2\\n- c\\n', '- {b: 1, a: abc}\\n- 2\\n- c\\n',\n '- a: abc\\n b: 1\\n- 2\\n- c\\n', '- b: 1\\n a: abc\\n- 2\\n- c\\n'\n ]\n\n _test_handler('yaml', obj_for_test, yaml_checker)\n\n\ndef test_pickle():\n\n def pickle_checker(dump_str):\n import pickle\n assert pickle.loads(dump_str) == obj_for_test\n\n _test_handler('pickle', obj_for_test, pickle_checker, mode='rb+')\n\n\ndef test_exception():\n test_obj = [{'a': 'abc', 'b': 1}, 2, 'c']\n\n with pytest.raises(ValueError):\n mmcv.dump(test_obj)\n\n with pytest.raises(TypeError):\n mmcv.dump(test_obj, 'tmp.txt')\n\n\ndef test_register_handler():\n\n @mmcv.register_handler('txt')\n class TxtHandler1(mmcv.BaseFileHandler):\n\n def load_from_fileobj(self, file):\n return file.read()\n\n def dump_to_fileobj(self, obj, file):\n file.write(str(obj))\n\n def dump_to_str(self, obj, **kwargs):\n return str(obj)\n\n @mmcv.register_handler(['txt1', 'txt2'])\n class TxtHandler2(mmcv.BaseFileHandler):\n\n def load_from_fileobj(self, file):\n return file.read()\n\n def dump_to_fileobj(self, obj, file):\n file.write('\\n')\n file.write(str(obj))\n\n def dump_to_str(self, obj, **kwargs):\n return str(obj)\n\n content = mmcv.load(osp.join(osp.dirname(__file__), 'data/filelist.txt'))\n assert content == '1.jpg\\n2.jpg\\n3.jpg\\n4.jpg\\n5.jpg'\n tmp_filename = osp.join(tempfile.gettempdir(), 'mmcv_test.txt2')\n mmcv.dump(content, tmp_filename)\n with open(tmp_filename, 'r') as f:\n written = f.read()\n os.remove(tmp_filename)\n assert written == '\\n' + content\n\n\ndef test_list_from_file():\n # get list from disk\n filename = osp.join(osp.dirname(__file__), 'data/filelist.txt')\n filelist = mmcv.list_from_file(filename)\n assert filelist == ['1.jpg', '2.jpg', '3.jpg', '4.jpg', '5.jpg']\n filelist = mmcv.list_from_file(filename, prefix='a/')\n assert filelist == ['a/1.jpg', 'a/2.jpg', 'a/3.jpg', 'a/4.jpg', 'a/5.jpg']\n filelist = mmcv.list_from_file(filename, offset=2)\n assert filelist == ['3.jpg', '4.jpg', '5.jpg']\n filelist = mmcv.list_from_file(filename, max_num=2)\n assert filelist == ['1.jpg', '2.jpg']\n filelist = mmcv.list_from_file(filename, offset=3, max_num=3)\n assert filelist == ['4.jpg', '5.jpg']\n\n # get list from http\n with patch.object(\n HTTPBackend, 'get_text', return_value='1.jpg\\n2.jpg\\n3.jpg'):\n filename = 'http://path/of/your/file'\n filelist = mmcv.list_from_file(\n filename, file_client_args={'backend': 'http'})\n assert filelist == ['1.jpg', '2.jpg', '3.jpg']\n filelist = mmcv.list_from_file(\n filename, file_client_args={'prefix': 'http'})\n assert filelist == ['1.jpg', '2.jpg', '3.jpg']\n filelist = mmcv.list_from_file(filename)\n assert filelist == ['1.jpg', '2.jpg', '3.jpg']\n\n # get list from petrel\n with patch.object(\n PetrelBackend, 'get_text', return_value='1.jpg\\n2.jpg\\n3.jpg'):\n filename = 's3://path/of/your/file'\n filelist = mmcv.list_from_file(\n filename, file_client_args={'backend': 'petrel'})\n assert filelist == ['1.jpg', '2.jpg', '3.jpg']\n filelist = mmcv.list_from_file(\n filename, file_client_args={'prefix': 's3'})\n assert filelist == ['1.jpg', '2.jpg', '3.jpg']\n filelist = mmcv.list_from_file(filename)\n assert filelist == ['1.jpg', '2.jpg', '3.jpg']\n\n\ndef test_dict_from_file():\n # get dict from disk\n filename = osp.join(osp.dirname(__file__), 'data/mapping.txt')\n mapping = mmcv.dict_from_file(filename)\n assert mapping == {'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n mapping = mmcv.dict_from_file(filename, key_type=int)\n assert mapping == {1: 'cat', 2: ['dog', 'cow'], 3: 'panda'}\n\n # get dict from http\n with patch.object(\n HTTPBackend, 'get_text', return_value='1 cat\\n2 dog cow\\n3 panda'):\n filename = 'http://path/of/your/file'\n mapping = mmcv.dict_from_file(\n filename, file_client_args={'backend': 'http'})\n assert mapping == {'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n mapping = mmcv.dict_from_file(\n filename, file_client_args={'prefix': 'http'})\n assert mapping == {'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n mapping = mmcv.dict_from_file(filename)\n assert mapping == {'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n\n # get dict from petrel\n with patch.object(\n PetrelBackend, 'get_text',\n return_value='1 cat\\n2 dog cow\\n3 panda'):\n filename = 's3://path/of/your/file'\n mapping = mmcv.dict_from_file(\n filename, file_client_args={'backend': 'petrel'})\n assert mapping == {'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n mapping = mmcv.dict_from_file(\n filename, file_client_args={'prefix': 's3'})\n assert mapping == {'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n mapping = mmcv.dict_from_file(filename)\n assert mapping == {'1': 'cat', '2': ['dog', 'cow'], '3': 'panda'}\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_image/test_colorspace.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport cv2\nimport numpy as np\nimport pytest\nfrom numpy.testing import assert_array_almost_equal, assert_array_equal\n\nimport mmcv\nfrom mmcv.image.colorspace import (_convert_input_type_range,\n _convert_output_type_range)\n\n\ndef test_bgr2gray():\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.bgr2gray(in_img)\n computed_gray = (\n in_img[:, :, 0] * 0.114 + in_img[:, :, 1] * 0.587 +\n in_img[:, :, 2] * 0.299)\n assert_array_almost_equal(out_img, computed_gray, decimal=4)\n out_img_3d = mmcv.bgr2gray(in_img, True)\n assert out_img_3d.shape == (10, 10, 1)\n assert_array_almost_equal(out_img_3d[..., 0], out_img, decimal=4)\n\n\ndef test_rgb2gray():\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.rgb2gray(in_img)\n computed_gray = (\n in_img[:, :, 0] * 0.299 + in_img[:, :, 1] * 0.587 +\n in_img[:, :, 2] * 0.114)\n assert_array_almost_equal(out_img, computed_gray, decimal=4)\n out_img_3d = mmcv.rgb2gray(in_img, True)\n assert out_img_3d.shape == (10, 10, 1)\n assert_array_almost_equal(out_img_3d[..., 0], out_img, decimal=4)\n\n\ndef test_gray2bgr():\n in_img = np.random.rand(10, 10).astype(np.float32)\n out_img = mmcv.gray2bgr(in_img)\n assert out_img.shape == (10, 10, 3)\n for i in range(3):\n assert_array_almost_equal(out_img[..., i], in_img, decimal=4)\n\n\ndef test_gray2rgb():\n in_img = np.random.rand(10, 10).astype(np.float32)\n out_img = mmcv.gray2rgb(in_img)\n assert out_img.shape == (10, 10, 3)\n for i in range(3):\n assert_array_almost_equal(out_img[..., i], in_img, decimal=4)\n\n\ndef test_bgr2rgb():\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.bgr2rgb(in_img)\n assert out_img.shape == in_img.shape\n assert_array_equal(out_img[..., 0], in_img[..., 2])\n assert_array_equal(out_img[..., 1], in_img[..., 1])\n assert_array_equal(out_img[..., 2], in_img[..., 0])\n\n\ndef test_rgb2bgr():\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.rgb2bgr(in_img)\n assert out_img.shape == in_img.shape\n assert_array_equal(out_img[..., 0], in_img[..., 2])\n assert_array_equal(out_img[..., 1], in_img[..., 1])\n assert_array_equal(out_img[..., 2], in_img[..., 0])\n\n\ndef test_bgr2hsv():\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.bgr2hsv(in_img)\n argmax = in_img.argmax(axis=2)\n computed_hsv = np.empty_like(in_img)\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n b, g, r = in_img[i, j]\n v = max(r, g, b)\n s = (v - min(r, g, b)) / v if v != 0 else 0\n if argmax[i, j] == 0:\n h = 240 + 60 * (r - g) / (v - min(r, g, b))\n elif argmax[i, j] == 1:\n h = 120 + 60 * (b - r) / (v - min(r, g, b))\n else:\n h = 60 * (g - b) / (v - min(r, g, b))\n if h < 0:\n h += 360\n computed_hsv[i, j, :] = [h, s, v]\n assert_array_almost_equal(out_img, computed_hsv, decimal=2)\n\n\ndef test_convert_input_type_range():\n with pytest.raises(TypeError):\n # The img type should be np.float32 or np.uint8\n in_img = np.random.rand(10, 10, 3).astype(np.uint64)\n _convert_input_type_range(in_img)\n # np.float32\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = _convert_input_type_range(in_img)\n assert out_img.dtype == np.float32\n assert np.absolute(out_img).mean() < 1\n # np.uint8\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.uint8)\n out_img = _convert_input_type_range(in_img)\n assert out_img.dtype == np.float32\n assert np.absolute(out_img).mean() < 1\n\n\ndef test_convert_output_type_range():\n with pytest.raises(TypeError):\n # The dst_type should be np.float32 or np.uint8\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n _convert_output_type_range(in_img, np.uint64)\n # np.float32\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.float32)\n out_img = _convert_output_type_range(in_img, np.float32)\n assert out_img.dtype == np.float32\n assert np.absolute(out_img).mean() < 1\n # np.uint8\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.float32)\n out_img = _convert_output_type_range(in_img, np.uint8)\n assert out_img.dtype == np.uint8\n assert np.absolute(out_img).mean() > 1\n\n\ndef assert_image_almost_equal(x, y, atol=1):\n assert x.dtype == np.uint8\n assert y.dtype == np.uint8\n assert np.all(np.abs(x.astype(np.int32) - y.astype(np.int32)) <= atol)\n\n\ndef test_rgb2ycbcr():\n with pytest.raises(TypeError):\n # The img type should be np.float32 or np.uint8\n in_img = np.random.rand(10, 10, 3).astype(np.uint64)\n mmcv.rgb2ycbcr(in_img)\n\n # float32\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.rgb2ycbcr(in_img)\n computed_ycbcr = np.empty_like(in_img)\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n r, g, b = in_img[i, j]\n y = 16 + r * 65.481 + g * 128.553 + b * 24.966\n cb = 128 - r * 37.797 - g * 74.203 + b * 112.0\n cr = 128 + r * 112.0 - g * 93.786 - b * 18.214\n computed_ycbcr[i, j, :] = [y, cb, cr]\n computed_ycbcr /= 255.\n assert_array_almost_equal(out_img, computed_ycbcr, decimal=2)\n # y_only=True\n out_img = mmcv.rgb2ycbcr(in_img, y_only=True)\n computed_y = np.empty_like(out_img, dtype=out_img.dtype)\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n r, g, b = in_img[i, j]\n y = 16 + r * 65.481 + g * 128.553 + b * 24.966\n computed_y[i, j] = y\n computed_y /= 255.\n assert_array_almost_equal(out_img, computed_y, decimal=2)\n\n # uint8\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.uint8)\n out_img = mmcv.rgb2ycbcr(in_img)\n computed_ycbcr = np.empty_like(in_img)\n in_img = in_img / 255.\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n r, g, b = in_img[i, j]\n y = 16 + r * 65.481 + g * 128.553 + b * 24.966\n cb = 128 - r * 37.797 - g * 74.203 + b * 112.0\n cr = 128 + r * 112.0 - g * 93.786 - b * 18.214\n y, cb, cr = y.round(), cb.round(), cr.round()\n computed_ycbcr[i, j, :] = [y, cb, cr]\n assert_image_almost_equal(out_img, computed_ycbcr)\n # y_only=True\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.uint8)\n out_img = mmcv.rgb2ycbcr(in_img, y_only=True)\n computed_y = np.empty_like(out_img, dtype=out_img.dtype)\n in_img = in_img / 255.\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n r, g, b = in_img[i, j]\n y = 16 + r * 65.481 + g * 128.553 + b * 24.966\n y = y.round()\n computed_y[i, j] = y\n assert_image_almost_equal(out_img, computed_y)\n\n\ndef test_bgr2ycbcr():\n # float32\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.bgr2ycbcr(in_img)\n computed_ycbcr = np.empty_like(in_img)\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n b, g, r = in_img[i, j]\n y = 16 + r * 65.481 + g * 128.553 + b * 24.966\n cb = 128 - r * 37.797 - g * 74.203 + b * 112.0\n cr = 128 + r * 112.0 - g * 93.786 - b * 18.214\n computed_ycbcr[i, j, :] = [y, cb, cr]\n computed_ycbcr /= 255.\n assert_array_almost_equal(out_img, computed_ycbcr, decimal=2)\n # y_only=True\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.bgr2ycbcr(in_img, y_only=True)\n computed_y = np.empty_like(out_img, dtype=out_img.dtype)\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n b, g, r = in_img[i, j]\n y = 16 + r * 65.481 + g * 128.553 + b * 24.966\n computed_y[i, j] = y\n computed_y /= 255.\n assert_array_almost_equal(out_img, computed_y, decimal=2)\n\n # uint8\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.uint8)\n out_img = mmcv.bgr2ycbcr(in_img)\n computed_ycbcr = np.empty_like(in_img)\n in_img = in_img / 255.\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n b, g, r = in_img[i, j]\n y = 16 + r * 65.481 + g * 128.553 + b * 24.966\n cb = 128 - r * 37.797 - g * 74.203 + b * 112.0\n cr = 128 + r * 112.0 - g * 93.786 - b * 18.214\n y, cb, cr = y.round(), cb.round(), cr.round()\n computed_ycbcr[i, j, :] = [y, cb, cr]\n assert_image_almost_equal(out_img, computed_ycbcr)\n # y_only = True\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.uint8)\n out_img = mmcv.bgr2ycbcr(in_img, y_only=True)\n computed_y = np.empty_like(out_img, dtype=out_img.dtype)\n in_img = in_img / 255.\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n b, g, r = in_img[i, j]\n y = 16 + r * 65.481 + g * 128.553 + b * 24.966\n y = y.round()\n computed_y[i, j] = y\n assert_image_almost_equal(out_img, computed_y)\n\n\ndef test_ycbcr2rgb():\n with pytest.raises(TypeError):\n # The img type should be np.float32 or np.uint8\n in_img = np.random.rand(10, 10, 3).astype(np.uint64)\n mmcv.ycbcr2rgb(in_img)\n\n # float32\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.ycbcr2rgb(in_img)\n computed_rgb = np.empty_like(in_img)\n in_img *= 255.\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n y, cb, cr = in_img[i, j]\n r = -222.921 + y * 0.00456621 * 255 + cr * 0.00625893 * 255\n g = 135.576 + y * 0.00456621 * 255 - cb * 0.00153632 * 255 - \\\n cr * 0.00318811 * 255\n b = -276.836 + y * 0.00456621 * 255. + cb * 0.00791071 * 255\n computed_rgb[i, j, :] = [r, g, b]\n computed_rgb /= 255.\n assert_array_almost_equal(out_img, computed_rgb, decimal=2)\n\n # uint8\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.uint8)\n out_img = mmcv.ycbcr2rgb(in_img)\n computed_rgb = np.empty_like(in_img)\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n y, cb, cr = in_img[i, j]\n r = -222.921 + y * 0.00456621 * 255 + cr * 0.00625893 * 255\n g = 135.576 + y * 0.00456621 * 255 - cb * 0.00153632 * 255 - \\\n cr * 0.00318811 * 255\n b = -276.836 + y * 0.00456621 * 255. + cb * 0.00791071 * 255\n r, g, b = r.round(), g.round(), b.round()\n computed_rgb[i, j, :] = [r, g, b]\n assert_image_almost_equal(out_img, computed_rgb)\n\n\ndef test_ycbcr2bgr():\n # float32\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.ycbcr2bgr(in_img)\n computed_bgr = np.empty_like(in_img)\n in_img *= 255.\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n y, cb, cr = in_img[i, j]\n r = -222.921 + y * 0.00456621 * 255 + cr * 0.00625893 * 255\n g = 135.576 + y * 0.00456621 * 255 - cb * 0.00153632 * 255 - \\\n cr * 0.00318811 * 255\n b = -276.836 + y * 0.00456621 * 255. + cb * 0.00791071 * 255\n computed_bgr[i, j, :] = [b, g, r]\n computed_bgr /= 255.\n assert_array_almost_equal(out_img, computed_bgr, decimal=2)\n\n # uint8\n in_img = (np.random.rand(10, 10, 3) * 255).astype(np.uint8)\n out_img = mmcv.ycbcr2bgr(in_img)\n computed_bgr = np.empty_like(in_img)\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n y, cb, cr = in_img[i, j]\n r = -222.921 + y * 0.00456621 * 255 + cr * 0.00625893 * 255\n g = 135.576 + y * 0.00456621 * 255 - cb * 0.00153632 * 255 - \\\n cr * 0.00318811 * 255\n b = -276.836 + y * 0.00456621 * 255. + cb * 0.00791071 * 255\n r, g, b = r.round(), g.round(), b.round()\n computed_bgr[i, j, :] = [b, g, r]\n assert_image_almost_equal(out_img, computed_bgr)\n\n\ndef test_bgr2hls():\n in_img = np.random.rand(10, 10, 3).astype(np.float32)\n out_img = mmcv.bgr2hls(in_img)\n argmax = in_img.argmax(axis=2)\n computed_hls = np.empty_like(in_img)\n for i in range(in_img.shape[0]):\n for j in range(in_img.shape[1]):\n b, g, r = in_img[i, j]\n maxc = max(r, g, b)\n minc = min(r, g, b)\n _l = (minc + maxc) / 2.0\n if minc == maxc:\n h = 0.0\n s = 0.0\n if _l <= 0.5:\n s = (maxc - minc) / (maxc + minc)\n else:\n s = (maxc - minc) / (2.0 - maxc - minc)\n if argmax[i, j] == 2:\n h = 60 * (g - b) / (maxc - minc)\n elif argmax[i, j] == 1:\n h = 60 * (2.0 + (b - r) / (maxc - minc))\n else:\n h = 60 * (4.0 + (r - g) / (maxc - minc))\n if h < 0:\n h += 360\n computed_hls[i, j, :] = [h, _l, s]\n assert_array_almost_equal(out_img, computed_hls, decimal=2)\n\n\n@pytest.mark.parametrize('src,dst,ref', [('bgr', 'gray', cv2.COLOR_BGR2GRAY),\n ('rgb', 'gray', cv2.COLOR_RGB2GRAY),\n ('bgr', 'rgb', cv2.COLOR_BGR2RGB),\n ('rgb', 'bgr', cv2.COLOR_RGB2BGR),\n ('bgr', 'hsv', cv2.COLOR_BGR2HSV),\n ('hsv', 'bgr', cv2.COLOR_HSV2BGR),\n ('bgr', 'hls', cv2.COLOR_BGR2HLS),\n ('hls', 'bgr', cv2.COLOR_HLS2BGR)])\ndef test_imconvert(src, dst, ref):\n img = np.random.rand(10, 10, 3).astype(np.float32)\n assert_array_equal(mmcv.imconvert(img, src, dst), cv2.cvtColor(img, ref))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_image/test_geometric.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\n\nimport cv2\nimport numpy as np\nimport pytest\nfrom numpy.testing import assert_array_equal\n\nimport mmcv\n\n\nclass TestGeometric:\n\n @classmethod\n def setup_class(cls):\n cls.data_dir = osp.join(osp.dirname(__file__), '../data')\n # the test img resolution is 400x300\n cls.img_path = osp.join(cls.data_dir, 'color.jpg')\n cls.img = cv2.imread(cls.img_path)\n\n def test_imresize(self):\n resized_img = mmcv.imresize(self.img, (1000, 600))\n assert resized_img.shape == (600, 1000, 3)\n resized_img, w_scale, h_scale = mmcv.imresize(self.img, (1000, 600),\n True)\n assert (resized_img.shape == (600, 1000, 3) and w_scale == 2.5\n and h_scale == 2.0)\n resized_img_dst = np.empty((600, 1000, 3), dtype=self.img.dtype)\n resized_img = mmcv.imresize(self.img, (1000, 600), out=resized_img_dst)\n assert id(resized_img_dst) == id(resized_img)\n assert_array_equal(resized_img_dst,\n mmcv.imresize(self.img, (1000, 600)))\n for mode in ['nearest', 'bilinear', 'bicubic', 'area', 'lanczos']:\n resized_img = mmcv.imresize(\n self.img, (1000, 600), interpolation=mode)\n assert resized_img.shape == (600, 1000, 3)\n\n # test pillow resize\n for mode in [\n 'nearest', 'bilinear', 'bicubic', 'box', 'lanczos', 'hamming'\n ]:\n resized_img = mmcv.imresize(\n self.img, (1000, 600), interpolation=mode, backend='pillow')\n assert resized_img.shape == (600, 1000, 3)\n\n # resize backend must be 'cv2' or 'pillow'\n with pytest.raises(ValueError):\n mmcv.imresize(self.img, (1000, 600), backend='not support')\n\n def test_imresize_to_multiple(self):\n # test size and keep_ratio = False\n resized_img = mmcv.imresize_to_multiple(\n self.img, divisor=16, size=(511, 513), keep_ratio=False)\n assert resized_img.shape == (528, 512, 3)\n resized_img = mmcv.imresize_to_multiple(\n self.img, divisor=(16, 32), size=(511, 513), keep_ratio=False)\n assert resized_img.shape == (544, 512, 3)\n\n # test size, keep_ratio = True, and return_scale\n resized_img, w_scale, h_scale = mmcv.imresize_to_multiple(\n self.img,\n divisor=16,\n size=(1000, 600),\n keep_ratio=True,\n return_scale=True)\n assert resized_img.shape == (\n 608, 800, 3) and h_scale == 608 / 300 and w_scale == 800 / 400\n resized_img, w_scale, h_scale = mmcv.imresize_to_multiple(\n self.img,\n divisor=(18, 16),\n size=(1000, 600),\n keep_ratio=True,\n return_scale=True)\n assert resized_img.shape == (\n 608, 810, 3) and h_scale == 608 / 300 and w_scale == 810 / 400\n\n # test scale_factor and return_scale\n resized_img, w_scale, h_scale = mmcv.imresize_to_multiple(\n self.img, divisor=16, scale_factor=2, return_scale=True)\n assert resized_img.shape == (\n 608, 800, 3) and h_scale == 608 / 300 and w_scale == 800 / 400\n resized_img, w_scale, h_scale = mmcv.imresize_to_multiple(\n self.img, divisor=16, scale_factor=(2, 3), return_scale=True)\n assert resized_img.shape == (\n 912, 800, 3) and h_scale == 912 / 300 and w_scale == 800 / 400\n resized_img, w_scale, h_scale = mmcv.imresize_to_multiple(\n self.img, divisor=(18, 16), scale_factor=(2, 3), return_scale=True)\n assert resized_img.shape == (\n 912, 810, 3) and h_scale == 912 / 300 and w_scale == 810 / 400\n\n # one of size and scale_factor should be given\n with pytest.raises(ValueError):\n mmcv.imresize_to_multiple(\n self.img, divisor=16, size=(1000, 600), scale_factor=2)\n with pytest.raises(ValueError):\n mmcv.imresize_to_multiple(\n self.img, divisor=16, size=None, scale_factor=None)\n\n def test_imresize_like(self):\n a = np.zeros((100, 200, 3))\n resized_img = mmcv.imresize_like(self.img, a)\n assert resized_img.shape == (100, 200, 3)\n\n def test_rescale_size(self):\n new_size, scale_factor = mmcv.rescale_size((400, 300), 1.5, True)\n assert new_size == (600, 450) and scale_factor == 1.5\n new_size, scale_factor = mmcv.rescale_size((400, 300), 0.934, True)\n assert new_size == (374, 280) and scale_factor == 0.934\n\n new_size = mmcv.rescale_size((400, 300), 1.5)\n assert new_size == (600, 450)\n new_size = mmcv.rescale_size((400, 300), 0.934)\n assert new_size == (374, 280)\n\n new_size, scale_factor = mmcv.rescale_size((400, 300), (1000, 600),\n True)\n assert new_size == (800, 600) and scale_factor == 2.0\n new_size, scale_factor = mmcv.rescale_size((400, 300), (180, 200),\n True)\n assert new_size == (200, 150) and scale_factor == 0.5\n\n new_size = mmcv.rescale_size((400, 300), (1000, 600))\n assert new_size == (800, 600)\n new_size = mmcv.rescale_size((400, 300), (180, 200))\n assert new_size == (200, 150)\n\n with pytest.raises(ValueError):\n mmcv.rescale_size((400, 300), -0.5)\n with pytest.raises(TypeError):\n mmcv.rescale_size()((400, 300), [100, 100])\n\n def test_imrescale(self):\n # rescale by a certain factor\n resized_img = mmcv.imrescale(self.img, 1.5)\n assert resized_img.shape == (450, 600, 3)\n resized_img = mmcv.imrescale(self.img, 0.934)\n assert resized_img.shape == (280, 374, 3)\n\n # rescale by a certain max_size\n # resize (400, 300) to (max_1000, max_600)\n resized_img = mmcv.imrescale(self.img, (1000, 600))\n assert resized_img.shape == (600, 800, 3)\n resized_img, scale = mmcv.imrescale(\n self.img, (1000, 600), return_scale=True)\n assert resized_img.shape == (600, 800, 3) and scale == 2.0\n # resize (400, 300) to (max_200, max_180)\n resized_img = mmcv.imrescale(self.img, (180, 200))\n assert resized_img.shape == (150, 200, 3)\n resized_img, scale = mmcv.imrescale(\n self.img, (180, 200), return_scale=True)\n assert resized_img.shape == (150, 200, 3) and scale == 0.5\n\n # test exceptions\n with pytest.raises(ValueError):\n mmcv.imrescale(self.img, -0.5)\n with pytest.raises(TypeError):\n mmcv.imrescale(self.img, [100, 100])\n\n def test_imflip(self):\n # direction must be \"horizontal\" or \"vertical\" or \"diagonal\"\n with pytest.raises(AssertionError):\n mmcv.imflip(np.random.rand(80, 60, 3), direction='random')\n\n # test horizontal flip (color image)\n img = np.random.rand(80, 60, 3)\n h, w, c = img.shape\n flipped_img = mmcv.imflip(img)\n assert flipped_img.shape == img.shape\n for i in range(h):\n for j in range(w):\n for k in range(c):\n assert flipped_img[i, j, k] == img[i, w - 1 - j, k]\n\n # test vertical flip (color image)\n flipped_img = mmcv.imflip(img, direction='vertical')\n assert flipped_img.shape == img.shape\n for i in range(h):\n for j in range(w):\n for k in range(c):\n assert flipped_img[i, j, k] == img[h - 1 - i, j, k]\n\n # test diagonal flip (color image)\n flipped_img = mmcv.imflip(img, direction='diagonal')\n assert flipped_img.shape == img.shape\n for i in range(h):\n for j in range(w):\n for k in range(c):\n assert flipped_img[i, j, k] == img[h - 1 - i, w - 1 - j, k]\n\n # test horizontal flip (grayscale image)\n img = np.random.rand(80, 60)\n h, w = img.shape\n flipped_img = mmcv.imflip(img)\n assert flipped_img.shape == img.shape\n for i in range(h):\n for j in range(w):\n assert flipped_img[i, j] == img[i, w - 1 - j]\n\n # test vertical flip (grayscale image)\n flipped_img = mmcv.imflip(img, direction='vertical')\n assert flipped_img.shape == img.shape\n for i in range(h):\n for j in range(w):\n assert flipped_img[i, j] == img[h - 1 - i, j]\n\n # test diagonal flip (grayscale image)\n flipped_img = mmcv.imflip(img, direction='diagonal')\n assert flipped_img.shape == img.shape\n for i in range(h):\n for j in range(w):\n assert flipped_img[i, j] == img[h - 1 - i, w - 1 - j]\n\n def test_imflip_(self):\n # direction must be \"horizontal\" or \"vertical\" or \"diagonal\"\n with pytest.raises(AssertionError):\n mmcv.imflip_(np.random.rand(80, 60, 3), direction='random')\n\n # test horizontal flip (color image)\n img = np.random.rand(80, 60, 3)\n h, w, c = img.shape\n img_for_flip = img.copy()\n flipped_img = mmcv.imflip_(img_for_flip)\n assert flipped_img.shape == img.shape\n assert flipped_img.shape == img_for_flip.shape\n assert id(flipped_img) == id(img_for_flip)\n for i in range(h):\n for j in range(w):\n for k in range(c):\n assert flipped_img[i, j, k] == img[i, w - 1 - j, k]\n assert flipped_img[i, j, k] == img_for_flip[i, j, k]\n\n # test vertical flip (color image)\n img_for_flip = img.copy()\n flipped_img = mmcv.imflip_(img_for_flip, direction='vertical')\n assert flipped_img.shape == img.shape\n assert flipped_img.shape == img_for_flip.shape\n assert id(flipped_img) == id(img_for_flip)\n for i in range(h):\n for j in range(w):\n for k in range(c):\n assert flipped_img[i, j, k] == img[h - 1 - i, j, k]\n assert flipped_img[i, j, k] == img_for_flip[i, j, k]\n\n # test diagonal flip (color image)\n img_for_flip = img.copy()\n flipped_img = mmcv.imflip_(img_for_flip, direction='diagonal')\n assert flipped_img.shape == img.shape\n assert flipped_img.shape == img_for_flip.shape\n assert id(flipped_img) == id(img_for_flip)\n for i in range(h):\n for j in range(w):\n for k in range(c):\n assert flipped_img[i, j, k] == img[h - 1 - i, w - 1 - j, k]\n assert flipped_img[i, j, k] == img_for_flip[i, j, k]\n\n # test horizontal flip (grayscale image)\n img = np.random.rand(80, 60)\n h, w = img.shape\n img_for_flip = img.copy()\n flipped_img = mmcv.imflip_(img_for_flip)\n assert flipped_img.shape == img.shape\n assert flipped_img.shape == img_for_flip.shape\n assert id(flipped_img) == id(img_for_flip)\n for i in range(h):\n for j in range(w):\n assert flipped_img[i, j] == img[i, w - 1 - j]\n assert flipped_img[i, j] == img_for_flip[i, j]\n\n # test vertical flip (grayscale image)\n img_for_flip = img.copy()\n flipped_img = mmcv.imflip_(img_for_flip, direction='vertical')\n assert flipped_img.shape == img.shape\n assert flipped_img.shape == img_for_flip.shape\n assert id(flipped_img) == id(img_for_flip)\n for i in range(h):\n for j in range(w):\n assert flipped_img[i, j] == img[h - 1 - i, j]\n assert flipped_img[i, j] == img_for_flip[i, j]\n\n # test diagonal flip (grayscale image)\n img_for_flip = img.copy()\n flipped_img = mmcv.imflip_(img_for_flip, direction='diagonal')\n assert flipped_img.shape == img.shape\n assert flipped_img.shape == img_for_flip.shape\n assert id(flipped_img) == id(img_for_flip)\n for i in range(h):\n for j in range(w):\n assert flipped_img[i, j] == img[h - 1 - i, w - 1 - j]\n assert flipped_img[i, j] == img_for_flip[i, j]\n\n def test_imcrop(self):\n # yapf: disable\n bboxes = np.array([[100, 100, 199, 199], # center\n [0, 0, 150, 100], # left-top corner\n [250, 200, 399, 299], # right-bottom corner\n [0, 100, 399, 199], # wide\n [150, 0, 299, 299]]) # tall\n # yapf: enable\n\n # crop one bbox\n patch = mmcv.imcrop(self.img, bboxes[0, :])\n patches = mmcv.imcrop(self.img, bboxes[[0], :])\n assert patch.shape == (100, 100, 3)\n patch_path = osp.join(self.data_dir, 'patches')\n ref_patch = np.load(patch_path + '/0.npy')\n assert_array_equal(patch, ref_patch)\n assert isinstance(patches, list) and len(patches) == 1\n assert_array_equal(patches[0], ref_patch)\n\n # crop with no scaling and padding\n patches = mmcv.imcrop(self.img, bboxes)\n assert len(patches) == bboxes.shape[0]\n for i in range(len(patches)):\n ref_patch = np.load(patch_path + f'/{i}.npy')\n assert_array_equal(patches[i], ref_patch)\n\n # crop with scaling and no padding\n patches = mmcv.imcrop(self.img, bboxes, 1.2)\n for i in range(len(patches)):\n ref_patch = np.load(patch_path + f'/scale_{i}.npy')\n assert_array_equal(patches[i], ref_patch)\n\n # crop with scaling and padding\n patches = mmcv.imcrop(self.img, bboxes, 1.2, pad_fill=[255, 255, 0])\n for i in range(len(patches)):\n ref_patch = np.load(patch_path + f'/pad_{i}.npy')\n assert_array_equal(patches[i], ref_patch)\n patches = mmcv.imcrop(self.img, bboxes, 1.2, pad_fill=0)\n for i in range(len(patches)):\n ref_patch = np.load(patch_path + f'/pad0_{i}.npy')\n assert_array_equal(patches[i], ref_patch)\n\n def test_impad(self):\n # grayscale image\n img = np.random.rand(10, 10).astype(np.float32)\n padded_img = mmcv.impad(img, padding=(0, 0, 2, 5), pad_val=0)\n assert_array_equal(img, padded_img[:10, :10])\n assert_array_equal(\n np.zeros((5, 12), dtype='float32'), padded_img[10:, :])\n assert_array_equal(\n np.zeros((15, 2), dtype='float32'), padded_img[:, 10:])\n\n # RGB image\n img = np.random.rand(10, 10, 3).astype(np.float32)\n padded_img = mmcv.impad(img, padding=(0, 0, 2, 5), pad_val=0)\n assert_array_equal(img, padded_img[:10, :10, :])\n assert_array_equal(\n np.zeros((5, 12, 3), dtype='float32'), padded_img[10:, :, :])\n assert_array_equal(\n np.zeros((15, 2, 3), dtype='float32'), padded_img[:, 10:, :])\n\n # RGB image with different values for three channels.\n img = np.random.randint(256, size=(10, 10, 3)).astype('uint8')\n padded_img = mmcv.impad(\n img, padding=(0, 0, 2, 5), pad_val=(100, 110, 120))\n assert_array_equal(img, padded_img[:10, :10, :])\n assert_array_equal(\n np.array([100, 110, 120], dtype='uint8') * np.ones(\n (5, 12, 3), dtype='uint8'), padded_img[10:, :, :])\n assert_array_equal(\n np.array([100, 110, 120], dtype='uint8') * np.ones(\n (15, 2, 3), dtype='uint8'), padded_img[:, 10:, :])\n\n # Pad the grayscale image to shape (15, 12)\n img = np.random.rand(10, 10).astype(np.float32)\n padded_img = mmcv.impad(img, shape=(15, 12))\n assert_array_equal(img, padded_img[:10, :10])\n assert_array_equal(\n np.zeros((5, 12), dtype='float32'), padded_img[10:, :])\n assert_array_equal(\n np.zeros((15, 2), dtype='float32'), padded_img[:, 10:])\n\n # Pad the RGB image to shape (15, 12)\n img = np.random.rand(10, 10, 3).astype(np.float32)\n padded_img = mmcv.impad(img, shape=(15, 12))\n assert_array_equal(img, padded_img[:10, :10, :])\n assert_array_equal(\n np.zeros((5, 12, 3), dtype='float32'), padded_img[10:, :, :])\n assert_array_equal(\n np.zeros((15, 2, 3), dtype='float32'), padded_img[:, 10:, :])\n\n # Pad the RGB image to shape (15, 12) with different values for\n # three channels.\n img = np.random.randint(256, size=(10, 10, 3)).astype('uint8')\n padded_img = mmcv.impad(img, shape=(15, 12), pad_val=(100, 110, 120))\n assert_array_equal(img, padded_img[:10, :10, :])\n assert_array_equal(\n np.array([100, 110, 120], dtype='uint8') * np.ones(\n (5, 12, 3), dtype='uint8'), padded_img[10:, :, :])\n assert_array_equal(\n np.array([100, 110, 120], dtype='uint8') * np.ones(\n (15, 2, 3), dtype='uint8'), padded_img[:, 10:, :])\n\n # RGB image with padding=[5, 2]\n img = np.random.rand(10, 10, 3).astype(np.float32)\n padded_img = mmcv.impad(img, padding=(5, 2), pad_val=0)\n\n assert padded_img.shape == (14, 20, 3)\n assert_array_equal(img, padded_img[2:12, 5:15, :])\n assert_array_equal(\n np.zeros((2, 5, 3), dtype='float32'), padded_img[:2, :5, :])\n assert_array_equal(\n np.zeros((2, 5, 3), dtype='float32'), padded_img[12:, :5, :])\n assert_array_equal(\n np.zeros((2, 5, 3), dtype='float32'), padded_img[:2, 15:, :])\n assert_array_equal(\n np.zeros((2, 5, 3), dtype='float32'), padded_img[12:, 15:, :])\n\n # RGB image with type(pad_val) = tuple\n pad_val = (0, 1, 2)\n img = np.random.rand(10, 10, 3).astype(np.float32)\n padded_img = mmcv.impad(img, padding=(0, 0, 5, 2), pad_val=pad_val)\n\n assert padded_img.shape == (12, 15, 3)\n assert_array_equal(img, padded_img[:10, :10, :])\n assert_array_equal(pad_val[0] * np.ones((2, 15, 1), dtype='float32'),\n padded_img[10:, :, 0:1])\n assert_array_equal(pad_val[1] * np.ones((2, 15, 1), dtype='float32'),\n padded_img[10:, :, 1:2])\n assert_array_equal(pad_val[2] * np.ones((2, 15, 1), dtype='float32'),\n padded_img[10:, :, 2:3])\n\n assert_array_equal(pad_val[0] * np.ones((12, 5, 1), dtype='float32'),\n padded_img[:, 10:, 0:1])\n assert_array_equal(pad_val[1] * np.ones((12, 5, 1), dtype='float32'),\n padded_img[:, 10:, 1:2])\n assert_array_equal(pad_val[2] * np.ones((12, 5, 1), dtype='float32'),\n padded_img[:, 10:, 2:3])\n\n # test different padding mode with channel number = 3\n for mode in ['constant', 'edge', 'reflect', 'symmetric']:\n img = np.random.rand(10, 10, 3).astype(np.float32)\n padded_img = mmcv.impad(\n img, padding=(0, 0, 5, 2), pad_val=pad_val, padding_mode=mode)\n assert padded_img.shape == (12, 15, 3)\n\n # test different padding mode with channel number = 1\n for mode in ['constant', 'edge', 'reflect', 'symmetric']:\n img = np.random.rand(10, 10).astype(np.float32)\n padded_img = mmcv.impad(\n img, padding=(0, 0, 5, 2), pad_val=0, padding_mode=mode)\n assert padded_img.shape == (12, 15)\n\n # Padding must be a int or a 2, or 4 element tuple.\n with pytest.raises(ValueError):\n mmcv.impad(img, padding=(1, 1, 1))\n\n # pad_val must be a int or a tuple\n with pytest.raises(TypeError):\n mmcv.impad(img, padding=(1, 1, 1, 1), pad_val='wrong')\n\n # When pad_val is a tuple,\n # len(pad_val) should be equal to img.shape[-1]\n img = np.random.rand(10, 10, 3).astype(np.float32)\n with pytest.raises(AssertionError):\n mmcv.impad(img, padding=3, pad_val=(100, 200))\n\n with pytest.raises(AssertionError):\n mmcv.impad(img, padding=2, pad_val=0, padding_mode='unknown')\n\n with pytest.raises(AssertionError):\n mmcv.impad(img, shape=(12, 15), padding=(0, 0, 5, 2))\n\n def test_impad_to_multiple(self):\n img = np.random.rand(11, 14, 3).astype(np.float32)\n padded_img = mmcv.impad_to_multiple(img, 4)\n assert padded_img.shape == (12, 16, 3)\n img = np.random.rand(20, 12).astype(np.float32)\n padded_img = mmcv.impad_to_multiple(img, 5)\n assert padded_img.shape == (20, 15)\n img = np.random.rand(20, 12).astype(np.float32)\n padded_img = mmcv.impad_to_multiple(img, 2)\n assert padded_img.shape == (20, 12)\n\n def test_cutout(self):\n img = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]).astype(np.uint8)\n\n # shape must be int or tuple\n with pytest.raises(AssertionError):\n mmcv.cutout(img, 2.5)\n # pad_val must be int or float or tuple with the same length\n # of img channels\n with pytest.raises(AssertionError):\n mmcv.cutout(img, 1, (1, 2, 3))\n with pytest.raises(TypeError):\n mmcv.cutout(img, 1, None)\n\n # test cutout the whole img\n assert_array_equal(mmcv.cutout(img, 6), np.zeros_like(img))\n # test not cutout\n assert_array_equal(mmcv.cutout(img, 0), img)\n # test cutout when shape is int\n np.random.seed(0)\n img_cutout = np.array([[1, 2, 3], [4, 0, 6], [7, 8,\n 9]]).astype(np.uint8)\n assert_array_equal(mmcv.cutout(img, 1), img_cutout)\n img_cutout = np.array([[1, 2, 3], [4, 10, 6], [7, 8,\n 9]]).astype(np.uint8)\n assert_array_equal(mmcv.cutout(img, 1, pad_val=10), img_cutout)\n # test cutout when shape is tuple\n np.random.seed(0)\n img_cutout = np.array([[1, 2, 3], [0, 0, 6], [7, 8,\n 9]]).astype(np.uint8)\n assert_array_equal(mmcv.cutout(img, (1, 2)), img_cutout)\n img_cutout = np.array([[1, 2, 3], [10, 10, 6], [7, 8,\n 9]]).astype(np.uint8)\n assert_array_equal(mmcv.cutout(img, (1, 2), pad_val=10), img_cutout)\n\n def test_imrotate(self):\n img = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]).astype(np.uint8)\n assert_array_equal(mmcv.imrotate(img, 0), img)\n img_r = np.array([[7, 4, 1], [8, 5, 2], [9, 6, 3]])\n assert_array_equal(mmcv.imrotate(img, 90), img_r)\n img_r = np.array([[3, 6, 9], [2, 5, 8], [1, 4, 7]])\n assert_array_equal(mmcv.imrotate(img, -90), img_r)\n\n img = np.array([[1, 2, 3, 4], [5, 6, 7, 8]]).astype(np.uint8)\n img_r = np.array([[0, 6, 2, 0], [0, 7, 3, 0]])\n assert_array_equal(mmcv.imrotate(img, 90), img_r)\n img_r = np.array([[1, 0, 0, 0], [2, 0, 0, 0]])\n assert_array_equal(mmcv.imrotate(img, 90, center=(0, 0)), img_r)\n img_r = np.array([[255, 6, 2, 255], [255, 7, 3, 255]])\n assert_array_equal(mmcv.imrotate(img, 90, border_value=255), img_r)\n img_r = np.array([[5, 1], [6, 2], [7, 3], [8, 4]])\n assert_array_equal(mmcv.imrotate(img, 90, auto_bound=True), img_r)\n\n with pytest.raises(ValueError):\n mmcv.imrotate(img, 90, center=(0, 0), auto_bound=True)\n\n def test_imshear(self):\n img = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]).astype(np.uint8)\n assert_array_equal(mmcv.imshear(img, 0), img)\n # magnitude=1, horizontal\n img_sheared = np.array([[1, 2, 3], [0, 4, 5], [0, 0, 7]],\n dtype=np.uint8)\n assert_array_equal(mmcv.imshear(img, 1), img_sheared)\n # magnitude=-1, vertical\n img_sheared = np.array([[1, 5, 9], [4, 8, 0], [7, 0, 0]],\n dtype=np.uint8)\n assert_array_equal(mmcv.imshear(img, -1, 'vertical'), img_sheared)\n # magnitude=1, vertical, borderValue=100\n borderValue = 100\n img_sheared = np.array(\n [[1, borderValue, borderValue], [4, 2, borderValue], [7, 5, 3]],\n dtype=np.uint8)\n assert_array_equal(\n mmcv.imshear(img, 1, 'vertical', borderValue), img_sheared)\n # magnitude=1, vertical, borderValue=100, img shape (h,w,3)\n img = np.stack([img, img, img], axis=-1)\n img_sheared = np.stack([img_sheared, img_sheared, img_sheared],\n axis=-1)\n assert_array_equal(\n mmcv.imshear(img, 1, 'vertical', borderValue), img_sheared)\n # test tuple format of borderValue\n assert_array_equal(\n mmcv.imshear(img, 1, 'vertical',\n (borderValue, borderValue, borderValue)), img_sheared)\n\n # test invalid length of borderValue\n with pytest.raises(AssertionError):\n mmcv.imshear(img, 0.5, 'horizontal', (borderValue, ))\n\n # test invalid type of borderValue\n with pytest.raises(ValueError):\n mmcv.imshear(img, 0.5, 'horizontal', [borderValue])\n\n # test invalid value of direction\n with pytest.raises(AssertionError):\n mmcv.imshear(img, 0.5, 'diagonal')\n\n def test_imtranslate(self):\n img = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=np.uint8)\n assert_array_equal(mmcv.imtranslate(img, 0), img)\n # offset=1, horizontal\n img_translated = np.array([[128, 1, 2], [128, 4, 5], [128, 7, 8]],\n dtype=np.uint8)\n assert_array_equal(\n mmcv.imtranslate(img, 1, border_value=128), img_translated)\n # offset=-1, vertical\n img_translated = np.array([[4, 5, 6], [7, 8, 9], [0, 0, 0]],\n dtype=np.uint8)\n assert_array_equal(\n mmcv.imtranslate(img, -1, 'vertical'), img_translated)\n # offset=-2, horizontal\n img = np.array([[1, 2, 3, 4], [5, 6, 7, 8]], dtype=np.uint8)\n img = np.stack([img, img, img], axis=-1)\n img_translated = [[3, 4, 128, 128], [7, 8, 128, 128]]\n img_translated = np.stack(\n [img_translated, img_translated, img_translated], axis=-1)\n assert_array_equal(\n mmcv.imtranslate(img, -2, border_value=128), img_translated)\n # offset=2, vertical\n border_value = (110, 120, 130)\n img_translated = np.stack([\n np.ones((2, 4)) * border_value[0],\n np.ones((2, 4)) * border_value[1],\n np.ones((2, 4)) * border_value[2]\n ],\n axis=-1).astype(np.uint8)\n assert_array_equal(\n mmcv.imtranslate(img, 2, 'vertical', border_value), img_translated)\n # test invalid number elements in border_value\n with pytest.raises(AssertionError):\n mmcv.imtranslate(img, 1, border_value=(1, ))\n # test invalid type of border_value\n with pytest.raises(ValueError):\n mmcv.imtranslate(img, 1, border_value=[1, 2, 3])\n # test invalid value of direction\n with pytest.raises(AssertionError):\n mmcv.imtranslate(img, 1, 'diagonal')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_image/test_image_misc.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numpy as np\nimport pytest\nfrom numpy.testing import assert_array_equal\n\nimport mmcv\n\ntry:\n import torch\nexcept ImportError:\n torch = None\n\n\n@pytest.mark.skipif(torch is None, reason='requires torch library')\ndef test_tensor2imgs():\n\n # test tensor obj\n with pytest.raises(AssertionError):\n tensor = np.random.rand(2, 3, 3)\n mmcv.tensor2imgs(tensor)\n\n # test tensor ndim\n with pytest.raises(AssertionError):\n tensor = torch.randn(2, 3, 3)\n mmcv.tensor2imgs(tensor)\n\n # test tensor dim-1\n with pytest.raises(AssertionError):\n tensor = torch.randn(2, 4, 3, 3)\n mmcv.tensor2imgs(tensor)\n\n # test mean length\n with pytest.raises(AssertionError):\n tensor = torch.randn(2, 3, 5, 5)\n mmcv.tensor2imgs(tensor, mean=(1, ))\n tensor = torch.randn(2, 1, 5, 5)\n mmcv.tensor2imgs(tensor, mean=(0, 0, 0))\n\n # test std length\n with pytest.raises(AssertionError):\n tensor = torch.randn(2, 3, 5, 5)\n mmcv.tensor2imgs(tensor, std=(1, ))\n tensor = torch.randn(2, 1, 5, 5)\n mmcv.tensor2imgs(tensor, std=(1, 1, 1))\n\n # test to_rgb\n with pytest.raises(AssertionError):\n tensor = torch.randn(2, 1, 5, 5)\n mmcv.tensor2imgs(tensor, mean=(0, ), std=(1, ), to_rgb=True)\n\n # test rgb=True\n tensor = torch.randn(2, 3, 5, 5)\n gts = [\n t.cpu().numpy().transpose(1, 2, 0).astype(np.uint8)\n for t in tensor.flip(1)\n ]\n outputs = mmcv.tensor2imgs(tensor, to_rgb=True)\n for gt, output in zip(gts, outputs):\n assert_array_equal(gt, output)\n\n # test rgb=False\n tensor = torch.randn(2, 3, 5, 5)\n gts = [t.cpu().numpy().transpose(1, 2, 0).astype(np.uint8) for t in tensor]\n outputs = mmcv.tensor2imgs(tensor, to_rgb=False)\n for gt, output in zip(gts, outputs):\n assert_array_equal(gt, output)\n\n # test tensor channel 1 and rgb=False\n tensor = torch.randn(2, 1, 5, 5)\n gts = [t.squeeze(0).cpu().numpy().astype(np.uint8) for t in tensor]\n outputs = mmcv.tensor2imgs(tensor, to_rgb=False)\n for gt, output in zip(gts, outputs):\n assert_array_equal(gt, output)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_image/test_io.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport os.path as osp\nimport sys\nimport tempfile\nfrom pathlib import Path\nfrom unittest.mock import MagicMock, patch\n\nimport cv2\nimport numpy as np\nimport pytest\nfrom numpy.testing import assert_allclose, assert_array_equal\n\nimport mmcv\nfrom mmcv.fileio.file_client import HTTPBackend, PetrelBackend\n\n\nclass TestIO:\n\n @classmethod\n def setup_class(cls):\n cls.data_dir = osp.join(osp.dirname(__file__), '../data')\n # the test img resolution is 400x300\n cls.img_path = osp.join(cls.data_dir, 'color.jpg')\n cls.img_path_obj = Path(cls.img_path)\n cls.gray_img_path = osp.join(cls.data_dir, 'grayscale.jpg')\n cls.gray_img_path_obj = Path(cls.gray_img_path)\n cls.gray_img_dim3_path = osp.join(cls.data_dir, 'grayscale_dim3.jpg')\n cls.gray_alpha_img_path = osp.join(cls.data_dir, 'gray_alpha.png')\n cls.palette_img_path = osp.join(cls.data_dir, 'palette.gif')\n cls.exif_img_path = osp.join(cls.data_dir, 'color_exif.jpg')\n cls.img = cv2.imread(cls.img_path)\n cls.tiff_path = osp.join(cls.data_dir, 'uint16-5channel.tif')\n # petrel s3 path\n cls.s3_path = 's3://path/of/your/file.jpg'\n # http path\n cls.http_path = 'http://path/of/your/file.jpg'\n # add mock package\n sys.modules['petrel_client'] = MagicMock()\n sys.modules['petrel_client.client'] = MagicMock()\n\n @classmethod\n def teardown_class(cls):\n # clean instances avoid to influence other unittest\n mmcv.FileClient._instances = {}\n\n def assert_img_equal(self, img, ref_img, ratio_thr=0.999):\n assert img.shape == ref_img.shape\n assert img.dtype == ref_img.dtype\n area = ref_img.shape[0] * ref_img.shape[1]\n diff = np.abs(img.astype('int32') - ref_img.astype('int32'))\n assert np.sum(diff <= 1) / float(area) > ratio_thr\n\n def test_imread(self):\n # backend cv2\n mmcv.use_backend('cv2')\n\n # HardDiskBackend\n img_cv2_color_bgr = mmcv.imread(self.img_path)\n assert img_cv2_color_bgr.shape == (300, 400, 3)\n img_cv2_color_rgb = mmcv.imread(self.img_path, channel_order='rgb')\n assert img_cv2_color_rgb.shape == (300, 400, 3)\n assert_array_equal(img_cv2_color_rgb[:, :, ::-1], img_cv2_color_bgr)\n img_cv2_grayscale1 = mmcv.imread(self.img_path, 'grayscale')\n assert img_cv2_grayscale1.shape == (300, 400)\n img_cv2_grayscale2 = mmcv.imread(self.gray_img_path)\n assert img_cv2_grayscale2.shape == (300, 400, 3)\n img_cv2_unchanged = mmcv.imread(self.gray_img_path, 'unchanged')\n assert img_cv2_unchanged.shape == (300, 400)\n img_cv2_unchanged = mmcv.imread(img_cv2_unchanged)\n assert_array_equal(img_cv2_unchanged, mmcv.imread(img_cv2_unchanged))\n\n img_cv2_color_bgr = mmcv.imread(self.img_path_obj)\n assert img_cv2_color_bgr.shape == (300, 400, 3)\n img_cv2_color_rgb = mmcv.imread(self.img_path_obj, channel_order='rgb')\n assert img_cv2_color_rgb.shape == (300, 400, 3)\n assert_array_equal(img_cv2_color_rgb[:, :, ::-1], img_cv2_color_bgr)\n img_cv2_grayscale1 = mmcv.imread(self.img_path_obj, 'grayscale')\n assert img_cv2_grayscale1.shape == (300, 400)\n img_cv2_grayscale2 = mmcv.imread(self.gray_img_path_obj)\n assert img_cv2_grayscale2.shape == (300, 400, 3)\n img_cv2_unchanged = mmcv.imread(self.gray_img_path_obj, 'unchanged')\n assert img_cv2_unchanged.shape == (300, 400)\n with pytest.raises(TypeError):\n mmcv.imread(1)\n\n # PetrelBackend\n img_cv2_color_bgr = mmcv.imread(self.img_path)\n with patch.object(\n PetrelBackend, 'get',\n return_value=img_cv2_color_bgr) as mock_method:\n img_cv2_color_bgr_petrel = mmcv.imread(self.s3_path, backend='cv2')\n img_cv2_color_bgr_petrel_with_args = mmcv.imread(\n self.s3_path,\n backend='cv2',\n file_client_args={'backend': 'petrel'})\n mock_method.assert_called()\n assert_array_equal(img_cv2_color_bgr_petrel,\n img_cv2_color_bgr_petrel_with_args)\n\n # HTTPBackend\n img_cv2_color_bgr = mmcv.imread(self.img_path)\n with patch.object(\n HTTPBackend, 'get',\n return_value=img_cv2_color_bgr) as mock_method:\n img_cv2_color_bgr_http = mmcv.imread(self.http_path, backend='cv2')\n img_cv2_color_bgr_http_with_args = mmcv.imread(\n self.http_path,\n backend='cv2',\n file_client_args={'backend': 'http'})\n mock_method.assert_called()\n assert_array_equal(img_cv2_color_bgr_http,\n img_cv2_color_bgr_http_with_args)\n\n with pytest.raises(FileNotFoundError):\n mmcv.imread('/not/exists/' + self.img_path)\n\n # test arg backend pillow\n img_pil_gray_alpha = mmcv.imread(\n self.gray_alpha_img_path, 'grayscale', backend='pillow')\n assert img_pil_gray_alpha.shape == (400, 500)\n mean = img_pil_gray_alpha[300:, 400:].mean()\n assert_allclose(img_pil_gray_alpha[300:, 400:] - mean, 0)\n img_pil_gray_alpha = mmcv.imread(\n self.gray_alpha_img_path, backend='pillow')\n mean = img_pil_gray_alpha[300:, 400:].mean(axis=(0, 1))\n assert_allclose(img_pil_gray_alpha[300:, 400:] - mean, 0)\n assert img_pil_gray_alpha.shape == (400, 500, 3)\n img_pil_gray_alpha = mmcv.imread(\n self.gray_alpha_img_path, 'unchanged', backend='pillow')\n assert img_pil_gray_alpha.shape == (400, 500, 2)\n img_pil_palette = mmcv.imread(\n self.palette_img_path, 'grayscale', backend='pillow')\n assert img_pil_palette.shape == (300, 400)\n img_pil_palette = mmcv.imread(self.palette_img_path, backend='pillow')\n assert img_pil_palette.shape == (300, 400, 3)\n img_pil_palette = mmcv.imread(\n self.palette_img_path, 'unchanged', backend='pillow')\n assert img_pil_palette.shape == (300, 400)\n\n # backend pillow\n mmcv.use_backend('pillow')\n img_pil_grayscale1 = mmcv.imread(self.img_path, 'grayscale')\n assert img_pil_grayscale1.shape == (300, 400)\n img_pil_gray_alpha = mmcv.imread(self.gray_alpha_img_path, 'grayscale')\n assert img_pil_gray_alpha.shape == (400, 500)\n mean = img_pil_gray_alpha[300:, 400:].mean()\n assert_allclose(img_pil_gray_alpha[300:, 400:] - mean, 0)\n img_pil_gray_alpha = mmcv.imread(self.gray_alpha_img_path)\n mean = img_pil_gray_alpha[300:, 400:].mean(axis=(0, 1))\n assert_allclose(img_pil_gray_alpha[300:, 400:] - mean, 0)\n assert img_pil_gray_alpha.shape == (400, 500, 3)\n img_pil_gray_alpha = mmcv.imread(self.gray_alpha_img_path, 'unchanged')\n assert img_pil_gray_alpha.shape == (400, 500, 2)\n img_pil_palette = mmcv.imread(self.palette_img_path, 'grayscale')\n assert img_pil_palette.shape == (300, 400)\n img_pil_palette = mmcv.imread(self.palette_img_path)\n assert img_pil_palette.shape == (300, 400, 3)\n img_pil_palette = mmcv.imread(self.palette_img_path, 'unchanged')\n assert img_pil_palette.shape == (300, 400)\n img_pil_grayscale2 = mmcv.imread(self.gray_img_path)\n assert img_pil_grayscale2.shape == (300, 400, 3)\n img_pil_unchanged = mmcv.imread(self.gray_img_path, 'unchanged')\n assert img_pil_unchanged.shape == (300, 400)\n img_pil_unchanged = mmcv.imread(img_pil_unchanged)\n assert_array_equal(img_pil_unchanged, mmcv.imread(img_pil_unchanged))\n\n img_pil_color_bgr = mmcv.imread(self.img_path_obj)\n assert img_pil_color_bgr.shape == (300, 400, 3)\n img_pil_color_rgb = mmcv.imread(self.img_path_obj, channel_order='rgb')\n assert img_pil_color_rgb.shape == (300, 400, 3)\n assert (img_pil_color_rgb == img_cv2_color_rgb).sum() / float(\n img_cv2_color_rgb.size) > 0.5\n assert_array_equal(img_pil_color_rgb[:, :, ::-1], img_pil_color_bgr)\n img_pil_grayscale1 = mmcv.imread(self.img_path_obj, 'grayscale')\n assert img_pil_grayscale1.shape == (300, 400)\n img_pil_grayscale2 = mmcv.imread(self.gray_img_path_obj)\n assert img_pil_grayscale2.shape == (300, 400, 3)\n img_pil_unchanged = mmcv.imread(self.gray_img_path_obj, 'unchanged')\n assert img_pil_unchanged.shape == (300, 400)\n with pytest.raises(TypeError):\n mmcv.imread(1)\n\n # backend turbojpeg\n mmcv.use_backend('turbojpeg')\n\n img_turbojpeg_color_bgr = mmcv.imread(self.img_path)\n assert img_turbojpeg_color_bgr.shape == (300, 400, 3)\n assert_array_equal(img_turbojpeg_color_bgr, img_cv2_color_bgr)\n\n img_turbojpeg_color_rgb = mmcv.imread(\n self.img_path, channel_order='rgb')\n assert img_turbojpeg_color_rgb.shape == (300, 400, 3)\n assert_array_equal(img_turbojpeg_color_rgb, img_cv2_color_rgb)\n\n with pytest.raises(ValueError):\n mmcv.imread(self.img_path, channel_order='unsupport_order')\n\n img_turbojpeg_grayscale1 = mmcv.imread(self.img_path, flag='grayscale')\n assert img_turbojpeg_grayscale1.shape == (300, 400)\n assert_array_equal(img_turbojpeg_grayscale1, img_cv2_grayscale1)\n\n img_turbojpeg_grayscale2 = mmcv.imread(self.gray_img_path)\n assert img_turbojpeg_grayscale2.shape == (300, 400, 3)\n assert_array_equal(img_turbojpeg_grayscale2, img_cv2_grayscale2)\n\n img_turbojpeg_grayscale2 = mmcv.imread(img_turbojpeg_grayscale2)\n assert_array_equal(img_turbojpeg_grayscale2,\n mmcv.imread(img_turbojpeg_grayscale2))\n\n with pytest.raises(ValueError):\n mmcv.imread(self.gray_img_path, 'unchanged')\n\n with pytest.raises(TypeError):\n mmcv.imread(1)\n\n with pytest.raises(AssertionError):\n mmcv.use_backend('unsupport_backend')\n\n with pytest.raises(ValueError):\n mmcv.imread(self.img_path, 'unsupported_backend')\n\n # backend tifffile, multi channel tiff file(> 4 channels).\n mmcv.use_backend('tifffile')\n img_tifffile = mmcv.imread(self.tiff_path)\n assert img_tifffile.shape == (200, 150, 5)\n\n mmcv.use_backend('cv2')\n\n # consistent exif behaviour\n img_cv2_exif = mmcv.imread(self.exif_img_path)\n img_pil_exif = mmcv.imread(self.exif_img_path, backend='pillow')\n assert img_cv2_exif.shape == (400, 300, 3)\n assert img_pil_exif.shape == (400, 300, 3)\n img_cv2_exif_unchanged = mmcv.imread(\n self.exif_img_path, flag='unchanged')\n img_pil_exif_unchanged = mmcv.imread(\n self.exif_img_path, backend='pillow', flag='unchanged')\n assert img_cv2_exif_unchanged.shape == (300, 400, 3)\n assert img_pil_exif_unchanged.shape == (300, 400, 3)\n img_cv2_color_ignore_exif = mmcv.imread(\n self.exif_img_path, flag='color_ignore_orientation')\n img_pil_color_ignore_exif = mmcv.imread(\n self.exif_img_path,\n backend='pillow',\n flag='color_ignore_orientation')\n assert img_cv2_color_ignore_exif.shape == (300, 400, 3)\n assert img_pil_color_ignore_exif.shape == (300, 400, 3)\n img_cv2_grayscale_ignore_exif = mmcv.imread(\n self.exif_img_path, flag='grayscale_ignore_orientation')\n img_pil_grayscale_ignore_exif = mmcv.imread(\n self.exif_img_path,\n backend='pillow',\n flag='grayscale_ignore_orientation')\n assert img_cv2_grayscale_ignore_exif.shape == (300, 400)\n assert img_pil_grayscale_ignore_exif.shape == (300, 400)\n\n def test_imfrombytes(self):\n # backend cv2, channel order: bgr\n mmcv.use_backend('cv2')\n with open(self.img_path, 'rb') as f:\n img_bytes = f.read()\n img_cv2 = mmcv.imfrombytes(img_bytes)\n assert img_cv2.shape == (300, 400, 3)\n\n # backend cv2, channel order: rgb\n mmcv.use_backend('cv2')\n with open(self.img_path, 'rb') as f:\n img_bytes = f.read()\n img_rgb_cv2 = mmcv.imfrombytes(img_bytes, channel_order='rgb')\n assert img_rgb_cv2.shape == (300, 400, 3)\n assert_array_equal(img_rgb_cv2, img_cv2[:, :, ::-1])\n\n # backend cv2, grayscale, decode as 3 channels\n with open(self.gray_img_path, 'rb') as f:\n img_bytes = f.read()\n gray_img_rgb_cv2 = mmcv.imfrombytes(img_bytes)\n assert gray_img_rgb_cv2.shape == (300, 400, 3)\n\n # backend cv2, grayscale\n with open(self.gray_img_path, 'rb') as f:\n img_bytes = f.read()\n gray_img_cv2 = mmcv.imfrombytes(img_bytes, flag='grayscale')\n assert gray_img_cv2.shape == (300, 400)\n\n # backend cv2, grayscale dim3\n with open(self.gray_img_dim3_path, 'rb') as f:\n img_bytes = f.read()\n gray_img_dim3_cv2 = mmcv.imfrombytes(img_bytes, flag='grayscale')\n assert gray_img_dim3_cv2.shape == (300, 400)\n\n # arg backend pillow, channel order: bgr\n with open(self.img_path, 'rb') as f:\n img_bytes = f.read()\n img_pillow = mmcv.imfrombytes(img_bytes, backend='pillow')\n assert img_pillow.shape == (300, 400, 3)\n # Pillow and opencv decoding may not be the same\n assert (img_cv2 == img_pillow).sum() / float(img_cv2.size) > 0.5\n\n # backend pillow, channel order: bgr\n mmcv.use_backend('pillow')\n with open(self.img_path, 'rb') as f:\n img_bytes = f.read()\n img_pillow = mmcv.imfrombytes(img_bytes)\n assert img_pillow.shape == (300, 400, 3)\n # Pillow and opencv decoding may not be the same\n assert (img_cv2 == img_pillow).sum() / float(img_cv2.size) > 0.5\n\n # backend turbojpeg, channel order: bgr\n mmcv.use_backend('turbojpeg')\n with open(self.img_path, 'rb') as f:\n img_bytes = f.read()\n img_turbojpeg = mmcv.imfrombytes(img_bytes)\n assert img_turbojpeg.shape == (300, 400, 3)\n assert_array_equal(img_cv2, img_turbojpeg)\n\n # backend turbojpeg, channel order: rgb\n with open(self.img_path, 'rb') as f:\n img_bytes = f.read()\n img_rgb_turbojpeg = mmcv.imfrombytes(img_bytes, channel_order='rgb')\n assert img_rgb_turbojpeg.shape == (300, 400, 3)\n assert_array_equal(img_rgb_turbojpeg, img_cv2[:, :, ::-1])\n\n # backend turbojpeg, grayscale, decode as 3 channels\n with open(self.gray_img_path, 'rb') as f:\n img_bytes = f.read()\n gray_img_turbojpeg = mmcv.imfrombytes(img_bytes)\n assert gray_img_turbojpeg.shape == (300, 400, 3)\n assert_array_equal(gray_img_rgb_cv2, gray_img_turbojpeg)\n\n # backend turbojpeg, grayscale\n with open(self.gray_img_path, 'rb') as f:\n img_bytes = f.read()\n gray_img_turbojpeg = mmcv.imfrombytes(img_bytes, flag='grayscale')\n assert gray_img_turbojpeg.shape == (300, 400)\n assert_array_equal(gray_img_cv2, gray_img_turbojpeg)\n\n # backend turbojpeg, grayscale dim3\n with open(self.gray_img_dim3_path, 'rb') as f:\n img_bytes = f.read()\n gray_img_dim3_turbojpeg = mmcv.imfrombytes(img_bytes, flag='grayscale')\n assert gray_img_dim3_turbojpeg.shape == (300, 400)\n assert_array_equal(gray_img_dim3_cv2, gray_img_dim3_turbojpeg)\n\n mmcv.use_backend('cv2')\n\n with pytest.raises(ValueError):\n with open(self.img_path, 'rb') as f:\n img_bytes = f.read()\n mmcv.imfrombytes(img_bytes, backend='unsupported_backend')\n\n def test_imwrite(self):\n img = mmcv.imread(self.img_path)\n out_file = osp.join(tempfile.gettempdir(), 'mmcv_test.jpg')\n mmcv.imwrite(img, out_file)\n rewrite_img = mmcv.imread(out_file)\n os.remove(out_file)\n self.assert_img_equal(img, rewrite_img)\n\n # test petrel client\n with patch.object(\n PetrelBackend, 'put', return_value=None) as mock_method:\n ret = mmcv.imwrite(img, self.s3_path)\n ret_with_args = mmcv.imwrite(\n img, self.s3_path, file_client_args={'backend': 'petrel'})\n assert ret\n assert ret_with_args\n mock_method.assert_called()\n\n with pytest.raises(cv2.error):\n mmcv.imwrite(img, 'error_file.jppg')\n\n @patch('mmcv.image.io.TurboJPEG', None)\n def test_no_turbojpeg(self):\n with pytest.raises(ImportError):\n mmcv.use_backend('turbojpeg')\n\n mmcv.use_backend('cv2')\n\n @patch('mmcv.image.io.Image', None)\n def test_no_pillow(self):\n with pytest.raises(ImportError):\n mmcv.use_backend('pillow')\n\n mmcv.use_backend('cv2')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_image/test_photometric.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\n\nimport cv2\nimport numpy as np\nimport pytest\nfrom numpy.testing import assert_array_equal\n\nimport mmcv\n\n\nclass TestPhotometric:\n\n @classmethod\n def setup_class(cls):\n # the test img resolution is 400x300\n cls.img_path = osp.join(osp.dirname(__file__), '../data/color.jpg')\n cls.img = cv2.imread(cls.img_path)\n cls.mean = np.array([123.675, 116.28, 103.53], dtype=np.float32)\n cls.std = np.array([58.395, 57.12, 57.375], dtype=np.float32)\n\n def test_imnormalize(self):\n rgb_img = self.img[:, :, ::-1]\n baseline = (rgb_img - self.mean) / self.std\n img = mmcv.imnormalize(self.img, self.mean, self.std)\n assert np.allclose(img, baseline)\n assert id(img) != id(self.img)\n img = mmcv.imnormalize(rgb_img, self.mean, self.std, to_rgb=False)\n assert np.allclose(img, baseline)\n assert id(img) != id(rgb_img)\n\n def test_imnormalize_(self):\n img_for_normalize = np.float32(self.img)\n rgb_img_for_normalize = np.float32(self.img[:, :, ::-1])\n baseline = (rgb_img_for_normalize - self.mean) / self.std\n img = mmcv.imnormalize_(img_for_normalize, self.mean, self.std)\n assert np.allclose(img_for_normalize, baseline)\n assert id(img) == id(img_for_normalize)\n img = mmcv.imnormalize_(\n rgb_img_for_normalize, self.mean, self.std, to_rgb=False)\n assert np.allclose(img, baseline)\n assert id(img) == id(rgb_img_for_normalize)\n\n def test_imdenormalize(self):\n norm_img = (self.img[:, :, ::-1] - self.mean) / self.std\n rgb_baseline = (norm_img * self.std + self.mean)\n bgr_baseline = rgb_baseline[:, :, ::-1]\n img = mmcv.imdenormalize(norm_img, self.mean, self.std)\n assert np.allclose(img, bgr_baseline)\n img = mmcv.imdenormalize(norm_img, self.mean, self.std, to_bgr=False)\n assert np.allclose(img, rgb_baseline)\n\n def test_iminvert(self):\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img_r = np.array([[255, 127, 0], [254, 128, 1], [253, 126, 2]],\n dtype=np.uint8)\n assert_array_equal(mmcv.iminvert(img), img_r)\n\n def test_solarize(self):\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img_r = np.array([[0, 127, 0], [1, 127, 1], [2, 126, 2]],\n dtype=np.uint8)\n assert_array_equal(mmcv.solarize(img), img_r)\n img_r = np.array([[0, 127, 0], [1, 128, 1], [2, 126, 2]],\n dtype=np.uint8)\n assert_array_equal(mmcv.solarize(img, 100), img_r)\n\n def test_posterize(self):\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img_r = np.array([[0, 128, 128], [0, 0, 128], [0, 128, 128]],\n dtype=np.uint8)\n assert_array_equal(mmcv.posterize(img, 1), img_r)\n img_r = np.array([[0, 128, 224], [0, 96, 224], [0, 128, 224]],\n dtype=np.uint8)\n assert_array_equal(mmcv.posterize(img, 3), img_r)\n\n def test_adjust_color(self):\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img = np.stack([img, img, img], axis=-1)\n assert_array_equal(mmcv.adjust_color(img), img)\n img_gray = mmcv.bgr2gray(img)\n img_r = np.stack([img_gray, img_gray, img_gray], axis=-1)\n assert_array_equal(mmcv.adjust_color(img, 0), img_r)\n assert_array_equal(mmcv.adjust_color(img, 0, 1), img_r)\n assert_array_equal(\n mmcv.adjust_color(img, 0.5, 0.5),\n np.round(np.clip((img * 0.5 + img_r * 0.5), 0,\n 255)).astype(img.dtype))\n assert_array_equal(\n mmcv.adjust_color(img, 1, 1.5),\n np.round(np.clip(img * 1 + img_r * 1.5, 0, 255)).astype(img.dtype))\n assert_array_equal(\n mmcv.adjust_color(img, 0.8, -0.6, gamma=2),\n np.round(np.clip(img * 0.8 - 0.6 * img_r + 2, 0,\n 255)).astype(img.dtype))\n assert_array_equal(\n mmcv.adjust_color(img, 0.8, -0.6, gamma=-0.6),\n np.round(np.clip(img * 0.8 - 0.6 * img_r - 0.6, 0,\n 255)).astype(img.dtype))\n\n # test float type of image\n img = img.astype(np.float32)\n assert_array_equal(\n np.round(mmcv.adjust_color(img, 0.8, -0.6, gamma=-0.6)),\n np.round(np.clip(img * 0.8 - 0.6 * img_r - 0.6, 0, 255)))\n\n def test_imequalize(self, nb_rand_test=100):\n\n def _imequalize(img):\n # equalize the image using PIL.ImageOps.equalize\n from PIL import ImageOps, Image\n img = Image.fromarray(img)\n equalized_img = np.asarray(ImageOps.equalize(img))\n return equalized_img\n\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img = np.stack([img, img, img], axis=-1)\n equalized_img = mmcv.imequalize(img)\n assert_array_equal(equalized_img, _imequalize(img))\n\n # test equalize with case step=0\n img = np.array([[0, 0, 0], [120, 120, 120], [255, 255, 255]],\n dtype=np.uint8)\n img = np.stack([img, img, img], axis=-1)\n assert_array_equal(mmcv.imequalize(img), img)\n\n # test equalize with randomly sampled image.\n for _ in range(nb_rand_test):\n img = np.clip(np.random.normal(0, 1, (256, 256, 3)) * 260, 0,\n 255).astype(np.uint8)\n equalized_img = mmcv.imequalize(img)\n assert_array_equal(equalized_img, _imequalize(img))\n\n def test_adjust_brightness(self, nb_rand_test=100):\n\n def _adjust_brightness(img, factor):\n # adjust the brightness of image using\n # PIL.ImageEnhance.Brightness\n from PIL.ImageEnhance import Brightness\n from PIL import Image\n img = Image.fromarray(img)\n brightened_img = Brightness(img).enhance(factor)\n return np.asarray(brightened_img)\n\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img = np.stack([img, img, img], axis=-1)\n # test case with factor 1.0\n assert_array_equal(mmcv.adjust_brightness(img, 1.), img)\n # test case with factor 0.0\n assert_array_equal(mmcv.adjust_brightness(img, 0.), np.zeros_like(img))\n # test adjust_brightness with randomly sampled images and factors.\n for _ in range(nb_rand_test):\n img = np.clip(\n np.random.uniform(0, 1, (1000, 1200, 3)) * 260, 0,\n 255).astype(np.uint8)\n factor = np.random.uniform() + np.random.choice([0, 1])\n np.testing.assert_allclose(\n mmcv.adjust_brightness(img, factor).astype(np.int32),\n _adjust_brightness(img, factor).astype(np.int32),\n rtol=0,\n atol=1)\n\n def test_adjust_contrast(self, nb_rand_test=100):\n\n def _adjust_contrast(img, factor):\n from PIL.ImageEnhance import Contrast\n from PIL import Image\n # Image.fromarray defaultly supports RGB, not BGR.\n # convert from BGR to RGB\n img = Image.fromarray(img[..., ::-1], mode='RGB')\n contrasted_img = Contrast(img).enhance(factor)\n # convert from RGB to BGR\n return np.asarray(contrasted_img)[..., ::-1]\n\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img = np.stack([img, img, img], axis=-1)\n # test case with factor 1.0\n assert_array_equal(mmcv.adjust_contrast(img, 1.), img)\n # test case with factor 0.0\n assert_array_equal(\n mmcv.adjust_contrast(img, 0.), _adjust_contrast(img, 0.))\n # test adjust_contrast with randomly sampled images and factors.\n for _ in range(nb_rand_test):\n img = np.clip(\n np.random.uniform(0, 1, (1200, 1000, 3)) * 260, 0,\n 255).astype(np.uint8)\n factor = np.random.uniform() + np.random.choice([0, 1])\n # Note the gap (less_equal 1) between PIL.ImageEnhance.Contrast\n # and mmcv.adjust_contrast comes from the gap that converts from\n # a color image to gray image using mmcv or PIL.\n np.testing.assert_allclose(\n mmcv.adjust_contrast(img, factor).astype(np.int32),\n _adjust_contrast(img, factor).astype(np.int32),\n rtol=0,\n atol=1)\n\n def test_auto_contrast(self, nb_rand_test=100):\n\n def _auto_contrast(img, cutoff=0):\n from PIL.ImageOps import autocontrast\n from PIL import Image\n # Image.fromarray defaultly supports RGB, not BGR.\n # convert from BGR to RGB\n img = Image.fromarray(img[..., ::-1], mode='RGB')\n contrasted_img = autocontrast(img, cutoff)\n # convert from RGB to BGR\n return np.asarray(contrasted_img)[..., ::-1]\n\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img = np.stack([img, img, img], axis=-1)\n\n # test case without cut-off\n assert_array_equal(mmcv.auto_contrast(img), _auto_contrast(img))\n # test case with cut-off as int\n assert_array_equal(\n mmcv.auto_contrast(img, 10), _auto_contrast(img, 10))\n # test case with cut-off as float\n assert_array_equal(\n mmcv.auto_contrast(img, 12.5), _auto_contrast(img, 12.5))\n # test case with cut-off as tuple\n assert_array_equal(\n mmcv.auto_contrast(img, (10, 10)), _auto_contrast(img, 10))\n # test case with cut-off with sum over 100\n assert_array_equal(\n mmcv.auto_contrast(img, 60), _auto_contrast(img, 60))\n\n # test auto_contrast with randomly sampled images and factors.\n for _ in range(nb_rand_test):\n img = np.clip(\n np.random.uniform(0, 1, (1200, 1000, 3)) * 260, 0,\n 255).astype(np.uint8)\n # cut-offs are not set as tuple since in `build.yml`, pillow 6.2.2\n # is installed, which does not support setting low cut-off and high\n # cut-off differently.\n # With pillow above 8.0.0, cutoff can be set as tuple\n cutoff = np.random.rand() * 100\n assert_array_equal(\n mmcv.auto_contrast(img, cutoff), _auto_contrast(img, cutoff))\n\n def test_adjust_sharpness(self, nb_rand_test=100):\n\n def _adjust_sharpness(img, factor):\n # adjust the sharpness of image using\n # PIL.ImageEnhance.Sharpness\n from PIL.ImageEnhance import Sharpness\n from PIL import Image\n img = Image.fromarray(img)\n sharpened_img = Sharpness(img).enhance(factor)\n return np.asarray(sharpened_img)\n\n img = np.array([[0, 128, 255], [1, 127, 254], [2, 129, 253]],\n dtype=np.uint8)\n img = np.stack([img, img, img], axis=-1)\n\n # test case with invalid type of kernel\n with pytest.raises(AssertionError):\n mmcv.adjust_sharpness(img, 1., kernel=1.)\n # test case with invalid shape of kernel\n kernel = np.ones((3, 3, 3))\n with pytest.raises(AssertionError):\n mmcv.adjust_sharpness(img, 1., kernel=kernel)\n # test case with all-zero kernel, factor 0.0\n kernel = np.zeros((3, 3))\n assert_array_equal(\n mmcv.adjust_sharpness(img, 0., kernel=kernel), np.zeros_like(img))\n\n # test case with factor 1.0\n assert_array_equal(mmcv.adjust_sharpness(img, 1.), img)\n # test adjust_sharpness with randomly sampled images and factors.\n for _ in range(nb_rand_test):\n img = np.clip(\n np.random.uniform(0, 1, (1000, 1200, 3)) * 260, 0,\n 255).astype(np.uint8)\n factor = np.random.uniform()\n # Note the gap between PIL.ImageEnhance.Sharpness and\n # mmcv.adjust_sharpness mainly comes from the difference ways of\n # handling img edges when applying filters\n np.testing.assert_allclose(\n mmcv.adjust_sharpness(img, factor).astype(np.int32)[1:-1,\n 1:-1],\n _adjust_sharpness(img, factor).astype(np.int32)[1:-1, 1:-1],\n rtol=0,\n atol=1)\n\n def test_adjust_lighting(self):\n img = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]).astype(np.uint8)\n img = np.stack([img, img, img], axis=-1)\n\n # eigval and eigvec must be np.ndarray\n with pytest.raises(AssertionError):\n mmcv.adjust_lighting(img, 1, np.ones((3, 1)))\n with pytest.raises(AssertionError):\n mmcv.adjust_lighting(img, np.array([1]), (1, 1, 1))\n # we must have the same number of eigval and eigvec\n with pytest.raises(AssertionError):\n mmcv.adjust_lighting(img, np.array([1]), np.eye(2))\n with pytest.raises(AssertionError):\n mmcv.adjust_lighting(img, np.array([1]), np.array([1]))\n\n img_adjusted = mmcv.adjust_lighting(\n img,\n np.random.normal(0, 1, 2),\n np.random.normal(0, 1, (3, 2)),\n alphastd=0.)\n assert_array_equal(img_adjusted, img)\n\n def test_lut_transform(self):\n lut_table = np.array(list(range(256)))\n\n # test assertion image values should between 0 and 255.\n with pytest.raises(AssertionError):\n mmcv.lut_transform(np.array([256]), lut_table)\n with pytest.raises(AssertionError):\n mmcv.lut_transform(np.array([-1]), lut_table)\n\n # test assertion lut_table should be ndarray with shape (256, )\n with pytest.raises(AssertionError):\n mmcv.lut_transform(np.array([0]), list(range(256)))\n with pytest.raises(AssertionError):\n mmcv.lut_transform(np.array([1]), np.array(list(range(257))))\n\n img = mmcv.lut_transform(self.img, lut_table)\n baseline = cv2.LUT(self.img, lut_table)\n assert np.allclose(img, baseline)\n\n input_img = np.array(\n [[[0, 128, 255], [255, 128, 0]], [[0, 128, 255], [255, 128, 0]]],\n dtype=float)\n img = mmcv.lut_transform(input_img, lut_table)\n baseline = cv2.LUT(np.array(input_img, dtype=np.uint8), lut_table)\n assert np.allclose(img, baseline)\n\n input_img = np.random.randint(0, 256, size=(7, 8, 9, 10, 11))\n img = mmcv.lut_transform(input_img, lut_table)\n baseline = cv2.LUT(np.array(input_img, dtype=np.uint8), lut_table)\n assert np.allclose(img, baseline)\n\n def test_clahe(self):\n\n def _clahe(img, clip_limit=40.0, tile_grid_size=(8, 8)):\n clahe = cv2.createCLAHE(clip_limit, tile_grid_size)\n return clahe.apply(np.array(img, dtype=np.uint8))\n\n # test assertion image should have the right shape\n with pytest.raises(AssertionError):\n mmcv.clahe(self.img)\n\n # test assertion tile_grid_size should be a tuple with 2 integers\n with pytest.raises(AssertionError):\n mmcv.clahe(self.img[:, :, 0], tile_grid_size=(8.0, 8.0))\n with pytest.raises(AssertionError):\n mmcv.clahe(self.img[:, :, 0], tile_grid_size=(8, 8, 8))\n with pytest.raises(AssertionError):\n mmcv.clahe(self.img[:, :, 0], tile_grid_size=[8, 8])\n\n # test with different channels\n for i in range(self.img.shape[-1]):\n img = mmcv.clahe(self.img[:, :, i])\n img_std = _clahe(self.img[:, :, i])\n assert np.allclose(img, img_std)\n assert id(img) != id(self.img[:, :, i])\n assert id(img_std) != id(self.img[:, :, i])\n\n # test case with clip_limit=1.2\n for i in range(self.img.shape[-1]):\n img = mmcv.clahe(self.img[:, :, i], 1.2)\n img_std = _clahe(self.img[:, :, i], 1.2)\n assert np.allclose(img, img_std)\n assert id(img) != id(self.img[:, :, i])\n assert id(img_std) != id(self.img[:, :, i])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_load_model_zoo.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport os.path as osp\nfrom unittest.mock import patch\n\nimport pytest\n\nimport mmcv\nfrom mmcv.runner.checkpoint import (DEFAULT_CACHE_DIR, ENV_MMCV_HOME,\n ENV_XDG_CACHE_HOME, _get_mmcv_home,\n _load_checkpoint,\n get_deprecated_model_names,\n get_external_models)\nfrom mmcv.utils import TORCH_VERSION\n\n\n@patch('mmcv.__path__', [osp.join(osp.dirname(__file__), 'data/')])\ndef test_set_mmcv_home():\n os.environ.pop(ENV_MMCV_HOME, None)\n mmcv_home = osp.join(osp.dirname(__file__), 'data/model_zoo/mmcv_home/')\n os.environ[ENV_MMCV_HOME] = mmcv_home\n assert _get_mmcv_home() == mmcv_home\n\n\n@patch('mmcv.__path__', [osp.join(osp.dirname(__file__), 'data/')])\ndef test_default_mmcv_home():\n os.environ.pop(ENV_MMCV_HOME, None)\n os.environ.pop(ENV_XDG_CACHE_HOME, None)\n assert _get_mmcv_home() == os.path.expanduser(\n os.path.join(DEFAULT_CACHE_DIR, 'mmcv'))\n model_urls = get_external_models()\n assert model_urls == mmcv.load(\n osp.join(mmcv.__path__[0], 'model_zoo/open_mmlab.json'))\n\n\n@patch('mmcv.__path__', [osp.join(osp.dirname(__file__), 'data/')])\ndef test_get_external_models():\n os.environ.pop(ENV_MMCV_HOME, None)\n mmcv_home = osp.join(osp.dirname(__file__), 'data/model_zoo/mmcv_home/')\n os.environ[ENV_MMCV_HOME] = mmcv_home\n ext_urls = get_external_models()\n assert ext_urls == {\n 'train': 'https://localhost/train.pth',\n 'test': 'test.pth',\n 'val': 'val.pth',\n 'train_empty': 'train.pth'\n }\n\n\n@patch('mmcv.__path__', [osp.join(osp.dirname(__file__), 'data/')])\ndef test_get_deprecated_models():\n os.environ.pop(ENV_MMCV_HOME, None)\n mmcv_home = osp.join(osp.dirname(__file__), 'data/model_zoo/mmcv_home/')\n os.environ[ENV_MMCV_HOME] = mmcv_home\n dep_urls = get_deprecated_model_names()\n assert dep_urls == {\n 'train_old': 'train',\n 'test_old': 'test',\n }\n\n\ndef load_from_http(url, map_location=None):\n return 'url:' + url\n\n\ndef load_url(url, map_location=None, model_dir=None):\n return load_from_http(url)\n\n\ndef load(filepath, map_location=None):\n return 'local:' + filepath\n\n\n@patch('mmcv.__path__', [osp.join(osp.dirname(__file__), 'data/')])\n@patch('mmcv.runner.checkpoint.load_from_http', load_from_http)\n@patch('mmcv.runner.checkpoint.load_url', load_url)\n@patch('torch.load', load)\ndef test_load_external_url():\n # test modelzoo://\n url = _load_checkpoint('modelzoo://resnet50')\n if TORCH_VERSION < '1.9.0':\n assert url == ('url:https://download.pytorch.org/models/resnet50-19c8e'\n '357.pth')\n else:\n # filename of checkpoint is renamed in torch1.9.0\n assert url == ('url:https://download.pytorch.org/models/resnet50-0676b'\n 'a61.pth')\n\n # test torchvision://\n url = _load_checkpoint('torchvision://resnet50')\n if TORCH_VERSION < '1.9.0':\n assert url == ('url:https://download.pytorch.org/models/resnet50-19c8e'\n '357.pth')\n else:\n # filename of checkpoint is renamed in torch1.9.0\n assert url == ('url:https://download.pytorch.org/models/resnet50-0676b'\n 'a61.pth')\n\n # test open-mmlab:// with default MMCV_HOME\n os.environ.pop(ENV_MMCV_HOME, None)\n os.environ.pop(ENV_XDG_CACHE_HOME, None)\n url = _load_checkpoint('open-mmlab://train')\n assert url == 'url:https://localhost/train.pth'\n\n # test open-mmlab:// with deprecated model name\n os.environ.pop(ENV_MMCV_HOME, None)\n os.environ.pop(ENV_XDG_CACHE_HOME, None)\n with pytest.warns(\n Warning,\n match='open-mmlab://train_old is deprecated in favor of '\n 'open-mmlab://train'):\n url = _load_checkpoint('open-mmlab://train_old')\n assert url == 'url:https://localhost/train.pth'\n\n # test openmmlab:// with deprecated model name\n os.environ.pop(ENV_MMCV_HOME, None)\n os.environ.pop(ENV_XDG_CACHE_HOME, None)\n with pytest.warns(\n Warning,\n match='openmmlab://train_old is deprecated in favor of '\n 'openmmlab://train'):\n url = _load_checkpoint('openmmlab://train_old')\n assert url == 'url:https://localhost/train.pth'\n\n # test open-mmlab:// with user-defined MMCV_HOME\n os.environ.pop(ENV_MMCV_HOME, None)\n mmcv_home = osp.join(osp.dirname(__file__), 'data/model_zoo/mmcv_home')\n os.environ[ENV_MMCV_HOME] = mmcv_home\n url = _load_checkpoint('open-mmlab://train')\n assert url == 'url:https://localhost/train.pth'\n with pytest.raises(FileNotFoundError, match='train.pth can not be found.'):\n _load_checkpoint('open-mmlab://train_empty')\n url = _load_checkpoint('open-mmlab://test')\n assert url == f'local:{osp.join(_get_mmcv_home(), \"test.pth\")}'\n url = _load_checkpoint('open-mmlab://val')\n assert url == f'local:{osp.join(_get_mmcv_home(), \"val.pth\")}'\n\n # test http:// https://\n url = _load_checkpoint('http://localhost/train.pth')\n assert url == 'url:http://localhost/train.pth'\n\n # test local file\n with pytest.raises(FileNotFoundError, match='train.pth can not be found.'):\n _load_checkpoint('train.pth')\n url = _load_checkpoint(osp.join(_get_mmcv_home(), 'test.pth'))\n assert url == f'local:{osp.join(_get_mmcv_home(), \"test.pth\")}'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_active_rotated_filter.py", "content": "import numpy as np\nimport pytest\nimport torch\n\nfrom mmcv.ops import active_rotated_filter\n\nnp_feature = np.array([[[[[-1.4934e-01, 1.1341e+00, -1.6241e-01],\n [-1.0986e+00, -1.1463e+00, -1.3176e+00],\n [1.4808e+00, 7.6572e-01, -1.4548e+00]]]],\n [[[[1.9370e+00, 6.2799e-01, 2.5834e-02],\n [-1.4242e+00, 7.6566e-01, 1.0015e+00],\n [9.8669e-01, 4.1356e-01, 6.1068e-01]]]],\n [[[[1.4565e+00, 1.4960e+00, 2.4339e-01],\n [-2.2484e-01, 7.5942e-01, -8.1184e-01],\n [-1.7077e+00, 1.0658e+00, 3.8311e-01]]]],\n [[[[8.4734e-01, 1.0904e+00, 2.4356e+00],\n [9.5822e-01, 2.2260e-01, -2.4450e-01],\n [-1.5078e+00, 7.0902e-02, -1.5921e+00]]]],\n [[[[2.1173e+00, -7.3524e-01, 1.8888e+00],\n [1.0169e+00, 4.7033e-01, -1.0875e+00],\n [-1.0736e+00, -5.2245e-01, -2.8733e-01]]]],\n [[[[-5.6433e-01, 1.5835e+00, -1.5826e+00],\n [-8.8974e-01, -4.3128e-01, -2.2423e-01],\n [1.6552e-03, -1.7292e+00, 2.6639e-01]]]],\n [[[[-1.2951e-01, 1.3493e+00, -1.9329e+00],\n [5.6248e-01, -5.1189e-01, 1.3614e+00],\n [3.3680e-01, -8.7148e-01, 5.0592e-01]]]],\n [[[[1.6781e-02, -8.3929e-01, 1.2060e+00],\n [-1.0764e+00, 4.7821e-01, 1.5342e+00],\n [-4.4542e-01, -1.8606e+00, 3.0827e-01]]]]])\n\nnp_indices = np.array([[[[1, 2, 3, 6, 9, 8, 7, 4], [2, 3, 6, 9, 8, 7, 4, 1],\n [3, 6, 9, 8, 7, 4, 1, 2]],\n [[4, 1, 2, 3, 6, 9, 8, 7], [5, 5, 5, 5, 5, 5, 5, 5],\n [6, 9, 8, 7, 4, 1, 2, 3]],\n [[7, 4, 1, 2, 3, 6, 9, 8], [8, 7, 4, 1, 2, 3, 6, 9],\n [9, 8, 7, 4, 1, 2, 3, 6]]]])\n\nexpected_output = np.array([[[[-1.4934e-01, 1.1341e+00, -1.6241e-01],\n [-1.0986e+00, -1.1463e+00, -1.3176e+00],\n [1.4808e+00, 7.6572e-01, -1.4548e+00]]],\n [[[-1.0986e+00, -1.4934e-01, 1.1341e+00],\n [1.4808e+00, -1.1463e+00, -1.6241e-01],\n [7.6572e-01, -1.4548e+00, -1.3176e+00]]],\n [[[1.4808e+00, -1.0986e+00, -1.4934e-01],\n [7.6572e-01, -1.1463e+00, 1.1341e+00],\n [-1.4548e+00, -1.3176e+00, -1.6241e-01]]],\n [[[7.6572e-01, 1.4808e+00, -1.0986e+00],\n [-1.4548e+00, -1.1463e+00, -1.4934e-01],\n [-1.3176e+00, -1.6241e-01, 1.1341e+00]]],\n [[[-1.4548e+00, 7.6572e-01, 1.4808e+00],\n [-1.3176e+00, -1.1463e+00, -1.0986e+00],\n [-1.6241e-01, 1.1341e+00, -1.4934e-01]]],\n [[[-1.3176e+00, -1.4548e+00, 7.6572e-01],\n [-1.6241e-01, -1.1463e+00, 1.4808e+00],\n [1.1341e+00, -1.4934e-01, -1.0986e+00]]],\n [[[-1.6241e-01, -1.3176e+00, -1.4548e+00],\n [1.1341e+00, -1.1463e+00, 7.6572e-01],\n [-1.4934e-01, -1.0986e+00, 1.4808e+00]]],\n [[[1.1341e+00, -1.6241e-01, -1.3176e+00],\n [-1.4934e-01, -1.1463e+00, -1.4548e+00],\n [-1.0986e+00, 1.4808e+00, 7.6572e-01]]],\n [[[1.9370e+00, 6.2799e-01, 2.5834e-02],\n [-1.4242e+00, 7.6566e-01, 1.0015e+00],\n [9.8669e-01, 4.1356e-01, 6.1068e-01]]],\n [[[-1.4242e+00, 1.9370e+00, 6.2799e-01],\n [9.8669e-01, 7.6566e-01, 2.5834e-02],\n [4.1356e-01, 6.1068e-01, 1.0015e+00]]],\n [[[9.8669e-01, -1.4242e+00, 1.9370e+00],\n [4.1356e-01, 7.6566e-01, 6.2799e-01],\n [6.1068e-01, 1.0015e+00, 2.5834e-02]]],\n [[[4.1356e-01, 9.8669e-01, -1.4242e+00],\n [6.1068e-01, 7.6566e-01, 1.9370e+00],\n [1.0015e+00, 2.5834e-02, 6.2799e-01]]],\n [[[6.1068e-01, 4.1356e-01, 9.8669e-01],\n [1.0015e+00, 7.6566e-01, -1.4242e+00],\n [2.5834e-02, 6.2799e-01, 1.9370e+00]]],\n [[[1.0015e+00, 6.1068e-01, 4.1356e-01],\n [2.5834e-02, 7.6566e-01, 9.8669e-01],\n [6.2799e-01, 1.9370e+00, -1.4242e+00]]],\n [[[2.5834e-02, 1.0015e+00, 6.1068e-01],\n [6.2799e-01, 7.6566e-01, 4.1356e-01],\n [1.9370e+00, -1.4242e+00, 9.8669e-01]]],\n [[[6.2799e-01, 2.5834e-02, 1.0015e+00],\n [1.9370e+00, 7.6566e-01, 6.1068e-01],\n [-1.4242e+00, 9.8669e-01, 4.1356e-01]]],\n [[[1.4565e+00, 1.4960e+00, 2.4339e-01],\n [-2.2484e-01, 7.5942e-01, -8.1184e-01],\n [-1.7077e+00, 1.0658e+00, 3.8311e-01]]],\n [[[-2.2484e-01, 1.4565e+00, 1.4960e+00],\n [-1.7077e+00, 7.5942e-01, 2.4339e-01],\n [1.0658e+00, 3.8311e-01, -8.1184e-01]]],\n [[[-1.7077e+00, -2.2484e-01, 1.4565e+00],\n [1.0658e+00, 7.5942e-01, 1.4960e+00],\n [3.8311e-01, -8.1184e-01, 2.4339e-01]]],\n [[[1.0658e+00, -1.7077e+00, -2.2484e-01],\n [3.8311e-01, 7.5942e-01, 1.4565e+00],\n [-8.1184e-01, 2.4339e-01, 1.4960e+00]]],\n [[[3.8311e-01, 1.0658e+00, -1.7077e+00],\n [-8.1184e-01, 7.5942e-01, -2.2484e-01],\n [2.4339e-01, 1.4960e+00, 1.4565e+00]]],\n [[[-8.1184e-01, 3.8311e-01, 1.0658e+00],\n [2.4339e-01, 7.5942e-01, -1.7077e+00],\n [1.4960e+00, 1.4565e+00, -2.2484e-01]]],\n [[[2.4339e-01, -8.1184e-01, 3.8311e-01],\n [1.4960e+00, 7.5942e-01, 1.0658e+00],\n [1.4565e+00, -2.2484e-01, -1.7077e+00]]],\n [[[1.4960e+00, 2.4339e-01, -8.1184e-01],\n [1.4565e+00, 7.5942e-01, 3.8311e-01],\n [-2.2484e-01, -1.7077e+00, 1.0658e+00]]],\n [[[8.4734e-01, 1.0904e+00, 2.4356e+00],\n [9.5822e-01, 2.2260e-01, -2.4450e-01],\n [-1.5078e+00, 7.0902e-02, -1.5921e+00]]],\n [[[9.5822e-01, 8.4734e-01, 1.0904e+00],\n [-1.5078e+00, 2.2260e-01, 2.4356e+00],\n [7.0902e-02, -1.5921e+00, -2.4450e-01]]],\n [[[-1.5078e+00, 9.5822e-01, 8.4734e-01],\n [7.0902e-02, 2.2260e-01, 1.0904e+00],\n [-1.5921e+00, -2.4450e-01, 2.4356e+00]]],\n [[[7.0902e-02, -1.5078e+00, 9.5822e-01],\n [-1.5921e+00, 2.2260e-01, 8.4734e-01],\n [-2.4450e-01, 2.4356e+00, 1.0904e+00]]],\n [[[-1.5921e+00, 7.0902e-02, -1.5078e+00],\n [-2.4450e-01, 2.2260e-01, 9.5822e-01],\n [2.4356e+00, 1.0904e+00, 8.4734e-01]]],\n [[[-2.4450e-01, -1.5921e+00, 7.0902e-02],\n [2.4356e+00, 2.2260e-01, -1.5078e+00],\n [1.0904e+00, 8.4734e-01, 9.5822e-01]]],\n [[[2.4356e+00, -2.4450e-01, -1.5921e+00],\n [1.0904e+00, 2.2260e-01, 7.0902e-02],\n [8.4734e-01, 9.5822e-01, -1.5078e+00]]],\n [[[1.0904e+00, 2.4356e+00, -2.4450e-01],\n [8.4734e-01, 2.2260e-01, -1.5921e+00],\n [9.5822e-01, -1.5078e+00, 7.0902e-02]]],\n [[[2.1173e+00, -7.3524e-01, 1.8888e+00],\n [1.0169e+00, 4.7033e-01, -1.0875e+00],\n [-1.0736e+00, -5.2245e-01, -2.8733e-01]]],\n [[[1.0169e+00, 2.1173e+00, -7.3524e-01],\n [-1.0736e+00, 4.7033e-01, 1.8888e+00],\n [-5.2245e-01, -2.8733e-01, -1.0875e+00]]],\n [[[-1.0736e+00, 1.0169e+00, 2.1173e+00],\n [-5.2245e-01, 4.7033e-01, -7.3524e-01],\n [-2.8733e-01, -1.0875e+00, 1.8888e+00]]],\n [[[-5.2245e-01, -1.0736e+00, 1.0169e+00],\n [-2.8733e-01, 4.7033e-01, 2.1173e+00],\n [-1.0875e+00, 1.8888e+00, -7.3524e-01]]],\n [[[-2.8733e-01, -5.2245e-01, -1.0736e+00],\n [-1.0875e+00, 4.7033e-01, 1.0169e+00],\n [1.8888e+00, -7.3524e-01, 2.1173e+00]]],\n [[[-1.0875e+00, -2.8733e-01, -5.2245e-01],\n [1.8888e+00, 4.7033e-01, -1.0736e+00],\n [-7.3524e-01, 2.1173e+00, 1.0169e+00]]],\n [[[1.8888e+00, -1.0875e+00, -2.8733e-01],\n [-7.3524e-01, 4.7033e-01, -5.2245e-01],\n [2.1173e+00, 1.0169e+00, -1.0736e+00]]],\n [[[-7.3524e-01, 1.8888e+00, -1.0875e+00],\n [2.1173e+00, 4.7033e-01, -2.8733e-01],\n [1.0169e+00, -1.0736e+00, -5.2245e-01]]],\n [[[-5.6433e-01, 1.5835e+00, -1.5826e+00],\n [-8.8974e-01, -4.3128e-01, -2.2423e-01],\n [1.6552e-03, -1.7292e+00, 2.6639e-01]]],\n [[[-8.8974e-01, -5.6433e-01, 1.5835e+00],\n [1.6552e-03, -4.3128e-01, -1.5826e+00],\n [-1.7292e+00, 2.6639e-01, -2.2423e-01]]],\n [[[1.6552e-03, -8.8974e-01, -5.6433e-01],\n [-1.7292e+00, -4.3128e-01, 1.5835e+00],\n [2.6639e-01, -2.2423e-01, -1.5826e+00]]],\n [[[-1.7292e+00, 1.6552e-03, -8.8974e-01],\n [2.6639e-01, -4.3128e-01, -5.6433e-01],\n [-2.2423e-01, -1.5826e+00, 1.5835e+00]]],\n [[[2.6639e-01, -1.7292e+00, 1.6552e-03],\n [-2.2423e-01, -4.3128e-01, -8.8974e-01],\n [-1.5826e+00, 1.5835e+00, -5.6433e-01]]],\n [[[-2.2423e-01, 2.6639e-01, -1.7292e+00],\n [-1.5826e+00, -4.3128e-01, 1.6552e-03],\n [1.5835e+00, -5.6433e-01, -8.8974e-01]]],\n [[[-1.5826e+00, -2.2423e-01, 2.6639e-01],\n [1.5835e+00, -4.3128e-01, -1.7292e+00],\n [-5.6433e-01, -8.8974e-01, 1.6552e-03]]],\n [[[1.5835e+00, -1.5826e+00, -2.2423e-01],\n [-5.6433e-01, -4.3128e-01, 2.6639e-01],\n [-8.8974e-01, 1.6552e-03, -1.7292e+00]]],\n [[[-1.2951e-01, 1.3493e+00, -1.9329e+00],\n [5.6248e-01, -5.1189e-01, 1.3614e+00],\n [3.3680e-01, -8.7148e-01, 5.0592e-01]]],\n [[[5.6248e-01, -1.2951e-01, 1.3493e+00],\n [3.3680e-01, -5.1189e-01, -1.9329e+00],\n [-8.7148e-01, 5.0592e-01, 1.3614e+00]]],\n [[[3.3680e-01, 5.6248e-01, -1.2951e-01],\n [-8.7148e-01, -5.1189e-01, 1.3493e+00],\n [5.0592e-01, 1.3614e+00, -1.9329e+00]]],\n [[[-8.7148e-01, 3.3680e-01, 5.6248e-01],\n [5.0592e-01, -5.1189e-01, -1.2951e-01],\n [1.3614e+00, -1.9329e+00, 1.3493e+00]]],\n [[[5.0592e-01, -8.7148e-01, 3.3680e-01],\n [1.3614e+00, -5.1189e-01, 5.6248e-01],\n [-1.9329e+00, 1.3493e+00, -1.2951e-01]]],\n [[[1.3614e+00, 5.0592e-01, -8.7148e-01],\n [-1.9329e+00, -5.1189e-01, 3.3680e-01],\n [1.3493e+00, -1.2951e-01, 5.6248e-01]]],\n [[[-1.9329e+00, 1.3614e+00, 5.0592e-01],\n [1.3493e+00, -5.1189e-01, -8.7148e-01],\n [-1.2951e-01, 5.6248e-01, 3.3680e-01]]],\n [[[1.3493e+00, -1.9329e+00, 1.3614e+00],\n [-1.2951e-01, -5.1189e-01, 5.0592e-01],\n [5.6248e-01, 3.3680e-01, -8.7148e-01]]],\n [[[1.6781e-02, -8.3929e-01, 1.2060e+00],\n [-1.0764e+00, 4.7821e-01, 1.5342e+00],\n [-4.4542e-01, -1.8606e+00, 3.0827e-01]]],\n [[[-1.0764e+00, 1.6781e-02, -8.3929e-01],\n [-4.4542e-01, 4.7821e-01, 1.2060e+00],\n [-1.8606e+00, 3.0827e-01, 1.5342e+00]]],\n [[[-4.4542e-01, -1.0764e+00, 1.6781e-02],\n [-1.8606e+00, 4.7821e-01, -8.3929e-01],\n [3.0827e-01, 1.5342e+00, 1.2060e+00]]],\n [[[-1.8606e+00, -4.4542e-01, -1.0764e+00],\n [3.0827e-01, 4.7821e-01, 1.6781e-02],\n [1.5342e+00, 1.2060e+00, -8.3929e-01]]],\n [[[3.0827e-01, -1.8606e+00, -4.4542e-01],\n [1.5342e+00, 4.7821e-01, -1.0764e+00],\n [1.2060e+00, -8.3929e-01, 1.6781e-02]]],\n [[[1.5342e+00, 3.0827e-01, -1.8606e+00],\n [1.2060e+00, 4.7821e-01, -4.4542e-01],\n [-8.3929e-01, 1.6781e-02, -1.0764e+00]]],\n [[[1.2060e+00, 1.5342e+00, 3.0827e-01],\n [-8.3929e-01, 4.7821e-01, -1.8606e+00],\n [1.6781e-02, -1.0764e+00, -4.4542e-01]]],\n [[[-8.3929e-01, 1.2060e+00, 1.5342e+00],\n [1.6781e-02, 4.7821e-01, 3.0827e-01],\n [-1.0764e+00, -4.4542e-01, -1.8606e+00]]]])\n\nexpected_grad = np.array([[[[[8., 8., 8.], [8., 8., 8.], [8., 8., 8.]]]],\n [[[[8., 8., 8.], [8., 8., 8.], [8., 8., 8.]]]],\n [[[[8., 8., 8.], [8., 8., 8.], [8., 8., 8.]]]],\n [[[[8., 8., 8.], [8., 8., 8.], [8., 8., 8.]]]],\n [[[[8., 8., 8.], [8., 8., 8.], [8., 8., 8.]]]],\n [[[[8., 8., 8.], [8., 8., 8.], [8., 8., 8.]]]],\n [[[[8., 8., 8.], [8., 8., 8.], [8., 8., 8.]]]],\n [[[[8., 8., 8.], [8., 8., 8.], [8., 8., 8.]]]]])\n\n\n@pytest.mark.parametrize('device', [\n 'cpu',\n pytest.param(\n 'cuda',\n marks=pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')),\n])\ndef test_active_rotated_filter(device):\n feature = torch.tensor(\n np_feature, dtype=torch.float, device=device, requires_grad=True)\n indices = torch.tensor(np_indices, dtype=torch.int, device=device)\n output = active_rotated_filter(feature, indices)\n output.backward(torch.ones_like(output))\n assert np.allclose(output.data.cpu().numpy(), expected_output, atol=1e-3)\n assert np.allclose(\n feature.grad.data.cpu().numpy(), expected_grad, atol=1e-3)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_assign_score_withk.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import assign_score_withk\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_paconv_assign_scores():\n scores = torch.tensor([[[[0.06947571, 0.6065746], [0.28462553, 0.8378516],\n [0.7595994, 0.97220325], [0.519155, 0.766185]],\n [[0.15348864, 0.6051019], [0.21510637, 0.31916398],\n [0.00236845, 0.5842595], [0.6783676, 0.5216348]]],\n [[[0.23089725, 0.5568468], [0.7405102, 0.06438422],\n [0.6887394, 0.22089851], [0.0502342, 0.79228795]],\n [[0.44883424, 0.15427643],\n [0.13817799, 0.34856772], [0.7989621, 0.33788306],\n [0.15699774, 0.7693662]]]]).float().cuda()\n scores.requires_grad_()\n points = torch.tensor([[[[0.06001121, 0.92963666, 0.5753327, 0.7251477],\n [0.53563064, 0.23129565, 0.92366195, 0.44261628]],\n [[0.5770022, 0.56625944, 0.23560429, 0.11178821],\n [0.7735967, 0.95678777, 0.25468266, 0.02895975]],\n [[0.0589869, 0.09017515, 0.5977862, 0.02797985],\n [0.603862, 0.35991007, 0.85761684, 0.3096559]],\n [[0.22359002, 0.13983732, 0.5544243, 0.68863827],\n [0.85646236, 0.75651926, 0.8638947, 0.83600986]],\n [[0.45424145, 0.27458847, 0.6456112, 0.47162914],\n [0.15773582, 0.47645122, 0.79964715, 0.3323908]],\n [[0.8351399, 0.84696376, 0.9431732, 0.29418713],\n [0.77168906, 0.6996871, 0.19354361, 0.03392768]],\n [[0.30976456, 0.7074133, 0.581795, 0.976677],\n [0.69656056, 0.07199162, 0.4708506, 0.29117996]],\n [[0.5829035, 0.30201727, 0.76556486, 0.0935446],\n [0.88030535, 0.16129416, 0.9242525, 0.49545723]]],\n [[[0.50899494, 0.06482804, 0.44939405, 0.37704808],\n [0.47028124, 0.11969638, 0.62823206, 0.28560323]],\n [[0.40690207, 0.689753, 0.51636654, 0.23040164],\n [0.06935787, 0.00488842, 0.22462702, 0.09182382]],\n [[0.26611632, 0.00184339, 0.7730655, 0.5228131],\n [0.87776035, 0.77895886, 0.2787183, 0.16620636]],\n [[0.502574, 0.04039001, 0.5368497, 0.98379374],\n [0.40973026, 0.3238272, 0.9733018, 0.13988364]],\n [[0.04586202, 0.20983845, 0.20662665, 0.22270602],\n [0.60387236, 0.5155574, 0.51237285, 0.6528438]],\n [[0.45735973, 0.86821306, 0.61054605, 0.8370336],\n [0.45193362, 0.3734138, 0.7825672, 0.5699416]],\n [[0.44591594, 0.12447512, 0.09282011, 0.7055254],\n [0.25223452, 0.46696228, 0.7051136, 0.892151]],\n [[0.49615085, 0.47321403, 0.93138885, 0.7652197],\n [0.38766378, 0.30332977, 0.23131835,\n 0.02863514]]]]).float().cuda()\n points.requires_grad_()\n centers = torch.tensor([[[[0.83878064, 0.96658987, 0.8033424, 0.9598312],\n [0.45035273, 0.8768925, 0.977736, 0.54547966]],\n [[0.01041394, 0.597893, 0.36212963, 0.4410367],\n [0.94879234, 0.8372817, 0.21237361, 0.67945415]],\n [[0.5096087, 0.26401454, 0.60034937, 0.5417416],\n [0.87591463, 0.546456, 0.4096033, 0.16373193]],\n [[0.79547447, 0.1482386, 0.12840575, 0.45384115],\n [0.5640288, 0.944541, 0.5745328, 0.73229736]],\n [[0.93011934, 0.7406011, 0.62621707, 0.8677915],\n [0.91563636, 0.3595413, 0.6678378, 0.6085383]],\n [[0.22431666, 0.65617776, 0.7483924, 0.6263364],\n [0.30968404, 0.78204364, 0.14899081,\n 0.09628749]],\n [[0.73675203, 0.72104895, 0.4648038, 0.6101647],\n [0.7817645, 0.16572917, 0.3311919, 0.43407398]],\n [[0.8193154, 0.09559608, 0.05978829, 0.90262103],\n [0.4256065, 0.8165596, 0.8206446, 0.6604721]]],\n [[[0.7159653, 0.18600845, 0.21433902, 0.3159626],\n [0.3921569, 0.33221376, 0.5061177, 0.7961841]],\n [[0.95338356, 0.04785997, 0.67185795, 0.6538394],\n [0.4729132, 0.33404195, 0.17750603, 0.8445621]],\n [[0.6755793, 0.16193843, 0.75943846, 0.92123103],\n [0.2781859, 0.03114432, 0.710638, 0.52729136]],\n [[0.8376105, 0.10858494, 0.13208169, 0.365772],\n [0.5930795, 0.27390373, 0.14036089, 0.170403]],\n [[0.3479789, 0.89855295, 0.04844379, 0.9871029],\n [0.29781651, 0.0244137, 0.9179047, 0.8081611]],\n [[0.12460887, 0.44991326, 0.19382608, 0.35037738],\n [0.2773472, 0.4362057, 0.36757517, 0.5993509]],\n [[0.29630446, 0.90046406, 0.5417113, 0.13510644],\n [0.09623539, 0.04226565, 0.32001644,\n 0.44358212]],\n [[0.5274848, 0.82096446, 0.9415489, 0.7123748],\n [0.7537517, 0.8086482, 0.85345286,\n 0.7472754]]]]).float().cuda()\n centers.requires_grad_()\n knn_idx = torch.tensor([[[6, 7, 4, 6], [2, 4, 2, 4]],\n [[7, 1, 3, 2], [6, 0, 2, 6]]]).long().cuda()\n aggregate = 'sum'\n expected_output = torch.tensor(\n [[[[-0.08134781, 0.03877336, -0.8212776, -0.2869547],\n [-0.23378491, -0.24112664, -0.1600166, -0.4121864]],\n [[-0.05780616, -0.12298299, -0.0370461, -0.07889931],\n [-0.13956165, -0.02006848, -0.10940295, -0.0293439]],\n [[0.09284145, 0.58250105, 0.5927749, 0.16774094],\n [0.27070042, 0.13422406, 0.2617501, 0.23416464]],\n [[-0.06121218, -0.09561322, -0.20408826, 0.08079343],\n [0.00944228, 0.03874819, 0.08404065, 0.04041629]]],\n [[[-0.2110898, -0.13335688, -0.09315082, 0.08512095],\n [0.09121774, 0.15976946, 0.23994486, 0.14350912]],\n [[-0.36167958, -0.14891288, -0.64470863, -0.0646704],\n [-0.28276974, -0.08847666, -0.46904767, 0.20491874]],\n [[-0.34877953, -0.35533834, -0.25225785, -0.4638189],\n [-0.1420663, 0.09467781, 0.17088932, 0.22580585]],\n [[-0.3879708, -0.3991068, 0.05276498, -0.46989647],\n [0.32522714, -0.02163534, 0.21604237, 0.4346682]]]]).float()\n\n # test forward\n output = assign_score_withk(scores, points, centers, knn_idx, aggregate)\n assert torch.allclose(output.detach().cpu(), expected_output, atol=1e-6)\n\n # test backward\n loss = output.sum()\n loss.backward()\n expected_scores_grad = torch.tensor([[[[0.04288036, -0.18217683],\n [-0.78873926, 0.7485497],\n [-0.6866992, 0.05346543],\n [0.04288036, -0.18217683]],\n [[-1.1407862, 0.13533896],\n [-0.06964391, -0.22948086],\n [-1.1407862, 0.13533896],\n [-0.06964391, -0.22948086]]],\n [[[-0.3363995, -2.212181],\n [-1.1589496, -2.7724311],\n [-0.9387654, -1.3163853],\n [-1.4385346, -1.0614843]],\n [[-0.5048497, 1.4143617],\n [-0.47332114, 0.6017133],\n [-0.30974793, 1.1995442],\n [-0.5048497, 1.4143617]]]]).float()\n expected_points_grad = torch.tensor(\n [[[[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0.15585709, 0.15585709, 0.15585709, 0.15585709],\n [1.1893613, 1.1893613, 1.1893613, 1.1893613]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[1.6530733, 1.6530733, 1.6530733, 1.6530733],\n [1.8130021, 1.8130021, 1.8130021, 1.8130021]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0.58863074, 0.58863074, 0.58863074, 0.58863074],\n [1.3727596, 1.3727596, 1.3727596, 1.3727596]],\n [[0.28462553, 0.28462553, 0.28462553, 0.28462553],\n [0.8378516, 0.8378516, 0.8378516, 0.8378516]]],\n [[[0.13817799, 0.13817799, 0.13817799, 0.13817799],\n [0.34856772, 0.34856772, 0.34856772, 0.34856772]],\n [[0.7405102, 0.7405102, 0.7405102, 0.7405102],\n [0.06438422, 0.06438422, 0.06438422, 0.06438422]],\n [[0.8491963, 0.8491963, 0.8491963, 0.8491963],\n [1.1301711, 1.1301711, 1.1301711, 1.1301711]],\n [[0.6887394, 0.6887394, 0.6887394, 0.6887394],\n [0.22089851, 0.22089851, 0.22089851, 0.22089851]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0.605832, 0.605832, 0.605832, 0.605832],\n [0.92364264, 0.92364264, 0.92364264, 0.92364264]],\n [[0.23089725, 0.23089725, 0.23089725, 0.23089725],\n [0.5568468, 0.5568468, 0.5568468, 0.5568468]]]]).float()\n expected_centers_grad = torch.tensor(\n [[[[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[-1.0493311, -1.0493311, -1.0493311, -1.0493311],\n [-2.0301602, -2.0301602, -2.0301602, -2.0301602]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[-1.6328557, -1.6328557, -1.6328557, -1.6328557],\n [-3.1828144, -3.1828144, -3.1828144, -3.1828144]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]]],\n [[[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[0., 0., 0., 0.], [0., 0., 0., 0.]],\n [[-1.5429721, -1.5429721, -1.5429721, -1.5429721],\n [-1.6100934, -1.6100934, -1.6100934, -1.6100934]],\n [[-1.7103812, -1.7103812, -1.7103812, -1.7103812],\n [-1.6344175, -1.6344175, -1.6344175, -1.6344175]]]]).float()\n assert torch.allclose(\n scores.grad.detach().cpu(), expected_scores_grad, atol=1e-6)\n assert torch.allclose(\n points.grad.detach().cpu(), expected_points_grad, atol=1e-6)\n assert torch.allclose(\n centers.grad.detach().cpu(), expected_centers_grad, atol=1e-6)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_ball_query.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import ball_query\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_ball_query():\n new_xyz = torch.tensor([[[-0.0740, 1.3147, -1.3625],\n [-2.2769, 2.7817, -0.2334],\n [-0.4003, 2.4666, -0.5116],\n [-0.0740, 1.3147, -1.3625],\n [-0.0740, 1.3147, -1.3625]],\n [[-2.0289, 2.4952, -0.1708],\n [-2.0668, 6.0278, -0.4875],\n [0.4066, 1.4211, -0.2947],\n [-2.0289, 2.4952, -0.1708],\n [-2.0289, 2.4952, -0.1708]]]).cuda()\n\n xyz = torch.tensor([[[-0.0740, 1.3147, -1.3625], [0.5555, 1.0399, -1.3634],\n [-0.4003, 2.4666,\n -0.5116], [-0.5251, 2.4379, -0.8466],\n [-0.9691, 1.1418,\n -1.3733], [-0.2232, 0.9561, -1.3626],\n [-2.2769, 2.7817, -0.2334],\n [-0.2822, 1.3192, -1.3645], [0.1533, 1.5024, -1.0432],\n [0.4917, 1.1529, -1.3496]],\n [[-2.0289, 2.4952,\n -0.1708], [-0.7188, 0.9956, -0.5096],\n [-2.0668, 6.0278, -0.4875], [-1.9304, 3.3092, 0.6610],\n [0.0949, 1.4332, 0.3140], [-1.2879, 2.0008, -0.7791],\n [-0.7252, 0.9611, -0.6371], [0.4066, 1.4211, -0.2947],\n [0.3220, 1.4447, 0.3548], [-0.9744, 2.3856,\n -1.2000]]]).cuda()\n\n idx = ball_query(0, 0.2, 5, xyz, new_xyz)\n expected_idx = torch.tensor([[[0, 0, 0, 0, 0], [6, 6, 6, 6, 6],\n [2, 2, 2, 2, 2], [0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0]],\n [[0, 0, 0, 0, 0], [2, 2, 2, 2, 2],\n [7, 7, 7, 7, 7], [0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0]]]).cuda()\n assert torch.all(idx == expected_idx)\n\n # test dilated ball query\n idx = ball_query(0.2, 0.4, 5, xyz, new_xyz)\n expected_idx = torch.tensor([[[0, 5, 7, 0, 0], [6, 6, 6, 6, 6],\n [2, 3, 2, 2, 2], [0, 5, 7, 0, 0],\n [0, 5, 7, 0, 0]],\n [[0, 0, 0, 0, 0], [2, 2, 2, 2, 2],\n [7, 7, 7, 7, 7], [0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0]]]).cuda()\n assert torch.all(idx == expected_idx)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_bbox.py", "content": "import numpy as np\nimport pytest\nimport torch\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\nclass TestBBox(object):\n\n def _test_bbox_overlaps(self, dtype=torch.float):\n\n from mmcv.ops import bbox_overlaps\n b1 = torch.tensor([[1.0, 1.0, 3.0, 4.0], [2.0, 2.0, 3.0, 4.0],\n [7.0, 7.0, 8.0, 8.0]]).cuda().type(dtype)\n b2 = torch.tensor([[0.0, 2.0, 2.0, 5.0], [2.0, 1.0, 3.0,\n 3.0]]).cuda().type(dtype)\n should_output = np.array([[0.33333334, 0.5], [0.2, 0.5], [0.0, 0.0]])\n out = bbox_overlaps(b1, b2, offset=1)\n assert np.allclose(out.cpu().numpy(), should_output, 1e-2)\n\n b1 = torch.tensor([[1.0, 1.0, 3.0, 4.0], [2.0, 2.0, 3.0,\n 4.0]]).cuda().type(dtype)\n b2 = torch.tensor([[0.0, 2.0, 2.0, 5.0], [2.0, 1.0, 3.0,\n 3.0]]).cuda().type(dtype)\n should_output = np.array([0.33333334, 0.5])\n out = bbox_overlaps(b1, b2, aligned=True, offset=1)\n assert np.allclose(out.cpu().numpy(), should_output, 1e-2)\n\n b1 = torch.tensor([[0.0, 0.0, 3.0, 3.0]]).cuda().type(dtype)\n b1 = torch.tensor([[0.0, 0.0, 3.0, 3.0]]).cuda().type(dtype)\n b2 = torch.tensor([[4.0, 0.0, 5.0, 3.0], [3.0, 0.0, 4.0, 3.0],\n [2.0, 0.0, 3.0, 3.0], [1.0, 0.0, 2.0,\n 3.0]]).cuda().type(dtype)\n should_output = np.array([0, 0.2, 0.5, 0.5])\n out = bbox_overlaps(b1, b2, offset=1)\n assert np.allclose(out.cpu().numpy(), should_output, 1e-2)\n\n def test_bbox_overlaps_float(self):\n self._test_bbox_overlaps(torch.float)\n\n def test_bbox_overlaps_half(self):\n self._test_bbox_overlaps(torch.half)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_bilinear_grid_sample.py", "content": "import numpy as np\nimport torch\nimport torch.nn.functional as F\n\n\nclass TestBilinearGridSample(object):\n\n def _test_bilinear_grid_sample(self,\n dtype=torch.float,\n align_corners=False,\n multiplier=1,\n precision=1e-3):\n from mmcv.ops.point_sample import bilinear_grid_sample\n\n input = torch.rand(1, 1, 20, 20, dtype=dtype)\n grid = torch.Tensor([[[1, 0, 0], [0, 1, 0]]])\n grid = F.affine_grid(\n grid, (1, 1, 15, 15), align_corners=align_corners).type_as(input)\n grid *= multiplier\n\n out = bilinear_grid_sample(input, grid, align_corners=align_corners)\n ref_out = F.grid_sample(input, grid, align_corners=align_corners)\n\n assert np.allclose(out.data.detach().cpu().numpy(),\n ref_out.data.detach().cpu().numpy(), precision)\n\n def test_bilinear_grid_sample(self):\n self._test_bilinear_grid_sample(torch.double, False)\n self._test_bilinear_grid_sample(torch.double, True)\n self._test_bilinear_grid_sample(torch.float, False)\n self._test_bilinear_grid_sample(torch.float, True)\n self._test_bilinear_grid_sample(torch.float, False)\n self._test_bilinear_grid_sample(torch.float, True, 5)\n self._test_bilinear_grid_sample(torch.float, False, 10)\n self._test_bilinear_grid_sample(torch.float, True, -6)\n self._test_bilinear_grid_sample(torch.float, False, -10)\n self._test_bilinear_grid_sample(torch.double, True, 5)\n self._test_bilinear_grid_sample(torch.double, False, 10)\n self._test_bilinear_grid_sample(torch.double, True, -6)\n self._test_bilinear_grid_sample(torch.double, False, -10)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_border_align.py", "content": "import copy\n\nimport numpy as np\nimport pytest\nimport torch\n\n# [1,4c,h,w]\ninput_arr = [[[[1., 2., 3., 4.], [5., 6., 7., 8.], [9., 10., 11., 12.]],\n [[6, 7, 5, 8], [2, 1, 3, 4], [12, 9, 11, 10]],\n [[-2, -3, 2, 0], [-4, -5, 1, -1], [-1, -1, -1, -1]],\n [[0, -1, 2, 1], [-4, -3, -2, -1], [-1, -2, -3, -4]]]]\n# [1,h*w,4]\nboxes_arr = [[[0, 0, 2, 1], [1, 0, 3, 1], [1, 0, 2, 1], [0, 0, 3, 1],\n [0, 0, 1, 2], [0, 0, 2, 2], [1, 0, 2, 1], [1, 0, 3, 1],\n [0, 1, 1, 2], [0, 0, 3, 2], [1, 0, 3, 2], [2, 0, 3, 2]]]\noutput_dict = {\n # [1,c,h*w,4] for each value,\n # the output is manually checked for its correctness\n\n # pool_size=1\n 1: [[[[3., 6., 1., 2.], [4., 7., -1., 1.], [3., 7., 1., 2.],\n [4., 6., -1., 1.], [2., 12., -1., -1.], [3., 12., -1., 2.],\n [3., 7., 1., 2.], [4., 7., -1., 1.], [6., 12., -1., -2.],\n [4., 12., -1., 1.], [4., 9., -1., 1.], [4., 11., -1., 1.]]]],\n\n # pool_size=2\n 2: [[[[3., 6., 1., 2.], [4., 7., 1., 1.], [3., 7., 1., 2.],\n [4., 6., -1., 1.], [2., 12., -1., -1.], [3., 12., -1., 2.],\n [3., 7., 1., 2.], [4., 7., 1., 1.], [6., 12., -1., -2.],\n [4., 12., -1., 1.], [4., 9., -1., 1.], [4., 11., -1., 1.]]]],\n}\ninput_grad_dict = {\n # [1,4c,h,w] for each value\n # the grad is manually checked for its correctness\n\n # pool_size=1\n 1: [[[[0., 1., 4., 6.], [0., 1., 0., 0.], [0., 0., 0., 0.]],\n [[2., 4., 0., 0.], [0., 0., 0., 0.], [4., 1., 1., 0.]],\n [[0., 0., 0., 0.], [0., 0., 3., 3.], [0., 2., 1., 3.]],\n [[0., 1., 4., 6.], [0., 0., 0., 0.], [0., 1., 0., 0.]]]],\n\n # pool_size=2\n 2: [[[[0., 1., 4., 6.], [0., 1., 0., 0.], [0., 0., 0., 0.]],\n [[2., 4., 0., 0.], [0., 0., 0., 0.], [4., 1., 1., 0.]],\n [[0., 0., 0., 0.], [0., 0., 5., 1.], [0., 2., 1., 3.]],\n [[0., 1., 4., 6.], [0., 0., 0., 0.], [0., 1., 0., 0.]]]],\n}\n\n\ndef _test_border_align_allclose(device, dtype, pool_size):\n if not torch.cuda.is_available() and device == 'cuda':\n pytest.skip('test requires GPU')\n try:\n from mmcv.ops import border_align, BorderAlign\n except ModuleNotFoundError:\n pytest.skip('BorderAlign op is not successfully compiled')\n\n np_input = np.array(input_arr)\n np_boxes = np.array(boxes_arr)\n np_output = np.array(output_dict[pool_size])\n np_grad = np.array(input_grad_dict[pool_size])\n\n input = torch.tensor(\n np_input, dtype=dtype, device=device, requires_grad=True)\n boxes = torch.tensor(np_boxes, dtype=dtype, device=device)\n\n # test for border_align\n input_cp = copy.deepcopy(input)\n output = border_align(input_cp, boxes, pool_size)\n output.backward(torch.ones_like(output))\n assert np.allclose(\n output.data.type(dtype).cpu().numpy(), np_output, atol=1e-5)\n assert np.allclose(\n input_cp.grad.data.type(dtype).cpu().numpy(), np_grad, atol=1e-5)\n\n # test for BorderAlign\n pool_module = BorderAlign(pool_size)\n output = pool_module(input, boxes)\n output.backward(torch.ones_like(output))\n assert np.allclose(\n output.data.type(dtype).cpu().numpy(), np_output, atol=1e-5)\n assert np.allclose(\n input.grad.data.type(dtype).cpu().numpy(), np_grad, atol=1e-5)\n\n\n@pytest.mark.parametrize('device', ['cuda'])\n@pytest.mark.parametrize('dtype', [torch.float, torch.half, torch.double])\n@pytest.mark.parametrize('pool_size', [1, 2])\ndef test_border_align(device, dtype, pool_size):\n _test_border_align_allclose(device, dtype, pool_size)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_box_iou_rotated.py", "content": "import numpy as np\nimport pytest\nimport torch\n\n\nclass TestBoxIoURotated(object):\n\n def test_box_iou_rotated_cpu(self):\n from mmcv.ops import box_iou_rotated\n np_boxes1 = np.asarray(\n [[1.0, 1.0, 3.0, 4.0, 0.5], [2.0, 2.0, 3.0, 4.0, 0.6],\n [7.0, 7.0, 8.0, 8.0, 0.4]],\n dtype=np.float32)\n np_boxes2 = np.asarray(\n [[0.0, 2.0, 2.0, 5.0, 0.3], [2.0, 1.0, 3.0, 3.0, 0.5],\n [5.0, 5.0, 6.0, 7.0, 0.4]],\n dtype=np.float32)\n np_expect_ious = np.asarray(\n [[0.3708, 0.4351, 0.0000], [0.1104, 0.4487, 0.0424],\n [0.0000, 0.0000, 0.3622]],\n dtype=np.float32)\n np_expect_ious_aligned = np.asarray([0.3708, 0.4487, 0.3622],\n dtype=np.float32)\n\n boxes1 = torch.from_numpy(np_boxes1)\n boxes2 = torch.from_numpy(np_boxes2)\n\n # test cw angle definition\n ious = box_iou_rotated(boxes1, boxes2)\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n\n ious = box_iou_rotated(boxes1, boxes2, aligned=True)\n assert np.allclose(\n ious.cpu().numpy(), np_expect_ious_aligned, atol=1e-4)\n\n # test ccw angle definition\n boxes1[..., -1] *= -1\n boxes2[..., -1] *= -1\n ious = box_iou_rotated(boxes1, boxes2, clockwise=False)\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n\n ious = box_iou_rotated(boxes1, boxes2, aligned=True, clockwise=False)\n assert np.allclose(\n ious.cpu().numpy(), np_expect_ious_aligned, atol=1e-4)\n\n @pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\n def test_box_iou_rotated_cuda(self):\n from mmcv.ops import box_iou_rotated\n np_boxes1 = np.asarray(\n [[1.0, 1.0, 3.0, 4.0, 0.5], [2.0, 2.0, 3.0, 4.0, 0.6],\n [7.0, 7.0, 8.0, 8.0, 0.4]],\n dtype=np.float32)\n np_boxes2 = np.asarray(\n [[0.0, 2.0, 2.0, 5.0, 0.3], [2.0, 1.0, 3.0, 3.0, 0.5],\n [5.0, 5.0, 6.0, 7.0, 0.4]],\n dtype=np.float32)\n np_expect_ious = np.asarray(\n [[0.3708, 0.4351, 0.0000], [0.1104, 0.4487, 0.0424],\n [0.0000, 0.0000, 0.3622]],\n dtype=np.float32)\n np_expect_ious_aligned = np.asarray([0.3708, 0.4487, 0.3622],\n dtype=np.float32)\n\n boxes1 = torch.from_numpy(np_boxes1).cuda()\n boxes2 = torch.from_numpy(np_boxes2).cuda()\n\n # test cw angle definition\n ious = box_iou_rotated(boxes1, boxes2)\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n\n ious = box_iou_rotated(boxes1, boxes2, aligned=True)\n assert np.allclose(\n ious.cpu().numpy(), np_expect_ious_aligned, atol=1e-4)\n\n # test ccw angle definition\n boxes1[..., -1] *= -1\n boxes2[..., -1] *= -1\n ious = box_iou_rotated(boxes1, boxes2, clockwise=False)\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n\n ious = box_iou_rotated(boxes1, boxes2, aligned=True, clockwise=False)\n assert np.allclose(\n ious.cpu().numpy(), np_expect_ious_aligned, atol=1e-4)\n\n def test_box_iou_rotated_iof_cpu(self):\n from mmcv.ops import box_iou_rotated\n np_boxes1 = np.asarray(\n [[1.0, 1.0, 3.0, 4.0, 0.5], [2.0, 2.0, 3.0, 4.0, 0.6],\n [7.0, 7.0, 8.0, 8.0, 0.4]],\n dtype=np.float32)\n np_boxes2 = np.asarray(\n [[0.0, 2.0, 2.0, 5.0, 0.3], [2.0, 1.0, 3.0, 3.0, 0.5],\n [5.0, 5.0, 6.0, 7.0, 0.4]],\n dtype=np.float32)\n np_expect_ious = np.asarray(\n [[0.4959, 0.5306, 0.0000], [0.1823, 0.5420, 0.1832],\n [0.0000, 0.0000, 0.4404]],\n dtype=np.float32)\n np_expect_ious_aligned = np.asarray([0.4959, 0.5420, 0.4404],\n dtype=np.float32)\n\n boxes1 = torch.from_numpy(np_boxes1)\n boxes2 = torch.from_numpy(np_boxes2)\n\n # test cw angle definition\n ious = box_iou_rotated(boxes1, boxes2, mode='iof')\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n ious = box_iou_rotated(boxes1, boxes2, mode='iof', aligned=True)\n assert np.allclose(\n ious.cpu().numpy(), np_expect_ious_aligned, atol=1e-4)\n\n # test ccw angle definition\n boxes1[..., -1] *= -1\n boxes2[..., -1] *= -1\n ious = box_iou_rotated(boxes1, boxes2, mode='iof', clockwise=False)\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n ious = box_iou_rotated(\n boxes1, boxes2, mode='iof', aligned=True, clockwise=False)\n assert np.allclose(\n ious.cpu().numpy(), np_expect_ious_aligned, atol=1e-4)\n\n @pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\n def test_box_iou_rotated_iof_cuda(self):\n from mmcv.ops import box_iou_rotated\n np_boxes1 = np.asarray(\n [[1.0, 1.0, 3.0, 4.0, 0.5], [2.0, 2.0, 3.0, 4.0, 0.6],\n [7.0, 7.0, 8.0, 8.0, 0.4]],\n dtype=np.float32)\n np_boxes2 = np.asarray(\n [[0.0, 2.0, 2.0, 5.0, 0.3], [2.0, 1.0, 3.0, 3.0, 0.5],\n [5.0, 5.0, 6.0, 7.0, 0.4]],\n dtype=np.float32)\n np_expect_ious = np.asarray(\n [[0.4959, 0.5306, 0.0000], [0.1823, 0.5420, 0.1832],\n [0.0000, 0.0000, 0.4404]],\n dtype=np.float32)\n np_expect_ious_aligned = np.asarray([0.4959, 0.5420, 0.4404],\n dtype=np.float32)\n\n boxes1 = torch.from_numpy(np_boxes1).cuda()\n boxes2 = torch.from_numpy(np_boxes2).cuda()\n\n # test cw angle definition\n ious = box_iou_rotated(boxes1, boxes2, mode='iof')\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n\n ious = box_iou_rotated(boxes1, boxes2, mode='iof', aligned=True)\n assert np.allclose(\n ious.cpu().numpy(), np_expect_ious_aligned, atol=1e-4)\n\n # test ccw angle definition\n boxes1[..., -1] *= -1\n boxes2[..., -1] *= -1\n ious = box_iou_rotated(boxes1, boxes2, mode='iof', clockwise=False)\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n\n ious = box_iou_rotated(\n boxes1, boxes2, mode='iof', aligned=True, clockwise=False)\n assert np.allclose(\n ious.cpu().numpy(), np_expect_ious_aligned, atol=1e-4)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_carafe.py", "content": "import torch\nfrom torch.autograd import gradcheck\n\n\nclass TestCarafe(object):\n\n def test_carafe_naive_gradcheck(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import CARAFENaive\n feat = torch.randn(\n 2, 64, 3, 3, requires_grad=True, device='cuda').double()\n mask = torch.randn(\n 2, 100, 6, 6, requires_grad=True,\n device='cuda').sigmoid().double()\n gradcheck(CARAFENaive(5, 4, 2), (feat, mask), atol=1e-4, eps=1e-4)\n\n def test_carafe_gradcheck(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import CARAFE\n feat = torch.randn(\n 2, 64, 3, 3, requires_grad=True, device='cuda').double()\n mask = torch.randn(\n 2, 100, 6, 6, requires_grad=True,\n device='cuda').sigmoid().double()\n gradcheck(CARAFE(5, 4, 2), (feat, mask), atol=1e-4, eps=1e-4)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_cc_attention.py", "content": "import numpy as np\nimport torch\nimport torch.nn as nn\n\n\nclass Loss(nn.Module):\n\n def __init__(self):\n super().__init__()\n\n def forward(self, input, target):\n input = input.view(-1)\n target = target.view(-1)\n return torch.mean(input - target)\n\n\nclass TestCrissCrossAttention(object):\n\n def test_cc_attention(self):\n device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n\n from mmcv.ops import CrissCrossAttention\n loss_func = Loss()\n\n input = np.fromfile(\n 'tests/data/for_ccattention/ccattention_input.bin',\n dtype=np.float32)\n output = np.fromfile(\n 'tests/data/for_ccattention/ccattention_output.bin',\n dtype=np.float32)\n input = input.reshape((1, 32, 45, 45))\n output = output.reshape((1, 32, 45, 45))\n label = torch.ones((1, 32, 45, 45))\n\n input = torch.FloatTensor(input)\n output = torch.FloatTensor(output)\n\n input.requires_grad = True\n\n shape = input.shape\n channel = shape[1]\n\n cca = CrissCrossAttention(channel)\n cca.to(device)\n input = input.to(device)\n label = label.to(device)\n cca.train()\n test_output = cca(input)\n test_loss = loss_func(test_output, label)\n test_loss.backward()\n test_output = test_output.detach().cpu().numpy()\n output = output.numpy()\n\n assert np.allclose(test_output, output)\n assert test_output.shape == shape\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_contour_expand.py", "content": "import numpy as np\nimport torch\n\n\ndef test_contour_expand():\n from mmcv.ops import contour_expand\n\n np_internal_kernel_label = np.array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 1, 1, 0, 0, 0, 0, 2, 0],\n [0, 0, 1, 1, 0, 0, 0, 0, 2, 0],\n [0, 0, 1, 1, 0, 0, 0, 0, 2, 0],\n [0, 0, 1, 1, 0, 0, 0, 0, 2, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0]]).astype(np.int32)\n np_kernel_mask1 = np.array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 1, 1, 1, 1, 1, 1, 1, 0],\n [0, 0, 1, 1, 1, 1, 1, 1, 1, 0],\n [0, 0, 1, 1, 1, 1, 1, 1, 1, 0],\n [0, 0, 1, 1, 1, 1, 1, 1, 1, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0]]).astype(np.uint8)\n np_kernel_mask2 = (np_internal_kernel_label > 0).astype(np.uint8)\n\n np_kernel_mask = np.stack([np_kernel_mask1, np_kernel_mask2])\n min_area = 1\n kernel_region_num = 3\n result = contour_expand(np_kernel_mask, np_internal_kernel_label, min_area,\n kernel_region_num)\n gt = [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 1, 1, 1, 1, 2, 2, 2, 0],\n [0, 0, 1, 1, 1, 1, 2, 2, 2, 0], [0, 0, 1, 1, 1, 1, 2, 2, 2, 0],\n [0, 0, 1, 1, 1, 1, 2, 2, 2, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]\n assert np.allclose(result, gt)\n\n np_kernel_mask_t = torch.from_numpy(np_kernel_mask)\n np_internal_kernel_label_t = torch.from_numpy(np_internal_kernel_label)\n result = contour_expand(np_kernel_mask_t, np_internal_kernel_label_t,\n min_area, kernel_region_num)\n assert np.allclose(result, gt)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_convex_iou.py", "content": "import numpy as np\nimport pytest\nimport torch\n\nfrom mmcv.ops import convex_giou, convex_iou\n\nnp_pointsets = np.asarray([[\n 1.0, 1.0, 2.0, 2.0, 1.0, 2.0, 2.0, 1.0, 1.0, 3.0, 3.0, 1.0, 2.0, 3.0, 3.0,\n 2.0, 1.5, 1.5\n],\n [\n 1.5, 1.5, 2.5, 2.5, 1.5, 2.5, 2.5, 1.5, 1.5,\n 3.5, 3.5, 1.5, 2.5, 3.5, 3.5, 2.5, 2.0, 2.0\n ]])\n\nnp_polygons = np.asarray([[1.0, 1.0, 1.0, 2.0, 2.0, 2.0, 2.0, 1.0],\n [1.0, 1.0, 1.0, 3.0, 3.0, 3.0, 3.0, 1.0]])\n\nnp_expected_iou = np.asarray([[0.2857, 0.8750], [0.0588, 0.4286]])\n\nnp_expected_giou = np.asarray([0.2857, 0.3831])\n\nnp_expected_grad = np.asarray([[\n 0.0204, 0.0408, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0612,\n -0.0408, -0.0408, 0.0816, -0.0408, -0.0816, -0.0816, -0.0408, 0.0000,\n 0.0000\n],\n [\n -0.1848, -0.1848, 0.0000, 0.0000, 0.0000,\n 0.0000, 0.0000, 0.0000, -0.1076, -0.0801,\n -0.0801, -0.1076, -0.0367, -0.0734, -0.0734,\n -0.0367, 0.0000, 0.0000\n ]])\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_convex_iou():\n pointsets = torch.from_numpy(np_pointsets).cuda().float()\n polygons = torch.from_numpy(np_polygons).cuda().float()\n expected_iou = torch.from_numpy(np_expected_iou).cuda().float()\n assert torch.allclose(\n convex_iou(pointsets, polygons), expected_iou, atol=1e-3)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_convex_giou():\n pointsets = torch.from_numpy(np_pointsets).cuda().float()\n polygons = torch.from_numpy(np_polygons).cuda().float()\n expected_giou = torch.from_numpy(np_expected_giou).cuda().float()\n expected_grad = torch.from_numpy(np_expected_grad).cuda().float()\n giou, grad = convex_giou(pointsets, polygons)\n assert torch.allclose(giou, expected_giou, atol=1e-3)\n assert torch.allclose(grad, expected_grad, atol=1e-3)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_corner_pool.py", "content": "\"\"\"\nCommandLine:\n pytest tests/test_corner_pool.py\n\"\"\"\nimport pytest\nimport torch\n\nfrom mmcv.ops import CornerPool\n\n\ndef test_corner_pool_device_and_dtypes_cpu():\n \"\"\"\n CommandLine:\n xdoctest -m tests/test_corner_pool.py \\\n test_corner_pool_device_and_dtypes_cpu\n \"\"\"\n with pytest.raises(AssertionError):\n # pool mode must in ['bottom', 'left', 'right', 'top']\n pool = CornerPool('corner')\n\n lr_tensor = torch.tensor([[[[0, 0, 0, 0, 0], [2, 1, 3, 0, 2],\n [5, 4, 1, 1, 6], [0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0]]]])\n tb_tensor = torch.tensor([[[[0, 3, 1, 0, 0], [0, 1, 1, 0, 0],\n [0, 3, 4, 0, 0], [0, 2, 2, 0, 0],\n [0, 0, 2, 0, 0]]]])\n # Left Pool\n left_answer = torch.tensor([[[[0, 0, 0, 0, 0], [3, 3, 3, 2, 2],\n [6, 6, 6, 6, 6], [0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0]]]])\n pool = CornerPool('left')\n left_tensor = pool(lr_tensor)\n assert left_tensor.type() == lr_tensor.type()\n assert torch.equal(left_tensor, left_answer)\n # Right Pool\n right_answer = torch.tensor([[[[0, 0, 0, 0, 0], [2, 2, 3, 3, 3],\n [5, 5, 5, 5, 6], [0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0]]]])\n pool = CornerPool('right')\n right_tensor = pool(lr_tensor)\n assert right_tensor.type() == lr_tensor.type()\n assert torch.equal(right_tensor, right_answer)\n # Top Pool\n top_answer = torch.tensor([[[[0, 3, 4, 0, 0], [0, 3, 4, 0, 0],\n [0, 3, 4, 0, 0], [0, 2, 2, 0, 0],\n [0, 0, 2, 0, 0]]]])\n pool = CornerPool('top')\n top_tensor = pool(tb_tensor)\n assert top_tensor.type() == tb_tensor.type()\n assert torch.equal(top_tensor, top_answer)\n # Bottom Pool\n bottom_answer = torch.tensor([[[[0, 3, 1, 0, 0], [0, 3, 1, 0, 0],\n [0, 3, 4, 0, 0], [0, 3, 4, 0, 0],\n [0, 3, 4, 0, 0]]]])\n pool = CornerPool('bottom')\n bottom_tensor = pool(tb_tensor)\n assert bottom_tensor.type() == tb_tensor.type()\n assert torch.equal(bottom_tensor, bottom_answer)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_correlation.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport pytest\nimport torch\n\nfrom mmcv.ops import Correlation\n\n_input1 = [[[[1., 2., 3.], [0., 1., 2.], [3., 5., 2.]]]]\n_input2 = [[[[1., 2., 3.], [3., 1., 2.], [8., 5., 2.]]]]\n\ngt_out_shape = (1, 1, 1, 3, 3)\n_gt_out = [[[[[1., 4., 9.], [0., 1., 4.], [24., 25., 4.]]]]]\ngt_input1_grad = [[[[1., 2., 3.], [3., 1., 2.], [8., 5., 2.]]]]\n\n\ndef assert_equal_tensor(tensor_a, tensor_b):\n\n assert tensor_a.eq(tensor_b).all()\n\n\nclass TestCorrelation:\n\n def _test_correlation(self, dtype=torch.float):\n\n layer = Correlation(max_displacement=0)\n\n input1 = torch.tensor(_input1, dtype=dtype).cuda()\n input2 = torch.tensor(_input2, dtype=dtype).cuda()\n input1.requires_grad = True\n input2.requires_grad = True\n out = layer(input1, input2)\n out.backward(torch.ones_like(out))\n\n # `eq_cpu` is not implemented for 'Half' in torch1.5.0,\n # so we need to make a comparison for cuda tensor\n # rather than cpu tensor\n gt_out = torch.tensor(_gt_out, dtype=dtype).cuda()\n assert_equal_tensor(out, gt_out)\n assert_equal_tensor(input1.grad.detach(), input2)\n assert_equal_tensor(input2.grad.detach(), input1)\n\n @pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\n def test_correlation(self):\n self._test_correlation(torch.float)\n self._test_correlation(torch.double)\n self._test_correlation(torch.half)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_deform_conv.py", "content": "import numpy as np\nimport pytest\nimport torch\n\nfrom mmcv.utils import TORCH_VERSION, digit_version\n\ntry:\n # If PyTorch version >= 1.6.0 and fp16 is enabled, torch.cuda.amp.autocast\n # would be imported and used; we should test if our modules support it.\n from torch.cuda.amp import autocast\nexcept ImportError:\n pass\n\ninput = [[[[1., 2., 3.], [0., 1., 2.], [3., 5., 2.]]]]\noffset_weight = [[[0.1, 0.4, 0.6, 0.1]], [[0.3, 0.2, 0.1, 0.3]],\n [[0.5, 0.5, 0.2, 0.8]], [[0.8, 0.3, 0.9, 0.1]],\n [[0.3, 0.1, 0.2, 0.5]], [[0.3, 0.7, 0.5, 0.3]],\n [[0.6, 0.2, 0.5, 0.3]], [[0.4, 0.1, 0.8, 0.4]]]\noffset_bias = [0.7, 0.1, 0.8, 0.5, 0.6, 0.5, 0.4, 0.7]\ndeform_weight = [[[0.4, 0.2, 0.1, 0.9]]]\n\ngt_out = [[[[1.650, 0.], [0.000, 0.]]]]\ngt_x_grad = [[[[-0.666, 0.204, 0.000], [0.030, -0.416, 0.012],\n [0.000, 0.252, 0.129]]]]\ngt_offset_weight_grad = [[[[1.44, 2.88], [0.00, 1.44]]],\n [[[-0.72, -1.44], [0.00, -0.72]]],\n [[[0.00, 0.00], [0.00, 0.00]]],\n [[[0.00, 0.00], [0.00, 0.00]]],\n [[[-0.10, -0.20], [0.00, -0.10]]],\n [[[-0.08, -0.16], [0.00, -0.08]]],\n [[[-0.54, -1.08], [0.00, -0.54]]],\n [[[-0.54, -1.08], [0.00, -0.54]]]]\ngt_offset_bias_grad = [1.44, -0.72, 0., 0., -0.10, -0.08, -0.54, -0.54],\ngt_deform_weight_grad = [[[[3.62, 0.], [0.40, 0.18]]]]\n\n\nclass TestDeformconv(object):\n\n def _test_deformconv(self,\n dtype=torch.float,\n threshold=1e-3,\n device='cuda',\n batch_size=10,\n im2col_step=2):\n if not torch.cuda.is_available() and device == 'cuda':\n pytest.skip('test requires GPU')\n from mmcv.ops import DeformConv2dPack\n c_in = 1\n c_out = 1\n batch_size = 10\n repeated_input = np.repeat(input, batch_size, axis=0)\n repeated_gt_out = np.repeat(gt_out, batch_size, axis=0)\n repeated_gt_x_grad = np.repeat(gt_x_grad, batch_size, axis=0)\n x = torch.tensor(repeated_input, device=device, dtype=dtype)\n x.requires_grad = True\n model = DeformConv2dPack(\n in_channels=c_in,\n out_channels=c_out,\n kernel_size=2,\n stride=1,\n padding=0,\n im2col_step=im2col_step)\n model.conv_offset.weight.data = torch.nn.Parameter(\n torch.Tensor(offset_weight).reshape(8, 1, 2, 2))\n model.conv_offset.bias.data = torch.nn.Parameter(\n torch.Tensor(offset_bias).reshape(8))\n model.weight.data = torch.nn.Parameter(\n torch.Tensor(deform_weight).reshape(1, 1, 2, 2))\n if device == 'cuda':\n model.cuda()\n model.type(dtype)\n\n out = model(x)\n out.backward(torch.ones_like(out))\n\n assert np.allclose(out.data.detach().cpu().numpy(), repeated_gt_out,\n threshold)\n assert np.allclose(x.grad.detach().cpu().numpy(), repeated_gt_x_grad,\n threshold)\n # the batch size of the input is increased which results in\n # a larger gradient so we need to divide by the batch_size\n assert np.allclose(\n model.conv_offset.weight.grad.detach().cpu().numpy() / batch_size,\n gt_offset_weight_grad, threshold)\n assert np.allclose(\n model.conv_offset.bias.grad.detach().cpu().numpy() / batch_size,\n gt_offset_bias_grad, threshold)\n assert np.allclose(\n model.weight.grad.detach().cpu().numpy() / batch_size,\n gt_deform_weight_grad, threshold)\n\n from mmcv.ops import DeformConv2d\n\n # test bias\n model = DeformConv2d(1, 1, 2, stride=1, padding=0)\n assert not hasattr(model, 'bias')\n # test bias=True\n with pytest.raises(AssertionError):\n model = DeformConv2d(1, 1, 2, stride=1, padding=0, bias=True)\n # test in_channels % group != 0\n with pytest.raises(AssertionError):\n model = DeformConv2d(3, 2, 3, groups=2)\n # test out_channels % group != 0\n with pytest.raises(AssertionError):\n model = DeformConv2d(3, 4, 3, groups=3)\n\n def _test_amp_deformconv(self,\n input_dtype,\n threshold=1e-3,\n batch_size=10,\n im2col_step=2):\n \"\"\"The function to test amp released on pytorch 1.6.0.\n\n The type of input data might be torch.float or torch.half,\n so we should test deform_conv in both cases. With amp, the\n data type of model will NOT be set manually.\n\n Args:\n input_dtype: torch.float or torch.half.\n threshold: the same as above function.\n \"\"\"\n if not torch.cuda.is_available():\n return\n from mmcv.ops import DeformConv2dPack\n c_in = 1\n c_out = 1\n repeated_input = np.repeat(input, batch_size, axis=0)\n repeated_gt_out = np.repeat(gt_out, batch_size, axis=0)\n repeated_gt_x_grad = np.repeat(gt_x_grad, batch_size, axis=0)\n x = torch.Tensor(repeated_input).cuda().type(input_dtype)\n x.requires_grad = True\n model = DeformConv2dPack(\n in_channels=c_in,\n out_channels=c_out,\n kernel_size=2,\n stride=1,\n padding=0,\n im2col_step=im2col_step)\n model.conv_offset.weight.data = torch.nn.Parameter(\n torch.Tensor(offset_weight).reshape(8, 1, 2, 2))\n model.conv_offset.bias.data = torch.nn.Parameter(\n torch.Tensor(offset_bias).reshape(8))\n model.weight.data = torch.nn.Parameter(\n torch.Tensor(deform_weight).reshape(1, 1, 2, 2))\n model.cuda()\n\n out = model(x)\n out.backward(torch.ones_like(out))\n\n assert np.allclose(out.data.detach().cpu().numpy(), repeated_gt_out,\n threshold)\n assert np.allclose(x.grad.detach().cpu().numpy(), repeated_gt_x_grad,\n threshold)\n assert np.allclose(\n model.conv_offset.weight.grad.detach().cpu().numpy() / batch_size,\n gt_offset_weight_grad, threshold)\n assert np.allclose(\n model.conv_offset.bias.grad.detach().cpu().numpy() / batch_size,\n gt_offset_bias_grad, threshold)\n assert np.allclose(\n model.weight.grad.detach().cpu().numpy() / batch_size,\n gt_deform_weight_grad, threshold)\n\n from mmcv.ops import DeformConv2d\n\n # test bias\n model = DeformConv2d(1, 1, 2, stride=1, padding=0)\n assert not hasattr(model, 'bias')\n # test bias=True\n with pytest.raises(AssertionError):\n model = DeformConv2d(1, 1, 2, stride=1, padding=0, bias=True)\n # test in_channels % group != 0\n with pytest.raises(AssertionError):\n model = DeformConv2d(3, 2, 3, groups=2)\n # test out_channels % group != 0\n with pytest.raises(AssertionError):\n model = DeformConv2d(3, 4, 3, groups=3)\n\n def test_deformconv(self):\n self._test_deformconv(torch.double, device='cpu')\n self._test_deformconv(torch.float, device='cpu', threshold=1e-1)\n self._test_deformconv(torch.double)\n self._test_deformconv(torch.float)\n self._test_deformconv(torch.half, threshold=1e-1)\n # test batch_size < im2col_step\n self._test_deformconv(torch.float, batch_size=1, im2col_step=2)\n # test bach_size % im2col_step != 0\n with pytest.raises(\n AssertionError,\n match='batch size must be divisible by im2col_step'):\n self._test_deformconv(torch.float, batch_size=10, im2col_step=3)\n\n # test amp when torch version >= '1.6.0', the type of\n # input data for deformconv might be torch.float or torch.half\n if (TORCH_VERSION != 'parrots'\n and digit_version(TORCH_VERSION) >= digit_version('1.6.0')):\n with autocast(enabled=True):\n self._test_amp_deformconv(torch.float, 1e-1)\n self._test_amp_deformconv(torch.half, 1e-1)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_deform_roi_pool.py", "content": "import os\n\nimport numpy as np\nimport torch\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck\n _USING_PARROTS = False\n\ncur_dir = os.path.dirname(os.path.abspath(__file__))\n\ninputs = [([[[[1., 2.], [3., 4.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2.], [3., 4.]], [[4., 3.], [2.,\n 1.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.], [9., 10., 13., 14.],\n [11., 12., 15., 16.]]]], [[0., 0., 0., 3., 3.]])]\noutputs = [([[[[1, 1.25], [1.5, 1.75]]]], [[[[3.0625, 0.4375],\n [0.4375, 0.0625]]]]),\n ([[[[1., 1.25], [1.5, 1.75]], [[4, 3.75],\n [3.5, 3.25]]]], [[[[3.0625, 0.4375],\n [0.4375, 0.0625]],\n [[3.0625, 0.4375],\n [0.4375,\n 0.0625]]]]),\n ([[[[1.9375, 4.75],\n [7.5625,\n 10.375]]]], [[[[0.47265625, 0.4296875, 0.4296875, 0.04296875],\n [0.4296875, 0.390625, 0.390625, 0.0390625],\n [0.4296875, 0.390625, 0.390625, 0.0390625],\n [0.04296875, 0.0390625, 0.0390625,\n 0.00390625]]]])]\n\n\nclass TestDeformRoIPool(object):\n\n def test_deform_roi_pool_gradcheck(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import DeformRoIPoolPack\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n sampling_ratio = 2\n\n for case in inputs:\n np_input = np.array(case[0])\n np_rois = np.array(case[1])\n\n x = torch.tensor(\n np_input, device='cuda', dtype=torch.float, requires_grad=True)\n rois = torch.tensor(np_rois, device='cuda', dtype=torch.float)\n output_c = x.size(1)\n\n droipool = DeformRoIPoolPack((pool_h, pool_w),\n output_c,\n spatial_scale=spatial_scale,\n sampling_ratio=sampling_ratio).cuda()\n\n if _USING_PARROTS:\n gradcheck(droipool, (x, rois), no_grads=[rois])\n else:\n gradcheck(droipool, (x, rois), eps=1e-2, atol=1e-2)\n\n def test_modulated_deform_roi_pool_gradcheck(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import ModulatedDeformRoIPoolPack\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n sampling_ratio = 2\n\n for case in inputs:\n np_input = np.array(case[0])\n np_rois = np.array(case[1])\n\n x = torch.tensor(\n np_input, device='cuda', dtype=torch.float, requires_grad=True)\n rois = torch.tensor(np_rois, device='cuda', dtype=torch.float)\n output_c = x.size(1)\n\n droipool = ModulatedDeformRoIPoolPack(\n (pool_h, pool_w),\n output_c,\n spatial_scale=spatial_scale,\n sampling_ratio=sampling_ratio).cuda()\n\n if _USING_PARROTS:\n gradcheck(droipool, (x, rois), no_grads=[rois])\n else:\n gradcheck(droipool, (x, rois), eps=1e-2, atol=1e-2)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_focal_loss.py", "content": "import numpy as np\nimport torch\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck\n _USING_PARROTS = False\n\n# torch.set_printoptions(precision=8, threshold=100)\n\ninputs = [\n ([[1., 0], [0, 1.]], [0, 1]),\n ([[1., 0, -1.], [0, 1., 2.]], [2, 1]),\n ([[1e-6, 2e-6, 3e-6], [4e-6, 5e-5, 6e-4], [7e-3, 8e-2, 9e-1]], [1, 2, 0]),\n]\n\nsoftmax_outputs = [(0.00566451, [[-0.00657264, 0.00657264],\n [0.00657264, -0.00657264]]),\n (0.34956908, [[0.10165970, 0.03739851, -0.13905823],\n [0.01227554, -0.10298023, 0.09070466]]),\n (0.15754992, [[0.02590877, -0.05181759, 0.02590882],\n [0.02589641, 0.02589760, -0.05179400],\n [-0.07307514, 0.02234372, 0.05073142]])]\n\nsigmoid_outputs = [(0.13562961, [[-0.00657264, 0.11185755],\n [0.11185755, -0.00657264]]),\n (1.10251057, [[0.28808805, 0.11185755, -0.09602935],\n [0.11185755, -0.00657264, 0.40376765]]),\n (0.42287254, [[0.07457182, -0.02485716, 0.07457201],\n [0.07457211, 0.07457669, -0.02483728],\n [-0.02462499, 0.08277918, 0.18050370]])]\n\n\nclass Testfocalloss(object):\n\n def _test_softmax(self, dtype=torch.float):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import softmax_focal_loss\n alpha = 0.25\n gamma = 2.0\n for case, output in zip(inputs, softmax_outputs):\n np_x = np.array(case[0])\n np_y = np.array(case[1])\n np_x_grad = np.array(output[1])\n\n x = torch.from_numpy(np_x).cuda().type(dtype)\n x.requires_grad_()\n y = torch.from_numpy(np_y).cuda().long()\n\n loss = softmax_focal_loss(x, y, gamma, alpha, None, 'mean')\n loss.backward()\n\n assert np.allclose(loss.data.cpu().numpy(), output[0], 1e-2)\n assert np.allclose(x.grad.data.cpu(), np_x_grad, 1e-2)\n\n def _test_sigmoid(self, dtype=torch.float):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import sigmoid_focal_loss\n alpha = 0.25\n gamma = 2.0\n for case, output in zip(inputs, sigmoid_outputs):\n np_x = np.array(case[0])\n np_y = np.array(case[1])\n np_x_grad = np.array(output[1])\n\n x = torch.from_numpy(np_x).cuda().type(dtype)\n x.requires_grad_()\n y = torch.from_numpy(np_y).cuda().long()\n\n loss = sigmoid_focal_loss(x, y, gamma, alpha, None, 'mean')\n loss.backward()\n\n assert np.allclose(loss.data.cpu().numpy(), output[0], 1e-2)\n assert np.allclose(x.grad.data.cpu(), np_x_grad, 1e-2)\n\n def _test_grad_softmax(self, dtype=torch.float):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import SoftmaxFocalLoss\n alpha = 0.25\n gamma = 2.0\n for case in inputs:\n np_x = np.array(case[0])\n np_y = np.array(case[1])\n\n x = torch.from_numpy(np_x).cuda().type(dtype)\n x.requires_grad_()\n y = torch.from_numpy(np_y).cuda().long()\n\n floss = SoftmaxFocalLoss(gamma, alpha)\n if _USING_PARROTS:\n # gradcheck(floss, (x, y),\n # no_grads=[y])\n pass\n else:\n gradcheck(floss, (x, y), eps=1e-2, atol=1e-2)\n\n def _test_grad_sigmoid(self, dtype=torch.float):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import SigmoidFocalLoss\n alpha = 0.25\n gamma = 2.0\n for case in inputs:\n np_x = np.array(case[0])\n np_y = np.array(case[1])\n\n x = torch.from_numpy(np_x).cuda().type(dtype)\n x.requires_grad_()\n y = torch.from_numpy(np_y).cuda().long()\n\n floss = SigmoidFocalLoss(gamma, alpha)\n if _USING_PARROTS:\n # gradcheck(floss, (x, y),\n # no_grads=[y])\n pass\n else:\n gradcheck(floss, (x, y), eps=1e-2, atol=1e-2)\n\n def test_softmax_float(self):\n self._test_softmax(dtype=torch.float)\n\n def test_softmax_half(self):\n self._test_softmax(dtype=torch.half)\n\n def test_sigmoid_float(self):\n self._test_sigmoid(dtype=torch.float)\n\n def test_sigmoid_half(self):\n self._test_sigmoid(dtype=torch.half)\n\n def test_grad_softmax_float(self):\n self._test_grad_softmax(dtype=torch.float)\n\n def test_grad_sigmoid_float(self):\n self._test_grad_sigmoid(dtype=torch.float)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_furthest_point_sample.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import furthest_point_sample, furthest_point_sample_with_dist\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_fps():\n xyz = torch.tensor([[[-0.2748, 1.0020, -1.1674], [0.1015, 1.3952, -1.2681],\n [-0.8070, 2.4137,\n -0.5845], [-1.0001, 2.1982, -0.5859],\n [0.3841, 1.8983, -0.7431]],\n [[-1.0696, 3.0758,\n -0.1899], [-0.2559, 3.5521, -0.1402],\n [0.8164, 4.0081, -0.1839], [-1.1000, 3.0213, -0.8205],\n [-0.0518, 3.7251, -0.3950]]]).cuda()\n\n idx = furthest_point_sample(xyz, 3)\n expected_idx = torch.tensor([[0, 2, 4], [0, 2, 1]]).cuda()\n assert torch.all(idx == expected_idx)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_fps_with_dist():\n xyz = torch.tensor([[[-0.2748, 1.0020, -1.1674], [0.1015, 1.3952, -1.2681],\n [-0.8070, 2.4137,\n -0.5845], [-1.0001, 2.1982, -0.5859],\n [0.3841, 1.8983, -0.7431]],\n [[-1.0696, 3.0758,\n -0.1899], [-0.2559, 3.5521, -0.1402],\n [0.8164, 4.0081, -0.1839], [-1.1000, 3.0213, -0.8205],\n [-0.0518, 3.7251, -0.3950]]]).cuda()\n\n expected_idx = torch.tensor([[0, 2, 4], [0, 2, 1]]).cuda()\n xyz_square_dist = ((xyz.unsqueeze(dim=1) -\n xyz.unsqueeze(dim=2))**2).sum(-1)\n idx = furthest_point_sample_with_dist(xyz_square_dist, 3)\n assert torch.all(idx == expected_idx)\n\n import numpy as np\n fps_idx = np.load('tests/data/for_3d_ops/fps_idx.npy')\n features_for_fps_distance = np.load(\n 'tests/data/for_3d_ops/features_for_fps_distance.npy')\n expected_idx = torch.from_numpy(fps_idx).cuda()\n features_for_fps_distance = torch.from_numpy(\n features_for_fps_distance).cuda()\n\n idx = furthest_point_sample_with_dist(features_for_fps_distance, 16)\n assert torch.all(idx == expected_idx)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_fused_bias_leakyrelu.py", "content": "import pytest\nimport torch\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck, gradgradcheck\n _USING_PARROTS = False\n\n\nclass TestFusedBiasLeakyReLU(object):\n\n @classmethod\n def setup_class(cls):\n if not torch.cuda.is_available():\n return\n cls.input_tensor = torch.randn((2, 2, 2, 2), requires_grad=True).cuda()\n cls.bias = torch.zeros(2, requires_grad=True).cuda()\n\n @pytest.mark.skipif(not torch.cuda.is_available(), reason='requires cuda')\n def test_gradient(self):\n\n from mmcv.ops import FusedBiasLeakyReLU\n if _USING_PARROTS:\n gradcheck(\n FusedBiasLeakyReLU(2).cuda(),\n self.input_tensor,\n delta=1e-4,\n pt_atol=1e-3)\n else:\n gradcheck(\n FusedBiasLeakyReLU(2).cuda(),\n self.input_tensor,\n eps=1e-4,\n atol=1e-3)\n\n @pytest.mark.skipif(\n not torch.cuda.is_available() or _USING_PARROTS,\n reason='requires cuda')\n def test_gradgradient(self):\n\n from mmcv.ops import FusedBiasLeakyReLU\n gradgradcheck(\n FusedBiasLeakyReLU(2).cuda(),\n self.input_tensor,\n eps=1e-4,\n atol=1e-3)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_gather_points.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import gather_points\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_gather_points():\n features = torch.tensor([[[\n -1.6095, -0.1029, -0.8876, -1.2447, -2.4031, 0.3708, -1.1586, -1.4967,\n -0.4800, 0.2252\n ],\n [\n 1.9138, 3.4979, 1.6854, 1.5631, 3.6776,\n 3.1154, 2.1705, 2.5221, 2.0411, 3.1446\n ],\n [\n -1.4173, 0.3073, -1.4339, -1.4340, -1.2770,\n -0.2867, -1.4162, -1.4044, -1.4245, -1.4074\n ]],\n [[\n 0.2160, 0.0842, 0.3661, -0.2749, -0.4909,\n -0.6066, -0.8773, -0.0745, -0.9496, 0.1434\n ],\n [\n 1.3644, 1.8087, 1.6855, 1.9563, 1.2746,\n 1.9662, 0.9566, 1.8778, 1.1437, 1.3639\n ],\n [\n -0.7172, 0.1692, 0.2241, 0.0721, -0.7540,\n 0.0462, -0.6227, 0.3223, -0.6944, -0.5294\n ]]]).cuda()\n\n idx = torch.tensor([[0, 1, 4, 0, 0, 0], [0, 5, 6, 0, 0, 0]]).int().cuda()\n\n output = gather_points(features, idx)\n expected_output = torch.tensor(\n [[[-1.6095, -0.1029, -2.4031, -1.6095, -1.6095, -1.6095],\n [1.9138, 3.4979, 3.6776, 1.9138, 1.9138, 1.9138],\n [-1.4173, 0.3073, -1.2770, -1.4173, -1.4173, -1.4173]],\n [[0.2160, -0.6066, -0.8773, 0.2160, 0.2160, 0.2160],\n [1.3644, 1.9662, 0.9566, 1.3644, 1.3644, 1.3644],\n [-0.7172, 0.0462, -0.6227, -0.7172, -0.7172, -0.7172]]]).cuda()\n\n assert torch.allclose(output, expected_output)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_group_points.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import grouping_operation\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_grouping_points():\n idx = torch.tensor([[[0, 0, 0], [3, 3, 3], [8, 8, 8], [0, 0, 0], [0, 0, 0],\n [0, 0, 0]],\n [[0, 0, 0], [6, 6, 6], [9, 9, 9], [0, 0, 0], [0, 0, 0],\n [0, 0, 0]]]).int().cuda()\n festures = torch.tensor([[[\n 0.5798, -0.7981, -0.9280, -1.3311, 1.3687, 0.9277, -0.4164, -1.8274,\n 0.9268, 0.8414\n ],\n [\n 5.4247, 1.5113, 2.3944, 1.4740, 5.0300,\n 5.1030, 1.9360, 2.1939, 2.1581, 3.4666\n ],\n [\n -1.6266, -1.0281, -1.0393, -1.6931, -1.3982,\n -0.5732, -1.0830, -1.7561, -1.6786, -1.6967\n ]],\n [[\n -0.0380, -0.1880, -1.5724, 0.6905, -0.3190,\n 0.7798, -0.3693, -0.9457, -0.2942, -1.8527\n ],\n [\n 1.1773, 1.5009, 2.6399, 5.9242, 1.0962,\n 2.7346, 6.0865, 1.5555, 4.3303, 2.8229\n ],\n [\n -0.6646, -0.6870, -0.1125, -0.2224, -0.3445,\n -1.4049, 0.4990, -0.7037, -0.9924, 0.0386\n ]]]).cuda()\n\n output = grouping_operation(festures, idx)\n expected_output = torch.tensor([[[[0.5798, 0.5798, 0.5798],\n [-1.3311, -1.3311, -1.3311],\n [0.9268, 0.9268, 0.9268],\n [0.5798, 0.5798, 0.5798],\n [0.5798, 0.5798, 0.5798],\n [0.5798, 0.5798, 0.5798]],\n [[5.4247, 5.4247, 5.4247],\n [1.4740, 1.4740, 1.4740],\n [2.1581, 2.1581, 2.1581],\n [5.4247, 5.4247, 5.4247],\n [5.4247, 5.4247, 5.4247],\n [5.4247, 5.4247, 5.4247]],\n [[-1.6266, -1.6266, -1.6266],\n [-1.6931, -1.6931, -1.6931],\n [-1.6786, -1.6786, -1.6786],\n [-1.6266, -1.6266, -1.6266],\n [-1.6266, -1.6266, -1.6266],\n [-1.6266, -1.6266, -1.6266]]],\n [[[-0.0380, -0.0380, -0.0380],\n [-0.3693, -0.3693, -0.3693],\n [-1.8527, -1.8527, -1.8527],\n [-0.0380, -0.0380, -0.0380],\n [-0.0380, -0.0380, -0.0380],\n [-0.0380, -0.0380, -0.0380]],\n [[1.1773, 1.1773, 1.1773],\n [6.0865, 6.0865, 6.0865],\n [2.8229, 2.8229, 2.8229],\n [1.1773, 1.1773, 1.1773],\n [1.1773, 1.1773, 1.1773],\n [1.1773, 1.1773, 1.1773]],\n [[-0.6646, -0.6646, -0.6646],\n [0.4990, 0.4990, 0.4990],\n [0.0386, 0.0386, 0.0386],\n [-0.6646, -0.6646, -0.6646],\n [-0.6646, -0.6646, -0.6646],\n [-0.6646, -0.6646, -0.6646]]]]).cuda()\n assert torch.allclose(output, expected_output)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_info.py", "content": "import torch\n\n\nclass TestInfo(object):\n\n def test_info(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import get_compiler_version, get_compiling_cuda_version\n cv = get_compiler_version()\n ccv = get_compiling_cuda_version()\n assert cv is not None\n assert ccv is not None\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_iou3d.py", "content": "import numpy as np\nimport pytest\nimport torch\n\nfrom mmcv.ops import boxes_iou_bev, nms_bev, nms_normal_bev\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_boxes_iou_bev():\n np_boxes1 = np.asarray(\n [[1.0, 1.0, 3.0, 4.0, 0.5], [2.0, 2.0, 3.0, 4.0, 0.6],\n [7.0, 7.0, 8.0, 8.0, 0.4]],\n dtype=np.float32)\n np_boxes2 = np.asarray(\n [[0.0, 2.0, 2.0, 5.0, 0.3], [2.0, 1.0, 3.0, 3.0, 0.5],\n [5.0, 5.0, 6.0, 7.0, 0.4]],\n dtype=np.float32)\n np_expect_ious = np.asarray(\n [[0.2621, 0.2948, 0.0000], [0.0549, 0.1587, 0.0000],\n [0.0000, 0.0000, 0.0000]],\n dtype=np.float32)\n\n boxes1 = torch.from_numpy(np_boxes1).cuda()\n boxes2 = torch.from_numpy(np_boxes2).cuda()\n\n ious = boxes_iou_bev(boxes1, boxes2)\n assert np.allclose(ious.cpu().numpy(), np_expect_ious, atol=1e-4)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_nms_bev():\n np_boxes = np.array(\n [[6.0, 3.0, 8.0, 7.0, 2.0], [3.0, 6.0, 9.0, 11.0, 1.0],\n [3.0, 7.0, 10.0, 12.0, 1.0], [1.0, 4.0, 13.0, 7.0, 3.0]],\n dtype=np.float32)\n np_scores = np.array([0.6, 0.9, 0.7, 0.2], dtype=np.float32)\n np_inds = np.array([1, 0, 3])\n boxes = torch.from_numpy(np_boxes)\n scores = torch.from_numpy(np_scores)\n inds = nms_bev(boxes.cuda(), scores.cuda(), thresh=0.3)\n\n assert np.allclose(inds.cpu().numpy(), np_inds)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_nms_normal_bev():\n np_boxes = np.array(\n [[6.0, 3.0, 8.0, 7.0, 2.0], [3.0, 6.0, 9.0, 11.0, 1.0],\n [3.0, 7.0, 10.0, 12.0, 1.0], [1.0, 4.0, 13.0, 7.0, 3.0]],\n dtype=np.float32)\n np_scores = np.array([0.6, 0.9, 0.7, 0.2], dtype=np.float32)\n np_inds = np.array([1, 0, 3])\n boxes = torch.from_numpy(np_boxes)\n scores = torch.from_numpy(np_scores)\n inds = nms_normal_bev(boxes.cuda(), scores.cuda(), thresh=0.3)\n\n assert np.allclose(inds.cpu().numpy(), np_inds)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_knn.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import knn\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_knn():\n new_xyz = torch.tensor([[[-0.0740, 1.3147, -1.3625],\n [-2.2769, 2.7817, -0.2334],\n [-0.4003, 2.4666, -0.5116],\n [-0.0740, 1.3147, -1.3625],\n [-0.0740, 1.3147, -1.3625]],\n [[-2.0289, 2.4952, -0.1708],\n [-2.0668, 6.0278, -0.4875],\n [0.4066, 1.4211, -0.2947],\n [-2.0289, 2.4952, -0.1708],\n [-2.0289, 2.4952, -0.1708]]]).cuda()\n\n xyz = torch.tensor([[[-0.0740, 1.3147, -1.3625], [0.5555, 1.0399, -1.3634],\n [-0.4003, 2.4666,\n -0.5116], [-0.5251, 2.4379, -0.8466],\n [-0.9691, 1.1418,\n -1.3733], [-0.2232, 0.9561, -1.3626],\n [-2.2769, 2.7817, -0.2334],\n [-0.2822, 1.3192, -1.3645], [0.1533, 1.5024, -1.0432],\n [0.4917, 1.1529, -1.3496]],\n [[-2.0289, 2.4952,\n -0.1708], [-0.7188, 0.9956, -0.5096],\n [-2.0668, 6.0278, -0.4875], [-1.9304, 3.3092, 0.6610],\n [0.0949, 1.4332, 0.3140], [-1.2879, 2.0008, -0.7791],\n [-0.7252, 0.9611, -0.6371], [0.4066, 1.4211, -0.2947],\n [0.3220, 1.4447, 0.3548], [-0.9744, 2.3856,\n -1.2000]]]).cuda()\n\n idx = knn(5, xyz, new_xyz)\n new_xyz_ = new_xyz.unsqueeze(2).repeat(1, 1, xyz.shape[1], 1)\n xyz_ = xyz.unsqueeze(1).repeat(1, new_xyz.shape[1], 1, 1)\n dist = ((new_xyz_ - xyz_) * (new_xyz_ - xyz_)).sum(-1)\n expected_idx = dist.topk(k=5, dim=2, largest=False)[1].transpose(2, 1)\n assert torch.all(idx == expected_idx)\n\n idx = knn(5,\n xyz.transpose(1, 2).contiguous(),\n new_xyz.transpose(1, 2).contiguous(), True)\n assert torch.all(idx == expected_idx)\n\n idx = knn(5, xyz, xyz)\n xyz_ = xyz.unsqueeze(2).repeat(1, 1, xyz.shape[1], 1)\n xyz__ = xyz.unsqueeze(1).repeat(1, xyz.shape[1], 1, 1)\n dist = ((xyz_ - xyz__) * (xyz_ - xyz__)).sum(-1)\n expected_idx = dist.topk(k=5, dim=2, largest=False)[1].transpose(2, 1)\n assert torch.all(idx == expected_idx)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_masked_conv2d.py", "content": "import torch\n\n\nclass TestMaskedConv2d(object):\n\n def test_masked_conv2d(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import MaskedConv2d\n input = torch.randn(1, 3, 16, 16, requires_grad=True, device='cuda')\n mask = torch.randn(1, 16, 16, requires_grad=True, device='cuda')\n conv = MaskedConv2d(3, 3, 3).cuda()\n output = conv(input, mask)\n assert output is not None\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_merge_cells.py", "content": "\"\"\"\nCommandLine:\n pytest tests/test_merge_cells.py\n\"\"\"\nimport torch\nimport torch.nn.functional as F\n\nfrom mmcv.ops.merge_cells import (BaseMergeCell, ConcatCell, GlobalPoolingCell,\n SumCell)\n\n\ndef test_sum_cell():\n inputs_x = torch.randn([2, 256, 32, 32])\n inputs_y = torch.randn([2, 256, 16, 16])\n sum_cell = SumCell(256, 256)\n output = sum_cell(inputs_x, inputs_y, out_size=inputs_x.shape[-2:])\n assert output.size() == inputs_x.size()\n output = sum_cell(inputs_x, inputs_y, out_size=inputs_y.shape[-2:])\n assert output.size() == inputs_y.size()\n output = sum_cell(inputs_x, inputs_y)\n assert output.size() == inputs_x.size()\n\n\ndef test_concat_cell():\n inputs_x = torch.randn([2, 256, 32, 32])\n inputs_y = torch.randn([2, 256, 16, 16])\n concat_cell = ConcatCell(256, 256)\n output = concat_cell(inputs_x, inputs_y, out_size=inputs_x.shape[-2:])\n assert output.size() == inputs_x.size()\n output = concat_cell(inputs_x, inputs_y, out_size=inputs_y.shape[-2:])\n assert output.size() == inputs_y.size()\n output = concat_cell(inputs_x, inputs_y)\n assert output.size() == inputs_x.size()\n\n\ndef test_global_pool_cell():\n inputs_x = torch.randn([2, 256, 32, 32])\n inputs_y = torch.randn([2, 256, 32, 32])\n gp_cell = GlobalPoolingCell(with_out_conv=False)\n gp_cell_out = gp_cell(inputs_x, inputs_y, out_size=inputs_x.shape[-2:])\n assert (gp_cell_out.size() == inputs_x.size())\n gp_cell = GlobalPoolingCell(256, 256)\n gp_cell_out = gp_cell(inputs_x, inputs_y, out_size=inputs_x.shape[-2:])\n assert (gp_cell_out.size() == inputs_x.size())\n\n\ndef test_resize_methods():\n inputs_x = torch.randn([2, 256, 128, 128])\n target_resize_sizes = [(128, 128), (256, 256)]\n resize_methods_list = ['nearest', 'bilinear']\n\n for method in resize_methods_list:\n merge_cell = BaseMergeCell(upsample_mode=method)\n for target_size in target_resize_sizes:\n merge_cell_out = merge_cell._resize(inputs_x, target_size)\n gt_out = F.interpolate(inputs_x, size=target_size, mode=method)\n assert merge_cell_out.equal(gt_out)\n\n target_size = (64, 64) # resize to a smaller size\n merge_cell = BaseMergeCell()\n merge_cell_out = merge_cell._resize(inputs_x, target_size)\n kernel_size = inputs_x.shape[-1] // target_size[-1]\n gt_out = F.max_pool2d(\n inputs_x, kernel_size=kernel_size, stride=kernel_size)\n assert (merge_cell_out == gt_out).all()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_min_area_polygons.py", "content": "import numpy as np\nimport pytest\nimport torch\n\nfrom mmcv.ops import min_area_polygons\n\nnp_pointsets = np.asarray([[\n 1.0, 1.0, 2.0, 2.0, 1.0, 2.0, 2.0, 1.0, 1.0, 3.0, 3.0, 1.0, 2.0, 3.0, 3.0,\n 2.0, 1.5, 1.5\n],\n [\n 1.0, 1.0, 8.0, 8.0, 1.0, 2.0, 2.0, 1.0, 1.0,\n 3.0, 3.0, 1.0, 2.0, 3.0, 3.0, 2.0, 1.5, 1.5\n ]])\n\nexpected_polygons = np.asarray(\n [[3.0000, 1.0000, 1.0000, 1.0000, 1.0000, 3.0000, 3.0000, 3.0000],\n [8.0, 8.0, 2.3243, 0.0541, 0.0541, 1.6757, 5.7297, 9.6216]])\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_min_area_polygons():\n pointsets = torch.from_numpy(np_pointsets).cuda().float()\n\n assert np.allclose(\n min_area_polygons(pointsets).cpu().numpy(),\n expected_polygons,\n atol=1e-4)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_modulated_deform_conv.py", "content": "import os\n\nimport numpy\nimport pytest\nimport torch\n\nfrom mmcv.utils import TORCH_VERSION, digit_version\n\ntry:\n # If PyTorch version >= 1.6.0 and fp16 is enabled, torch.cuda.amp.autocast\n # would be imported and used; we should test if our modules support it.\n from torch.cuda.amp import autocast\nexcept ImportError:\n pass\n\ncur_dir = os.path.dirname(os.path.abspath(__file__))\n\ninput_t = [[[[1., 2., 3.], [1., 2., 3.], [1., 2., 3.]]]]\noutput_t = [[[[0.5, 1.5, 2.5, 1.5], [1.0, 3.0, 5.0, 3.0], [1.0, 3.0, 5.0, 3.0],\n [0.5, 1.5, 2.5, 1.5]]]]\ninput_grad = [[[[2., 2., 2.], [2., 2., 2.], [2., 2., 2.]]]]\ndcn_w_grad = [[[[9., 9.], [9., 9.]]]]\ndcn_offset_w_grad = [[[[-7.0, -4.0], [0.0, 0.0]]], [[[-9.0, 7.5], [-6.0,\n 5.0]]],\n [[[-4.0, -7.0], [0.0, 0.0]]],\n [[[-7.5, -9.0], [-5.0, -6.0]]],\n [[[-7.0, -4.0], [-7.0, -4.0]]],\n [[[-6.0, 5.0], [-9.0, 7.5]]],\n [[[-4.0, -7.0], [-4.0, -7.0]]],\n [[[-5.0, -6.0], [-7.5, -9.0]]], [[[10.5, 6.0], [7.0,\n 4.0]]],\n [[[6.0, 10.5], [4.0, 7.0]]], [[[7.0, 4.0], [10.5, 6.0]]],\n [[[4.0, 7.0], [6.0, 10.5]]]]\ndcn_offset_b_grad = [\n -3.0, -1.5, -3.0, -1.5, -3.0, -1.5, -3.0, -1.5, 4.5, 4.5, 4.5, 4.5\n]\n\n\nclass TestMdconv(object):\n\n def _test_mdconv(self, dtype=torch.float, device='cuda'):\n if not torch.cuda.is_available() and device == 'cuda':\n pytest.skip('test requires GPU')\n from mmcv.ops import ModulatedDeformConv2dPack\n input = torch.tensor(input_t, dtype=dtype, device=device)\n input.requires_grad = True\n\n dcn = ModulatedDeformConv2dPack(\n 1,\n 1,\n kernel_size=(2, 2),\n stride=1,\n padding=1,\n deform_groups=1,\n bias=False)\n\n if device == 'cuda':\n dcn.cuda()\n\n dcn.weight.data.fill_(1.)\n dcn.type(dtype)\n output = dcn(input)\n output.sum().backward()\n assert numpy.allclose(output.cpu().detach().numpy(), output_t, 1e-2)\n assert numpy.allclose(input.grad.cpu().detach().numpy(), input_grad,\n 1e-2)\n assert numpy.allclose(dcn.weight.grad.cpu().detach().numpy(),\n dcn_w_grad, 1e-2)\n assert numpy.allclose(\n dcn.conv_offset.weight.grad.cpu().detach().numpy(),\n dcn_offset_w_grad, 1e-2)\n assert numpy.allclose(dcn.conv_offset.bias.grad.cpu().detach().numpy(),\n dcn_offset_b_grad, 1e-2)\n\n def _test_amp_mdconv(self, input_dtype=torch.float):\n \"\"\"The function to test amp released on pytorch 1.6.0.\n\n The type of input data might be torch.float or torch.half,\n so we should test mdconv in both cases. With amp, the data\n type of model will NOT be set manually.\n\n Args:\n input_dtype: torch.float or torch.half.\n \"\"\"\n if not torch.cuda.is_available():\n return\n from mmcv.ops import ModulatedDeformConv2dPack\n input = torch.tensor(input_t).cuda().type(input_dtype)\n input.requires_grad = True\n\n dcn = ModulatedDeformConv2dPack(\n 1,\n 1,\n kernel_size=(2, 2),\n stride=1,\n padding=1,\n deform_groups=1,\n bias=False).cuda()\n dcn.weight.data.fill_(1.)\n output = dcn(input)\n output.sum().backward()\n assert numpy.allclose(output.cpu().detach().numpy(), output_t, 1e-2)\n assert numpy.allclose(input.grad.cpu().detach().numpy(), input_grad,\n 1e-2)\n assert numpy.allclose(dcn.weight.grad.cpu().detach().numpy(),\n dcn_w_grad, 1e-2)\n assert numpy.allclose(\n dcn.conv_offset.weight.grad.cpu().detach().numpy(),\n dcn_offset_w_grad, 1e-2)\n assert numpy.allclose(dcn.conv_offset.bias.grad.cpu().detach().numpy(),\n dcn_offset_b_grad, 1e-2)\n\n def test_mdconv(self):\n self._test_mdconv(torch.double, device='cpu')\n self._test_mdconv(torch.float, device='cpu')\n self._test_mdconv(torch.double)\n self._test_mdconv(torch.float)\n self._test_mdconv(torch.half)\n\n # test amp when torch version >= '1.6.0', the type of\n # input data for mdconv might be torch.float or torch.half\n if (TORCH_VERSION != 'parrots'\n and digit_version(TORCH_VERSION) >= digit_version('1.6.0')):\n with autocast(enabled=True):\n self._test_amp_mdconv(torch.float)\n self._test_amp_mdconv(torch.half)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_ms_deformable_attn.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops.multi_scale_deform_attn import (\n MultiScaleDeformableAttention, MultiScaleDeformableAttnFunction,\n multi_scale_deformable_attn_pytorch)\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck\n _USING_PARROTS = False\n\n\n@pytest.mark.parametrize('device_type', [\n 'cpu',\n pytest.param(\n 'cuda:0',\n marks=pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support'))\n])\ndef test_multiscale_deformable_attention(device_type):\n\n with pytest.raises(ValueError):\n # embed_dims must be divisible by num_heads,\n MultiScaleDeformableAttention(\n embed_dims=256,\n num_heads=7,\n )\n device = torch.device(device_type)\n msda = MultiScaleDeformableAttention(\n embed_dims=3, num_levels=2, num_heads=3)\n msda.init_weights()\n num_query = 5\n bs = 1\n embed_dims = 3\n query = torch.rand(num_query, bs, embed_dims).to(device)\n key = torch.rand(num_query, bs, embed_dims).to(device)\n spatial_shapes = torch.Tensor([[2, 2], [1, 1]]).long().to(device)\n level_start_index = torch.Tensor([0, 4]).long().to(device)\n reference_points = torch.rand(bs, num_query, 2, 2).to(device)\n msda.to(device)\n msda(\n query,\n key,\n key,\n reference_points=reference_points,\n spatial_shapes=spatial_shapes,\n level_start_index=level_start_index)\n\n\ndef test_forward_multi_scale_deformable_attn_pytorch():\n N, M, D = 1, 2, 2\n Lq, L, P = 2, 2, 2\n shapes = torch.as_tensor([(6, 4), (3, 2)], dtype=torch.long)\n S = sum([(H * W).item() for H, W in shapes])\n\n torch.manual_seed(3)\n value = torch.rand(N, S, M, D) * 0.01\n sampling_locations = torch.rand(N, Lq, M, L, P, 2)\n attention_weights = torch.rand(N, Lq, M, L, P) + 1e-5\n attention_weights /= attention_weights.sum(\n -1, keepdim=True).sum(\n -2, keepdim=True)\n\n multi_scale_deformable_attn_pytorch(value.double(), shapes,\n sampling_locations.double(),\n attention_weights.double()).detach()\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_forward_equal_with_pytorch_double():\n N, M, D = 1, 2, 2\n Lq, L, P = 2, 2, 2\n shapes = torch.as_tensor([(6, 4), (3, 2)], dtype=torch.long).cuda()\n level_start_index = torch.cat((shapes.new_zeros(\n (1, )), shapes.prod(1).cumsum(0)[:-1]))\n S = sum([(H * W).item() for H, W in shapes])\n\n torch.manual_seed(3)\n value = torch.rand(N, S, M, D).cuda() * 0.01\n sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()\n attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5\n attention_weights /= attention_weights.sum(\n -1, keepdim=True).sum(\n -2, keepdim=True)\n im2col_step = 2\n output_pytorch = multi_scale_deformable_attn_pytorch(\n value.double(), shapes, sampling_locations.double(),\n attention_weights.double()).detach().cpu()\n\n output_cuda = MultiScaleDeformableAttnFunction.apply(\n value.double(), shapes, level_start_index, sampling_locations.double(),\n attention_weights.double(), im2col_step).detach().cpu()\n assert torch.allclose(output_cuda, output_pytorch)\n max_abs_err = (output_cuda - output_pytorch).abs().max()\n max_rel_err = ((output_cuda - output_pytorch).abs() /\n output_pytorch.abs()).max()\n assert max_abs_err < 1e-18\n assert max_rel_err < 1e-15\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_forward_equal_with_pytorch_float():\n N, M, D = 1, 2, 2\n Lq, L, P = 2, 2, 2\n shapes = torch.as_tensor([(6, 4), (3, 2)], dtype=torch.long).cuda()\n level_start_index = torch.cat((shapes.new_zeros(\n (1, )), shapes.prod(1).cumsum(0)[:-1]))\n S = sum([(H * W).item() for H, W in shapes])\n\n torch.manual_seed(3)\n value = torch.rand(N, S, M, D).cuda() * 0.01\n sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()\n attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5\n attention_weights /= attention_weights.sum(\n -1, keepdim=True).sum(\n -2, keepdim=True)\n im2col_step = 2\n output_pytorch = multi_scale_deformable_attn_pytorch(\n value, shapes, sampling_locations, attention_weights).detach().cpu()\n\n output_cuda = MultiScaleDeformableAttnFunction.apply(\n value, shapes, level_start_index, sampling_locations,\n attention_weights, im2col_step).detach().cpu()\n assert torch.allclose(output_cuda, output_pytorch, rtol=1e-2, atol=1e-3)\n max_abs_err = (output_cuda - output_pytorch).abs().max()\n max_rel_err = ((output_cuda - output_pytorch).abs() /\n output_pytorch.abs()).max()\n assert max_abs_err < 1e-9\n assert max_rel_err < 1e-6\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\n@pytest.mark.parametrize('channels', [\n 4,\n 30,\n 32,\n 64,\n 71,\n 1025,\n])\ndef test_gradient_numerical(channels,\n grad_value=True,\n grad_sampling_loc=True,\n grad_attn_weight=True):\n\n N, M, _ = 1, 2, 2\n Lq, L, P = 2, 2, 2\n shapes = torch.as_tensor([(3, 2), (2, 1)], dtype=torch.long).cuda()\n level_start_index = torch.cat((shapes.new_zeros(\n (1, )), shapes.prod(1).cumsum(0)[:-1]))\n S = sum([(H * W).item() for H, W in shapes])\n\n value = torch.rand(N, S, M, channels).cuda() * 0.01\n sampling_locations = torch.rand(N, Lq, M, L, P, 2).cuda()\n attention_weights = torch.rand(N, Lq, M, L, P).cuda() + 1e-5\n attention_weights /= attention_weights.sum(\n -1, keepdim=True).sum(\n -2, keepdim=True)\n im2col_step = 2\n\n func = MultiScaleDeformableAttnFunction.apply\n\n value.requires_grad = grad_value\n sampling_locations.requires_grad = grad_sampling_loc\n attention_weights.requires_grad = grad_attn_weight\n if _USING_PARROTS:\n assert gradcheck(\n func, (value.double(), shapes, level_start_index,\n sampling_locations.double(), attention_weights.double(),\n im2col_step),\n no_grads=[shapes, level_start_index])\n else:\n assert gradcheck(func, (value.double(), shapes, level_start_index,\n sampling_locations.double(),\n attention_weights.double(), im2col_step))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_nms.py", "content": "import numpy as np\nimport pytest\nimport torch\n\n\nclass Testnms(object):\n\n def test_nms_allclose(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import nms\n np_boxes = np.array([[6.0, 3.0, 8.0, 7.0], [3.0, 6.0, 9.0, 11.0],\n [3.0, 7.0, 10.0, 12.0], [1.0, 4.0, 13.0, 7.0]],\n dtype=np.float32)\n np_scores = np.array([0.6, 0.9, 0.7, 0.2], dtype=np.float32)\n np_inds = np.array([1, 0, 3])\n np_dets = np.array([[3.0, 6.0, 9.0, 11.0, 0.9],\n [6.0, 3.0, 8.0, 7.0, 0.6],\n [1.0, 4.0, 13.0, 7.0, 0.2]])\n boxes = torch.from_numpy(np_boxes)\n scores = torch.from_numpy(np_scores)\n dets, inds = nms(boxes, scores, iou_threshold=0.3, offset=0)\n assert np.allclose(dets, np_dets) # test cpu\n assert np.allclose(inds, np_inds) # test cpu\n dets, inds = nms(\n boxes.cuda(), scores.cuda(), iou_threshold=0.3, offset=0)\n assert np.allclose(dets.cpu().numpy(), np_dets) # test gpu\n assert np.allclose(inds.cpu().numpy(), np_inds) # test gpu\n\n def test_softnms_allclose(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import soft_nms\n np_boxes = np.array([[6.0, 3.0, 8.0, 7.0], [3.0, 6.0, 9.0, 11.0],\n [3.0, 7.0, 10.0, 12.0], [1.0, 4.0, 13.0, 7.0]],\n dtype=np.float32)\n np_scores = np.array([0.6, 0.9, 0.7, 0.2], dtype=np.float32)\n\n np_output = {\n 'linear': {\n 'dets':\n np.array(\n [[3., 6., 9., 11., 0.9], [6., 3., 8., 7., 0.6],\n [3., 7., 10., 12., 0.29024392], [1., 4., 13., 7., 0.2]],\n dtype=np.float32),\n 'inds':\n np.array([1, 0, 2, 3], dtype=np.int64)\n },\n 'gaussian': {\n 'dets':\n np.array([[3., 6., 9., 11., 0.9], [6., 3., 8., 7., 0.59630775],\n [3., 7., 10., 12., 0.35275510],\n [1., 4., 13., 7., 0.18650459]],\n dtype=np.float32),\n 'inds':\n np.array([1, 0, 2, 3], dtype=np.int64)\n },\n 'naive': {\n 'dets':\n np.array([[3., 6., 9., 11., 0.9], [6., 3., 8., 7., 0.6],\n [1., 4., 13., 7., 0.2]],\n dtype=np.float32),\n 'inds':\n np.array([1, 0, 3], dtype=np.int64)\n }\n }\n\n boxes = torch.from_numpy(np_boxes)\n scores = torch.from_numpy(np_scores)\n\n configs = [[0.3, 0.5, 0.01, 'linear'], [0.3, 0.5, 0.01, 'gaussian'],\n [0.3, 0.5, 0.01, 'naive']]\n\n for iou, sig, mscore, m in configs:\n dets, inds = soft_nms(\n boxes,\n scores,\n iou_threshold=iou,\n sigma=sig,\n min_score=mscore,\n method=m)\n assert np.allclose(dets.cpu().numpy(), np_output[m]['dets'])\n assert np.allclose(inds.cpu().numpy(), np_output[m]['inds'])\n\n if torch.__version__ != 'parrots':\n boxes = boxes.cuda()\n scores = scores.cuda()\n for iou, sig, mscore, m in configs:\n dets, inds = soft_nms(\n boxes,\n scores,\n iou_threshold=iou,\n sigma=sig,\n min_score=mscore,\n method=m)\n assert np.allclose(dets.cpu().numpy(), np_output[m]['dets'])\n assert np.allclose(inds.cpu().numpy(), np_output[m]['inds'])\n\n def test_nms_match(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import nms, nms_match\n iou_thr = 0.6\n # empty input\n empty_dets = np.array([])\n assert len(nms_match(empty_dets, iou_thr)) == 0\n\n # non empty ndarray input\n np_dets = np.array(\n [[49.1, 32.4, 51.0, 35.9, 0.9], [49.3, 32.9, 51.0, 35.3, 0.9],\n [35.3, 11.5, 39.9, 14.5, 0.4], [35.2, 11.7, 39.7, 15.7, 0.3]],\n dtype=np.float32)\n np_groups = nms_match(np_dets, iou_thr)\n assert isinstance(np_groups[0], np.ndarray)\n assert len(np_groups) == 2\n tensor_dets = torch.from_numpy(np_dets)\n boxes = tensor_dets[:, :4]\n scores = tensor_dets[:, 4]\n nms_keep_inds = nms(boxes.contiguous(), scores.contiguous(),\n iou_thr)[1]\n assert set([g[0].item()\n for g in np_groups]) == set(nms_keep_inds.tolist())\n\n # non empty tensor input\n tensor_dets = torch.from_numpy(np_dets)\n tensor_groups = nms_match(tensor_dets, iou_thr)\n assert isinstance(tensor_groups[0], torch.Tensor)\n for i in range(len(tensor_groups)):\n assert np.equal(tensor_groups[i].numpy(), np_groups[i]).all()\n\n # input of wrong shape\n wrong_dets = np.zeros((2, 3))\n with pytest.raises(AssertionError):\n nms_match(wrong_dets, iou_thr)\n\n def test_batched_nms(self):\n import mmcv\n from mmcv.ops import batched_nms\n results = mmcv.load('./tests/data/batched_nms_data.pkl')\n\n nms_max_num = 100\n nms_cfg = dict(\n type='nms',\n iou_threshold=0.7,\n score_threshold=0.5,\n max_num=nms_max_num)\n boxes, keep = batched_nms(\n torch.from_numpy(results['boxes']),\n torch.from_numpy(results['scores']),\n torch.from_numpy(results['idxs']),\n nms_cfg,\n class_agnostic=False)\n\n nms_cfg.update(split_thr=100)\n seq_boxes, seq_keep = batched_nms(\n torch.from_numpy(results['boxes']),\n torch.from_numpy(results['scores']),\n torch.from_numpy(results['idxs']),\n nms_cfg,\n class_agnostic=False)\n\n assert torch.equal(keep, seq_keep)\n assert torch.equal(boxes, seq_boxes)\n assert torch.equal(keep,\n torch.from_numpy(results['keep'][:nms_max_num]))\n\n nms_cfg = dict(type='soft_nms', iou_threshold=0.7)\n boxes, keep = batched_nms(\n torch.from_numpy(results['boxes']),\n torch.from_numpy(results['scores']),\n torch.from_numpy(results['idxs']),\n nms_cfg,\n class_agnostic=False)\n\n nms_cfg.update(split_thr=100)\n seq_boxes, seq_keep = batched_nms(\n torch.from_numpy(results['boxes']),\n torch.from_numpy(results['scores']),\n torch.from_numpy(results['idxs']),\n nms_cfg,\n class_agnostic=False)\n\n assert torch.equal(keep, seq_keep)\n assert torch.equal(boxes, seq_boxes)\n\n # test skip nms when `nms_cfg` is None\n seq_boxes, seq_keep = batched_nms(\n torch.from_numpy(results['boxes']),\n torch.from_numpy(results['scores']),\n torch.from_numpy(results['idxs']),\n None,\n class_agnostic=False)\n assert len(seq_keep) == len(results['boxes'])\n # assert score is descending order\n assert ((seq_boxes[:, -1][1:] - seq_boxes[:, -1][:-1]) < 0).all()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_nms_rotated.py", "content": "import numpy as np\nimport pytest\nimport torch\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(),\n reason='GPU is required to test NMSRotated op')\nclass TestNmsRotated:\n\n def test_ml_nms_rotated(self):\n from mmcv.ops import nms_rotated\n np_boxes = np.array(\n [[6.0, 3.0, 8.0, 7.0, 0.5, 0.7], [3.0, 6.0, 9.0, 11.0, 0.6, 0.8],\n [3.0, 7.0, 10.0, 12.0, 0.3, 0.5], [1.0, 4.0, 13.0, 7.0, 0.6, 0.9]\n ],\n dtype=np.float32)\n np_labels = np.array([1, 0, 1, 0], dtype=np.float32)\n\n np_expect_dets = np.array(\n [[1.0, 4.0, 13.0, 7.0, 0.6], [3.0, 6.0, 9.0, 11.0, 0.6],\n [6.0, 3.0, 8.0, 7.0, 0.5]],\n dtype=np.float32)\n np_expect_keep_inds = np.array([3, 1, 0], dtype=np.int64)\n\n boxes = torch.from_numpy(np_boxes).cuda()\n labels = torch.from_numpy(np_labels).cuda()\n\n # test cw angle definition\n dets, keep_inds = nms_rotated(boxes[:, :5], boxes[:, -1], 0.5, labels)\n\n assert np.allclose(dets.cpu().numpy()[:, :5], np_expect_dets)\n assert np.allclose(keep_inds.cpu().numpy(), np_expect_keep_inds)\n\n # test ccw angle definition\n boxes[..., -2] *= -1\n dets, keep_inds = nms_rotated(\n boxes[:, :5], boxes[:, -1], 0.5, labels, clockwise=False)\n dets[..., -2] *= -1\n assert np.allclose(dets.cpu().numpy()[:, :5], np_expect_dets)\n assert np.allclose(keep_inds.cpu().numpy(), np_expect_keep_inds)\n\n def test_nms_rotated(self):\n from mmcv.ops import nms_rotated\n np_boxes = np.array(\n [[6.0, 3.0, 8.0, 7.0, 0.5, 0.7], [3.0, 6.0, 9.0, 11.0, 0.6, 0.8],\n [3.0, 7.0, 10.0, 12.0, 0.3, 0.5], [1.0, 4.0, 13.0, 7.0, 0.6, 0.9]\n ],\n dtype=np.float32)\n\n np_expect_dets = np.array(\n [[1.0, 4.0, 13.0, 7.0, 0.6], [3.0, 6.0, 9.0, 11.0, 0.6],\n [6.0, 3.0, 8.0, 7.0, 0.5]],\n dtype=np.float32)\n np_expect_keep_inds = np.array([3, 1, 0], dtype=np.int64)\n\n boxes = torch.from_numpy(np_boxes).cuda()\n\n # test cw angle definition\n dets, keep_inds = nms_rotated(boxes[:, :5], boxes[:, -1], 0.5)\n assert np.allclose(dets.cpu().numpy()[:, :5], np_expect_dets)\n assert np.allclose(keep_inds.cpu().numpy(), np_expect_keep_inds)\n\n # test ccw angle definition\n boxes[..., -2] *= -1\n dets, keep_inds = nms_rotated(\n boxes[:, :5], boxes[:, -1], 0.5, clockwise=False)\n dets[..., -2] *= -1\n assert np.allclose(dets.cpu().numpy()[:, :5], np_expect_dets)\n assert np.allclose(keep_inds.cpu().numpy(), np_expect_keep_inds)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_onnx.py", "content": "import os\nimport warnings\nfrom functools import partial\n\nimport numpy as np\nimport onnx\nimport onnxruntime as rt\nimport pytest\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom packaging import version\n\nonnx_file = 'tmp.onnx'\n\n\n@pytest.fixture(autouse=True)\ndef run_before_and_after_test():\n # clear onnx_file before test\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n\n yield\n\n # clear onnx_file after test\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n\n\nclass WrapFunction(nn.Module):\n\n def __init__(self, wrapped_function):\n super(WrapFunction, self).__init__()\n self.wrapped_function = wrapped_function\n\n def forward(self, *args, **kwargs):\n return self.wrapped_function(*args, **kwargs)\n\n\ndef process_grid_sample(func, input, grid, ort_custom_op_path=''):\n wrapped_model = WrapFunction(func).eval()\n\n input_names = ['input', 'grid']\n output_names = ['output']\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (input, grid),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=input_names,\n output_names=output_names,\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n\n session_options = rt.SessionOptions()\n if ort_custom_op_path:\n session_options.register_custom_ops_library(ort_custom_op_path)\n\n # get onnx output\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [node.name for node in onnx_model.graph.initializer]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 2)\n sess = rt.InferenceSession(onnx_file, session_options)\n ort_result = sess.run(None, {\n 'input': input.detach().numpy(),\n 'grid': grid.detach().numpy()\n })\n pytorch_results = wrapped_model(input.clone(), grid.clone())\n assert np.allclose(pytorch_results, ort_result, atol=1e-3)\n\n\n@pytest.mark.parametrize('mode', ['bilinear', 'nearest'])\n@pytest.mark.parametrize('padding_mode', ['zeros', 'border', 'reflection'])\n@pytest.mark.parametrize('align_corners', [True, False])\ndef test_grid_sample(mode, padding_mode, align_corners):\n from mmcv.onnx.symbolic import register_extra_symbolics\n opset_version = 11\n register_extra_symbolics(opset_version)\n\n from mmcv.ops import get_onnxruntime_op_path\n ort_custom_op_path = get_onnxruntime_op_path()\n if not os.path.exists(ort_custom_op_path):\n pytest.skip('custom ops for onnxruntime are not compiled.')\n\n input = torch.rand(1, 1, 10, 10)\n grid = torch.Tensor([[[1, 0, 0], [0, 1, 0]]])\n grid = F.affine_grid(\n grid, (1, 1, 15, 15), align_corners=align_corners).type_as(input)\n\n def func(input, grid):\n return F.grid_sample(\n input,\n grid,\n mode=mode,\n padding_mode=padding_mode,\n align_corners=align_corners)\n\n return process_grid_sample(func, input, grid, ort_custom_op_path)\n\n\n@pytest.mark.parametrize('align_corners', [True, False])\ndef test_bilinear_grid_sample(align_corners):\n from mmcv.ops.point_sample import bilinear_grid_sample\n\n # only support pytorch >= 1.5.0\n if version.parse(torch.__version__) < version.parse('1.5.0'):\n pytest.skip('Only support PyTorch >= 1.5.0')\n\n input = torch.rand(1, 1, 10, 10)\n grid = torch.Tensor([[[1, 0, 0], [0, 1, 0]]])\n grid = F.affine_grid(\n grid, (1, 1, 15, 15), align_corners=align_corners).type_as(input)\n\n def func(input, grid):\n return bilinear_grid_sample(input, grid, align_corners=align_corners)\n\n return process_grid_sample(func, input, grid)\n\n\ndef test_nms():\n if torch.__version__ == 'parrots':\n pytest.skip('onnx is not supported in parrots directly')\n from mmcv.ops import get_onnxruntime_op_path, nms\n np_boxes = np.array([[6.0, 3.0, 8.0, 7.0], [3.0, 6.0, 9.0, 11.0],\n [3.0, 7.0, 10.0, 12.0], [1.0, 4.0, 13.0, 7.0]],\n dtype=np.float32)\n np_scores = np.array([0.6, 0.9, 0.7, 0.2], dtype=np.float32)\n boxes = torch.from_numpy(np_boxes)\n scores = torch.from_numpy(np_scores)\n\n nms = partial(\n nms, iou_threshold=0.3, offset=0, score_threshold=0, max_num=0)\n pytorch_dets, _ = nms(boxes, scores)\n pytorch_score = pytorch_dets[:, 4]\n\n wrapped_model = WrapFunction(nms)\n wrapped_model.cpu().eval()\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (boxes, scores),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['boxes', 'scores'],\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n ort_custom_op_path = get_onnxruntime_op_path()\n session_options = rt.SessionOptions()\n if os.path.exists(ort_custom_op_path):\n session_options.register_custom_ops_library(ort_custom_op_path)\n\n # get onnx output\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [node.name for node in onnx_model.graph.initializer]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 2)\n sess = rt.InferenceSession(onnx_file, session_options)\n onnx_dets, _ = sess.run(None, {\n 'scores': scores.detach().numpy(),\n 'boxes': boxes.detach().numpy()\n })\n onnx_score = onnx_dets[:, 4]\n assert np.allclose(pytorch_score, onnx_score, atol=1e-3)\n\n\n@pytest.mark.skipif(not torch.cuda.is_available(), reason='test requires GPU')\ndef test_softnms():\n if torch.__version__ == 'parrots':\n pytest.skip('onnx is not supported in parrots directly')\n from mmcv.ops import get_onnxruntime_op_path, soft_nms\n\n # only support pytorch >= 1.7.0\n if version.parse(torch.__version__) < version.parse('1.7.0'):\n warnings.warn('test_softnms should be ran with pytorch >= 1.7.0')\n return\n\n # only support onnxruntime >= 1.5.1\n assert version.parse(rt.__version__) >= version.parse(\n '1.5.1'), 'test_softnms should be ran with onnxruntime >= 1.5.1'\n\n ort_custom_op_path = get_onnxruntime_op_path()\n if not os.path.exists(ort_custom_op_path):\n pytest.skip('softnms for onnxruntime is not compiled.')\n\n np_boxes = np.array([[6.0, 3.0, 8.0, 7.0], [3.0, 6.0, 9.0, 11.0],\n [3.0, 7.0, 10.0, 12.0], [1.0, 4.0, 13.0, 7.0]],\n dtype=np.float32)\n np_scores = np.array([0.6, 0.9, 0.7, 0.2], dtype=np.float32)\n\n boxes = torch.from_numpy(np_boxes)\n scores = torch.from_numpy(np_scores)\n\n configs = [[0.3, 0.5, 0.01, 'linear'], [0.3, 0.5, 0.01, 'gaussian'],\n [0.3, 0.5, 0.01, 'naive']]\n\n session_options = rt.SessionOptions()\n session_options.register_custom_ops_library(ort_custom_op_path)\n\n for _iou_threshold, _sigma, _min_score, _method in configs:\n pytorch_dets, pytorch_inds = soft_nms(\n boxes,\n scores,\n iou_threshold=_iou_threshold,\n sigma=_sigma,\n min_score=_min_score,\n method=_method)\n nms = partial(\n soft_nms,\n iou_threshold=_iou_threshold,\n sigma=_sigma,\n min_score=_min_score,\n method=_method)\n\n wrapped_model = WrapFunction(nms)\n wrapped_model.cpu().eval()\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (boxes, scores),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['boxes', 'scores'],\n opset_version=11)\n onnx_model = onnx.load(onnx_file)\n\n # get onnx output\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [\n node.name for node in onnx_model.graph.initializer\n ]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 2)\n sess = rt.InferenceSession(onnx_file, session_options)\n onnx_dets, onnx_inds = sess.run(None, {\n 'scores': scores.detach().numpy(),\n 'boxes': boxes.detach().numpy()\n })\n\n assert np.allclose(pytorch_dets, onnx_dets, atol=1e-3)\n assert np.allclose(onnx_inds, onnx_inds, atol=1e-3)\n\n\ndef test_roialign():\n if torch.__version__ == 'parrots':\n pytest.skip('onnx is not supported in parrots directly')\n try:\n from mmcv.ops import get_onnxruntime_op_path, roi_align\n except (ImportError, ModuleNotFoundError):\n pytest.skip('roi_align op is not successfully compiled')\n\n ort_custom_op_path = get_onnxruntime_op_path()\n # roi align config\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n sampling_ratio = 2\n\n inputs = [([[[[1., 2.], [3., 4.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2.], [3., 4.]], [[4., 3.],\n [2., 1.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.], [9., 10., 13., 14.],\n [11., 12., 15., 16.]]]], [[0., 0., 0., 3., 3.]])]\n\n def warpped_function(torch_input, torch_rois):\n return roi_align(torch_input, torch_rois, (pool_w, pool_h),\n spatial_scale, sampling_ratio, 'avg', True)\n\n for case in inputs:\n np_input = np.array(case[0], dtype=np.float32)\n np_rois = np.array(case[1], dtype=np.float32)\n input = torch.from_numpy(np_input)\n rois = torch.from_numpy(np_rois)\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_output = roi_align(input, rois, (pool_w, pool_h),\n spatial_scale, sampling_ratio, 'avg',\n True)\n\n # export and load onnx model\n wrapped_model = WrapFunction(warpped_function)\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (input, rois),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input', 'rois'],\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n session_options = rt.SessionOptions()\n if os.path.exists(ort_custom_op_path):\n session_options.register_custom_ops_library(ort_custom_op_path)\n\n # compute onnx_output\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [\n node.name for node in onnx_model.graph.initializer\n ]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 2)\n sess = rt.InferenceSession(onnx_file, session_options)\n onnx_output = sess.run(None, {\n 'input': input.detach().numpy(),\n 'rois': rois.detach().numpy()\n })\n onnx_output = onnx_output[0]\n\n # allclose\n\n assert np.allclose(pytorch_output, onnx_output, atol=1e-3)\n\n\ndef test_roialign_rotated():\n if torch.__version__ == 'parrots':\n pytest.skip('onnx is not supported in parrots directly')\n try:\n from mmcv.ops import get_onnxruntime_op_path, roi_align_rotated\n except (ImportError, ModuleNotFoundError):\n pytest.skip('roi_align_aligned op is not successfully compiled')\n\n ort_custom_op_path = get_onnxruntime_op_path()\n if not os.path.exists(ort_custom_op_path):\n pytest.skip('custom ops for onnxruntime are not compiled.')\n # roi align config\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n sampling_ratio = 2\n\n inputs = [([[[[1., 2.], [3., 4.]]]], [[0., 0.5, 0.5, 1., 1., 0]]),\n ([[[[1., 2.], [3., 4.]]]], [[0., 0.5, 0.5, 1., 1., np.pi / 2]]),\n ([[[[1., 2.], [3., 4.]],\n [[4., 3.], [2., 1.]]]], [[0., 0.5, 0.5, 1., 1., 0]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.], [9., 10., 13., 14.],\n [11., 12., 15., 16.]]]], [[0., 1.5, 1.5, 3., 3., 0]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.], [9., 10., 13., 14.],\n [11., 12., 15., 16.]]]], [[0., 1.5, 1.5, 3., 3.,\n np.pi / 2]])]\n\n def warpped_function(torch_input, torch_rois):\n return roi_align_rotated(torch_input, torch_rois, (pool_w, pool_h),\n spatial_scale, sampling_ratio, True, False)\n\n for case in inputs:\n np_input = np.array(case[0], dtype=np.float32)\n np_rois = np.array(case[1], dtype=np.float32)\n input = torch.from_numpy(np_input)\n rois = torch.from_numpy(np_rois)\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_output = roi_align_rotated(input, rois, (pool_w, pool_h),\n spatial_scale, sampling_ratio,\n True, False)\n\n # export and load onnx model\n wrapped_model = WrapFunction(warpped_function)\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (input, rois),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['features', 'rois'],\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n session_options = rt.SessionOptions()\n if os.path.exists(ort_custom_op_path):\n session_options.register_custom_ops_library(ort_custom_op_path)\n\n # compute onnx_output\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [\n node.name for node in onnx_model.graph.initializer\n ]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 2)\n sess = rt.InferenceSession(onnx_file, session_options)\n onnx_output = sess.run(None, {\n 'features': input.detach().numpy(),\n 'rois': rois.detach().numpy()\n })\n onnx_output = onnx_output[0]\n\n # allclose\n\n assert np.allclose(pytorch_output, onnx_output, atol=1e-3)\n\n\n@pytest.mark.skipif(not torch.cuda.is_available(), reason='test requires GPU')\ndef test_roipool():\n if torch.__version__ == 'parrots':\n pytest.skip('onnx is not supported in parrots directly')\n from mmcv.ops import roi_pool\n\n # roi pool config\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n\n inputs = [([[[[1., 2.], [3., 4.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2.], [3., 4.]], [[4., 3.],\n [2., 1.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.], [9., 10., 13., 14.],\n [11., 12., 15., 16.]]]], [[0., 0., 0., 3., 3.]])]\n\n def warpped_function(torch_input, torch_rois):\n return roi_pool(torch_input, torch_rois, (pool_w, pool_h),\n spatial_scale)\n\n for case in inputs:\n np_input = np.array(case[0], dtype=np.float32)\n np_rois = np.array(case[1], dtype=np.float32)\n input = torch.from_numpy(np_input).cuda()\n rois = torch.from_numpy(np_rois).cuda()\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_output = roi_pool(input, rois, (pool_w, pool_h),\n spatial_scale)\n pytorch_output = pytorch_output.cpu()\n\n # export and load onnx model\n wrapped_model = WrapFunction(warpped_function)\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (input, rois),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input', 'rois'],\n opset_version=11)\n onnx_model = onnx.load(onnx_file)\n\n # compute onnx_output\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [\n node.name for node in onnx_model.graph.initializer\n ]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 2)\n sess = rt.InferenceSession(onnx_file)\n onnx_output = sess.run(\n None, {\n 'input': input.detach().cpu().numpy(),\n 'rois': rois.detach().cpu().numpy()\n })\n onnx_output = onnx_output[0]\n\n # allclose\n assert np.allclose(pytorch_output, onnx_output, atol=1e-3)\n\n\ndef test_interpolate():\n from mmcv.onnx.symbolic import register_extra_symbolics\n opset_version = 11\n register_extra_symbolics(opset_version)\n\n def func(feat, scale_factor=2):\n out = F.interpolate(feat, scale_factor=scale_factor)\n return out\n\n net = WrapFunction(func)\n net = net.cpu().eval()\n dummy_input = torch.randn(2, 4, 8, 8).cpu()\n torch.onnx.export(\n net,\n dummy_input,\n onnx_file,\n input_names=['input'],\n opset_version=opset_version)\n sess = rt.InferenceSession(onnx_file)\n onnx_result = sess.run(None, {'input': dummy_input.detach().numpy()})\n pytorch_result = func(dummy_input).detach().numpy()\n\n assert np.allclose(pytorch_result, onnx_result, atol=1e-3)\n\n\n@pytest.mark.parametrize('mode', ['top', 'bottom', 'left', 'right'])\ndef test_corner_pool(mode, opset=11):\n if torch.__version__ == 'parrots':\n pytest.skip('onnx is not supported in parrots directly')\n\n from mmcv.ops import get_onnxruntime_op_path\n ort_custom_op_path = get_onnxruntime_op_path()\n if not os.path.exists(ort_custom_op_path):\n pytest.skip('custom ops for onnxruntime are not compiled.')\n\n from mmcv.ops.corner_pool import CornerPool\n\n def corner_pool_func(input):\n corner_pool_module = CornerPool(mode)\n return corner_pool_module.corner_pool.apply(input)\n\n wrapped_model = WrapFunction(corner_pool_func).eval()\n\n input = torch.rand((2, 3, 9, 12)) # (n,c,h,w)\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model,\n input,\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input'],\n output_names=['output'],\n opset_version=opset)\n\n onnx_model = onnx.load(onnx_file)\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [node.name for node in onnx_model.graph.initializer]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 1)\n\n session_options = rt.SessionOptions()\n session_options.register_custom_ops_library(ort_custom_op_path)\n sess = rt.InferenceSession(onnx_file, session_options)\n ort_result = sess.run(None, {'input': input.detach().numpy()})\n pytorch_results = wrapped_model(input.clone())\n\n assert np.allclose(pytorch_results, ort_result, atol=1e-5)\n\n\n@pytest.mark.parametrize('key', ['cummax', 'cummin'])\ndef test_cummax_cummin(key, opset=11):\n if torch.__version__ == 'parrots':\n pytest.skip('onnx is not supported in parrots directly')\n\n # Note generally `cummax` or `cummin` is exportable to ONNX\n # as long as the pytorch version >= 1.5.0, since `torch.cummax`\n # is only supported with torch >= 1.5.0.\n # But when `cummax` or `cummin` serves as an intermediate component\n # whose outputs is used as inputs for another modules, it's expected\n # that pytorch version must be >= 1.7.0. Otherwise error appears like:\n # `RuntimeError: tuple appears in op that does not forward tuples,\n # unsupported 'kind: prim::PythonOp`.\n if version.parse(torch.__version__) < version.parse('1.7.0'):\n pytest.skip('test_cummax_cummin should be ran with pytorch >= 1.7.0')\n\n # register custom op `mmcv::cummax` and `mmcv::cummin`\n from mmcv.onnx.symbolic import register_extra_symbolics\n register_extra_symbolics(opset)\n\n from mmcv.ops import get_onnxruntime_op_path\n ort_custom_op_path = get_onnxruntime_op_path()\n if not os.path.exists(ort_custom_op_path):\n pytest.skip('custom ops for onnxruntime are not compiled.')\n\n input_list = [\n # arbitrary shape, e.g. 1-D, 2-D, 3-D, ...\n torch.rand((2, 3, 4, 1, 5)),\n torch.rand((1)),\n torch.rand((2, 0, 1)), # tensor.numel() is 0\n torch.FloatTensor(), # empty tensor\n ]\n\n cummax_cummin_funcs = {'cummax': torch.cummax, 'cummin': torch.cummin}\n\n for input in input_list:\n ndims = input.dim()\n # valid dim range is [-ndims, ndims-1]\n # test for all `dim` value which is valid\n for dim in range(-ndims, ndims):\n cummax_func = partial(cummax_cummin_funcs[key], dim=dim)\n wrapped_model = WrapFunction(cummax_func).eval()\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model,\n input,\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input'],\n output_names=['output', 'indices'],\n opset_version=opset)\n\n onnx_model = onnx.load(onnx_file)\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [\n node.name for node in onnx_model.graph.initializer\n ]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 1)\n\n session_options = rt.SessionOptions()\n session_options.register_custom_ops_library(ort_custom_op_path)\n sess = rt.InferenceSession(onnx_file, session_options)\n ort_output, ort_inds = sess.run(None,\n {'input': input.detach().numpy()})\n pytorch_output, pytorch_inds = wrapped_model(input.clone())\n pytorch_output = pytorch_output.detach().numpy()\n pytorch_inds = pytorch_inds.detach().numpy()\n assert np.allclose(pytorch_output, ort_output, atol=1e-5)\n assert np.all(pytorch_inds == ort_inds)\n\n\n@pytest.mark.parametrize('shifts_dims_pair', [([-3, 5], [2, 0]), (5, None)])\ndef test_roll(shifts_dims_pair):\n opset = 11\n from mmcv.onnx.symbolic import register_extra_symbolics\n register_extra_symbolics(opset)\n\n input = torch.arange(0, 4 * 5 * 6, dtype=torch.float32).view(4, 5, 6)\n\n shifts, dims = shifts_dims_pair\n func = partial(torch.roll, shifts=shifts, dims=dims)\n wrapped_model = WrapFunction(func).eval()\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model,\n input,\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input'],\n output_names=['output'],\n opset_version=opset)\n\n onnx_model = onnx.load(onnx_file)\n input_all = [node.name for node in onnx_model.graph.input]\n input_initializer = [node.name for node in onnx_model.graph.initializer]\n net_feed_input = list(set(input_all) - set(input_initializer))\n assert (len(net_feed_input) == 1)\n\n sess = rt.InferenceSession(onnx_file)\n ort_output = sess.run(None, {'input': input.detach().numpy()})[0]\n\n with torch.no_grad():\n pytorch_output = wrapped_model(input.clone())\n\n torch.testing.assert_allclose(ort_output, pytorch_output)\n\n\n@pytest.mark.skipif(\n torch.__version__ == 'parrots',\n reason='onnx is not supported in parrots directly')\n@pytest.mark.skipif(\n not torch.cuda.is_available(),\n reason='modulated_deform_conv2d only supports in GPU')\ndef test_modulated_deform_conv2d():\n try:\n from mmcv.ops import ModulatedDeformConv2d, get_onnxruntime_op_path\n except (ImportError, ModuleNotFoundError):\n pytest.skip('modulated_deform_conv op is not successfully compiled')\n\n ort_custom_op_path = get_onnxruntime_op_path()\n # modulated deform conv config\n in_channels = 3\n out_channels = 64\n stride = 1\n padding = 0\n dilation = 1\n groups = 1\n deform_groups = 1\n kernel_size = 3\n\n input = torch.rand(1, in_channels, 28, 28).cuda() # (n, c, h, w)\n conv_offset = nn.Conv2d(\n in_channels=3,\n out_channels=deform_groups * 3 * kernel_size * kernel_size,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=True).cuda()\n conv_offset.cuda()\n out = conv_offset(input)\n o1, o2, mask = torch.chunk(out, 3, dim=1)\n offset = torch.cat((o1, o2), dim=1)\n mask = torch.sigmoid(mask)\n\n model_with_bias = ModulatedDeformConv2d(\n in_channels,\n out_channels,\n kernel_size,\n stride,\n padding,\n dilation,\n groups,\n deform_groups,\n bias=True)\n model_without_bias = ModulatedDeformConv2d(\n in_channels,\n out_channels,\n kernel_size,\n stride,\n padding,\n dilation,\n groups,\n deform_groups,\n bias=False)\n models = [model_with_bias.cuda(), model_without_bias.cuda()]\n\n for model in models:\n # export and load onnx model\n with torch.no_grad():\n torch.onnx.export(\n model, (input, offset, mask),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input', 'offset', 'mask'],\n opset_version=11)\n\n session_options = rt.SessionOptions()\n if os.path.exists(ort_custom_op_path):\n session_options.register_custom_ops_library(ort_custom_op_path)\n\n # compute onnx_output\n sess = rt.InferenceSession(onnx_file, session_options)\n onnx_output = sess.run(\n None, {\n 'input': input.cpu().detach().numpy(),\n 'offset': offset.cpu().detach().numpy(),\n 'mask': mask.cpu().detach().numpy()\n })[0]\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_output = model(input, offset, mask).cpu()\n # allclose\n assert np.allclose(pytorch_output, onnx_output, atol=1e-3)\n\n\n@pytest.mark.skipif(\n torch.__version__ == 'parrots',\n reason='onnx is not supported in parrots directly')\ndef test_deform_conv2d(threshold=1e-3):\n try:\n from mmcv.ops import DeformConv2d, get_onnxruntime_op_path\n except (ImportError, ModuleNotFoundError):\n pytest.skip('deform_conv op is not successfully compiled')\n\n ort_custom_op_path = get_onnxruntime_op_path()\n if not os.path.exists(ort_custom_op_path):\n pytest.skip('custom ops for onnxruntime are not compiled.')\n\n # deform conv config\n # modulated deform conv config\n in_channels = 1\n out_channels = 64\n stride = 1\n padding = 0\n dilation = 1\n groups = 1\n deform_groups = 1\n kernel_size = 2\n input = [[[[1., 2., 3.], [0., 1., 2.], [3., 5., 2.]]]]\n offset_weight = [[[0.1, 0.4, 0.6, 0.1]], [[0.3, 0.2, 0.1, 0.3]],\n [[0.5, 0.5, 0.2, 0.8]], [[0.8, 0.3, 0.9, 0.1]],\n [[0.3, 0.1, 0.2, 0.5]], [[0.3, 0.7, 0.5, 0.3]],\n [[0.6, 0.2, 0.5, 0.3]], [[0.4, 0.1, 0.8, 0.4]]]\n offset_bias = [0.7, 0.1, 0.8, 0.5, 0.6, 0.5, 0.4, 0.7]\n deform_weight = [[[0.4, 0.2, 0.1, 0.9]]]\n\n x = torch.tensor(input)\n conv_offset = nn.Conv2d(\n in_channels=in_channels,\n out_channels=deform_groups * 2 * kernel_size * kernel_size,\n kernel_size=kernel_size,\n stride=stride,\n padding=padding,\n dilation=dilation,\n bias=True)\n\n conv_offset.weight.data = torch.nn.Parameter(\n torch.Tensor(offset_weight).reshape(8, 1, 2, 2))\n conv_offset.bias.data = torch.nn.Parameter(\n torch.Tensor(offset_bias).reshape(8))\n\n offset = conv_offset(x)\n\n model = DeformConv2d(in_channels, out_channels, kernel_size, stride,\n padding, dilation, groups, deform_groups)\n\n model.weight.data = torch.nn.Parameter(\n torch.Tensor(deform_weight).reshape(1, 1, 2, 2))\n\n with torch.no_grad():\n torch.onnx.export(\n model, (x, offset),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input', 'offset'],\n opset_version=11)\n\n session_options = rt.SessionOptions()\n if os.path.exists(ort_custom_op_path):\n session_options.register_custom_ops_library(ort_custom_op_path)\n\n # compute onnx_output\n sess = rt.InferenceSession(onnx_file, session_options)\n onnx_output = sess.run(\n None, {\n 'input': x.cpu().detach().numpy(),\n 'offset': offset.cpu().detach().numpy(),\n })[0]\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_output = model(x, offset).cpu()\n # allclose\n assert np.allclose(pytorch_output, onnx_output, atol=1e-3)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_pixel_group.py", "content": "import numpy as np\nimport torch\n\n\ndef test_pixel_group():\n from mmcv.ops import pixel_group\n np_score = np.array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0],\n [0, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0],\n [0, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0],\n [0, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]).astype(np.float32)\n np_mask = (np_score > 0.5)\n np_embedding = np.zeros((10, 10, 8)).astype(np.float32)\n np_embedding[:, :7] = 0.9\n np_embedding[:, 7:] = 10.0\n np_kernel_label = np.array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 1, 1, 1, 0, 0, 0, 2, 0],\n [0, 0, 1, 1, 1, 0, 0, 0, 2, 0],\n [0, 0, 1, 1, 1, 0, 0, 0, 2, 0],\n [0, 0, 1, 1, 1, 0, 0, 0, 2, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0]]).astype(np.int32)\n np_kernel_contour = np.array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 1, 1, 1, 0, 0, 0, 1, 0],\n [0, 0, 1, 0, 1, 0, 0, 0, 1, 0],\n [0, 0, 1, 0, 1, 0, 0, 0, 1, 0],\n [0, 0, 1, 1, 1, 0, 0, 0, 1, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n [0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0]]).astype(np.uint8)\n kernel_region_num = 3\n distance_threshold = float(0.8)\n result = pixel_group(np_score, np_mask, np_embedding, np_kernel_label,\n np_kernel_contour, kernel_region_num,\n distance_threshold)\n gt_1 = [\n 0.8999997973442078, 24.0, 1.0, 3.0, 2.0, 3.0, 3.0, 3.0, 4.0, 3.0, 5.0,\n 3.0, 6.0, 3.0, 1.0, 4.0, 2.0, 4.0, 3.0, 4.0, 4.0, 4.0, 5.0, 4.0, 6.0,\n 4.0, 1.0, 5.0, 2.0, 5.0, 3.0, 5.0, 4.0, 5.0, 5.0, 5.0, 6.0, 5.0, 1.0,\n 6.0, 2.0, 6.0, 3.0, 6.0, 4.0, 6.0, 5.0, 6.0, 6.0, 6.0\n ]\n\n gt_2 = [\n 0.9000000357627869, 8.0, 7.0, 3.0, 8.0, 3.0, 7.0, 4.0, 8.0, 4.0, 7.0,\n 5.0, 8.0, 5.0, 7.0, 6.0, 8.0, 6.0\n ]\n\n assert np.allclose(result[0], [0, 0])\n assert np.allclose(result[1], gt_1)\n assert np.allclose(result[2], gt_2)\n\n # test torch Tensor\n np_score_t = torch.from_numpy(np_score)\n np_mask_t = torch.from_numpy(np_mask)\n np_embedding_t = torch.from_numpy(np_embedding)\n np_kernel_label_t = torch.from_numpy(np_kernel_label)\n np_kernel_contour_t = torch.from_numpy(np_kernel_contour)\n\n result = pixel_group(np_score_t, np_mask_t, np_embedding_t,\n np_kernel_label_t, np_kernel_contour_t,\n kernel_region_num, distance_threshold)\n\n assert np.allclose(result[0], [0, 0])\n assert np.allclose(result[1], gt_1)\n assert np.allclose(result[2], gt_2)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_points_in_polygons.py", "content": "import numpy as np\nimport pytest\nimport torch\n\nfrom mmcv.ops import points_in_polygons\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_points_in_polygons():\n points = np.array([[300., 300.], [400., 400.], [100., 100], [300, 250],\n [100, 0]])\n polygons = np.array([[200., 200., 400., 400., 500., 200., 400., 100.],\n [400., 400., 500., 500., 600., 300., 500., 200.],\n [300., 300., 600., 700., 700., 700., 700., 100.]])\n expected_output = np.array([[0., 0., 0.], [0., 0., 1.], [0., 0., 0.],\n [1., 0., 0.], [0., 0., 0.]])\n points = torch.from_numpy(points).cuda().float()\n polygons = torch.from_numpy(polygons).cuda().float()\n expected_output = torch.from_numpy(expected_output).cuda().float()\n assert torch.allclose(\n points_in_polygons(points, polygons), expected_output, 1e-3)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_psa_mask.py", "content": "import numpy as np\nimport torch\nimport torch.nn as nn\n\n\nclass Loss(nn.Module):\n\n def __init__(self):\n super().__init__()\n\n def forward(self, input, target):\n input = input.view(-1)\n target = target.view(-1)\n return torch.mean(input - target)\n\n\nclass TestPSAMask(object):\n\n def test_psa_mask_collect(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import PSAMask\n test_loss = Loss()\n\n input = np.fromfile(\n 'tests/data/for_psa_mask/psa_input.bin', dtype=np.float32)\n output_collect = np.fromfile(\n 'tests/data/for_psa_mask/psa_output_collect.bin', dtype=np.float32)\n\n input = input.reshape((4, 16, 8, 8))\n output_collect = output_collect.reshape((4, 64, 8, 8))\n label = torch.ones((4, 64, 8, 8))\n\n input = torch.FloatTensor(input)\n input.requires_grad = True\n\n psamask_collect = PSAMask('collect', (4, 4))\n\n # test collect cpu\n test_output = psamask_collect(input)\n loss = test_loss(test_output, label)\n loss.backward()\n test_output = test_output.detach().numpy()\n assert np.allclose(test_output, output_collect)\n assert test_output.shape == output_collect.shape\n\n psamask_collect.cuda()\n input = input.cuda()\n label = label.cuda()\n\n # test collect cuda\n test_output = psamask_collect(input)\n loss = test_loss(test_output, label)\n loss.backward()\n test_output = test_output.detach().cpu().numpy()\n assert np.allclose(test_output, output_collect)\n assert test_output.shape == output_collect.shape\n\n def test_psa_mask_distribute(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import PSAMask\n test_loss = Loss()\n\n input = np.fromfile(\n 'tests/data/for_psa_mask/psa_input.bin', dtype=np.float32)\n output_distribute = np.fromfile(\n 'tests/data/for_psa_mask/psa_output_distribute.bin',\n dtype=np.float32)\n\n input = input.reshape((4, 16, 8, 8))\n output_distribute = output_distribute.reshape((4, 64, 8, 8))\n label = torch.ones((4, 64, 8, 8))\n\n input = torch.FloatTensor(input)\n input.requires_grad = True\n\n psamask_distribute = PSAMask('distribute', (4, 4))\n\n # test distribute cpu\n test_output = psamask_distribute(input)\n loss = test_loss(test_output, label)\n loss.backward()\n test_output = test_output.detach().numpy()\n assert np.allclose(test_output, output_distribute)\n assert test_output.shape == output_distribute.shape\n\n psamask_distribute.cuda()\n input = input.cuda()\n label = label.cuda()\n\n # test distribute cuda\n test_output = psamask_distribute(input)\n loss = test_loss(test_output, label)\n loss.backward()\n test_output = test_output.detach().cpu().numpy()\n assert np.allclose(test_output, output_distribute)\n assert test_output.shape == output_distribute.shape\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_riroi_align_rotated.py", "content": "import numpy as np\nimport pytest\nimport torch\nfrom torch.autograd import gradcheck\n\nfrom mmcv.ops import RiRoIAlignRotated\n\nnp_feature = np.array([[[[1, 2], [3, 4]], [[1, 2], [4, 3]], [[4, 3], [2, 1]],\n [[1, 2], [5, 6]], [[3, 4], [7, 8]], [[9, 10], [13,\n 14]],\n [[11, 12], [15, 16]], [[1, 1], [2, 2]]]])\nnp_rois = np.array([[0., 0.5, 0.5, 1., 1., np.pi / 3],\n [0., 1., 1., 3., 3., np.pi / 2]])\nexpect_output = np.array([[[[1.8425, 1.3516], [2.3151, 1.8241]],\n [[2.4779, 1.7416], [3.2173, 2.5632]],\n [[2.7149, 2.2638], [2.6540, 2.3673]],\n [[2.9461, 2.8638], [2.8028, 2.7205]],\n [[4.1943, 2.7214], [5.6119, 4.1391]],\n [[7.5276, 6.0547], [8.9453, 7.4724]],\n [[12.1943, 10.7214], [13.6119, 12.1391]],\n [[9.5489, 8.4237], [10.5763, 9.4511]]],\n [[[7.6562, 12.5625], [4.0000, 6.6250]],\n [[1.0000, 1.3125], [0.5000, 0.6562]],\n [[1.6562, 1.9375], [1.0000, 1.3125]],\n [[1.8438, 2.0547], [0.7500, 1.1562]],\n [[0.8438, 3.0625], [0.2500, 1.1875]],\n [[2.6562, 2.5625], [1.5000, 1.6250]],\n [[3.6562, 4.5625], [2.0000, 2.6250]],\n [[6.6562, 10.5625], [3.5000, 5.6250]]]])\n\nexpect_grad = np.array([[[[1.4727, 1.5586], [1.5586, 1.6602]],\n [[1.4727, 1.5586], [1.5586, 1.6602]],\n [[1.4727, 1.5586], [1.5586, 1.6602]],\n [[1.4727, 1.5586], [1.5586, 1.6602]],\n [[1.4727, 1.5586], [1.5586, 1.6602]],\n [[1.4727, 1.5586], [1.5586, 1.6602]],\n [[1.4727, 1.5586], [1.5586, 1.6602]],\n [[1.4727, 1.5586], [1.5586, 1.6602]]]])\n\npool_h = 2\npool_w = 2\nspatial_scale = 1.0\nnum_samples = 2\nsampling_ratio = 2\nnum_orientations = 8\nclockwise = False\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_roialign_rotated_gradcheck():\n x = torch.tensor(\n np_feature, dtype=torch.float, device='cuda', requires_grad=True)\n rois = torch.tensor(np_rois, dtype=torch.float, device='cuda')\n froipool = RiRoIAlignRotated((pool_h, pool_w), spatial_scale, num_samples,\n num_orientations, clockwise)\n gradcheck(froipool, (x, rois), eps=1e-3, atol=1e-3)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_roialign_rotated_allclose():\n x = torch.tensor(\n np_feature, dtype=torch.float, device='cuda', requires_grad=True)\n rois = torch.tensor(np_rois, dtype=torch.float, device='cuda')\n froipool = RiRoIAlignRotated((pool_h, pool_w), spatial_scale, num_samples,\n num_orientations, clockwise)\n output = froipool(x, rois)\n output.backward(torch.ones_like(output))\n assert np.allclose(\n output.data.type(torch.float).cpu().numpy(), expect_output, atol=1e-3)\n assert np.allclose(\n x.grad.data.type(torch.float).cpu().numpy(), expect_grad, atol=1e-3)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_roi_align.py", "content": "import numpy as np\nimport pytest\nimport torch\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck\n _USING_PARROTS = False\n\n# yapf:disable\ninputs = [([[[[1., 2.], [3., 4.]]]],\n [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2.], [3., 4.]],\n [[4., 3.], [2., 1.]]]],\n [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.],\n [9., 10., 13., 14.], [11., 12., 15., 16.]]]],\n [[0., 0., 0., 3., 3.]])]\noutputs = [([[[[1.0, 1.25], [1.5, 1.75]]]],\n [[[[3.0625, 0.4375], [0.4375, 0.0625]]]]),\n ([[[[1.0, 1.25], [1.5, 1.75]],\n [[4.0, 3.75], [3.5, 3.25]]]],\n [[[[3.0625, 0.4375], [0.4375, 0.0625]],\n [[3.0625, 0.4375], [0.4375, 0.0625]]]]),\n ([[[[1.9375, 4.75], [7.5625, 10.375]]]],\n [[[[0.47265625, 0.42968750, 0.42968750, 0.04296875],\n [0.42968750, 0.39062500, 0.39062500, 0.03906250],\n [0.42968750, 0.39062500, 0.39062500, 0.03906250],\n [0.04296875, 0.03906250, 0.03906250, 0.00390625]]]])]\n# yapf:enable\n\npool_h = 2\npool_w = 2\nspatial_scale = 1.0\nsampling_ratio = 2\n\n\ndef _test_roialign_gradcheck(device, dtype):\n if not torch.cuda.is_available() and device == 'cuda':\n pytest.skip('test requires GPU')\n try:\n from mmcv.ops import RoIAlign\n except ModuleNotFoundError:\n pytest.skip('RoIAlign op is not successfully compiled')\n if dtype is torch.half:\n pytest.skip('grad check does not support fp16')\n for case in inputs:\n np_input = np.array(case[0])\n np_rois = np.array(case[1])\n\n x = torch.tensor(\n np_input, dtype=dtype, device=device, requires_grad=True)\n rois = torch.tensor(np_rois, dtype=dtype, device=device)\n\n froipool = RoIAlign((pool_h, pool_w), spatial_scale, sampling_ratio)\n\n if torch.__version__ == 'parrots':\n gradcheck(\n froipool, (x, rois), no_grads=[rois], delta=1e-5, pt_atol=1e-5)\n else:\n gradcheck(froipool, (x, rois), eps=1e-5, atol=1e-5)\n\n\ndef _test_roialign_allclose(device, dtype):\n if not torch.cuda.is_available() and device == 'cuda':\n pytest.skip('test requires GPU')\n try:\n from mmcv.ops import roi_align\n except ModuleNotFoundError:\n pytest.skip('test requires compilation')\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n sampling_ratio = 2\n\n for case, output in zip(inputs, outputs):\n np_input = np.array(case[0])\n np_rois = np.array(case[1])\n np_output = np.array(output[0])\n np_grad = np.array(output[1])\n\n x = torch.tensor(\n np_input, dtype=dtype, device=device, requires_grad=True)\n rois = torch.tensor(np_rois, dtype=dtype, device=device)\n\n output = roi_align(x, rois, (pool_h, pool_w), spatial_scale,\n sampling_ratio, 'avg', True)\n output.backward(torch.ones_like(output))\n assert np.allclose(\n output.data.type(torch.float).cpu().numpy(), np_output, atol=1e-3)\n assert np.allclose(\n x.grad.data.type(torch.float).cpu().numpy(), np_grad, atol=1e-3)\n\n\n@pytest.mark.parametrize('device', ['cuda', 'cpu'])\n@pytest.mark.parametrize('dtype', [torch.float, torch.double, torch.half])\ndef test_roialign(device, dtype):\n # check double only\n if dtype is torch.double:\n _test_roialign_gradcheck(device=device, dtype=dtype)\n _test_roialign_allclose(device=device, dtype=dtype)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_roi_align_rotated.py", "content": "import numpy as np\nimport pytest\nimport torch\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck\n _USING_PARROTS = False\n\n# yapf:disable\ninputs = [([[[[1., 2.], [3., 4.]]]],\n [[0., 0.5, 0.5, 1., 1., 0]]),\n ([[[[1., 2.], [3., 4.]]]],\n [[0., 0.5, 0.5, 1., 1., np.pi / 2]]),\n ([[[[1., 2.], [3., 4.]],\n [[4., 3.], [2., 1.]]]],\n [[0., 0.5, 0.5, 1., 1., 0]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.],\n [9., 10., 13., 14.], [11., 12., 15., 16.]]]],\n [[0., 1.5, 1.5, 3., 3., 0]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.],\n [9., 10., 13., 14.], [11., 12., 15., 16.]]]],\n [[0., 1.5, 1.5, 3., 3., np.pi / 2]])]\noutputs = [([[[[1.0, 1.25], [1.5, 1.75]]]],\n [[[[3.0625, 0.4375], [0.4375, 0.0625]]]]),\n ([[[[1.5, 1], [1.75, 1.25]]]],\n [[[[3.0625, 0.4375], [0.4375, 0.0625]]]]),\n ([[[[1.0, 1.25], [1.5, 1.75]],\n [[4.0, 3.75], [3.5, 3.25]]]],\n [[[[3.0625, 0.4375], [0.4375, 0.0625]],\n [[3.0625, 0.4375], [0.4375, 0.0625]]]]),\n ([[[[1.9375, 4.75], [7.5625, 10.375]]]],\n [[[[0.47265625, 0.42968750, 0.42968750, 0.04296875],\n [0.42968750, 0.39062500, 0.39062500, 0.03906250],\n [0.42968750, 0.39062500, 0.39062500, 0.03906250],\n [0.04296875, 0.03906250, 0.03906250, 0.00390625]]]]),\n ([[[[7.5625, 1.9375], [10.375, 4.75]]]],\n [[[[0.47265625, 0.42968750, 0.42968750, 0.04296875],\n [0.42968750, 0.39062500, 0.39062500, 0.03906250],\n [0.42968750, 0.39062500, 0.39062500, 0.03906250],\n [0.04296875, 0.03906250, 0.03906250, 0.00390625]]]])]\n# yapf:enable\n\npool_h = 2\npool_w = 2\nspatial_scale = 1.0\nsampling_ratio = 2\n\n\ndef _test_roialign_rotated_gradcheck(device, dtype):\n if not torch.cuda.is_available() and device == 'cuda':\n pytest.skip('unittest does not support GPU yet.')\n try:\n from mmcv.ops import RoIAlignRotated\n except ModuleNotFoundError:\n pytest.skip('RoIAlignRotated op is not successfully compiled')\n if dtype is torch.half:\n pytest.skip('grad check does not support fp16')\n for case in inputs:\n np_input = np.array(case[0])\n np_rois = np.array(case[1])\n\n x = torch.tensor(\n np_input, dtype=dtype, device=device, requires_grad=True)\n rois = torch.tensor(np_rois, dtype=dtype, device=device)\n\n froipool = RoIAlignRotated((pool_h, pool_w), spatial_scale,\n sampling_ratio)\n\n if torch.__version__ == 'parrots':\n gradcheck(\n froipool, (x, rois), no_grads=[rois], delta=1e-5, pt_atol=1e-5)\n else:\n gradcheck(froipool, (x, rois), eps=1e-5, atol=1e-5)\n\n\ndef _test_roialign_rotated_allclose(device, dtype):\n if not torch.cuda.is_available() and device == 'cuda':\n pytest.skip('unittest does not support GPU yet.')\n try:\n from mmcv.ops import roi_align_rotated\n except ModuleNotFoundError:\n pytest.skip('test requires compilation')\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n sampling_ratio = 2\n\n for case, output in zip(inputs, outputs):\n np_input = np.array(case[0])\n np_rois = np.array(case[1])\n np_output = np.array(output[0])\n np_grad = np.array(output[1])\n\n x = torch.tensor(\n np_input, dtype=dtype, device=device, requires_grad=True)\n rois = torch.tensor(np_rois, dtype=dtype, device=device)\n\n output = roi_align_rotated(x, rois, (pool_h, pool_w), spatial_scale,\n sampling_ratio, True)\n output.backward(torch.ones_like(output))\n assert np.allclose(\n output.data.type(torch.float).cpu().numpy(), np_output, atol=1e-3)\n assert np.allclose(\n x.grad.data.type(torch.float).cpu().numpy(), np_grad, atol=1e-3)\n\n\n@pytest.mark.parametrize('device', ['cuda', 'cpu'])\n@pytest.mark.parametrize('dtype', [torch.float, torch.double, torch.half])\ndef test_roialign_rotated(device, dtype):\n # check double only\n if (dtype is torch.double):\n _test_roialign_rotated_gradcheck(device=device, dtype=dtype)\n _test_roialign_rotated_allclose(device=device, dtype=dtype)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_roi_pool.py", "content": "import os\n\nimport numpy as np\nimport torch\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck\n\n _USING_PARROTS = False\n\ncur_dir = os.path.dirname(os.path.abspath(__file__))\n\ninputs = [([[[[1., 2.], [3., 4.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2.], [3., 4.]], [[4., 3.], [2.,\n 1.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.], [9., 10., 13., 14.],\n [11., 12., 15., 16.]]]], [[0., 0., 0., 3., 3.]])]\noutputs = [([[[[1., 2.], [3., 4.]]]], [[[[1., 1.], [1., 1.]]]]),\n ([[[[1., 2.], [3., 4.]], [[4., 3.], [2., 1.]]]], [[[[1., 1.],\n [1., 1.]],\n [[1., 1.],\n [1., 1.]]]]),\n ([[[[4., 8.], [12., 16.]]]], [[[[0., 0., 0., 0.], [0., 1., 0., 1.],\n [0., 0., 0., 0.], [0., 1., 0.,\n 1.]]]])]\n\n\nclass TestRoiPool(object):\n\n def test_roipool_gradcheck(self):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import RoIPool\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n\n for case in inputs:\n np_input = np.array(case[0])\n np_rois = np.array(case[1])\n\n x = torch.tensor(np_input, device='cuda', requires_grad=True)\n rois = torch.tensor(np_rois, device='cuda')\n\n froipool = RoIPool((pool_h, pool_w), spatial_scale)\n\n if _USING_PARROTS:\n pass\n # gradcheck(froipool, (x, rois), no_grads=[rois])\n else:\n gradcheck(froipool, (x, rois), eps=1e-2, atol=1e-2)\n\n def _test_roipool_allclose(self, dtype=torch.float):\n if not torch.cuda.is_available():\n return\n from mmcv.ops import roi_pool\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n\n for case, output in zip(inputs, outputs):\n np_input = np.array(case[0])\n np_rois = np.array(case[1])\n np_output = np.array(output[0])\n np_grad = np.array(output[1])\n\n x = torch.tensor(\n np_input, dtype=dtype, device='cuda', requires_grad=True)\n rois = torch.tensor(np_rois, dtype=dtype, device='cuda')\n\n output = roi_pool(x, rois, (pool_h, pool_w), spatial_scale)\n output.backward(torch.ones_like(output))\n assert np.allclose(output.data.cpu().numpy(), np_output, 1e-3)\n assert np.allclose(x.grad.data.cpu().numpy(), np_grad, 1e-3)\n\n def test_roipool_allclose(self):\n self._test_roipool_allclose(torch.double)\n self._test_roipool_allclose(torch.float)\n self._test_roipool_allclose(torch.half)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_roiaware_pool3d.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numpy as np\nimport pytest\nimport torch\n\nfrom mmcv.ops import (RoIAwarePool3d, points_in_boxes_all, points_in_boxes_cpu,\n points_in_boxes_part)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_RoIAwarePool3d():\n roiaware_pool3d_max = RoIAwarePool3d(\n out_size=4, max_pts_per_voxel=128, mode='max')\n roiaware_pool3d_avg = RoIAwarePool3d(\n out_size=4, max_pts_per_voxel=128, mode='avg')\n rois = torch.tensor(\n [[1.0, 2.0, 3.0, 5.0, 4.0, 6.0, -0.3 - np.pi / 2],\n [-10.0, 23.0, 16.0, 20.0, 10.0, 20.0, -0.5 - np.pi / 2]],\n dtype=torch.float32).cuda(\n ) # boxes (m, 7) with bottom center in lidar coordinate\n pts = torch.tensor(\n [[1, 2, 3.3], [1.2, 2.5, 3.0], [0.8, 2.1, 3.5], [1.6, 2.6, 3.6],\n [0.8, 1.2, 3.9], [-9.2, 21.0, 18.2], [3.8, 7.9, 6.3],\n [4.7, 3.5, -12.2], [3.8, 7.6, -2], [-10.6, -12.9, -20], [-16, -18, 9],\n [-21.3, -52, -5], [0, 0, 0], [6, 7, 8], [-2, -3, -4]],\n dtype=torch.float32).cuda() # points (n, 3) in lidar coordinate\n pts_feature = pts.clone()\n\n pooled_features_max = roiaware_pool3d_max(\n rois=rois, pts=pts, pts_feature=pts_feature)\n assert pooled_features_max.shape == torch.Size([2, 4, 4, 4, 3])\n assert torch.allclose(pooled_features_max.sum(),\n torch.tensor(51.100).cuda(), 1e-3)\n\n pooled_features_avg = roiaware_pool3d_avg(\n rois=rois, pts=pts, pts_feature=pts_feature)\n assert pooled_features_avg.shape == torch.Size([2, 4, 4, 4, 3])\n assert torch.allclose(pooled_features_avg.sum(),\n torch.tensor(49.750).cuda(), 1e-3)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_points_in_boxes_part():\n boxes = torch.tensor(\n [[[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 0.3]],\n [[-10.0, 23.0, 16.0, 10, 20, 20, 0.5]]],\n dtype=torch.float32).cuda(\n ) # boxes (b, t, 7) with bottom center in lidar coordinate\n pts = torch.tensor(\n [[[1, 2, 3.3], [1.2, 2.5, 3.0], [0.8, 2.1, 3.5], [1.6, 2.6, 3.6],\n [0.8, 1.2, 3.9], [-9.2, 21.0, 18.2], [3.8, 7.9, 6.3],\n [4.7, 3.5, -12.2]],\n [[3.8, 7.6, -2], [-10.6, -12.9, -20], [-16, -18, 9], [-21.3, -52, -5],\n [0, 0, 0], [6, 7, 8], [-2, -3, -4], [6, 4, 9]]],\n dtype=torch.float32).cuda() # points (b, m, 3) in lidar coordinate\n\n point_indices = points_in_boxes_part(points=pts, boxes=boxes)\n expected_point_indices = torch.tensor(\n [[0, 0, 0, 0, 0, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1]],\n dtype=torch.int32).cuda()\n assert point_indices.shape == torch.Size([2, 8])\n assert (point_indices == expected_point_indices).all()\n\n boxes = torch.tensor([[[0.0, 0.0, 0.0, 1.0, 20.0, 1.0, 0.523598]]],\n dtype=torch.float32).cuda() # 30 degrees\n pts = torch.tensor(\n [[[4, 6.928, 0], [6.928, 4, 0], [4, -6.928, 0], [6.928, -4, 0],\n [-4, 6.928, 0], [-6.928, 4, 0], [-4, -6.928, 0], [-6.928, -4, 0]]],\n dtype=torch.float32).cuda()\n point_indices = points_in_boxes_part(points=pts, boxes=boxes)\n expected_point_indices = torch.tensor([[-1, -1, 0, -1, 0, -1, -1, -1]],\n dtype=torch.int32).cuda()\n assert (point_indices == expected_point_indices).all()\n\n\ndef test_points_in_boxes_cpu():\n boxes = torch.tensor(\n [[[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 0.3],\n [-10.0, 23.0, 16.0, 10, 20, 20, 0.5]]],\n dtype=torch.float32\n ) # boxes (m, 7) with bottom center in lidar coordinate\n pts = torch.tensor(\n [[[1, 2, 3.3], [1.2, 2.5, 3.0], [0.8, 2.1, 3.5], [1.6, 2.6, 3.6],\n [0.8, 1.2, 3.9], [-9.2, 21.0, 18.2], [3.8, 7.9, 6.3],\n [4.7, 3.5, -12.2], [3.8, 7.6, -2], [-10.6, -12.9, -20], [\n -16, -18, 9\n ], [-21.3, -52, -5], [0, 0, 0], [6, 7, 8], [-2, -3, -4]]],\n dtype=torch.float32) # points (n, 3) in lidar coordinate\n\n point_indices = points_in_boxes_cpu(points=pts, boxes=boxes)\n expected_point_indices = torch.tensor(\n [[[1, 0], [1, 0], [1, 0], [1, 0], [1, 0], [0, 1], [0, 0], [0, 0],\n [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]]],\n dtype=torch.int32)\n assert point_indices.shape == torch.Size([1, 15, 2])\n assert (point_indices == expected_point_indices).all()\n\n boxes = torch.tensor([[[0.0, 0.0, 0.0, 1.0, 20.0, 1.0, 0.523598]]],\n dtype=torch.float32) # 30 degrees\n pts = torch.tensor(\n [[[4, 6.928, 0], [6.928, 4, 0], [4, -6.928, 0], [6.928, -4, 0],\n [-4, 6.928, 0], [-6.928, 4, 0], [-4, -6.928, 0], [-6.928, -4, 0]]],\n dtype=torch.float32)\n point_indices = points_in_boxes_cpu(points=pts, boxes=boxes)\n expected_point_indices = torch.tensor(\n [[[0], [0], [1], [0], [1], [0], [0], [0]]], dtype=torch.int32)\n assert (point_indices == expected_point_indices).all()\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_points_in_boxes_all():\n\n boxes = torch.tensor(\n [[[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 0.3],\n [-10.0, 23.0, 16.0, 10, 20, 20, 0.5]]],\n dtype=torch.float32).cuda(\n ) # boxes (m, 7) with bottom center in lidar coordinate\n pts = torch.tensor(\n [[[1, 2, 3.3], [1.2, 2.5, 3.0], [0.8, 2.1, 3.5], [1.6, 2.6, 3.6],\n [0.8, 1.2, 3.9], [-9.2, 21.0, 18.2], [3.8, 7.9, 6.3],\n [4.7, 3.5, -12.2], [3.8, 7.6, -2], [-10.6, -12.9, -20], [\n -16, -18, 9\n ], [-21.3, -52, -5], [0, 0, 0], [6, 7, 8], [-2, -3, -4]]],\n dtype=torch.float32).cuda() # points (n, 3) in lidar coordinate\n\n point_indices = points_in_boxes_all(points=pts, boxes=boxes)\n expected_point_indices = torch.tensor(\n [[[1, 0], [1, 0], [1, 0], [1, 0], [1, 0], [0, 1], [0, 0], [0, 0],\n [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]]],\n dtype=torch.int32).cuda()\n assert point_indices.shape == torch.Size([1, 15, 2])\n assert (point_indices == expected_point_indices).all()\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_roipoint_pool3d.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import RoIPointPool3d\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_gather_points():\n feats = torch.tensor(\n [[1, 2, 3.3], [1.2, 2.5, 3.0], [0.8, 2.1, 3.5], [1.6, 2.6, 3.6],\n [0.8, 1.2, 3.9], [-9.2, 21.0, 18.2], [3.8, 7.9, 6.3],\n [4.7, 3.5, -12.2], [3.8, 7.6, -2], [-10.6, -12.9, -20], [-16, -18, 9],\n [-21.3, -52, -5], [0, 0, 0], [6, 7, 8], [-2, -3, -4]],\n dtype=torch.float32).unsqueeze(0).cuda()\n points = feats.clone()\n rois = torch.tensor([[[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 0.3],\n [-10.0, 23.0, 16.0, 10, 20, 20, 0.5]]],\n dtype=torch.float32).cuda()\n\n roipoint_pool3d = RoIPointPool3d(num_sampled_points=4)\n roi_feat, empty_flag = roipoint_pool3d(feats, points, rois)\n expected_roi_feat = torch.tensor([[[[1, 2, 3.3, 1, 2, 3.3],\n [1.2, 2.5, 3, 1.2, 2.5, 3],\n [0.8, 2.1, 3.5, 0.8, 2.1, 3.5],\n [1.6, 2.6, 3.6, 1.6, 2.6, 3.6]],\n [[-9.2, 21, 18.2, -9.2, 21, 18.2],\n [-9.2, 21, 18.2, -9.2, 21, 18.2],\n [-9.2, 21, 18.2, -9.2, 21, 18.2],\n [-9.2, 21, 18.2, -9.2, 21,\n 18.2]]]]).cuda()\n expected_empty_flag = torch.tensor([[0, 0]]).int().cuda()\n\n assert torch.allclose(roi_feat, expected_roi_feat)\n assert torch.allclose(empty_flag, expected_empty_flag)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_rotated_feature_align.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import rotated_feature_align\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_rotated_feature_align():\n feature = torch.tensor([[[[1.2924, -0.2172, -0.5222, 0.1172],\n [0.9144, 1.2248, 1.3115, -0.9690],\n [-0.8949, -1.1797, -0.9093, -0.3961],\n [-0.4586, 0.5062, -0.7947, -0.7397]],\n [[-1.0943, -0.7495, 1.3461, -1.1652],\n [0.2034, 0.6763, -1.2357, 0.5231],\n [-1.0062, 1.2592, 1.4225, -0.3951],\n [-0.1242, -1.6240, 0.1932, 2.7181]],\n [[-1.6271, -1.0276, 0.0578, -0.2997],\n [-0.9684, -1.6946, -1.3188, -1.1938],\n [-1.6744, -0.8917, -0.6556,\n 1.0073], [-0.1205, 0.3671, -0.3731, -0.5347]]],\n [[[0.7035, 0.2089, -0.1774, 3.4670],\n [-0.8505, -0.9278, 1.4714, 0.1644],\n [0.0898, 0.3531, -0.4007, 0.1927],\n [1.2569, -0.2636, -0.5223, 0.0616]],\n [[0.1760, -0.7639, -0.4600, -1.3260],\n [-0.9921, -0.2970, -0.8955, 1.0508],\n [1.3515, -0.1641, 1.9679, 1.1986],\n [-0.3616, 0.6287, 0.4933, 0.3360]],\n [[-0.5860, 0.2124, -0.8700, 2.4200],\n [-0.0551, -1.5103, -1.6779, 0.8399],\n [0.8431, 1.2414, -1.1243, -0.3887],\n [-2.1254, 0.6047, -0.3515, 0.7254]]]],\n device='cuda',\n requires_grad=True)\n\n bbox = torch.tensor(\n [[[[1.3080e+01, 1.2688e+01, 1.1214e+01, 9.3944e+01, -9.1905e-01],\n [3.8104e+01, 1.0134e+01, 1.4659e+02, 9.0306e+01, -9.8211e-01],\n [-5.3213e+01, 4.9508e+01, 5.1513e+01, 3.2055e+01, -3.1954e-01],\n [2.6974e+01, 2.5248e+01, 5.4495e+01, 3.1083e+00, -6.2127e-01]],\n [[-1.5604e+01, -5.1908e+01, 2.3998e+02, 1.5008e+01, -1.2546e+00],\n [3.1354e+01, -7.3635e+00, 6.7879e+01, 3.5081e+01, -3.3851e-01],\n [-5.3292e+00, 9.1946e+00, 1.2834e+01, 1.0485e+01, -1.3039e+00],\n [-2.3925e+01, 3.6623e+01, 3.9875e+01, 7.2009e+01, -6.5934e-01]],\n [[7.2114e+01, -2.3781e+01, 2.9106e+01, 8.4501e+01, -1.1340e+00],\n [2.6258e+01, -7.7034e+00, 1.7629e+02, 1.0615e+02, -1.2156e+00],\n [3.8057e+01, 4.6016e+01, 1.2965e+01, 6.9384e+00, -1.0855e+00],\n [2.4428e+01, -1.6189e+01, 2.0572e+02, 3.1622e+01, -1.5719e-01]],\n [[3.8226e+00, 2.9608e+01, 1.4457e+01, 6.8179e+01, -9.1997e-01],\n [2.5003e+01, -4.2490e+01, 9.6007e+01, 4.9086e+01, -1.4786e+00],\n [8.5983e+01, 5.4980e+01, 7.8080e+01, 1.0003e+02, -1.0926e+00],\n [9.9065e+00, 4.1457e+01, 5.9799e+00, 1.7973e+01, -5.6313e-01]]],\n [[[-1.8244e+01, 4.6309e+00, 5.3010e+01, 2.4310e+01, -7.0345e-01],\n [1.9419e+01, 3.6704e+01, 5.2390e+01, 5.4133e+01, -3.7730e-01],\n [5.6387e+01, 2.3752e+01, 9.0441e+00, 1.7792e+01, -1.5583e+00],\n [3.6303e+01, 1.6396e+01, 2.0283e+01, 1.9148e+01, -8.3419e-01]],\n [[3.2169e+01, 3.0521e+01, 2.6283e+01, 1.9680e+02, -3.0454e-01],\n [2.5788e+01, -3.2189e+01, 8.8882e+01, 1.0207e+02, -1.5328e+00],\n [8.4676e+00, -1.6668e+01, 2.4657e+01, 1.1275e+02, -4.0388e-01],\n [-1.0799e+01, 6.0422e+00, 9.5807e+00, 3.3677e+01, -3.5438e-01]],\n [[6.9363e+01, 1.0850e+01, 2.5968e+01, 2.2311e+01, -1.6408e-01],\n [2.8140e+00, 4.6843e+00, 3.1289e+00, 2.1480e+01, -6.7583e-01],\n [2.6661e+01, 4.5290e+01, 6.1679e+00, 3.0005e+01, -8.9806e-01],\n [5.0871e+00, 1.3234e+01, 9.2087e+01, 4.9622e+01, -2.8020e-01]],\n [[-1.2643e+01, 2.5176e+01, 5.0488e+01, 5.4246e+01, -4.4840e-01],\n [-3.4521e+01, 9.8435e-01, 5.2413e+01, 9.7996e+00, -8.4218e-01],\n [4.9829e+01, -1.0808e+01, 2.9848e+01, 7.3579e+01, -6.2672e-01],\n [8.0446e+01, 2.8064e+01, 4.5273e+01, 5.3809e+01, -1.2359e+00]]]],\n device='cuda',\n requires_grad=True)\n\n expected_output = torch.tensor([[[[1.1095, -0.2172, -0.5222, -0.6225],\n [0.9144, 0.7662, 1.0487, -0.9690],\n [-0.8949, -1.6384, -0.9093, -0.3961],\n [-0.8604, 0.5062, -0.7947, -0.7397]],\n [[-0.3961, -0.7495, 1.3461, 1.5528],\n [0.2034, 0.5522, -1.6722, 0.5231],\n [-1.0062, 1.1350, 1.4225, -0.3951],\n [-0.4826, -1.6240, 0.1932, 2.7181]],\n [[-2.6436, -1.0276, 0.0578, -0.8344],\n [-0.9684, -1.8151, -2.1843, -1.1938],\n [-1.6744, -1.0121, -0.6556, 1.0073],\n [-0.8474, 0.3671, -0.3731, -0.5347]]],\n [[[0.7035, 0.2089, -0.1774, 3.4670],\n [-0.8505, -0.9278, 1.4714, 0.1644],\n [0.0898, 0.3064, -0.4007, 0.5849],\n [1.2569, -0.2636, -0.5223, 0.0616]],\n [[0.1760, -0.7639, -0.4600, -1.3260],\n [-0.9921, -0.2970, -0.8955, 1.0508],\n [1.3515, -0.6125, 1.9679, 0.5550],\n [-0.3616, 0.6287, 0.4933, 0.3360]],\n [[-0.5860, 0.2124, -0.8700, 2.4200],\n [-0.0551, -1.5103, -1.6779, 0.8399],\n [0.8431, 0.8455, -1.1243, -1.5994],\n [-2.1254, 0.6047, -0.3515,\n 0.7254]]]]).cuda()\n\n expected_grad = torch.tensor([[[[1.0000, 1.8507, 1.1493, 1.5222],\n [1.0000, 1.1511, 1.2139, 1.4778],\n [1.0000, 1.2629, 1.3721, 1.0000],\n [3.0000, 1.0000, 1.0000, 2.0000]],\n [[1.0000, 1.8507, 1.1493, 1.5222],\n [1.0000, 1.1511, 1.2139, 1.4778],\n [1.0000, 1.2629, 1.3721, 1.0000],\n [3.0000, 1.0000, 1.0000, 2.0000]],\n [[1.0000, 1.8507, 1.1493, 1.5222],\n [1.0000, 1.1511, 1.2139, 1.4778],\n [1.0000, 1.2629, 1.3721, 1.0000],\n [3.0000, 1.0000, 1.0000, 2.0000]]],\n [[[1.2687, 1.5055, 1.2382, 1.0000],\n [1.1458, 1.4258, 1.4160, 1.0000],\n [1.0000, 1.0000, 1.0000, 1.0000],\n [1.0000, 1.0000, 1.0000, 1.0000]],\n [[1.2687, 1.5055, 1.2382, 1.0000],\n [1.1458, 1.4258, 1.4160, 1.0000],\n [1.0000, 1.0000, 1.0000, 1.0000],\n [1.0000, 1.0000, 1.0000, 1.0000]],\n [[1.2687, 1.5055, 1.2382, 1.0000],\n [1.1458, 1.4258, 1.4160, 1.0000],\n [1.0000, 1.0000, 1.0000, 1.0000],\n [1.0000, 1.0000, 1.0000,\n 1.0000]]]]).cuda()\n\n output = rotated_feature_align(\n feature, bbox, spatial_scale=1 / 8, points=1)\n output.backward(torch.ones_like(output))\n assert torch.allclose(output, expected_output, 1e-2)\n assert torch.allclose(feature.grad, expected_grad, 1e-2)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_saconv.py", "content": "import torch\nimport torch.nn as nn\n\nfrom mmcv.ops import SAConv2d\n\n\ndef test_sacconv():\n\n # test with normal cast\n x = torch.rand(1, 3, 256, 256)\n saconv = SAConv2d(3, 5, kernel_size=3, padding=1)\n sac_out = saconv(x)\n refer_conv = nn.Conv2d(3, 5, kernel_size=3, padding=1)\n refer_out = refer_conv(x)\n assert sac_out.shape == refer_out.shape\n\n # test with dilation >= 2\n dalited_saconv = SAConv2d(3, 5, kernel_size=3, padding=2, dilation=2)\n dalited_sac_out = dalited_saconv(x)\n refer_conv = nn.Conv2d(3, 5, kernel_size=3, padding=2, dilation=2)\n refer_out = refer_conv(x)\n assert dalited_sac_out.shape == refer_out.shape\n\n # test with deform\n deform_saconv = SAConv2d(3, 5, kernel_size=3, padding=1, use_deform=True)\n if torch.cuda.is_available():\n x = torch.rand(1, 3, 256, 256).cuda()\n deform_saconv = SAConv2d(\n 3, 5, kernel_size=3, padding=1, use_deform=True).cuda()\n deform_sac_out = deform_saconv(x).cuda()\n refer_conv = nn.Conv2d(3, 5, kernel_size=3, padding=1).cuda()\n refer_out = refer_conv(x)\n assert deform_sac_out.shape == refer_out.shape\n else:\n deform_sac_out = deform_saconv(x)\n refer_conv = nn.Conv2d(3, 5, kernel_size=3, padding=1)\n refer_out = refer_conv(x)\n assert deform_sac_out.shape == refer_out.shape\n\n # test with groups >= 2\n x = torch.rand(1, 4, 256, 256)\n group_saconv = SAConv2d(4, 4, kernel_size=3, padding=1, groups=2)\n group_sac_out = group_saconv(x)\n refer_conv = nn.Conv2d(4, 4, kernel_size=3, padding=1, groups=2)\n refer_out = refer_conv(x)\n assert group_sac_out.shape == refer_out.shape\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_scatter_points.py", "content": "import pytest\nimport torch\nfrom torch.autograd import gradcheck\n\nfrom mmcv.ops import DynamicScatter\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_dynamic_scatter():\n feats = torch.rand(\n size=(200000, 3), dtype=torch.float32, device='cuda') * 100 - 50\n coors = torch.randint(\n low=-1, high=20, size=(200000, 3), dtype=torch.int32, device='cuda')\n\n dsmean = DynamicScatter([0.32, 0.32, 6],\n [-74.88, -74.88, -2, 74.88, 74.88, 4], True)\n dsmax = DynamicScatter([0.32, 0.32, 6],\n [-74.88, -74.88, -2, 74.88, 74.88, 4], False)\n\n # test empty input\n empty_feats = torch.empty(size=(0, 3), dtype=torch.float32, device='cuda')\n empty_coors = torch.empty(size=(0, 3), dtype=torch.int32, device='cuda')\n\n empty_feats.requires_grad_()\n empty_feats_out_mean, empty_coors_out_mean = dsmean(\n empty_feats, empty_coors)\n empty_feats_out_mean.sum().backward()\n empty_feats_out_max, empty_coors_out_max = dsmax(empty_feats, empty_coors)\n empty_feats_out_max.sum().backward()\n\n assert empty_feats_out_mean.shape == empty_feats.shape\n assert empty_feats_out_max.shape == empty_feats.shape\n assert empty_coors_out_mean.shape == empty_coors.shape\n assert empty_coors_out_max.shape == empty_coors.shape\n\n # test empty reduced output\n empty_o_feats = torch.rand(\n size=(200000, 3), dtype=torch.float32, device='cuda') * 100 - 50\n empty_o_coors = torch.randint(\n low=-1, high=0, size=(200000, 3), dtype=torch.int32, device='cuda')\n\n empty_o_feats.requires_grad_()\n empty_o_feats_out_mean, empty_o_coors_out_mean = dsmean(\n empty_o_feats, empty_o_coors)\n empty_o_feats_out_mean.sum().backward()\n assert (empty_o_feats.grad == 0).all()\n\n empty_o_feats_out_max, empty_o_coors_out_max = dsmax(\n empty_o_feats, empty_o_coors)\n empty_o_feats_out_max.sum().backward()\n assert (empty_o_feats.grad == 0).all()\n\n # test non-empty input\n ref_voxel_coors = coors.unique(dim=0, sorted=True)\n ref_voxel_coors = ref_voxel_coors[ref_voxel_coors.min(dim=-1).values >= 0]\n ref_voxel_feats_mean = []\n ref_voxel_feats_max = []\n for ref_voxel_coor in ref_voxel_coors:\n voxel_mask = (coors == ref_voxel_coor).all(dim=-1)\n ref_voxel_feats_mean.append(feats[voxel_mask].mean(dim=0))\n ref_voxel_feats_max.append(feats[voxel_mask].max(dim=0).values)\n ref_voxel_feats_mean = torch.stack(ref_voxel_feats_mean)\n ref_voxel_feats_max = torch.stack(ref_voxel_feats_max)\n\n feats_out_mean, coors_out_mean = dsmean(feats, coors)\n seq_mean = (coors_out_mean[:, 0] * 400 + coors_out_mean[:, 1] * 20 +\n coors_out_mean[:, 2]).argsort()\n feats_out_mean = feats_out_mean[seq_mean]\n coors_out_mean = coors_out_mean[seq_mean]\n\n feats_out_max, coors_out_max = dsmax(feats, coors)\n seq_max = (coors_out_max[:, 0] * 400 + coors_out_max[:, 1] * 20 +\n coors_out_max[:, 2]).argsort()\n feats_out_max = feats_out_max[seq_max]\n coors_cout_max = coors_out_max[seq_max]\n\n assert (coors_out_mean == ref_voxel_coors).all()\n assert torch.allclose(\n feats_out_mean, ref_voxel_feats_mean, atol=1e-2, rtol=1e-5)\n assert (coors_cout_max == ref_voxel_coors).all()\n assert torch.allclose(\n feats_out_max, ref_voxel_feats_max, atol=1e-2, rtol=1e-5)\n\n # test grad #\n feats = torch.rand(\n size=(100, 4), dtype=torch.float32, device='cuda') * 100 - 50\n coors = torch.randint(\n low=-1, high=3, size=(100, 3), dtype=torch.int32, device='cuda')\n feats.requires_grad_()\n gradcheck(dsmean, (feats, coors), eps=1e-2, atol=1e-2, rtol=1e-5)\n gradcheck(dsmax, (feats, coors), eps=1e-2, atol=1e-2, rtol=1e-5)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_syncbn.py", "content": "import os\nimport platform\n\nimport numpy as np\nimport pytest\nimport torch\nimport torch.distributed as dist\nimport torch.nn as nn\n\nif platform.system() == 'Windows':\n import regex as re\nelse:\n import re\n\n\nclass TestSyncBN(object):\n\n def dist_init(self):\n rank = int(os.environ['SLURM_PROCID'])\n world_size = int(os.environ['SLURM_NTASKS'])\n local_rank = int(os.environ['SLURM_LOCALID'])\n node_list = str(os.environ['SLURM_NODELIST'])\n\n node_parts = re.findall('[0-9]+', node_list)\n os.environ['MASTER_ADDR'] = (f'{node_parts[1]}.{node_parts[2]}' +\n f'.{node_parts[3]}.{node_parts[4]}')\n os.environ['MASTER_PORT'] = '12341'\n os.environ['WORLD_SIZE'] = str(world_size)\n os.environ['RANK'] = str(rank)\n\n dist.init_process_group('nccl')\n torch.cuda.set_device(local_rank)\n\n def _test_syncbn_train(self, size=1, half=False):\n\n if 'SLURM_NTASKS' not in os.environ or int(\n os.environ['SLURM_NTASKS']) != 4:\n print('must run with slurm has 4 processes!\\n'\n 'srun -p test --gres=gpu:4 -n4')\n return\n else:\n print('Running syncbn test')\n from mmcv.ops import SyncBatchNorm\n\n assert size in (1, 2, 4)\n if not dist.is_initialized():\n self.dist_init()\n rank = dist.get_rank()\n\n torch.manual_seed(9)\n torch.cuda.manual_seed(9)\n\n self.x = torch.rand(16, 3, 2, 3).cuda()\n self.y_bp = torch.rand(16, 3, 2, 3).cuda()\n\n if half:\n self.x = self.x.half()\n self.y_bp = self.y_bp.half()\n dist.broadcast(self.x, src=0)\n dist.broadcast(self.y_bp, src=0)\n\n torch.cuda.synchronize()\n if size == 1:\n groups = [None, None, None, None]\n groups[0] = dist.new_group([0])\n groups[1] = dist.new_group([1])\n groups[2] = dist.new_group([2])\n groups[3] = dist.new_group([3])\n group = groups[rank]\n elif size == 2:\n groups = [None, None, None, None]\n groups[0] = groups[1] = dist.new_group([0, 1])\n groups[2] = groups[3] = dist.new_group([2, 3])\n group = groups[rank]\n elif size == 4:\n group = dist.group.WORLD\n syncbn = SyncBatchNorm(3, group=group).cuda()\n syncbn.weight.data[0] = 0.2\n syncbn.weight.data[1] = 0.5\n syncbn.weight.data[2] = 0.7\n syncbn.train()\n\n bn = nn.BatchNorm2d(3).cuda()\n bn.weight.data[0] = 0.2\n bn.weight.data[1] = 0.5\n bn.weight.data[2] = 0.7\n bn.train()\n\n sx = self.x[rank * 4:rank * 4 + 4]\n sx.requires_grad_()\n sy = syncbn(sx)\n sy.backward(self.y_bp[rank * 4:rank * 4 + 4])\n\n smean = syncbn.running_mean\n svar = syncbn.running_var\n sx_grad = sx.grad\n sw_grad = syncbn.weight.grad\n sb_grad = syncbn.bias.grad\n\n if size == 1:\n x = self.x[rank * 4:rank * 4 + 4]\n y_bp = self.y_bp[rank * 4:rank * 4 + 4]\n elif size == 2:\n x = self.x[rank // 2 * 8:rank // 2 * 8 + 8]\n y_bp = self.y_bp[rank // 2 * 8:rank // 2 * 8 + 8]\n elif size == 4:\n x = self.x\n y_bp = self.y_bp\n x.requires_grad_()\n y = bn(x)\n y.backward(y_bp)\n\n if size == 2:\n y = y[rank % 2 * 4:rank % 2 * 4 + 4]\n elif size == 4:\n y = y[rank * 4:rank * 4 + 4]\n\n mean = bn.running_mean\n var = bn.running_var\n if size == 1:\n x_grad = x.grad\n w_grad = bn.weight.grad\n b_grad = bn.bias.grad\n elif size == 2:\n x_grad = x.grad[rank % 2 * 4:rank % 2 * 4 + 4]\n w_grad = bn.weight.grad / 2\n b_grad = bn.bias.grad / 2\n elif size == 4:\n x_grad = x.grad[rank * 4:rank * 4 + 4]\n w_grad = bn.weight.grad / 4\n b_grad = bn.bias.grad / 4\n\n assert np.allclose(mean.data.cpu().numpy(),\n smean.data.cpu().numpy(), 1e-3)\n assert np.allclose(var.data.cpu().numpy(),\n svar.data.cpu().numpy(), 1e-3)\n assert np.allclose(y.data.cpu().numpy(), sy.data.cpu().numpy(), 1e-3)\n assert np.allclose(w_grad.data.cpu().numpy(),\n sw_grad.data.cpu().numpy(), 1e-3)\n assert np.allclose(b_grad.data.cpu().numpy(),\n sb_grad.data.cpu().numpy(), 1e-3)\n assert np.allclose(x_grad.data.cpu().numpy(),\n sx_grad.data.cpu().numpy(), 1e-2)\n\n def _test_syncbn_empty_train(self, size=1, half=False):\n\n if 'SLURM_NTASKS' not in os.environ or int(\n os.environ['SLURM_NTASKS']) != 4:\n print('must run with slurm has 4 processes!\\n'\n 'srun -p test --gres=gpu:4 -n4')\n return\n else:\n print('Running syncbn test')\n from mmcv.ops import SyncBatchNorm\n\n assert size in (1, 2, 4)\n if not dist.is_initialized():\n self.dist_init()\n rank = dist.get_rank()\n\n torch.manual_seed(9)\n torch.cuda.manual_seed(9)\n\n self.x = torch.rand(0, 3, 2, 3).cuda()\n self.y_bp = torch.rand(0, 3, 2, 3).cuda()\n\n if half:\n self.x = self.x.half()\n self.y_bp = self.y_bp.half()\n dist.broadcast(self.x, src=0)\n dist.broadcast(self.y_bp, src=0)\n\n torch.cuda.synchronize()\n if size == 1:\n groups = [None, None, None, None]\n groups[0] = dist.new_group([0])\n groups[1] = dist.new_group([1])\n groups[2] = dist.new_group([2])\n groups[3] = dist.new_group([3])\n group = groups[rank]\n elif size == 2:\n groups = [None, None, None, None]\n groups[0] = groups[1] = dist.new_group([0, 1])\n groups[2] = groups[3] = dist.new_group([2, 3])\n group = groups[rank]\n elif size == 4:\n group = dist.group.WORLD\n\n syncbn = SyncBatchNorm(3, group=group, stats_mode='N').cuda()\n syncbn.weight.data[0] = 0.2\n syncbn.weight.data[1] = 0.5\n syncbn.weight.data[2] = 0.7\n syncbn.train()\n\n bn = nn.BatchNorm2d(3).cuda()\n bn.weight.data[0] = 0.2\n bn.weight.data[1] = 0.5\n bn.weight.data[2] = 0.7\n bn.train()\n\n sx = self.x[rank * 4:rank * 4 + 4]\n sx.requires_grad_()\n sy = syncbn(sx)\n sy.backward(self.y_bp[rank * 4:rank * 4 + 4])\n smean = syncbn.running_mean\n svar = syncbn.running_var\n sx_grad = sx.grad\n sw_grad = syncbn.weight.grad\n sb_grad = syncbn.bias.grad\n\n if size == 1:\n x = self.x[rank * 4:rank * 4 + 4]\n y_bp = self.y_bp[rank * 4:rank * 4 + 4]\n elif size == 2:\n x = self.x[rank // 2 * 8:rank // 2 * 8 + 8]\n y_bp = self.y_bp[rank // 2 * 8:rank // 2 * 8 + 8]\n elif size == 4:\n x = self.x\n y_bp = self.y_bp\n x.requires_grad_()\n y = bn(x)\n y.backward(y_bp)\n\n if size == 2:\n y = y[rank % 2 * 4:rank % 2 * 4 + 4]\n elif size == 4:\n y = y[rank * 4:rank * 4 + 4]\n\n mean = bn.running_mean\n var = bn.running_var\n if size == 1:\n x_grad = x.grad\n w_grad = bn.weight.grad\n b_grad = bn.bias.grad\n elif size == 2:\n x_grad = x.grad[rank % 2 * 4:rank % 2 * 4 + 4]\n w_grad = bn.weight.grad / 2\n b_grad = bn.bias.grad / 2\n elif size == 4:\n x_grad = x.grad[rank * 4:rank * 4 + 4]\n w_grad = bn.weight.grad / 4\n b_grad = bn.bias.grad / 4\n\n assert np.allclose(mean.data.cpu().numpy(),\n smean.data.cpu().numpy(), 1e-3)\n assert np.allclose(var.data.cpu().numpy(),\n svar.data.cpu().numpy(), 1e-3)\n assert np.allclose(y.data.cpu().numpy(), sy.data.cpu().numpy(), 1e-3)\n assert np.allclose(w_grad.data.cpu().numpy(),\n sw_grad.data.cpu().numpy(), 1e-3)\n assert np.allclose(b_grad.data.cpu().numpy(),\n sb_grad.data.cpu().numpy(), 1e-3)\n assert np.allclose(x_grad.data.cpu().numpy(),\n sx_grad.data.cpu().numpy(), 1e-2)\n\n # 'stats_mode' only allows 'default' and 'N'\n with pytest.raises(AssertionError):\n SyncBatchNorm(3, group=group, stats_mode='X')\n\n def test_syncbn_1(self):\n self._test_syncbn_train(size=1)\n\n def test_syncbn_2(self):\n self._test_syncbn_train(size=2)\n\n def test_syncbn_4(self):\n self._test_syncbn_train(size=4)\n\n def test_syncbn_1_half(self):\n self._test_syncbn_train(size=1, half=True)\n\n def test_syncbn_2_half(self):\n self._test_syncbn_train(size=2, half=True)\n\n def test_syncbn_4_half(self):\n self._test_syncbn_train(size=4, half=True)\n\n def test_syncbn_empty_1(self):\n self._test_syncbn_empty_train(size=1)\n\n def test_syncbn_empty_2(self):\n self._test_syncbn_empty_train(size=2)\n\n def test_syncbn_empty_4(self):\n self._test_syncbn_empty_train(size=4)\n\n def test_syncbn_empty_1_half(self):\n self._test_syncbn_empty_train(size=1, half=True)\n\n def test_syncbn_empty_2_half(self):\n self._test_syncbn_empty_train(size=2, half=True)\n\n def test_syncbn_empty_4_half(self):\n self._test_syncbn_empty_train(size=4, half=True)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_tensorrt.py", "content": "import os\nfrom functools import partial\nfrom typing import Callable\n\nimport numpy as np\nimport onnx\nimport pytest\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\ntry:\n from mmcv.tensorrt import (TRTWrapper, is_tensorrt_plugin_loaded, onnx2trt,\n save_trt_engine)\nexcept ImportError:\n pytest.skip(\n 'TensorRT should be installed from source.', allow_module_level=True)\n\nif not torch.cuda.is_available():\n pytest.skip(\n 'CUDA is required for this test module', allow_module_level=True)\n\nif not is_tensorrt_plugin_loaded():\n pytest.skip(\n 'Test requires to complie TensorRT plugins in mmcv',\n allow_module_level=True)\n\n\nclass WrapFunction(nn.Module):\n\n def __init__(self, wrapped_function):\n super(WrapFunction, self).__init__()\n self.wrapped_function = wrapped_function\n\n def forward(self, *args, **kwargs):\n return self.wrapped_function(*args, **kwargs)\n\n\nonnx_file = 'tmp.onnx'\ntrt_file = 'tmp.engine'\n\n\ndef test_roialign():\n try:\n from mmcv.ops import RoIAlign\n except (ImportError, ModuleNotFoundError):\n pytest.skip('test requires compilation')\n\n # trt config\n fp16_mode = False\n max_workspace_size = 1 << 30\n\n # roi align config\n pool_h = 2\n pool_w = 2\n spatial_scale = 1.0\n sampling_ratio = 2\n\n inputs = [([[[[1., 2.], [3., 4.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2.], [3., 4.]], [[4., 3.],\n [2., 1.]]]], [[0., 0., 0., 1., 1.]]),\n ([[[[1., 2., 5., 6.], [3., 4., 7., 8.], [9., 10., 13., 14.],\n [11., 12., 15., 16.]]]], [[0., 0., 0., 3., 3.]])]\n\n wrapped_model = RoIAlign((pool_w, pool_h), spatial_scale, sampling_ratio,\n 'avg', True).cuda()\n for case in inputs:\n np_input = np.array(case[0], dtype=np.float32)\n np_rois = np.array(case[1], dtype=np.float32)\n input = torch.from_numpy(np_input).cuda()\n rois = torch.from_numpy(np_rois).cuda()\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (input, rois),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input', 'rois'],\n output_names=['roi_feat'],\n opset_version=11)\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n opt_shape_dict = {\n 'input': [list(input.shape),\n list(input.shape),\n list(input.shape)],\n 'rois': [list(rois.shape),\n list(rois.shape),\n list(rois.shape)]\n }\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, ['input', 'rois'], ['roi_feat'])\n\n with torch.no_grad():\n trt_outputs = trt_model({'input': input, 'rois': rois})\n trt_roi_feat = trt_outputs['roi_feat']\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_roi_feat = wrapped_model(input, rois)\n\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n assert torch.allclose(pytorch_roi_feat, trt_roi_feat)\n\n\ndef test_nms():\n try:\n import mmcv\n from mmcv.ops import nms\n except (ImportError, ModuleNotFoundError):\n pytest.skip('test requires compilation')\n os.environ['ONNX_BACKEND'] = 'MMCVTensorRT'\n # trt config\n fp16_mode = False\n max_workspace_size = 1 << 30\n data = mmcv.load('./tests/data/batched_nms_data.pkl')\n boxes = torch.from_numpy(data['boxes']).cuda()\n scores = torch.from_numpy(data['scores']).cuda()\n nms = partial(\n nms, iou_threshold=0.7, offset=0, score_threshold=0.1, max_num=100)\n wrapped_model = WrapFunction(nms)\n wrapped_model.cpu().eval()\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (boxes.detach().cpu(), scores.detach().cpu()),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['boxes', 'scores'],\n output_names=['dets', 'inds'],\n opset_version=11)\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n opt_shape_dict = {\n 'boxes': [list(boxes.shape),\n list(boxes.shape),\n list(boxes.shape)],\n 'scores': [list(scores.shape),\n list(scores.shape),\n list(scores.shape)]\n }\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, ['boxes', 'scores'], ['dets', 'inds'])\n\n with torch.no_grad():\n trt_outputs = trt_model({'boxes': boxes, 'scores': scores})\n trt_dets = trt_outputs['dets']\n trt_inds = trt_outputs['inds']\n trt_inds = trt_inds.long()\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_outputs = wrapped_model(boxes, scores)\n pytorch_dets, pytorch_inds = pytorch_outputs\n\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n num_boxes = pytorch_dets.shape[0]\n trt_dets = trt_dets[:num_boxes, ...]\n trt_inds = trt_inds[:num_boxes]\n trt_scores = trt_dets[:, 4]\n pytorch_scores = pytorch_dets[:, 4]\n os.environ.pop('ONNX_BACKEND')\n assert torch.allclose(pytorch_scores, trt_scores, atol=1e-3)\n assert torch.equal(pytorch_inds, trt_inds)\n\n\ndef test_batched_nms():\n try:\n import mmcv\n from mmcv.ops import batched_nms\n except (ImportError, ModuleNotFoundError):\n pytest.skip('test requires compilation')\n\n # trt config\n os.environ['ONNX_BACKEND'] = 'MMCVTensorRT'\n fp16_mode = False\n max_workspace_size = 1 << 30\n data = mmcv.load('./tests/data/batched_nms_data.pkl')\n nms_cfg = dict(type='nms', iou_threshold=0.7, score_threshold=0.1)\n boxes = torch.from_numpy(data['boxes']).cuda()\n scores = torch.from_numpy(data['scores']).cuda()\n idxs = torch.from_numpy(data['idxs']).cuda()\n class_agnostic = False\n\n nms = partial(batched_nms, nms_cfg=nms_cfg, class_agnostic=class_agnostic)\n wrapped_model = WrapFunction(nms)\n wrapped_model.cpu().eval()\n input_data = (boxes.detach().cpu(), scores.detach().cpu(),\n idxs.detach().cpu())\n input_names = ['boxes', 'scores', 'idxs']\n output_names = ['dets', 'inds']\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model,\n input_data,\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=input_names,\n output_names=output_names,\n opset_version=11)\n onnx_model = onnx.load(onnx_file)\n # create trt engine and wrapper\n opt_shape_dict = {\n 'boxes': [list(boxes.shape),\n list(boxes.shape),\n list(boxes.shape)],\n 'scores': [list(scores.shape),\n list(scores.shape),\n list(scores.shape)],\n 'idxs': [list(idxs.shape),\n list(idxs.shape),\n list(idxs.shape)]\n }\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, input_names, output_names)\n\n with torch.no_grad():\n trt_outputs = trt_model({\n 'boxes': boxes,\n 'scores': scores,\n 'idxs': idxs\n })\n trt_dets = trt_outputs['dets']\n trt_inds = trt_outputs['inds']\n trt_inds = trt_inds.long()\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_outputs = wrapped_model(boxes, scores, idxs)\n pytorch_dets, pytorch_inds = pytorch_outputs\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n num_boxes = pytorch_dets.shape[0]\n trt_dets = trt_dets[:num_boxes, ...]\n trt_inds = trt_inds[:num_boxes]\n trt_scores = trt_dets[:, 4]\n pytorch_scores = pytorch_dets[:, 4]\n\n os.environ.pop('ONNX_BACKEND')\n assert torch.allclose(pytorch_scores, trt_scores)\n assert torch.equal(pytorch_inds, trt_inds)\n\n\ndef test_scatternd():\n\n def func(data):\n data[:, :-2] += 1\n data[:2, :] -= 1\n return data\n\n data = torch.zeros(4, 4).cuda()\n wrapped_model = WrapFunction(func).eval().cuda()\n\n input_names = ['input']\n output_names = ['output']\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (data.clone(), ),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=input_names,\n output_names=output_names,\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n opt_shape_dict = {\n 'input': [list(data.shape),\n list(data.shape),\n list(data.shape)],\n }\n # trt config\n fp16_mode = False\n max_workspace_size = 1 << 30\n\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, input_names, output_names)\n\n with torch.no_grad():\n trt_outputs = trt_model({'input': data.clone()})\n trt_results = trt_outputs['output']\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_results = wrapped_model(data.clone())\n\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n assert torch.allclose(pytorch_results, trt_results)\n\n\ndef test_deform_conv():\n try:\n from mmcv.ops import DeformConv2dPack\n except (ImportError, ModuleNotFoundError):\n pytest.skip('test requires compilation')\n\n input = [[[[1., 2., 3.], [0., 1., 2.], [3., 5., 2.]]]]\n offset_weight = [[[0.1, 0.4, 0.6, 0.1]], [[0.3, 0.2, 0.1, 0.3]],\n [[0.5, 0.5, 0.2, 0.8]], [[0.8, 0.3, 0.9, 0.1]],\n [[0.3, 0.1, 0.2, 0.5]], [[0.3, 0.7, 0.5, 0.3]],\n [[0.6, 0.2, 0.5, 0.3]], [[0.4, 0.1, 0.8, 0.4]]]\n offset_bias = [0.7, 0.1, 0.8, 0.5, 0.6, 0.5, 0.4, 0.7]\n deform_weight = [[[0.4, 0.2, 0.1, 0.9]]]\n\n c_in = 1\n c_out = 1\n x = torch.Tensor(input).cuda()\n x.requires_grad = True\n model = DeformConv2dPack(c_in, c_out, 2, stride=1, padding=0)\n model.conv_offset.weight.data = torch.nn.Parameter(\n torch.Tensor(offset_weight).reshape(8, 1, 2, 2))\n model.conv_offset.bias.data = torch.nn.Parameter(\n torch.Tensor(offset_bias).reshape(8))\n model.weight.data = torch.nn.Parameter(\n torch.Tensor(deform_weight).reshape(1, 1, 2, 2))\n model.cuda().eval()\n\n input_names = ['input']\n output_names = ['output']\n\n with torch.no_grad():\n torch.onnx.export(\n model, (x.clone(), ),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=input_names,\n output_names=output_names,\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n opt_shape_dict = {\n 'input': [list(x.shape), list(x.shape),\n list(x.shape)],\n }\n # trt config\n fp16_mode = False\n max_workspace_size = 1 << 30\n\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, input_names, output_names)\n\n with torch.no_grad():\n trt_outputs = trt_model({'input': x.clone()})\n trt_results = trt_outputs['output']\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_results = model(x.clone())\n\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n assert torch.allclose(pytorch_results, trt_results)\n\n\n@pytest.mark.parametrize('with_bias', [True, False])\ndef test_modulated_deform_conv(with_bias):\n try:\n from mmcv.ops import ModulatedDeformConv2dPack\n except (ImportError, ModuleNotFoundError):\n pytest.skip('test requires compilation')\n\n input = [[[[1., 2., 3.], [0., 1., 2.], [3., 5., 2.]]]]\n\n x = torch.Tensor(input).cuda()\n model = ModulatedDeformConv2dPack(\n 1,\n 1,\n kernel_size=(2, 2),\n stride=1,\n padding=1,\n deform_groups=1,\n bias=with_bias)\n model.weight.data.fill_(1.)\n model.type(torch.float32)\n model = model.cuda().eval()\n\n input_names = ['input']\n output_names = ['output']\n\n with torch.no_grad():\n torch.onnx.export(\n model, (x.clone(), ),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=input_names,\n output_names=output_names,\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n opt_shape_dict = {\n 'input': [list(x.shape), list(x.shape),\n list(x.shape)],\n }\n # trt config\n fp16_mode = False\n max_workspace_size = 1 << 30\n\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, input_names, output_names)\n\n with torch.no_grad():\n trt_outputs = trt_model({'input': x.clone()})\n trt_results = trt_outputs['output']\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_results = model(x.clone())\n\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n torch.testing.assert_allclose(pytorch_results, trt_results)\n\n\n@pytest.mark.parametrize('mode', ['bilinear', 'nearest'])\n@pytest.mark.parametrize('padding_mode', ['zeros', 'border', 'reflection'])\n@pytest.mark.parametrize('align_corners', [True, False])\ndef test_grid_sample(mode, padding_mode, align_corners):\n from mmcv.onnx.symbolic import register_extra_symbolics\n\n register_extra_symbolics(11)\n\n input = torch.rand(1, 1, 10, 10).cuda()\n grid = torch.Tensor([[[1, 0, 0], [0, 1, 0]]])\n grid = F.affine_grid(grid, (1, 1, 15, 15)).type_as(input).cuda()\n\n def func(input, grid):\n return F.grid_sample(\n input,\n grid,\n mode=mode,\n padding_mode=padding_mode,\n align_corners=align_corners)\n\n wrapped_model = WrapFunction(func).eval().cuda()\n\n input_names = ['input', 'grid']\n output_names = ['output']\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (input.clone(), grid.clone()),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=input_names,\n output_names=output_names,\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n opt_shape_dict = {\n 'input': [list(input.shape),\n list(input.shape),\n list(input.shape)],\n 'grid': [list(grid.shape),\n list(grid.shape),\n list(grid.shape)],\n }\n # trt config\n fp16_mode = False\n max_workspace_size = 1 << 30\n\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, input_names, output_names)\n\n with torch.no_grad():\n trt_outputs = trt_model({'input': input.clone(), 'grid': grid.clone()})\n trt_results = trt_outputs['output']\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_results = wrapped_model(input.clone(), grid.clone())\n\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n assert torch.allclose(pytorch_results, trt_results)\n\n\n@pytest.mark.parametrize('func', [torch.cummax, torch.cummin])\ndef test_cummin_cummax(func: Callable):\n # Note generally `cummax` or `cummin` is exportable to ONNX\n # as long as the pytorch version >= 1.5.0, since `torch.cummax`\n # is only supported with torch >= 1.5.0.\n # But when `cummax` or `cummin` serves as an intermediate component\n # whose outputs is used as inputs for another modules, it's expected\n # that pytorch version must be >= 1.7.0. Otherwise error appears like:\n # `RuntimeError: tuple appears in op that does not forward tuples,\n # unsupported 'kind: prim::PythonOp`.\n from packaging import version\n if version.parse(torch.__version__) < version.parse('1.7.0'):\n pytest.skip('test_cummax_cummin should be ran with pytorch >= 1.7.0')\n\n opset = 11\n # register custom op `mmcv::cummax` and `mmcv::cummin`\n from mmcv.onnx.symbolic import register_extra_symbolics\n register_extra_symbolics(opset)\n\n input_list = [\n # arbitrary shape, e.g. 1-D, 2-D, 3-D, ...\n torch.rand((2, 3, 4, 1, 5)).cuda(),\n torch.rand((1)).cuda()\n ]\n\n input_names = ['input']\n output_names = ['output', 'indices']\n\n for input in input_list:\n ndims = input.dim()\n # valid dim range is [-ndims, ndims-1]\n # test for all `dim` value which is valid\n for dim in range(-ndims, ndims):\n cummax_func = partial(func, dim=dim)\n wrapped_model = WrapFunction(cummax_func).eval().cuda()\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model,\n input,\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=False,\n input_names=input_names,\n output_names=output_names,\n opset_version=opset)\n\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n opt_shape_dict = {\n 'input':\n [list(input.shape),\n list(input.shape),\n list(input.shape)]\n }\n # trt config\n fp16_mode = False\n max_workspace_size = 1 << 30\n\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n\n # remove ONNX model after conversion\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n\n # save TensorRT model\n save_trt_engine(trt_engine, trt_file)\n\n # load and wrap TensorRT model\n trt_model = TRTWrapper(trt_file)\n\n # remove trt model after loading\n if os.path.exists(trt_file):\n os.remove(trt_file)\n\n # compute trt output\n with torch.no_grad():\n trt_results = trt_model({'input': input.contiguous().clone()})\n trt_output = trt_results['output']\n trt_indices = trt_results['indices']\n\n # compute pytorch output\n with torch.no_grad():\n pytorch_results = wrapped_model(input.clone())\n pytorch_output = pytorch_results[0]\n pytorch_indices = pytorch_results[1]\n\n torch.testing.assert_allclose(trt_output, pytorch_output)\n torch.testing.assert_allclose(trt_indices, pytorch_indices)\n\n\n@pytest.mark.parametrize('dynamic_export', [True, False])\n@pytest.mark.parametrize('fp16_mode', [True, False])\ndef test_instance_norm(dynamic_export, fp16_mode):\n\n n, c, h, w = 2, 3, 10, 10\n data = torch.randn(n, c, h, w).cuda()\n norm = nn.InstanceNorm2d(c, affine=True)\n\n wrapped_model = WrapFunction(norm).eval().cuda()\n\n input_names = ['input']\n output_names = ['output']\n dynamic_axes = None\n if dynamic_export:\n dynamic_axes = {\n 'input': {\n 0: 'n',\n 2: 'h',\n 3: 'w',\n },\n 'output': {\n 0: 'n',\n 2: 'h',\n 3: 'w',\n },\n }\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (data.clone(), ),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=input_names,\n output_names=output_names,\n dynamic_axes=dynamic_axes,\n opset_version=11)\n\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n if dynamic_export:\n opt_shape_dict = {\n 'input':\n [list(data.shape),\n list(data.shape), [2 * n, c, 2 * h, 2 * w]],\n }\n else:\n opt_shape_dict = {\n 'input': [list(data.shape),\n list(data.shape),\n list(data.shape)],\n }\n # trt config\n max_workspace_size = 1 << 30\n\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, input_names, output_names)\n\n with torch.no_grad():\n trt_outputs = trt_model({'input': data.clone()})\n trt_results = trt_outputs['output']\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_results = wrapped_model(data.clone())\n\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n assert torch.allclose(pytorch_results, trt_results)\n\n\n@pytest.mark.parametrize('mode', ['top', 'bottom', 'left', 'right'])\ndef test_corner_pool(mode):\n try:\n from mmcv.ops import CornerPool\n except (ImportError, ModuleNotFoundError):\n pytest.skip('test requires compilation')\n\n opset = 11\n # register custom op `mmcv::MMCVCornerPool`\n from mmcv.onnx.symbolic import register_extra_symbolics\n register_extra_symbolics(opset)\n\n # trt config\n fp16_mode = False\n max_workspace_size = 1 << 30\n\n inputs = [\n # (n, c, h, w)\n torch.rand((2, 3, 5, 5)),\n torch.rand((1, 2, 4, 6)),\n torch.rand((2, 1, 3, 2)),\n ]\n\n class CornerPoolWrapper(CornerPool):\n\n def __init__(self, mode):\n super(CornerPoolWrapper, self).__init__(mode)\n\n def forward(self, x):\n # no use `torch.cummax`, instead `corner_pool` is used\n # for various torch version\n return self.corner_pool.apply(x)\n\n wrapped_model = CornerPoolWrapper(mode).cuda()\n for input in inputs:\n input = input.cuda()\n\n with torch.no_grad():\n torch.onnx.export(\n wrapped_model, (input, ),\n onnx_file,\n export_params=True,\n keep_initializers_as_inputs=True,\n input_names=['input'],\n output_names=['output'],\n opset_version=opset)\n onnx_model = onnx.load(onnx_file)\n\n # create trt engine and wrapper\n opt_shape_dict = {\n 'input': [list(input.shape),\n list(input.shape),\n list(input.shape)],\n }\n trt_engine = onnx2trt(\n onnx_model,\n opt_shape_dict,\n fp16_mode=fp16_mode,\n max_workspace_size=max_workspace_size)\n save_trt_engine(trt_engine, trt_file)\n trt_model = TRTWrapper(trt_file, ['input'], ['output'])\n\n with torch.no_grad():\n trt_outputs = trt_model({'input': input})\n trt_pool_feat = trt_outputs['output']\n\n # compute pytorch_output\n with torch.no_grad():\n pytorch_pool_feat = wrapped_model(input)\n\n # allclose\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n if os.path.exists(trt_file):\n os.remove(trt_file)\n assert torch.allclose(pytorch_pool_feat, trt_pool_feat, atol=1e-5)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_tensorrt_preprocess.py", "content": "import os\nfrom functools import wraps\n\nimport onnx\nimport torch\n\nfrom mmcv.ops import nms\nfrom mmcv.tensorrt.preprocess import preprocess_onnx\n\n\ndef remove_tmp_file(func):\n\n @wraps(func)\n def wrapper(*args, **kwargs):\n onnx_file = 'tmp.onnx'\n kwargs['onnx_file'] = onnx_file\n try:\n result = func(*args, **kwargs)\n finally:\n if os.path.exists(onnx_file):\n os.remove(onnx_file)\n return result\n\n return wrapper\n\n\n@remove_tmp_file\ndef export_nms_module_to_onnx(module, onnx_file):\n torch_model = module()\n torch_model.eval()\n\n input = (torch.rand([100, 4], dtype=torch.float32),\n torch.rand([100], dtype=torch.float32))\n\n torch.onnx.export(\n torch_model,\n input,\n onnx_file,\n opset_version=11,\n input_names=['boxes', 'scores'],\n output_names=['output'])\n\n onnx_model = onnx.load(onnx_file)\n return onnx_model\n\n\ndef test_can_handle_nms_with_constant_maxnum():\n\n class ModuleNMS(torch.nn.Module):\n\n def forward(self, boxes, scores):\n return nms(boxes, scores, iou_threshold=0.4, max_num=10)\n\n onnx_model = export_nms_module_to_onnx(ModuleNMS)\n preprocess_onnx_model = preprocess_onnx(onnx_model)\n for node in preprocess_onnx_model.graph.node:\n if 'NonMaxSuppression' in node.name:\n assert len(node.attribute) == 5, 'The NMS must have 5 attributes.'\n\n\ndef test_can_handle_nms_with_undefined_maxnum():\n\n class ModuleNMS(torch.nn.Module):\n\n def forward(self, boxes, scores):\n return nms(boxes, scores, iou_threshold=0.4)\n\n onnx_model = export_nms_module_to_onnx(ModuleNMS)\n preprocess_onnx_model = preprocess_onnx(onnx_model)\n for node in preprocess_onnx_model.graph.node:\n if 'NonMaxSuppression' in node.name:\n assert len(node.attribute) == 5, \\\n 'The NMS must have 5 attributes.'\n assert node.attribute[2].i > 0, \\\n 'The max_output_boxes_per_class is not defined correctly.'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_three_interpolate.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import three_interpolate\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_three_interpolate():\n features = torch.tensor([[[2.4350, 4.7516, 4.4995, 2.4350, 2.4350, 2.4350],\n [3.1236, 2.6278, 3.0447, 3.1236, 3.1236, 3.1236],\n [2.6732, 2.8677, 2.6436, 2.6732, 2.6732, 2.6732],\n [0.0124, 7.0150, 7.0199, 0.0124, 0.0124, 0.0124],\n [0.3207, 0.0000, 0.3411, 0.3207, 0.3207,\n 0.3207]],\n [[0.0000, 0.9544, 2.4532, 0.0000, 0.0000, 0.0000],\n [0.5346, 1.9176, 1.4715, 0.5346, 0.5346, 0.5346],\n [0.0000, 0.2744, 2.0842, 0.0000, 0.0000, 0.0000],\n [0.3414, 1.5063, 1.6209, 0.3414, 0.3414, 0.3414],\n [0.5814, 0.0103, 0.0000, 0.5814, 0.5814,\n 0.5814]]]).cuda()\n\n idx = torch.tensor([[[0, 1, 2], [2, 3, 4], [2, 3, 4], [0, 1, 2], [0, 1, 2],\n [0, 1, 3]],\n [[0, 2, 3], [1, 3, 4], [2, 1, 4], [0, 2, 4], [0, 2, 4],\n [0, 1, 2]]]).int().cuda()\n\n weight = torch.tensor([[[3.3333e-01, 3.3333e-01, 3.3333e-01],\n [1.0000e+00, 5.8155e-08, 2.2373e-08],\n [1.0000e+00, 1.7737e-08, 1.7356e-08],\n [3.3333e-01, 3.3333e-01, 3.3333e-01],\n [3.3333e-01, 3.3333e-01, 3.3333e-01],\n [3.3333e-01, 3.3333e-01, 3.3333e-01]],\n [[3.3333e-01, 3.3333e-01, 3.3333e-01],\n [1.0000e+00, 1.3651e-08, 7.7312e-09],\n [1.0000e+00, 1.7148e-08, 1.4070e-08],\n [3.3333e-01, 3.3333e-01, 3.3333e-01],\n [3.3333e-01, 3.3333e-01, 3.3333e-01],\n [3.3333e-01, 3.3333e-01, 3.3333e-01]]]).cuda()\n\n output = three_interpolate(features, idx, weight)\n expected_output = torch.tensor([[[\n 3.8953e+00, 4.4995e+00, 4.4995e+00, 3.8953e+00, 3.8953e+00, 3.2072e+00\n ], [\n 2.9320e+00, 3.0447e+00, 3.0447e+00, 2.9320e+00, 2.9320e+00, 2.9583e+00\n ], [\n 2.7281e+00, 2.6436e+00, 2.6436e+00, 2.7281e+00, 2.7281e+00, 2.7380e+00\n ], [\n 4.6824e+00, 7.0199e+00, 7.0199e+00, 4.6824e+00, 4.6824e+00, 2.3466e+00\n ], [\n 2.2060e-01, 3.4110e-01, 3.4110e-01, 2.2060e-01, 2.2060e-01, 2.1380e-01\n ]],\n [[\n 8.1773e-01, 9.5440e-01, 2.4532e+00,\n 8.1773e-01, 8.1773e-01, 1.1359e+00\n ],\n [\n 8.4689e-01, 1.9176e+00, 1.4715e+00,\n 8.4689e-01, 8.4689e-01, 1.3079e+00\n ],\n [\n 6.9473e-01, 2.7440e-01, 2.0842e+00,\n 6.9473e-01, 6.9473e-01, 7.8619e-01\n ],\n [\n 7.6789e-01, 1.5063e+00, 1.6209e+00,\n 7.6789e-01, 7.6789e-01, 1.1562e+00\n ],\n [\n 3.8760e-01, 1.0300e-02, 8.3569e-09,\n 3.8760e-01, 3.8760e-01, 1.9723e-01\n ]]]).cuda()\n\n assert torch.allclose(output, expected_output, 1e-4)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_three_nn.py", "content": "import pytest\nimport torch\n\nfrom mmcv.ops import three_nn\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_three_nn():\n known = torch.tensor([[[-1.8373, 3.5605,\n -0.7867], [0.7615, 2.9420, 0.2314],\n [-0.6503, 3.6637, -1.0622],\n [-1.8373, 3.5605, -0.7867],\n [-1.8373, 3.5605, -0.7867]],\n [[-1.3399, 1.9991, -0.3698],\n [-0.0799, 0.9698,\n -0.8457], [0.0858, 2.4721, -0.1928],\n [-1.3399, 1.9991, -0.3698],\n [-1.3399, 1.9991, -0.3698]]]).cuda()\n\n unknown = torch.tensor([[[-1.8373, 3.5605, -0.7867],\n [0.7615, 2.9420, 0.2314],\n [-0.6503, 3.6637, -1.0622],\n [-1.5237, 2.3976, -0.8097],\n [-0.0722, 3.4017, -0.2880],\n [0.5198, 3.0661, -0.4605],\n [-2.0185, 3.5019, -0.3236],\n [0.5098, 3.1020, 0.5799],\n [-1.6137, 3.8443, -0.5269],\n [0.7341, 2.9626, -0.3189]],\n [[-1.3399, 1.9991, -0.3698],\n [-0.0799, 0.9698, -0.8457],\n [0.0858, 2.4721, -0.1928],\n [-0.9022, 1.6560, -1.3090],\n [0.1156, 1.6901, -0.4366],\n [-0.6477, 2.3576, -0.1563],\n [-0.8482, 1.1466, -1.2704],\n [-0.8753, 2.0845, -0.3460],\n [-0.5621, 1.4233, -1.2858],\n [-0.5883, 1.3114, -1.2899]]]).cuda()\n\n dist, idx = three_nn(unknown, known)\n expected_dist = torch.tensor([[[0.0000, 0.0000, 0.0000],\n [0.0000, 2.0463, 2.8588],\n [0.0000, 1.2229, 1.2229],\n [1.2047, 1.2047, 1.2047],\n [1.0011, 1.0845, 1.8411],\n [0.7433, 1.4451, 2.4304],\n [0.5007, 0.5007, 0.5007],\n [0.4587, 2.0875, 2.7544],\n [0.4450, 0.4450, 0.4450],\n [0.5514, 1.7206, 2.6811]],\n [[0.0000, 0.0000, 0.0000],\n [0.0000, 1.6464, 1.6952],\n [0.0000, 1.5125, 1.5125],\n [1.0915, 1.0915, 1.0915],\n [0.8197, 0.8511, 1.4894],\n [0.7433, 0.8082, 0.8082],\n [0.8955, 1.3340, 1.3340],\n [0.4730, 0.4730, 0.4730],\n [0.7949, 1.3325, 1.3325],\n [0.7566, 1.3727, 1.3727]]]).cuda()\n expected_idx = torch.tensor([[[0, 3, 4], [1, 2, 0], [2, 0, 3], [0, 3, 4],\n [2, 1, 0], [1, 2, 0], [0, 3, 4], [1, 2, 0],\n [0, 3, 4], [1, 2, 0]],\n [[0, 3, 4], [1, 2, 0], [2, 0, 3], [0, 3, 4],\n [2, 1, 0], [2, 0, 3], [1, 0, 3], [0, 3, 4],\n [1, 0, 3], [1, 0, 3]]]).cuda()\n\n assert torch.allclose(dist, expected_dist, 1e-4)\n assert torch.all(idx == expected_idx)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_tin_shift.py", "content": "import os\n\nimport numpy as np\nimport pytest\nimport torch\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck\n\n _USING_PARROTS = False\n\ncur_dir = os.path.dirname(os.path.abspath(__file__))\n\ninputs = ([[[[0.88572276, 0.46422583], [0.97408265, 0.59547687],\n [0.030812204, 0.96236038], [0.75418317, 0.44058233],\n [0.33279222, 0.00084149837], [0.7069388, 0.23255438],\n [0.13547045, 0.81549376], [0.40174931, 0.36317211]],\n [[0.57444429, 0.15905505], [0.39897251, 0.25790238],\n [0.93282568, 0.18451685], [0.92526674, 0.18283755],\n [0.31664443, 0.59323865], [0.1957739, 0.42505842],\n [0.081158757, 0.81340349], [0.43456328, 0.30195212]],\n [[0.8198145, 0.05990988], [0.98062474, 0.34803438],\n [0.10412294, 0.37183142], [0.15021622, 0.038857818],\n [0.40985721, 0.42253625], [0.71150124, 0.59778064],\n [0.83851069, 0.15194464], [0.097513378, 0.74820143]],\n [[0.80680406, 0.49327564], [0.17821097, 0.12980539],\n [0.50657678, 0.14446253], [0.04178369, 0.53071898],\n [0.84983683, 0.3826949], [0.32193625, 0.91275406],\n [0.75628334, 0.52934098], [0.27994192, 0.3053292]]],\n [[[0.082397044, 0.4210068], [0.23563534, 0.7938987],\n [0.63669145, 0.69397897], [0.8844561, 0.97854084],\n [0.79027033, 0.60640401], [0.63528901, 0.72172403],\n [0.0097346902, 0.70800996], [0.87891227, 0.13674974]],\n [[0.74329448, 0.0243572], [0.82178867, 0.85750699],\n [0.7568835, 0.73146772], [0.5031184, 0.30479157],\n [0.28713053, 0.47414285], [0.4682079, 0.067471564],\n [0.48368263, 0.14590704], [0.25397325, 0.19946373]],\n [[0.4291026, 0.068739474], [0.7159555, 0.79903615],\n [0.76412082, 0.85348046], [0.081224024, 0.82264912],\n [0.97173303, 0.24291694], [0.48957139, 0.43488795],\n [0.67382395, 0.21889746], [0.36712623, 0.67127824]],\n [[0.12054044, 0.18096751], [0.86675781, 0.54755616],\n [0.68208277, 0.15164375], [0.79991871, 0.80811197],\n [0.85256428, 0.68253738], [0.185983, 0.95642138],\n [0.48102546, 0.28009653], [0.35726011, 0.58168036]]]])\n\nshifts = [([[1, 0, 1, -2], [-2, 1, -1, 1]]), ([[2, 1, 2, -1], [-1, 2, 0, 2]])]\n\noutputs = [([[[[0.0, 0.0], [0.0, 0.0], [0.030812, 0.96236], [0.75418, 0.44058],\n [0.0, 0.0], [0.0, 0.0], [0.83851, 0.15194], [0.097513, 0.7482]],\n [[0.88572, 0.46423], [0.97408, 0.59548], [0.93283, 0.18452],\n [0.92527, 0.18284], [0.33279, 0.0008415], [0.70694, 0.23255],\n [0.75628, 0.52934], [0.27994, 0.30533]],\n [[0.57444, 0.15906], [0.39897, 0.2579], [0.10412, 0.37183],\n [0.15022, 0.038858], [0.31664, 0.59324], [0.19577, 0.42506],\n [0.0, 0.0], [0.0, 0.0]],\n [[0.81981, 0.05991], [0.98062, 0.34803], [0.50658, 0.14446],\n [0.041784, 0.53072], [0.40986, 0.42254], [0.7115, 0.59778],\n [0.0, 0.0], [0.0, 0.0]]],\n [[[0.4291, 0.068739], [0.71596, 0.79904], [0.0, 0.0], [0.0, 0.0],\n [0.28713, 0.47414], [0.46821, 0.067472], [0.0, 0.0], [0.0,\n 0.0]],\n [[0.12054, 0.18097], [0.86676, 0.54756], [0.63669, 0.69398],\n [0.88446, 0.97854], [0.97173, 0.24292], [0.48957, 0.43489],\n [0.0097347, 0.70801], [0.87891, 0.13675]],\n [[0.0, 0.0], [0.0, 0.0], [0.75688, 0.73147], [0.50312, 0.30479],\n [0.85256, 0.68254], [0.18598, 0.95642], [0.48368, 0.14591],\n [0.25397, 0.19946]],\n [[0.0, 0.0], [0.0, 0.0], [0.76412, 0.85348], [0.081224, 0.82265],\n [0.0, 0.0], [0.0, 0.0], [0.67382, 0.2189], [0.36713,\n 0.67128]]]]),\n ([[[[0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0],\n [0.0, 0.0], [0.081159, 0.8134], [0.43456, 0.30195]],\n [[0.0, 0.0], [0.0, 0.0], [0.030812, 0.96236], [0.75418, 0.44058],\n [0.0, 0.0], [0.0, 0.0], [0.83851, 0.15194], [0.097513, 0.7482]],\n [[0.88572, 0.46423], [0.97408, 0.59548], [0.93283, 0.18452],\n [0.92527, 0.18284], [0.33279, 0.0008415], [0.70694, 0.23255],\n [0.75628, 0.52934], [0.27994, 0.30533]],\n [[0.57444, 0.15906], [0.39897, 0.2579], [0.10412, 0.37183],\n [0.15022, 0.038858], [0.31664, 0.59324], [0.19577, 0.42506],\n [0.0, 0.0], [0.0, 0.0]]],\n [[[0.74329, 0.024357], [0.82179, 0.85751], [0.0, 0.0], [0.0, 0.0],\n [0.79027, 0.6064], [0.63529, 0.72172], [0.0, 0.0], [0.0, 0.0]],\n [[0.4291, 0.068739], [0.71596, 0.79904], [0.0, 0.0], [0.0, 0.0],\n [0.28713, 0.47414], [0.46821, 0.067472], [0.0, 0.0], [0.0,\n 0.0]],\n [[0.12054, 0.18097], [0.86676, 0.54756], [0.63669, 0.69398],\n [0.88446, 0.97854], [0.97173, 0.24292], [0.48957, 0.43489],\n [0.0097347, 0.70801], [0.87891, 0.13675]],\n [[0.0, 0.0], [0.0, 0.0], [0.75688, 0.73147], [0.50312, 0.30479],\n [0.85256, 0.68254], [0.18598, 0.95642], [0.48368, 0.14591],\n [0.25397, 0.19946]]]])]\n\ngrads = [\n [[[[0., 0.], [0., 0.], [1., 1.], [1., 1.], [0., 0.], [0., 0.], [1., 1.],\n [1., 1.]],\n [[1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.],\n [1., 1.]],\n [[1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [0., 0.],\n [0., 0.]],\n [[1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [0., 0.],\n [0., 0.]]],\n [[[1., 1.], [1., 1.], [0., 0.], [0., 0.], [1., 1.], [1., 1.], [0., 0.],\n [0., 0.]],\n [[1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.],\n [1., 1.]],\n [[0., 0.], [0., 0.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.],\n [1., 1.]],\n [[0., 0.], [0., 0.], [1., 1.], [1., 1.], [0., 0.], [0., 0.], [1., 1.],\n [1., 1.]]]],\n [[[[0., 0.], [0., 0.], [0., 0.], [0., 0.], [0., 0.], [0., 0.], [1., 1.],\n [1., 1.]],\n [[0., 0.], [0., 0.], [1., 1.], [1., 1.], [0., 0.], [0., 0.], [1., 1.],\n [1., 1.]],\n [[1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.],\n [1., 1.]],\n [[1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [0., 0.],\n [0., 0.]]],\n [[[1., 1.], [1., 1.], [0., 0.], [0., 0.], [1., 1.], [1., 1.], [0., 0.],\n [0., 0.]],\n [[1., 1.], [1., 1.], [0., 0.], [0., 0.], [1., 1.], [1., 1.], [0., 0.],\n [0., 0.]],\n [[1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.],\n [1., 1.]],\n [[0., 0.], [0., 0.], [1., 1.], [1., 1.], [1., 1.], [1., 1.], [1., 1.],\n [1., 1.]]]]\n]\n\n\ndef _test_tinshift_gradcheck(dtype):\n try:\n from mmcv.ops import tin_shift\n except ModuleNotFoundError:\n pytest.skip('TINShift op is not successfully compiled')\n\n if dtype == torch.half:\n pytest.skip('\"add_cpu/sub_cpu\" not implemented for Half')\n\n for shift in shifts:\n np_input = np.array(inputs)\n np_shift = np.array(shift)\n\n x = torch.tensor(\n np_input, dtype=dtype, device='cuda', requires_grad=True)\n shift = torch.tensor(np_shift, device='cuda').int()\n if torch.__version__ == 'parrots':\n gradcheck(tin_shift, (x, shift))\n else:\n gradcheck(tin_shift, (x, shift), atol=1, rtol=0.1)\n\n\ndef _test_tinshift_allclose(dtype):\n try:\n from mmcv.ops import tin_shift\n except ModuleNotFoundError:\n pytest.skip('TINShift op is not successfully compiled')\n\n for shift, output, grad in zip(shifts, outputs, grads):\n np_input = np.array(inputs)\n np_shift = np.array(shift)\n np_output = np.array(output)\n np_grad = np.array(grad)\n\n x = torch.tensor(\n np_input, dtype=dtype, device='cuda', requires_grad=True)\n shift = torch.tensor(np_shift, device='cuda').int()\n\n output = tin_shift(x, shift)\n output.backward(torch.ones_like(output))\n assert np.allclose(\n output.data.type(torch.float).cpu().numpy(), np_output, 1e-3)\n assert np.allclose(\n x.grad.data.type(torch.float).cpu().numpy(), np_grad, 1e-3)\n\n\ndef _test_tinshift_assert(dtype):\n try:\n from mmcv.ops import tin_shift\n except ModuleNotFoundError:\n pytest.skip('TINShift op is not successfully compiled')\n\n inputs = [torch.rand(2, 3, 4, 2), torch.rand(2, 3, 4, 2)]\n shifts = [torch.rand(2, 3), torch.rand(2, 5)]\n\n for x, shift in zip(inputs, shifts):\n x = x.cuda()\n shift = shift.cuda()\n\n # A ValueError should be raised if ops get inputs with wrong shapes.\n with pytest.raises(ValueError):\n tin_shift(x, shift)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\n@pytest.mark.parametrize('dtype', [torch.float, torch.double, torch.half])\ndef test_tinshift(dtype):\n _test_tinshift_allclose(dtype=dtype)\n _test_tinshift_gradcheck(dtype=dtype)\n _test_tinshift_assert(dtype=dtype)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_upfirdn2d.py", "content": "import pytest\nimport torch\n\n_USING_PARROTS = True\ntry:\n from parrots.autograd import gradcheck\nexcept ImportError:\n from torch.autograd import gradcheck, gradgradcheck\n _USING_PARROTS = False\n\n\nclass TestUpFirDn2d(object):\n \"\"\"Unit test for UpFirDn2d.\n\n Here, we just test the basic case of upsample version. More gerneal tests\n will be included in other unit test for UpFirDnUpsample and\n UpFirDnDownSample modules.\n \"\"\"\n\n @classmethod\n def setup_class(cls):\n kernel_1d = torch.tensor([1., 3., 3., 1.])\n cls.kernel = kernel_1d[:, None] * kernel_1d[None, :]\n cls.kernel = cls.kernel / cls.kernel.sum()\n cls.factor = 2\n pad = cls.kernel.shape[0] - cls.factor\n cls.pad = ((pad + 1) // 2 + cls.factor - 1, pad // 2)\n\n cls.input_tensor = torch.randn((2, 3, 4, 4), requires_grad=True)\n\n @pytest.mark.skipif(not torch.cuda.is_available(), reason='requires cuda')\n def test_upfirdn2d(self):\n from mmcv.ops import upfirdn2d\n if _USING_PARROTS:\n gradcheck(\n upfirdn2d,\n (self.input_tensor.cuda(),\n self.kernel.type_as(\n self.input_tensor).cuda(), self.factor, 1, self.pad),\n delta=1e-4,\n pt_atol=1e-3)\n else:\n gradcheck(\n upfirdn2d,\n (self.input_tensor.cuda(),\n self.kernel.type_as(\n self.input_tensor).cuda(), self.factor, 1, self.pad),\n eps=1e-4,\n atol=1e-3)\n\n gradgradcheck(\n upfirdn2d,\n (self.input_tensor.cuda(),\n self.kernel.type_as(\n self.input_tensor).cuda(), self.factor, 1, self.pad),\n eps=1e-4,\n atol=1e-3)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_ops/test_voxelization.py", "content": "import numpy as np\nimport pytest\nimport torch\n\nfrom mmcv.ops import Voxelization\n\n\ndef _get_voxel_points_indices(points, coors, voxel):\n result_form = np.equal(coors, voxel)\n return result_form[:, 0] & result_form[:, 1] & result_form[:, 2]\n\n\n@pytest.mark.parametrize('device_type', [\n 'cpu',\n pytest.param(\n 'cuda:0',\n marks=pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support'))\n])\ndef test_voxelization(device_type):\n voxel_size = [0.5, 0.5, 0.5]\n point_cloud_range = [0, -40, -3, 70.4, 40, 1]\n\n voxel_dict = np.load(\n 'tests/data/for_3d_ops/test_voxel.npy', allow_pickle=True).item()\n expected_coors = voxel_dict['coors']\n expected_voxels = voxel_dict['voxels']\n expected_num_points_per_voxel = voxel_dict['num_points_per_voxel']\n points = voxel_dict['points']\n\n points = torch.tensor(points)\n max_num_points = -1\n dynamic_voxelization = Voxelization(voxel_size, point_cloud_range,\n max_num_points)\n max_num_points = 1000\n hard_voxelization = Voxelization(voxel_size, point_cloud_range,\n max_num_points)\n\n device = torch.device(device_type)\n\n # test hard_voxelization on cpu/gpu\n points = points.contiguous().to(device)\n coors, voxels, num_points_per_voxel = hard_voxelization.forward(points)\n coors = coors.cpu().detach().numpy()\n voxels = voxels.cpu().detach().numpy()\n num_points_per_voxel = num_points_per_voxel.cpu().detach().numpy()\n assert np.all(coors == expected_coors)\n assert np.all(voxels == expected_voxels)\n assert np.all(num_points_per_voxel == expected_num_points_per_voxel)\n\n # test dynamic_voxelization on cpu/gpu\n coors = dynamic_voxelization.forward(points)\n coors = coors.cpu().detach().numpy()\n points = points.cpu().detach().numpy()\n for i in range(expected_voxels.shape[0]):\n indices = _get_voxel_points_indices(points, coors, expected_voxels[i])\n num_points_current_voxel = points[indices].shape[0]\n assert num_points_current_voxel > 0\n assert np.all(\n points[indices] == expected_coors[i][:num_points_current_voxel])\n assert num_points_current_voxel == expected_num_points_per_voxel[i]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_parallel.py", "content": "from unittest.mock import MagicMock, patch\n\nimport pytest\nimport torch\nimport torch.nn as nn\nfrom torch.nn.parallel import DataParallel, DistributedDataParallel\n\nfrom mmcv.parallel import (MODULE_WRAPPERS, MMDataParallel,\n MMDistributedDataParallel, is_module_wrapper)\nfrom mmcv.parallel._functions import Scatter, get_input_device, scatter\nfrom mmcv.parallel.distributed_deprecated import \\\n MMDistributedDataParallel as DeprecatedMMDDP\n\n\ndef mock(*args, **kwargs):\n pass\n\n\n@patch('torch.distributed._broadcast_coalesced', mock)\n@patch('torch.distributed.broadcast', mock)\n@patch('torch.nn.parallel.DistributedDataParallel._ddp_init_helper', mock)\ndef test_is_module_wrapper():\n\n class Model(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.conv = nn.Conv2d(2, 2, 1)\n\n def forward(self, x):\n return self.conv(x)\n\n # _verify_model_across_ranks is added in torch1.9.0 so we should check\n # whether _verify_model_across_ranks is the member of torch.distributed\n # before mocking\n if hasattr(torch.distributed, '_verify_model_across_ranks'):\n torch.distributed._verify_model_across_ranks = mock\n\n model = Model()\n assert not is_module_wrapper(model)\n\n dp = DataParallel(model)\n assert is_module_wrapper(dp)\n\n mmdp = MMDataParallel(model)\n assert is_module_wrapper(mmdp)\n\n ddp = DistributedDataParallel(model, process_group=MagicMock())\n assert is_module_wrapper(ddp)\n\n mmddp = MMDistributedDataParallel(model, process_group=MagicMock())\n assert is_module_wrapper(mmddp)\n\n deprecated_mmddp = DeprecatedMMDDP(model)\n assert is_module_wrapper(deprecated_mmddp)\n\n # test module wrapper registry\n @MODULE_WRAPPERS.register_module()\n class ModuleWrapper(object):\n\n def __init__(self, module):\n self.module = module\n\n def forward(self, *args, **kwargs):\n return self.module(*args, **kwargs)\n\n module_wraper = ModuleWrapper(model)\n assert is_module_wrapper(module_wraper)\n\n\ndef test_get_input_device():\n # if the device is CPU, return -1\n input = torch.zeros([1, 3, 3, 3])\n assert get_input_device(input) == -1\n inputs = [torch.zeros([1, 3, 3, 3]), torch.zeros([1, 4, 4, 4])]\n assert get_input_device(inputs) == -1\n\n # if the device is GPU, return the index of device\n if torch.cuda.is_available():\n input = torch.zeros([1, 3, 3, 3]).cuda()\n assert get_input_device(input) == 0\n inputs = [\n torch.zeros([1, 3, 3, 3]).cuda(),\n torch.zeros([1, 4, 4, 4]).cuda()\n ]\n assert get_input_device(inputs) == 0\n\n # input should be a tensor or list of tensor\n with pytest.raises(Exception):\n get_input_device(5)\n\n\ndef test_scatter():\n # if the device is CPU, just return the input\n input = torch.zeros([1, 3, 3, 3])\n output = scatter(input=input, devices=[-1])\n assert torch.allclose(input, output)\n\n inputs = [torch.zeros([1, 3, 3, 3]), torch.zeros([1, 4, 4, 4])]\n outputs = scatter(input=inputs, devices=[-1])\n for input, output in zip(inputs, outputs):\n assert torch.allclose(input, output)\n\n # if the device is GPU, copy the input from CPU to GPU\n if torch.cuda.is_available():\n input = torch.zeros([1, 3, 3, 3])\n output = scatter(input=input, devices=[0])\n assert torch.allclose(input.cuda(), output)\n\n inputs = [torch.zeros([1, 3, 3, 3]), torch.zeros([1, 4, 4, 4])]\n outputs = scatter(input=inputs, devices=[0])\n for input, output in zip(inputs, outputs):\n assert torch.allclose(input.cuda(), output)\n\n # input should be a tensor or list of tensor\n with pytest.raises(Exception):\n scatter(5, [-1])\n\n\ndef test_Scatter():\n # if the device is CPU, just return the input\n target_gpus = [-1]\n input = torch.zeros([1, 3, 3, 3])\n outputs = Scatter.forward(target_gpus, input)\n assert isinstance(outputs, tuple)\n assert torch.allclose(input, outputs[0])\n\n target_gpus = [-1]\n inputs = [torch.zeros([1, 3, 3, 3]), torch.zeros([1, 4, 4, 4])]\n outputs = Scatter.forward(target_gpus, inputs)\n assert isinstance(outputs, tuple)\n for input, output in zip(inputs, outputs):\n assert torch.allclose(input, output)\n\n # if the device is GPU, copy the input from CPU to GPU\n if torch.cuda.is_available():\n target_gpus = [0]\n input = torch.zeros([1, 3, 3, 3])\n outputs = Scatter.forward(target_gpus, input)\n assert isinstance(outputs, tuple)\n assert torch.allclose(input.cuda(), outputs[0])\n\n target_gpus = [0]\n inputs = [torch.zeros([1, 3, 3, 3]), torch.zeros([1, 4, 4, 4])]\n outputs = Scatter.forward(target_gpus, inputs)\n assert isinstance(outputs, tuple)\n for input, output in zip(inputs, outputs):\n assert torch.allclose(input.cuda(), output[0])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_basemodule.py", "content": "import tempfile\n\nimport pytest\nimport torch\nfrom torch import nn\n\nimport mmcv\nfrom mmcv.cnn.utils.weight_init import update_init_info\nfrom mmcv.runner import BaseModule, ModuleDict, ModuleList, Sequential\nfrom mmcv.utils import Registry, build_from_cfg\n\nCOMPONENTS = Registry('component')\nFOOMODELS = Registry('model')\n\n\n@COMPONENTS.register_module()\nclass FooConv1d(BaseModule):\n\n def __init__(self, init_cfg=None):\n super().__init__(init_cfg)\n self.conv1d = nn.Conv1d(4, 1, 4)\n\n def forward(self, x):\n return self.conv1d(x)\n\n\n@COMPONENTS.register_module()\nclass FooConv2d(BaseModule):\n\n def __init__(self, init_cfg=None):\n super().__init__(init_cfg)\n self.conv2d = nn.Conv2d(3, 1, 3)\n\n def forward(self, x):\n return self.conv2d(x)\n\n\n@COMPONENTS.register_module()\nclass FooLinear(BaseModule):\n\n def __init__(self, init_cfg=None):\n super().__init__(init_cfg)\n self.linear = nn.Linear(3, 4)\n\n def forward(self, x):\n return self.linear(x)\n\n\n@COMPONENTS.register_module()\nclass FooLinearConv1d(BaseModule):\n\n def __init__(self, linear=None, conv1d=None, init_cfg=None):\n super().__init__(init_cfg)\n if linear is not None:\n self.linear = build_from_cfg(linear, COMPONENTS)\n if conv1d is not None:\n self.conv1d = build_from_cfg(conv1d, COMPONENTS)\n\n def forward(self, x):\n x = self.linear(x)\n return self.conv1d(x)\n\n\n@FOOMODELS.register_module()\nclass FooModel(BaseModule):\n\n def __init__(self,\n component1=None,\n component2=None,\n component3=None,\n component4=None,\n init_cfg=None) -> None:\n super().__init__(init_cfg)\n if component1 is not None:\n self.component1 = build_from_cfg(component1, COMPONENTS)\n if component2 is not None:\n self.component2 = build_from_cfg(component2, COMPONENTS)\n if component3 is not None:\n self.component3 = build_from_cfg(component3, COMPONENTS)\n if component4 is not None:\n self.component4 = build_from_cfg(component4, COMPONENTS)\n\n # its type is not BaseModule, it can be initialized\n # with \"override\" key.\n self.reg = nn.Linear(3, 4)\n\n\ndef test_initilization_info_logger():\n # 'override' has higher priority\n\n import torch.nn as nn\n from mmcv.utils.logging import get_logger\n import os\n\n class OverloadInitConv(nn.Conv2d, BaseModule):\n\n def init_weights(self):\n for p in self.parameters():\n with torch.no_grad():\n p.fill_(1)\n\n class CheckLoggerModel(BaseModule):\n\n def __init__(self, init_cfg=None):\n super(CheckLoggerModel, self).__init__(init_cfg)\n self.conv1 = nn.Conv2d(1, 1, 1, 1)\n self.conv2 = OverloadInitConv(1, 1, 1, 1)\n self.conv3 = nn.Conv2d(1, 1, 1, 1)\n self.fc1 = nn.Linear(1, 1)\n\n init_cfg = [\n dict(\n type='Normal',\n layer='Conv2d',\n std=0.01,\n override=dict(\n type='Normal', name='conv3', std=0.01, bias_prob=0.01)),\n dict(type='Constant', layer='Linear', val=0., bias=1.)\n ]\n\n model = CheckLoggerModel(init_cfg=init_cfg)\n\n train_log = '20210720_132454.log'\n workdir = tempfile.mkdtemp()\n log_file = os.path.join(workdir, train_log)\n # create a logger\n get_logger('init_logger', log_file=log_file)\n assert not hasattr(model, '_params_init_info')\n model.init_weights()\n # assert `_params_init_info` would be deleted after `init_weights`\n assert not hasattr(model, '_params_init_info')\n # assert initialization information has been dumped\n assert os.path.exists(log_file)\n\n lines = mmcv.list_from_file(log_file)\n\n # check initialization information is right\n for i, line in enumerate(lines):\n if 'conv1.weight' in line:\n assert 'NormalInit' in lines[i + 1]\n if 'conv2.weight' in line:\n assert 'OverloadInitConv' in lines[i + 1]\n if 'fc1.weight' in line:\n assert 'ConstantInit' in lines[i + 1]\n\n # test corner case\n\n class OverloadInitConvFc(nn.Conv2d, BaseModule):\n\n def __init__(self, *args, **kwargs):\n super(OverloadInitConvFc, self).__init__(*args, **kwargs)\n self.conv1 = nn.Linear(1, 1)\n\n def init_weights(self):\n for p in self.parameters():\n with torch.no_grad():\n p.fill_(1)\n\n class CheckLoggerModel(BaseModule):\n\n def __init__(self, init_cfg=None):\n super(CheckLoggerModel, self).__init__(init_cfg)\n self.conv1 = nn.Conv2d(1, 1, 1, 1)\n self.conv2 = OverloadInitConvFc(1, 1, 1, 1)\n self.conv3 = nn.Conv2d(1, 1, 1, 1)\n self.fc1 = nn.Linear(1, 1)\n\n class TopLevelModule(BaseModule):\n\n def __init__(self, init_cfg=None, checklog_init_cfg=None):\n super(TopLevelModule, self).__init__(init_cfg)\n self.module1 = CheckLoggerModel(checklog_init_cfg)\n self.module2 = OverloadInitConvFc(1, 1, 1, 1)\n\n checklog_init_cfg = [\n dict(\n type='Normal',\n layer='Conv2d',\n std=0.01,\n override=dict(\n type='Normal', name='conv3', std=0.01, bias_prob=0.01)),\n dict(type='Constant', layer='Linear', val=0., bias=1.)\n ]\n\n top_level_init_cfg = [\n dict(\n type='Normal',\n layer='Conv2d',\n std=0.01,\n override=dict(\n type='Normal', name='module2', std=0.01, bias_prob=0.01))\n ]\n\n model = TopLevelModule(\n init_cfg=top_level_init_cfg, checklog_init_cfg=checklog_init_cfg)\n\n model.module1.init_weights()\n model.module2.init_weights()\n model.init_weights()\n model.module1.init_weights()\n model.module2.init_weights()\n\n assert not hasattr(model, '_params_init_info')\n model.init_weights()\n # assert `_params_init_info` would be deleted after `init_weights`\n assert not hasattr(model, '_params_init_info')\n # assert initialization information has been dumped\n assert os.path.exists(log_file)\n\n lines = mmcv.list_from_file(log_file)\n # check initialization information is right\n for i, line in enumerate(lines):\n if 'TopLevelModule' in line and 'init_cfg' not in line:\n # have been set init_flag\n assert 'the same' in line\n\n\ndef test_update_init_info():\n\n class DummyModel(BaseModule):\n\n def __init__(self, init_cfg=None):\n super().__init__(init_cfg)\n self.conv1 = nn.Conv2d(1, 1, 1, 1)\n self.conv3 = nn.Conv2d(1, 1, 1, 1)\n self.fc1 = nn.Linear(1, 1)\n\n model = DummyModel()\n from collections import defaultdict\n model._params_init_info = defaultdict(dict)\n for name, param in model.named_parameters():\n model._params_init_info[param]['init_info'] = 'init'\n model._params_init_info[param]['tmp_mean_value'] = param.data.mean()\n\n with torch.no_grad():\n for p in model.parameters():\n p.fill_(1)\n\n update_init_info(model, init_info='fill_1')\n\n for item in model._params_init_info.values():\n assert item['init_info'] == 'fill_1'\n assert item['tmp_mean_value'] == 1\n\n # test assert for new parameters\n model.conv1.bias = nn.Parameter(torch.ones_like(model.conv1.bias))\n with pytest.raises(AssertionError):\n update_init_info(model, init_info=' ')\n\n\ndef test_model_weight_init():\n \"\"\"\n Config\n model (FooModel, Linear: weight=1, bias=2, Conv1d: weight=3, bias=4,\n Conv2d: weight=5, bias=6)\n ├──component1 (FooConv1d)\n ├──component2 (FooConv2d)\n ├──component3 (FooLinear)\n ├──component4 (FooLinearConv1d)\n ├──linear (FooLinear)\n ├──conv1d (FooConv1d)\n ├──reg (nn.Linear)\n\n Parameters after initialization\n model (FooModel)\n ├──component1 (FooConv1d, weight=3, bias=4)\n ├──component2 (FooConv2d, weight=5, bias=6)\n ├──component3 (FooLinear, weight=1, bias=2)\n ├──component4 (FooLinearConv1d)\n ├──linear (FooLinear, weight=1, bias=2)\n ├──conv1d (FooConv1d, weight=3, bias=4)\n ├──reg (nn.Linear, weight=1, bias=2)\n \"\"\"\n model_cfg = dict(\n type='FooModel',\n init_cfg=[\n dict(type='Constant', val=1, bias=2, layer='Linear'),\n dict(type='Constant', val=3, bias=4, layer='Conv1d'),\n dict(type='Constant', val=5, bias=6, layer='Conv2d')\n ],\n component1=dict(type='FooConv1d'),\n component2=dict(type='FooConv2d'),\n component3=dict(type='FooLinear'),\n component4=dict(\n type='FooLinearConv1d',\n linear=dict(type='FooLinear'),\n conv1d=dict(type='FooConv1d')))\n\n model = build_from_cfg(model_cfg, FOOMODELS)\n model.init_weights()\n\n assert torch.equal(model.component1.conv1d.weight,\n torch.full(model.component1.conv1d.weight.shape, 3.0))\n assert torch.equal(model.component1.conv1d.bias,\n torch.full(model.component1.conv1d.bias.shape, 4.0))\n assert torch.equal(model.component2.conv2d.weight,\n torch.full(model.component2.conv2d.weight.shape, 5.0))\n assert torch.equal(model.component2.conv2d.bias,\n torch.full(model.component2.conv2d.bias.shape, 6.0))\n assert torch.equal(model.component3.linear.weight,\n torch.full(model.component3.linear.weight.shape, 1.0))\n assert torch.equal(model.component3.linear.bias,\n torch.full(model.component3.linear.bias.shape, 2.0))\n assert torch.equal(\n model.component4.linear.linear.weight,\n torch.full(model.component4.linear.linear.weight.shape, 1.0))\n assert torch.equal(\n model.component4.linear.linear.bias,\n torch.full(model.component4.linear.linear.bias.shape, 2.0))\n assert torch.equal(\n model.component4.conv1d.conv1d.weight,\n torch.full(model.component4.conv1d.conv1d.weight.shape, 3.0))\n assert torch.equal(\n model.component4.conv1d.conv1d.bias,\n torch.full(model.component4.conv1d.conv1d.bias.shape, 4.0))\n assert torch.equal(model.reg.weight, torch.full(model.reg.weight.shape,\n 1.0))\n assert torch.equal(model.reg.bias, torch.full(model.reg.bias.shape, 2.0))\n\n\ndef test_nest_components_weight_init():\n \"\"\"\n Config\n model (FooModel, Linear: weight=1, bias=2, Conv1d: weight=3, bias=4,\n Conv2d: weight=5, bias=6)\n ├──component1 (FooConv1d, Conv1d: weight=7, bias=8)\n ├──component2 (FooConv2d, Conv2d: weight=9, bias=10)\n ├──component3 (FooLinear)\n ├──component4 (FooLinearConv1d, Linear: weight=11, bias=12)\n ├──linear (FooLinear, Linear: weight=11, bias=12)\n ├──conv1d (FooConv1d)\n ├──reg (nn.Linear, weight=13, bias=14)\n\n Parameters after initialization\n model (FooModel)\n ├──component1 (FooConv1d, weight=7, bias=8)\n ├──component2 (FooConv2d, weight=9, bias=10)\n ├──component3 (FooLinear, weight=1, bias=2)\n ├──component4 (FooLinearConv1d)\n ├──linear (FooLinear, weight=1, bias=2)\n ├──conv1d (FooConv1d, weight=3, bias=4)\n ├──reg (nn.Linear, weight=13, bias=14)\n \"\"\"\n\n model_cfg = dict(\n type='FooModel',\n init_cfg=[\n dict(\n type='Constant',\n val=1,\n bias=2,\n layer='Linear',\n override=dict(type='Constant', name='reg', val=13, bias=14)),\n dict(type='Constant', val=3, bias=4, layer='Conv1d'),\n dict(type='Constant', val=5, bias=6, layer='Conv2d'),\n ],\n component1=dict(\n type='FooConv1d',\n init_cfg=dict(type='Constant', layer='Conv1d', val=7, bias=8)),\n component2=dict(\n type='FooConv2d',\n init_cfg=dict(type='Constant', layer='Conv2d', val=9, bias=10)),\n component3=dict(type='FooLinear'),\n component4=dict(\n type='FooLinearConv1d',\n linear=dict(type='FooLinear'),\n conv1d=dict(type='FooConv1d')))\n\n model = build_from_cfg(model_cfg, FOOMODELS)\n model.init_weights()\n\n assert torch.equal(model.component1.conv1d.weight,\n torch.full(model.component1.conv1d.weight.shape, 7.0))\n assert torch.equal(model.component1.conv1d.bias,\n torch.full(model.component1.conv1d.bias.shape, 8.0))\n assert torch.equal(model.component2.conv2d.weight,\n torch.full(model.component2.conv2d.weight.shape, 9.0))\n assert torch.equal(model.component2.conv2d.bias,\n torch.full(model.component2.conv2d.bias.shape, 10.0))\n assert torch.equal(model.component3.linear.weight,\n torch.full(model.component3.linear.weight.shape, 1.0))\n assert torch.equal(model.component3.linear.bias,\n torch.full(model.component3.linear.bias.shape, 2.0))\n assert torch.equal(\n model.component4.linear.linear.weight,\n torch.full(model.component4.linear.linear.weight.shape, 1.0))\n assert torch.equal(\n model.component4.linear.linear.bias,\n torch.full(model.component4.linear.linear.bias.shape, 2.0))\n assert torch.equal(\n model.component4.conv1d.conv1d.weight,\n torch.full(model.component4.conv1d.conv1d.weight.shape, 3.0))\n assert torch.equal(\n model.component4.conv1d.conv1d.bias,\n torch.full(model.component4.conv1d.conv1d.bias.shape, 4.0))\n assert torch.equal(model.reg.weight,\n torch.full(model.reg.weight.shape, 13.0))\n assert torch.equal(model.reg.bias, torch.full(model.reg.bias.shape, 14.0))\n\n\ndef test_without_layer_weight_init():\n model_cfg = dict(\n type='FooModel',\n init_cfg=[\n dict(type='Constant', val=1, bias=2, layer='Linear'),\n dict(type='Constant', val=3, bias=4, layer='Conv1d'),\n dict(type='Constant', val=5, bias=6, layer='Conv2d')\n ],\n component1=dict(\n type='FooConv1d', init_cfg=dict(type='Constant', val=7, bias=8)),\n component2=dict(type='FooConv2d'),\n component3=dict(type='FooLinear'))\n model = build_from_cfg(model_cfg, FOOMODELS)\n model.init_weights()\n\n assert torch.equal(model.component1.conv1d.weight,\n torch.full(model.component1.conv1d.weight.shape, 3.0))\n assert torch.equal(model.component1.conv1d.bias,\n torch.full(model.component1.conv1d.bias.shape, 4.0))\n\n # init_cfg in component1 does not have layer key, so it does nothing\n assert torch.equal(model.component2.conv2d.weight,\n torch.full(model.component2.conv2d.weight.shape, 5.0))\n assert torch.equal(model.component2.conv2d.bias,\n torch.full(model.component2.conv2d.bias.shape, 6.0))\n assert torch.equal(model.component3.linear.weight,\n torch.full(model.component3.linear.weight.shape, 1.0))\n assert torch.equal(model.component3.linear.bias,\n torch.full(model.component3.linear.bias.shape, 2.0))\n\n assert torch.equal(model.reg.weight, torch.full(model.reg.weight.shape,\n 1.0))\n assert torch.equal(model.reg.bias, torch.full(model.reg.bias.shape, 2.0))\n\n\ndef test_override_weight_init():\n\n # only initialize 'override'\n model_cfg = dict(\n type='FooModel',\n init_cfg=[\n dict(type='Constant', val=10, bias=20, override=dict(name='reg'))\n ],\n component1=dict(type='FooConv1d'),\n component3=dict(type='FooLinear'))\n model = build_from_cfg(model_cfg, FOOMODELS)\n model.init_weights()\n assert torch.equal(model.reg.weight,\n torch.full(model.reg.weight.shape, 10.0))\n assert torch.equal(model.reg.bias, torch.full(model.reg.bias.shape, 20.0))\n # do not initialize others\n assert not torch.equal(\n model.component1.conv1d.weight,\n torch.full(model.component1.conv1d.weight.shape, 10.0))\n assert not torch.equal(\n model.component1.conv1d.bias,\n torch.full(model.component1.conv1d.bias.shape, 20.0))\n assert not torch.equal(\n model.component3.linear.weight,\n torch.full(model.component3.linear.weight.shape, 10.0))\n assert not torch.equal(\n model.component3.linear.bias,\n torch.full(model.component3.linear.bias.shape, 20.0))\n\n # 'override' has higher priority\n model_cfg = dict(\n type='FooModel',\n init_cfg=[\n dict(\n type='Constant',\n val=1,\n bias=2,\n override=dict(name='reg', type='Constant', val=30, bias=40))\n ],\n component1=dict(type='FooConv1d'),\n component2=dict(type='FooConv2d'),\n component3=dict(type='FooLinear'))\n model = build_from_cfg(model_cfg, FOOMODELS)\n model.init_weights()\n\n assert torch.equal(model.reg.weight,\n torch.full(model.reg.weight.shape, 30.0))\n assert torch.equal(model.reg.bias, torch.full(model.reg.bias.shape, 40.0))\n\n\ndef test_sequential_model_weight_init():\n seq_model_cfg = [\n dict(\n type='FooConv1d',\n init_cfg=dict(type='Constant', layer='Conv1d', val=0., bias=1.)),\n dict(\n type='FooConv2d',\n init_cfg=dict(type='Constant', layer='Conv2d', val=2., bias=3.)),\n ]\n layers = [build_from_cfg(cfg, COMPONENTS) for cfg in seq_model_cfg]\n seq_model = Sequential(*layers)\n seq_model.init_weights()\n assert torch.equal(seq_model[0].conv1d.weight,\n torch.full(seq_model[0].conv1d.weight.shape, 0.))\n assert torch.equal(seq_model[0].conv1d.bias,\n torch.full(seq_model[0].conv1d.bias.shape, 1.))\n assert torch.equal(seq_model[1].conv2d.weight,\n torch.full(seq_model[1].conv2d.weight.shape, 2.))\n assert torch.equal(seq_model[1].conv2d.bias,\n torch.full(seq_model[1].conv2d.bias.shape, 3.))\n # inner init_cfg has higher priority\n layers = [build_from_cfg(cfg, COMPONENTS) for cfg in seq_model_cfg]\n seq_model = Sequential(\n *layers,\n init_cfg=dict(\n type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.))\n seq_model.init_weights()\n assert torch.equal(seq_model[0].conv1d.weight,\n torch.full(seq_model[0].conv1d.weight.shape, 0.))\n assert torch.equal(seq_model[0].conv1d.bias,\n torch.full(seq_model[0].conv1d.bias.shape, 1.))\n assert torch.equal(seq_model[1].conv2d.weight,\n torch.full(seq_model[1].conv2d.weight.shape, 2.))\n assert torch.equal(seq_model[1].conv2d.bias,\n torch.full(seq_model[1].conv2d.bias.shape, 3.))\n\n\ndef test_modulelist_weight_init():\n models_cfg = [\n dict(\n type='FooConv1d',\n init_cfg=dict(type='Constant', layer='Conv1d', val=0., bias=1.)),\n dict(\n type='FooConv2d',\n init_cfg=dict(type='Constant', layer='Conv2d', val=2., bias=3.)),\n ]\n layers = [build_from_cfg(cfg, COMPONENTS) for cfg in models_cfg]\n modellist = ModuleList(layers)\n modellist.init_weights()\n assert torch.equal(modellist[0].conv1d.weight,\n torch.full(modellist[0].conv1d.weight.shape, 0.))\n assert torch.equal(modellist[0].conv1d.bias,\n torch.full(modellist[0].conv1d.bias.shape, 1.))\n assert torch.equal(modellist[1].conv2d.weight,\n torch.full(modellist[1].conv2d.weight.shape, 2.))\n assert torch.equal(modellist[1].conv2d.bias,\n torch.full(modellist[1].conv2d.bias.shape, 3.))\n # inner init_cfg has higher priority\n layers = [build_from_cfg(cfg, COMPONENTS) for cfg in models_cfg]\n modellist = ModuleList(\n layers,\n init_cfg=dict(\n type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.))\n modellist.init_weights()\n assert torch.equal(modellist[0].conv1d.weight,\n torch.full(modellist[0].conv1d.weight.shape, 0.))\n assert torch.equal(modellist[0].conv1d.bias,\n torch.full(modellist[0].conv1d.bias.shape, 1.))\n assert torch.equal(modellist[1].conv2d.weight,\n torch.full(modellist[1].conv2d.weight.shape, 2.))\n assert torch.equal(modellist[1].conv2d.bias,\n torch.full(modellist[1].conv2d.bias.shape, 3.))\n\n\ndef test_moduledict_weight_init():\n models_cfg = dict(\n foo_conv_1d=dict(\n type='FooConv1d',\n init_cfg=dict(type='Constant', layer='Conv1d', val=0., bias=1.)),\n foo_conv_2d=dict(\n type='FooConv2d',\n init_cfg=dict(type='Constant', layer='Conv2d', val=2., bias=3.)),\n )\n layers = {\n name: build_from_cfg(cfg, COMPONENTS)\n for name, cfg in models_cfg.items()\n }\n modeldict = ModuleDict(layers)\n modeldict.init_weights()\n assert torch.equal(\n modeldict['foo_conv_1d'].conv1d.weight,\n torch.full(modeldict['foo_conv_1d'].conv1d.weight.shape, 0.))\n assert torch.equal(\n modeldict['foo_conv_1d'].conv1d.bias,\n torch.full(modeldict['foo_conv_1d'].conv1d.bias.shape, 1.))\n assert torch.equal(\n modeldict['foo_conv_2d'].conv2d.weight,\n torch.full(modeldict['foo_conv_2d'].conv2d.weight.shape, 2.))\n assert torch.equal(\n modeldict['foo_conv_2d'].conv2d.bias,\n torch.full(modeldict['foo_conv_2d'].conv2d.bias.shape, 3.))\n # inner init_cfg has higher priority\n layers = {\n name: build_from_cfg(cfg, COMPONENTS)\n for name, cfg in models_cfg.items()\n }\n modeldict = ModuleDict(\n layers,\n init_cfg=dict(\n type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.))\n modeldict.init_weights()\n assert torch.equal(\n modeldict['foo_conv_1d'].conv1d.weight,\n torch.full(modeldict['foo_conv_1d'].conv1d.weight.shape, 0.))\n assert torch.equal(\n modeldict['foo_conv_1d'].conv1d.bias,\n torch.full(modeldict['foo_conv_1d'].conv1d.bias.shape, 1.))\n assert torch.equal(\n modeldict['foo_conv_2d'].conv2d.weight,\n torch.full(modeldict['foo_conv_2d'].conv2d.weight.shape, 2.))\n assert torch.equal(\n modeldict['foo_conv_2d'].conv2d.bias,\n torch.full(modeldict['foo_conv_2d'].conv2d.bias.shape, 3.))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_checkpoint.py", "content": "import sys\nfrom collections import OrderedDict\nfrom tempfile import TemporaryDirectory\nfrom unittest.mock import MagicMock, patch\n\nimport pytest\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.nn.parallel import DataParallel\n\nfrom mmcv.fileio.file_client import PetrelBackend\nfrom mmcv.parallel.registry import MODULE_WRAPPERS\nfrom mmcv.runner.checkpoint import (_load_checkpoint_with_prefix,\n get_state_dict, load_checkpoint,\n load_from_local, load_from_pavi,\n save_checkpoint)\n\nsys.modules['petrel_client'] = MagicMock()\nsys.modules['petrel_client.client'] = MagicMock()\n\n\n@MODULE_WRAPPERS.register_module()\nclass DDPWrapper(object):\n\n def __init__(self, module):\n self.module = module\n\n\nclass Block(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.conv = nn.Conv2d(3, 3, 1)\n self.norm = nn.BatchNorm2d(3)\n\n\nclass Model(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.block = Block()\n self.conv = nn.Conv2d(3, 3, 1)\n\n\nclass Mockpavimodel(object):\n\n def __init__(self, name='fakename'):\n self.name = name\n\n def download(self, file):\n pass\n\n\ndef assert_tensor_equal(tensor_a, tensor_b):\n assert tensor_a.eq(tensor_b).all()\n\n\ndef test_get_state_dict():\n if torch.__version__ == 'parrots':\n state_dict_keys = set([\n 'block.conv.weight', 'block.conv.bias', 'block.norm.weight',\n 'block.norm.bias', 'block.norm.running_mean',\n 'block.norm.running_var', 'conv.weight', 'conv.bias'\n ])\n else:\n state_dict_keys = set([\n 'block.conv.weight', 'block.conv.bias', 'block.norm.weight',\n 'block.norm.bias', 'block.norm.running_mean',\n 'block.norm.running_var', 'block.norm.num_batches_tracked',\n 'conv.weight', 'conv.bias'\n ])\n\n model = Model()\n state_dict = get_state_dict(model)\n assert isinstance(state_dict, OrderedDict)\n assert set(state_dict.keys()) == state_dict_keys\n\n assert_tensor_equal(state_dict['block.conv.weight'],\n model.block.conv.weight)\n assert_tensor_equal(state_dict['block.conv.bias'], model.block.conv.bias)\n assert_tensor_equal(state_dict['block.norm.weight'],\n model.block.norm.weight)\n assert_tensor_equal(state_dict['block.norm.bias'], model.block.norm.bias)\n assert_tensor_equal(state_dict['block.norm.running_mean'],\n model.block.norm.running_mean)\n assert_tensor_equal(state_dict['block.norm.running_var'],\n model.block.norm.running_var)\n if torch.__version__ != 'parrots':\n assert_tensor_equal(state_dict['block.norm.num_batches_tracked'],\n model.block.norm.num_batches_tracked)\n assert_tensor_equal(state_dict['conv.weight'], model.conv.weight)\n assert_tensor_equal(state_dict['conv.bias'], model.conv.bias)\n\n wrapped_model = DDPWrapper(model)\n state_dict = get_state_dict(wrapped_model)\n assert isinstance(state_dict, OrderedDict)\n assert set(state_dict.keys()) == state_dict_keys\n assert_tensor_equal(state_dict['block.conv.weight'],\n wrapped_model.module.block.conv.weight)\n assert_tensor_equal(state_dict['block.conv.bias'],\n wrapped_model.module.block.conv.bias)\n assert_tensor_equal(state_dict['block.norm.weight'],\n wrapped_model.module.block.norm.weight)\n assert_tensor_equal(state_dict['block.norm.bias'],\n wrapped_model.module.block.norm.bias)\n assert_tensor_equal(state_dict['block.norm.running_mean'],\n wrapped_model.module.block.norm.running_mean)\n assert_tensor_equal(state_dict['block.norm.running_var'],\n wrapped_model.module.block.norm.running_var)\n if torch.__version__ != 'parrots':\n assert_tensor_equal(\n state_dict['block.norm.num_batches_tracked'],\n wrapped_model.module.block.norm.num_batches_tracked)\n assert_tensor_equal(state_dict['conv.weight'],\n wrapped_model.module.conv.weight)\n assert_tensor_equal(state_dict['conv.bias'],\n wrapped_model.module.conv.bias)\n\n # wrapped inner module\n for name, module in wrapped_model.module._modules.items():\n module = DataParallel(module)\n wrapped_model.module._modules[name] = module\n state_dict = get_state_dict(wrapped_model)\n assert isinstance(state_dict, OrderedDict)\n assert set(state_dict.keys()) == state_dict_keys\n assert_tensor_equal(state_dict['block.conv.weight'],\n wrapped_model.module.block.module.conv.weight)\n assert_tensor_equal(state_dict['block.conv.bias'],\n wrapped_model.module.block.module.conv.bias)\n assert_tensor_equal(state_dict['block.norm.weight'],\n wrapped_model.module.block.module.norm.weight)\n assert_tensor_equal(state_dict['block.norm.bias'],\n wrapped_model.module.block.module.norm.bias)\n assert_tensor_equal(state_dict['block.norm.running_mean'],\n wrapped_model.module.block.module.norm.running_mean)\n assert_tensor_equal(state_dict['block.norm.running_var'],\n wrapped_model.module.block.module.norm.running_var)\n if torch.__version__ != 'parrots':\n assert_tensor_equal(\n state_dict['block.norm.num_batches_tracked'],\n wrapped_model.module.block.module.norm.num_batches_tracked)\n assert_tensor_equal(state_dict['conv.weight'],\n wrapped_model.module.conv.module.weight)\n assert_tensor_equal(state_dict['conv.bias'],\n wrapped_model.module.conv.module.bias)\n\n\ndef test_load_pavimodel_dist():\n\n sys.modules['pavi'] = MagicMock()\n sys.modules['pavi.modelcloud'] = MagicMock()\n pavimodel = Mockpavimodel()\n import pavi\n pavi.modelcloud.get = MagicMock(return_value=pavimodel)\n with pytest.raises(AssertionError):\n # test pavi prefix\n _ = load_from_pavi('MyPaviFolder/checkpoint.pth')\n\n with pytest.raises(FileNotFoundError):\n # there is not such checkpoint for us to load\n _ = load_from_pavi('pavi://checkpoint.pth')\n\n\ndef test_load_checkpoint_with_prefix():\n\n class FooModule(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.linear = nn.Linear(1, 2)\n self.conv2d = nn.Conv2d(3, 1, 3)\n self.conv2d_2 = nn.Conv2d(3, 2, 3)\n\n model = FooModule()\n nn.init.constant_(model.linear.weight, 1)\n nn.init.constant_(model.linear.bias, 2)\n nn.init.constant_(model.conv2d.weight, 3)\n nn.init.constant_(model.conv2d.bias, 4)\n nn.init.constant_(model.conv2d_2.weight, 5)\n nn.init.constant_(model.conv2d_2.bias, 6)\n\n with TemporaryDirectory():\n torch.save(model.state_dict(), 'model.pth')\n prefix = 'conv2d'\n state_dict = _load_checkpoint_with_prefix(prefix, 'model.pth')\n assert torch.equal(model.conv2d.state_dict()['weight'],\n state_dict['weight'])\n assert torch.equal(model.conv2d.state_dict()['bias'],\n state_dict['bias'])\n\n # test whether prefix is in pretrained model\n with pytest.raises(AssertionError):\n prefix = 'back'\n _load_checkpoint_with_prefix(prefix, 'model.pth')\n\n\ndef test_load_checkpoint():\n import os\n\n import re\n import tempfile\n\n class PrefixModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.backbone = Model()\n\n pmodel = PrefixModel()\n model = Model()\n checkpoint_path = os.path.join(tempfile.gettempdir(), 'checkpoint.pth')\n\n # add prefix\n torch.save(model.state_dict(), checkpoint_path)\n state_dict = load_checkpoint(\n pmodel, checkpoint_path, revise_keys=[(r'^', 'backbone.')])\n for key in pmodel.backbone.state_dict().keys():\n assert torch.equal(pmodel.backbone.state_dict()[key], state_dict[key])\n # strip prefix\n torch.save(pmodel.state_dict(), checkpoint_path)\n state_dict = load_checkpoint(\n model, checkpoint_path, revise_keys=[(r'^backbone\\.', '')])\n\n for key in state_dict.keys():\n key_stripped = re.sub(r'^backbone\\.', '', key)\n assert torch.equal(model.state_dict()[key_stripped], state_dict[key])\n os.remove(checkpoint_path)\n\n\ndef test_load_checkpoint_metadata():\n import os\n\n import tempfile\n\n from mmcv.runner import load_checkpoint, save_checkpoint\n\n class ModelV1(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.block = Block()\n self.conv1 = nn.Conv2d(3, 3, 1)\n self.conv2 = nn.Conv2d(3, 3, 1)\n nn.init.normal_(self.conv1.weight)\n nn.init.normal_(self.conv2.weight)\n\n class ModelV2(nn.Module):\n _version = 2\n\n def __init__(self):\n super().__init__()\n self.block = Block()\n self.conv0 = nn.Conv2d(3, 3, 1)\n self.conv1 = nn.Conv2d(3, 3, 1)\n nn.init.normal_(self.conv0.weight)\n nn.init.normal_(self.conv1.weight)\n\n def _load_from_state_dict(self, state_dict, prefix, local_metadata,\n *args, **kwargs):\n \"\"\"load checkpoints.\"\"\"\n\n # Names of some parameters in has been changed.\n version = local_metadata.get('version', None)\n if version is None or version < 2:\n state_dict_keys = list(state_dict.keys())\n convert_map = {'conv1': 'conv0', 'conv2': 'conv1'}\n for k in state_dict_keys:\n for ori_str, new_str in convert_map.items():\n if k.startswith(prefix + ori_str):\n new_key = k.replace(ori_str, new_str)\n state_dict[new_key] = state_dict[k]\n del state_dict[k]\n\n super()._load_from_state_dict(state_dict, prefix, local_metadata,\n *args, **kwargs)\n\n model_v1 = ModelV1()\n model_v1_conv0_weight = model_v1.conv1.weight.detach()\n model_v1_conv1_weight = model_v1.conv2.weight.detach()\n model_v2 = ModelV2()\n model_v2_conv0_weight = model_v2.conv0.weight.detach()\n model_v2_conv1_weight = model_v2.conv1.weight.detach()\n ckpt_v1_path = os.path.join(tempfile.gettempdir(), 'checkpoint_v1.pth')\n ckpt_v2_path = os.path.join(tempfile.gettempdir(), 'checkpoint_v2.pth')\n\n # Save checkpoint\n save_checkpoint(model_v1, ckpt_v1_path)\n save_checkpoint(model_v2, ckpt_v2_path)\n\n # test load v1 model\n load_checkpoint(model_v2, ckpt_v1_path)\n assert torch.allclose(model_v2.conv0.weight, model_v1_conv0_weight)\n assert torch.allclose(model_v2.conv1.weight, model_v1_conv1_weight)\n\n # test load v2 model\n load_checkpoint(model_v2, ckpt_v2_path)\n assert torch.allclose(model_v2.conv0.weight, model_v2_conv0_weight)\n assert torch.allclose(model_v2.conv1.weight, model_v2_conv1_weight)\n\n\ndef test_load_classes_name():\n import os\n\n import tempfile\n\n from mmcv.runner import load_checkpoint, save_checkpoint\n checkpoint_path = os.path.join(tempfile.gettempdir(), 'checkpoint.pth')\n model = Model()\n save_checkpoint(model, checkpoint_path)\n checkpoint = load_checkpoint(model, checkpoint_path)\n assert 'meta' in checkpoint and 'CLASSES' not in checkpoint['meta']\n\n model.CLASSES = ('class1', 'class2')\n save_checkpoint(model, checkpoint_path)\n checkpoint = load_checkpoint(model, checkpoint_path)\n assert 'meta' in checkpoint and 'CLASSES' in checkpoint['meta']\n assert checkpoint['meta']['CLASSES'] == ('class1', 'class2')\n\n model = Model()\n wrapped_model = DDPWrapper(model)\n save_checkpoint(wrapped_model, checkpoint_path)\n checkpoint = load_checkpoint(wrapped_model, checkpoint_path)\n assert 'meta' in checkpoint and 'CLASSES' not in checkpoint['meta']\n\n wrapped_model.module.CLASSES = ('class1', 'class2')\n save_checkpoint(wrapped_model, checkpoint_path)\n checkpoint = load_checkpoint(wrapped_model, checkpoint_path)\n assert 'meta' in checkpoint and 'CLASSES' in checkpoint['meta']\n assert checkpoint['meta']['CLASSES'] == ('class1', 'class2')\n\n # remove the temp file\n os.remove(checkpoint_path)\n\n\ndef test_checkpoint_loader():\n import os\n\n import tempfile\n\n from mmcv.runner import CheckpointLoader, _load_checkpoint, save_checkpoint\n checkpoint_path = os.path.join(tempfile.gettempdir(), 'checkpoint.pth')\n model = Model()\n save_checkpoint(model, checkpoint_path)\n checkpoint = _load_checkpoint(checkpoint_path)\n assert 'meta' in checkpoint and 'CLASSES' not in checkpoint['meta']\n # remove the temp file\n os.remove(checkpoint_path)\n\n filenames = [\n 'http://xx.xx/xx.pth', 'https://xx.xx/xx.pth',\n 'modelzoo://xx.xx/xx.pth', 'torchvision://xx.xx/xx.pth',\n 'open-mmlab://xx.xx/xx.pth', 'openmmlab://xx.xx/xx.pth',\n 'mmcls://xx.xx/xx.pth', 'pavi://xx.xx/xx.pth', 's3://xx.xx/xx.pth',\n 'ss3://xx.xx/xx.pth', ' s3://xx.xx/xx.pth',\n 'open-mmlab:s3://xx.xx/xx.pth', 'openmmlab:s3://xx.xx/xx.pth',\n 'openmmlabs3://xx.xx/xx.pth', ':s3://xx.xx/xx.path'\n ]\n fn_names = [\n 'load_from_http', 'load_from_http', 'load_from_torchvision',\n 'load_from_torchvision', 'load_from_openmmlab', 'load_from_openmmlab',\n 'load_from_mmcls', 'load_from_pavi', 'load_from_ceph',\n 'load_from_local', 'load_from_local', 'load_from_ceph',\n 'load_from_ceph', 'load_from_local', 'load_from_local'\n ]\n\n for filename, fn_name in zip(filenames, fn_names):\n loader = CheckpointLoader._get_checkpoint_loader(filename)\n assert loader.__name__ == fn_name\n\n @CheckpointLoader.register_scheme(prefixes='ftp://')\n def load_from_ftp(filename, map_location):\n return dict(filename=filename)\n\n # test register_loader\n filename = 'ftp://xx.xx/xx.pth'\n loader = CheckpointLoader._get_checkpoint_loader(filename)\n assert loader.__name__ == 'load_from_ftp'\n\n def load_from_ftp1(filename, map_location):\n return dict(filename=filename)\n\n # test duplicate registered error\n with pytest.raises(KeyError):\n CheckpointLoader.register_scheme('ftp://', load_from_ftp1)\n\n # test force param\n CheckpointLoader.register_scheme('ftp://', load_from_ftp1, force=True)\n checkpoint = CheckpointLoader.load_checkpoint(filename)\n assert checkpoint['filename'] == filename\n\n # test print function name\n loader = CheckpointLoader._get_checkpoint_loader(filename)\n assert loader.__name__ == 'load_from_ftp1'\n\n # test sort\n @CheckpointLoader.register_scheme(prefixes='a/b')\n def load_from_ab(filename, map_location):\n return dict(filename=filename)\n\n @CheckpointLoader.register_scheme(prefixes='a/b/c')\n def load_from_abc(filename, map_location):\n return dict(filename=filename)\n\n filename = 'a/b/c/d'\n loader = CheckpointLoader._get_checkpoint_loader(filename)\n assert loader.__name__ == 'load_from_abc'\n\n\ndef test_save_checkpoint(tmp_path):\n model = Model()\n optimizer = optim.SGD(model.parameters(), lr=0.1, momentum=0.9)\n # meta is not a dict\n with pytest.raises(TypeError):\n save_checkpoint(model, '/path/of/your/filename', meta='invalid type')\n\n # 1. save to disk\n filename = str(tmp_path / 'checkpoint1.pth')\n save_checkpoint(model, filename)\n\n filename = str(tmp_path / 'checkpoint2.pth')\n save_checkpoint(model, filename, optimizer)\n\n filename = str(tmp_path / 'checkpoint3.pth')\n save_checkpoint(model, filename, meta={'test': 'test'})\n\n filename = str(tmp_path / 'checkpoint4.pth')\n save_checkpoint(model, filename, file_client_args={'backend': 'disk'})\n\n # 2. save to petrel oss\n with patch.object(PetrelBackend, 'put') as mock_method:\n filename = 's3://path/of/your/checkpoint1.pth'\n save_checkpoint(model, filename)\n mock_method.assert_called()\n\n with patch.object(PetrelBackend, 'put') as mock_method:\n filename = 's3://path//of/your/checkpoint2.pth'\n save_checkpoint(\n model, filename, file_client_args={'backend': 'petrel'})\n mock_method.assert_called()\n\n\ndef test_load_from_local():\n import os\n home_path = os.path.expanduser('~')\n checkpoint_path = os.path.join(\n home_path, 'dummy_checkpoint_used_to_test_load_from_local.pth')\n model = Model()\n save_checkpoint(model, checkpoint_path)\n checkpoint = load_from_local(\n '~/dummy_checkpoint_used_to_test_load_from_local.pth',\n map_location=None)\n assert_tensor_equal(checkpoint['state_dict']['block.conv.weight'],\n model.block.conv.weight)\n os.remove(checkpoint_path)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_dist_utils.py", "content": "import os\nfrom unittest.mock import patch\n\nimport pytest\n\nfrom mmcv.runner import init_dist\n\n\n@patch('torch.cuda.device_count', return_value=1)\n@patch('torch.cuda.set_device')\n@patch('torch.distributed.init_process_group')\n@patch('subprocess.getoutput', return_value='127.0.0.1')\ndef test_init_dist(mock_getoutput, mock_dist_init, mock_set_device,\n mock_device_count):\n with pytest.raises(ValueError):\n # launcher must be one of {'pytorch', 'mpi', 'slurm'}\n init_dist('invaliad_launcher')\n\n # test initialize with slurm launcher\n os.environ['SLURM_PROCID'] = '0'\n os.environ['SLURM_NTASKS'] = '1'\n os.environ['SLURM_NODELIST'] = '[0]' # haven't check the correct form\n\n init_dist('slurm')\n # no port is specified, use default port 29500\n assert os.environ['MASTER_PORT'] == '29500'\n assert os.environ['MASTER_ADDR'] == '127.0.0.1'\n assert os.environ['WORLD_SIZE'] == '1'\n assert os.environ['RANK'] == '0'\n mock_set_device.assert_called_with(0)\n mock_getoutput.assert_called_with('scontrol show hostname [0] | head -n1')\n mock_dist_init.assert_called_with(backend='nccl')\n\n init_dist('slurm', port=29505)\n # port is specified with argument 'port'\n assert os.environ['MASTER_PORT'] == '29505'\n assert os.environ['MASTER_ADDR'] == '127.0.0.1'\n assert os.environ['WORLD_SIZE'] == '1'\n assert os.environ['RANK'] == '0'\n mock_set_device.assert_called_with(0)\n mock_getoutput.assert_called_with('scontrol show hostname [0] | head -n1')\n mock_dist_init.assert_called_with(backend='nccl')\n\n init_dist('slurm')\n # port is specified by environment variable 'MASTER_PORT'\n assert os.environ['MASTER_PORT'] == '29505'\n assert os.environ['MASTER_ADDR'] == '127.0.0.1'\n assert os.environ['WORLD_SIZE'] == '1'\n assert os.environ['RANK'] == '0'\n mock_set_device.assert_called_with(0)\n mock_getoutput.assert_called_with('scontrol show hostname [0] | head -n1')\n mock_dist_init.assert_called_with(backend='nccl')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_eval_hook.py", "content": "import json\nimport os.path as osp\nimport sys\nimport tempfile\nimport unittest.mock as mock\nfrom collections import OrderedDict\nfrom unittest.mock import MagicMock, patch\n\nimport pytest\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.data import DataLoader, Dataset\n\nfrom mmcv.fileio.file_client import PetrelBackend\nfrom mmcv.runner import DistEvalHook as BaseDistEvalHook\nfrom mmcv.runner import EpochBasedRunner\nfrom mmcv.runner import EvalHook as BaseEvalHook\nfrom mmcv.runner import IterBasedRunner\nfrom mmcv.utils import get_logger, scandir\n\nsys.modules['petrel_client'] = MagicMock()\nsys.modules['petrel_client.client'] = MagicMock()\n\n\nclass ExampleDataset(Dataset):\n\n def __init__(self):\n self.index = 0\n self.eval_result = [1, 4, 3, 7, 2, -3, 4, 6]\n\n def __getitem__(self, idx):\n results = dict(x=torch.tensor([1]))\n return results\n\n def __len__(self):\n return 1\n\n @mock.create_autospec\n def evaluate(self, results, logger=None):\n pass\n\n\nclass EvalDataset(ExampleDataset):\n\n def evaluate(self, results, logger=None):\n acc = self.eval_result[self.index]\n output = OrderedDict(\n acc=acc, index=self.index, score=acc, loss_top=acc)\n self.index += 1\n return output\n\n\nclass Model(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.param = nn.Parameter(torch.tensor([1.0]))\n\n def forward(self, x, **kwargs):\n return self.param * x\n\n def train_step(self, data_batch, optimizer, **kwargs):\n return {'loss': torch.sum(self(data_batch['x']))}\n\n def val_step(self, data_batch, optimizer, **kwargs):\n return {'loss': torch.sum(self(data_batch['x']))}\n\n\ndef _build_epoch_runner():\n\n model = Model()\n tmp_dir = tempfile.mkdtemp()\n\n runner = EpochBasedRunner(\n model=model, work_dir=tmp_dir, logger=get_logger('demo'))\n return runner\n\n\ndef _build_iter_runner():\n\n model = Model()\n tmp_dir = tempfile.mkdtemp()\n\n runner = IterBasedRunner(\n model=model, work_dir=tmp_dir, logger=get_logger('demo'))\n return runner\n\n\nclass EvalHook(BaseEvalHook):\n\n _default_greater_keys = ['acc', 'top']\n _default_less_keys = ['loss', 'loss_top']\n\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n\n\nclass DistEvalHook(BaseDistEvalHook):\n\n greater_keys = ['acc', 'top']\n less_keys = ['loss', 'loss_top']\n\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n\n\ndef test_eval_hook():\n with pytest.raises(AssertionError):\n # `save_best` should be a str\n test_dataset = Model()\n data_loader = DataLoader(test_dataset)\n EvalHook(data_loader, save_best=True)\n\n with pytest.raises(TypeError):\n # dataloader must be a pytorch DataLoader\n test_dataset = Model()\n data_loader = [DataLoader(test_dataset)]\n EvalHook(data_loader)\n\n with pytest.raises(ValueError):\n # key_indicator must be valid when rule_map is None\n test_dataset = ExampleDataset()\n data_loader = DataLoader(test_dataset)\n EvalHook(data_loader, save_best='unsupport')\n\n with pytest.raises(KeyError):\n # rule must be in keys of rule_map\n test_dataset = ExampleDataset()\n data_loader = DataLoader(test_dataset)\n EvalHook(data_loader, save_best='auto', rule='unsupport')\n\n # if eval_res is an empty dict, print a warning information\n with pytest.warns(UserWarning) as record_warnings:\n\n class _EvalDataset(ExampleDataset):\n\n def evaluate(self, results, logger=None):\n return {}\n\n test_dataset = _EvalDataset()\n data_loader = DataLoader(test_dataset)\n eval_hook = EvalHook(data_loader, save_best='auto')\n runner = _build_epoch_runner()\n runner.register_hook(eval_hook)\n runner.run([data_loader], [('train', 1)], 1)\n # Since there will be many warnings thrown, we just need to check if the\n # expected exceptions are thrown\n expected_message = ('Since `eval_res` is an empty dict, the behavior to '\n 'save the best checkpoint will be skipped in this '\n 'evaluation.')\n for warning in record_warnings:\n if str(warning.message) == expected_message:\n break\n else:\n assert False\n\n test_dataset = ExampleDataset()\n loader = DataLoader(test_dataset)\n model = Model()\n data_loader = DataLoader(test_dataset)\n eval_hook = EvalHook(data_loader, save_best=None)\n\n with tempfile.TemporaryDirectory() as tmpdir:\n\n # total_epochs = 1\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 1)\n test_dataset.evaluate.assert_called_with(\n test_dataset, [torch.tensor([1])], logger=runner.logger)\n assert runner.meta is None or 'best_score' not in runner.meta[\n 'hook_msgs']\n assert runner.meta is None or 'best_ckpt' not in runner.meta[\n 'hook_msgs']\n\n # when `save_best` is set to 'auto', first metric will be used.\n loader = DataLoader(EvalDataset())\n model = Model()\n data_loader = DataLoader(EvalDataset())\n eval_hook = EvalHook(data_loader, interval=1, save_best='auto')\n\n with tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 8)\n\n ckpt_path = osp.join(tmpdir, 'best_acc_epoch_4.pth')\n\n assert runner.meta['hook_msgs']['best_ckpt'] == ckpt_path\n assert osp.exists(ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == 7\n\n # total_epochs = 8, return the best acc and corresponding epoch\n loader = DataLoader(EvalDataset())\n model = Model()\n data_loader = DataLoader(EvalDataset())\n eval_hook = EvalHook(data_loader, interval=1, save_best='acc')\n\n with tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 8)\n\n ckpt_path = osp.join(tmpdir, 'best_acc_epoch_4.pth')\n\n assert runner.meta['hook_msgs']['best_ckpt'] == ckpt_path\n assert osp.exists(ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == 7\n\n # total_epochs = 8, return the best loss_top and corresponding epoch\n loader = DataLoader(EvalDataset())\n model = Model()\n data_loader = DataLoader(EvalDataset())\n eval_hook = EvalHook(data_loader, interval=1, save_best='loss_top')\n\n with tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 8)\n\n ckpt_path = osp.join(tmpdir, 'best_loss_top_epoch_6.pth')\n\n assert runner.meta['hook_msgs']['best_ckpt'] == ckpt_path\n assert osp.exists(ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == -3\n\n # total_epochs = 8, return the best score and corresponding epoch\n data_loader = DataLoader(EvalDataset())\n eval_hook = EvalHook(\n data_loader, interval=1, save_best='score', rule='greater')\n with tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 8)\n\n ckpt_path = osp.join(tmpdir, 'best_score_epoch_4.pth')\n\n assert runner.meta['hook_msgs']['best_ckpt'] == ckpt_path\n assert osp.exists(ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == 7\n\n # total_epochs = 8, return the best score using less compare func\n # and indicate corresponding epoch\n data_loader = DataLoader(EvalDataset())\n eval_hook = EvalHook(data_loader, save_best='acc', rule='less')\n with tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 8)\n\n ckpt_path = osp.join(tmpdir, 'best_acc_epoch_6.pth')\n\n assert runner.meta['hook_msgs']['best_ckpt'] == ckpt_path\n assert osp.exists(ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == -3\n\n # Test the EvalHook when resume happened\n data_loader = DataLoader(EvalDataset())\n eval_hook = EvalHook(data_loader, save_best='acc')\n with tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 2)\n\n old_ckpt_path = osp.join(tmpdir, 'best_acc_epoch_2.pth')\n\n assert runner.meta['hook_msgs']['best_ckpt'] == old_ckpt_path\n assert osp.exists(old_ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == 4\n\n resume_from = old_ckpt_path\n loader = DataLoader(ExampleDataset())\n eval_hook = EvalHook(data_loader, save_best='acc')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n\n runner.resume(resume_from)\n assert runner.meta['hook_msgs']['best_ckpt'] == old_ckpt_path\n assert osp.exists(old_ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == 4\n\n runner.run([loader], [('train', 1)], 8)\n\n ckpt_path = osp.join(tmpdir, 'best_acc_epoch_4.pth')\n\n assert runner.meta['hook_msgs']['best_ckpt'] == ckpt_path\n assert osp.exists(ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == 7\n assert not osp.exists(old_ckpt_path)\n\n # test EvalHook with customer test_fn and greater/less keys\n loader = DataLoader(EvalDataset())\n model = Model()\n data_loader = DataLoader(EvalDataset())\n\n eval_hook = EvalHook(\n data_loader,\n save_best='acc',\n test_fn=mock.MagicMock(return_value={}),\n greater_keys=[],\n less_keys=['acc'])\n\n with tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 8)\n\n ckpt_path = osp.join(tmpdir, 'best_acc_epoch_6.pth')\n\n assert runner.meta['hook_msgs']['best_ckpt'] == ckpt_path\n assert osp.exists(ckpt_path)\n assert runner.meta['hook_msgs']['best_score'] == -3\n\n # test EvalHook with specified `out_dir`\n loader = DataLoader(EvalDataset())\n model = Model()\n data_loader = DataLoader(EvalDataset())\n out_dir = 's3://user/data'\n eval_hook = EvalHook(\n data_loader, interval=1, save_best='auto', out_dir=out_dir)\n\n with patch.object(PetrelBackend, 'put') as mock_put, \\\n patch.object(PetrelBackend, 'remove') as mock_remove, \\\n patch.object(PetrelBackend, 'isfile') as mock_isfile, \\\n tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_eval')\n runner = EpochBasedRunner(model=model, work_dir=tmpdir, logger=logger)\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_hook(eval_hook)\n runner.run([loader], [('train', 1)], 8)\n\n basename = osp.basename(runner.work_dir.rstrip(osp.sep))\n ckpt_path = f'{out_dir}/{basename}/best_acc_epoch_4.pth'\n\n assert runner.meta['hook_msgs']['best_ckpt'] == ckpt_path\n assert runner.meta['hook_msgs']['best_score'] == 7\n\n assert mock_put.call_count == 3\n assert mock_remove.call_count == 2\n assert mock_isfile.call_count == 2\n\n\n@patch('mmcv.engine.single_gpu_test', MagicMock)\n@patch('mmcv.engine.multi_gpu_test', MagicMock)\n@pytest.mark.parametrize('EvalHookParam', [EvalHook, DistEvalHook])\n@pytest.mark.parametrize('_build_demo_runner,by_epoch',\n [(_build_epoch_runner, True),\n (_build_iter_runner, False)])\ndef test_start_param(EvalHookParam, _build_demo_runner, by_epoch):\n # create dummy data\n dataloader = DataLoader(EvalDataset())\n\n # 0.1. dataloader is not a DataLoader object\n with pytest.raises(TypeError):\n EvalHookParam(dataloader=MagicMock(), interval=-1)\n\n # 0.2. negative interval\n with pytest.raises(ValueError):\n EvalHookParam(dataloader, interval=-1)\n\n # 0.3. negative start\n with pytest.raises(ValueError):\n EvalHookParam(dataloader, start=-1)\n\n # 1. start=None, interval=1: perform evaluation after each epoch.\n runner = _build_demo_runner()\n evalhook = EvalHookParam(dataloader, interval=1, by_epoch=by_epoch)\n evalhook.evaluate = MagicMock()\n runner.register_hook(evalhook)\n runner.run([dataloader], [('train', 1)], 2)\n assert evalhook.evaluate.call_count == 2 # after epoch 1 & 2\n\n # 2. start=1, interval=1: perform evaluation after each epoch.\n runner = _build_demo_runner()\n evalhook = EvalHookParam(\n dataloader, start=1, interval=1, by_epoch=by_epoch)\n evalhook.evaluate = MagicMock()\n runner.register_hook(evalhook)\n runner.run([dataloader], [('train', 1)], 2)\n assert evalhook.evaluate.call_count == 2 # after epoch 1 & 2\n\n # 3. start=None, interval=2: perform evaluation after epoch 2, 4, 6, etc\n runner = _build_demo_runner()\n evalhook = EvalHookParam(dataloader, interval=2, by_epoch=by_epoch)\n evalhook.evaluate = MagicMock()\n runner.register_hook(evalhook)\n runner.run([dataloader], [('train', 1)], 2)\n assert evalhook.evaluate.call_count == 1 # after epoch 2\n\n # 4. start=1, interval=2: perform evaluation after epoch 1, 3, 5, etc\n runner = _build_demo_runner()\n evalhook = EvalHookParam(\n dataloader, start=1, interval=2, by_epoch=by_epoch)\n evalhook.evaluate = MagicMock()\n runner.register_hook(evalhook)\n runner.run([dataloader], [('train', 1)], 3)\n assert evalhook.evaluate.call_count == 2 # after epoch 1 & 3\n\n # 5. start=0, interval=1: perform evaluation after each epoch and\n # before epoch 1.\n runner = _build_demo_runner()\n evalhook = EvalHookParam(dataloader, start=0, by_epoch=by_epoch)\n evalhook.evaluate = MagicMock()\n runner.register_hook(evalhook)\n runner.run([dataloader], [('train', 1)], 2)\n assert evalhook.evaluate.call_count == 3 # before epoch1 and after e1 & e2\n\n # 6. resuming from epoch i, start = x (x<=i), interval =1: perform\n # evaluation after each epoch and before the first epoch.\n runner = _build_demo_runner()\n evalhook = EvalHookParam(dataloader, start=1, by_epoch=by_epoch)\n evalhook.evaluate = MagicMock()\n runner.register_hook(evalhook)\n if by_epoch:\n runner._epoch = 2\n else:\n runner._iter = 2\n runner.run([dataloader], [('train', 1)], 3)\n assert evalhook.evaluate.call_count == 2 # before & after epoch 3\n\n # 7. resuming from epoch i, start = i+1/None, interval =1: perform\n # evaluation after each epoch.\n runner = _build_demo_runner()\n evalhook = EvalHookParam(dataloader, start=2, by_epoch=by_epoch)\n evalhook.evaluate = MagicMock()\n runner.register_hook(evalhook)\n if by_epoch:\n runner._epoch = 1\n else:\n runner._iter = 1\n runner.run([dataloader], [('train', 1)], 3)\n assert evalhook.evaluate.call_count == 2 # after epoch 2 & 3\n\n\n@pytest.mark.parametrize('runner,by_epoch,eval_hook_priority',\n [(EpochBasedRunner, True, 'NORMAL'),\n (EpochBasedRunner, True, 'LOW'),\n (IterBasedRunner, False, 'LOW')])\ndef test_logger(runner, by_epoch, eval_hook_priority):\n loader = DataLoader(EvalDataset())\n model = Model()\n data_loader = DataLoader(EvalDataset())\n eval_hook = EvalHook(\n data_loader, interval=1, by_epoch=by_epoch, save_best='acc')\n\n with tempfile.TemporaryDirectory() as tmpdir:\n logger = get_logger('test_logger')\n optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)\n runner = EpochBasedRunner(\n model=model, optimizer=optimizer, work_dir=tmpdir, logger=logger)\n runner.register_logger_hooks(\n dict(\n interval=1,\n hooks=[dict(type='TextLoggerHook', by_epoch=by_epoch)]))\n runner.register_timer_hook(dict(type='IterTimerHook'))\n runner.register_hook(eval_hook, priority=eval_hook_priority)\n runner.run([loader], [('train', 1)], 1)\n\n path = osp.join(tmpdir, next(scandir(tmpdir, '.json')))\n with open(path) as fr:\n fr.readline() # skip the first line which is `hook_msg`\n train_log = json.loads(fr.readline())\n assert train_log['mode'] == 'train' and 'time' in train_log\n val_log = json.loads(fr.readline())\n assert val_log['mode'] == 'val' and 'time' not in val_log\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_fp16.py", "content": "import numpy as np\nimport pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.runner.fp16_utils import auto_fp16, cast_tensor_type, force_fp32\n\n\ndef test_cast_tensor_type():\n inputs = torch.FloatTensor([5.])\n src_type = torch.float32\n dst_type = torch.int32\n outputs = cast_tensor_type(inputs, src_type, dst_type)\n assert isinstance(outputs, torch.Tensor)\n assert outputs.dtype == dst_type\n\n # convert torch.float to torch.half\n inputs = torch.FloatTensor([5.])\n src_type = torch.float\n dst_type = torch.half\n outputs = cast_tensor_type(inputs, src_type, dst_type)\n assert isinstance(outputs, torch.Tensor)\n assert outputs.dtype == dst_type\n\n # skip the conversion when the type of input is not the same as src_type\n inputs = torch.IntTensor([5])\n src_type = torch.float\n dst_type = torch.half\n outputs = cast_tensor_type(inputs, src_type, dst_type)\n assert isinstance(outputs, torch.Tensor)\n assert outputs.dtype == inputs.dtype\n\n inputs = 'tensor'\n src_type = str\n dst_type = str\n outputs = cast_tensor_type(inputs, src_type, dst_type)\n assert isinstance(outputs, str)\n\n inputs = np.array([5.])\n src_type = np.ndarray\n dst_type = np.ndarray\n outputs = cast_tensor_type(inputs, src_type, dst_type)\n assert isinstance(outputs, np.ndarray)\n\n inputs = dict(\n tensor_a=torch.FloatTensor([1.]), tensor_b=torch.FloatTensor([2.]))\n src_type = torch.float32\n dst_type = torch.int32\n outputs = cast_tensor_type(inputs, src_type, dst_type)\n assert isinstance(outputs, dict)\n assert outputs['tensor_a'].dtype == dst_type\n assert outputs['tensor_b'].dtype == dst_type\n\n inputs = [torch.FloatTensor([1.]), torch.FloatTensor([2.])]\n src_type = torch.float32\n dst_type = torch.int32\n outputs = cast_tensor_type(inputs, src_type, dst_type)\n assert isinstance(outputs, list)\n assert outputs[0].dtype == dst_type\n assert outputs[1].dtype == dst_type\n\n inputs = 5\n outputs = cast_tensor_type(inputs, None, None)\n assert isinstance(outputs, int)\n\n\ndef test_auto_fp16():\n\n with pytest.raises(TypeError):\n # ExampleObject is not a subclass of nn.Module\n\n class ExampleObject(object):\n\n @auto_fp16()\n def __call__(self, x):\n return x\n\n model = ExampleObject()\n input_x = torch.ones(1, dtype=torch.float32)\n model(input_x)\n\n # apply to all input args\n class ExampleModule(nn.Module):\n\n @auto_fp16()\n def forward(self, x, y):\n return x, y\n\n model = ExampleModule()\n input_x = torch.ones(1, dtype=torch.float32)\n input_y = torch.ones(1, dtype=torch.float32)\n output_x, output_y = model(input_x, input_y)\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n\n model.fp16_enabled = True\n output_x, output_y = model(input_x, input_y)\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n\n if torch.cuda.is_available():\n model.cuda()\n output_x, output_y = model(input_x.cuda(), input_y.cuda())\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n\n # apply to specified input args\n class ExampleModule(nn.Module):\n\n @auto_fp16(apply_to=('x', ))\n def forward(self, x, y):\n return x, y\n\n model = ExampleModule()\n input_x = torch.ones(1, dtype=torch.float32)\n input_y = torch.ones(1, dtype=torch.float32)\n output_x, output_y = model(input_x, input_y)\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n\n model.fp16_enabled = True\n output_x, output_y = model(input_x, input_y)\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.float32\n\n if torch.cuda.is_available():\n model.cuda()\n output_x, output_y = model(input_x.cuda(), input_y.cuda())\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.float32\n\n # apply to optional input args\n class ExampleModule(nn.Module):\n\n @auto_fp16(apply_to=('x', 'y'))\n def forward(self, x, y=None, z=None):\n return x, y, z\n\n model = ExampleModule()\n input_x = torch.ones(1, dtype=torch.float32)\n input_y = torch.ones(1, dtype=torch.float32)\n input_z = torch.ones(1, dtype=torch.float32)\n output_x, output_y, output_z = model(input_x, y=input_y, z=input_z)\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n assert output_z.dtype == torch.float32\n\n model.fp16_enabled = True\n output_x, output_y, output_z = model(input_x, y=input_y, z=input_z)\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n assert output_z.dtype == torch.float32\n\n if torch.cuda.is_available():\n model.cuda()\n output_x, output_y, output_z = model(\n input_x.cuda(), y=input_y.cuda(), z=input_z.cuda())\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n assert output_z.dtype == torch.float32\n\n # out_fp32=True\n class ExampleModule(nn.Module):\n\n @auto_fp16(apply_to=('x', 'y'), out_fp32=True)\n def forward(self, x, y=None, z=None):\n return x, y, z\n\n model = ExampleModule()\n input_x = torch.ones(1, dtype=torch.half)\n input_y = torch.ones(1, dtype=torch.float32)\n input_z = torch.ones(1, dtype=torch.float32)\n output_x, output_y, output_z = model(input_x, y=input_y, z=input_z)\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.float32\n assert output_z.dtype == torch.float32\n\n model.fp16_enabled = True\n output_x, output_y, output_z = model(input_x, y=input_y, z=input_z)\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n assert output_z.dtype == torch.float32\n\n if torch.cuda.is_available():\n model.cuda()\n output_x, output_y, output_z = model(\n input_x.cuda(), y=input_y.cuda(), z=input_z.cuda())\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n assert output_z.dtype == torch.float32\n\n\ndef test_force_fp32():\n\n with pytest.raises(TypeError):\n # ExampleObject is not a subclass of nn.Module\n\n class ExampleObject(object):\n\n @force_fp32()\n def __call__(self, x):\n return x\n\n model = ExampleObject()\n input_x = torch.ones(1, dtype=torch.float32)\n model(input_x)\n\n # apply to all input args\n class ExampleModule(nn.Module):\n\n @force_fp32()\n def forward(self, x, y):\n return x, y\n\n model = ExampleModule()\n input_x = torch.ones(1, dtype=torch.half)\n input_y = torch.ones(1, dtype=torch.half)\n output_x, output_y = model(input_x, input_y)\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n\n model.fp16_enabled = True\n output_x, output_y = model(input_x, input_y)\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n\n if torch.cuda.is_available():\n model.cuda()\n output_x, output_y = model(input_x.cuda(), input_y.cuda())\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n\n # apply to specified input args\n class ExampleModule(nn.Module):\n\n @force_fp32(apply_to=('x', ))\n def forward(self, x, y):\n return x, y\n\n model = ExampleModule()\n input_x = torch.ones(1, dtype=torch.half)\n input_y = torch.ones(1, dtype=torch.half)\n output_x, output_y = model(input_x, input_y)\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n\n model.fp16_enabled = True\n output_x, output_y = model(input_x, input_y)\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.half\n\n if torch.cuda.is_available():\n model.cuda()\n output_x, output_y = model(input_x.cuda(), input_y.cuda())\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.half\n\n # apply to optional input args\n class ExampleModule(nn.Module):\n\n @force_fp32(apply_to=('x', 'y'))\n def forward(self, x, y=None, z=None):\n return x, y, z\n\n model = ExampleModule()\n input_x = torch.ones(1, dtype=torch.half)\n input_y = torch.ones(1, dtype=torch.half)\n input_z = torch.ones(1, dtype=torch.half)\n output_x, output_y, output_z = model(input_x, y=input_y, z=input_z)\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n assert output_z.dtype == torch.half\n\n model.fp16_enabled = True\n output_x, output_y, output_z = model(input_x, y=input_y, z=input_z)\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n assert output_z.dtype == torch.half\n\n if torch.cuda.is_available():\n model.cuda()\n output_x, output_y, output_z = model(\n input_x.cuda(), y=input_y.cuda(), z=input_z.cuda())\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.float32\n assert output_z.dtype == torch.half\n\n # out_fp16=True\n class ExampleModule(nn.Module):\n\n @force_fp32(apply_to=('x', 'y'), out_fp16=True)\n def forward(self, x, y=None, z=None):\n return x, y, z\n\n model = ExampleModule()\n input_x = torch.ones(1, dtype=torch.float32)\n input_y = torch.ones(1, dtype=torch.half)\n input_z = torch.ones(1, dtype=torch.half)\n output_x, output_y, output_z = model(input_x, y=input_y, z=input_z)\n assert output_x.dtype == torch.float32\n assert output_y.dtype == torch.half\n assert output_z.dtype == torch.half\n\n model.fp16_enabled = True\n output_x, output_y, output_z = model(input_x, y=input_y, z=input_z)\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n assert output_z.dtype == torch.half\n\n if torch.cuda.is_available():\n model.cuda()\n output_x, output_y, output_z = model(\n input_x.cuda(), y=input_y.cuda(), z=input_z.cuda())\n assert output_x.dtype == torch.half\n assert output_y.dtype == torch.half\n assert output_z.dtype == torch.half\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_hooks.py", "content": "\"\"\"Tests the hooks with runners.\n\nCommandLine:\n pytest tests/test_runner/test_hooks.py\n xdoctest tests/test_hooks.py zero\n\"\"\"\nimport logging\nimport os.path as osp\nimport platform\nimport random\nimport re\nimport shutil\nimport sys\nimport tempfile\nfrom unittest.mock import MagicMock, Mock, call, patch\n\nimport pytest\nimport torch\nimport torch.nn as nn\nfrom torch.nn.init import constant_\nfrom torch.utils.data import DataLoader\n\nfrom mmcv.fileio.file_client import PetrelBackend\nfrom mmcv.runner import (CheckpointHook, DvcliveLoggerHook, EMAHook,\n Fp16OptimizerHook,\n GradientCumulativeFp16OptimizerHook,\n GradientCumulativeOptimizerHook, IterTimerHook,\n MlflowLoggerHook, NeptuneLoggerHook, OptimizerHook,\n PaviLoggerHook, WandbLoggerHook, build_runner)\nfrom mmcv.runner.fp16_utils import auto_fp16\nfrom mmcv.runner.hooks.hook import HOOKS, Hook\nfrom mmcv.runner.hooks.lr_updater import (CosineRestartLrUpdaterHook,\n CyclicLrUpdaterHook,\n FlatCosineAnnealingLrUpdaterHook,\n OneCycleLrUpdaterHook,\n StepLrUpdaterHook)\nfrom mmcv.utils import TORCH_VERSION\n\nsys.modules['petrel_client'] = MagicMock()\nsys.modules['petrel_client.client'] = MagicMock()\n\n\ndef test_optimizerhook():\n\n class Model(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.conv1 = nn.Conv2d(\n in_channels=1,\n out_channels=2,\n kernel_size=3,\n stride=1,\n padding=1,\n dilation=1)\n self.conv2 = nn.Conv2d(\n in_channels=2,\n out_channels=2,\n kernel_size=3,\n stride=1,\n padding=1,\n dilation=1)\n self.conv3 = nn.Conv2d(\n in_channels=1,\n out_channels=2,\n kernel_size=3,\n stride=1,\n padding=1,\n dilation=1)\n\n def forward(self, x):\n x1 = self.conv1(x)\n x2 = self.conv2(x1)\n return x1, x2\n\n model = Model()\n x = torch.rand(1, 1, 3, 3)\n\n dummy_runner = Mock()\n dummy_runner.optimizer.zero_grad = Mock(return_value=None)\n dummy_runner.optimizer.step = Mock(return_value=None)\n dummy_runner.model = model\n dummy_runner.outputs = dict()\n\n dummy_runner.outputs['num_samples'] = 0\n\n class DummyLogger():\n\n def __init__(self):\n self.msg = ''\n\n def log(self, msg=None, **kwargs):\n self.msg += msg\n\n dummy_runner.logger = DummyLogger()\n optimizer_hook = OptimizerHook(\n dict(max_norm=2), detect_anomalous_params=True)\n\n dummy_runner.outputs['loss'] = model(x)[0].sum()\n optimizer_hook.after_train_iter(dummy_runner)\n # assert the parameters of conv2 and conv3 are not in the\n # computational graph which is with x1.sum() as root.\n assert 'conv2.weight' in dummy_runner.logger.msg\n assert 'conv2.bias' in dummy_runner.logger.msg\n assert 'conv3.weight' in dummy_runner.logger.msg\n assert 'conv3.bias' in dummy_runner.logger.msg\n assert 'conv1.weight' not in dummy_runner.logger.msg\n assert 'conv1.bias' not in dummy_runner.logger.msg\n\n dummy_runner.outputs['loss'] = model(x)[1].sum()\n dummy_runner.logger.msg = ''\n optimizer_hook.after_train_iter(dummy_runner)\n # assert the parameters of conv3 are not in the computational graph\n assert 'conv3.weight' in dummy_runner.logger.msg\n assert 'conv3.bias' in dummy_runner.logger.msg\n assert 'conv2.weight' not in dummy_runner.logger.msg\n assert 'conv2.bias' not in dummy_runner.logger.msg\n assert 'conv1.weight' not in dummy_runner.logger.msg\n assert 'conv1.bias' not in dummy_runner.logger.msg\n\n\ndef test_checkpoint_hook(tmp_path):\n \"\"\"xdoctest -m tests/test_runner/test_hooks.py test_checkpoint_hook.\"\"\"\n\n # test epoch based runner\n loader = DataLoader(torch.ones((5, 2)))\n runner = _build_demo_runner('EpochBasedRunner', max_epochs=1)\n runner.meta = dict()\n checkpointhook = CheckpointHook(interval=1, by_epoch=True)\n runner.register_hook(checkpointhook)\n runner.run([loader], [('train', 1)])\n assert runner.meta['hook_msgs']['last_ckpt'] == osp.join(\n runner.work_dir, 'epoch_1.pth')\n shutil.rmtree(runner.work_dir)\n\n # test petrel oss when the type of runner is `EpochBasedRunner`\n runner = _build_demo_runner('EpochBasedRunner', max_epochs=4)\n runner.meta = dict()\n out_dir = 's3://user/data'\n with patch.object(PetrelBackend, 'put') as mock_put, \\\n patch.object(PetrelBackend, 'remove') as mock_remove, \\\n patch.object(PetrelBackend, 'isfile') as mock_isfile:\n checkpointhook = CheckpointHook(\n interval=1, out_dir=out_dir, by_epoch=True, max_keep_ckpts=2)\n runner.register_hook(checkpointhook)\n runner.run([loader], [('train', 1)])\n basename = osp.basename(runner.work_dir.rstrip(osp.sep))\n assert runner.meta['hook_msgs']['last_ckpt'] == \\\n '/'.join([out_dir, basename, 'epoch_4.pth'])\n mock_put.assert_called()\n mock_remove.assert_called()\n mock_isfile.assert_called()\n shutil.rmtree(runner.work_dir)\n\n # test iter based runner\n runner = _build_demo_runner(\n 'IterBasedRunner', max_iters=1, max_epochs=None)\n runner.meta = dict()\n checkpointhook = CheckpointHook(interval=1, by_epoch=False)\n runner.register_hook(checkpointhook)\n runner.run([loader], [('train', 1)])\n assert runner.meta['hook_msgs']['last_ckpt'] == osp.join(\n runner.work_dir, 'iter_1.pth')\n shutil.rmtree(runner.work_dir)\n\n # test petrel oss when the type of runner is `IterBasedRunner`\n runner = _build_demo_runner(\n 'IterBasedRunner', max_iters=4, max_epochs=None)\n runner.meta = dict()\n out_dir = 's3://user/data'\n with patch.object(PetrelBackend, 'put') as mock_put, \\\n patch.object(PetrelBackend, 'remove') as mock_remove, \\\n patch.object(PetrelBackend, 'isfile') as mock_isfile:\n checkpointhook = CheckpointHook(\n interval=1, out_dir=out_dir, by_epoch=False, max_keep_ckpts=2)\n runner.register_hook(checkpointhook)\n runner.run([loader], [('train', 1)])\n basename = osp.basename(runner.work_dir.rstrip(osp.sep))\n assert runner.meta['hook_msgs']['last_ckpt'] == \\\n '/'.join([out_dir, basename, 'iter_4.pth'])\n mock_put.assert_called()\n mock_remove.assert_called()\n mock_isfile.assert_called()\n shutil.rmtree(runner.work_dir)\n\n\ndef test_ema_hook():\n \"\"\"xdoctest -m tests/test_hooks.py test_ema_hook.\"\"\"\n\n class DemoModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.conv = nn.Conv2d(\n in_channels=1,\n out_channels=2,\n kernel_size=1,\n padding=1,\n bias=True)\n self._init_weight()\n\n def _init_weight(self):\n constant_(self.conv.weight, 0)\n constant_(self.conv.bias, 0)\n\n def forward(self, x):\n return self.conv(x).sum()\n\n def train_step(self, x, optimizer, **kwargs):\n return dict(loss=self(x))\n\n def val_step(self, x, optimizer, **kwargs):\n return dict(loss=self(x))\n\n loader = DataLoader(torch.ones((1, 1, 1, 1)))\n runner = _build_demo_runner()\n demo_model = DemoModel()\n runner.model = demo_model\n emahook = EMAHook(momentum=0.1, interval=2, warm_up=100, resume_from=None)\n checkpointhook = CheckpointHook(interval=1, by_epoch=True)\n runner.register_hook(emahook, priority='HIGHEST')\n runner.register_hook(checkpointhook)\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n checkpoint = torch.load(f'{runner.work_dir}/epoch_1.pth')\n contain_ema_buffer = False\n for name, value in checkpoint['state_dict'].items():\n if 'ema' in name:\n contain_ema_buffer = True\n assert value.sum() == 0\n value.fill_(1)\n else:\n assert value.sum() == 0\n assert contain_ema_buffer\n torch.save(checkpoint, f'{runner.work_dir}/epoch_1.pth')\n work_dir = runner.work_dir\n resume_ema_hook = EMAHook(\n momentum=0.5, warm_up=0, resume_from=f'{work_dir}/epoch_1.pth')\n runner = _build_demo_runner(max_epochs=2)\n runner.model = demo_model\n runner.register_hook(resume_ema_hook, priority='HIGHEST')\n checkpointhook = CheckpointHook(interval=1, by_epoch=True)\n runner.register_hook(checkpointhook)\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n checkpoint = torch.load(f'{runner.work_dir}/epoch_2.pth')\n contain_ema_buffer = False\n for name, value in checkpoint['state_dict'].items():\n if 'ema' in name:\n contain_ema_buffer = True\n assert value.sum() == 2\n else:\n assert value.sum() == 1\n assert contain_ema_buffer\n shutil.rmtree(runner.work_dir)\n shutil.rmtree(work_dir)\n\n\ndef test_custom_hook():\n\n @HOOKS.register_module()\n class ToyHook(Hook):\n\n def __init__(self, info, *args, **kwargs):\n super().__init__()\n self.info = info\n\n runner = _build_demo_runner_without_hook('EpochBasedRunner', max_epochs=1)\n # test if custom_hooks is None\n runner.register_custom_hooks(None)\n assert len(runner.hooks) == 0\n # test if custom_hooks is dict list\n custom_hooks_cfg = [\n dict(type='ToyHook', priority=51, info=51),\n dict(type='ToyHook', priority=49, info=49)\n ]\n runner.register_custom_hooks(custom_hooks_cfg)\n assert [hook.info for hook in runner.hooks] == [49, 51]\n # test if custom_hooks is object and without priority\n runner.register_custom_hooks(ToyHook(info='default'))\n assert len(runner.hooks) == 3 and runner.hooks[1].info == 'default'\n shutil.rmtree(runner.work_dir)\n\n runner = _build_demo_runner_without_hook('EpochBasedRunner', max_epochs=1)\n # test custom_hooks with string priority setting\n priority_ranks = [\n 'HIGHEST', 'VERY_HIGH', 'HIGH', 'ABOVE_NORMAL', 'NORMAL',\n 'BELOW_NORMAL', 'LOW', 'VERY_LOW', 'LOWEST'\n ]\n random_priority_ranks = priority_ranks.copy()\n random.shuffle(random_priority_ranks)\n custom_hooks_cfg = [\n dict(type='ToyHook', priority=rank, info=rank)\n for rank in random_priority_ranks\n ]\n runner.register_custom_hooks(custom_hooks_cfg)\n assert [hook.info for hook in runner.hooks] == priority_ranks\n shutil.rmtree(runner.work_dir)\n\n runner = _build_demo_runner_without_hook('EpochBasedRunner', max_epochs=1)\n # test register_training_hooks order\n custom_hooks_cfg = [\n dict(type='ToyHook', priority=1, info='custom 1'),\n dict(type='ToyHook', priority='NORMAL', info='custom normal'),\n dict(type='ToyHook', priority=89, info='custom 89')\n ]\n runner.register_training_hooks(\n lr_config=ToyHook('lr'),\n optimizer_config=ToyHook('optimizer'),\n checkpoint_config=ToyHook('checkpoint'),\n log_config=dict(interval=1, hooks=[dict(type='ToyHook', info='log')]),\n momentum_config=ToyHook('momentum'),\n timer_config=ToyHook('timer'),\n custom_hooks_config=custom_hooks_cfg)\n # If custom hooks have same priority with default hooks, custom hooks\n # will be triggered after default hooks.\n hooks_order = [\n 'custom 1', 'lr', 'momentum', 'optimizer', 'checkpoint',\n 'custom normal', 'timer', 'custom 89', 'log'\n ]\n assert [hook.info for hook in runner.hooks] == hooks_order\n shutil.rmtree(runner.work_dir)\n\n\ndef test_pavi_hook():\n sys.modules['pavi'] = MagicMock()\n\n loader = DataLoader(torch.ones((5, 2)))\n runner = _build_demo_runner()\n runner.meta = dict(config_dict=dict(lr=0.02, gpu_ids=range(1)))\n hook = PaviLoggerHook(add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n shutil.rmtree(runner.work_dir)\n\n assert hasattr(hook, 'writer')\n hook.writer.add_scalars.assert_called_with('val', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 1)\n # in Windows environment, the latest checkpoint is copied from epoch_1.pth\n if platform.system() == 'Windows':\n snapshot_file_path = osp.join(runner.work_dir, 'latest.pth')\n else:\n snapshot_file_path = osp.join(runner.work_dir, 'epoch_1.pth')\n hook.writer.add_snapshot_file.assert_called_with(\n tag=runner.work_dir.split('/')[-1],\n snapshot_file_path=snapshot_file_path,\n iteration=1)\n\n\ndef test_sync_buffers_hook():\n loader = DataLoader(torch.ones((5, 2)))\n runner = _build_demo_runner()\n runner.register_hook_from_cfg(dict(type='SyncBuffersHook'))\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n shutil.rmtree(runner.work_dir)\n\n\n@pytest.mark.parametrize('multi_optimizers, max_iters, gamma, cyclic_times',\n [(True, 8, 1, 1), (False, 8, 0.5, 2)])\ndef test_momentum_runner_hook(multi_optimizers, max_iters, gamma,\n cyclic_times):\n \"\"\"xdoctest -m tests/test_hooks.py test_momentum_runner_hook.\"\"\"\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add momentum scheduler\n hook_cfg = dict(\n type='CyclicMomentumUpdaterHook',\n by_epoch=False,\n target_ratio=(0.85 / 0.95, 1),\n cyclic_times=cyclic_times,\n step_ratio_up=0.4,\n gamma=gamma)\n runner.register_hook_from_cfg(hook_cfg)\n\n # add momentum LR scheduler\n hook_cfg = dict(\n type='CyclicLrUpdaterHook',\n by_epoch=False,\n target_ratio=(10, 1),\n cyclic_times=1,\n step_ratio_up=0.4)\n runner.register_hook_from_cfg(hook_cfg)\n runner.register_hook_from_cfg(dict(type='IterTimerHook'))\n\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n # TODO: use a more elegant way to check values\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.01999999999999999,\n 'learning_rate/model2': 0.009999999999999995,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.2,\n 'learning_rate/model2': 0.1,\n 'momentum/model1': 0.85,\n 'momentum/model2': 0.8052631578947369,\n }, 5),\n call(\n 'train', {\n 'learning_rate/model1': 0.155,\n 'learning_rate/model2': 0.0775,\n 'momentum/model1': 0.875,\n 'momentum/model2': 0.8289473684210527,\n }, 7)\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.01999999999999999,\n 'momentum': 0.95\n }, 1),\n call('train', {\n 'learning_rate': 0.11,\n 'momentum': 0.85\n }, 3),\n call('train', {\n 'learning_rate': 0.1879422863405995,\n 'momentum': 0.95\n }, 6),\n call('train', {\n 'learning_rate': 0.11000000000000001,\n 'momentum': 0.9\n }, 8),\n ]\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n # test constant momentum warmup\n sys.modules['pavi'] = MagicMock()\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add momentum scheduler\n hook_cfg = dict(\n type='StepMomentumUpdaterHook',\n by_epoch=False,\n warmup='constant',\n warmup_iters=5,\n warmup_ratio=0.5,\n step=[10],\n )\n runner.register_hook_from_cfg(hook_cfg)\n runner.register_hook_from_cfg(dict(type='IterTimerHook'))\n\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 1.9,\n 'momentum/model2': 1.8,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 1.9,\n 'momentum/model2': 1.8,\n }, 5),\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 10),\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 1.9\n }, 1),\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 1.9\n }, 5),\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 10),\n ]\n\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n # test linear momentum warmup\n sys.modules['pavi'] = MagicMock()\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add momentum scheduler\n hook_cfg = dict(\n type='StepMomentumUpdaterHook',\n by_epoch=False,\n warmup='linear',\n warmup_iters=5,\n warmup_ratio=0.5,\n step=[10],\n )\n runner.register_hook_from_cfg(hook_cfg)\n runner.register_hook_from_cfg(dict(type='IterTimerHook'))\n\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 1.9,\n 'momentum/model2': 1.8,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 1.3571428571428572,\n 'momentum/model2': 1.2857142857142858,\n }, 3),\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 10),\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 1.9\n }, 1),\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 1.3571428571428572\n }, 3),\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 10),\n ]\n\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n # test exponentially momentum warmup\n sys.modules['pavi'] = MagicMock()\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add momentum scheduler\n hook_cfg = dict(\n type='StepMomentumUpdaterHook',\n by_epoch=False,\n warmup='exp',\n warmup_iters=5,\n warmup_ratio=0.5,\n step=[10],\n )\n runner.register_hook_from_cfg(hook_cfg)\n runner.register_hook_from_cfg(dict(type='IterTimerHook'))\n\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 1.9,\n 'momentum/model2': 1.8,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 1.4399307381848783,\n 'momentum/model2': 1.3641449098593583,\n }, 3),\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 10),\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 1.9\n }, 1),\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 1.4399307381848783\n }, 3),\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 10),\n ]\n\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n\n@pytest.mark.parametrize('multi_optimizers', (True, False))\ndef test_cosine_runner_hook(multi_optimizers):\n \"\"\"xdoctest -m tests/test_hooks.py test_cosine_runner_hook.\"\"\"\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add momentum scheduler\n hook_cfg = dict(\n type='CosineAnnealingMomentumUpdaterHook',\n min_momentum_ratio=0.99 / 0.95,\n by_epoch=False,\n warmup_iters=2,\n warmup_ratio=0.9 / 0.95)\n runner.register_hook_from_cfg(hook_cfg)\n\n # add momentum LR scheduler\n hook_cfg = dict(\n type='CosineAnnealingLrUpdaterHook',\n by_epoch=False,\n min_lr_ratio=0,\n warmup_iters=2,\n warmup_ratio=0.9)\n runner.register_hook_from_cfg(hook_cfg)\n runner.register_hook_from_cfg(dict(type='IterTimerHook'))\n runner.register_hook(IterTimerHook())\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n # TODO: use a more elegant way to check values\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.01,\n 'learning_rate/model2': 0.005,\n 'momentum/model1': 0.97,\n 'momentum/model2': 0.9189473684210527,\n }, 6),\n call(\n 'train', {\n 'learning_rate/model1': 0.0004894348370484647,\n 'learning_rate/model2': 0.00024471741852423234,\n 'momentum/model1': 0.9890211303259032,\n 'momentum/model2': 0.9369673866245399,\n }, 10)\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 1),\n call('train', {\n 'learning_rate': 0.01,\n 'momentum': 0.97\n }, 6),\n call(\n 'train', {\n 'learning_rate': 0.0004894348370484647,\n 'momentum': 0.9890211303259032\n }, 10)\n ]\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n\n@pytest.mark.parametrize('multi_optimizers, by_epoch', [(False, False),\n (True, False),\n (False, True),\n (True, True)])\ndef test_flat_cosine_runner_hook(multi_optimizers, by_epoch):\n \"\"\"xdoctest -m tests/test_hooks.py test_flat_cosine_runner_hook.\"\"\"\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n max_epochs = 10 if by_epoch else 1\n runner = _build_demo_runner(\n multi_optimizers=multi_optimizers, max_epochs=max_epochs)\n\n with pytest.raises(ValueError):\n # start_percent: expected float between 0 and 1\n FlatCosineAnnealingLrUpdaterHook(start_percent=-0.1, min_lr_ratio=0)\n\n # add LR scheduler\n hook_cfg = dict(\n type='FlatCosineAnnealingLrUpdaterHook',\n by_epoch=by_epoch,\n min_lr_ratio=0,\n warmup='linear',\n warmup_iters=10 if by_epoch else 2,\n warmup_ratio=0.9,\n start_percent=0.5)\n runner.register_hook_from_cfg(hook_cfg)\n runner.register_hook_from_cfg(dict(type='IterTimerHook'))\n runner.register_hook(IterTimerHook())\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n # TODO: use a more elegant way to check values\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n if by_epoch:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.018000000000000002,\n 'learning_rate/model2': 0.009000000000000001,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 11),\n call(\n 'train', {\n 'learning_rate/model1': 0.018090169943749474,\n 'learning_rate/model2': 0.009045084971874737,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 61),\n call(\n 'train', {\n 'learning_rate/model1': 0.0019098300562505265,\n 'learning_rate/model2': 0.0009549150281252633,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 100)\n ]\n else:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.018000000000000002,\n 'learning_rate/model2': 0.009000000000000001,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9\n }, 6),\n call(\n 'train', {\n 'learning_rate/model1': 0.018090169943749474,\n 'learning_rate/model2': 0.009045084971874737,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9\n }, 7),\n call(\n 'train', {\n 'learning_rate/model1': 0.0019098300562505265,\n 'learning_rate/model2': 0.0009549150281252633,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9\n }, 10)\n ]\n else:\n if by_epoch:\n calls = [\n call('train', {\n 'learning_rate': 0.018000000000000002,\n 'momentum': 0.95\n }, 1),\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 11),\n call('train', {\n 'learning_rate': 0.018090169943749474,\n 'momentum': 0.95\n }, 61),\n call('train', {\n 'learning_rate': 0.0019098300562505265,\n 'momentum': 0.95\n }, 100)\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.018000000000000002,\n 'momentum': 0.95\n }, 1),\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 6),\n call('train', {\n 'learning_rate': 0.018090169943749474,\n 'momentum': 0.95\n }, 7),\n call('train', {\n 'learning_rate': 0.0019098300562505265,\n 'momentum': 0.95\n }, 10)\n ]\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n\n@pytest.mark.parametrize('multi_optimizers, max_iters', [(True, 10), (True, 2),\n (False, 10),\n (False, 2)])\ndef test_one_cycle_runner_hook(multi_optimizers, max_iters):\n \"\"\"Test OneCycleLrUpdaterHook and OneCycleMomentumUpdaterHook.\"\"\"\n with pytest.raises(AssertionError):\n # by_epoch should be False\n OneCycleLrUpdaterHook(max_lr=0.1, by_epoch=True)\n\n with pytest.raises(ValueError):\n # expected float between 0 and 1\n OneCycleLrUpdaterHook(max_lr=0.1, pct_start=-0.1)\n\n with pytest.raises(ValueError):\n # anneal_strategy should be either 'cos' or 'linear'\n OneCycleLrUpdaterHook(max_lr=0.1, anneal_strategy='sin')\n\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add momentum scheduler\n hook_cfg = dict(\n type='OneCycleMomentumUpdaterHook',\n base_momentum=0.85,\n max_momentum=0.95,\n pct_start=0.5,\n anneal_strategy='cos',\n three_phase=False)\n runner.register_hook_from_cfg(hook_cfg)\n\n # add LR scheduler\n hook_cfg = dict(\n type='OneCycleLrUpdaterHook',\n max_lr=0.01,\n pct_start=0.5,\n anneal_strategy='cos',\n div_factor=25,\n final_div_factor=1e4,\n three_phase=False)\n runner.register_hook_from_cfg(hook_cfg)\n runner.register_hook_from_cfg(dict(type='IterTimerHook'))\n runner.register_hook(IterTimerHook())\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n # TODO: use a more elegant way to check values\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.0003999999999999993,\n 'learning_rate/model2': 0.0003999999999999993,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.95,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.00904508879153485,\n 'learning_rate/model2': 0.00904508879153485,\n 'momentum/model1': 0.8595491502812526,\n 'momentum/model2': 0.8595491502812526,\n }, 6),\n call(\n 'train', {\n 'learning_rate/model1': 4e-08,\n 'learning_rate/model2': 4e-08,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.95,\n }, 10)\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.0003999999999999993,\n 'momentum': 0.95\n }, 1),\n call(\n 'train', {\n 'learning_rate': 0.00904508879153485,\n 'momentum': 0.8595491502812526\n }, 6),\n call('train', {\n 'learning_rate': 4e-08,\n 'momentum': 0.95\n }, 10)\n ]\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n # Test OneCycleLrUpdaterHook\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n runner = _build_demo_runner(\n runner_type='IterBasedRunner', max_epochs=None, max_iters=max_iters)\n\n args = dict(\n max_lr=0.01,\n total_steps=5,\n pct_start=0.5,\n anneal_strategy='linear',\n div_factor=25,\n final_div_factor=1e4,\n )\n hook = OneCycleLrUpdaterHook(**args)\n runner.register_hook(hook)\n if max_iters == 10:\n # test total_steps < max_iters\n with pytest.raises(ValueError):\n runner.run([loader], [('train', 1)])\n else:\n # test total_steps > max_iters\n runner.run([loader], [('train', 1)])\n lr_last = runner.current_lr()\n t = torch.tensor([0.0], requires_grad=True)\n optim = torch.optim.SGD([t], lr=0.01)\n lr_scheduler = torch.optim.lr_scheduler.OneCycleLR(optim, **args)\n lr_target = []\n for _ in range(max_iters):\n optim.step()\n lr_target.append(optim.param_groups[0]['lr'])\n lr_scheduler.step()\n assert lr_target[-1] == lr_last[0]\n\n\n@pytest.mark.parametrize('multi_optimizers', (True, False))\ndef test_cosine_restart_lr_update_hook(multi_optimizers):\n \"\"\"Test CosineRestartLrUpdaterHook.\"\"\"\n with pytest.raises(AssertionError):\n # either `min_lr` or `min_lr_ratio` should be specified\n CosineRestartLrUpdaterHook(\n by_epoch=False,\n periods=[2, 10],\n restart_weights=[0.5, 0.5],\n min_lr=0.1,\n min_lr_ratio=0)\n\n with pytest.raises(AssertionError):\n # periods and restart_weights should have the same length\n CosineRestartLrUpdaterHook(\n by_epoch=False,\n periods=[2, 10],\n restart_weights=[0.5],\n min_lr_ratio=0)\n\n with pytest.raises(ValueError):\n # the last cumulative_periods 7 (out of [5, 7]) should >= 10\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n runner = _build_demo_runner()\n\n # add cosine restart LR scheduler\n hook = CosineRestartLrUpdaterHook(\n by_epoch=False,\n periods=[5, 2], # cumulative_periods [5, 7 (5 + 2)]\n restart_weights=[0.5, 0.5],\n min_lr=0.0001)\n runner.register_hook(hook)\n runner.register_hook(IterTimerHook())\n\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add cosine restart LR scheduler\n hook = CosineRestartLrUpdaterHook(\n by_epoch=False,\n periods=[5, 5],\n restart_weights=[0.5, 0.5],\n min_lr_ratio=0)\n runner.register_hook(hook)\n runner.register_hook(IterTimerHook())\n\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n # TODO: use a more elegant way to check values\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.01,\n 'learning_rate/model2': 0.005,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.01,\n 'learning_rate/model2': 0.005,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 6),\n call(\n 'train', {\n 'learning_rate/model1': 0.0009549150281252633,\n 'learning_rate/model2': 0.00047745751406263163,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 10)\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.01,\n 'momentum': 0.95\n }, 1),\n call('train', {\n 'learning_rate': 0.01,\n 'momentum': 0.95\n }, 6),\n call('train', {\n 'learning_rate': 0.0009549150281252633,\n 'momentum': 0.95\n }, 10)\n ]\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n\n@pytest.mark.parametrize('multi_optimizers', (True, False))\ndef test_step_runner_hook(multi_optimizers):\n \"\"\"Test StepLrUpdaterHook.\"\"\"\n with pytest.raises(TypeError):\n # `step` should be specified\n StepLrUpdaterHook()\n with pytest.raises(AssertionError):\n # if `step` is int, should be positive\n StepLrUpdaterHook(-10)\n with pytest.raises(AssertionError):\n # if `step` is list of int, should all be positive\n StepLrUpdaterHook([10, 16, -20])\n\n # test StepLrUpdaterHook with int `step` value\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((30, 2)))\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add momentum scheduler\n hook_cfg = dict(\n type='StepMomentumUpdaterHook',\n by_epoch=False,\n step=5,\n gamma=0.5,\n min_momentum=0.05)\n runner.register_hook_from_cfg(hook_cfg)\n\n # add step LR scheduler\n hook = StepLrUpdaterHook(by_epoch=False, step=5, gamma=0.5, min_lr=1e-3)\n runner.register_hook(hook)\n runner.register_hook(IterTimerHook())\n\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n # TODO: use a more elegant way to check values\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.01,\n 'learning_rate/model2': 0.005,\n 'momentum/model1': 0.475,\n 'momentum/model2': 0.45\n }, 6),\n call(\n 'train', {\n 'learning_rate/model1': 0.0025,\n 'learning_rate/model2': 0.00125,\n 'momentum/model1': 0.11875,\n 'momentum/model2': 0.1125\n }, 16),\n call(\n 'train', {\n 'learning_rate/model1': 0.00125,\n 'learning_rate/model2': 0.001,\n 'momentum/model1': 0.059375,\n 'momentum/model2': 0.05625\n }, 21),\n call(\n 'train', {\n 'learning_rate/model1': 0.001,\n 'learning_rate/model2': 0.001,\n 'momentum/model1': 0.05,\n 'momentum/model2': 0.05\n }, 26),\n call(\n 'train', {\n 'learning_rate/model1': 0.001,\n 'learning_rate/model2': 0.001,\n 'momentum/model1': 0.05,\n 'momentum/model2': 0.05\n }, 30)\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 1),\n call('train', {\n 'learning_rate': 0.01,\n 'momentum': 0.475\n }, 6),\n call('train', {\n 'learning_rate': 0.0025,\n 'momentum': 0.11875\n }, 16),\n call('train', {\n 'learning_rate': 0.00125,\n 'momentum': 0.059375\n }, 21),\n call('train', {\n 'learning_rate': 0.001,\n 'momentum': 0.05\n }, 26),\n call('train', {\n 'learning_rate': 0.001,\n 'momentum': 0.05\n }, 30)\n ]\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n # test StepLrUpdaterHook with list[int] `step` value\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n runner = _build_demo_runner(multi_optimizers=multi_optimizers)\n\n # add momentum scheduler\n hook_cfg = dict(\n type='StepMomentumUpdaterHook',\n by_epoch=False,\n step=[4, 6, 8],\n gamma=0.1)\n runner.register_hook_from_cfg(hook_cfg)\n\n # add step LR scheduler\n hook = StepLrUpdaterHook(by_epoch=False, step=[4, 6, 8], gamma=0.1)\n runner.register_hook(hook)\n runner.register_hook(IterTimerHook())\n\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n # TODO: use a more elegant way to check values\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.002,\n 'learning_rate/model2': 0.001,\n 'momentum/model1': 9.5e-2,\n 'momentum/model2': 9.000000000000001e-2\n }, 5),\n call(\n 'train', {\n 'learning_rate/model1': 2.0000000000000004e-4,\n 'learning_rate/model2': 1.0000000000000002e-4,\n 'momentum/model1': 9.500000000000001e-3,\n 'momentum/model2': 9.000000000000003e-3\n }, 7),\n call(\n 'train', {\n 'learning_rate/model1': 2.0000000000000005e-05,\n 'learning_rate/model2': 1.0000000000000003e-05,\n 'momentum/model1': 9.500000000000002e-4,\n 'momentum/model2': 9.000000000000002e-4\n }, 9)\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 1),\n call('train', {\n 'learning_rate': 0.002,\n 'momentum': 0.095\n }, 5),\n call(\n 'train', {\n 'learning_rate': 2.0000000000000004e-4,\n 'momentum': 9.500000000000001e-3\n }, 7),\n call(\n 'train', {\n 'learning_rate': 2.0000000000000005e-05,\n 'momentum': 9.500000000000002e-4\n }, 9)\n ]\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n\n@pytest.mark.parametrize('multi_optimizers, max_iters, gamma, cyclic_times',\n [(True, 8, 1, 1), (False, 8, 0.5, 2)])\ndef test_cyclic_lr_update_hook(multi_optimizers, max_iters, gamma,\n cyclic_times):\n \"\"\"Test CyclicLrUpdateHook.\"\"\"\n with pytest.raises(AssertionError):\n # by_epoch should be False\n CyclicLrUpdaterHook(by_epoch=True)\n\n with pytest.raises(AssertionError):\n # target_ratio must be either float or tuple/list of two floats\n CyclicLrUpdaterHook(by_epoch=False, target_ratio=(10.0, 0.1, 0.2))\n\n with pytest.raises(AssertionError):\n # step_ratio_up must be in range [0,1)\n CyclicLrUpdaterHook(by_epoch=False, step_ratio_up=1.4)\n\n with pytest.raises(ValueError):\n # anneal_strategy must be one of \"cos\" or \"linear\"\n CyclicLrUpdaterHook(by_epoch=False, anneal_strategy='sin')\n\n with pytest.raises(AssertionError):\n # gamma must be in range (0, 1]\n CyclicLrUpdaterHook(by_epoch=False, gamma=0)\n\n sys.modules['pavi'] = MagicMock()\n loader = DataLoader(torch.ones((10, 2)))\n runner = _build_demo_runner(\n runner_type='IterBasedRunner',\n max_epochs=None,\n max_iters=max_iters,\n multi_optimizers=multi_optimizers)\n\n # add cyclic LR scheduler\n schedule_hook = CyclicLrUpdaterHook(\n by_epoch=False,\n target_ratio=(10.0, 1.0),\n cyclic_times=cyclic_times,\n step_ratio_up=0.5,\n anneal_strategy='linear',\n gamma=gamma)\n runner.register_hook(schedule_hook)\n runner.register_hook_from_cfg(dict(type='IterTimerHook'))\n runner.register_hook(IterTimerHook())\n # add pavi hook\n hook = PaviLoggerHook(interval=1, add_graph=False, add_last_ckpt=True)\n runner.register_hook(hook)\n runner.run([loader], [('train', 1)])\n shutil.rmtree(runner.work_dir)\n\n assert hasattr(hook, 'writer')\n if multi_optimizers:\n calls = [\n call(\n 'train', {\n 'learning_rate/model1': 0.02,\n 'learning_rate/model2': 0.01,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 1),\n call(\n 'train', {\n 'learning_rate/model1': 0.155,\n 'learning_rate/model2': 0.0775,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 4),\n call(\n 'train', {\n 'learning_rate/model1': 0.155,\n 'learning_rate/model2': 0.0775,\n 'momentum/model1': 0.95,\n 'momentum/model2': 0.9,\n }, 6)\n ]\n else:\n calls = [\n call('train', {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, 1),\n call('train', {\n 'learning_rate': 0.11,\n 'momentum': 0.95\n }, 4),\n call('train', {\n 'learning_rate': 0.065,\n 'momentum': 0.95\n }, 6),\n call('train', {\n 'learning_rate': 0.11,\n 'momentum': 0.95\n }, 7),\n ]\n hook.writer.add_scalars.assert_has_calls(calls, any_order=True)\n\n\n@pytest.mark.parametrize('log_model', (True, False))\ndef test_mlflow_hook(log_model):\n sys.modules['mlflow'] = MagicMock()\n sys.modules['mlflow.pytorch'] = MagicMock()\n\n runner = _build_demo_runner()\n loader = DataLoader(torch.ones((5, 2)))\n\n hook = MlflowLoggerHook(exp_name='test', log_model=log_model)\n runner.register_hook(hook)\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n shutil.rmtree(runner.work_dir)\n\n hook.mlflow.set_experiment.assert_called_with('test')\n hook.mlflow.log_metrics.assert_called_with(\n {\n 'learning_rate': 0.02,\n 'momentum': 0.95\n }, step=6)\n if log_model:\n hook.mlflow_pytorch.log_model.assert_called_with(\n runner.model,\n 'models',\n pip_requirements=[f'torch=={TORCH_VERSION}'])\n else:\n assert not hook.mlflow_pytorch.log_model.called\n\n\ndef test_wandb_hook():\n sys.modules['wandb'] = MagicMock()\n runner = _build_demo_runner()\n hook = WandbLoggerHook(log_artifact=True)\n loader = DataLoader(torch.ones((5, 2)))\n\n runner.register_hook(hook)\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n\n shutil.rmtree(runner.work_dir)\n\n hook.wandb.init.assert_called_with()\n hook.wandb.log.assert_called_with({\n 'learning_rate': 0.02,\n 'momentum': 0.95\n },\n step=6,\n commit=True)\n hook.wandb.log_artifact.assert_called()\n hook.wandb.join.assert_called_with()\n\n\ndef test_neptune_hook():\n sys.modules['neptune'] = MagicMock()\n sys.modules['neptune.new'] = MagicMock()\n runner = _build_demo_runner()\n hook = NeptuneLoggerHook()\n\n loader = DataLoader(torch.ones((5, 2)))\n\n runner.register_hook(hook)\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n shutil.rmtree(runner.work_dir)\n\n hook.neptune.init.assert_called_with()\n hook.run['momentum'].log.assert_called_with(0.95, step=6)\n hook.run.stop.assert_called_with()\n\n\ndef test_dvclive_hook():\n sys.modules['dvclive'] = MagicMock()\n runner = _build_demo_runner()\n\n hook = DvcliveLoggerHook()\n dvclive_mock = hook.dvclive\n loader = DataLoader(torch.ones((5, 2)))\n\n runner.register_hook(hook)\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n shutil.rmtree(runner.work_dir)\n\n dvclive_mock.set_step.assert_called_with(6)\n dvclive_mock.log.assert_called_with('momentum', 0.95)\n\n\ndef test_dvclive_hook_model_file(tmp_path):\n sys.modules['dvclive'] = MagicMock()\n runner = _build_demo_runner()\n\n hook = DvcliveLoggerHook(model_file=osp.join(runner.work_dir, 'model.pth'))\n runner.register_hook(hook)\n\n loader = torch.utils.data.DataLoader(torch.ones((5, 2)))\n loader = DataLoader(torch.ones((5, 2)))\n\n runner.run([loader, loader], [('train', 1), ('val', 1)])\n\n assert osp.exists(osp.join(runner.work_dir, 'model.pth'))\n\n shutil.rmtree(runner.work_dir)\n\n\ndef _build_demo_runner_without_hook(runner_type='EpochBasedRunner',\n max_epochs=1,\n max_iters=None,\n multi_optimizers=False):\n\n class Model(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.linear = nn.Linear(2, 1)\n self.conv = nn.Conv2d(3, 3, 3)\n\n def forward(self, x):\n return self.linear(x)\n\n def train_step(self, x, optimizer, **kwargs):\n return dict(loss=self(x))\n\n def val_step(self, x, optimizer, **kwargs):\n return dict(loss=self(x))\n\n model = Model()\n\n if multi_optimizers:\n optimizer = {\n 'model1':\n torch.optim.SGD(model.linear.parameters(), lr=0.02, momentum=0.95),\n 'model2':\n torch.optim.SGD(model.conv.parameters(), lr=0.01, momentum=0.9),\n }\n else:\n optimizer = torch.optim.SGD(model.parameters(), lr=0.02, momentum=0.95)\n\n tmp_dir = tempfile.mkdtemp()\n runner = build_runner(\n dict(type=runner_type),\n default_args=dict(\n model=model,\n work_dir=tmp_dir,\n optimizer=optimizer,\n logger=logging.getLogger(),\n max_epochs=max_epochs,\n max_iters=max_iters))\n return runner\n\n\ndef _build_demo_runner(runner_type='EpochBasedRunner',\n max_epochs=1,\n max_iters=None,\n multi_optimizers=False):\n log_config = dict(\n interval=1, hooks=[\n dict(type='TextLoggerHook'),\n ])\n\n runner = _build_demo_runner_without_hook(runner_type, max_epochs,\n max_iters, multi_optimizers)\n\n runner.register_checkpoint_hook(dict(interval=1))\n runner.register_logger_hooks(log_config)\n return runner\n\n\ndef test_runner_with_revise_keys():\n import os\n\n class Model(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.conv = nn.Conv2d(3, 3, 1)\n\n class PrefixModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.backbone = Model()\n\n pmodel = PrefixModel()\n model = Model()\n checkpoint_path = os.path.join(tempfile.gettempdir(), 'checkpoint.pth')\n\n # add prefix\n torch.save(model.state_dict(), checkpoint_path)\n runner = _build_demo_runner(runner_type='EpochBasedRunner')\n runner.model = pmodel\n state_dict = runner.load_checkpoint(\n checkpoint_path, revise_keys=[(r'^', 'backbone.')])\n for key in pmodel.backbone.state_dict().keys():\n assert torch.equal(pmodel.backbone.state_dict()[key], state_dict[key])\n # strip prefix\n torch.save(pmodel.state_dict(), checkpoint_path)\n runner.model = model\n state_dict = runner.load_checkpoint(\n checkpoint_path, revise_keys=[(r'^backbone\\.', '')])\n for key in state_dict.keys():\n key_stripped = re.sub(r'^backbone\\.', '', key)\n assert torch.equal(model.state_dict()[key_stripped], state_dict[key])\n os.remove(checkpoint_path)\n\n\ndef test_get_triggered_stages():\n\n class ToyHook(Hook):\n # test normal stage\n def before_run():\n pass\n\n # test the method mapped to multi stages.\n def after_epoch():\n pass\n\n hook = ToyHook()\n # stages output have order, so here is list instead of set.\n expected_stages = ['before_run', 'after_train_epoch', 'after_val_epoch']\n assert hook.get_triggered_stages() == expected_stages\n\n\ndef test_gradient_cumulative_optimizer_hook():\n\n class ToyModel(nn.Module):\n\n def __init__(self, with_norm=False):\n super().__init__()\n self.fp16_enabled = False\n self.fc = nn.Linear(3, 2)\n nn.init.constant_(self.fc.weight, 1.)\n nn.init.constant_(self.fc.bias, 1.)\n self.with_norm = with_norm\n if with_norm:\n self.norm = nn.BatchNorm1d(2)\n\n def forward(self, x):\n x = self.fc(x)\n if self.with_norm:\n x = self.norm(x)\n return x\n\n def train_step(self, x, optimizer, **kwargs):\n return dict(loss=self(x).mean(), num_samples=x.shape[0])\n\n def val_step(self, x, optimizer, **kwargs):\n return dict(loss=self(x).mean(), num_samples=x.shape[0])\n\n def build_toy_runner(config=dict(type='EpochBasedRunner', max_epochs=3)):\n model = ToyModel()\n optimizer = torch.optim.SGD(model.parameters(), lr=0.02)\n tmp_dir = tempfile.mkdtemp()\n\n runner = build_runner(\n config,\n default_args=dict(\n model=model,\n work_dir=tmp_dir,\n optimizer=optimizer,\n logger=logging.getLogger(),\n meta=dict()))\n return runner\n\n with pytest.raises(AssertionError):\n # cumulative_iters only accepts int\n GradientCumulativeOptimizerHook(cumulative_iters='str')\n\n with pytest.raises(AssertionError):\n # cumulative_iters only accepts positive number\n GradientCumulativeOptimizerHook(cumulative_iters=-1)\n\n # test epoch based runner\n data = torch.rand((6, 3))\n # optimize with cumulative_iters\n loader_1 = DataLoader(data, batch_size=1)\n runner_1 = build_toy_runner()\n optimizer_hook = GradientCumulativeOptimizerHook(\n grad_clip=dict(max_norm=0.2), cumulative_iters=3)\n runner_1.register_hook(optimizer_hook)\n runner_1.run([loader_1], [('train', 1)])\n\n # optimize without cumulative_iters\n loader_2 = DataLoader(data, batch_size=3)\n runner_2 = build_toy_runner()\n optimizer_hook = OptimizerHook(grad_clip=dict(max_norm=0.2))\n runner_2.register_hook(optimizer_hook)\n runner_2.run([loader_2], [('train', 1)])\n\n # test optimizer works well\n assert (runner_1.model.fc.weight < 1).all()\n assert (runner_1.model.fc.bias < 1).all()\n # test optimizer with cumulative_iters gets the same results\n assert torch.allclose(runner_1.model.fc.weight, runner_2.model.fc.weight)\n assert torch.allclose(runner_1.model.fc.bias, runner_2.model.fc.bias)\n shutil.rmtree(runner_1.work_dir)\n shutil.rmtree(runner_2.work_dir)\n\n # test iter based runner\n data = torch.rand((8, 3))\n # optimize with cumulative_iters\n loader_1 = DataLoader(data, batch_size=1)\n runner_1 = build_toy_runner(dict(type='IterBasedRunner', max_iters=8))\n optimizer_hook = GradientCumulativeOptimizerHook(\n grad_clip=dict(max_norm=0.2), cumulative_iters=3)\n runner_1.register_hook(optimizer_hook)\n runner_1.run([loader_1], [('train', 1)])\n\n # optimize without cumulative_iters\n loader_2_divisible = DataLoader(data[:6], batch_size=3)\n loader_2_remainder = DataLoader(data[6:], batch_size=2)\n runner_2 = build_toy_runner(dict(type='IterBasedRunner', max_iters=3))\n optimizer_hook = OptimizerHook(grad_clip=dict(max_norm=0.2))\n runner_2.register_hook(optimizer_hook)\n runner_2.run([loader_2_divisible, loader_2_remainder], [('train', 2),\n ('train', 1)])\n\n # test optimizer works well\n assert (runner_1.model.fc.weight < 1).all()\n assert (runner_1.model.fc.bias < 1).all()\n # test optimizer with cumulative_iters gets the same results\n assert torch.allclose(runner_1.model.fc.weight, runner_2.model.fc.weight)\n assert torch.allclose(runner_1.model.fc.bias, runner_2.model.fc.bias)\n shutil.rmtree(runner_1.work_dir)\n shutil.rmtree(runner_2.work_dir)\n\n # test has_batch_norm\n model = ToyModel(with_norm=True)\n optimizer_hook = GradientCumulativeOptimizerHook(\n grad_clip=dict(max_norm=0.2), cumulative_iters=3)\n assert optimizer_hook.has_batch_norm(model)\n\n\n@pytest.mark.skipif(\n not torch.cuda.is_available(), reason='requires CUDA support')\ndef test_gradient_cumulative_fp16_optimizer_hook():\n\n class ToyModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.fp16_enabled = False\n self.fc = nn.Linear(3, 2)\n nn.init.constant_(self.fc.weight, 1.)\n nn.init.constant_(self.fc.bias, 1.)\n\n @auto_fp16(apply_to=('x', ))\n def forward(self, x):\n x = self.fc(x)\n return x\n\n def train_step(self, x, optimizer, **kwargs):\n return dict(loss=self(x).mean(), num_samples=x.shape[0])\n\n def val_step(self, x, optimizer, **kwargs):\n return dict(loss=self(x).mean(), num_samples=x.shape[0])\n\n def build_toy_runner(config=dict(type='EpochBasedRunner', max_epochs=3)):\n model = ToyModel().cuda()\n optimizer = torch.optim.SGD(model.parameters(), lr=0.02)\n tmp_dir = tempfile.mkdtemp()\n\n runner = build_runner(\n config,\n default_args=dict(\n model=model,\n work_dir=tmp_dir,\n optimizer=optimizer,\n logger=logging.getLogger(),\n meta=dict()))\n return runner\n\n # test epoch based runner\n data = torch.rand((6, 3)).cuda()\n # optimize with cumulative_iters\n loader_1 = DataLoader(data, batch_size=1)\n runner_1 = build_toy_runner()\n optimizer_hook = GradientCumulativeFp16OptimizerHook(\n grad_clip=dict(max_norm=0.2), cumulative_iters=3)\n runner_1.register_hook(optimizer_hook)\n runner_1.run([loader_1], [('train', 1)])\n\n # optimize without cumulative_iters\n loader_2 = DataLoader(data, batch_size=3)\n runner_2 = build_toy_runner()\n optimizer_hook = Fp16OptimizerHook(grad_clip=dict(max_norm=0.2))\n runner_2.register_hook(optimizer_hook)\n runner_2.run([loader_2], [('train', 1)])\n\n # test optimizer works well\n assert (runner_1.model.fc.weight < 1).all()\n assert (runner_1.model.fc.bias < 1).all()\n # test optimizer with cumulative_iters gets the same results\n assert torch.allclose(runner_1.model.fc.weight, runner_2.model.fc.weight)\n assert torch.allclose(runner_1.model.fc.bias, runner_2.model.fc.bias)\n shutil.rmtree(runner_1.work_dir)\n shutil.rmtree(runner_2.work_dir)\n\n # test iter based runner\n data = torch.rand((8, 3)).cuda()\n # optimize with cumulative_iters\n loader_1 = DataLoader(data, batch_size=1)\n runner_1 = build_toy_runner(dict(type='IterBasedRunner', max_iters=8))\n optimizer_hook = GradientCumulativeFp16OptimizerHook(\n grad_clip=dict(max_norm=0.2), cumulative_iters=3)\n runner_1.register_hook(optimizer_hook)\n runner_1.run([loader_1], [('train', 1)])\n\n # optimize without cumulative_iters\n loader_2_divisible = DataLoader(data[:6], batch_size=3)\n loader_2_remainder = DataLoader(data[6:], batch_size=2)\n runner_2 = build_toy_runner(dict(type='IterBasedRunner', max_iters=3))\n optimizer_hook = Fp16OptimizerHook(grad_clip=dict(max_norm=0.2))\n runner_2.register_hook(optimizer_hook)\n runner_2.run([loader_2_divisible, loader_2_remainder], [('train', 2),\n ('train', 1)])\n\n # test optimizer works well\n assert (runner_1.model.fc.weight < 1).all()\n assert (runner_1.model.fc.bias < 1).all()\n # test optimizer with cumulative_iters gets the same results\n assert torch.allclose(runner_1.model.fc.weight, runner_2.model.fc.weight)\n assert torch.allclose(runner_1.model.fc.bias, runner_2.model.fc.bias)\n shutil.rmtree(runner_1.work_dir)\n shutil.rmtree(runner_2.work_dir)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_optimizer.py", "content": "import sys\nimport warnings\nfrom unittest.mock import MagicMock\n\nimport pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.runner import OPTIMIZER_BUILDERS, DefaultOptimizerConstructor\nfrom mmcv.runner.optimizer import build_optimizer, build_optimizer_constructor\nfrom mmcv.runner.optimizer.builder import TORCH_OPTIMIZERS\nfrom mmcv.utils.ext_loader import check_ops_exist\n\nOPS_AVAILABLE = check_ops_exist()\nif not OPS_AVAILABLE:\n sys.modules['mmcv.ops'] = MagicMock(\n DeformConv2d=dict, ModulatedDeformConv2d=dict)\n\n\nclass SubModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.conv1 = nn.Conv2d(2, 2, kernel_size=1, groups=2)\n self.gn = nn.GroupNorm(2, 2)\n self.param1 = nn.Parameter(torch.ones(1))\n\n def forward(self, x):\n return x\n\n\nclass ExampleModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.param1 = nn.Parameter(torch.ones(1))\n self.conv1 = nn.Conv2d(3, 4, kernel_size=1, bias=False)\n self.conv2 = nn.Conv2d(4, 2, kernel_size=1)\n self.bn = nn.BatchNorm2d(2)\n self.sub = SubModel()\n if OPS_AVAILABLE:\n from mmcv.ops import DeformConv2dPack\n self.dcn = DeformConv2dPack(\n 3, 4, kernel_size=3, deformable_groups=1)\n\n def forward(self, x):\n return x\n\n\nclass ExampleDuplicateModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.param1 = nn.Parameter(torch.ones(1))\n self.conv1 = nn.Sequential(nn.Conv2d(3, 4, kernel_size=1, bias=False))\n self.conv2 = nn.Sequential(nn.Conv2d(4, 2, kernel_size=1))\n self.bn = nn.BatchNorm2d(2)\n self.sub = SubModel()\n self.conv3 = nn.Sequential(nn.Conv2d(3, 4, kernel_size=1, bias=False))\n self.conv3[0] = self.conv1[0]\n if OPS_AVAILABLE:\n from mmcv.ops import DeformConv2dPack\n self.dcn = DeformConv2dPack(\n 3, 4, kernel_size=3, deformable_groups=1)\n\n def forward(self, x):\n return x\n\n\nclass PseudoDataParallel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.module = ExampleModel()\n\n def forward(self, x):\n return x\n\n\nbase_lr = 0.01\nbase_wd = 0.0001\nmomentum = 0.9\n\n\ndef check_default_optimizer(optimizer, model, prefix=''):\n assert isinstance(optimizer, torch.optim.SGD)\n assert optimizer.defaults['lr'] == base_lr\n assert optimizer.defaults['momentum'] == momentum\n assert optimizer.defaults['weight_decay'] == base_wd\n param_groups = optimizer.param_groups[0]\n if OPS_AVAILABLE:\n param_names = [\n 'param1', 'conv1.weight', 'conv2.weight', 'conv2.bias',\n 'bn.weight', 'bn.bias', 'sub.param1', 'sub.conv1.weight',\n 'sub.conv1.bias', 'sub.gn.weight', 'sub.gn.bias', 'dcn.weight',\n 'dcn.conv_offset.weight', 'dcn.conv_offset.bias'\n ]\n else:\n param_names = [\n 'param1', 'conv1.weight', 'conv2.weight', 'conv2.bias',\n 'bn.weight', 'bn.bias', 'sub.param1', 'sub.conv1.weight',\n 'sub.conv1.bias', 'sub.gn.weight', 'sub.gn.bias'\n ]\n param_dict = dict(model.named_parameters())\n assert len(param_groups['params']) == len(param_names)\n for i in range(len(param_groups['params'])):\n assert torch.equal(param_groups['params'][i],\n param_dict[prefix + param_names[i]])\n\n\ndef check_sgd_optimizer(optimizer,\n model,\n prefix='',\n bias_lr_mult=1,\n bias_decay_mult=1,\n norm_decay_mult=1,\n dwconv_decay_mult=1,\n dcn_offset_lr_mult=1,\n bypass_duplicate=False):\n param_groups = optimizer.param_groups\n assert isinstance(optimizer, torch.optim.SGD)\n assert optimizer.defaults['lr'] == base_lr\n assert optimizer.defaults['momentum'] == momentum\n assert optimizer.defaults['weight_decay'] == base_wd\n model_parameters = list(model.parameters())\n assert len(param_groups) == len(model_parameters)\n for i, param in enumerate(model_parameters):\n param_group = param_groups[i]\n assert torch.equal(param_group['params'][0], param)\n assert param_group['momentum'] == momentum\n\n # param1\n param1 = param_groups[0]\n assert param1['lr'] == base_lr\n assert param1['weight_decay'] == base_wd\n # conv1.weight\n conv1_weight = param_groups[1]\n assert conv1_weight['lr'] == base_lr\n assert conv1_weight['weight_decay'] == base_wd\n # conv2.weight\n conv2_weight = param_groups[2]\n assert conv2_weight['lr'] == base_lr\n assert conv2_weight['weight_decay'] == base_wd\n # conv2.bias\n conv2_bias = param_groups[3]\n assert conv2_bias['lr'] == base_lr * bias_lr_mult\n assert conv2_bias['weight_decay'] == base_wd * bias_decay_mult\n # bn.weight\n bn_weight = param_groups[4]\n assert bn_weight['lr'] == base_lr\n assert bn_weight['weight_decay'] == base_wd * norm_decay_mult\n # bn.bias\n bn_bias = param_groups[5]\n assert bn_bias['lr'] == base_lr\n assert bn_bias['weight_decay'] == base_wd * norm_decay_mult\n # sub.param1\n sub_param1 = param_groups[6]\n assert sub_param1['lr'] == base_lr\n assert sub_param1['weight_decay'] == base_wd\n # sub.conv1.weight\n sub_conv1_weight = param_groups[7]\n assert sub_conv1_weight['lr'] == base_lr\n assert sub_conv1_weight['weight_decay'] == base_wd * dwconv_decay_mult\n # sub.conv1.bias\n sub_conv1_bias = param_groups[8]\n assert sub_conv1_bias['lr'] == base_lr * bias_lr_mult\n assert sub_conv1_bias['weight_decay'] == base_wd * dwconv_decay_mult\n # sub.gn.weight\n sub_gn_weight = param_groups[9]\n assert sub_gn_weight['lr'] == base_lr\n assert sub_gn_weight['weight_decay'] == base_wd * norm_decay_mult\n # sub.gn.bias\n sub_gn_bias = param_groups[10]\n assert sub_gn_bias['lr'] == base_lr\n assert sub_gn_bias['weight_decay'] == base_wd * norm_decay_mult\n\n if torch.cuda.is_available():\n dcn_conv_weight = param_groups[11]\n assert dcn_conv_weight['lr'] == base_lr\n assert dcn_conv_weight['weight_decay'] == base_wd\n\n dcn_offset_weight = param_groups[12]\n assert dcn_offset_weight['lr'] == base_lr * dcn_offset_lr_mult\n assert dcn_offset_weight['weight_decay'] == base_wd\n\n dcn_offset_bias = param_groups[13]\n assert dcn_offset_bias['lr'] == base_lr * dcn_offset_lr_mult\n assert dcn_offset_bias['weight_decay'] == base_wd\n\n\ndef test_default_optimizer_constructor():\n model = ExampleModel()\n\n with pytest.raises(TypeError):\n # optimizer_cfg must be a dict\n optimizer_cfg = []\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg)\n optim_constructor(model)\n\n with pytest.raises(TypeError):\n # paramwise_cfg must be a dict or None\n optimizer_cfg = dict(lr=0.0001)\n paramwise_cfg = ['error']\n optim_constructor = DefaultOptimizerConstructor(\n optimizer_cfg, paramwise_cfg)\n optim_constructor(model)\n\n with pytest.raises(ValueError):\n # bias_decay_mult/norm_decay_mult is specified but weight_decay is None\n optimizer_cfg = dict(lr=0.0001, weight_decay=None)\n paramwise_cfg = dict(bias_decay_mult=1, norm_decay_mult=1)\n optim_constructor = DefaultOptimizerConstructor(\n optimizer_cfg, paramwise_cfg)\n optim_constructor(model)\n\n # basic config with ExampleModel\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg)\n optimizer = optim_constructor(model)\n check_default_optimizer(optimizer, model)\n\n # basic config with pseudo data parallel\n model = PseudoDataParallel()\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = None\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg)\n optimizer = optim_constructor(model)\n check_default_optimizer(optimizer, model, prefix='module.')\n\n # basic config with DataParallel\n if torch.cuda.is_available():\n model = torch.nn.DataParallel(ExampleModel())\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = None\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg)\n optimizer = optim_constructor(model)\n check_default_optimizer(optimizer, model, prefix='module.')\n\n # Empty paramwise_cfg with ExampleModel\n model = ExampleModel()\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = dict()\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg,\n paramwise_cfg)\n optimizer = optim_constructor(model)\n check_default_optimizer(optimizer, model)\n\n # Empty paramwise_cfg with ExampleModel and no grad\n model = ExampleModel()\n for param in model.parameters():\n param.requires_grad = False\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = dict()\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg)\n optimizer = optim_constructor(model)\n check_default_optimizer(optimizer, model)\n\n # paramwise_cfg with ExampleModel\n model = ExampleModel()\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = dict(\n bias_lr_mult=2,\n bias_decay_mult=0.5,\n norm_decay_mult=0,\n dwconv_decay_mult=0.1,\n dcn_offset_lr_mult=0.1)\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg,\n paramwise_cfg)\n optimizer = optim_constructor(model)\n check_sgd_optimizer(optimizer, model, **paramwise_cfg)\n\n # paramwise_cfg with ExampleModel, weight decay is None\n model = ExampleModel()\n optimizer_cfg = dict(type='Rprop', lr=base_lr)\n paramwise_cfg = dict(bias_lr_mult=2)\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg,\n paramwise_cfg)\n optimizer = optim_constructor(model)\n\n param_groups = optimizer.param_groups\n assert isinstance(optimizer, torch.optim.Rprop)\n assert optimizer.defaults['lr'] == base_lr\n model_parameters = list(model.parameters())\n assert len(param_groups) == len(model_parameters)\n for i, param in enumerate(model_parameters):\n param_group = param_groups[i]\n assert torch.equal(param_group['params'][0], param)\n # param1\n assert param_groups[0]['lr'] == base_lr\n # conv1.weight\n assert param_groups[1]['lr'] == base_lr\n # conv2.weight\n assert param_groups[2]['lr'] == base_lr\n # conv2.bias\n assert param_groups[3]['lr'] == base_lr * paramwise_cfg['bias_lr_mult']\n # bn.weight\n assert param_groups[4]['lr'] == base_lr\n # bn.bias\n assert param_groups[5]['lr'] == base_lr\n # sub.param1\n assert param_groups[6]['lr'] == base_lr\n # sub.conv1.weight\n assert param_groups[7]['lr'] == base_lr\n # sub.conv1.bias\n assert param_groups[8]['lr'] == base_lr * paramwise_cfg['bias_lr_mult']\n # sub.gn.weight\n assert param_groups[9]['lr'] == base_lr\n # sub.gn.bias\n assert param_groups[10]['lr'] == base_lr\n\n if OPS_AVAILABLE:\n # dcn.weight\n assert param_groups[11]['lr'] == base_lr\n # dcn.conv_offset.weight\n assert param_groups[12]['lr'] == base_lr\n # dcn.conv_offset.bias\n assert param_groups[13]['lr'] == base_lr\n\n # paramwise_cfg with pseudo data parallel\n model = PseudoDataParallel()\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = dict(\n bias_lr_mult=2,\n bias_decay_mult=0.5,\n norm_decay_mult=0,\n dwconv_decay_mult=0.1,\n dcn_offset_lr_mult=0.1)\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg,\n paramwise_cfg)\n optimizer = optim_constructor(model)\n check_sgd_optimizer(optimizer, model, prefix='module.', **paramwise_cfg)\n\n # paramwise_cfg with DataParallel\n if torch.cuda.is_available():\n model = torch.nn.DataParallel(ExampleModel())\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = dict(\n bias_lr_mult=2,\n bias_decay_mult=0.5,\n norm_decay_mult=0,\n dwconv_decay_mult=0.1,\n dcn_offset_lr_mult=0.1)\n optim_constructor = DefaultOptimizerConstructor(\n optimizer_cfg, paramwise_cfg)\n optimizer = optim_constructor(model)\n check_sgd_optimizer(\n optimizer, model, prefix='module.', **paramwise_cfg)\n\n # paramwise_cfg with ExampleModel and no grad\n for param in model.parameters():\n param.requires_grad = False\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg,\n paramwise_cfg)\n optimizer = optim_constructor(model)\n param_groups = optimizer.param_groups\n assert isinstance(optimizer, torch.optim.SGD)\n assert optimizer.defaults['lr'] == base_lr\n assert optimizer.defaults['momentum'] == momentum\n assert optimizer.defaults['weight_decay'] == base_wd\n for i, (name, param) in enumerate(model.named_parameters()):\n param_group = param_groups[i]\n assert torch.equal(param_group['params'][0], param)\n assert param_group['momentum'] == momentum\n assert param_group['lr'] == base_lr\n assert param_group['weight_decay'] == base_wd\n\n # paramwise_cfg with bypass_duplicate option\n model = ExampleDuplicateModel()\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = dict(\n bias_lr_mult=2,\n bias_decay_mult=0.5,\n norm_decay_mult=0,\n dwconv_decay_mult=0.1)\n with pytest.raises(ValueError) as excinfo:\n optim_constructor = DefaultOptimizerConstructor(\n optimizer_cfg, paramwise_cfg)\n optim_constructor(model)\n assert 'some parameters appear in more than one parameter ' \\\n 'group' == excinfo.value\n\n paramwise_cfg = dict(\n bias_lr_mult=2,\n bias_decay_mult=0.5,\n norm_decay_mult=0,\n dwconv_decay_mult=0.1,\n dcn_offset_lr_mult=0.1,\n bypass_duplicate=True)\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg,\n paramwise_cfg)\n with warnings.catch_warnings(record=True) as w:\n optimizer = optim_constructor(model)\n warnings.simplefilter('always')\n assert len(w) == 1\n assert str(w[0].message) == 'conv3.0 is duplicate. It is skipped ' \\\n 'since bypass_duplicate=True'\n model_parameters = list(model.parameters())\n num_params = 14 if OPS_AVAILABLE else 11\n assert len(optimizer.param_groups) == len(model_parameters) == num_params\n check_sgd_optimizer(optimizer, model, **paramwise_cfg)\n\n # test DefaultOptimizerConstructor with custom_keys and ExampleModel\n model = ExampleModel()\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = dict(\n custom_keys={\n 'param1': dict(lr_mult=10),\n 'sub': dict(lr_mult=0.1, decay_mult=0),\n 'sub.gn': dict(lr_mult=0.01),\n 'non_exist_key': dict(lr_mult=0.0)\n },\n norm_decay_mult=0.5)\n\n with pytest.raises(TypeError):\n # custom_keys should be a dict\n paramwise_cfg_ = dict(custom_keys=[0.1, 0.0001])\n optim_constructor = DefaultOptimizerConstructor(\n optimizer_cfg, paramwise_cfg_)\n optimizer = optim_constructor(model)\n\n with pytest.raises(ValueError):\n # if 'decay_mult' is specified in custom_keys, weight_decay should be\n # specified\n optimizer_cfg_ = dict(type='SGD', lr=0.01)\n paramwise_cfg_ = dict(custom_keys={'.backbone': dict(decay_mult=0.5)})\n optim_constructor = DefaultOptimizerConstructor(\n optimizer_cfg_, paramwise_cfg_)\n optimizer = optim_constructor(model)\n\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg,\n paramwise_cfg)\n optimizer = optim_constructor(model)\n # check optimizer type and default config\n assert isinstance(optimizer, torch.optim.SGD)\n assert optimizer.defaults['lr'] == base_lr\n assert optimizer.defaults['momentum'] == momentum\n assert optimizer.defaults['weight_decay'] == base_wd\n\n # check params groups\n param_groups = optimizer.param_groups\n\n groups = []\n group_settings = []\n # group 1, matches of 'param1'\n # 'param1' is the longest match for 'sub.param1'\n groups.append(['param1', 'sub.param1'])\n group_settings.append({\n 'lr': base_lr * 10,\n 'momentum': momentum,\n 'weight_decay': base_wd,\n })\n # group 2, matches of 'sub.gn'\n groups.append(['sub.gn.weight', 'sub.gn.bias'])\n group_settings.append({\n 'lr': base_lr * 0.01,\n 'momentum': momentum,\n 'weight_decay': base_wd,\n })\n # group 3, matches of 'sub'\n groups.append(['sub.conv1.weight', 'sub.conv1.bias'])\n group_settings.append({\n 'lr': base_lr * 0.1,\n 'momentum': momentum,\n 'weight_decay': 0,\n })\n # group 4, bn is configured by 'norm_decay_mult'\n groups.append(['bn.weight', 'bn.bias'])\n group_settings.append({\n 'lr': base_lr,\n 'momentum': momentum,\n 'weight_decay': base_wd * 0.5,\n })\n # group 5, default group\n groups.append(['conv1.weight', 'conv2.weight', 'conv2.bias'])\n group_settings.append({\n 'lr': base_lr,\n 'momentum': momentum,\n 'weight_decay': base_wd\n })\n\n num_params = 14 if OPS_AVAILABLE else 11\n assert len(param_groups) == num_params\n for i, (name, param) in enumerate(model.named_parameters()):\n assert torch.equal(param_groups[i]['params'][0], param)\n for group, settings in zip(groups, group_settings):\n if name in group:\n for setting in settings:\n assert param_groups[i][setting] == settings[\n setting], f'{name} {setting}'\n\n # test DefaultOptimizerConstructor with custom_keys and ExampleModel 2\n model = ExampleModel()\n optimizer_cfg = dict(type='SGD', lr=base_lr, momentum=momentum)\n paramwise_cfg = dict(custom_keys={'param1': dict(lr_mult=10)})\n\n optim_constructor = DefaultOptimizerConstructor(optimizer_cfg,\n paramwise_cfg)\n optimizer = optim_constructor(model)\n # check optimizer type and default config\n assert isinstance(optimizer, torch.optim.SGD)\n assert optimizer.defaults['lr'] == base_lr\n assert optimizer.defaults['momentum'] == momentum\n assert optimizer.defaults['weight_decay'] == 0\n\n # check params groups\n param_groups = optimizer.param_groups\n\n groups = []\n group_settings = []\n # group 1, matches of 'param1'\n groups.append(['param1', 'sub.param1'])\n group_settings.append({\n 'lr': base_lr * 10,\n 'momentum': momentum,\n 'weight_decay': 0,\n })\n # group 2, default group\n groups.append([\n 'sub.conv1.weight', 'sub.conv1.bias', 'sub.gn.weight', 'sub.gn.bias',\n 'conv1.weight', 'conv2.weight', 'conv2.bias', 'bn.weight', 'bn.bias'\n ])\n group_settings.append({\n 'lr': base_lr,\n 'momentum': momentum,\n 'weight_decay': 0\n })\n\n num_params = 14 if OPS_AVAILABLE else 11\n assert len(param_groups) == num_params\n for i, (name, param) in enumerate(model.named_parameters()):\n assert torch.equal(param_groups[i]['params'][0], param)\n for group, settings in zip(groups, group_settings):\n if name in group:\n for setting in settings:\n assert param_groups[i][setting] == settings[\n setting], f'{name} {setting}'\n\n\ndef test_torch_optimizers():\n torch_optimizers = [\n 'ASGD', 'Adadelta', 'Adagrad', 'Adam', 'AdamW', 'Adamax', 'LBFGS',\n 'Optimizer', 'RMSprop', 'Rprop', 'SGD', 'SparseAdam'\n ]\n assert set(torch_optimizers).issubset(set(TORCH_OPTIMIZERS))\n\n\ndef test_build_optimizer_constructor():\n model = ExampleModel()\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n paramwise_cfg = dict(\n bias_lr_mult=2,\n bias_decay_mult=0.5,\n norm_decay_mult=0,\n dwconv_decay_mult=0.1,\n dcn_offset_lr_mult=0.1)\n optim_constructor_cfg = dict(\n type='DefaultOptimizerConstructor',\n optimizer_cfg=optimizer_cfg,\n paramwise_cfg=paramwise_cfg)\n optim_constructor = build_optimizer_constructor(optim_constructor_cfg)\n optimizer = optim_constructor(model)\n check_sgd_optimizer(optimizer, model, **paramwise_cfg)\n\n from mmcv.runner import OPTIMIZERS\n from mmcv.utils import build_from_cfg\n\n @OPTIMIZER_BUILDERS.register_module()\n class MyOptimizerConstructor(DefaultOptimizerConstructor):\n\n def __call__(self, model):\n if hasattr(model, 'module'):\n model = model.module\n\n conv1_lr_mult = self.paramwise_cfg.get('conv1_lr_mult', 1.)\n\n params = []\n for name, param in model.named_parameters():\n param_group = {'params': [param]}\n if name.startswith('conv1') and param.requires_grad:\n param_group['lr'] = self.base_lr * conv1_lr_mult\n params.append(param_group)\n optimizer_cfg['params'] = params\n\n return build_from_cfg(optimizer_cfg, OPTIMIZERS)\n\n paramwise_cfg = dict(conv1_lr_mult=5)\n optim_constructor_cfg = dict(\n type='MyOptimizerConstructor',\n optimizer_cfg=optimizer_cfg,\n paramwise_cfg=paramwise_cfg)\n optim_constructor = build_optimizer_constructor(optim_constructor_cfg)\n optimizer = optim_constructor(model)\n\n param_groups = optimizer.param_groups\n assert isinstance(optimizer, torch.optim.SGD)\n assert optimizer.defaults['lr'] == base_lr\n assert optimizer.defaults['momentum'] == momentum\n assert optimizer.defaults['weight_decay'] == base_wd\n for i, param in enumerate(model.parameters()):\n param_group = param_groups[i]\n assert torch.equal(param_group['params'][0], param)\n assert param_group['momentum'] == momentum\n # conv1.weight\n assert param_groups[1]['lr'] == base_lr * paramwise_cfg['conv1_lr_mult']\n assert param_groups[1]['weight_decay'] == base_wd\n\n\ndef test_build_optimizer():\n model = ExampleModel()\n optimizer_cfg = dict(\n type='SGD', lr=base_lr, weight_decay=base_wd, momentum=momentum)\n optimizer = build_optimizer(model, optimizer_cfg)\n check_default_optimizer(optimizer, model)\n\n model = ExampleModel()\n optimizer_cfg = dict(\n type='SGD',\n lr=base_lr,\n weight_decay=base_wd,\n momentum=momentum,\n paramwise_cfg=dict(\n bias_lr_mult=2,\n bias_decay_mult=0.5,\n norm_decay_mult=0,\n dwconv_decay_mult=0.1,\n dcn_offset_lr_mult=0.1))\n optimizer = build_optimizer(model, optimizer_cfg)\n check_sgd_optimizer(optimizer, model, **optimizer_cfg['paramwise_cfg'])\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_runner.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport logging\nimport os\nimport os.path as osp\nimport platform\nimport random\nimport string\nimport tempfile\n\nimport pytest\nimport torch\nimport torch.nn as nn\n\nfrom mmcv.parallel import MMDataParallel\nfrom mmcv.runner import (RUNNERS, EpochBasedRunner, IterBasedRunner,\n build_runner)\nfrom mmcv.runner.hooks import IterTimerHook\n\n\nclass OldStyleModel(nn.Module):\n\n def __init__(self):\n super().__init__()\n self.conv = nn.Conv2d(3, 3, 1)\n\n\nclass Model(OldStyleModel):\n\n def train_step(self):\n pass\n\n def val_step(self):\n pass\n\n\ndef test_build_runner():\n temp_root = tempfile.gettempdir()\n dir_name = ''.join(\n [random.choice(string.ascii_letters) for _ in range(10)])\n\n default_args = dict(\n model=Model(),\n work_dir=osp.join(temp_root, dir_name),\n logger=logging.getLogger())\n cfg = dict(type='EpochBasedRunner', max_epochs=1)\n runner = build_runner(cfg, default_args=default_args)\n assert runner._max_epochs == 1\n cfg = dict(type='IterBasedRunner', max_iters=1)\n runner = build_runner(cfg, default_args=default_args)\n assert runner._max_iters == 1\n\n with pytest.raises(ValueError, match='Only one of'):\n cfg = dict(type='IterBasedRunner', max_epochs=1, max_iters=1)\n runner = build_runner(cfg, default_args=default_args)\n\n\n@pytest.mark.parametrize('runner_class', RUNNERS.module_dict.values())\ndef test_epoch_based_runner(runner_class):\n\n with pytest.warns(DeprecationWarning):\n # batch_processor is deprecated\n model = OldStyleModel()\n\n def batch_processor():\n pass\n\n _ = runner_class(model, batch_processor, logger=logging.getLogger())\n\n with pytest.raises(TypeError):\n # batch_processor must be callable\n model = OldStyleModel()\n _ = runner_class(model, batch_processor=0, logger=logging.getLogger())\n\n with pytest.raises(TypeError):\n # optimizer must be a optimizer or a dict of optimizers\n model = Model()\n optimizer = 'NotAOptimizer'\n _ = runner_class(\n model, optimizer=optimizer, logger=logging.getLogger())\n\n with pytest.raises(TypeError):\n # optimizer must be a optimizer or a dict of optimizers\n model = Model()\n optimizers = dict(optim1=torch.optim.Adam(), optim2='NotAOptimizer')\n _ = runner_class(\n model, optimizer=optimizers, logger=logging.getLogger())\n\n with pytest.raises(TypeError):\n # logger must be a logging.Logger\n model = Model()\n _ = runner_class(model, logger=None)\n\n with pytest.raises(TypeError):\n # meta must be a dict or None\n model = Model()\n _ = runner_class(model, logger=logging.getLogger(), meta=['list'])\n\n with pytest.raises(AssertionError):\n # model must implement the method train_step()\n model = OldStyleModel()\n _ = runner_class(model, logger=logging.getLogger())\n\n with pytest.raises(TypeError):\n # work_dir must be a str or None\n model = Model()\n _ = runner_class(model, work_dir=1, logger=logging.getLogger())\n\n with pytest.raises(RuntimeError):\n # batch_processor and train_step() cannot be both set\n\n def batch_processor():\n pass\n\n model = Model()\n _ = runner_class(model, batch_processor, logger=logging.getLogger())\n\n # test work_dir\n model = Model()\n temp_root = tempfile.gettempdir()\n dir_name = ''.join(\n [random.choice(string.ascii_letters) for _ in range(10)])\n work_dir = osp.join(temp_root, dir_name)\n _ = runner_class(model, work_dir=work_dir, logger=logging.getLogger())\n assert osp.isdir(work_dir)\n _ = runner_class(model, work_dir=work_dir, logger=logging.getLogger())\n assert osp.isdir(work_dir)\n os.removedirs(work_dir)\n\n\n@pytest.mark.parametrize('runner_class', RUNNERS.module_dict.values())\ndef test_runner_with_parallel(runner_class):\n\n def batch_processor():\n pass\n\n model = MMDataParallel(OldStyleModel())\n _ = runner_class(model, batch_processor, logger=logging.getLogger())\n\n model = MMDataParallel(Model())\n _ = runner_class(model, logger=logging.getLogger())\n\n with pytest.raises(RuntimeError):\n # batch_processor and train_step() cannot be both set\n\n def batch_processor():\n pass\n\n model = MMDataParallel(Model())\n _ = runner_class(model, batch_processor, logger=logging.getLogger())\n\n\n@pytest.mark.parametrize('runner_class', RUNNERS.module_dict.values())\ndef test_save_checkpoint(runner_class):\n model = Model()\n runner = runner_class(model=model, logger=logging.getLogger())\n\n with pytest.raises(TypeError):\n # meta should be None or dict\n runner.save_checkpoint('.', meta=list())\n\n with tempfile.TemporaryDirectory() as root:\n runner.save_checkpoint(root)\n\n latest_path = osp.join(root, 'latest.pth')\n assert osp.exists(latest_path)\n\n if isinstance(runner, EpochBasedRunner):\n first_ckp_path = osp.join(root, 'epoch_1.pth')\n elif isinstance(runner, IterBasedRunner):\n first_ckp_path = osp.join(root, 'iter_1.pth')\n\n assert osp.exists(first_ckp_path)\n\n if platform.system() != 'Windows':\n assert osp.realpath(latest_path) == osp.realpath(first_ckp_path)\n else:\n # use copy instead of symlink on windows\n pass\n\n torch.load(latest_path)\n\n\n@pytest.mark.parametrize('runner_class', RUNNERS.module_dict.values())\ndef test_build_lr_momentum_hook(runner_class):\n model = Model()\n runner = runner_class(model=model, logger=logging.getLogger())\n\n # test policy that is already title\n lr_config = dict(\n policy='CosineAnnealing',\n by_epoch=False,\n min_lr_ratio=0,\n warmup_iters=2,\n warmup_ratio=0.9)\n runner.register_lr_hook(lr_config)\n assert len(runner.hooks) == 1\n\n # test policy that is already title\n lr_config = dict(\n policy='Cyclic',\n by_epoch=False,\n target_ratio=(10, 1),\n cyclic_times=1,\n step_ratio_up=0.4)\n runner.register_lr_hook(lr_config)\n assert len(runner.hooks) == 2\n\n # test policy that is not title\n lr_config = dict(\n policy='cyclic',\n by_epoch=False,\n target_ratio=(0.85 / 0.95, 1),\n cyclic_times=1,\n step_ratio_up=0.4)\n runner.register_lr_hook(lr_config)\n assert len(runner.hooks) == 3\n\n # test policy that is title\n lr_config = dict(\n policy='Step',\n warmup='linear',\n warmup_iters=500,\n warmup_ratio=1.0 / 3,\n step=[8, 11])\n runner.register_lr_hook(lr_config)\n assert len(runner.hooks) == 4\n\n # test policy that is not title\n lr_config = dict(\n policy='step',\n warmup='linear',\n warmup_iters=500,\n warmup_ratio=1.0 / 3,\n step=[8, 11])\n runner.register_lr_hook(lr_config)\n assert len(runner.hooks) == 5\n\n # test policy that is already title\n mom_config = dict(\n policy='CosineAnnealing',\n min_momentum_ratio=0.99 / 0.95,\n by_epoch=False,\n warmup_iters=2,\n warmup_ratio=0.9 / 0.95)\n runner.register_momentum_hook(mom_config)\n assert len(runner.hooks) == 6\n\n # test policy that is already title\n mom_config = dict(\n policy='Cyclic',\n by_epoch=False,\n target_ratio=(0.85 / 0.95, 1),\n cyclic_times=1,\n step_ratio_up=0.4)\n runner.register_momentum_hook(mom_config)\n assert len(runner.hooks) == 7\n\n # test policy that is already title\n mom_config = dict(\n policy='cyclic',\n by_epoch=False,\n target_ratio=(0.85 / 0.95, 1),\n cyclic_times=1,\n step_ratio_up=0.4)\n runner.register_momentum_hook(mom_config)\n assert len(runner.hooks) == 8\n\n\n@pytest.mark.parametrize('runner_class', RUNNERS.module_dict.values())\ndef test_register_timer_hook(runner_class):\n model = Model()\n runner = runner_class(model=model, logger=logging.getLogger())\n\n # test register None\n timer_config = None\n runner.register_timer_hook(timer_config)\n assert len(runner.hooks) == 0\n\n # test register IterTimerHook with config\n timer_config = dict(type='IterTimerHook')\n runner.register_timer_hook(timer_config)\n assert len(runner.hooks) == 1\n assert isinstance(runner.hooks[0], IterTimerHook)\n\n # test register IterTimerHook\n timer_config = IterTimerHook()\n runner.register_timer_hook(timer_config)\n assert len(runner.hooks) == 2\n assert isinstance(runner.hooks[1], IterTimerHook)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_runner/test_utils.py", "content": "import os\nimport random\n\nimport numpy as np\nimport torch\n\nfrom mmcv.runner import set_random_seed\nfrom mmcv.utils import TORCH_VERSION, digit_version\n\nis_rocm_pytorch = False\nif digit_version(TORCH_VERSION) >= digit_version('1.5'):\n from torch.utils.cpp_extension import ROCM_HOME\n is_rocm_pytorch = True if ((torch.version.hip is not None) and\n (ROCM_HOME is not None)) else False\n\n\ndef test_set_random_seed():\n set_random_seed(0)\n a_random = random.randint(0, 10)\n a_np_random = np.random.rand(2, 2)\n a_torch_random = torch.rand(2, 2)\n assert torch.backends.cudnn.deterministic is False\n assert torch.backends.cudnn.benchmark is False\n assert os.environ['PYTHONHASHSEED'] == str(0)\n\n set_random_seed(0, True)\n b_random = random.randint(0, 10)\n b_np_random = np.random.rand(2, 2)\n b_torch_random = torch.rand(2, 2)\n assert torch.backends.cudnn.deterministic is True\n if is_rocm_pytorch:\n assert torch.backends.cudnn.benchmark is True\n else:\n assert torch.backends.cudnn.benchmark is False\n\n assert a_random == b_random\n assert np.equal(a_np_random, b_np_random).all()\n assert torch.equal(a_torch_random, b_torch_random)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_config.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport argparse\nimport json\nimport os\nimport os.path as osp\nimport shutil\nimport tempfile\nfrom pathlib import Path\n\nimport pytest\nimport yaml\n\nfrom mmcv import Config, ConfigDict, DictAction, dump, load\n\ndata_path = osp.join(osp.dirname(osp.dirname(__file__)), 'data')\n\n\ndef test_construct():\n cfg = Config()\n assert cfg.filename is None\n assert cfg.text == ''\n assert len(cfg) == 0\n assert cfg._cfg_dict == {}\n\n with pytest.raises(TypeError):\n Config([0, 1])\n\n cfg_dict = dict(item1=[1, 2], item2=dict(a=0), item3=True, item4='test')\n # test a.py\n cfg_file = osp.join(data_path, 'config/a.py')\n cfg = Config(cfg_dict, filename=cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n assert cfg.text == open(cfg_file, 'r').read()\n assert cfg.dump() == cfg.pretty_text\n with tempfile.TemporaryDirectory() as temp_config_dir:\n dump_file = osp.join(temp_config_dir, 'a.py')\n cfg.dump(dump_file)\n assert cfg.dump() == open(dump_file, 'r').read()\n assert Config.fromfile(dump_file)\n\n # test b.json\n cfg_file = osp.join(data_path, 'config/b.json')\n cfg = Config(cfg_dict, filename=cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n assert cfg.text == open(cfg_file, 'r').read()\n assert cfg.dump() == json.dumps(cfg_dict)\n with tempfile.TemporaryDirectory() as temp_config_dir:\n dump_file = osp.join(temp_config_dir, 'b.json')\n cfg.dump(dump_file)\n assert cfg.dump() == open(dump_file, 'r').read()\n assert Config.fromfile(dump_file)\n\n # test c.yaml\n cfg_file = osp.join(data_path, 'config/c.yaml')\n cfg = Config(cfg_dict, filename=cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n assert cfg.text == open(cfg_file, 'r').read()\n assert cfg.dump() == yaml.dump(cfg_dict)\n with tempfile.TemporaryDirectory() as temp_config_dir:\n dump_file = osp.join(temp_config_dir, 'c.yaml')\n cfg.dump(dump_file)\n assert cfg.dump() == open(dump_file, 'r').read()\n assert Config.fromfile(dump_file)\n\n # test h.py\n cfg_file = osp.join(data_path, 'config/h.py')\n path = osp.join(osp.dirname(__file__), 'data', 'config')\n # the value of osp.dirname(__file__) may be `D:\\a\\xxx` in windows\n # environment. When dumping the cfg_dict to file, `D:\\a\\xxx` will be\n # converted to `D:\\x07\\xxx` and it will cause unexpected result when\n # checking whether `D:\\a\\xxx` equals to `D:\\x07\\xxx`. Therefore, we forcely\n # convert a string representation of the path with forward slashes (/)\n path = Path(path).as_posix()\n cfg_dict = dict(item1='h.py', item2=path, item3='abc_h')\n cfg = Config(cfg_dict, filename=cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n assert cfg.text == open(cfg_file, 'r').read()\n assert cfg.dump() == cfg.pretty_text\n with tempfile.TemporaryDirectory() as temp_config_dir:\n dump_file = osp.join(temp_config_dir, 'h.py')\n cfg.dump(dump_file)\n assert cfg.dump() == open(dump_file, 'r').read()\n assert Config.fromfile(dump_file)\n assert Config.fromfile(dump_file)['item1'] == cfg_dict['item1']\n assert Config.fromfile(dump_file)['item2'] == cfg_dict['item2']\n assert Config.fromfile(dump_file)['item3'] == cfg_dict['item3']\n\n # test no use_predefined_variable\n cfg_dict = dict(\n item1='{{fileBasename}}',\n item2='{{ fileDirname}}',\n item3='abc_{{ fileBasenameNoExtension }}')\n assert Config.fromfile(cfg_file, False)\n assert Config.fromfile(cfg_file, False)['item1'] == cfg_dict['item1']\n assert Config.fromfile(cfg_file, False)['item2'] == cfg_dict['item2']\n assert Config.fromfile(cfg_file, False)['item3'] == cfg_dict['item3']\n\n # test p.yaml\n cfg_file = osp.join(data_path, 'config/p.yaml')\n cfg_dict = dict(item1=osp.join(osp.dirname(__file__), 'data', 'config'))\n cfg = Config(cfg_dict, filename=cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n assert cfg.text == open(cfg_file, 'r').read()\n assert cfg.dump() == yaml.dump(cfg_dict)\n with tempfile.TemporaryDirectory() as temp_config_dir:\n dump_file = osp.join(temp_config_dir, 'p.yaml')\n cfg.dump(dump_file)\n assert cfg.dump() == open(dump_file, 'r').read()\n assert Config.fromfile(dump_file)\n assert Config.fromfile(dump_file)['item1'] == cfg_dict['item1']\n\n # test no use_predefined_variable\n assert Config.fromfile(cfg_file, False)\n assert Config.fromfile(cfg_file, False)['item1'] == '{{ fileDirname }}'\n\n # test o.json\n cfg_file = osp.join(data_path, 'config/o.json')\n cfg_dict = dict(item1=osp.join(osp.dirname(__file__), 'data', 'config'))\n cfg = Config(cfg_dict, filename=cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n assert cfg.text == open(cfg_file, 'r').read()\n assert cfg.dump() == json.dumps(cfg_dict)\n with tempfile.TemporaryDirectory() as temp_config_dir:\n dump_file = osp.join(temp_config_dir, 'o.json')\n cfg.dump(dump_file)\n assert cfg.dump() == open(dump_file, 'r').read()\n assert Config.fromfile(dump_file)\n assert Config.fromfile(dump_file)['item1'] == cfg_dict['item1']\n\n # test no use_predefined_variable\n assert Config.fromfile(cfg_file, False)\n assert Config.fromfile(cfg_file, False)['item1'] == '{{ fileDirname }}'\n\n\ndef test_fromfile():\n for filename in ['a.py', 'a.b.py', 'b.json', 'c.yaml']:\n cfg_file = osp.join(data_path, 'config', filename)\n cfg = Config.fromfile(cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n assert cfg.text == osp.abspath(osp.expanduser(cfg_file)) + '\\n' + \\\n open(cfg_file, 'r').read()\n\n # test custom_imports for Config.fromfile\n cfg_file = osp.join(data_path, 'config', 'q.py')\n imported_file = osp.join(data_path, 'config', 'r.py')\n target_pkg = osp.join(osp.dirname(__file__), 'r.py')\n\n # Since the imported config will be regarded as a tmp file\n # it should be copied to the directory at the same level\n shutil.copy(imported_file, target_pkg)\n Config.fromfile(cfg_file, import_custom_modules=True)\n\n assert os.environ.pop('TEST_VALUE') == 'test'\n os.remove(target_pkg)\n\n with pytest.raises(FileNotFoundError):\n Config.fromfile('no_such_file.py')\n with pytest.raises(IOError):\n Config.fromfile(osp.join(data_path, 'color.jpg'))\n\n\ndef test_fromstring():\n for filename in ['a.py', 'a.b.py', 'b.json', 'c.yaml']:\n cfg_file = osp.join(data_path, 'config', filename)\n file_format = osp.splitext(filename)[-1]\n in_cfg = Config.fromfile(cfg_file)\n\n out_cfg = Config.fromstring(in_cfg.pretty_text, '.py')\n assert in_cfg._cfg_dict == out_cfg._cfg_dict\n\n cfg_str = open(cfg_file, 'r').read()\n out_cfg = Config.fromstring(cfg_str, file_format)\n assert in_cfg._cfg_dict == out_cfg._cfg_dict\n\n # test pretty_text only supports py file format\n cfg_file = osp.join(data_path, 'config', 'b.json')\n in_cfg = Config.fromfile(cfg_file)\n with pytest.raises(Exception):\n Config.fromstring(in_cfg.pretty_text, '.json')\n\n # test file format error\n cfg_str = open(cfg_file, 'r').read()\n with pytest.raises(Exception):\n Config.fromstring(cfg_str, '.py')\n\n\ndef test_merge_from_base():\n cfg_file = osp.join(data_path, 'config/d.py')\n cfg = Config.fromfile(cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n base_cfg_file = osp.join(data_path, 'config/base.py')\n merge_text = osp.abspath(osp.expanduser(base_cfg_file)) + '\\n' + \\\n open(base_cfg_file, 'r').read()\n merge_text += '\\n' + osp.abspath(osp.expanduser(cfg_file)) + '\\n' + \\\n open(cfg_file, 'r').read()\n assert cfg.text == merge_text\n assert cfg.item1 == [2, 3]\n assert cfg.item2.a == 1\n assert cfg.item3 is False\n assert cfg.item4 == 'test_base'\n\n with pytest.raises(TypeError):\n Config.fromfile(osp.join(data_path, 'config/e.py'))\n\n\ndef test_merge_from_multiple_bases():\n cfg_file = osp.join(data_path, 'config/l.py')\n cfg = Config.fromfile(cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n # cfg.field\n assert cfg.item1 == [1, 2]\n assert cfg.item2.a == 0\n assert cfg.item3 is False\n assert cfg.item4 == 'test'\n assert cfg.item5 == dict(a=0, b=1)\n assert cfg.item6 == [dict(a=0), dict(b=1)]\n assert cfg.item7 == dict(a=[0, 1, 2], b=dict(c=[3.1, 4.2, 5.3]))\n\n with pytest.raises(KeyError):\n Config.fromfile(osp.join(data_path, 'config/m.py'))\n\n\ndef test_base_variables():\n for file in ['t.py', 't.json', 't.yaml']:\n cfg_file = osp.join(data_path, f'config/{file}')\n cfg = Config.fromfile(cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n # cfg.field\n assert cfg.item1 == [1, 2]\n assert cfg.item2.a == 0\n assert cfg.item3 is False\n assert cfg.item4 == 'test'\n assert cfg.item5 == dict(a=0, b=1)\n assert cfg.item6 == [dict(a=0), dict(b=1)]\n assert cfg.item7 == dict(a=[0, 1, 2], b=dict(c=[3.1, 4.2, 5.3]))\n assert cfg.item8 == file\n assert cfg.item9 == dict(a=0)\n assert cfg.item10 == [3.1, 4.2, 5.3]\n\n # test nested base\n for file in ['u.py', 'u.json', 'u.yaml']:\n cfg_file = osp.join(data_path, f'config/{file}')\n cfg = Config.fromfile(cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n # cfg.field\n assert cfg.base == '_base_.item8'\n assert cfg.item1 == [1, 2]\n assert cfg.item2.a == 0\n assert cfg.item3 is False\n assert cfg.item4 == 'test'\n assert cfg.item5 == dict(a=0, b=1)\n assert cfg.item6 == [dict(a=0), dict(b=1)]\n assert cfg.item7 == dict(a=[0, 1, 2], b=dict(c=[3.1, 4.2, 5.3]))\n assert cfg.item8 == 't.py'\n assert cfg.item9 == dict(a=0)\n assert cfg.item10 == [3.1, 4.2, 5.3]\n assert cfg.item11 == 't.py'\n assert cfg.item12 == dict(a=0)\n assert cfg.item13 == [3.1, 4.2, 5.3]\n assert cfg.item14 == [1, 2]\n assert cfg.item15 == dict(\n a=dict(b=dict(a=0)),\n b=[False],\n c=['test'],\n d=[[{\n 'e': 0\n }], [{\n 'a': 0\n }, {\n 'b': 1\n }]],\n e=[1, 2])\n\n # test reference assignment for py\n cfg_file = osp.join(data_path, 'config/v.py')\n cfg = Config.fromfile(cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n assert cfg.item21 == 't.py'\n assert cfg.item22 == 't.py'\n assert cfg.item23 == [3.1, 4.2, 5.3]\n assert cfg.item24 == [3.1, 4.2, 5.3]\n assert cfg.item25 == dict(\n a=dict(b=[3.1, 4.2, 5.3]),\n b=[[3.1, 4.2, 5.3]],\n c=[[{\n 'e': 't.py'\n }], [{\n 'a': 0\n }, {\n 'b': 1\n }]],\n e='t.py')\n\n\ndef test_merge_recursive_bases():\n cfg_file = osp.join(data_path, 'config/f.py')\n cfg = Config.fromfile(cfg_file)\n assert isinstance(cfg, Config)\n assert cfg.filename == cfg_file\n # cfg.field\n assert cfg.item1 == [2, 3]\n assert cfg.item2.a == 1\n assert cfg.item3 is False\n assert cfg.item4 == 'test_recursive_bases'\n\n\ndef test_merge_from_dict():\n cfg_file = osp.join(data_path, 'config/a.py')\n cfg = Config.fromfile(cfg_file)\n input_options = {'item2.a': 1, 'item2.b': 0.1, 'item3': False}\n cfg.merge_from_dict(input_options)\n assert cfg.item2 == dict(a=1, b=0.1)\n assert cfg.item3 is False\n\n cfg_file = osp.join(data_path, 'config/s.py')\n cfg = Config.fromfile(cfg_file)\n\n # Allow list keys\n input_options = {'item.0.a': 1, 'item.1.b': 1}\n cfg.merge_from_dict(input_options, allow_list_keys=True)\n assert cfg.item == [{'a': 1}, {'b': 1, 'c': 0}]\n\n # allow_list_keys is False\n input_options = {'item.0.a': 1, 'item.1.b': 1}\n with pytest.raises(TypeError):\n cfg.merge_from_dict(input_options, allow_list_keys=False)\n\n # Overflowed index number\n input_options = {'item.2.a': 1}\n with pytest.raises(KeyError):\n cfg.merge_from_dict(input_options, allow_list_keys=True)\n\n\ndef test_merge_delete():\n cfg_file = osp.join(data_path, 'config/delete.py')\n cfg = Config.fromfile(cfg_file)\n # cfg.field\n assert cfg.item1 == dict(a=0)\n assert cfg.item2 == dict(a=0, b=0)\n assert cfg.item3 is True\n assert cfg.item4 == 'test'\n assert '_delete_' not in cfg.item2\n\n # related issue: https://github.com/open-mmlab/mmcv/issues/1570\n assert type(cfg.item1) == ConfigDict\n assert type(cfg.item2) == ConfigDict\n\n\ndef test_merge_intermediate_variable():\n\n cfg_file = osp.join(data_path, 'config/i_child.py')\n cfg = Config.fromfile(cfg_file)\n # cfg.field\n assert cfg.item1 == [1, 2]\n assert cfg.item2 == dict(a=0)\n assert cfg.item3 is True\n assert cfg.item4 == 'test'\n assert cfg.item_cfg == dict(b=2)\n assert cfg.item5 == dict(cfg=dict(b=1))\n assert cfg.item6 == dict(cfg=dict(b=2))\n\n\ndef test_fromfile_in_config():\n cfg_file = osp.join(data_path, 'config/code.py')\n cfg = Config.fromfile(cfg_file)\n # cfg.field\n assert cfg.cfg.item1 == [1, 2]\n assert cfg.cfg.item2 == dict(a=0)\n assert cfg.cfg.item3 is True\n assert cfg.cfg.item4 == 'test'\n assert cfg.item5 == 1\n\n\ndef test_dict():\n cfg_dict = dict(item1=[1, 2], item2=dict(a=0), item3=True, item4='test')\n\n for filename in ['a.py', 'b.json', 'c.yaml']:\n cfg_file = osp.join(data_path, 'config', filename)\n cfg = Config.fromfile(cfg_file)\n\n # len(cfg)\n assert len(cfg) == 4\n # cfg.keys()\n assert set(cfg.keys()) == set(cfg_dict.keys())\n assert set(cfg._cfg_dict.keys()) == set(cfg_dict.keys())\n # cfg.values()\n for value in cfg.values():\n assert value in cfg_dict.values()\n # cfg.items()\n for name, value in cfg.items():\n assert name in cfg_dict\n assert value in cfg_dict.values()\n # cfg.field\n assert cfg.item1 == cfg_dict['item1']\n assert cfg.item2 == cfg_dict['item2']\n assert cfg.item2.a == 0\n assert cfg.item3 == cfg_dict['item3']\n assert cfg.item4 == cfg_dict['item4']\n with pytest.raises(AttributeError):\n cfg.not_exist\n # field in cfg, cfg[field], cfg.get()\n for name in ['item1', 'item2', 'item3', 'item4']:\n assert name in cfg\n assert cfg[name] == cfg_dict[name]\n assert cfg.get(name) == cfg_dict[name]\n assert cfg.get('not_exist') is None\n assert cfg.get('not_exist', 0) == 0\n with pytest.raises(KeyError):\n cfg['not_exist']\n assert 'item1' in cfg\n assert 'not_exist' not in cfg\n # cfg.update()\n cfg.update(dict(item1=0))\n assert cfg.item1 == 0\n cfg.update(dict(item2=dict(a=1)))\n assert cfg.item2.a == 1\n\n\ndef test_setattr():\n cfg = Config()\n cfg.item1 = [1, 2]\n cfg.item2 = {'a': 0}\n cfg['item5'] = {'a': {'b': None}}\n assert cfg._cfg_dict['item1'] == [1, 2]\n assert cfg.item1 == [1, 2]\n assert cfg._cfg_dict['item2'] == {'a': 0}\n assert cfg.item2.a == 0\n assert cfg._cfg_dict['item5'] == {'a': {'b': None}}\n assert cfg.item5.a.b is None\n\n\ndef test_pretty_text():\n cfg_file = osp.join(data_path, 'config/l.py')\n cfg = Config.fromfile(cfg_file)\n with tempfile.TemporaryDirectory() as temp_config_dir:\n text_cfg_filename = osp.join(temp_config_dir, '_text_config.py')\n with open(text_cfg_filename, 'w') as f:\n f.write(cfg.pretty_text)\n text_cfg = Config.fromfile(text_cfg_filename)\n assert text_cfg._cfg_dict == cfg._cfg_dict\n\n\ndef test_dict_action():\n parser = argparse.ArgumentParser(description='Train a detector')\n parser.add_argument(\n '--options', nargs='+', action=DictAction, help='custom options')\n # Nested brackets\n args = parser.parse_args(\n ['--options', 'item2.a=a,b', 'item2.b=[(a,b), [1,2], false]'])\n out_dict = {'item2.a': ['a', 'b'], 'item2.b': [('a', 'b'), [1, 2], False]}\n assert args.options == out_dict\n # Single Nested brackets\n args = parser.parse_args(['--options', 'item2.a=[[1]]'])\n out_dict = {'item2.a': [[1]]}\n assert args.options == out_dict\n # Imbalance bracket\n with pytest.raises(AssertionError):\n parser.parse_args(['--options', 'item2.a=[(a,b), [1,2], false'])\n # Normal values\n args = parser.parse_args(\n ['--options', 'item2.a=1', 'item2.b=0.1', 'item2.c=x', 'item3=false'])\n out_dict = {'item2.a': 1, 'item2.b': 0.1, 'item2.c': 'x', 'item3': False}\n assert args.options == out_dict\n cfg_file = osp.join(data_path, 'config/a.py')\n cfg = Config.fromfile(cfg_file)\n cfg.merge_from_dict(args.options)\n assert cfg.item2 == dict(a=1, b=0.1, c='x')\n assert cfg.item3 is False\n\n\ndef test_dump_mapping():\n cfg_file = osp.join(data_path, 'config/n.py')\n cfg = Config.fromfile(cfg_file)\n\n with tempfile.TemporaryDirectory() as temp_config_dir:\n text_cfg_filename = osp.join(temp_config_dir, '_text_config.py')\n cfg.dump(text_cfg_filename)\n text_cfg = Config.fromfile(text_cfg_filename)\n\n assert text_cfg._cfg_dict == cfg._cfg_dict\n\n\ndef test_reserved_key():\n cfg_file = osp.join(data_path, 'config/g.py')\n with pytest.raises(KeyError):\n Config.fromfile(cfg_file)\n\n\ndef test_syntax_error():\n # the name can not be used to open the file a second time in windows,\n # so `delete` should be set as `False` and we need to manually remove it\n # more details can be found at https://github.com/open-mmlab/mmcv/pull/1077\n temp_cfg_file = tempfile.NamedTemporaryFile(suffix='.py', delete=False)\n temp_cfg_path = temp_cfg_file.name\n # write a file with syntax error\n with open(temp_cfg_path, 'w') as f:\n f.write('a=0b=dict(c=1)')\n with pytest.raises(\n SyntaxError, match='There are syntax errors in config file'):\n Config.fromfile(temp_cfg_path)\n temp_cfg_file.close()\n os.remove(temp_cfg_path)\n\n\ndef test_pickle_support():\n cfg_file = osp.join(data_path, 'config/n.py')\n cfg = Config.fromfile(cfg_file)\n\n with tempfile.TemporaryDirectory() as temp_config_dir:\n pkl_cfg_filename = osp.join(temp_config_dir, '_pickle.pkl')\n dump(cfg, pkl_cfg_filename)\n pkl_cfg = load(pkl_cfg_filename)\n\n assert pkl_cfg._cfg_dict == cfg._cfg_dict\n\n\ndef test_deprecation():\n deprecated_cfg_files = [\n osp.join(data_path, 'config/deprecated.py'),\n osp.join(data_path, 'config/deprecated_as_base.py')\n ]\n\n for cfg_file in deprecated_cfg_files:\n with pytest.warns(DeprecationWarning):\n cfg = Config.fromfile(cfg_file)\n assert cfg.item1 == 'expected'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_env.py", "content": "import sys\n\nimport pytest\n\nimport mmcv\n\n\ndef test_collect_env():\n try:\n import torch # noqa: F401\n except ModuleNotFoundError:\n pytest.skip('skipping tests that require PyTorch')\n\n from mmcv.utils import collect_env\n env_info = collect_env()\n expected_keys = [\n 'sys.platform', 'Python', 'CUDA available', 'PyTorch',\n 'PyTorch compiling details', 'OpenCV', 'MMCV', 'MMCV Compiler',\n 'MMCV CUDA Compiler'\n ]\n for key in expected_keys:\n assert key in env_info\n\n if env_info['CUDA available']:\n for key in ['CUDA_HOME', 'NVCC']:\n assert key in env_info\n\n if sys.platform != 'win32':\n assert 'GCC' in env_info\n\n assert env_info['sys.platform'] == sys.platform\n assert env_info['Python'] == sys.version.replace('\\n', '')\n assert env_info['MMCV'] == mmcv.__version__\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_hub.py", "content": "import pytest\nfrom torch.utils import model_zoo\n\nfrom mmcv.utils import TORCH_VERSION, digit_version, load_url\n\n\ndef test_load_url():\n url1 = 'https://download.openmmlab.com/mmcv/test_data/saved_in_pt1.5.pth'\n url2 = 'https://download.openmmlab.com/mmcv/test_data/saved_in_pt1.6.pth'\n\n # The 1.6 release of PyTorch switched torch.save to use a new zipfile-based\n # file format. It will cause RuntimeError when a checkpoint was saved in\n # torch >= 1.6.0 but loaded in torch < 1.7.0.\n # More details at https://github.com/open-mmlab/mmpose/issues/904\n if digit_version(TORCH_VERSION) < digit_version('1.7.0'):\n model_zoo.load_url(url1)\n with pytest.raises(RuntimeError):\n model_zoo.load_url(url2)\n else:\n # high version of PyTorch can load checkpoints from url, regardless\n # of which version they were saved in\n model_zoo.load_url(url1)\n model_zoo.load_url(url2)\n\n load_url(url1)\n # if a checkpoint was saved in torch >= 1.6.0 but loaded in torch < 1.5.0,\n # it will raise a RuntimeError\n if digit_version(TORCH_VERSION) < digit_version('1.5.0'):\n with pytest.raises(RuntimeError):\n load_url(url2)\n else:\n load_url(url2)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_logging.py", "content": "import logging\nimport os\nimport platform\nimport tempfile\nfrom unittest.mock import patch\n\nimport pytest\n\nfrom mmcv import get_logger, print_log\n\nif platform.system() == 'Windows':\n import regex as re\nelse:\n import re\n\n\n@patch('torch.distributed.get_rank', lambda: 0)\n@patch('torch.distributed.is_initialized', lambda: True)\n@patch('torch.distributed.is_available', lambda: True)\ndef test_get_logger_rank0():\n logger = get_logger('rank0.pkg1')\n assert isinstance(logger, logging.Logger)\n assert len(logger.handlers) == 1\n assert isinstance(logger.handlers[0], logging.StreamHandler)\n assert logger.handlers[0].level == logging.INFO\n\n logger = get_logger('rank0.pkg2', log_level=logging.DEBUG)\n assert isinstance(logger, logging.Logger)\n assert len(logger.handlers) == 1\n assert logger.handlers[0].level == logging.DEBUG\n\n # the name can not be used to open the file a second time in windows,\n # so `delete` should be set as `False` and we need to manually remove it\n # more details can be found at https://github.com/open-mmlab/mmcv/pull/1077\n with tempfile.NamedTemporaryFile(delete=False) as f:\n logger = get_logger('rank0.pkg3', log_file=f.name)\n assert isinstance(logger, logging.Logger)\n assert len(logger.handlers) == 2\n assert isinstance(logger.handlers[0], logging.StreamHandler)\n assert isinstance(logger.handlers[1], logging.FileHandler)\n logger_pkg3 = get_logger('rank0.pkg3')\n assert id(logger_pkg3) == id(logger)\n # flushing and closing all handlers in order to remove `f.name`\n logging.shutdown()\n\n os.remove(f.name)\n\n logger_pkg3 = get_logger('rank0.pkg3.subpkg')\n assert logger_pkg3.handlers == logger_pkg3.handlers\n\n\n@patch('torch.distributed.get_rank', lambda: 1)\n@patch('torch.distributed.is_initialized', lambda: True)\n@patch('torch.distributed.is_available', lambda: True)\ndef test_get_logger_rank1():\n logger = get_logger('rank1.pkg1')\n assert isinstance(logger, logging.Logger)\n assert len(logger.handlers) == 1\n assert isinstance(logger.handlers[0], logging.StreamHandler)\n assert logger.handlers[0].level == logging.INFO\n\n # the name can not be used to open the file a second time in windows,\n # so `delete` should be set as `False` and we need to manually remove it\n # more details can be found at https://github.com/open-mmlab/mmcv/pull/1077\n with tempfile.NamedTemporaryFile(delete=False) as f:\n logger = get_logger('rank1.pkg2', log_file=f.name)\n assert isinstance(logger, logging.Logger)\n assert len(logger.handlers) == 1\n assert logger.handlers[0].level == logging.INFO\n # flushing and closing all handlers in order to remove `f.name`\n logging.shutdown()\n\n os.remove(f.name)\n\n\ndef test_print_log_print(capsys):\n print_log('welcome', logger=None)\n out, _ = capsys.readouterr()\n assert out == 'welcome\\n'\n\n\ndef test_print_log_silent(capsys, caplog):\n print_log('welcome', logger='silent')\n out, _ = capsys.readouterr()\n assert out == ''\n assert len(caplog.records) == 0\n\n\ndef test_print_log_logger(caplog):\n print_log('welcome', logger='mmcv')\n assert caplog.record_tuples[-1] == ('mmcv', logging.INFO, 'welcome')\n\n print_log('welcome', logger='mmcv', level=logging.ERROR)\n assert caplog.record_tuples[-1] == ('mmcv', logging.ERROR, 'welcome')\n\n # the name can not be used to open the file a second time in windows,\n # so `delete` should be set as `False` and we need to manually remove it\n # more details can be found at https://github.com/open-mmlab/mmcv/pull/1077\n with tempfile.NamedTemporaryFile(delete=False) as f:\n logger = get_logger('abc', log_file=f.name)\n print_log('welcome', logger=logger)\n assert caplog.record_tuples[-1] == ('abc', logging.INFO, 'welcome')\n with open(f.name, 'r') as fin:\n log_text = fin.read()\n regex_time = r'\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2},\\d{3}'\n match = re.fullmatch(regex_time + r' - abc - INFO - welcome\\n',\n log_text)\n assert match is not None\n # flushing and closing all handlers in order to remove `f.name`\n logging.shutdown()\n\n os.remove(f.name)\n\n\ndef test_print_log_exception():\n with pytest.raises(TypeError):\n print_log('welcome', logger=0)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_misc.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport pytest\n\nimport mmcv\nfrom mmcv import deprecated_api_warning\nfrom mmcv.utils.misc import has_method\n\n\ndef test_to_ntuple():\n single_number = 2\n assert mmcv.utils.to_1tuple(single_number) == (single_number, )\n assert mmcv.utils.to_2tuple(single_number) == (single_number,\n single_number)\n assert mmcv.utils.to_3tuple(single_number) == (single_number,\n single_number,\n single_number)\n assert mmcv.utils.to_4tuple(single_number) == (single_number,\n single_number,\n single_number,\n single_number)\n assert mmcv.utils.to_ntuple(5)(single_number) == (single_number,\n single_number,\n single_number,\n single_number,\n single_number)\n assert mmcv.utils.to_ntuple(6)(single_number) == (single_number,\n single_number,\n single_number,\n single_number,\n single_number,\n single_number)\n\n\ndef test_iter_cast():\n assert mmcv.list_cast([1, 2, 3], int) == [1, 2, 3]\n assert mmcv.list_cast(['1.1', 2, '3'], float) == [1.1, 2.0, 3.0]\n assert mmcv.list_cast([1, 2, 3], str) == ['1', '2', '3']\n assert mmcv.tuple_cast((1, 2, 3), str) == ('1', '2', '3')\n assert next(mmcv.iter_cast([1, 2, 3], str)) == '1'\n with pytest.raises(TypeError):\n mmcv.iter_cast([1, 2, 3], '')\n with pytest.raises(TypeError):\n mmcv.iter_cast(1, str)\n\n\ndef test_is_seq_of():\n assert mmcv.is_seq_of([1.0, 2.0, 3.0], float)\n assert mmcv.is_seq_of([(1, ), (2, ), (3, )], tuple)\n assert mmcv.is_seq_of((1.0, 2.0, 3.0), float)\n assert mmcv.is_list_of([1.0, 2.0, 3.0], float)\n assert not mmcv.is_seq_of((1.0, 2.0, 3.0), float, seq_type=list)\n assert not mmcv.is_tuple_of([1.0, 2.0, 3.0], float)\n assert not mmcv.is_seq_of([1.0, 2, 3], int)\n assert not mmcv.is_seq_of((1.0, 2, 3), int)\n\n\ndef test_slice_list():\n in_list = [1, 2, 3, 4, 5, 6]\n assert mmcv.slice_list(in_list, [1, 2, 3]) == [[1], [2, 3], [4, 5, 6]]\n assert mmcv.slice_list(in_list, [len(in_list)]) == [in_list]\n with pytest.raises(TypeError):\n mmcv.slice_list(in_list, 2.0)\n with pytest.raises(ValueError):\n mmcv.slice_list(in_list, [1, 2])\n\n\ndef test_concat_list():\n assert mmcv.concat_list([[1, 2]]) == [1, 2]\n assert mmcv.concat_list([[1, 2], [3, 4, 5], [6]]) == [1, 2, 3, 4, 5, 6]\n\n\ndef test_requires_package(capsys):\n\n @mmcv.requires_package('nnn')\n def func_a():\n pass\n\n @mmcv.requires_package(['numpy', 'n1', 'n2'])\n def func_b():\n pass\n\n @mmcv.requires_package('numpy')\n def func_c():\n return 1\n\n with pytest.raises(RuntimeError):\n func_a()\n out, _ = capsys.readouterr()\n assert out == ('Prerequisites \"nnn\" are required in method \"func_a\" but '\n 'not found, please install them first.\\n')\n\n with pytest.raises(RuntimeError):\n func_b()\n out, _ = capsys.readouterr()\n assert out == (\n 'Prerequisites \"n1, n2\" are required in method \"func_b\" but not found,'\n ' please install them first.\\n')\n\n assert func_c() == 1\n\n\ndef test_requires_executable(capsys):\n\n @mmcv.requires_executable('nnn')\n def func_a():\n pass\n\n @mmcv.requires_executable(['ls', 'n1', 'n2'])\n def func_b():\n pass\n\n @mmcv.requires_executable('mv')\n def func_c():\n return 1\n\n with pytest.raises(RuntimeError):\n func_a()\n out, _ = capsys.readouterr()\n assert out == ('Prerequisites \"nnn\" are required in method \"func_a\" but '\n 'not found, please install them first.\\n')\n\n with pytest.raises(RuntimeError):\n func_b()\n out, _ = capsys.readouterr()\n assert out == (\n 'Prerequisites \"n1, n2\" are required in method \"func_b\" but not found,'\n ' please install them first.\\n')\n\n assert func_c() == 1\n\n\ndef test_import_modules_from_strings():\n # multiple imports\n import os.path as osp_\n\n import sys as sys_\n osp, sys = mmcv.import_modules_from_strings(['os.path', 'sys'])\n assert osp == osp_\n assert sys == sys_\n\n # single imports\n osp = mmcv.import_modules_from_strings('os.path')\n assert osp == osp_\n # No imports\n assert mmcv.import_modules_from_strings(None) is None\n assert mmcv.import_modules_from_strings([]) is None\n assert mmcv.import_modules_from_strings('') is None\n # Unsupported types\n with pytest.raises(TypeError):\n mmcv.import_modules_from_strings(1)\n with pytest.raises(TypeError):\n mmcv.import_modules_from_strings([1])\n # Failed imports\n with pytest.raises(ImportError):\n mmcv.import_modules_from_strings('_not_implemented_module')\n with pytest.warns(UserWarning):\n imported = mmcv.import_modules_from_strings(\n '_not_implemented_module', allow_failed_imports=True)\n assert imported is None\n with pytest.warns(UserWarning):\n imported = mmcv.import_modules_from_strings(\n ['os.path', '_not_implemented'], allow_failed_imports=True)\n assert imported[0] == osp\n assert imported[1] is None\n\n\ndef test_is_method_overridden():\n\n class Base:\n\n def foo1():\n pass\n\n def foo2():\n pass\n\n class Sub(Base):\n\n def foo1():\n pass\n\n # test passing sub class directly\n assert mmcv.is_method_overridden('foo1', Base, Sub)\n assert not mmcv.is_method_overridden('foo2', Base, Sub)\n\n # test passing instance of sub class\n sub_instance = Sub()\n assert mmcv.is_method_overridden('foo1', Base, sub_instance)\n assert not mmcv.is_method_overridden('foo2', Base, sub_instance)\n\n # base_class should be a class, not instance\n base_instance = Base()\n with pytest.raises(AssertionError):\n mmcv.is_method_overridden('foo1', base_instance, sub_instance)\n\n\ndef test_has_method():\n\n class Foo:\n\n def __init__(self, name):\n self.name = name\n\n def print_name(self):\n print(self.name)\n\n foo = Foo('foo')\n assert not has_method(foo, 'name')\n assert has_method(foo, 'print_name')\n\n\ndef test_deprecated_api_warning():\n\n @deprecated_api_warning(name_dict=dict(old_key='new_key'))\n def dummy_func(new_key=1):\n return new_key\n\n # replace `old_key` to `new_key`\n assert dummy_func(old_key=2) == 2\n\n # The expected behavior is to replace the\n # deprecated key `old_key` to `new_key`,\n # but got them in the arguments at the same time\n with pytest.raises(AssertionError):\n dummy_func(old_key=1, new_key=2)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_parrots_jit.py", "content": "import pytest\nimport torch\n\nimport mmcv\nfrom mmcv.utils import TORCH_VERSION\n\nskip_no_parrots = pytest.mark.skipif(\n TORCH_VERSION != 'parrots', reason='test case under parrots environment')\n\n\nclass TestJit(object):\n\n def test_add_dict(self):\n\n @mmcv.jit\n def add_dict(oper):\n rets = oper['x'] + oper['y']\n return {'result': rets}\n\n def add_dict_pyfunc(oper):\n rets = oper['x'] + oper['y']\n return {'result': rets}\n\n a = torch.rand((3, 4))\n b = torch.rand((3, 4))\n oper = {'x': a, 'y': b}\n\n rets_t = add_dict(oper)\n rets = add_dict_pyfunc(oper)\n assert 'result' in rets\n assert (rets_t['result'] == rets['result']).all()\n\n def test_add_list(self):\n\n @mmcv.jit\n def add_list(oper, x, y):\n rets = {}\n for idx, pair in enumerate(oper):\n rets[f'k{idx}'] = pair['x'] + pair['y']\n rets[f'k{len(oper)}'] = x + y\n return rets\n\n def add_list_pyfunc(oper, x, y):\n rets = {}\n for idx, pair in enumerate(oper):\n rets[f'k{idx}'] = pair['x'] + pair['y']\n rets[f'k{len(oper)}'] = x + y\n return rets\n\n pair_num = 3\n oper = []\n for _ in range(pair_num):\n oper.append({'x': torch.rand((3, 4)), 'y': torch.rand((3, 4))})\n a = torch.rand((3, 4))\n b = torch.rand((3, 4))\n rets = add_list_pyfunc(oper, x=a, y=b)\n rets_t = add_list(oper, x=a, y=b)\n for idx in range(pair_num + 1):\n assert f'k{idx}' in rets_t\n assert (rets[f'k{idx}'] == rets_t[f'k{idx}']).all()\n\n @skip_no_parrots\n def test_jit_cache(self):\n\n @mmcv.jit\n def func(oper):\n if oper['const'] > 1:\n return oper['x'] * 2 + oper['y']\n else:\n return oper['x'] * 2 - oper['y']\n\n def pyfunc(oper):\n if oper['const'] > 1:\n return oper['x'] * 2 + oper['y']\n else:\n return oper['x'] * 2 - oper['y']\n\n assert len(func._cache._cache) == 0\n\n oper = {'const': 2, 'x': torch.rand((3, 4)), 'y': torch.rand((3, 4))}\n rets_plus = pyfunc(oper)\n rets_plus_t = func(oper)\n assert (rets_plus == rets_plus_t).all()\n assert len(func._cache._cache) == 1\n\n oper['const'] = 0.5\n rets_minus = pyfunc(oper)\n rets_minus_t = func(oper)\n assert (rets_minus == rets_minus_t).all()\n assert len(func._cache._cache) == 2\n\n rets_a = (rets_minus_t + rets_plus_t) / 4\n assert torch.allclose(oper['x'], rets_a)\n\n @skip_no_parrots\n def test_jit_shape(self):\n\n @mmcv.jit\n def func(a):\n return a + 1\n\n assert len(func._cache._cache) == 0\n\n a = torch.ones((3, 4))\n r = func(a)\n assert r.shape == (3, 4)\n assert (r == 2).all()\n assert len(func._cache._cache) == 1\n\n a = torch.ones((2, 3, 4))\n r = func(a)\n assert r.shape == (2, 3, 4)\n assert (r == 2).all()\n assert len(func._cache._cache) == 2\n\n @skip_no_parrots\n def test_jit_kwargs(self):\n\n @mmcv.jit\n def func(a, b):\n return torch.mean((a - b) * (a - b))\n\n assert len(func._cache._cache) == 0\n x = torch.rand((16, 32))\n y = torch.rand((16, 32))\n func(x, y)\n assert len(func._cache._cache) == 1\n func(x, b=y)\n assert len(func._cache._cache) == 1\n func(b=y, a=x)\n assert len(func._cache._cache) == 1\n\n def test_jit_derivate(self):\n\n @mmcv.jit(derivate=True)\n def func(x, y):\n return (x + 2) * (y - 2)\n\n a = torch.rand((3, 4))\n b = torch.rand((3, 4))\n a.requires_grad = True\n\n c = func(a, b)\n assert c.requires_grad\n d = torch.empty_like(c)\n d.fill_(1.0)\n c.backward(d)\n assert torch.allclose(a.grad, (b - 2))\n assert b.grad is None\n\n a.grad = None\n c = func(a, b)\n assert c.requires_grad\n d = torch.empty_like(c)\n d.fill_(2.7)\n c.backward(d)\n assert torch.allclose(a.grad, 2.7 * (b - 2))\n assert b.grad is None\n\n def test_jit_optimize(self):\n\n @mmcv.jit(optimize=True)\n def func(a, b):\n return torch.mean((a - b) * (a - b))\n\n def pyfunc(a, b):\n return torch.mean((a - b) * (a - b))\n\n a = torch.rand((16, 32))\n b = torch.rand((16, 32))\n\n c = func(a, b)\n d = pyfunc(a, b)\n assert torch.allclose(c, d)\n\n @mmcv.skip_no_elena\n def test_jit_coderize(self):\n if not torch.cuda.is_available():\n return\n\n @mmcv.jit(coderize=True)\n def func(a, b):\n return (a + b) * (a - b)\n\n def pyfunc(a, b):\n return (a + b) * (a - b)\n\n a = torch.rand((16, 32), device='cuda')\n b = torch.rand((16, 32), device='cuda')\n\n c = func(a, b)\n d = pyfunc(a, b)\n assert torch.allclose(c, d)\n\n def test_jit_value_dependent(self):\n\n @mmcv.jit\n def func(a, b):\n torch.nonzero(a)\n return torch.mean((a - b) * (a - b))\n\n def pyfunc(a, b):\n torch.nonzero(a)\n return torch.mean((a - b) * (a - b))\n\n a = torch.rand((16, 32))\n b = torch.rand((16, 32))\n\n c = func(a, b)\n d = pyfunc(a, b)\n assert torch.allclose(c, d)\n\n @skip_no_parrots\n def test_jit_check_input(self):\n\n def func(x):\n y = torch.rand_like(x)\n return x + y\n\n a = torch.ones((3, 4))\n with pytest.raises(AssertionError):\n func = mmcv.jit(func, check_input=(a, ))\n\n @skip_no_parrots\n def test_jit_partial_shape(self):\n\n @mmcv.jit(full_shape=False)\n def func(a, b):\n return torch.mean((a - b) * (a - b))\n\n def pyfunc(a, b):\n return torch.mean((a - b) * (a - b))\n\n a = torch.rand((3, 4))\n b = torch.rand((3, 4))\n assert torch.allclose(func(a, b), pyfunc(a, b))\n assert len(func._cache._cache) == 1\n\n a = torch.rand((6, 5))\n b = torch.rand((6, 5))\n assert torch.allclose(func(a, b), pyfunc(a, b))\n assert len(func._cache._cache) == 1\n\n a = torch.rand((3, 4, 5))\n b = torch.rand((3, 4, 5))\n assert torch.allclose(func(a, b), pyfunc(a, b))\n assert len(func._cache._cache) == 2\n\n a = torch.rand((1, 9, 8))\n b = torch.rand((1, 9, 8))\n assert torch.allclose(func(a, b), pyfunc(a, b))\n assert len(func._cache._cache) == 2\n\n def test_instance_method(self):\n\n class T(object):\n\n def __init__(self, shape):\n self._c = torch.rand(shape)\n\n @mmcv.jit\n def test_method(self, x, y):\n return (x * self._c) + y\n\n shape = (16, 32)\n t = T(shape)\n a = torch.rand(shape)\n b = torch.rand(shape)\n res = (a * t._c) + b\n jit_res = t.test_method(a, b)\n assert torch.allclose(res, jit_res)\n\n t = T(shape)\n res = (a * t._c) + b\n jit_res = t.test_method(a, b)\n assert torch.allclose(res, jit_res)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_path.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os.path as osp\nfrom pathlib import Path\n\nimport pytest\n\nimport mmcv\n\n\ndef test_is_filepath():\n assert mmcv.is_filepath(__file__)\n assert mmcv.is_filepath('abc')\n assert mmcv.is_filepath(Path('/etc'))\n assert not mmcv.is_filepath(0)\n\n\ndef test_fopen():\n assert hasattr(mmcv.fopen(__file__), 'read')\n assert hasattr(mmcv.fopen(Path(__file__)), 'read')\n\n\ndef test_check_file_exist():\n mmcv.check_file_exist(__file__)\n with pytest.raises(FileNotFoundError):\n mmcv.check_file_exist('no_such_file.txt')\n\n\ndef test_scandir():\n folder = osp.join(osp.dirname(osp.dirname(__file__)), 'data/for_scan')\n filenames = ['a.bin', '1.txt', '2.txt', '1.json', '2.json', '3.TXT']\n assert set(mmcv.scandir(folder)) == set(filenames)\n assert set(mmcv.scandir(Path(folder))) == set(filenames)\n assert set(mmcv.scandir(folder, '.txt')) == set(\n [filename for filename in filenames if filename.endswith('.txt')])\n assert set(mmcv.scandir(folder, ('.json', '.txt'))) == set([\n filename for filename in filenames\n if filename.endswith(('.txt', '.json'))\n ])\n assert set(mmcv.scandir(folder, '.png')) == set()\n\n # path of sep is `\\\\` in windows but `/` in linux, so osp.join should be\n # used to join string for compatibility\n filenames_recursive = [\n 'a.bin', '1.txt', '2.txt', '1.json', '2.json', '3.TXT',\n osp.join('sub', '1.json'),\n osp.join('sub', '1.txt'), '.file'\n ]\n # .file starts with '.' and is a file so it will not be scanned\n assert set(mmcv.scandir(folder, recursive=True)) == set(\n [filename for filename in filenames_recursive if filename != '.file'])\n assert set(mmcv.scandir(Path(folder), recursive=True)) == set(\n [filename for filename in filenames_recursive if filename != '.file'])\n assert set(mmcv.scandir(folder, '.txt', recursive=True)) == set([\n filename for filename in filenames_recursive\n if filename.endswith('.txt')\n ])\n assert set(\n mmcv.scandir(folder, '.TXT', recursive=True,\n case_sensitive=False)) == set([\n filename for filename in filenames_recursive\n if filename.endswith(('.txt', '.TXT'))\n ])\n assert set(\n mmcv.scandir(\n folder, ('.TXT', '.JSON'), recursive=True,\n case_sensitive=False)) == set([\n filename for filename in filenames_recursive\n if filename.endswith(('.txt', '.json', '.TXT'))\n ])\n with pytest.raises(TypeError):\n list(mmcv.scandir(123))\n with pytest.raises(TypeError):\n list(mmcv.scandir(folder, 111))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_progressbar.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport time\n\ntry:\n from unittest.mock import patch\nexcept ImportError:\n from mock import patch\n\ntry:\n from StringIO import StringIO\nexcept ImportError:\n from io import StringIO\n\nimport mmcv # isort:skip\n\n\ndef reset_string_io(io):\n io.truncate(0)\n io.seek(0)\n\n\nclass TestProgressBar:\n\n def test_start(self):\n out = StringIO()\n bar_width = 20\n # without total task num\n prog_bar = mmcv.ProgressBar(bar_width=bar_width, file=out)\n assert out.getvalue() == 'completed: 0, elapsed: 0s'\n reset_string_io(out)\n prog_bar = mmcv.ProgressBar(bar_width=bar_width, start=False, file=out)\n assert out.getvalue() == ''\n reset_string_io(out)\n prog_bar.start()\n assert out.getvalue() == 'completed: 0, elapsed: 0s'\n # with total task num\n reset_string_io(out)\n prog_bar = mmcv.ProgressBar(10, bar_width=bar_width, file=out)\n assert out.getvalue() == f'[{\" \" * bar_width}] 0/10, elapsed: 0s, ETA:'\n reset_string_io(out)\n prog_bar = mmcv.ProgressBar(\n 10, bar_width=bar_width, start=False, file=out)\n assert out.getvalue() == ''\n reset_string_io(out)\n prog_bar.start()\n assert out.getvalue() == f'[{\" \" * bar_width}] 0/10, elapsed: 0s, ETA:'\n\n def test_update(self):\n out = StringIO()\n bar_width = 20\n # without total task num\n prog_bar = mmcv.ProgressBar(bar_width=bar_width, file=out)\n time.sleep(1)\n reset_string_io(out)\n prog_bar.update()\n assert out.getvalue() == 'completed: 1, elapsed: 1s, 1.0 tasks/s'\n reset_string_io(out)\n # with total task num\n prog_bar = mmcv.ProgressBar(10, bar_width=bar_width, file=out)\n time.sleep(1)\n reset_string_io(out)\n prog_bar.update()\n assert out.getvalue() == f'\\r[{\">\" * 2 + \" \" * 18}] 1/10, 1.0 ' \\\n 'task/s, elapsed: 1s, ETA: 9s'\n\n def test_adaptive_length(self):\n with patch.dict('os.environ', {'COLUMNS': '80'}):\n out = StringIO()\n bar_width = 20\n prog_bar = mmcv.ProgressBar(10, bar_width=bar_width, file=out)\n time.sleep(1)\n reset_string_io(out)\n prog_bar.update()\n assert len(out.getvalue()) == 66\n\n os.environ['COLUMNS'] = '30'\n reset_string_io(out)\n prog_bar.update()\n assert len(out.getvalue()) == 48\n\n os.environ['COLUMNS'] = '60'\n reset_string_io(out)\n prog_bar.update()\n assert len(out.getvalue()) == 60\n\n\ndef sleep_1s(num):\n time.sleep(1)\n return num\n\n\ndef test_track_progress_list():\n out = StringIO()\n ret = mmcv.track_progress(sleep_1s, [1, 2, 3], bar_width=3, file=out)\n assert out.getvalue() == (\n '[ ] 0/3, elapsed: 0s, ETA:'\n '\\r[> ] 1/3, 1.0 task/s, elapsed: 1s, ETA: 2s'\n '\\r[>> ] 2/3, 1.0 task/s, elapsed: 2s, ETA: 1s'\n '\\r[>>>] 3/3, 1.0 task/s, elapsed: 3s, ETA: 0s\\n')\n assert ret == [1, 2, 3]\n\n\ndef test_track_progress_iterator():\n out = StringIO()\n ret = mmcv.track_progress(\n sleep_1s, ((i for i in [1, 2, 3]), 3), bar_width=3, file=out)\n assert out.getvalue() == (\n '[ ] 0/3, elapsed: 0s, ETA:'\n '\\r[> ] 1/3, 1.0 task/s, elapsed: 1s, ETA: 2s'\n '\\r[>> ] 2/3, 1.0 task/s, elapsed: 2s, ETA: 1s'\n '\\r[>>>] 3/3, 1.0 task/s, elapsed: 3s, ETA: 0s\\n')\n assert ret == [1, 2, 3]\n\n\ndef test_track_iter_progress():\n out = StringIO()\n ret = []\n for num in mmcv.track_iter_progress([1, 2, 3], bar_width=3, file=out):\n ret.append(sleep_1s(num))\n assert out.getvalue() == (\n '[ ] 0/3, elapsed: 0s, ETA:'\n '\\r[> ] 1/3, 1.0 task/s, elapsed: 1s, ETA: 2s'\n '\\r[>> ] 2/3, 1.0 task/s, elapsed: 2s, ETA: 1s'\n '\\r[>>>] 3/3, 1.0 task/s, elapsed: 3s, ETA: 0s\\n')\n assert ret == [1, 2, 3]\n\n\ndef test_track_enum_progress():\n out = StringIO()\n ret = []\n count = []\n for i, num in enumerate(\n mmcv.track_iter_progress([1, 2, 3], bar_width=3, file=out)):\n ret.append(sleep_1s(num))\n count.append(i)\n assert out.getvalue() == (\n '[ ] 0/3, elapsed: 0s, ETA:'\n '\\r[> ] 1/3, 1.0 task/s, elapsed: 1s, ETA: 2s'\n '\\r[>> ] 2/3, 1.0 task/s, elapsed: 2s, ETA: 1s'\n '\\r[>>>] 3/3, 1.0 task/s, elapsed: 3s, ETA: 0s\\n')\n assert ret == [1, 2, 3]\n assert count == [0, 1, 2]\n\n\ndef test_track_parallel_progress_list():\n out = StringIO()\n results = mmcv.track_parallel_progress(\n sleep_1s, [1, 2, 3, 4], 2, bar_width=4, file=out)\n # The following cannot pass CI on Github Action\n # assert out.getvalue() == (\n # '[ ] 0/4, elapsed: 0s, ETA:'\n # '\\r[> ] 1/4, 1.0 task/s, elapsed: 1s, ETA: 3s'\n # '\\r[>> ] 2/4, 2.0 task/s, elapsed: 1s, ETA: 1s'\n # '\\r[>>> ] 3/4, 1.5 task/s, elapsed: 2s, ETA: 1s'\n # '\\r[>>>>] 4/4, 2.0 task/s, elapsed: 2s, ETA: 0s\\n')\n assert results == [1, 2, 3, 4]\n\n\ndef test_track_parallel_progress_iterator():\n out = StringIO()\n results = mmcv.track_parallel_progress(\n sleep_1s, ((i for i in [1, 2, 3, 4]), 4), 2, bar_width=4, file=out)\n # The following cannot pass CI on Github Action\n # assert out.getvalue() == (\n # '[ ] 0/4, elapsed: 0s, ETA:'\n # '\\r[> ] 1/4, 1.0 task/s, elapsed: 1s, ETA: 3s'\n # '\\r[>> ] 2/4, 2.0 task/s, elapsed: 1s, ETA: 1s'\n # '\\r[>>> ] 3/4, 1.5 task/s, elapsed: 2s, ETA: 1s'\n # '\\r[>>>>] 4/4, 2.0 task/s, elapsed: 2s, ETA: 0s\\n')\n assert results == [1, 2, 3, 4]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_registry.py", "content": "import pytest\n\nimport mmcv\n\n\ndef test_registry():\n CATS = mmcv.Registry('cat')\n assert CATS.name == 'cat'\n assert CATS.module_dict == {}\n assert len(CATS) == 0\n\n @CATS.register_module()\n class BritishShorthair:\n pass\n\n assert len(CATS) == 1\n assert CATS.get('BritishShorthair') is BritishShorthair\n\n class Munchkin:\n pass\n\n CATS.register_module(Munchkin)\n assert len(CATS) == 2\n assert CATS.get('Munchkin') is Munchkin\n assert 'Munchkin' in CATS\n\n with pytest.raises(KeyError):\n CATS.register_module(Munchkin)\n\n CATS.register_module(Munchkin, force=True)\n assert len(CATS) == 2\n\n # force=False\n with pytest.raises(KeyError):\n\n @CATS.register_module()\n class BritishShorthair:\n pass\n\n @CATS.register_module(force=True)\n class BritishShorthair:\n pass\n\n assert len(CATS) == 2\n\n assert CATS.get('PersianCat') is None\n assert 'PersianCat' not in CATS\n\n @CATS.register_module(name=['Siamese', 'Siamese2'])\n class SiameseCat:\n pass\n\n assert CATS.get('Siamese').__name__ == 'SiameseCat'\n assert CATS.get('Siamese2').__name__ == 'SiameseCat'\n\n class SphynxCat:\n pass\n\n CATS.register_module(name='Sphynx', module=SphynxCat)\n assert CATS.get('Sphynx') is SphynxCat\n\n CATS.register_module(name=['Sphynx1', 'Sphynx2'], module=SphynxCat)\n assert CATS.get('Sphynx2') is SphynxCat\n\n repr_str = 'Registry(name=cat, items={'\n repr_str += (\"'BritishShorthair': .BritishShorthair'>, \")\n repr_str += (\"'Munchkin': .Munchkin'>, \")\n repr_str += (\"'Siamese': .SiameseCat'>, \")\n repr_str += (\"'Siamese2': .SiameseCat'>, \")\n repr_str += (\"'Sphynx': .SphynxCat'>, \")\n repr_str += (\"'Sphynx1': .SphynxCat'>, \")\n repr_str += (\"'Sphynx2': .SphynxCat'>\")\n repr_str += '})'\n assert repr(CATS) == repr_str\n\n # name type\n with pytest.raises(TypeError):\n CATS.register_module(name=7474741, module=SphynxCat)\n\n # the registered module should be a class\n with pytest.raises(TypeError):\n CATS.register_module(0)\n\n # can only decorate a class\n with pytest.raises(TypeError):\n\n @CATS.register_module()\n def some_method():\n pass\n\n # begin: test old APIs\n with pytest.warns(DeprecationWarning):\n CATS.register_module(SphynxCat)\n assert CATS.get('SphynxCat').__name__ == 'SphynxCat'\n\n with pytest.warns(DeprecationWarning):\n CATS.register_module(SphynxCat, force=True)\n assert CATS.get('SphynxCat').__name__ == 'SphynxCat'\n\n with pytest.warns(DeprecationWarning):\n\n @CATS.register_module\n class NewCat:\n pass\n\n assert CATS.get('NewCat').__name__ == 'NewCat'\n\n with pytest.warns(DeprecationWarning):\n CATS.deprecated_register_module(SphynxCat, force=True)\n assert CATS.get('SphynxCat').__name__ == 'SphynxCat'\n\n with pytest.warns(DeprecationWarning):\n\n @CATS.deprecated_register_module\n class CuteCat:\n pass\n\n assert CATS.get('CuteCat').__name__ == 'CuteCat'\n\n with pytest.warns(DeprecationWarning):\n\n @CATS.deprecated_register_module(force=True)\n class NewCat2:\n pass\n\n assert CATS.get('NewCat2').__name__ == 'NewCat2'\n\n # end: test old APIs\n\n\ndef test_multi_scope_registry():\n DOGS = mmcv.Registry('dogs')\n assert DOGS.name == 'dogs'\n assert DOGS.scope == 'test_registry'\n assert DOGS.module_dict == {}\n assert len(DOGS) == 0\n\n @DOGS.register_module()\n class GoldenRetriever:\n pass\n\n assert len(DOGS) == 1\n assert DOGS.get('GoldenRetriever') is GoldenRetriever\n\n HOUNDS = mmcv.Registry('dogs', parent=DOGS, scope='hound')\n\n @HOUNDS.register_module()\n class BloodHound:\n pass\n\n assert len(HOUNDS) == 1\n assert HOUNDS.get('BloodHound') is BloodHound\n assert DOGS.get('hound.BloodHound') is BloodHound\n assert HOUNDS.get('hound.BloodHound') is BloodHound\n\n LITTLE_HOUNDS = mmcv.Registry('dogs', parent=HOUNDS, scope='little_hound')\n\n @LITTLE_HOUNDS.register_module()\n class Dachshund:\n pass\n\n assert len(LITTLE_HOUNDS) == 1\n assert LITTLE_HOUNDS.get('Dachshund') is Dachshund\n assert LITTLE_HOUNDS.get('hound.BloodHound') is BloodHound\n assert HOUNDS.get('little_hound.Dachshund') is Dachshund\n assert DOGS.get('hound.little_hound.Dachshund') is Dachshund\n\n MID_HOUNDS = mmcv.Registry('dogs', parent=HOUNDS, scope='mid_hound')\n\n @MID_HOUNDS.register_module()\n class Beagle:\n pass\n\n assert MID_HOUNDS.get('Beagle') is Beagle\n assert HOUNDS.get('mid_hound.Beagle') is Beagle\n assert DOGS.get('hound.mid_hound.Beagle') is Beagle\n assert LITTLE_HOUNDS.get('hound.mid_hound.Beagle') is Beagle\n assert MID_HOUNDS.get('hound.BloodHound') is BloodHound\n assert MID_HOUNDS.get('hound.Dachshund') is None\n\n\ndef test_build_from_cfg():\n BACKBONES = mmcv.Registry('backbone')\n\n @BACKBONES.register_module()\n class ResNet:\n\n def __init__(self, depth, stages=4):\n self.depth = depth\n self.stages = stages\n\n @BACKBONES.register_module()\n class ResNeXt:\n\n def __init__(self, depth, stages=4):\n self.depth = depth\n self.stages = stages\n\n cfg = dict(type='ResNet', depth=50)\n model = mmcv.build_from_cfg(cfg, BACKBONES)\n assert isinstance(model, ResNet)\n assert model.depth == 50 and model.stages == 4\n\n cfg = dict(type='ResNet', depth=50)\n model = mmcv.build_from_cfg(cfg, BACKBONES, default_args={'stages': 3})\n assert isinstance(model, ResNet)\n assert model.depth == 50 and model.stages == 3\n\n cfg = dict(type='ResNeXt', depth=50, stages=3)\n model = mmcv.build_from_cfg(cfg, BACKBONES)\n assert isinstance(model, ResNeXt)\n assert model.depth == 50 and model.stages == 3\n\n cfg = dict(type=ResNet, depth=50)\n model = mmcv.build_from_cfg(cfg, BACKBONES)\n assert isinstance(model, ResNet)\n assert model.depth == 50 and model.stages == 4\n\n # type defined using default_args\n cfg = dict(depth=50)\n model = mmcv.build_from_cfg(\n cfg, BACKBONES, default_args=dict(type='ResNet'))\n assert isinstance(model, ResNet)\n assert model.depth == 50 and model.stages == 4\n\n cfg = dict(depth=50)\n model = mmcv.build_from_cfg(cfg, BACKBONES, default_args=dict(type=ResNet))\n assert isinstance(model, ResNet)\n assert model.depth == 50 and model.stages == 4\n\n # not a registry\n with pytest.raises(TypeError):\n cfg = dict(type='VGG')\n model = mmcv.build_from_cfg(cfg, 'BACKBONES')\n\n # non-registered class\n with pytest.raises(KeyError):\n cfg = dict(type='VGG')\n model = mmcv.build_from_cfg(cfg, BACKBONES)\n\n # default_args must be a dict or None\n with pytest.raises(TypeError):\n cfg = dict(type='ResNet', depth=50)\n model = mmcv.build_from_cfg(cfg, BACKBONES, default_args=1)\n\n # cfg['type'] should be a str or class\n with pytest.raises(TypeError):\n cfg = dict(type=1000)\n model = mmcv.build_from_cfg(cfg, BACKBONES)\n\n # cfg should contain the key \"type\"\n with pytest.raises(KeyError, match='must contain the key \"type\"'):\n cfg = dict(depth=50, stages=4)\n model = mmcv.build_from_cfg(cfg, BACKBONES)\n\n # cfg or default_args should contain the key \"type\"\n with pytest.raises(KeyError, match='must contain the key \"type\"'):\n cfg = dict(depth=50)\n model = mmcv.build_from_cfg(\n cfg, BACKBONES, default_args=dict(stages=4))\n\n # incorrect registry type\n with pytest.raises(TypeError):\n cfg = dict(type='ResNet', depth=50)\n model = mmcv.build_from_cfg(cfg, 'BACKBONES')\n\n # incorrect default_args type\n with pytest.raises(TypeError):\n cfg = dict(type='ResNet', depth=50)\n model = mmcv.build_from_cfg(cfg, BACKBONES, default_args=0)\n\n # incorrect arguments\n with pytest.raises(TypeError):\n cfg = dict(type='ResNet', non_existing_arg=50)\n model = mmcv.build_from_cfg(cfg, BACKBONES)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_testing.py", "content": "import numpy as np\nimport pytest\n\nimport mmcv\n\ntry:\n import torch\nexcept ImportError:\n torch = None\nelse:\n import torch.nn as nn\n\n\ndef test_assert_dict_contains_subset():\n dict_obj = {'a': 'test1', 'b': 2, 'c': (4, 6)}\n\n # case 1\n expected_subset = {'a': 'test1', 'b': 2, 'c': (4, 6)}\n assert mmcv.assert_dict_contains_subset(dict_obj, expected_subset)\n\n # case 2\n expected_subset = {'a': 'test1', 'b': 2, 'c': (6, 4)}\n assert not mmcv.assert_dict_contains_subset(dict_obj, expected_subset)\n\n # case 3\n expected_subset = {'a': 'test1', 'b': 2, 'c': None}\n assert not mmcv.assert_dict_contains_subset(dict_obj, expected_subset)\n\n # case 4\n expected_subset = {'a': 'test1', 'b': 2, 'd': (4, 6)}\n assert not mmcv.assert_dict_contains_subset(dict_obj, expected_subset)\n\n # case 5\n dict_obj = {\n 'a': 'test1',\n 'b': 2,\n 'c': (4, 6),\n 'd': np.array([[5, 3, 5], [1, 2, 3]])\n }\n expected_subset = {\n 'a': 'test1',\n 'b': 2,\n 'c': (4, 6),\n 'd': np.array([[5, 3, 5], [6, 2, 3]])\n }\n assert not mmcv.assert_dict_contains_subset(dict_obj, expected_subset)\n\n # case 6\n dict_obj = {'a': 'test1', 'b': 2, 'c': (4, 6), 'd': np.array([[1]])}\n expected_subset = {'a': 'test1', 'b': 2, 'c': (4, 6), 'd': np.array([[1]])}\n assert mmcv.assert_dict_contains_subset(dict_obj, expected_subset)\n\n if torch is not None:\n dict_obj = {\n 'a': 'test1',\n 'b': 2,\n 'c': (4, 6),\n 'd': torch.tensor([5, 3, 5])\n }\n\n # case 7\n expected_subset = {'d': torch.tensor([5, 5, 5])}\n assert not mmcv.assert_dict_contains_subset(dict_obj, expected_subset)\n\n # case 8\n expected_subset = {'d': torch.tensor([[5, 3, 5], [4, 1, 2]])}\n assert not mmcv.assert_dict_contains_subset(dict_obj, expected_subset)\n\n\ndef test_assert_attrs_equal():\n\n class TestExample(object):\n a, b, c = 1, ('wvi', 3), [4.5, 3.14]\n\n def test_func(self):\n return self.b\n\n # case 1\n assert mmcv.assert_attrs_equal(TestExample, {\n 'a': 1,\n 'b': ('wvi', 3),\n 'c': [4.5, 3.14]\n })\n\n # case 2\n assert not mmcv.assert_attrs_equal(TestExample, {\n 'a': 1,\n 'b': ('wvi', 3),\n 'c': [4.5, 3.14, 2]\n })\n\n # case 3\n assert not mmcv.assert_attrs_equal(TestExample, {\n 'bc': 54,\n 'c': [4.5, 3.14]\n })\n\n # case 4\n assert mmcv.assert_attrs_equal(TestExample, {\n 'b': ('wvi', 3),\n 'test_func': TestExample.test_func\n })\n\n if torch is not None:\n\n class TestExample(object):\n a, b = torch.tensor([1]), torch.tensor([4, 5])\n\n # case 5\n assert mmcv.assert_attrs_equal(TestExample, {\n 'a': torch.tensor([1]),\n 'b': torch.tensor([4, 5])\n })\n\n # case 6\n assert not mmcv.assert_attrs_equal(TestExample, {\n 'a': torch.tensor([1]),\n 'b': torch.tensor([4, 6])\n })\n\n\nassert_dict_has_keys_data_1 = [({\n 'res_layer': 1,\n 'norm_layer': 2,\n 'dense_layer': 3\n})]\nassert_dict_has_keys_data_2 = [(['res_layer', 'dense_layer'], True),\n (['res_layer', 'conv_layer'], False)]\n\n\n@pytest.mark.parametrize('obj', assert_dict_has_keys_data_1)\n@pytest.mark.parametrize('expected_keys, ret_value',\n assert_dict_has_keys_data_2)\ndef test_assert_dict_has_keys(obj, expected_keys, ret_value):\n assert mmcv.assert_dict_has_keys(obj, expected_keys) == ret_value\n\n\nassert_keys_equal_data_1 = [(['res_layer', 'norm_layer', 'dense_layer'])]\nassert_keys_equal_data_2 = [(['res_layer', 'norm_layer', 'dense_layer'], True),\n (['res_layer', 'dense_layer', 'norm_layer'], True),\n (['res_layer', 'norm_layer'], False),\n (['res_layer', 'conv_layer', 'norm_layer'], False)]\n\n\n@pytest.mark.parametrize('result_keys', assert_keys_equal_data_1)\n@pytest.mark.parametrize('target_keys, ret_value', assert_keys_equal_data_2)\ndef test_assert_keys_equal(result_keys, target_keys, ret_value):\n assert mmcv.assert_keys_equal(result_keys, target_keys) == ret_value\n\n\n@pytest.mark.skipif(torch is None, reason='requires torch library')\ndef test_assert_is_norm_layer():\n # case 1\n assert not mmcv.assert_is_norm_layer(nn.Conv3d(3, 64, 3))\n\n # case 2\n assert mmcv.assert_is_norm_layer(nn.BatchNorm3d(128))\n\n # case 3\n assert mmcv.assert_is_norm_layer(nn.GroupNorm(8, 64))\n\n # case 4\n assert not mmcv.assert_is_norm_layer(nn.Sigmoid())\n\n\n@pytest.mark.skipif(torch is None, reason='requires torch library')\ndef test_assert_params_all_zeros():\n demo_module = nn.Conv2d(3, 64, 3)\n nn.init.constant_(demo_module.weight, 0)\n nn.init.constant_(demo_module.bias, 0)\n assert mmcv.assert_params_all_zeros(demo_module)\n\n nn.init.xavier_normal_(demo_module.weight)\n nn.init.constant_(demo_module.bias, 0)\n assert not mmcv.assert_params_all_zeros(demo_module)\n\n demo_module = nn.Linear(2048, 400, bias=False)\n nn.init.constant_(demo_module.weight, 0)\n assert mmcv.assert_params_all_zeros(demo_module)\n\n nn.init.normal_(demo_module.weight, mean=0, std=0.01)\n assert not mmcv.assert_params_all_zeros(demo_module)\n\n\ndef test_check_python_script(capsys):\n mmcv.utils.check_python_script('./tests/data/scripts/hello.py zz')\n captured = capsys.readouterr().out\n assert captured == 'hello zz!\\n'\n mmcv.utils.check_python_script('./tests/data/scripts/hello.py agent')\n captured = capsys.readouterr().out\n assert captured == 'hello agent!\\n'\n # Make sure that wrong cmd raises an error\n with pytest.raises(SystemExit):\n mmcv.utils.check_python_script('./tests/data/scripts/hello.py li zz')\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_timer.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport time\n\nimport pytest\n\nimport mmcv\n\n\ndef test_timer_init():\n timer = mmcv.Timer(start=False)\n assert not timer.is_running\n timer.start()\n assert timer.is_running\n timer = mmcv.Timer()\n assert timer.is_running\n\n\ndef test_timer_run():\n timer = mmcv.Timer()\n time.sleep(1)\n assert abs(timer.since_start() - 1) < 1e-2\n time.sleep(1)\n assert abs(timer.since_last_check() - 1) < 1e-2\n assert abs(timer.since_start() - 2) < 1e-2\n timer = mmcv.Timer(False)\n with pytest.raises(mmcv.TimerError):\n timer.since_start()\n with pytest.raises(mmcv.TimerError):\n timer.since_last_check()\n\n\ndef test_timer_context(capsys):\n with mmcv.Timer():\n time.sleep(1)\n out, _ = capsys.readouterr()\n assert abs(float(out) - 1) < 1e-2\n with mmcv.Timer(print_tmpl='time: {:.1f}s'):\n time.sleep(1)\n out, _ = capsys.readouterr()\n assert out == 'time: 1.0s\\n'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_trace.py", "content": "import pytest\nimport torch\n\nfrom mmcv.utils import digit_version, is_jit_tracing\n\n\n@pytest.mark.skipif(\n digit_version(torch.__version__) < digit_version('1.6.0'),\n reason='torch.jit.is_tracing is not available before 1.6.0')\ndef test_is_jit_tracing():\n\n def foo(x):\n if is_jit_tracing():\n return x\n else:\n return x.tolist()\n\n x = torch.rand(3)\n # test without trace\n assert isinstance(foo(x), list)\n\n # test with trace\n traced_foo = torch.jit.trace(foo, (torch.rand(1), ))\n assert isinstance(traced_foo(x), torch.Tensor)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_utils/test_version_utils.py", "content": "from unittest.mock import patch\n\nimport pytest\n\nfrom mmcv import get_git_hash, parse_version_info\nfrom mmcv.utils import digit_version\n\n\ndef test_digit_version():\n assert digit_version('0.2.16') == (0, 2, 16, 0, 0, 0)\n assert digit_version('1.2.3') == (1, 2, 3, 0, 0, 0)\n assert digit_version('1.2.3rc0') == (1, 2, 3, 0, -1, 0)\n assert digit_version('1.2.3rc1') == (1, 2, 3, 0, -1, 1)\n assert digit_version('1.0rc0') == (1, 0, 0, 0, -1, 0)\n assert digit_version('1.0') == digit_version('1.0.0')\n assert digit_version('1.5.0+cuda90_cudnn7.6.3_lms') == digit_version('1.5')\n assert digit_version('1.0.0dev') < digit_version('1.0.0a')\n assert digit_version('1.0.0a') < digit_version('1.0.0a1')\n assert digit_version('1.0.0a') < digit_version('1.0.0b')\n assert digit_version('1.0.0b') < digit_version('1.0.0rc')\n assert digit_version('1.0.0rc1') < digit_version('1.0.0')\n assert digit_version('1.0.0') < digit_version('1.0.0post')\n assert digit_version('1.0.0post') < digit_version('1.0.0post1')\n assert digit_version('v1') == (1, 0, 0, 0, 0, 0)\n assert digit_version('v1.1.5') == (1, 1, 5, 0, 0, 0)\n with pytest.raises(AssertionError):\n digit_version('a')\n with pytest.raises(AssertionError):\n digit_version('1x')\n with pytest.raises(AssertionError):\n digit_version('1.x')\n\n\ndef test_parse_version_info():\n assert parse_version_info('0.2.16') == (0, 2, 16, 0, 0, 0)\n assert parse_version_info('1.2.3') == (1, 2, 3, 0, 0, 0)\n assert parse_version_info('1.2.3rc0') == (1, 2, 3, 0, 'rc', 0)\n assert parse_version_info('1.2.3rc1') == (1, 2, 3, 0, 'rc', 1)\n assert parse_version_info('1.0rc0') == (1, 0, 0, 0, 'rc', 0)\n\n\ndef _mock_cmd_success(cmd):\n return '3b46d33e90c397869ad5103075838fdfc9812aa0'.encode('ascii')\n\n\ndef _mock_cmd_fail(cmd):\n raise OSError\n\n\ndef test_get_git_hash():\n with patch('mmcv.utils.version_utils._minimal_ext_cmd', _mock_cmd_success):\n assert get_git_hash() == '3b46d33e90c397869ad5103075838fdfc9812aa0'\n assert get_git_hash(digits=6) == '3b46d3'\n assert get_git_hash(digits=100) == get_git_hash()\n with patch('mmcv.utils.version_utils._minimal_ext_cmd', _mock_cmd_fail):\n assert get_git_hash() == 'unknown'\n assert get_git_hash(fallback='n/a') == 'n/a'\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_video/test_optflow.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport os.path as osp\nimport tempfile\n\nimport cv2\nimport numpy as np\nimport pytest\nfrom numpy.testing import assert_array_almost_equal, assert_array_equal\n\nimport mmcv\n\n\ndef test_flowread():\n data_dir = osp.join(osp.dirname(__file__), '../data')\n flow_shape = (60, 80, 2)\n\n # read .flo file\n flow = mmcv.flowread(osp.join(data_dir, 'optflow.flo'))\n assert flow.shape == flow_shape\n\n # pseudo read\n flow_same = mmcv.flowread(flow)\n assert_array_equal(flow, flow_same)\n\n # read quantized flow concatenated vertically\n flow = mmcv.flowread(\n osp.join(data_dir, 'optflow_concat0.jpg'), quantize=True, denorm=True)\n assert flow.shape == flow_shape\n\n # read quantized flow concatenated horizontally\n flow = mmcv.flowread(\n osp.join(data_dir, 'optflow_concat1.jpg'),\n quantize=True,\n concat_axis=1,\n denorm=True)\n assert flow.shape == flow_shape\n\n # test exceptions\n notflow_file = osp.join(data_dir, 'color.jpg')\n with pytest.raises(TypeError):\n mmcv.flowread(1)\n with pytest.raises(IOError):\n mmcv.flowread(notflow_file)\n with pytest.raises(IOError):\n mmcv.flowread(notflow_file, quantize=True)\n with pytest.raises(ValueError):\n mmcv.flowread(np.zeros((100, 100, 1)))\n\n\ndef test_flowwrite():\n flow = np.random.rand(100, 100, 2).astype(np.float32)\n\n # write to a .flo file\n tmp_filehandler, filename = tempfile.mkstemp()\n mmcv.flowwrite(flow, filename)\n flow_from_file = mmcv.flowread(filename)\n assert_array_equal(flow, flow_from_file)\n os.close(tmp_filehandler)\n os.remove(filename)\n\n # write to two .jpg files\n tmp_filename = osp.join(tempfile.gettempdir(), 'mmcv_test_flow.jpg')\n for concat_axis in range(2):\n mmcv.flowwrite(\n flow, tmp_filename, quantize=True, concat_axis=concat_axis)\n shape = (200, 100) if concat_axis == 0 else (100, 200)\n assert osp.isfile(tmp_filename)\n assert mmcv.imread(tmp_filename, flag='unchanged').shape == shape\n os.remove(tmp_filename)\n\n # test exceptions\n with pytest.raises(AssertionError):\n mmcv.flowwrite(flow, tmp_filename, quantize=True, concat_axis=2)\n\n\ndef test_quantize_flow():\n flow = (np.random.rand(10, 8, 2).astype(np.float32) - 0.5) * 15\n max_val = 5.0\n dx, dy = mmcv.quantize_flow(flow, max_val=max_val, norm=False)\n ref = np.zeros_like(flow, dtype=np.uint8)\n for i in range(ref.shape[0]):\n for j in range(ref.shape[1]):\n for k in range(ref.shape[2]):\n val = flow[i, j, k] + max_val\n val = min(max(val, 0), 2 * max_val)\n ref[i, j, k] = min(np.floor(255 * val / (2 * max_val)), 254)\n assert_array_equal(dx, ref[..., 0])\n assert_array_equal(dy, ref[..., 1])\n max_val = 0.5\n dx, dy = mmcv.quantize_flow(flow, max_val=max_val, norm=True)\n ref = np.zeros_like(flow, dtype=np.uint8)\n for i in range(ref.shape[0]):\n for j in range(ref.shape[1]):\n for k in range(ref.shape[2]):\n scale = flow.shape[1] if k == 0 else flow.shape[0]\n val = flow[i, j, k] / scale + max_val\n val = min(max(val, 0), 2 * max_val)\n ref[i, j, k] = min(np.floor(255 * val / (2 * max_val)), 254)\n assert_array_equal(dx, ref[..., 0])\n assert_array_equal(dy, ref[..., 1])\n\n\ndef test_dequantize_flow():\n dx = np.random.randint(256, size=(10, 8), dtype=np.uint8)\n dy = np.random.randint(256, size=(10, 8), dtype=np.uint8)\n max_val = 5.0\n flow = mmcv.dequantize_flow(dx, dy, max_val=max_val, denorm=False)\n ref = np.zeros_like(flow, dtype=np.float32)\n for i in range(ref.shape[0]):\n for j in range(ref.shape[1]):\n ref[i, j, 0] = float(dx[i, j] + 0.5) * 2 * max_val / 255 - max_val\n ref[i, j, 1] = float(dy[i, j] + 0.5) * 2 * max_val / 255 - max_val\n assert_array_almost_equal(flow, ref)\n max_val = 0.5\n flow = mmcv.dequantize_flow(dx, dy, max_val=max_val, denorm=True)\n h, w = dx.shape\n ref = np.zeros_like(flow, dtype=np.float32)\n for i in range(ref.shape[0]):\n for j in range(ref.shape[1]):\n ref[i, j,\n 0] = (float(dx[i, j] + 0.5) * 2 * max_val / 255 - max_val) * w\n ref[i, j,\n 1] = (float(dy[i, j] + 0.5) * 2 * max_val / 255 - max_val) * h\n assert_array_almost_equal(flow, ref)\n\n\ndef test_flow2rgb():\n flow = np.array([[[0, 0], [0.5, 0.5], [1, 1], [2, 1], [3, np.inf]]],\n dtype=np.float32)\n flow_img = mmcv.flow2rgb(flow)\n # yapf: disable\n assert_array_almost_equal(\n flow_img,\n np.array([[[1., 1., 1.],\n [1., 0.826074731, 0.683772236],\n [1., 0.652149462, 0.367544472],\n [1., 0.265650552, 5.96046448e-08],\n [0., 0., 0.]]],\n dtype=np.float32))\n # yapf: enable\n\n\ndef test_flow_warp():\n\n img = np.zeros((5, 5, 3))\n img[2, 2, 0] = 1\n flow = np.ones((5, 5, 2))\n\n res_nn = mmcv.flow_warp(img, flow, interpolate_mode='nearest')\n res_bi = mmcv.flow_warp(img, flow, interpolate_mode='bilinear')\n\n assert_array_almost_equal(res_nn, res_bi, decimal=5)\n\n img = np.zeros((5, 5, 1))\n img[2, 2, 0] = 1\n img[2, 3, 0] = 0.75\n flow = np.zeros((5, 5, 2))\n flow[2, 2, :] = [0.5, 0.7]\n\n res_ = np.copy(img)\n res_[2, 2] = 0.5 * 0.3 + 0.75 * 0.5 * 0.3\n res_bi = mmcv.flow_warp(img, flow, interpolate_mode='bilinear')\n assert_array_almost_equal(res_, res_bi, decimal=5)\n\n with pytest.raises(NotImplementedError):\n _ = mmcv.flow_warp(img, flow, interpolate_mode='xxx')\n\n with pytest.raises(AssertionError):\n _ = mmcv.flow_warp(img, flow[:, :, 0], interpolate_mode='xxx')\n\n\ndef test_make_color_wheel():\n default_color_wheel = mmcv.make_color_wheel()\n color_wheel = mmcv.make_color_wheel([2, 2, 2, 2, 2, 2])\n # yapf: disable\n assert_array_equal(default_color_wheel, np.array(\n [[1. , 0. , 0. ], # noqa\n [1. , 0.06666667, 0. ], # noqa\n [1. , 0.13333334, 0. ], # noqa\n [1. , 0.2 , 0. ], # noqa\n [1. , 0.26666668, 0. ], # noqa\n [1. , 0.33333334, 0. ], # noqa\n [1. , 0.4 , 0. ], # noqa\n [1. , 0.46666667, 0. ], # noqa\n [1. , 0.53333336, 0. ], # noqa\n [1. , 0.6 , 0. ], # noqa\n [1. , 0.6666667 , 0. ], # noqa\n [1. , 0.73333335, 0. ], # noqa\n [1. , 0.8 , 0. ], # noqa\n [1. , 0.8666667 , 0. ], # noqa\n [1. , 0.93333334, 0. ], # noqa\n [1. , 1. , 0. ], # noqa\n [0.8333333 , 1. , 0. ], # noqa\n [0.6666667 , 1. , 0. ], # noqa\n [0.5 , 1. , 0. ], # noqa\n [0.33333334, 1. , 0. ], # noqa\n [0.16666667, 1. , 0. ], # noqa\n [0. , 1. , 0. ], # noqa\n [0. , 1. , 0.25 ], # noqa\n [0. , 1. , 0.5 ], # noqa\n [0. , 1. , 0.75 ], # noqa\n [0. , 1. , 1. ], # noqa\n [0. , 0.90909094, 1. ], # noqa\n [0. , 0.8181818 , 1. ], # noqa\n [0. , 0.72727275, 1. ], # noqa\n [0. , 0.6363636 , 1. ], # noqa\n [0. , 0.54545456, 1. ], # noqa\n [0. , 0.45454547, 1. ], # noqa\n [0. , 0.36363637, 1. ], # noqa\n [0. , 0.27272728, 1. ], # noqa\n [0. , 0.18181819, 1. ], # noqa\n [0. , 0.09090909, 1. ], # noqa\n [0. , 0. , 1. ], # noqa\n [0.07692308, 0. , 1. ], # noqa\n [0.15384616, 0. , 1. ], # noqa\n [0.23076923, 0. , 1. ], # noqa\n [0.30769232, 0. , 1. ], # noqa\n [0.3846154 , 0. , 1. ], # noqa\n [0.46153846, 0. , 1. ], # noqa\n [0.53846157, 0. , 1. ], # noqa\n [0.61538464, 0. , 1. ], # noqa\n [0.6923077 , 0. , 1. ], # noqa\n [0.7692308 , 0. , 1. ], # noqa\n [0.84615386, 0. , 1. ], # noqa\n [0.9230769 , 0. , 1. ], # noqa\n [1. , 0. , 1. ], # noqa\n [1. , 0. , 0.8333333 ], # noqa\n [1. , 0. , 0.6666667 ], # noqa\n [1. , 0. , 0.5 ], # noqa\n [1. , 0. , 0.33333334], # noqa\n [1. , 0. , 0.16666667]], dtype=np.float32)) # noqa\n\n assert_array_equal(\n color_wheel,\n np.array([[1., 0. , 0. ], # noqa\n [1. , 0.5, 0. ], # noqa\n [1. , 1. , 0. ], # noqa\n [0.5, 1. , 0. ], # noqa\n [0. , 1. , 0. ], # noqa\n [0. , 1. , 0.5], # noqa\n [0. , 1. , 1. ], # noqa\n [0. , 0.5, 1. ], # noqa\n [0. , 0. , 1. ], # noqa\n [0.5, 0. , 1. ], # noqa\n [1. , 0. , 1. ], # noqa\n [1. , 0. , 0.5]], dtype=np.float32)) # noqa\n # yapf: enable\n\n\ndef test_flow_from_bytes():\n data_dir = osp.join(osp.dirname(__file__), '../data')\n flow_shape = (60, 80, 2)\n flow_file = osp.join(data_dir, 'optflow.flo')\n\n # read .flo file\n flow_fromfile = mmcv.flowread(flow_file)\n\n with open(flow_file, 'rb') as f:\n flow_bytes = f.read()\n flow_frombytes = mmcv.flow_from_bytes(flow_bytes)\n\n assert flow_frombytes.shape == flow_shape\n assert np.all(flow_frombytes == flow_fromfile)\n\n\ndef test_sparse_flow_from_bytes():\n data_dir = osp.join(osp.dirname(__file__), '../data')\n flow_file = osp.join(data_dir, 'sparse_flow.png')\n\n with open(flow_file, 'rb') as f:\n flow_bytes = f.read()\n # read flow from bytes\n flow_frombytes, valid_frombytes = mmcv.sparse_flow_from_bytes(flow_bytes)\n\n # test flow shape is [H, W, 2] and valid shape is [H, W]\n assert flow_frombytes.shape[:2] == valid_frombytes.shape\n assert flow_frombytes.shape[2] == 2\n\n def read_sparse_flow_from_file():\n flow = cv2.imread(flow_file, cv2.IMREAD_ANYDEPTH | cv2.IMREAD_COLOR)\n flow = flow[:, :, ::-1].astype(np.float32)\n flow, valid = flow[:, :, :2], flow[:, :, 2]\n flow = (flow - 2**15) / 64.0\n return flow, valid\n\n # read flow from file\n flow_flowfile, valid_fromfile = read_sparse_flow_from_file()\n\n assert np.all(flow_frombytes == flow_flowfile)\n assert np.all(valid_frombytes == valid_fromfile)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_video/test_processing.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport os.path as osp\nimport platform\nimport tempfile\n\nimport pytest\n\nimport mmcv\n\n\nclass TestVideoEditor:\n\n @classmethod\n def setup_class(cls):\n cls.video_path = osp.join(osp.dirname(__file__), '../data/test.mp4')\n cls.num_frames = 168\n\n @pytest.mark.skipif(platform.system() == 'Windows', reason='skip windows')\n def test_cut_concat_video(self):\n part1_file = osp.join(tempfile.gettempdir(), '.mmcv_test1.mp4')\n part2_file = osp.join(tempfile.gettempdir(), '.mmcv_test2.mp4')\n mmcv.cut_video(self.video_path, part1_file, end=3, vcodec='h264')\n mmcv.cut_video(self.video_path, part2_file, start=3, vcodec='h264')\n v1 = mmcv.VideoReader(part1_file)\n v2 = mmcv.VideoReader(part2_file)\n assert len(v1) == 75\n assert len(v2) == self.num_frames - 75\n\n out_file = osp.join(tempfile.gettempdir(), '.mmcv_test.mp4')\n mmcv.concat_video([part1_file, part2_file], out_file)\n v = mmcv.VideoReader(out_file)\n assert len(v) == self.num_frames\n os.remove(part1_file)\n os.remove(part2_file)\n os.remove(out_file)\n\n @pytest.mark.skipif(platform.system() == 'Windows', reason='skip windows')\n def test_resize_video(self):\n out_file = osp.join(tempfile.gettempdir(), '.mmcv_test.mp4')\n mmcv.resize_video(\n self.video_path, out_file, (200, 100), log_level='panic')\n v = mmcv.VideoReader(out_file)\n assert v.resolution == (200, 100)\n os.remove(out_file)\n mmcv.resize_video(self.video_path, out_file, ratio=2)\n v = mmcv.VideoReader(out_file)\n assert v.resolution == (294 * 2, 240 * 2)\n os.remove(out_file)\n mmcv.resize_video(self.video_path, out_file, (1000, 480), keep_ar=True)\n v = mmcv.VideoReader(out_file)\n assert v.resolution == (294 * 2, 240 * 2)\n os.remove(out_file)\n mmcv.resize_video(\n self.video_path, out_file, ratio=(2, 1.5), keep_ar=True)\n v = mmcv.VideoReader(out_file)\n assert v.resolution == (294 * 2, 360)\n os.remove(out_file)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_video/test_reader.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport os\nimport os.path as osp\nimport shutil\nimport tempfile\nfrom collections import OrderedDict\n\nimport pytest\n\nimport mmcv\n\n\nclass TestCache:\n\n def test_init(self):\n with pytest.raises(ValueError):\n mmcv.Cache(0)\n cache = mmcv.Cache(100)\n assert cache.capacity == 100\n assert cache.size == 0\n\n def test_put(self):\n cache = mmcv.Cache(3)\n for i in range(1, 4):\n cache.put(f'k{i}', i)\n assert cache.size == i\n assert cache._cache == OrderedDict([('k1', 1), ('k2', 2), ('k3', 3)])\n cache.put('k4', 4)\n assert cache.size == 3\n assert cache._cache == OrderedDict([('k2', 2), ('k3', 3), ('k4', 4)])\n cache.put('k2', 2)\n assert cache._cache == OrderedDict([('k2', 2), ('k3', 3), ('k4', 4)])\n\n def test_get(self):\n cache = mmcv.Cache(3)\n assert cache.get('key_none') is None\n assert cache.get('key_none', 0) == 0\n cache.put('k1', 1)\n assert cache.get('k1') == 1\n\n\nclass TestVideoReader:\n\n @classmethod\n def setup_class(cls):\n cls.video_path = osp.join(osp.dirname(__file__), '../data/test.mp4')\n cls.num_frames = 168\n cls.video_url = 'https://www.learningcontainer.com/wp-content/uploads/2020/05/sample-mp4-file.mp4' # noqa: E501\n\n def test_load(self):\n # read from video file\n v = mmcv.VideoReader(self.video_path)\n assert v.width == 294\n assert v.height == 240\n assert v.fps == 25\n assert v.frame_cnt == self.num_frames\n assert len(v) == self.num_frames\n assert v.opened\n import cv2\n assert isinstance(v.vcap, type(cv2.VideoCapture()))\n\n # read from video url\n v = mmcv.VideoReader(self.video_url)\n assert v.width == 320\n assert v.height == 240\n assert v.fps == 15\n assert v.frame_cnt == 1889\n assert len(v) == 1889\n assert v.opened\n assert isinstance(v.vcap, type(cv2.VideoCapture()))\n\n def test_read(self):\n v = mmcv.VideoReader(self.video_path)\n img = v.read()\n assert int(round(img.mean())) == 94\n img = v.get_frame(63)\n assert int(round(img.mean())) == 94\n img = v[64]\n assert int(round(img.mean())) == 205\n img = v[-104]\n assert int(round(img.mean())) == 205\n img = v[63]\n assert int(round(img.mean())) == 94\n img = v[-105]\n assert int(round(img.mean())) == 94\n img = v.read()\n assert int(round(img.mean())) == 205\n with pytest.raises(IndexError):\n v.get_frame(self.num_frames + 1)\n with pytest.raises(IndexError):\n v[-self.num_frames - 1]\n\n def test_slice(self):\n v = mmcv.VideoReader(self.video_path)\n imgs = v[-105:-103]\n assert int(round(imgs[0].mean())) == 94\n assert int(round(imgs[1].mean())) == 205\n assert len(imgs) == 2\n imgs = v[63:65]\n assert int(round(imgs[0].mean())) == 94\n assert int(round(imgs[1].mean())) == 205\n assert len(imgs) == 2\n imgs = v[64:62:-1]\n assert int(round(imgs[0].mean())) == 205\n assert int(round(imgs[1].mean())) == 94\n assert len(imgs) == 2\n imgs = v[:5]\n assert len(imgs) == 5\n for img in imgs:\n assert int(round(img.mean())) == 94\n imgs = v[165:]\n assert len(imgs) == 3\n for img in imgs:\n assert int(round(img.mean())) == 0\n imgs = v[-3:]\n assert len(imgs) == 3\n for img in imgs:\n assert int(round(img.mean())) == 0\n\n def test_current_frame(self):\n v = mmcv.VideoReader(self.video_path)\n assert v.current_frame() is None\n v.read()\n img = v.current_frame()\n assert int(round(img.mean())) == 94\n\n def test_position(self):\n v = mmcv.VideoReader(self.video_path)\n assert v.position == 0\n for _ in range(10):\n v.read()\n assert v.position == 10\n v.get_frame(99)\n assert v.position == 100\n\n def test_iterator(self):\n cnt = 0\n for img in mmcv.VideoReader(self.video_path):\n cnt += 1\n assert img.shape == (240, 294, 3)\n assert cnt == self.num_frames\n\n def test_with(self):\n with mmcv.VideoReader(self.video_path) as v:\n assert v.opened\n assert not v.opened\n\n def test_cvt2frames(self):\n v = mmcv.VideoReader(self.video_path)\n frame_dir = tempfile.mkdtemp()\n v.cvt2frames(frame_dir)\n assert osp.isdir(frame_dir)\n for i in range(self.num_frames):\n filename = f'{frame_dir}/{i:06d}.jpg'\n assert osp.isfile(filename)\n os.remove(filename)\n\n v = mmcv.VideoReader(self.video_path)\n v.cvt2frames(frame_dir, show_progress=False)\n assert osp.isdir(frame_dir)\n for i in range(self.num_frames):\n filename = f'{frame_dir}/{i:06d}.jpg'\n assert osp.isfile(filename)\n os.remove(filename)\n\n v = mmcv.VideoReader(self.video_path)\n v.cvt2frames(\n frame_dir,\n file_start=100,\n filename_tmpl='{:03d}.JPEG',\n start=100,\n max_num=20)\n assert osp.isdir(frame_dir)\n for i in range(100, 120):\n filename = f'{frame_dir}/{i:03d}.JPEG'\n assert osp.isfile(filename)\n os.remove(filename)\n shutil.rmtree(frame_dir)\n\n def test_frames2video(self):\n v = mmcv.VideoReader(self.video_path)\n frame_dir = tempfile.mkdtemp()\n v.cvt2frames(frame_dir)\n assert osp.isdir(frame_dir)\n for i in range(self.num_frames):\n filename = f'{frame_dir}/{i:06d}.jpg'\n assert osp.isfile(filename)\n\n out_filename = osp.join(tempfile.gettempdir(), 'mmcv_test.avi')\n mmcv.frames2video(frame_dir, out_filename)\n v = mmcv.VideoReader(out_filename)\n assert v.fps == 30\n assert len(v) == self.num_frames\n\n mmcv.frames2video(\n frame_dir,\n out_filename,\n fps=25,\n start=10,\n end=50,\n show_progress=False)\n\n with mmcv.VideoReader(out_filename) as v:\n assert v.fps == 25\n assert len(v) == 40\n\n for i in range(self.num_frames):\n filename = f'{frame_dir}/{i:06d}.jpg'\n os.remove(filename)\n shutil.rmtree(frame_dir)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/mmcv/tests/test_visualization.py", "content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport numpy as np\nimport pytest\n\nimport mmcv\n\n\ndef test_color():\n assert mmcv.color_val(mmcv.Color.blue) == (255, 0, 0)\n assert mmcv.color_val('green') == (0, 255, 0)\n assert mmcv.color_val((1, 2, 3)) == (1, 2, 3)\n assert mmcv.color_val(100) == (100, 100, 100)\n assert mmcv.color_val(np.zeros(3, dtype=int)) == (0, 0, 0)\n with pytest.raises(TypeError):\n mmcv.color_val([255, 255, 255])\n with pytest.raises(TypeError):\n mmcv.color_val(1.0)\n with pytest.raises(AssertionError):\n mmcv.color_val((0, 0, 500))\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/common/dataset.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch.utils.data as data\nimport torch\nimport h5py\nimport cv2\nimport numpy as np\n\nclass Dataset_Pro(data.Dataset):\n def __init__(self, file_path, img_scale):\n super(Dataset_Pro, self).__init__()\n\n data = h5py.File(file_path) # NxCxHxW = 0x1x2x3\n\n print(f\"loading Dataset_Pro: {file_path} with {img_scale}\")\n # tensor type:\n gt1 = data[\"gt\"][...] # convert to np tpye for CV2.filter\n gt1 = np.array(gt1, dtype=np.float32) / img_scale\n self.gt = torch.from_numpy(gt1) # NxCxHxW:\n\n ms1 = data[\"ms\"][...] # convert to np tpye for CV2.filter\n ms1 = np.array(ms1, dtype=np.float32) / img_scale\n\n self.ms = torch.from_numpy(ms1)\n\n lms1 = data[\"lms\"][...] # convert to np tpye for CV2.filter\n lms1 = np.array(lms1, dtype=np.float32) / img_scale\n self.lms = torch.from_numpy(lms1)\n\n\n pan1 = data['pan'][...] # Nx1xHxW\n pan1 = np.array(pan1, dtype=np.float32) / img_scale # Nx1xHxW\n self.pan = torch.from_numpy(pan1) # Nx1xHxW:\n\n if 'valid' in file_path:\n self.gt = self.gt.permute([0, 2, 3, 1])\n\n print(pan1.shape, lms1.shape, gt1.shape, ms1.shape)\n #####必要函数\n def __getitem__(self, index):\n return {'gt':self.gt[index, :, :, :].float(),\n 'lms':self.lms[index, :, :, :].float(),\n 'ms':self.ms[index, :, :, :].float(),\n 'pan':self.pan[index, :, :, :].float()}\n\n #####必要函数\n def __len__(self):\n return self.gt.shape[0]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/common/dataset_hp.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch.utils.data as data\nimport torch\nimport h5py\nimport cv2\nimport numpy as np\n\n\ndef get_edge(data): # for training: HxWxC\n rs = np.zeros_like(data)\n N = data.shape[0]\n for i in range(N):\n if len(data.shape) == 3:\n rs[i, :, :] = data[i, :, :] - cv2.boxFilter(data[i, :, :], -1, (5, 5))\n else:\n rs[i, :, :, :] = data[i, :, :, :] - cv2.boxFilter(data[i, :, :, :], -1, (5, 5))\n return rs\n\n\nclass Dataset_Pro(data.Dataset):\n def __init__(self, file_path, img_scale):\n super(Dataset_Pro, self).__init__()\n data = h5py.File(file_path) # NxCxHxW = 0x1x2x3=8806x8x64x64\n\n # tensor type:\n gt1 = data[\"gt\"][...] # convert to np tpye for CV2.filter\n gt1 = np.array(gt1, dtype=np.float32) / img_scale\n self.gt = torch.from_numpy(gt1) # NxCxHxW:\n\n lms1 = data[\"lms\"][...] # convert to np tpye for CV2.filter\n lms1 = np.array(lms1, dtype=np.float32) / img_scale\n self.lms = torch.from_numpy(lms1)\n\n ms1 = data[\"ms\"][...] # NxCxHxW=0,1,2,3\n ms1 = np.array(ms1.transpose(0, 2, 3, 1), dtype=np.float32) / img_scale # NxHxWxC\n ms1_tmp = get_edge(ms1) # NxHxWxC\n self.ms_hp = torch.from_numpy(ms1_tmp).permute(0, 3, 1, 2) # NxCxHxW:\n\n pan1 = data['pan'][...] # Nx1xHxW\n pan1 = np.array(pan1.transpose(0, 2, 3, 1), dtype=np.float32) / img_scale # NxHxWx1\n pan1 = np.squeeze(pan1, axis=3) # NxHxW\n pan_hp_tmp = get_edge(pan1) # NxHxW\n pan_hp_tmp = np.expand_dims(pan_hp_tmp, axis=3) # NxHxWx1\n self.pan_hp = torch.from_numpy(pan_hp_tmp).permute(0, 3, 1, 2) # Nx1xHxW:\n print(\n f\"gt: {self.gt.size()}, lms: {self.lms.size()}, pan_hp: {self.pan_hp.size()}, ms_hp: {self.ms_hp.size()} with {img_scale}\")\n\n #####必要函数\n def __getitem__(self, index):\n return {'gt': self.gt[index, :, :, :].float(),\n 'lms': self.lms[index, :, :, :].float(),\n 'ms_hp': self.ms_hp[index, :, :, :].float(),\n 'pan_hp': self.pan_hp[index, :, :, :].float()}\n\n #####必要函数\n\n def __len__(self):\n return self.gt.shape[0]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/common/evaluate.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport math\nimport torch\nimport torch.nn.functional as F\nimport numpy as np\n\n\ndef q2n(gt, x, q_blocks_size, q_shift):\n '''\n '''\n if isinstance(gt, torch.Tensor):\n gt = gt.cpu().numpy()\n x = x.cpu().numpy()\n\n N, N1, N2, N3 = gt.shape # 255 255 8\n size2 = q_blocks_size # 32\n\n stepx = math.ceil(N1 / q_shift) # 8\n stepy = math.ceil(N2 / q_shift) # 8\n\n if stepy <= 0:\n stepy = 1\n stepx = 1\n\n est1 = (stepx - 1) * q_shift + q_blocks_size - N1 # 1\n est2 = (stepy - 1) * q_shift + q_blocks_size - N2 # 1\n # if np.sum(np.array([est1 != 0, est2 != 0])) > 0:\n # refref = np.zeros(shape=[N1+1, N2+1])\n # fusfus = refref.copy()\n\n for i in range(N3):\n a1 = gt[..., 0]\n\n ia1 = np.zeros(shape=[N, N1 + est1, N2 + est2])\n ia1[:, : N1, : N2] = a1\n ia1[:, :, N2:N2 + est2] = ia1[:, :, N2 - 1:-1:N2 - est2 + 1]\n ia1[:, N1:N1 + est1, ...] = ia1[:, N1 - 1:-1:N1 - est1 + 1, ...]\n if i == 0:\n refref = ia1[..., np.newaxis] # np.concatenate(refref, ia1, axis=3)\n else:\n refref = np.concatenate([refref, ia1[..., np.newaxis]], axis=-1)\n if i < N3:\n gt = gt[..., 1:]\n\n gt = refref\n\n for i in range(N3):\n\n a2 = x[..., 0]\n ia2 = np.zeros(shape=[N, N1 + est1, N2 + est2])\n ia2[:, : N1, : N2] = a2\n ia2[:, :, N2:N2 + est2] = ia2[:, :, N2 - 1:-1:N2 - est2 + 1]\n ia2[:, N1:N1 + est1, ...] = ia2[:, N1 - 1:-1:N1 - est1 + 1, ...]\n if i == 0:\n fusfus = ia2[..., np.newaxis] # np.concatenate(refref, ia1, axis=3)\n else:\n fusfus = np.concatenate([fusfus, ia2[..., np.newaxis]], axis=-1)\n\n if i < N3:\n x = x[..., 1:]\n x = fusfus\n\n x = np.array(x, dtype=np.uint16)\n gt = np.array(gt, dtype=np.uint16)\n\n _, N1, N2, N3 = gt.shape\n\n if math.ceil(math.log2(N3)) - math.log2(N3) != 0:\n Ndif = pow(2, math.ceil(math.log2(N3))) - N3\n dif = np.zeros(shape=[N, N1, N2, Ndif], dtype=np.uint16)\n gt = np.concatenate(gt, dif, axis=-1)\n x = np.concatenate(x, dif, axis=-1)\n\n _, _, _, N3 = gt.shape\n\n valori = np.zeros(shape=[N, stepx, stepy, N3])\n\n for j in range(stepx):\n for i in range(stepy):\n o = onions_quality(gt[:, j * q_shift:j * q_shift + q_blocks_size,\n i * q_shift: i * q_shift + size2, :],\n x[:, j * q_shift:j * q_shift + q_blocks_size,\n i * q_shift: i * q_shift + size2, :],\n q_blocks_size)\n valori[:, j, i, :] = o\n q2n_idx_map = np.sqrt(np.sum(valori ** 2, axis=-1))\n # q2n_index = np.mean(q2n_idx_map)\n return q2n_idx_map\n\n\ndef norm_blocco(x, eps=1e-8):\n a = x.mean()\n c = x.std()\n if c == 0:\n c = eps\n return (x - a) / c + 1, a, c\n\n\ndef onions_quality(dat1, dat2, size1):\n dat1 = np.float64(dat1)\n dat2 = np.float64(dat2)\n\n dat2 = np.concatenate([dat2[..., 0, np.newaxis], -dat2[..., 1:]], axis=-1)\n N, _, _, N3 = dat1.shape\n size2 = size1\n\n for i in range(N3):\n a1, s, t = norm_blocco(np.squeeze(dat1[..., i]))\n # print(s,t)\n dat1[..., i] = a1\n if s == 0:\n if i == 0:\n dat2[..., i] = dat2[..., i] - s + 1\n else:\n dat2[..., i] = -(-dat2[..., i] - s + 1)\n else:\n if i == 0:\n dat2[..., i] = ((dat2[..., i] - s) / t) + 1\n else:\n dat2[..., i] = -(((-dat2[..., i] - s) / t) + 1)\n m1 = np.zeros(shape=[N, N3])\n m2 = m1.copy()\n\n mod_q1m = 0\n mod_q2m = 0\n mod_q1 = np.zeros(shape=[size1, size2])\n mod_q2 = np.zeros(shape=[size1, size2])\n\n for i in range(N3):\n m1[..., i] = np.mean(np.squeeze(dat1[..., i]))\n m2[..., i] = np.mean(np.squeeze(dat2[..., i]))\n mod_q1m += m1[..., i] ** 2\n mod_q2m += m2[..., i] ** 2\n mod_q1 += np.squeeze(dat1[..., i]) ** 2\n mod_q2 += np.squeeze(dat2[..., i]) ** 2\n\n mod_q1m = np.sqrt(mod_q1m)\n mod_q2m = np.sqrt(mod_q2m)\n mod_q1 = np.sqrt(mod_q1)\n mod_q2 = np.sqrt(mod_q2)\n\n termine2 = mod_q1m * mod_q2m # 7.97\n termine4 = mod_q1m ** 2 + mod_q2m ** 2 #\n int1 = (size1 * size2) / (size1 * size2 - 1) * np.mean(mod_q1 ** 2)\n int2 = (size1 * size2) / (size1 * size2 - 1) * np.mean(mod_q2 ** 2)\n termine3 = int1 + int2 - (size1 * size2) / ((size1 * size2 - 1)) * (mod_q1m ** 2 + mod_q2m ** 2) # 17.8988 ** 2\n mean_bias = 2 * termine2 / termine4 # 1\n if termine3 == 0:\n q = np.zeros(shape=[N, 1, N3])\n q[:, :, N3 - 1] = mean_bias\n else:\n cbm = 2 / termine3\n # 32 32 8\n qu = onion_mult2D(dat1, dat2)\n qm = onion_mult(m1.reshape(-1), m2.reshape(-1))\n qv = np.zeros(shape=[N, N3])\n for i in range(N3):\n qv[..., i] = (size1 * size2) / ((size1 * size2) - 1) * np.mean(np.squeeze(qu[:, :, i]))\n q = qv - (size1 * size2) / ((size1 * size2) - 1) * qm\n q = q * mean_bias * cbm\n return q\n\n\ndef onion_mult2D(onion1, onion2):\n _, _, _, N3 = onion1.shape\n\n if N3 > 1:\n L = N3 // 2\n a = onion1[..., : L]\n b = onion1[..., L:]\n b = np.concatenate([b[..., 0, np.newaxis], -b[..., 1:]], axis=-1)\n c = onion2[..., : L]\n d = onion2[..., L:]\n d = np.concatenate([d[..., 0, np.newaxis], -d[..., 1:]], axis=-1)\n\n if N3 == 2:\n ris = np.concatenate([a * c - d * b, a * d + c * b], axis=-1)\n else:\n ris1 = onion_mult2D(a, c)\n ris2 = onion_mult2D(d, np.concatenate([b[..., 0, np.newaxis], -b[..., 1:]], axis=-1))\n ris3 = onion_mult2D(np.concatenate([a[..., 0, np.newaxis], -a[..., 1:]], axis=-1), d)\n ris4 = onion_mult2D(c, b)\n\n aux1 = ris1 - ris2\n aux2 = ris3 + ris4\n\n ris = np.concatenate([aux1, aux2], axis=-1)\n else:\n ris = onion1 * onion2\n return ris\n\n\ndef onion_mult(onion1, onion2):\n # _, N = onion1.shape\n N = len(onion1)\n if N > 1:\n\n L = N // 2\n a = onion1[:L]\n b = onion1[L:]\n # b[1:] = -b[1:]\n b = np.append(np.array(b[0]), -b[1:])\n c = onion2[:L]\n d = onion2[L:]\n # d[1:] = -d[1:]\n d = np.append(np.array(d[0]), -d[1:])\n\n if N == 2:\n ris = np.append(a * c - d * b, a * d + c * b)\n else:\n\n ris1 = onion_mult(a, c)\n # b[1:] = -b[1:]\n ris2 = onion_mult(d, np.append(np.array(b[0]), -b[1:]))\n # a[1:] = -a[1:]\n ris3 = onion_mult(np.append(np.array(a[0]), -a[1:]), d)\n ris4 = onion_mult(c, b)\n\n aux1 = ris1 - ris2\n aux2 = ris3 + ris4\n ris = np.append(aux1, aux2)\n else:\n ris = np.array(onion1).reshape(-1) * np.array(onion2).reshape(-1)\n return ris\n\n\ndef compute_index(img_base, img_out, ratio):\n h = img_out.shape[0]\n w = img_out.shape[1]\n chanel = img_out.shape[2]\n # 计算SAM\n sum1 = torch.sum(img_base * img_out, 2)\n sum2 = torch.sum(img_base * img_base, 2)\n sum3 = torch.sum(img_out * img_out, 2)\n t = (sum2 * sum3) ** 0.5\n numlocal = torch.gt(t, 0)\n num = torch.sum(numlocal)\n t = sum1 / t\n angle = torch.acos(t)\n sumangle = torch.where(torch.isnan(angle), torch.full_like(angle, 0), angle).sum()\n if num == 0:\n averangle = sumangle\n else:\n averangle = sumangle / num\n SAM = averangle * 180 / 3.14159256\n\n # 计算ERGAS\n summ = 0\n for i in range(chanel):\n a1 = torch.mean((img_base[:, :, i] - img_out[:, :, i]) ** 2)\n m1 = torch.mean(img_base[:, :, i])\n a2 = m1 * m1\n summ = summ + a1 / a2\n ERGAS = 100 * (1 / ratio) * ((summ / chanel) ** 0.5)\n\n return SAM, ERGAS\n\n\nimport decimal\n\ndecimal.getcontext().rounding = \"ROUND_HALF_UP\"\nn_digits = 6\n\n\ndef analysis_accu(img_base, img_out, ratio, flag_cut_bounds=True, dim_cut=21, choices=4):\n if flag_cut_bounds:\n img_base = img_base[dim_cut - 1:-dim_cut, dim_cut - 1:-dim_cut, :] #:\n img_out = img_out[dim_cut - 1:-dim_cut, dim_cut - 1:-dim_cut, :] #:\n\n # q2n\n # q2n_index = q2n(img_base, img_out, q_blocks_size=32, q_shift=32)\n\n h = img_out.shape[0]\n w = img_out.shape[1]\n chanel = img_out.shape[2]\n\n # 计算SAM\n sum1 = torch.sum(img_base * img_out, 2)\n sum2 = torch.sum(img_base * img_base, 2)\n sum3 = torch.sum(img_out * img_out, 2)\n t = (sum2 * sum3) ** 0.5\n numlocal = torch.gt(t, 0)\n num = torch.sum(numlocal)\n t = sum1 / t\n angle = torch.acos(t)\n sumangle = torch.where(torch.isnan(angle), torch.full_like(angle, 0), angle).sum()\n if num == 0:\n averangle = sumangle\n else:\n averangle = sumangle / num\n\n # you can adopt https://segmentfault.com/a/1190000018929994 to compute, too.\n # averangle = math.ceil(averangle * 1000000) / 1000000\n averangle = (averangle * 10 ** n_digits).round() / (10 ** n_digits)\n # SAM = decimal.Decimal(averangle.cpu().numpy() * 180 / 3.14159256).quantize(decimal.Decimal(\"0.00000\"))\n SAM = averangle * 180 / 3.14159256\n\n # 计算ERGAS\n summ = 0\n for i in range(chanel):\n a1 = torch.mean((img_base[:, :, i] - img_out[:, :, i]) ** 2)\n m1 = torch.mean(img_base[:, :, i])\n a2 = m1 * m1\n summ = summ + a1 / a2\n ERGAS = 100 * (1 / ratio) * ((summ / chanel) ** 0.5)\n\n # 计算PSNR\n # mse = torch.mean((img_base - img_out) ** 2, 0)\n # mse = torch.mean(mse, 0)\n # rmse = mse ** 0.5\n # temp = torch.log(1 / rmse) / math.log(10)\n PSNR = 10 * torch.log10(math.pow(1.0, 2) / torch.mean((img_out-img_base)**2, [0, 1]))\n\n # SSIM\n # img_base = img_base.permute(2, 0, 1)\n # img_out = img_out.permute(2, 0, 1)\n # img_base = img_base.unsqueeze(0)\n # img_out = img_out.unsqueeze(0)\n # SSIM = _ssim(img_base.permute(2, 0, 1).unsqueeze(0), img_out.permute(2, 0, 1).unsqueeze(0))\n\n # index = torch.zeros((5, chanel + 1))\n # index[0, 1:chanel + 1] = CC\n # index[1, 1:chanel + 1] = PSNR\n # index[2, 1:chanel + 1] = SSIM\n # index[0, 0] = torch.mean(CC)\n # index[1, 0] = torch.mean(PSNR)\n # index[2, 0] = torch.mean(SSIM)\n # index[3, 0] = SAM\n # index[4, 0] = ERGAS\n\n PSNR = torch.mean(PSNR)\n # SSIM = torch.mean(SSIM)\n # q2n_index = np.mean(q2n_index)\n\n if choices == 5:\n # 计算CC\n C1 = torch.sum(torch.sum(img_base * img_out, 0), 0) - h * w * (\n torch.mean(torch.mean(img_base, 0), 0) * torch.mean(torch.mean(img_out, 0), 0))\n C2 = torch.sum(torch.sum(img_out ** 2, 0), 0) - h * w * (torch.mean(torch.mean(img_out, 0), 0) ** 2)\n C3 = torch.sum(torch.sum(img_base ** 2, 0), 0) - h * w * (torch.mean(torch.mean(img_base, 0), 0) ** 2)\n CC = C1 / ((C2 * C3) ** 0.5)\n CC = torch.mean(CC)\n return {'SAM': SAM, 'ERGAS': ERGAS, 'PSNR': PSNR, 'CC': CC} # , q2n_index\n\n return {'SAM': SAM, 'ERGAS': ERGAS, 'PSNR': PSNR, }\n\n\ndef _ssim(img1, img2):\n img1 = img1.float()\n img2 = img2.float()\n\n channel = img1.shape[1]\n max_val = 1\n _, c, w, h = img1.size()\n window_size = min(w, h, 11)\n sigma = 1.5 * window_size / 11\n window = create_window(window_size, sigma, channel).cuda()\n mu1 = F.conv2d(img1, window, padding=window_size // 2, groups=channel)\n mu2 = F.conv2d(img2, window, padding=window_size // 2, groups=channel)\n\n mu1_sq = mu1.pow(2)\n mu2_sq = mu2.pow(2)\n mu1_mu2 = mu1 * mu2\n\n sigma1_sq = F.conv2d(img1 * img1, window, padding=window_size // 2, groups=channel) - mu1_sq\n sigma2_sq = F.conv2d(img2 * img2, window, padding=window_size // 2, groups=channel) - mu2_sq\n sigma12 = F.conv2d(img1 * img2, window, padding=window_size // 2, groups=channel) - mu1_mu2\n C1 = (0.01 * max_val) ** 2\n C2 = (0.03 * max_val) ** 2\n V1 = 2.0 * sigma12 + C2\n V2 = sigma1_sq + sigma2_sq + C2\n ssim_map = ((2 * mu1_mu2 + C1) * V1) / ((mu1_sq + mu2_sq + C1) * V2)\n t = ssim_map.shape\n return ssim_map.mean(2).mean(2)\n\n\nfrom torch.autograd import Variable\n\n\ndef gaussian(window_size, sigma):\n gauss = torch.Tensor([math.exp(-(x - window_size // 2) ** 2 / float(2 * sigma ** 2)) for x in range(window_size)])\n return gauss / gauss.sum()\n\n\ndef create_window(window_size, sigma, channel):\n _1D_window = gaussian(window_size, sigma).unsqueeze(1)\n _2D_window = _1D_window.mm(_1D_window.t()).float().unsqueeze(0).unsqueeze(0)\n window = Variable(_2D_window.expand(channel, 1, window_size, window_size).contiguous())\n return window\n\n\ndef compare_index(A):\n A_size = A.shape\n ite_n = A_size[2]\n band_n = A_size[1]\n C_better = A[:, 0, 0]\n ind = 0\n for i in range(ite_n):\n score_b = 0\n score_c = 0\n C_compare = A[:, 0, i]\n if (C_better[0] > C_compare[0]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n if (C_better[1] > C_compare[1]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n if (C_better[2] > C_compare[2]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n if (C_better[3] < C_compare[3]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n if (C_better[4] < C_compare[4]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n\n if (score_c > score_b):\n C_better = A[:, 0, i]\n ind = i\n\n C_best = A[:, :, ind]\n best_ind = ind + 1\n return C_best, best_ind\n\n\nif __name__ == \"__main__\":\n # a = np.zeros(shape=[256, 256])\n # print(a[:255, :255].shape)\n from scipy import io as sio\n ms = sio.loadmat('../../tests/I_MS.mat')['I_MS'] / 2047.0\n gt = sio.loadmat('../../tests/I_GT.mat')['I_GT'] / 2047.0\n ms = torch.from_numpy(ms).float() #* 2047.0\n gt = torch.from_numpy(gt).float() #* 2047.0\n print(analysis_accu(ms, gt, ratio=4, dim_cut=21))" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/common/psdata.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport glob\nimport torch\nfrom torch.utils.data import DataLoader\n\n\nclass PansharpeningSession():\n def __init__(self, args):\n self.dataloaders = {}\n self.samples_per_gpu = args.samples_per_gpu\n self.workers_per_gpu = args.workers_per_gpu\n # self.patch_size = args.patch_size\n self.writers = {}\n self.args = args\n\n\n def get_dataloader(self, dataset_name, distributed):\n\n if any(list(map(lambda x: x in dataset_name, ['wv2', 'wv3', 'wv4', 'qb']))):\n if \"hp\" in dataset_name:\n # high-pass filter\n from UDL.pansharpening.common.dataset_hp import Dataset_Pro\n dataset_name = dataset_name.split('_')[0] #'wv2_hp'\n dataset = Dataset_Pro('/'.join([self.args.data_dir, 'training_data', f'train_{dataset_name}_10000.h5']), img_scale=self.args.img_range)\n else:\n\n from UDL.pansharpening.common.dataset import Dataset_Pro\n dataset = Dataset_Pro('/'.join([self.args.data_dir, 'training_data', f'train_{dataset_name}_10000.h5']), img_scale=self.args.img_range)\n\n else:\n print(f\"train_{dataset_name} is not supported.\")\n raise NotImplementedError\n\n\n sampler = None\n if distributed:\n sampler = torch.utils.data.distributed.DistributedSampler(dataset)\n\n # if not dataset_name in self.dataloaders:\n dataloaders = \\\n DataLoader(dataset, batch_size=self.samples_per_gpu,\n persistent_workers=(True if self.workers_per_gpu > 0 else False), pin_memory=True,\n shuffle=(sampler is None), num_workers=self.workers_per_gpu, drop_last=True, sampler=sampler)\n\n return dataloaders, sampler\n\n def get_eval_dataloader(self, dataset_name, distributed):\n\n if 'valid' in dataset_name:\n if \"hp\" in dataset_name:\n from UDL.pansharpening.common.dataset_hp import Dataset_Pro\n dataset = Dataset_Pro(\n '/'.join([self.args.data_dir, 'validation_data', f'{dataset_name}.h5']), img_scale=self.args.img_range)\n\n else:\n from UDL.pansharpening.common.dataset import Dataset_Pro\n dataset = Dataset_Pro('/'.join([self.args.data_dir, 'validation_data', f'{dataset_name}.h5']), img_scale=self.args.img_range)\n\n elif 'TestData' in dataset_name:\n if 'hp' in dataset_name:\n satellite = dataset_name.split('_')[-2]\n else:\n satellite = dataset_name.split('_')[-1]\n\n from UDL.pansharpening.evaluation.ps_evaluate import MultiExmTest_h5\n dataset = MultiExmTest_h5('/'.join([self.args.data_dir, 'test_data', satellite.lower(), f\"{dataset_name.replace('_hp', '')}.h5\"]),\n dataset_name, img_scale=self.args.img_range)\n\n elif 'RR' in dataset_name or 'FR' in dataset_name:\n splits = dataset_name.split('_')\n if 'hp' in dataset_name:\n satellite = splits[-3]\n else:\n satellite = splits[-2]\n\n from UDL.pansharpening.evaluation.ps_evaluate import SingleDataset\n\n dataset = SingleDataset(['/'.join([self.args.data_dir, 'test_data', satellite.lower(),\n dataset_name.replace('_hp', '')+\".mat\"])], dataset_name, img_scale=self.args.img_range)\n\n\n else:\n print(f\"{dataset_name} is not supported.\")\n raise NotImplementedError\n\n sampler = None\n if distributed:\n sampler = torch.utils.data.distributed.DistributedSampler(dataset)\n\n # if not dataset_name in self.dataloaders:\n dataloaders = \\\n DataLoader(dataset, batch_size=1,\n shuffle=False, num_workers=1, drop_last=False, sampler=sampler)\n return dataloaders, sampler\n\n\n\nif __name__ == '__main__':\n # from option import args\n import argparse\n parser = argparse.ArgumentParser()\n\n args = parser.parse_args()\n args.samples_per_gpu = 8\n args.workers_per_gpu = 0\n args.data_dir = \"C:/Datasets/pansharpening_2\"\n args.dataset = 'gf2'\n\n # survey\n # wv3 9714 16-64\n # wv2 15084 16-64\n # gf2 19809 16-64\n # qb 17139 16-64\n sess = PansharpeningSession(args)\n train_loader, _ = sess.get_test_dataloader(args.dataset, False)\n print(len(train_loader))\n\n # import scipy.io as sio\n #\n # x = sio.loadmat(\"D:/Datasets/pansharpening/training_data/train1.mat\")\n # print(x.keys())\n\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/__init__.py", "content": "# from .models.builder import PANSHARPENING_MODELS, build_model\nfrom importlib import import_module\nimport os\njoin = os.path.join\ndirname = os.path.dirname\n\npkg_list = [import_module('.' + pkg.replace('.py', ''), package=\"UDL.pansharpening.configs\")\n for pkg in os.listdir(dirname(__file__)) if '.py' in pkg]\ndel pkg_list" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/hook_configs.py", "content": "# checkpoint saving\n# checkpoint_config = dict(interval=1)\ncheckpoint_config = dict(type='ModelCheckpoint', indicator='loss')\n# yapf:disable\nlog_config = dict(\n interval=100,\n hooks=[\n dict(type='TextLoggerHook'),\n # dict(type='TensorboardLoggerHook')\n ])\n# yapf:enable\n\n# dist_params = dict(backend='nccl')\nlog_level = 'INFO'\nload_from = \"D:/ProjectSets/NDA/UDL/UDL/results/pansharpening/wv3/FusionNet/Test/model_2022-04-02-12-02-55/275.pth.tar\"\nresume_from = \"D:/ProjectSets/NDA/UDL/UDL/results/pansharpening/wv3/FusionNet/Test/model_2022-04-02-12-02-55/275.pth.tar\"\nworkflow = [('train', 1)]\n\n# optimizer\noptimizer = dict(type='Adam', lr=3e-4)\noptimizer_config = dict(grad_clip=None)\nlr_config = None\n# learning policy\nrunner = dict(type='EpochBasedRunner', max_epochs=275)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/option_bdpn.py", "content": "import argparse\nfrom UDL.AutoDL import TaskDispatcher\nimport os\n\nclass parser_args(TaskDispatcher, name='BDPN'):\n def __init__(self, cfg=None):\n super(parser_args, self).__init__()\n if cfg is None:\n from UDL.Basis.option import panshaprening_cfg\n cfg = panshaprening_cfg()\n\n script_path = os.path.dirname(os.path.dirname(__file__))\n root_dir = script_path.split(cfg.task)[0]\n\n model_path = f'{root_dir}/results/{cfg.task}/wv3/BDPN/Test/.pth.tar'\n\n parser = argparse.ArgumentParser(description='PyTorch Pansharpening Training')\n # * Logger\n parser.add_argument('--out_dir', metavar='DIR', default=f'{root_dir}/results/{cfg.task}',\n help='path to save model')\n # * Training\n parser.add_argument('--lr', default=0.0001, type=float) # 1e-4 2e-4 8\n parser.add_argument('--lr_scheduler', default=True, type=bool)\n parser.add_argument('--samples_per_gpu', default=8, type=int, # 8\n metavar='N', help='mini-batch size (default: 256)')\n parser.add_argument('--print-freq', '-p', default=50, type=int,\n metavar='N', help='print frequency (default: 10)')\n parser.add_argument('--epochs', default=1000, type=int)\n parser.add_argument('--workers_per_gpu', default=0, type=int)\n parser.add_argument('--resume_from',\n default=model_path,\n type=str, metavar='PATH',\n help='path to latest checkpoint (default: none)')\n # * Model and Dataset\n parser.add_argument('--arch', '-a', metavar='ARCH', default='BDPN', type=str,\n choices=['PanNet', 'DiCNN', 'PNN', 'FusionNet'])\n parser.add_argument('--dataset', default={'train': 'wv3', 'val': 'NY1_WV3_RR'}, type=str,\n choices=[None, 'wv2', 'wv3', 'wv4', 'qb',\n 'TestData_qb', 'TestData_wv2', 'TestData_wv3', 'TestData_wv4',\n 'San_Francisco_QB_RR', 'San_Francisco_QB_FR', 'NY1_WV3_FR',\n 'NY1_WV3_RR', 'Alice_WV4_FR', 'Alice_WV4_RR', 'Rio_WV2_FR', 'Rio_WV2_RR'],\n help=\"training choices: ['wv2', 'wv3', 'wv4', 'qb'],\"\n \"validation choices: ['valid_wv2_10000','valid_wv3_10000', 'valid_wv4_10000', 'valid_qb_10000']\"\n \"test choices is ['TestData_wv2', 'TestData_wv3', 'TestData_wv4', 'TestData_qb'], and others with RR/FR\")\n parser.add_argument('--eval', default=False, type=bool,\n help=\"performing evalution for patch2entire\")\n\n args = parser.parse_args()\n args.start_epoch = args.best_epoch = 1\n args.experimental_desc = \"Test\"\n # cfg.save_fmt = 'png'\n cfg.img_range = 2047.0\n\n cfg.merge_args2cfg(args)\n print(cfg.pretty_text)\n # cfg.workflow = [('train', 50), ('val', 1)]\n # cfg.workflow = [('val', 1)] # only val workflow means perform test.\n cfg.workflow = [('train', 50)]\n self.merge_from_dict(cfg)\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/option_dicnn.py", "content": "import argparse\n# from UDL.Basis.option import panshaprening_cfg, Config, os\nfrom UDL.AutoDL import TaskDispatcher\nimport os\n\nclass parser_args(TaskDispatcher, name='DiCNN1'):\n def __init__(self, cfg=None):\n super(parser_args, self).__init__()\n if cfg is None:\n from UDL.Basis.option import panshaprening_cfg\n cfg = panshaprening_cfg()\n\n script_path = os.path.dirname(os.path.dirname(__file__))\n root_dir = script_path.split(cfg.task)[0]\n\n # model_path = f'{root_dir}/results/{cfg.task}/gf2/DiCNN1/Test/.pth.tar'\n # model_path = f'{root_dir}/results/{cfg.task}/qb/DiCNN1/Test/m.pth.tar'\n model_path = f'{root_dir}/results/{cfg.task}/wv3/DiCNN1/Test/.pth.tar'\n\n parser = argparse.ArgumentParser(description='PyTorch Pansharpening Training')\n # * Logger\n parser.add_argument('--out_dir', metavar='DIR', default=f'{root_dir}/results/{cfg.task}',\n help='path to save model')\n # * Training\n parser.add_argument('--lr', default=2e-4, type=float) # 1e-4 2e-4 8\n parser.add_argument('--lr_scheduler', default=True, type=bool)\n parser.add_argument('--samples_per_gpu', default=64, type=int, # 8\n metavar='N', help='mini-batch size (default: 256)')\n parser.add_argument('--print-freq', '-p', default=1, type=int,\n metavar='N', help='print frequency (default: 10)')\n parser.add_argument('--epochs', default=5000, type=int)\n parser.add_argument('--workers_per_gpu', default=0, type=int)\n parser.add_argument('--resume_from',\n default=model_path,\n type=str, metavar='PATH',\n help='path to latest checkpoint (default: none)')\n # * Model and Dataset\n parser.add_argument('--arch', '-a', metavar='ARCH', default='DiCNN1', type=str,\n choices=['PanNet', 'DiCNN', 'PNN', 'FusionNet'])\n parser.add_argument('--dataset', default={'train': 'wv3', 'val': 'NY1_WV3_RR'}, type=str,\n choices=[None, 'wv2', 'wv3', 'wv4', 'qb',\n 'TestData_qb', 'TestData_wv2', 'TestData_wv3', 'TestData_wv4',\n 'San_Francisco_QB_RR', 'San_Francisco_QB_FR', 'NY1_WV3_FR',\n 'NY1_WV3_RR', 'Alice_WV4_FR', 'Alice_WV4_RR', 'Rio_WV2_FR', 'Rio_WV2_RR'],\n help=\"training choices: ['wv2', 'wv3', 'wv4', 'qb'],\"\n \"validation choices: ['valid_wv2_10000','valid_wv3_10000', 'valid_wv4_10000', 'valid_qb_10000']\"\n \"test choices is ['TestData_wv2', 'TestData_wv3', 'TestData_wv4', 'TestData_qb'], and others with RR/FR\")\n parser.add_argument('--eval', default=False, type=bool,\n help=\"performing evalution for patch2entire\")\n\n args = parser.parse_args()\n args.start_epoch = args.best_epoch = 1\n args.experimental_desc = \"Test\"\n # cfg.save_fmt = 'png'\n cfg.img_range = 2047.0#1023.0\n\n cfg.merge_args2cfg(args)\n print(cfg.pretty_text)\n cfg.workflow = [('train', 1)]\n self.merge_from_dict(cfg)\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/option_drpnn.py", "content": "import argparse\n# from UDL.Basis.option import panshaprening_cfg, Config, os\nfrom UDL.AutoDL import TaskDispatcher\nimport os\n\nclass parser_args(TaskDispatcher, name='DRPNN'):\n def __init__(self, cfg=None):\n super(parser_args, self).__init__()\n if cfg is None:\n from UDL.Basis.option import panshaprening_cfg\n cfg = panshaprening_cfg()\n\n script_path = os.path.dirname(os.path.dirname(__file__))\n root_dir = script_path.split(cfg.task)[0]\n\n model_path = f'{root_dir}/results/{cfg.task}/wv3/DRPNN/Test/.pth.tar'\n\n parser = argparse.ArgumentParser(description='PyTorch Pansharpening Training')\n # * Logger\n parser.add_argument('--out_dir', metavar='DIR', default=f'{root_dir}/results/{cfg.task}',\n help='path to save model')\n # * Training\n parser.add_argument('--lr', default=2e-4, type=float) # 1e-4 2e-4 8\n parser.add_argument('--lr_scheduler', default=True, type=bool)\n parser.add_argument('--samples_per_gpu', default=32, type=int, # 8\n metavar='N', help='mini-batch size (default: 256)')\n parser.add_argument('--print-freq', '-p', default=50, type=int,\n metavar='N', help='print frequency (default: 10)')\n parser.add_argument('--epochs', default=500, type=int)\n parser.add_argument('--workers_per_gpu', default=0, type=int)\n parser.add_argument('--resume_from',\n default=model_path,\n type=str, metavar='PATH',\n help='path to latest checkpoint (default: none)')\n # * Model and Dataset\n parser.add_argument('--arch', '-a', metavar='ARCH', default='DRPNN', type=str,\n choices=['PanNet', 'DiCNN', 'PNN', 'FusionNet'])\n parser.add_argument('--dataset', default={'train': 'wv3', 'val': 'NY1_WV3_RR'}, type=str,\n choices=[None, 'wv2', 'wv3', 'wv4', 'qb',\n 'TestData_qb', 'TestData_wv2', 'TestData_wv3', 'TestData_wv4',\n 'San_Francisco_QB_RR', 'San_Francisco_QB_FR', 'NY1_WV3_FR',\n 'NY1_WV3_RR', 'Alice_WV4_FR', 'Alice_WV4_RR', 'Rio_WV2_FR', 'Rio_WV2_RR'],\n help=\"training choices: ['wv2', 'wv3', 'wv4', 'qb'],\"\n \"validation choices: ['valid_wv2_10000','valid_wv3_10000', 'valid_wv4_10000', 'valid_qb_10000']\"\n \"test choices is ['TestData_wv2', 'TestData_wv3', 'TestData_wv4', 'TestData_qb'], and others with RR/FR\")\n parser.add_argument('--eval', default=False, type=bool,\n help=\"performing evalution for patch2entire\")\n\n args = parser.parse_args()\n args.start_epoch = args.best_epoch = 1\n args.experimental_desc = \"Test\"\n # cfg.save_fmt = 'png'\n cfg.img_range = 2047.0\n cfg.seed = 1\n cfg.merge_args2cfg(args)\n print(cfg.pretty_text)\n # cfg.workflow = [('train', 50), ('val', 1)]\n cfg.workflow = [('val', 1)]\n self.merge_from_dict(cfg)\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/option_fusionnet.py", "content": "import argparse\n# from UDL.Basis.option import panshaprening_cfg, Config, os\nfrom UDL.AutoDL import TaskDispatcher\nimport os\n\nclass parser_args(TaskDispatcher, name='FusionNet'):\n def __init__(self, cfg=None):\n super(parser_args, self).__init__()\n if cfg is None:\n from UDL.Basis.option import panshaprening_cfg\n cfg = panshaprening_cfg()\n\n script_path = os.path.dirname(os.path.dirname(__file__))\n root_dir = script_path.split(cfg.task)[0]\n\n model_path = f'./.pth.tar'\n\n\n parser = argparse.ArgumentParser(description='PyTorch Pansharpening Training')\n # * Logger\n parser.add_argument('--out_dir', metavar='DIR', default=f'{root_dir}/results/{cfg.task}',\n help='path to save model')\n parser.add_argument('--mode', default=argparse.SUPPRESS, help='protective declare, please ignore it')\n\n parser.add_argument('--lr', default=3e-4, type=float)\n # parser.add_argument('--lr_scheduler', default=True, type=bool)\n parser.add_argument('--samples_per_gpu', default=32, type=int,\n metavar='N', help='mini-batch size (default: 256)')\n parser.add_argument('--print-freq', '-p', default=50, type=int,\n metavar='N', help='print frequency (default: 10)')\n parser.add_argument('--seed', default=1, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--epochs', default=400, type=int)\n parser.add_argument('--workers_per_gpu', default=0, type=int)\n parser.add_argument('--resume_from',\n default=model_path,\n type=str, metavar='PATH',\n help='path to latest checkpoint (default: none)')\n ##\n parser.add_argument('--arch', '-a', metavar='ARCH', default='FusionNet', type=str,\n choices=['PanNet', 'DiCNN', 'PNN', 'FusionNet'])\n parser.add_argument('--dataset', default={'train': 'wv3', 'val': 'NY1_WV3_RR'}, type=str,\n choices=[None, 'wv2', 'wv3', 'wv4', 'qb',\n 'TestData_qb', 'TestData_wv2', 'TestData_wv3', 'TestData_wv4',\n 'San_Francisco_QB_RR', 'San_Francisco_QB_FR', 'NY1_WV3_FR',\n 'NY1_WV3_RR', 'Alice_WV4_FR', 'Alice_WV4_RR', 'Rio_WV2_FR', 'Rio_WV2_RR'],\n help=\"training choices: ['wv2', 'wv3', 'wv4', 'qb'],\"\n \"validation choices: ['valid_wv2_10000','valid_wv3_10000', 'valid_wv4_10000', 'valid_qb_10000']\"\n \"test choices is ['TestData_wv2', 'TestData_wv3', 'TestData_wv4', 'TestData_qb'], and others with RR/FR\")\n parser.add_argument('--eval', default=False, type=bool,\n help=\"performing evalution for patch2entire\")\n\n\n args = parser.parse_args()\n args.start_epoch = args.best_epoch = 1\n args.experimental_desc = 'Test'\n cfg.merge_args2cfg(args)\n cfg.save_fmt = \"mat\"\n # cfg.workflow = [('train', 10), ('val', 1)]\n cfg.workflow = [('val', 1), ('train', 1)]\n # cfg.config = f\"{script_path}/configs/hook_configs.py\"\n cfg.use_tfb = False\n cfg.img_range = 2047.0#1023.0\n\n self.merge_from_dict(cfg)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/option_msdcnn.py", "content": "import argparse\n# from UDL.Basis.option import panshaprening_cfg, Config, os\nfrom UDL.AutoDL import TaskDispatcher\nimport os\n\nclass parser_args(TaskDispatcher, name='MSDCNN'):\n def __init__(self, cfg=None):\n super(parser_args, self).__init__()\n if cfg is None:\n from UDL.Basis.option import panshaprening_cfg\n cfg = panshaprening_cfg()\n\n script_path = os.path.dirname(os.path.dirname(__file__))\n root_dir = script_path.split(cfg.task)[0]\n\n model_path = f'{root_dir}/results/{cfg.task}/wv3/MSDCNN/Test/.pth.tar'\n\n parser = argparse.ArgumentParser(description='PyTorch Pansharpening Training')\n # * Logger\n parser.add_argument('--out_dir', metavar='DIR', default=f'{root_dir}/results/{cfg.task}',\n help='path to save model')\n # * Training\n parser.add_argument('--lr', default=0.000001, type=float) # 1e-4 2e-4 8\n parser.add_argument('--lr_scheduler', default=True, type=bool)\n parser.add_argument('--samples_per_gpu', default=64, type=int, # 8\n metavar='N', help='mini-batch size (default: 256)')\n parser.add_argument('--print-freq', '-p', default=50, type=int,\n metavar='N', help='print frequency (default: 10)')\n parser.add_argument('--epochs', default=500, type=int)\n parser.add_argument('--workers_per_gpu', default=0, type=int)\n parser.add_argument('--resume_from',\n default=model_path,\n type=str, metavar='PATH',\n help='path to latest checkpoint (default: none)')\n # * Model and Dataset\n parser.add_argument('--arch', '-a', metavar='ARCH', default='MSDCNN', type=str,\n choices=['PanNet', 'DiCNN', 'PNN', 'FusionNet'])\n parser.add_argument('--dataset', default={'train': 'wv3', 'val': 'NY1_WV3_RR'}, type=str,\n choices=[None, 'wv2', 'wv3', 'wv4', 'qb',\n 'TestData_qb', 'TestData_wv2', 'TestData_wv3', 'TestData_wv4',\n 'San_Francisco_QB_RR', 'San_Francisco_QB_FR', 'NY1_WV3_FR',\n 'NY1_WV3_RR', 'Alice_WV4_FR', 'Alice_WV4_RR', 'Rio_WV2_FR', 'Rio_WV2_RR'],\n help=\"training choices: ['wv2', 'wv3', 'wv4', 'qb'],\"\n \"validation choices: ['valid_wv2_10000','valid_wv3_10000', 'valid_wv4_10000', 'valid_qb_10000']\"\n \"test choices is ['TestData_wv2', 'TestData_wv3', 'TestData_wv4', 'TestData_qb'], and others with RR/FR\")\n parser.add_argument('--eval', default=False, type=bool,\n help=\"performing evalution for patch2entire\")\n\n args = parser.parse_args()\n args.start_epoch = args.best_epoch = 1\n args.experimental_desc = \"Test\"\n # cfg.save_fmt = 'png'\n cfg.img_range = 2047.0\n\n cfg.merge_args2cfg(args)\n print(cfg.pretty_text)\n # cfg.workflow = [('train', 50), ('val', 1)]\n # cfg.workflow = [('val', 1)]\n cfg.workflow = [('train', 50)]\n self.merge_from_dict(cfg)\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/option_pannet.py", "content": "import argparse\n# from UDL.Basis.option import panshaprening_cfg, Config, os\nfrom UDL.AutoDL import TaskDispatcher\nimport os\n\nclass parser_args(TaskDispatcher, name='PanNet'):\n def __init__(self, cfg=None):\n super(parser_args, self).__init__()\n if cfg is None:\n from UDL.Basis.option import panshaprening_cfg\n cfg = panshaprening_cfg()\n\n script_path = os.path.dirname(os.path.dirname(__file__))\n root_dir = script_path.split(cfg.task)[0]\n\n # model_path = f'{root_dir}/results/{cfg.task}/qb_hp/PanNet/Test/.pth.tar'\n model_path = f'{root_dir}/results/{cfg.task}/wv3_hp/PanNet/Test/.pth.tar'\n # model_path = f''\n parser = argparse.ArgumentParser(description='PyTorch Pansharpening Training')\n # * Logger\n parser.add_argument('--out_dir', metavar='DIR', default=f'{root_dir}/results/{cfg.task}',\n help='path to save model')\n # * Training\n parser.add_argument('--lr', default=1e-3, type=float) # 1e-4 2e-4 8\n parser.add_argument('--lr_scheduler', default=True, type=bool)\n parser.add_argument('--samples_per_gpu', default=32, type=int, # 8\n metavar='N', help='mini-batch size (default: 256)')\n parser.add_argument('--print-freq', '-p', default=50, type=int,\n metavar='N', help='print frequency (default: 10)')\n parser.add_argument('--seed', default=1, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--epochs', default=450, type=int)\n parser.add_argument('--workers_per_gpu', default=0, type=int)\n parser.add_argument('--resume_from',\n default=model_path,\n type=str, metavar='PATH',\n help='path to latest checkpoint (default: none)')\n\n # * Model and Dataset\n parser.add_argument('--arch', '-a', metavar='ARCH', default='PanNet', type=str,\n choices=['PanNet', 'DiCNN', 'PNN', 'FusionNet'])\n parser.add_argument('--dataset', default={'train': 'wv3_hp', 'val': 'NY1_WV3_RR_hp'}, type=str,\n choices=[None, 'wv2_hp', 'wv3_hp', 'wv4_hp', 'qb_hp',\n 'TestData_qb_hp', 'TestData_wv2_hp', 'TestData_wv3_hp', 'TestData_wv4_hp',\n 'San_Francisco_QB_RR_hp', 'San_Francisco_QB_FR_hp', 'NY1_WV3_FR_hp',\n 'NY1_WV3_RR_hp', 'Alice_WV4_FR', 'Alice_WV4_RR_hp', 'Rio_WV2_FR_hp', 'Rio_WV2_RR_hp'],\n help=\"training choices: ['wv2', 'wv3', 'wv4', 'qb'],\"\n \"validation choices: ['valid_wv2_10000','valid_wv3_10000', 'valid_wv4_10000', 'valid_qb_10000']\"\n \"test choices is ['TestData_wv2', 'TestData_wv3', 'TestData_wv4', 'TestData_qb'], and others with RR/FR\")\n parser.add_argument('--eval', default=False, type=bool,\n help=\"performing evalution for patch2entire\")\n\n args = parser.parse_args()\n args.start_epoch = args.best_epoch = 1\n args.experimental_desc = \"Test\"\n\n\n cfg.merge_args2cfg(args)\n cfg.img_range = 2047.0\n cfg.reg = True\n cfg.workflow = [('train', 1)]\n print(cfg.pretty_text)\n cfg.workflow = [('train', 1)]\n self.merge_from_dict(cfg)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/configs/option_pnn.py", "content": "import argparse\n# from UDL.Basis.option import panshaprening_cfg, Config, os\nfrom UDL.AutoDL import TaskDispatcher\nimport os\n\nclass parser_args(TaskDispatcher, name='PNN'):\n def __init__(self, cfg=None):\n super(parser_args, self).__init__()\n if cfg is None:\n from UDL.Basis.option import panshaprening_cfg\n cfg = panshaprening_cfg()\n\n script_path = os.path.dirname(os.path.dirname(__file__))\n root_dir = script_path.split(cfg.task)[0].replace('\\\\', '/')\n\n model_path = f'.pth.tar'\n\n parser = argparse.ArgumentParser(description='PyTorch Pansharpening Training')\n # * Logger\n parser.add_argument('--out_dir', metavar='DIR', default=f'{root_dir}/results/{cfg.task}',\n help='path to save model')\n # * Training\n parser.add_argument('--lr', default=1e-3, type=float) # 1e-4 2e-4 8\n parser.add_argument('--lr_scheduler', default=True, type=bool)\n parser.add_argument('--samples_per_gpu', default=64, type=int, # 8\n metavar='N', help='mini-batch size (default: 256)')\n parser.add_argument('--print-freq', '-p', default=500, type=int,\n metavar='N', help='print frequency (default: 10)')\n parser.add_argument('--seed', default=1, type=int,\n help='seed for initializing training. ')\n parser.add_argument('--epochs', default=12000, type=int)\n parser.add_argument('--workers_per_gpu', default=0, type=int)\n parser.add_argument('--resume_from',\n default=model_path,\n type=str, metavar='PATH',\n help='path to latest checkpoint (default: none)')\n # * Model and Dataset\n parser.add_argument('--arch', '-a', metavar='ARCH', default='PNN', type=str,\n choices=['PanNet', 'DiCNN', 'PNN', 'FusionNet'])\n parser.add_argument('--dataset', default={'train': 'wv3', 'val': 'NY1_WV3_RR'}, type=str,\n choices=[None, 'wv2', 'wv3', 'wv4', 'qb',\n 'TestData_qb', 'TestData_wv2', 'TestData_wv3', 'TestData_wv4',\n 'San_Francisco_QB_RR', 'San_Francisco_QB_FR', 'NY1_WV3_FR',\n 'NY1_WV3_RR', 'Alice_WV4_FR', 'Alice_WV4_RR', 'Rio_WV2_FR', 'Rio_WV2_RR'],\n help=\"training choices: ['wv2', 'wv3', 'wv4', 'qb'],\"\n \"validation choices: ['valid_wv2_10000','valid_wv3_10000', 'valid_wv4_10000', 'valid_qb_10000']\"\n \"test choices is ['TestData_wv2', 'TestData_wv3', 'TestData_wv4', 'TestData_qb'], and others with RR/FR\")\n parser.add_argument('--eval', default=False, type=bool,\n help=\"performing evalution for patch2entire\")\n args = parser.parse_args()\n args.start_epoch = args.best_epoch = 1\n args.experimental_desc = 'Test'\n cfg.merge_args2cfg(args)\n cfg.workflow = [('train', 1)]\n cfg.img_range = 2047.0\n print(cfg.pretty_text)\n self.merge_from_dict(cfg)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/evaluation/ps_evaluate.py", "content": "import os\nimport datetime\nimport imageio\nimport numpy as np\nimport cv2\nimport h5py\nimport torch\nimport torch.nn.functional as F\nfrom scipy import io as sio\nfrom torch.utils.data import DataLoader, Dataset\nfrom UDL.Basis.auxiliary import MetricLogger, SmoothedValue, set_random_seed\nfrom UDL.Basis.dist_utils import init_dist, dist_train_v1, get_dist_info, reduce_mean\nfrom UDL.pansharpening.common.evaluate import analysis_accu\nfrom UDL.Basis.postprocess import showimage8\nimport matplotlib.pyplot as plt\n# from UDL.Basis.zoom_image_region import show_region_images\nfrom logging import info as log_string\n\n# dmd\ndef load_gt_compared(file_path_gt, file_path_compared):\n data1 = sio.loadmat(file_path_gt) # HxWxC\n data2 = sio.loadmat(file_path_compared)\n try:\n gt = torch.from_numpy(data1['gt'] / 2047.0)\n except KeyError:\n print(data1.keys())\n compared_data = torch.from_numpy(data2['output_dmdnet_newdata6'] * 2047.0)\n return gt, compared_data\n\n\ndef get_edge(data): # get high-frequency\n rs = np.zeros_like(data)\n if rs.ndim == 4:\n for b in range(data.shape[0]):\n for i in range(data.shape[1]):\n rs[b, i, :, :] = data[b, i, :, :] - cv2.boxFilter(data[b, i, :, :], -1, (5, 5))\n elif len(rs.shape) == 3:\n for i in range(data.shape[2]):\n rs[:, :, i] = data[:, :, i] - cv2.boxFilter(data[:, :, i], -1, (5, 5))\n else:\n rs = data - cv2.boxFilter(data, -1, (5, 5))\n\n return rs\n\n\ndef load_dataset_singlemat_hp(file_path, scale):\n data = sio.loadmat(file_path) # HxWxC\n\n # tensor type:\n lms = torch.from_numpy(data['I_MS'] / scale).permute(2, 0, 1) # CxHxW = 8x256x256\n ms_hp = torch.from_numpy(get_edge(data['I_MS_LR'] / scale)).permute(2, 0, 1).unsqueeze(dim=0) # CxHxW= 8x64x64\n pan_hp = torch.from_numpy(get_edge(data['I_PAN'] / scale)) # HxW = 256x256\n gt = torch.from_numpy(data['I_GT'] / scale)\n\n return lms.squeeze().float(), ms_hp.squeeze().float(), pan_hp.float(), gt.float()\n\n\ndef load_dataset_singlemat(file_path, scale):\n data = sio.loadmat(file_path) # HxWxC\n print(\"load_dataset_singlemat: \", data.keys())\n # tensor type:\n lms = torch.from_numpy(data['I_MS'] / scale).permute(2, 0, 1) # CxHxW = 8x256x256\n ms = torch.from_numpy(data['I_MS_LR'] / scale).permute(2, 0, 1).unsqueeze(dim=0) # CxHxW= 8x64x64\n\n pan = torch.from_numpy(data['I_PAN'] / scale) # HxW = 256x256\n if data.get('I_GT', None) is None:\n gt = torch.from_numpy(data['I_MS'] / scale)\n else:\n gt = torch.from_numpy(data['I_GT'] / scale)\n\n return lms.squeeze().float(), ms.squeeze().float(), pan.float(), gt.float()\n\n\ndef load_dataset_H5_hp(file_path, scale, use_cuda=True):\n data = h5py.File(file_path) # NxHxWxC\n shape_list = []\n # for k in data.keys():\n # shape_list.append((k, data[k].shape))\n # print(shape_list)\n\n # tensor type: NxCxHxW:\n\n lms = torch.from_numpy(data['lms'][...] / scale).float()#.permute(0, 3, 1, 2)\n ms_hp = torch.from_numpy(get_edge(data['ms'][...] / scale)).float()#.permute(0, 3, 1, 2) # NxCxHxW:\n mms_hp = torch.nn.functional.interpolate(ms_hp, size=(ms_hp.size(2) * 2, ms_hp.size(3) * 2),\n mode=\"bilinear\", align_corners=True)\n pan = np.squeeze(data['pan'][...])\n pan = pan[:, np.newaxis, :, :] # NxCxHxW (C=1)\n pan_hp = torch.from_numpy(get_edge(pan / scale)).float()#.permute(0, 3, 1, 2) # Nx1xHxW:\n if data.get('gt', None) is None:\n gt = torch.from_numpy(data['lms'][...]).float()\n else:\n gt = torch.from_numpy(data['gt'][...]).float()\n\n return {'lms': lms,\n 'mms:': mms_hp,\n 'ms': ms_hp,\n 'pan': pan_hp,\n 'gt': gt.permute([0, 2, 3, 1])\n }\n\ndef load_dataset_H5(file_path, scale, use_cuda=True):\n data = h5py.File(file_path) # CxHxW\n print(data.keys())\n # tensor type:\n if use_cuda:\n lms = torch.from_numpy(data['lms'][...] / scale).cuda().float() # CxHxW = 8x64x64\n\n ms = torch.from_numpy(data['ms'][...] / scale).cuda().float() # CxHxW= 8x64x64\n pan = torch.from_numpy(data['pan'][...] / scale).cuda().float() # HxW = 256x256\n\n gt = torch.from_numpy(data['gt'][...]).cuda().float()\n\n else:\n lms = torch.from_numpy(data['lms'][...] / scale).float() # CxHxW = 8x64x64\n\n ms = torch.from_numpy(data['ms'][...] / scale).float() # CxHxW= 8x64x64\n pan = torch.from_numpy(data['pan'][...] / scale).float() # HxW = 256x256\n if data.get('gt', None) is None:\n gt = torch.from_numpy(data['lms'][...]).float()\n else:\n gt = torch.from_numpy(data['gt'][...]).float()\n\n return {'lms': lms,\n 'ms': ms,\n 'pan': pan,\n 'gt': gt.permute([0, 2, 3, 1])\n }\n\n\nclass MultiExmTest_h5(Dataset):\n\n def __init__(self, file_path, dataset_name, img_scale, suffix='.h5'):\n super(MultiExmTest_h5, self).__init__()\n\n # self.scale = 2047.0\n # if 'gf' in dataset_name:\n # self.scale = 1023.0\n self.img_scale = img_scale\n print(f\"loading MultiExmTest_h5: {file_path} with {img_scale}\")\n # 一次性载入到内存\n if 'hp' not in dataset_name:\n data = load_dataset_H5(file_path, img_scale, False)\n\n elif 'hp' in dataset_name:\n file_path = file_path.replace('_hp', '')\n data = load_dataset_H5_hp(file_path, img_scale, False)\n\n else:\n print(f\"{dataset_name} is not supported in evaluation\")\n raise NotImplementedError\n if suffix == '.mat':\n self.lms = data['lms'].permute(0, 3, 1, 2) # CxHxW = 8x256x256\n self.ms = data['ms'].permute(0, 3, 1, 2) # CxHxW= 8x64x64\n self.pan = data['pan'].unsqueeze(1)\n self.gt = data['gt'].permute(0, 3, 1, 2)\n else:\n self.lms = data['lms']\n self.ms = data['ms']\n self.pan = data['pan']\n self.gt = data['gt']\n\n print(f\"lms: {self.lms.shape}, ms: {self.ms.shape}, pan: {self.pan.shape}, gt: {self.gt.shape}\")\n\n def __getitem__(self, item):\n return {'lms': self.lms[item, ...],\n 'ms': self.ms[item, ...],\n 'pan': self.pan[item, ...],\n 'gt': self.gt[item, ...]\n }\n\n def __len__(self):\n return self.gt.shape[0]\n\n\nclass SingleDataset(Dataset):\n\n\n\n def __init__(self, file_lists, dataset_name, img_scale, dataset=None):\n\n self.img_scale = img_scale\n self.file_lists = file_lists\n print(f\"loading SingleDataset: {file_lists} with {img_scale}\")\n self.file_nums = len(file_lists)\n self.dataset = {}\n self.dataset_name = dataset_name\n\n if 'hp' not in dataset_name:\n self.dataset = load_dataset_singlemat\n elif 'hp' in dataset_name:\n self.dataset = load_dataset_singlemat_hp\n else:\n print(f\"{dataset_name} is not supported in evaluation\")\n raise NotImplementedError\n\n def __getitem__(self, idx):\n file_path = self.file_lists[idx % self.file_nums]\n test_lms, test_ms, test_pan, gt = self.dataset(file_path, self.img_scale)\n\n if 'hp' not in self.dataset_name:\n return {'gt': gt,\n 'lms': test_lms,\n 'ms': test_ms,\n 'pan': test_pan.unsqueeze(dim=0),\n 'filename': file_path}\n else:\n return {'gt': gt,\n 'lms': test_lms,\n 'ms': test_ms,\n 'pan': test_pan.unsqueeze(dim=0),\n 'filename': file_path}\n\n def __len__(self):\n return self.file_nums\n\n\ndef save_results(idx, save_model_output, filename, save_fmt, output):\n if filename is None:\n save_name = os.path.join(f\"{save_model_output}\",\n \"output_mulExm_{}.mat\".format(idx))\n sio.savemat(save_name, {'sr': output.cpu().detach().numpy()})\n else:\n filename = os.path.basename(filename).split('.')[0]\n if save_fmt != 'mat':\n output = showimage8(output)\n filename = '/'.join([save_model_output, filename + \".png\"])\n # plt.imsave(filename, output, dpi=300)\n # show_region_images(output, xywh=[50, 100, 50, 50], #sub_width=\"20%\", sub_height=\"20%\",\n # sub_ax_anchor=(0, 0, 1, 1))\n # mpl_save_fig(filename)\n else:\n filename = '/'.join([save_model_output, \"output_\" + filename + \".mat\"])\n sio.savemat(filename, {'sr': output.cpu().detach().numpy()})\n\n\ndef mpl_save_fig(filename):\n plt.savefig(f\"{filename}\", format='svg', dpi=300, pad_inches=0, bbox_inches='tight')\n\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/data_qb.py", "content": "import torch.utils.data as data\nimport torch\nimport h5py\nimport numpy as np\n\n\nclass Dataset_Ft(data.Dataset):\n def __init__(self, file_path):\n super(Dataset_Ft, self).__init__()\n\n\nclass Dataset_Pro(data.Dataset):\n def __init__(self, file_path):\n super(Dataset_Pro, self).__init__()\n data = h5py.File(file_path) # NxCxHxW = 0x1x2x3\n\n # tensor type:\n gt1 = data[\"gt\"][...]\n gt1 = np.array(gt1, dtype=np.float32) / 2047\n self.gt = torch.from_numpy(gt1) # NxCxHxW:\n\n print(self.gt.size())\n\n lms1 = data[\"lms\"][...]\n lms1 = np.array(lms1, dtype=np.float32) / 2047\n self.lms = torch.from_numpy(lms1)\n\n ms1 = data[\"ms\"][...] # NxCxHxW\n ms1 = np.array(ms1.transpose(0, 2, 3, 1), dtype=np.float32) / 2047 # NxHxWxC\n self.ms = torch.from_numpy(ms1).permute(0, 3, 1, 2) # NxCxHxW:\n\n pan1 = data['pan'][...] # Nx1xHxW\n pan1 = np.array(pan1.transpose(0, 2, 3, 1), dtype=np.float32) / 2047 # NxHxWx1\n pan1 = np.squeeze(pan1, axis=3) # NxHxW\n pan_tmp = np.expand_dims(pan1, axis=3) # NxHxWx1\n self.pan = torch.from_numpy(pan_tmp).permute(0, 3, 1, 2) # Nx1xHxW:\n\n #####必要函数\n def __getitem__(self, index):\n return self.gt[index, :, :, :].float(), \\\n self.lms[index, :, :, :].float(), \\\n self.ms[index, :, :, :].float(), \\\n self.pan[index, :, :, :].float()\n\n #####必要函数\n def __len__(self):\n return self.gt.shape[0]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/data_single_read.py", "content": "import torch.nn.modules as nn\nimport torch\nimport cv2\nimport numpy as np\nimport h5py\nimport scipy.io as sio\nimport os\n\n\n\ndef load_set(file_path, blk):\n data = sio.loadmat(file_path) # HxWxC\n\n # tensor type:\n lms = np.array(data['lms'] / 2047.0, dtype=np.float32)\n pan_hp = np.expand_dims(np.array(data['pan'] / 2047.0,dtype=np.float32), axis=-1)\n lms = np.concatenate([lms, pan_hp], axis=-1)\n lms = np.pad(lms, ((blk, blk), (blk, blk), (0, 0)), mode='edge')\n lms = torch.from_numpy(lms).cuda().permute(2, 0, 1) # CxHxW = 8x256x256\n pan_hp = torch.from_numpy(pan_hp).cuda().permute(2, 0, 1) # HxW = 256x256\n ms_hp = torch.from_numpy(data['ms'] / 2047.0).cuda().permute(2, 0, 1) # CxHxW= 8x64x64\n gt = torch.from_numpy(data['gt'] / 2047.0).cuda()\n\n return lms, ms_hp, pan_hp, gt\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/data_wv2.py", "content": "import torch.utils.data as data\nimport torch\nimport h5py\nimport numpy as np\n\n\nclass Dataset_Ft(data.Dataset):\n def __init__(self, file_path):\n super(Dataset_Ft, self).__init__()\n\n\nclass Dataset_Pro(data.Dataset):\n def __init__(self, file_path):\n super(Dataset_Pro, self).__init__()\n data = h5py.File(file_path) # NxCxHxW = 0x1x2x3\n\n # tensor type:\n gt1 = data[\"gt\"][...]\n gt1 = np.array(gt1, dtype=np.float32) / 2047\n self.gt = torch.from_numpy(gt1) # NxCxHxW:\n\n print(self.gt.size())\n\n lms1 = data[\"lms\"][...]\n lms1 = np.array(lms1, dtype=np.float32) / 2047\n self.lms = torch.from_numpy(lms1)\n\n ms1 = data[\"ms\"][...] # NxCxHxW\n ms1 = np.array(ms1.transpose(0, 2, 3, 1), dtype=np.float32) / 2047 # NxHxWxC\n self.ms = torch.from_numpy(ms1).permute(0, 3, 1, 2) # NxCxHxW:\n\n pan1 = data['pan'][...] # Nx1xHxW\n pan1 = np.array(pan1.transpose(0, 2, 3, 1), dtype=np.float32) / 2047 # NxHxWx1\n pan1 = np.squeeze(pan1, axis=3) # NxHxW\n pan_tmp = np.expand_dims(pan1, axis=3) # NxHxWx1\n self.pan = torch.from_numpy(pan_tmp).permute(0, 3, 1, 2) # Nx1xHxW:\n\n #####必要函数\n def __getitem__(self, index):\n return self.gt[index, :, :, :].float(), \\\n self.lms[index, :, :, :].float(), \\\n self.ms[index, :, :, :].float(), \\\n self.pan[index, :, :, :].float()\n\n #####必要函数\n def __len__(self):\n return self.gt.shape[0]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/data_wv3.py", "content": "import torch.utils.data as data\nimport torch\nimport h5py\nimport numpy as np\n\n\nclass Dataset_Ft(data.Dataset):\n def __init__(self, file_path):\n super(Dataset_Ft, self).__init__()\n\n\nclass Dataset_Pro(data.Dataset):\n def __init__(self, file_path):\n super(Dataset_Pro, self).__init__()\n data = h5py.File(file_path) # NxCxHxW = 0x1x2x3\n # tensor type:\n gt1 = data[\"gt\"][...]\n gt1 = np.array(gt1, dtype=np.float32) / 2047\n self.gt = torch.from_numpy(gt1) # NxCxHxW:\n\n print(self.gt.size())\n\n lms1 = data[\"lms\"][...]\n lms1 = np.array(lms1, dtype=np.float32) / 2047\n self.lms = torch.from_numpy(lms1)\n\n ms1 = data[\"ms\"][...] # NxCxHxW\n ms1 = np.array(ms1.transpose(0, 2, 3, 1), dtype=np.float32) / 2047 # NxHxWxC\n self.ms = torch.from_numpy(ms1).permute(0, 3, 1, 2) # NxCxHxW:\n\n pan1 = data['pan'][...] # Nx1xHxW\n pan1 = np.array(pan1.transpose(0, 2, 3, 1), dtype=np.float32) / 2047 # NxHxWx1\n pan1 = np.squeeze(pan1, axis=3) # NxHxW\n pan_tmp = np.expand_dims(pan1, axis=3) # NxHxWx1\n self.pan = torch.from_numpy(pan_tmp).permute(0, 3, 1, 2) # Nx1xHxW:\n\n #####必要函数\n def __getitem__(self, index):\n return self.gt[index, :, :, :].float(), \\\n self.lms[index, :, :, :].float(), \\\n self.ms[index, :, :, :].float(), \\\n self.pan[index, :, :, :].float()\n\n #####必要函数\n def __len__(self):\n return self.gt.shape[0]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/data_wv4.py", "content": "import torch.utils.data as data\nimport torch\nimport h5py\nimport numpy as np\n\n\nclass Dataset_Ft(data.Dataset):\n def __init__(self, file_path):\n super(Dataset_Ft, self).__init__()\n\n\nclass Dataset_Pro(data.Dataset):\n def __init__(self, file_path):\n super(Dataset_Pro, self).__init__()\n data = h5py.File(file_path) # NxCxHxW = 0x1x2x3\n\n # tensor type:\n gt1 = data[\"gt\"][...]\n gt1 = np.array(gt1, dtype=np.float32) / 2047\n self.gt = torch.from_numpy(gt1) # NxCxHxW:\n\n print(self.gt.size())\n\n lms1 = data[\"lms\"][...]\n lms1 = np.array(lms1, dtype=np.float32) / 2047\n self.lms = torch.from_numpy(lms1)\n\n ms1 = data[\"ms\"][...] # NxCxHxW\n ms1 = np.array(ms1.transpose(0, 2, 3, 1), dtype=np.float32) / 2047 # NxHxWxC\n self.ms = torch.from_numpy(ms1).permute(0, 3, 1, 2) # NxCxHxW:\n\n pan1 = data['pan'][...] # Nx1xHxW\n pan1 = np.array(pan1.transpose(0, 2, 3, 1), dtype=np.float32) / 2047 # NxHxWx1\n pan1 = np.squeeze(pan1, axis=3) # NxHxW\n pan_tmp = np.expand_dims(pan1, axis=3) # NxHxWx1\n self.pan = torch.from_numpy(pan_tmp).permute(0, 3, 1, 2) # Nx1xHxW:\n\n #####必要函数\n def __getitem__(self, index):\n return self.gt[index, :, :, :].float(), \\\n self.lms[index, :, :, :].float(), \\\n self.ms[index, :, :, :].float(), \\\n self.pan[index, :, :, :].float()\n\n #####必要函数\n def __len__(self):\n return self.gt.shape[0]\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/evaluate.py", "content": "import math\nimport torch\nimport torch.nn.functional as F\nimport numpy as np\n\n\n# 由于dat及其方差等数值舍入存在误差，最终结果有0.001左右的误差\ndef q2n(gt, x, q_blocks_size, q_shift):\n '''\n '''\n if isinstance(gt , torch.Tensor):\n gt = gt.cpu().numpy()\n x = x.cpu().numpy()\n\n N, N1, N2, N3 = gt.shape # 255 255 8\n size2 = q_blocks_size # 32\n\n stepx = math.ceil(N1 / q_shift) # 8\n stepy = math.ceil(N2 / q_shift) # 8\n\n if stepy <= 0:\n stepy = 1\n stepx = 1\n\n est1 = (stepx - 1) * q_shift + q_blocks_size - N1 # 1\n est2 = (stepy - 1) * q_shift + q_blocks_size - N2 # 1\n # if np.sum(np.array([est1 != 0, est2 != 0])) > 0:\n # refref = np.zeros(shape=[N1+1, N2+1])\n # fusfus = refref.copy()\n\n for i in range(N3):\n a1 = gt[..., 0]\n\n ia1 = np.zeros(shape=[N, N1 + est1, N2 + est2])\n ia1[:, : N1, : N2] = a1\n ia1[:, :, N2:N2 + est2] = ia1[:, :, N2 - 1:-1:N2 - est2 + 1]\n ia1[:, N1:N1 + est1, ...] = ia1[:, N1 - 1:-1:N1 - est1 + 1, ...]\n if i == 0:\n refref = ia1[..., np.newaxis] # np.concatenate(refref, ia1, axis=3)\n else:\n refref = np.concatenate([refref, ia1[..., np.newaxis]], axis=-1)\n if i < N3:\n gt = gt[..., 1:]\n\n gt = refref\n\n for i in range(N3):\n\n a2 = x[..., 0]\n ia2 = np.zeros(shape=[N, N1 + est1, N2 + est2])\n ia2[:, : N1, : N2] = a2\n ia2[:, :, N2:N2 + est2] = ia2[:, :, N2 - 1:-1:N2 - est2 + 1]\n ia2[:, N1:N1 + est1, ...] = ia2[:, N1 - 1:-1:N1 - est1 + 1, ...]\n if i == 0:\n fusfus = ia2[..., np.newaxis] # np.concatenate(refref, ia1, axis=3)\n else:\n fusfus = np.concatenate([fusfus, ia2[..., np.newaxis]], axis=-1)\n\n if i < N3:\n x = x[..., 1:]\n x = fusfus\n\n x = np.array(x, dtype=np.uint16)\n gt = np.array(gt, dtype=np.uint16)\n\n _, N1, N2, N3 = gt.shape\n\n if math.ceil(math.log2(N3)) - math.log2(N3) != 0:\n Ndif = pow(2, math.ceil(math.log2(N3))) - N3\n dif = np.zeros(shape=[N, N1, N2, Ndif], dtype=np.uint16)\n gt = np.concatenate(gt, dif, axis=-1)\n x = np.concatenate(x, dif, axis=-1)\n\n _, _, _, N3 = gt.shape\n\n valori = np.zeros(shape=[N, stepx, stepy, N3])\n\n for j in range(stepx):\n for i in range(stepy):\n o = onions_quality(gt[:, j * q_shift:j * q_shift + q_blocks_size,\n i * q_shift: i * q_shift + size2, :],\n x[:, j * q_shift:j * q_shift + q_blocks_size,\n i * q_shift: i * q_shift + size2, :],\n q_blocks_size)\n # 0.971379489438014\t0.00553590637316723\t0.00305237797490489\t-0.0188289323262161\t-0.00420556598390016\t-0.0173947468044076\t-0.0202144450367593\t0.0102693855205061\n valori[:, j, i, :] = o\n q2n_idx_map = np.sqrt(np.sum(valori ** 2, axis=-1))\n # q2n_index = np.mean(q2n_idx_map)\n return q2n_idx_map\n\n\ndef norm_blocco(x, eps=1e-8):\n a = x.mean()\n c = x.std()\n if c == 0:\n c = eps\n return (x - a) / c + 1, a, c\n\n\ndef onions_quality(dat1, dat2, size1):\n dat1 = np.float64(dat1)\n dat2 = np.float64(dat2)\n\n dat2 = np.concatenate([dat2[..., 0, np.newaxis], -dat2[..., 1:]], axis=-1)\n N, _, _, N3 = dat1.shape\n size2 = size1\n\n # Block norm\n for i in range(N3):\n a1, s, t = norm_blocco(np.squeeze(dat1[..., i]))\n # print(s,t)\n dat1[..., i] = a1\n if s == 0:\n if i == 0:\n dat2[..., i] = dat2[..., i] - s + 1\n else:\n dat2[..., i] = -(-dat2[..., i] - s + 1)\n else:\n if i == 0:\n dat2[..., i] = ((dat2[..., i] - s) / t) + 1\n else:\n dat2[..., i] = -(((-dat2[..., i] - s) / t) + 1)\n m1 = np.zeros(shape=[N, N3])\n m2 = m1.copy()\n\n mod_q1m = 0\n mod_q2m = 0\n mod_q1 = np.zeros(shape=[size1, size2])\n mod_q2 = np.zeros(shape=[size1, size2])\n\n for i in range(N3):\n m1[..., i] = np.mean(np.squeeze(dat1[..., i]))\n m2[..., i] = np.mean(np.squeeze(dat2[..., i]))\n mod_q1m += m1[..., i] ** 2\n mod_q2m += m2[..., i] ** 2\n mod_q1 += np.squeeze(dat1[..., i]) ** 2\n mod_q2 += np.squeeze(dat2[..., i]) ** 2\n\n mod_q1m = np.sqrt(mod_q1m)\n mod_q2m = np.sqrt(mod_q2m)\n mod_q1 = np.sqrt(mod_q1)\n mod_q2 = np.sqrt(mod_q2)\n\n termine2 = mod_q1m * mod_q2m # 7.97\n termine4 = mod_q1m ** 2 + mod_q2m ** 2 #\n int1 = (size1 * size2) / (size1 * size2 - 1) * np.mean(mod_q1 ** 2)\n int2 = (size1 * size2) / (size1 * size2 - 1) * np.mean(mod_q2 ** 2)\n termine3 = int1 + int2 - (size1 * size2) / ((size1 * size2 - 1)) * (mod_q1m ** 2 + mod_q2m ** 2) # 17.8988 ** 2\n mean_bias = 2 * termine2 / termine4 # 1\n if termine3 == 0:\n q = np.zeros(shape=[N, 1, N3])\n q[:, :, N3 - 1] = mean_bias\n else:\n cbm = 2 / termine3\n # 32 32 8\n qu = onion_mult2D(dat1, dat2)\n qm = onion_mult(m1.reshape(-1), m2.reshape(-1))\n qv = np.zeros(shape=[N, N3])\n for i in range(N3):\n qv[..., i] = (size1 * size2) / ((size1 * size2) - 1) * np.mean(np.squeeze(qu[:, :, i]))\n q = qv - (size1 * size2) / ((size1 * size2) - 1) * qm\n q = q * mean_bias * cbm\n return q\n\n\ndef onion_mult2D(onion1, onion2):\n _, _, _, N3 = onion1.shape\n\n if N3 > 1:\n L = N3 // 2\n a = onion1[..., : L]\n b = onion1[..., L:]\n b = np.concatenate([b[..., 0, np.newaxis], -b[..., 1:]], axis=-1)\n c = onion2[..., : L]\n d = onion2[..., L:]\n d = np.concatenate([d[..., 0, np.newaxis], -d[..., 1:]], axis=-1)\n\n if N3 == 2:\n ris = np.concatenate([a * c - d * b, a * d + c * b], axis=-1)\n else:\n ris1 = onion_mult2D(a, c)\n ris2 = onion_mult2D(d, np.concatenate([b[..., 0, np.newaxis], -b[..., 1:]], axis=-1))\n ris3 = onion_mult2D(np.concatenate([a[..., 0, np.newaxis], -a[..., 1:]], axis=-1), d)\n ris4 = onion_mult2D(c, b)\n\n aux1 = ris1 - ris2\n aux2 = ris3 + ris4\n\n ris = np.concatenate([aux1, aux2], axis=-1)\n else:\n ris = onion1 * onion2\n return ris\n\n\ndef onion_mult(onion1, onion2):\n # _, N = onion1.shape\n N = len(onion1)\n if N > 1:\n\n L = N // 2\n a = onion1[:L]\n b = onion1[L:]\n # b[1:] = -b[1:]\n b = np.append(np.array(b[0]), -b[1:])\n c = onion2[:L]\n d = onion2[L:]\n # d[1:] = -d[1:]\n d = np.append(np.array(d[0]), -d[1:])\n\n if N == 2:\n ris = np.append(a * c - d * b, a * d + c * b)\n else:\n\n ris1 = onion_mult(a, c)\n # b[1:] = -b[1:]\n ris2 = onion_mult(d, np.append(np.array(b[0]), -b[1:]))\n # a[1:] = -a[1:]\n ris3 = onion_mult(np.append(np.array(a[0]), -a[1:]), d)\n ris4 = onion_mult(c, b)\n\n aux1 = ris1 - ris2\n aux2 = ris3 + ris4\n ris = np.append(aux1, aux2)\n else:\n ris = np.array(onion1).reshape(-1) * np.array(onion2).reshape(-1)\n return ris\n\n\ndef compute_index(img_base, img_out, ratio):\n h = img_out.shape[0]\n w = img_out.shape[1]\n chanel = img_out.shape[2]\n # 计算SAM\n sum1 = torch.sum(img_base * img_out, 2)\n sum2 = torch.sum(img_base * img_base, 2)\n sum3 = torch.sum(img_out * img_out, 2)\n t = (sum2 * sum3) ** 0.5\n numlocal = torch.gt(t, 0)\n num = torch.sum(numlocal)\n t = sum1 / t\n angle = torch.acos(t)\n sumangle = torch.where(torch.isnan(angle), torch.full_like(angle, 0), angle).sum()\n if num == 0:\n averangle = sumangle\n else:\n averangle = sumangle / num\n SAM = averangle * 180 / 3.14159256\n\n # 计算ERGAS\n summ = 0\n for i in range(chanel):\n a1 = torch.mean((img_base[:, :, i] - img_out[:, :, i]) ** 2)\n m1 = torch.mean(img_base[:, :, i])\n a2 = m1 * m1\n summ = summ + a1 / a2\n ERGAS = 100 * (1 / ratio) * ((summ / chanel) ** 0.5)\n\n return SAM, ERGAS\n\n\nimport decimal\n\ndecimal.getcontext().rounding = \"ROUND_HALF_UP\"\nn_digits = 6\n\n\n# panHrnet: 2.6565 |1.4651 | 0.98364 | 0.98024 | 0.98089-Q8\ndef analysis_accu(img_base, img_out, ratio, flag_cut_bounds=True, dim_cut=1):\n if flag_cut_bounds:\n img_base = img_base[dim_cut - 1:-dim_cut, dim_cut - 1:-dim_cut, :]#:\n img_out = img_out[dim_cut - 1:-dim_cut, dim_cut - 1:-dim_cut, :]#:\n\n # q2n\n # q2n_index = q2n(img_base, img_out, q_blocks_size=32, q_shift=32)\n\n h = img_out.shape[0]\n w = img_out.shape[1]\n chanel = img_out.shape[2]\n\n # 计算CC\n C1 = torch.sum(torch.sum(img_base * img_out, 0), 0) - h * w * (\n torch.mean(torch.mean(img_base, 0), 0) * torch.mean(torch.mean(img_out, 0), 0))\n C2 = torch.sum(torch.sum(img_out ** 2, 0), 0) - h * w * (torch.mean(torch.mean(img_out, 0), 0) ** 2)\n C3 = torch.sum(torch.sum(img_base ** 2, 0), 0) - h * w * (torch.mean(torch.mean(img_base, 0), 0) ** 2)\n CC = C1 / ((C2 * C3) ** 0.5)\n\n # 计算SAM\n sum1 = torch.sum(img_base * img_out, 2)\n sum2 = torch.sum(img_base * img_base, 2)\n sum3 = torch.sum(img_out * img_out, 2)\n t = (sum2 * sum3) ** 0.5\n numlocal = torch.gt(t, 0)\n num = torch.sum(numlocal)\n t = sum1 / t\n angle = torch.acos(t)\n sumangle = torch.where(torch.isnan(angle), torch.full_like(angle, 0), angle).sum()\n if num == 0:\n averangle = sumangle\n else:\n averangle = sumangle / num\n\n # 或者采用https://segmentfault.com/a/1190000018929994修改精度\n # averangle = math.ceil(averangle * 1000000) / 1000000\n averangle = (averangle * 10 ** n_digits).round() / (10 ** n_digits)\n # SAM = decimal.Decimal(averangle.cpu().numpy() * 180 / 3.14159256).quantize(decimal.Decimal(\"0.00000\"))\n SAM = averangle * 180 / 3.14159256\n\n # 计算ERGAS\n summ = 0\n for i in range(chanel):\n a1 = torch.mean((img_base[:, :, i] - img_out[:, :, i]) ** 2)\n m1 = torch.mean(img_base[:, :, i])\n a2 = m1 * m1\n summ = summ + a1 / a2\n ERGAS = 100 * (1 / ratio) * ((summ / chanel) ** 0.5)\n\n # 计算PSNR\n mse = torch.mean((img_base - img_out) ** 2, 0)\n mse = torch.mean(mse, 0)\n rmse = mse ** 0.5\n temp = torch.log(1 / rmse) / math.log(10)\n PSNR = 20 * temp\n\n # 计算SSIM\n img_base = img_base.permute(2, 0, 1)\n img_out = img_out.permute(2, 0, 1)\n img_base = img_base.unsqueeze(0)\n img_out = img_out.unsqueeze(0)\n SSIM = _ssim(img_base, img_out)\n\n index = torch.zeros((5, chanel + 1))\n index[0, 1:chanel + 1] = CC\n index[1, 1:chanel + 1] = PSNR\n index[2, 1:chanel + 1] = SSIM\n # index[0, 0] = torch.mean(CC)\n # index[1, 0] = torch.mean(PSNR)\n # index[2, 0] = torch.mean(SSIM)\n # index[3, 0] = SAM\n # index[4, 0] = ERGAS\n CC = torch.mean(CC)\n PSNR = torch.mean(PSNR)\n SSIM = torch.mean(SSIM)\n # q2n_index = np.mean(q2n_index)\n\n return CC, PSNR, SSIM, SAM, ERGAS#, q2n_index\n\n\ndef _ssim(img1, img2):\n img1 = img1.float()\n img2 = img2.float()\n\n channel = img1.shape[1]\n max_val = 1\n _, c, w, h = img1.size()\n window_size = min(w, h, 11)\n sigma = 1.5 * window_size / 11\n window = create_window(window_size, sigma, channel).cuda()\n mu1 = F.conv2d(img1, window, padding=window_size // 2, groups=channel)\n mu2 = F.conv2d(img2, window, padding=window_size // 2, groups=channel)\n\n mu1_sq = mu1.pow(2)\n mu2_sq = mu2.pow(2)\n mu1_mu2 = mu1 * mu2\n\n sigma1_sq = F.conv2d(img1 * img1, window, padding=window_size // 2, groups=channel) - mu1_sq\n sigma2_sq = F.conv2d(img2 * img2, window, padding=window_size // 2, groups=channel) - mu2_sq\n sigma12 = F.conv2d(img1 * img2, window, padding=window_size // 2, groups=channel) - mu1_mu2\n C1 = (0.01 * max_val) ** 2\n C2 = (0.03 * max_val) ** 2\n V1 = 2.0 * sigma12 + C2\n V2 = sigma1_sq + sigma2_sq + C2\n ssim_map = ((2 * mu1_mu2 + C1) * V1) / ((mu1_sq + mu2_sq + C1) * V2)\n t = ssim_map.shape\n return ssim_map.mean(2).mean(2)\n\n\nfrom torch.autograd import Variable\n\n\ndef gaussian(window_size, sigma):\n gauss = torch.Tensor([math.exp(-(x - window_size // 2) ** 2 / float(2 * sigma ** 2)) for x in range(window_size)])\n return gauss / gauss.sum()\n\n\ndef create_window(window_size, sigma, channel):\n _1D_window = gaussian(window_size, sigma).unsqueeze(1)\n _2D_window = _1D_window.mm(_1D_window.t()).float().unsqueeze(0).unsqueeze(0)\n window = Variable(_2D_window.expand(channel, 1, window_size, window_size).contiguous())\n return window\n\n\ndef compare_index(A):\n A_size = A.shape\n ite_n = A_size[2]\n band_n = A_size[1]\n C_better = A[:, 0, 0]\n ind = 0\n for i in range(ite_n):\n score_b = 0\n score_c = 0\n C_compare = A[:, 0, i]\n if (C_better[0] > C_compare[0]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n if (C_better[1] > C_compare[1]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n if (C_better[2] > C_compare[2]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n if (C_better[3] < C_compare[3]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n if (C_better[4] < C_compare[4]):\n score_b = score_b + 1\n else:\n score_c = score_c + 1\n\n if (score_c > score_b):\n C_better = A[:, 0, i]\n ind = i\n\n C_best = A[:, :, ind]\n best_ind = ind + 1\n return C_best, best_ind\n\n\nif __name__ == \"__main__\":\n a = np.zeros(shape=[256, 256])\n print(a[:255, :255].shape)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/main_pre_train_trainData_qb.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.tensorboard import SummaryWriter\nfrom torch.autograd import Variable\nfrom torch.utils.data import DataLoader\nfrom data_qb import Dataset_Pro\nfrom model_qb import APNN, summaries, loss_with_l2_regularization, weights_init\nfrom logger import create_logger, log_string\nimport numpy as np\n\n\nimport argparse\n\nparser = argparse.ArgumentParser(description='PyTorch ImageNet Training')\n\nparser.add_argument('--out_dir', metavar='DIR', default='../results',\n help='path to save model')\nparser.add_argument('--log_dir', metavar='DIR', default='logs',\n help='path to save log')\nparser.add_argument('--tfb_dir', metavar='DIR', default=None,\n help='useless in this script.')\nparser.add_argument('--arch', '-a', metavar='ARCH', default='APNN')\n\nargs = parser.parse_args()\nargs.experimental_desc = \"APNN\"\nargs.dataset = \"QB\"\n\nout_dir, model_save_dir, tfb_dir = create_logger(args, args.experimental_desc)\n\n###################################################################\n# ------------------- Pre-Define Part----------------------\n###################################################################\n# ================== Pre-Define =================== #\nSEED = 10\ntorch.manual_seed(SEED)\ntorch.cuda.manual_seed(SEED)\ntorch.cuda.manual_seed_all(SEED)\n# cudnn.benchmark = True ###自动寻找最优算法\ncudnn.deterministic = True\n\n# ============= 2) HYPER PARAMS(Pre-Defined) ==========#\n\nsensor = 'QB'\nnr_bands = 4 #selected by user or taken from data?\nlr = 0.0001*17*17*nr_bands\nepochs = 15000\nckpt = 50\nbatch_size = 128\nmodel_path = \"Weights/qb/.pth\"\n\n# ============= 3) Load Model + Loss + Optimizer + Learn_rate_update ==========#\nmodel = APNN().cuda()\nmodel.apply(weights_init)\nif os.path.isfile(model_path):\n model.load_state_dict(torch.load(model_path)) ## Load the pretrained Encoder\n log_string('APNN is Successfully Loaded from %s' % (model_path))\n\n# summaries(model, grad=True) ## Summary the Network\ncriterion = nn.L1Loss(reduction='mean').cuda()\nregularization = loss_with_l2_regularization().cuda()\ntarget_layerParam = list(map(id, model.conv3.parameters()))\nbase_layerParam = filter(lambda p: id(p) not in target_layerParam, model.parameters())\n\ntraining_parameters = [{'params': model.conv3.parameters(), 'lr': lr/10},\n {'params': base_layerParam}]\n\noptimizer = optim.SGD(training_parameters, lr=lr, momentum=0.9)\n\nlog_string(\"inspect optimizer setting: {}\\n\".format(optimizer.state_dict()))\nprint(\"target id: {}\".format(target_layerParam))\n\n# (input_size - kernel_size + 1) // 2 = 2* pad = 2 * blk = net_scope\nnet_scope = 0\nfor name, layer in model.named_parameters():\n if 'conv' in name and 'bias' not in name:\n net_scope += layer.shape[-1]-1\n\nnet_scope = np.sum(net_scope) + 1\nblk = net_scope//2 #8\n\nsave_best_file = './results/PNN/PNN_model.pth.tar'\n\nPNN_model = {'sensor': sensor,\n 'lr': lr,\n 'epochs': epochs,\n 'model_sampling_period': ckpt,\n 'net_scope': net_scope,\n 'batch_size': batch_size}\n\n\nwriter = SummaryWriter('./train_logs') ## Tensorboard_show: case 2\n\ndef save_checkpoint(model, epoch): # save model function\n model_out_path = 'Weights' + '/' + \"{}.pth\".format(epoch)\n torch.save(model.state_dict(), model_out_path)\n\n###################################################################\n# ------------------- Main Train (Run second)----------------------------------\n###################################################################\n\ndef train(training_data_loader, validate_data_loader, start_epoch=0):\n log_string('Start training...')\n vmin = 10000\n for epoch in range(start_epoch, epochs, 1):\n\n epoch += 1\n epoch_train_mae, epoch_train_mse, epoch_val_mae, epoch_val_mse = [], [], [], []\n\n # ============Epoch Train=============== #\n model.train()\n\n for iteration, batch in enumerate(training_data_loader, 1):\n\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n\n lms = torch.cat([lms, pan], dim=1)\n optimizer.zero_grad() # fixed\n\n sr = model(lms) # call model\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n\n loss = criterion(sr, gt) # compute loss\n new_loss = regularization(loss, model, flag=False)\n epoch_train_mae.append(loss.item()) # save all losses into a vector for one epoch\n\n new_loss.backward() # fixed\n optimizer.step() # fixed\n\n with torch.no_grad():\n loss = nn.MSELoss()(sr, gt)\n loss.requires_grad = False\n epoch_train_mse.append(loss.item())\n\n t_loss1 = np.nanmean(np.array(epoch_train_mae)) # compute the mean value of all losses, as one epoch loss\n t_loss2 = np.nanmean(np.array(epoch_train_mse))\n\n writer.add_scalar('mae_loss/t_mae', t_loss1, epoch) # write to tensorboard to check\n writer.add_scalar('mae_loss/t_mse', t_loss2, epoch)\n log_string('Epoch: {}/{} training L1-loss: {:.7f}, L2-loss: {:.7f}'.format(epochs, epoch, t_loss1, t_loss2)) # print loss for each epoch\n # if epoch % ckpt == 0: # if each ckpt epochs, then start to save model\n # save_checkpoint(model, epoch)\n\n # ============Epoch Validate=============== #\n model.eval()\n with torch.no_grad():\n for iteration, batch in enumerate(validate_data_loader, 1):\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n lms = torch.cat([lms, pan], dim=1)\n sr = model(lms)\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n \n loss1 = criterion(sr, gt)\n loss2 = nn.MSELoss()(sr, gt)\n epoch_val_mae.append(loss1.item())\n epoch_val_mse.append(loss2.item())\n\n v_loss1 = np.nanmean(np.array(epoch_val_mae))\n v_loss2 = np.nanmean(np.array(epoch_val_mse))\n writer.add_scalar('val/v_mae', v_loss1, epoch)\n writer.add_scalar('val/v_mse', v_loss2, epoch)\n log_string('Epoch: {}/{} validate L1-loss: {:.7f}, L2-loss: {:7f}'.format(epochs, epoch, v_loss1, v_loss2)) # print loss for each epoch\n\n\n ### during save and simple best save ###\n # vmin = 10000\n if (epoch + 1) % ckpt == 0:\n # print(\"saving PNN_model_{}.pth.tar\".format(epoch))\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss=v_loss1,\n train_params=PNN_model),\n '{}/PNN_model_{}.pth.tar'.format(model_save_dir, epoch + 1))\n\n if v_loss1 < vmin:\n if os.path.isfile(save_best_file):\n os.remove(save_best_file)\n # print(\"saving PNN_model.pth.tar\")\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss1=v_loss1,\n train_params=PNN_model),\n '{}/best_PNN_model_{}.pth.tar'.format(model_save_dir, epoch))\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss1=v_loss1,\n train_params=PNN_model),\n './pretrained_models/'+sensor+'_PNNplus_model.pth.tar')\n vmin = v_loss1\n\n\n writer.close() # close tensorboard\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == \"__main__\":\n train_set = Dataset_Pro('../training_data/train_qb_10000.h5') # creat data for training\n training_data_loader = DataLoader(dataset=train_set, num_workers=0, batch_size=batch_size, shuffle=True,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n validate_set = Dataset_Pro('../training_data/valid_qb_10000.h5') # creat data for validation\n validate_data_loader = DataLoader(dataset=validate_set, num_workers=0, batch_size=batch_size, shuffle=False,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n train(training_data_loader, validate_data_loader) # call train function (call: Line 53)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/main_pre_train_trainData_wv2.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.tensorboard import SummaryWriter\nfrom torch.autograd import Variable\nfrom torch.utils.data import DataLoader\nfrom data_wv2 import Dataset_Pro\nfrom model_wv2 import APNN, summaries, loss_with_l2_regularization, weights_init\nfrom logger import create_logger, log_string\nimport numpy as np\n\nimport argparse\n\nparser = argparse.ArgumentParser(description='PyTorch ImageNet Training')\n\nparser.add_argument('--out_dir', metavar='DIR', default='../results',\n help='path to save model')\nparser.add_argument('--log_dir', metavar='DIR', default='logs',\n help='path to save log')\nparser.add_argument('--tfb_dir', metavar='DIR', default=None,\n help='useless in this script.')\nparser.add_argument('--arch', '-a', metavar='ARCH', default='APNN')\n\nargs = parser.parse_args()\nargs.experimental_desc = \"APNN\"\nargs.dataset = \"WV2\"\n\nout_dir, model_save_dir, tfb_dir = create_logger(args, args.experimental_desc)\nprint(model_save_dir)\n# import shutil\n# from torch.utils.tensorboard import SummaryWriter\n\n###################################################################\n# ------------------- Pre-Define Part----------------------\n###################################################################\n# ================== Pre-Define =================== #\nSEED = 10\ntorch.manual_seed(SEED)\ntorch.cuda.manual_seed(SEED)\ntorch.cuda.manual_seed_all(SEED)\n# cudnn.benchmark = True ###自动寻找最优算法\ncudnn.deterministic = True\n\n# ============= 2) HYPER PARAMS(Pre-Defined) ==========#\n\"\"\" CHANGES: \n 1. row 48: in APNN the L1 loss is averaged only on the minibatch size,\n for the learning rate in case the loss is averaged on minibatches, patches size,\n and bands is lr=0.0001*17*17*nr_bands\n a. nr_bands takes into account the number of bands\n \n 2. row 46: should be indicated by user here, or taken from data \n \n 3. row 46: depends on the \n \n 4. the dataset is already normalized, so we do not need anymore and \n \n 5. row 49: in APNN epochs=10000\n \n 6. row 71: in APNN weight_decay=0\n \n 7. rows 182-195: in pretrained_models the best PNN model is saved\n\"\"\"\n\nsensor = 'WV2'\nnr_bands = 8 # selected by user or taken from data?\nlr = 0.1#0.0001 * 17 * 17 * nr_bands#=0.2302\nepochs = 15000\nckpt = 50\nbatch_size = 128\nmodel_path = \"../results/WV2/best_PNN_model_4765.pth.tar\"\nv_min = 10000\n'''\n- Epoch: 15000/4765 training L1-loss: 0.0145116, L2-loss: 0.0005868\n- Epoch: 15000/4765 validate L1-loss: 0.0145294, L2-loss: 0.000590\n- Epoch: 15000/4766 training L1-loss: 0.0145040, L2-loss: 0.0005864\n- Epoch: 15000/4766 validate L1-loss: 0.0145357, L2-loss: 0.000590\n- Epoch: 15000/4767 training L1-loss: 0.0145072, L2-loss: 0.0005866\n- Epoch: 15000/4767 validate L1-loss: 0.0145413, L2-loss: 0.000591\n'''\n# TODO L2 norm to do where\n# ============= 3) Load Model + Loss + Optimizer + Learn_rate_update ==========#\nmodel = APNN().cuda()\nmodel.apply(weights_init)\nif os.path.isfile(model_path):\n log_string(\"loading\")\n checkpoint = torch.load(model_path)\n model.load_state_dict(checkpoint[\"model_state\"]) ## Load the pretrained Encoder\n log_string('APNN is Successfully Loaded from %s' % (model_path))\n v_min = checkpoint[\"loss1\"]\n# summaries(model, grad=True) ## Summary the Network\ncriterion = nn.L1Loss(reduction='mean').cuda()\nregularization = loss_with_l2_regularization().cuda()\n# 用model里有的实例id去指定model中的其他参数,而不要遍历model.parameters()\ntarget_layerParam = list(map(id, model.conv3.parameters()))\nbase_layerParam = filter(lambda p: id(p) not in target_layerParam, model.parameters())\n\ntraining_parameters = [{'params': model.conv3.parameters(), 'lr': lr / 10},\n {'params': base_layerParam}]\n\noptimizer = optim.SGD(training_parameters, lr=lr, momentum=0.9)\n\nlog_string(\"inspect optimizer setting: {}\\n\".format(optimizer.state_dict()))\nlog_string(\"target id: {}\".format(target_layerParam))\n\n# 模型卷积层宽卷积零填充范围\n# (input_size - kernel_size + 1) // 2 = 2* pad = 2 * blk = net_scope\nnet_scope = 0\nfor name, layer in model.named_parameters():\n if 'conv' in name and 'bias' not in name:\n net_scope += layer.shape[-1] - 1\n\nnet_scope = np.sum(net_scope) + 1\nblk = net_scope // 2 # 8\n\nsave_best_file = './results/WV2/PNN_model.pth.tar'\n\nPNN_model = {'sensor': sensor,\n 'lr': lr,\n 'epochs': epochs,\n 'model_sampling_period': ckpt,\n 'net_scope': net_scope,\n 'batch_size': batch_size}\n\nwriter = SummaryWriter('../train_logs') ## Tensorboard_show: case 2\n\n\ndef save_checkpoint(model, epoch): # save model function\n model_out_path = 'Weights' + '/' + \"{}.pth\".format(epoch)\n torch.save(model.state_dict(), model_out_path)\n\n\n###################################################################\n# ------------------- Main Train (Run second)----------------------------------\n###################################################################\n\ndef train(training_data_loader, validate_data_loader, start_epoch=0, v_min=10000):\n log_string('Start training...')\n\n for epoch in range(start_epoch, epochs, 1):\n\n epoch += 1\n epoch_train_mae, epoch_train_mse, epoch_val_mae, epoch_val_mse = [], [], [], []\n\n # ============Epoch Train=============== #\n model.train()\n\n for iteration, batch in enumerate(training_data_loader, 1):\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n\n lms = torch.cat([lms, pan], dim=1)\n optimizer.zero_grad() # fixed\n\n sr = model(lms) # call model\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n\n loss = criterion(sr, gt) # compute loss\n new_loss = regularization(loss, model, flag=False)\n epoch_train_mae.append(loss.item()) # save all losses into a vector for one epoch\n\n new_loss.backward() # fixed\n optimizer.step() # fixed\n\n with torch.no_grad():\n loss = nn.MSELoss()(sr, gt)\n loss.requires_grad = False\n epoch_train_mse.append(loss.item())\n\n t_loss1 = np.nanmean(np.array(epoch_train_mae)) # compute the mean value of all losses, as one epoch loss\n t_loss2 = np.nanmean(np.array(epoch_train_mse))\n\n writer.add_scalar('mae_loss/t_mae', t_loss1, epoch) # write to tensorboard to check\n writer.add_scalar('mae_loss/t_mse', t_loss2, epoch)\n log_string('Epoch: {}/{} training L1-loss: {:.7f}, L2-loss: {:.7f}'.format(epochs, epoch, t_loss1,\n t_loss2)) # print loss for each epoch\n # if epoch % ckpt == 0: # if each ckpt epochs, then start to save model\n # save_checkpoint(model, epoch)\n\n # ============Epoch Validate=============== #\n model.eval()\n with torch.no_grad():\n for iteration, batch in enumerate(validate_data_loader, 1):\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n lms = torch.cat([lms, pan], dim=1)\n sr = model(lms)\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n\n loss1 = criterion(sr, gt)\n loss2 = nn.MSELoss()(sr, gt)\n epoch_val_mae.append(loss1.item())\n epoch_val_mse.append(loss2.item())\n\n v_loss1 = np.nanmean(np.array(epoch_val_mae))\n v_loss2 = np.nanmean(np.array(epoch_val_mse))\n writer.add_scalar('val/v_mae', v_loss1, epoch)\n writer.add_scalar('val/v_mse', v_loss2, epoch)\n log_string('Epoch: {}/{} validate L1-loss: {:.7f}, L2-loss: {:7f}'.format(epochs, epoch, v_loss1,\n v_loss2)) # print loss for each epoch\n\n ### during save and simple best save ###\n # vmin = 10000\n if (epoch + 1) % ckpt == 0:\n # print(\"saving PNN_model_{}.pth.tar\".format(epoch))\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss=v_loss1,\n train_params=PNN_model),\n '{}/PNN_model_{}.pth.tar'.format(model_save_dir, epoch + 1))\n\n if v_loss1 < v_min:\n if os.path.isfile(save_best_file):\n os.remove(save_best_file)\n # print(\"saving PNN_model.pth.tar\")\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss1=v_loss1,\n train_params=PNN_model),\n '{}/best_PNN_model_{}.pth.tar'.format(model_save_dir,epoch))\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss1=v_loss1,\n train_params=PNN_model),\n '../pretrained_models/'+sensor+'_PNNplus_model.pth.tar')\n v_min = v_loss1\n\n writer.close() # close tensorboard\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == \"__main__\":\n\n\n train_set = Dataset_Pro('../training_data/train_wv2_10000.h5') # creat data for training\n training_data_loader = DataLoader(dataset=train_set, num_workers=0, batch_size=batch_size, shuffle=True,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n validate_set = Dataset_Pro('../training_data/valid_wv2_10000.h5') # creat data for validation\n validate_data_loader = DataLoader(dataset=validate_set, num_workers=0, batch_size=batch_size, shuffle=False,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n train(training_data_loader, validate_data_loader, 6700, v_min=v_min) # call train function (call: Line 53)\n'''\n- Epoch: 15000/1942 validate L1-loss: 0.0152082, L2-loss: 0.000656\n- Epoch: 15000/1943 training L1-loss: 0.0151794, L2-loss: 0.0006520\n- Epoch: 15000/1943 validate L1-loss: 0.0152087, L2-loss: 0.000656\n- Epoch: 15000/1944 training L1-loss: 0.0151762, L2-loss: 0.0006518\n- Epoch: 15000/1944 validate L1-loss: 0.0152076, L2-loss: 0.000656\n- Epoch: 15000/1945 training L1-loss: 0.0151769, L2-loss: 0.0006519\n- Epoch: 15000/1945 validate L1-loss: 0.0152089, L2-loss: 0.000656\n- Epoch: 15000/1946 training L1-loss: 0.0151799, L2-loss: 0.0006520\n- Epoch: 15000/1946 validate L1-loss: 0.0152077, L2-loss: 0.000656\n'''" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/main_pre_train_trainData_wv3.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.tensorboard import SummaryWriter\nfrom torch.autograd import Variable\nfrom torch.utils.data import DataLoader\nfrom data_wv3 import Dataset_Pro\nfrom model_wv3 import APNN, loss_with_l2_regularization, weights_init\nimport numpy as np\nimport shutil\nfrom torch.utils.tensorboard import SummaryWriter\nfrom torchstat import stat\n###################################################################\n# ------------------- Pre-Define Part----------------------\n###################################################################\n# ================== Pre-Define =================== #\nSEED = 10\ntorch.manual_seed(SEED)\ntorch.cuda.manual_seed(SEED)\ntorch.cuda.manual_seed_all(SEED)\n# cudnn.benchmark = True ###自动寻找最优算法\ncudnn.deterministic = True\n\n# ============= 2) HYPER PARAMS(Pre-Defined) ==========#\n\n\nsensor = 'WV3'\nnr_bands = 8 # selected by user or taken from data?\nlr = 0.0001 * 17 * 17 * nr_bands\nepochs = 10000\nckpt = 50\nbatch_size = 128\n# 0.010094023841832365\nmodel_path = \"results/PNN/.pth.tar\"\n# ============= 3) Load Model + Loss + Optimizer + Learn_rate_update ==========#\nmodel = APNN().cuda()\nmodel.apply(weights_init)\n\nstat(model, input_size=[(9, 64, 64)])\n\nif os.path.isfile(model_path):\n checkpoint = torch.load(model_path)\n model.load_state_dict(checkpoint[\"model_state\"]) ## Load the pretrained Encoder\n print('APNN is Successfully Loaded from %s' % (model_path))\n if \"loss1\" in dict(checkpoint).keys():\n print(\"loss: {}\".format(checkpoint[\"loss1\"]))\n\n\n\ncriterion = nn.L1Loss(reduction='mean').cuda()\nregularization = loss_with_l2_regularization().cuda()\ntarget_layerParam = list(map(id, model.conv3.parameters()))\nbase_layerParam = filter(lambda p: id(p) not in target_layerParam, model.parameters())\n\ntraining_parameters = [{'params': model.conv3.parameters(), 'lr': lr / 10},\n {'params': base_layerParam}]\n\noptimizer = optim.SGD(training_parameters, lr=lr, momentum=0.9, weight_decay=0)\n\nprint(\"inspect optimizer setting:\\n\", optimizer.state_dict())\nprint(\"target id:\", target_layerParam)\n\n# (input_size - kernel_size + 1) // 2 = 2* pad = 2 * blk = net_scope\nnet_scope = 0\nfor name, layer in model.named_parameters():\n if 'conv' in name and 'bias' not in name:\n net_scope += layer.shape[-1] - 1\n\nnet_scope = np.sum(net_scope) + 1\nblk = net_scope // 2 # 8\n\nsave_best_file = './results/PNN/PNN_model.pth.tar'\n\nPNN_model = {'sensor': sensor,\n 'lr': lr,\n 'epochs': epochs,\n 'model_sampling_period': ckpt,\n 'net_scope': net_scope,\n 'batch_size': batch_size}\n\nwriter = SummaryWriter('./train_logs') ## Tensorboard_show: case 2\n\n\ndef save_checkpoint(model, epoch): # save model function\n model_out_path = 'Weights' + '/wv3/' + \"{}.pth\".format(epoch)\n torch.save(model.state_dict(), model_out_path)\n\n\n###################################################################\n# ------------------- Main Train (Run second)----------------------------------\n###################################################################\n\ndef train(training_data_loader, validate_data_loader, start_epoch=0):\n print('Start training...')\n print(model.conv1.weight.data[0, 0, 0, 0])\n vmin = 10000\n for epoch in range(start_epoch, epochs, 1):\n flag = (epoch == (epochs - 1)) or epoch == 0\n epoch += 1\n epoch_train_mae, epoch_train_mse, epoch_val_mae, epoch_val_mse = [], [], [], []\n\n # ============Epoch Train=============== #\n model.train()\n\n for iteration, batch in enumerate(training_data_loader, 1):\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n\n lms = torch.cat([lms, pan], dim=1)\n optimizer.zero_grad() # fixed\n\n sr = model(lms) # call model\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n\n loss = criterion(sr, gt) # compute loss\n new_loss = regularization(loss, model, flag=flag or (iteration == 0))\n epoch_train_mae.append(loss.item()) # save all losses into a vector for one epoch\n\n new_loss.backward() # fixed\n optimizer.step() # fixed\n\n with torch.no_grad():\n loss = nn.MSELoss()(sr, gt)\n loss.requires_grad = False\n epoch_train_mse.append(loss.item())\n\n t_loss1 = np.nanmean(np.array(epoch_train_mae)) # compute the mean value of all losses, as one epoch loss\n t_loss2 = np.nanmean(np.array(epoch_train_mse))\n\n writer.add_scalar('mae_loss/t_mae', t_loss1, epoch) # write to tensorboard to check\n writer.add_scalar('mae_loss/t_mse', t_loss2, epoch)\n print('Epoch: {}/{} training L1-loss: {:.7f}, L2-loss: {:.7f}'.format(epochs, epoch, t_loss1,\n t_loss2)) # print loss for each epoch\n # if epoch % ckpt == 0: # if each ckpt epochs, then start to save model\n # save_checkpoint(model, epoch)\n\n # ============Epoch Validate=============== #\n model.eval()\n with torch.no_grad():\n for iteration, batch in enumerate(validate_data_loader, 1):\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n lms = torch.cat([lms, pan], dim=1)\n sr = model(lms)\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n\n loss1 = criterion(sr, gt)\n loss2 = nn.MSELoss()(sr, gt)\n epoch_val_mae.append(loss1.item())\n epoch_val_mse.append(loss2.item())\n\n v_loss1 = np.nanmean(np.array(epoch_val_mae))\n v_loss2 = np.nanmean(np.array(epoch_val_mse))\n writer.add_scalar('val/v_mae', v_loss1, epoch)\n writer.add_scalar('val/v_mse', v_loss2, epoch)\n print('Epoch: {}/{} validate L1-loss: {:.7f}, L2-loss: {:7f}'.format(epochs, epoch, v_loss1,\n v_loss2)) # print loss for each epoch\n\n ### during save and simple best save ###\n # vmin = 10000\n if (epoch + 1) % ckpt == 0:\n # print(\"saving PNN_model_{}.pth.tar\".format(epoch))\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n optim_state=optimizer.state_dict(),\n loss=v_loss1,\n train_params=PNN_model),\n './results/PNN/PNN_model_{}.pth.tar'.format(epoch + 1))\n\n if v_loss1 < vmin:\n if os.path.isfile(save_best_file):\n os.remove(save_best_file)\n # print(\"saving PNN_model.pth.tar\")\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n optim_state=optimizer.state_dict(),\n loss1=v_loss1,\n train_params=PNN_model),\n './results/PNN/best_PNN_model_{}.pth.tar'.format(epoch))\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n optim_state=optimizer.state_dict(),\n loss1=v_loss1,\n train_params=PNN_model),\n './pretrained_models/' + sensor + '_PNNplus_model.pth.tar')\n vmin = v_loss1\n\n writer.close() # close tensorboard\n\n\ndef fine_tune_test(file_path, training_data_loader):\n from main_test_wv3 import Tester, eval_test\n\n # tester = Tester(file_path, mode='ft') # call initial model\n evaluator = Tester(file_path)\n criterion = nn.L1Loss(reduction='mean').cuda()\n \" LOAD PRETRAINED MODEL\"\n model_path = \"./results/PNN/.pth.tar\"\n if os.path.isfile(model_path):\n print(\"loading model\")\n checkpoint = torch.load(model_path)\n # checkpoint = torch.load('./pretrained_models/' + sensor_model)\n print(checkpoint.keys())\n net = checkpoint['model']\n print(net.conv1.weight.data[0, 0, 0, 0])\n net.load_state_dict(checkpoint['model_state'])\n train_params = checkpoint['train_params']\n lr = train_params['lr'] # learning rate\n print(\"lr\", lr)\n FT_epochs = 1000 # number of fine tuning epochs\n\n else:\n lr = 0.0001 * 17 * 17 * nr_bands\n FT_epochs = epochs\n net = APNN().cuda()\n print(net.conv1.weight.data[0, 0, 0, 0])\n '''\n tensor(-0.0003, device='cuda:0')\n tensor(-0.0204, device='cuda:0')\n '''\n print(net.conv1.weight.data[0, 0, 0, 0])\n print(net)\n # print(\"pretrain loss: \", checkpoint[\"loss1\"])\n\n\n\n\n print(dict(net.named_parameters()).keys())\n target_layerParam = list(map(id, net.conv3.parameters()))\n base_layerParam = filter(lambda p: id(p) not in target_layerParam, net.parameters())\n training_parameters = [{'params': net.conv3.parameters(), 'lr': lr/10},\n {'params': base_layerParam}]\n\n optimizer = optim.SGD(training_parameters, lr=lr, momentum=0.9, weight_decay=0)\n\n try:\n optimizer.load_state_dict(checkpoint[\"optim_state\"])\n except:\n print(\"default optim_state\")\n\n v_min = 10000\n ft_loss = np.zeros(FT_epochs)\n\n eval_test(net, evaluator, mode=\"eval\", mode2=\"pre\")#0.0114576\n for epoch in range(FT_epochs):\n net.train()\n epoch_train_mae = []\n for iteration, batch in enumerate(training_data_loader, 1):\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n\n lms = torch.cat([lms, pan], dim=1)\n optimizer.zero_grad() # fixed\n\n sr = net(lms) # call model\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n\n loss = criterion(sr, gt) # compute loss\n new_loss = regularization(loss, net, flag=False)\n\n epoch_train_mae.append(loss.item()) # save all losses into a vector for one epoch\n\n new_loss.backward() # fixed\n optimizer.step() # fixed\n\n running_loss = np.nanmean(epoch_train_mae)\n ft_loss[epoch] = running_loss\n\n if running_loss < v_min:\n v_min = running_loss\n PATH = './ft_network/'\n if not os.path.exists(PATH):\n os.makedirs(PATH)\n torch.save(dict(model=net,\n model_state=net.state_dict(),\n loss=ft_loss),\n PATH + '/net.pth.tar')\n net.eval()\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\")\n print('[%d] loss: %.20f' % (epoch + 1, running_loss))\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == \"__main__\":\n train_set = Dataset_Pro('./training_data/train_wv3_10000.h5') # creat data for training\n training_data_loader = DataLoader(dataset=train_set, num_workers=0, batch_size=batch_size, shuffle=True,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n validate_set = Dataset_Pro('./training_data/valid_wv3_10000.h5') # creat data for validation\n validate_data_loader = DataLoader(dataset=validate_set, num_workers=0, batch_size=batch_size, shuffle=False,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n train(training_data_loader, validate_data_loader) # call train function (call: Line 53)\n # file_path = \"./test_data/TestData_wv3.h5\"\n # fine_tune_test(file_path, training_data_loader)\n\n'''\n\n'''\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/main_pre_train_trainData_wv4.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.tensorboard import SummaryWriter\nfrom torch.autograd import Variable\nfrom torch.utils.data import DataLoader\nfrom data_wv4 import Dataset_Pro\nfrom model_wv4 import APNN, summaries, loss_with_l2_regularization, weights_init\nfrom logger import create_logger, log_string\nimport numpy as np\n\n\nimport argparse\n\nparser = argparse.ArgumentParser(description='PyTorch ImageNet Training')\n\nparser.add_argument('--out_dir', metavar='DIR', default='../results',\n help='path to save model')\nparser.add_argument('--log_dir', metavar='DIR', default='logs',\n help='path to save log')\nparser.add_argument('--tfb_dir', metavar='DIR', default=None,\n help='useless in this script.')\nparser.add_argument('--arch', '-a', metavar='ARCH', default='APNN')\n\nargs = parser.parse_args()\nargs.experimental_desc = \"APNN\"\nargs.dataset = \"WV4\"\n\nout_dir, model_save_dir, tfb_dir = create_logger(args, args.experimental_desc)\nprint(model_save_dir)\n#import shutil\n#from torch.utils.tensorboard import SummaryWriter\n\n###################################################################\n# ------------------- Pre-Define Part----------------------\n###################################################################\n# ================== Pre-Define =================== #\nSEED = 10\ntorch.manual_seed(SEED)\ntorch.cuda.manual_seed(SEED)\ntorch.cuda.manual_seed_all(SEED)\n# cudnn.benchmark = True ###自动寻找最优算法\ncudnn.deterministic = True\n\n# ============= 2) HYPER PARAMS(Pre-Defined) ==========#\n\"\"\" CHANGES: \n 1. row 48: in APNN the L1 loss is averaged only on the minibatch size,\n for the learning rate in case the loss is averaged on minibatches, patches size,\n and bands is lr=0.0001*17*17*nr_bands\n a. nr_bands takes into account the number of bands\n \n 2. row 46: should be indicated by user here, or taken from data \n \n 3. row 46: depends on the \n \n 4. the dataset is already normalized, so we do not need anymore and \n \n 5. row 49: in APNN epochs=10000\n \n 6. row 71: in APNN weight_decay=0\n \n 7. rows 182-195: in pretrained_models the best PNN model is saved\n\"\"\"\n# os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"1\"\nsensor = 'WV4'\nnr_bands = 4 #selected by user or taken from data?\nlr = 0.0001*17*17*nr_bands\nepochs = 15000\nckpt = 50\nbatch_size = 128\nmodel_path = \"../results/wv4/best_PNN_model_1706.pth.tar.pth.tar\"\nv_min = 10000\n#TODO L2 norm to do where\n# ============= 3) Load Model + Loss + Optimizer + Learn_rate_update ==========#\nmodel = APNN().cuda()\nmodel.apply(weights_init)\nif os.path.isfile(model_path):\n log_string(\"loading\")\n checkpoint = torch.load(model_path)\n model.load_state_dict(checkpoint[\"model_state\"]) ## Load the pretrained Encoder\n v_min = checkpoint[\"loss1\"]\n print(\"best_loss {:.7f}\".format(v_min))\n log_string('APNN is Successfully Loaded from %s' % (model_path))\n\n# summaries(model, grad=True) ## Summary the Network\ncriterion = nn.L1Loss(reduction='mean').cuda()\nregularization = loss_with_l2_regularization().cuda()\n#用model里有的实例id去指定model中的其他参数,而不要遍历model.parameters()\ntarget_layerParam = list(map(id, model.conv3.parameters()))\nbase_layerParam = filter(lambda p: id(p) not in target_layerParam, model.parameters())\n\ntraining_parameters = [{'params': model.conv3.parameters(), 'lr': lr/10},\n {'params': base_layerParam}]\n\noptimizer = optim.SGD(training_parameters, lr=lr, momentum=0.9)\n\nlog_string(\"inspect optimizer setting: {}\\n\".format(optimizer.state_dict()))\nlog_string(\"target id: {}\".format(target_layerParam))\n\n#模型卷积层宽卷积零填充范围\n# (input_size - kernel_size + 1) // 2 = 2* pad = 2 * blk = net_scope\nnet_scope = 0\nfor name, layer in model.named_parameters():\n if 'conv' in name and 'bias' not in name:\n net_scope += layer.shape[-1]-1\n\nnet_scope = np.sum(net_scope) + 1\nblk = net_scope//2 #8\n\nsave_best_file = '../results/PNN/PNN_model.pth.tar'\n\nPNN_model = {'sensor': sensor,\n 'lr': lr,\n 'epochs': epochs,\n 'model_sampling_period': ckpt,\n 'net_scope': net_scope,\n 'batch_size': batch_size}\n\n\nwriter = SummaryWriter('../train_logs') ## Tensorboard_show: case 2\n\ndef save_checkpoint(model, epoch): # save model function\n model_out_path = 'Weights' + '/' + \"{}.pth\".format(epoch)\n torch.save(model.state_dict(), model_out_path)\n\n###################################################################\n# ------------------- Main Train (Run second)----------------------------------\n###################################################################\n\ndef train(training_data_loader, validate_data_loader, start_epoch=0, v_min=10000):\n log_string('Start training...')\n\n for epoch in range(start_epoch, epochs, 1):\n\n epoch += 1\n epoch_train_mae, epoch_train_mse, epoch_val_mae, epoch_val_mse = [], [], [], []\n\n # ============Epoch Train=============== #\n model.train()\n\n for iteration, batch in enumerate(training_data_loader, 1):\n\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n\n lms = torch.cat([lms, pan], dim=1)\n optimizer.zero_grad() # fixed\n\n sr = model(lms) # call model\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n\n loss = criterion(sr, gt) # compute loss\n new_loss = regularization(loss, model, flag=False)\n epoch_train_mae.append(loss.item()) # save all losses into a vector for one epoch\n\n new_loss.backward() # fixed\n optimizer.step() # fixed\n\n with torch.no_grad():\n loss = nn.MSELoss()(sr, gt)\n loss.requires_grad = False\n epoch_train_mse.append(loss.item())\n\n t_loss1 = np.nanmean(np.array(epoch_train_mae)) # compute the mean value of all losses, as one epoch loss\n t_loss2 = np.nanmean(np.array(epoch_train_mse))\n\n writer.add_scalar('mae_loss/t_mae', t_loss1, epoch) # write to tensorboard to check\n writer.add_scalar('mae_loss/t_mse', t_loss2, epoch)\n log_string('Epoch: {}/{} training L1-loss: {:.7f}, L2-loss: {:.7f}'.format(epochs, epoch, t_loss1, t_loss2)) # print loss for each epoch\n # if epoch % ckpt == 0: # if each ckpt epochs, then start to save model\n # save_checkpoint(model, epoch)\n\n # ============Epoch Validate=============== #\n model.eval()\n with torch.no_grad():\n for iteration, batch in enumerate(validate_data_loader, 1):\n gt, lms, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda(), batch[3].cuda()\n gt = gt - lms\n lms = torch.cat([lms, pan], dim=1)\n sr = model(lms)\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n \n loss1 = criterion(sr, gt)\n loss2 = nn.MSELoss()(sr, gt)\n epoch_val_mae.append(loss1.item())\n epoch_val_mse.append(loss2.item())\n\n v_loss1 = np.nanmean(np.array(epoch_val_mae))\n v_loss2 = np.nanmean(np.array(epoch_val_mse))\n writer.add_scalar('val/v_mae', v_loss1, epoch)\n writer.add_scalar('val/v_mse', v_loss2, epoch)\n log_string('Epoch: {}/{} validate L1-loss: {:.7f}, L2-loss: {:7f}'.format(epochs, epoch, v_loss1, v_loss2)) # print loss for each epoch\n\n\n ### during save and simple best save ###\n # vmin = 10000\n if (epoch + 1) % ckpt == 0:\n # print(\"saving PNN_model_{}.pth.tar\".format(epoch))\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss=v_loss1,\n train_params=PNN_model),\n '{}/PNN_model_{}.pth.tar'.format(model_save_dir, epoch + 1))\n\n if v_loss1 < v_min:\n if os.path.isfile(save_best_file):\n os.remove(save_best_file)\n # print(\"saving PNN_model.pth.tar\")\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss1=v_loss1,\n train_params=PNN_model),\n '{}/best_PNN_model_{}.pth.tar'.format(model_save_dir, epoch))\n torch.save(dict(model=model,\n model_state=model.state_dict(),\n loss1=v_loss1,\n train_params=PNN_model),\n '../pretrained_models/'+sensor+'_PNNplus_model.pth.tar')\n v_min = v_loss1\n\n\n writer.close() # close tensorboard\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == \"__main__\":\n train_set = Dataset_Pro('../training_data/train_wv4_10000.h5') # creat data for training\n training_data_loader = DataLoader(dataset=train_set, num_workers=0, batch_size=batch_size, shuffle=True,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n validate_set = Dataset_Pro('../training_data/valid_wv4_10000.h5') # creat data for validation\n validate_data_loader = DataLoader(dataset=validate_set, num_workers=0, batch_size=batch_size, shuffle=False,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n train(training_data_loader, validate_data_loader, 1707, v_min) # call train function (call: Line 53)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/main_test_qb.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.autograd import Variable\nfrom torch.utils.data import DataLoader\nfrom data_qb import Dataset_Pro\nimport h5py\nfrom data_single_read import load_set\nfrom evaluate import compute_index\nfrom model_qb import APNN, summaries, weights_init, loss_with_l2_regularization\nimport numpy as np\nimport scipy.io as sio\nfrom time import time\nfrom evaluate import analysis_accu\n\n\n###################################################################\n# ------------------- Sub-Functions (will be used) -------------------\n###################################################################\n\ndef load_set(file_path, blk):\n\n suffix = file_path.split('.')\n if suffix[-1] == 'h5':\n data = h5py.File(file_path) # NxCxHxW = 0x1x2x3\n ms1 = data[\"ms\"][...] # NxCxHxW=0,1,2,3\n shape_size = len(ms1.shape)\n elif suffix[-1] == 'mat':\n ## ===== case2: HxWxC\n data = sio.loadmat(file_path) #\n print(data.keys())\n ms1 = data[\"I_MS_LR\"][...] # NxCxHxW=0,1,2,3\n shape_size = len(ms1.shape)\n else:\n raise NotImplemented(\"file foramt is not supported\")\n\n if suffix[-2][-2:] == 'FR':\n data['I_GT'] = data['I_MS_LR'] #exception\n\n if shape_size == 4: # NxCxHxW\n # tensor type:\n lms1 = data['lms'][...] # NxCxHxW = 4x8x512x512\n lms1 = np.array(lms1, dtype=np.float32) / 2047.0\n lms = torch.from_numpy(lms1)\n\n pan1 = data['pan'][...] # NxCxHxW = 4x8x512x512\n pan1 = np.array(pan1, dtype=np.float32) / 2047.0\n pan = torch.from_numpy(pan1)\n\n test_I_in1 = np.concatenate([lms1, pan1], axis=1) # NxCxHxW = Nx9xHxW\n test_I_in1 = np.pad(test_I_in1, ((0, 0), (0, 0), (blk, blk), (blk, blk)), mode='edge') # pading\n test_I_in = torch.from_numpy(test_I_in1) # NxCxHxW = Nx9xHxW\n\n ms1 = data['ms'][...] # NxCxHxW = 4x8x512x512\n ms1 = np.array(ms1, dtype=np.float32) / 2047.0\n ms = torch.from_numpy(ms1)\n\n gt1 = data['gt'][...] # NxCxHxW = 4x8x512x512\n gt1 = np.array(gt1, dtype=np.float32) / 2047.0\n gt = torch.from_numpy(gt1)\n\n return test_I_in, ms, pan, gt\n if shape_size == 3: # HxWxC\n\n # tensor type:\n lms1 = data['I_MS'][...] # HxWxC=0,1,2\n lms1 = np.expand_dims(lms1, axis=0) # 1xHxWxC\n lms1 = np.array(lms1, dtype=np.float32) / 2047.0 # 1xHxWxC\n lms = torch.from_numpy(lms1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n pan1 = data['I_PAN'][...] # HxW\n pan1 = np.expand_dims(pan1, axis=0) # 1xHxW\n pan1 = np.expand_dims(pan1, axis=3) # 1xHxWx1\n pan1 = np.array(pan1, dtype=np.float32) / 2047. # 1xHxWx1\n pan = torch.from_numpy(pan1).permute(0, 3, 1, 2) # Nx1xHxW:\n\n test_I_in1 = np.concatenate([lms1, pan1], axis=3) # 1xHxWx(C+1) = Nx9xHxW\n test_I_in1 = np.transpose(test_I_in1, (0, 3, 1, 2)) # 1x(C+1)xHxW\n test_I_in1 = np.pad(test_I_in1, ((0, 0), (0, 0), (blk, blk), (blk, blk)), mode='edge') # NCHW\n test_I_in = torch.from_numpy(test_I_in1) # NxCxHxW = Nx9xHxW\n\n ms1 = data['I_MS_LR'][...] # HxWxC=0,1,2\n ms1 = np.expand_dims(ms1, axis=0) # 1xHxWxC\n ms1 = np.array(ms1, dtype=np.float32) / 2047.0 # 1xHxWxC\n ms = torch.from_numpy(ms1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n gt1 = data['I_GT'][...] # HxWxC=0,1,2\n gt1 = np.expand_dims(gt1, axis=0) # 1xHxWxC\n gt1 = np.array(gt1, dtype=np.float32) / 2047.0 # 1xHxWxC\n gt = torch.from_numpy(gt1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n return test_I_in, ms, pan, gt\n\n\n###################################################################\n# ------------------- Main Test (Run second) -------------------\n###################################################################\n\n## 1) initial test by model ##\\\nblk = 8 # 4\n\n\nclass Tester():\n def __init__(self, file_path, mode):\n\n test_I_in, test_ms, test_pan, test_gt = load_set(file_path, blk)\n self.test_I_in = test_I_in\n self.test_ms = test_ms\n self.test_pan = test_pan\n self.test_gt = test_gt\n self.mode = mode\n self.file_path = file_path\n \"the fine tuning phase requires downgraded input resolution\"\n if mode == 'ft':\n from wald_utilities import wald_protocol\n ms_lr, pan_lr = wald_protocol(test_ms, test_pan, 4., 'QB', channels=4)\n self.test_I_in = torch.cat((ms_lr, pan_lr), dim=1)\n self.test_I_in = torch.nn.functional.pad(self.test_I_in, (8, 8, 8, 8), mode='reflect') # NCHW\n self.test_gt = self.test_ms\n\n def __call__(self, model):\n x = self.test_I_in # send to cuda, important!\n x = x.cuda().float() # convert to tensor type:\n out2 = model(x)\n if self.mode == 'test' or self.mode == 'RR':\n sr = out2 + self.test_I_in[:, :-1, blk:-blk, blk:-blk].cuda() # NxCxHxW\n sr = sr.permute(0, 2, 3, 1)\n gt = self.test_gt.permute(0, 2, 3, 1).cuda()\n our_CC, our_PSNR, our_SSIM, our_SAM, our_ERGAS = analysis_accu(gt[0, ...], sr[0, ...], 4)\n print(f'[{self.file_path}]: our_CC: {our_CC}, our_PSNR: {our_PSNR}, '\n f'our_SSIM: {our_SSIM},\\n'\n f'our_SAM: {our_SAM} our_ERGAS: {our_ERGAS}')\n return out2\n\n\n## 2) target-adative's fine_tune_training, i.e., PNNplus##\n\ndef test(file_path, sensor_model):\n suffix = file_path.split('.')[-2][-2:]\n if suffix == 'RR' or suffix == 'FR':\n simulated = suffix\n else:\n simulated = 'test'\n\n tester = Tester(file_path, mode='ft') # call initial model\n evaluator = Tester(file_path, mode=simulated)\n criterion = nn.L1Loss(reduction='mean').cuda()\n regularization = loss_with_l2_regularization().cuda()\n \" LOAD PRETRAINED MODEL\"\n init_loss = 0\n model_path = \"../pretrained_models/1QB_PNNplus_model.pth.tar\"\n if os.path.isfile(model_path):\n print(\"loading model\")\n checkpoint = torch.load(model_path)\n # checkpoint = torch.load('./pretrained_models/' + sensor_model)\n print(checkpoint.keys())\n net = checkpoint['model']\n print(net.conv1.weight.data[0, 0, 0, 0])\n net.load_state_dict(checkpoint['model_state'])\n '''\n ft: lr setting\n 1-SF\n 2e-4 epoch 150\n 1-ik\n 2e-4\n 7-h5\n 1e-5 \n '''\n lr_ = 2e-4#0.0001 * 17 * 17 * 8 # 1e-3#2e-4#0.0001 * 17 * 17 * 8\n FT_epochs = 50 # number of fine tuning epochs\n # init_loss = checkpoint[\"loss1\"]\n else:\n nr_bands = 4 # selected by user or taken from data?\n lr_ = 0.0001 * 17 * 17 * nr_bands\n FT_epochs = 5000\n net = APNN().cuda()\n net.apply(weights_init)\n print(net.conv1.weight.data[0, 0, 0, 0])\n\n print(net.conv1.weight.data[0, 0, 0, 0])\n print(net)\n\n test_gt = tester.test_gt - tester.test_I_in[:, :-1, blk:-blk, blk:-blk]\n pretrain_inIt_loss = criterion(net(tester.test_I_in.cuda()), test_gt.cuda()).item()\n print(\"init loss: {:.20f} pretrain_inIt loss: {:.20f}\".format(init_loss, pretrain_inIt_loss))\n eval_test(net, evaluator, mode=\"eval\", mode2=\"pre\")\n print('-'*100)\n \"scaling learning rate on last layer\"\n target_layerParam = list(map(id, net.conv3.parameters()))\n base_layerParam = filter(lambda p: id(p) not in target_layerParam, net.parameters())\n\n training_parameters = [{'params': net.conv3.parameters(), 'lr': lr_ / 10},\n {'params': base_layerParam}]\n\n optimizer = optim.SGD(training_parameters, lr=lr_, momentum=0.9)\n\n device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n v_min = 10000\n ft_loss = np.zeros(FT_epochs)\n Train_time = time()\n\n ## 2.1) \"FINE TUNING\"--training\n for epoch in range(FT_epochs): # loop over the testing image multiple times\n net.train()\n # running_loss = 0.0\n # loading testing image\n test_I_in = tester.test_I_in\n test_gt = tester.test_gt\n\n # residual\n test_gt = test_gt - test_I_in[:, :-1, blk:-blk, blk:-blk]\n\n # zero the parameter gradients\n optimizer.zero_grad()\n\n # forward + backward + optimize\n x1 = test_I_in # send to cuda, important!\n x2 = test_gt\n x1 = x1.cuda().float() # convert to tensor type:\n x2 = x2.cuda().float() # convert to tensor type:\n\n outputs = net(x1)\n\n loss = criterion(outputs, x2) # compute loss\n new_loss = regularization(loss, net, flag=False)\n\n new_loss.backward()\n optimizer.step()\n\n running_loss = loss.item()\n ft_loss[epoch] = running_loss\n\n if running_loss < v_min:\n\n PATH = '../ft_network/QB'\n if not os.path.exists(PATH):\n os.makedirs(PATH)\n torch.save(dict(model=net,\n model_state=net.state_dict(),\n loss=ft_loss),\n PATH + '/net.pth.tar')\n # if np.abs(running_loss - v_min) > 1e-3:\n net.eval()\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\")\n v_min = running_loss\n\n print('[%d] loss: %.7f' % (epoch + 1, running_loss))\n\n Train_time = time() - Train_time\n\n ## 2.2) \"FINE TUNING\"--testing\n \" LOAD BEST MODEL\"\n checkpoint = torch.load('../ft_network/QB/net.pth.tar')\n net = checkpoint['model']\n net.load_state_dict(checkpoint['model_state'])\n\n \" PANSHARPENING \"\n net.to(device)\n net.eval()\n print(net.conv3.weight.data[0, 0, 0, 0], net.conv2.weight.data[0, 0, 0, 0], net.conv1.weight.data[0, 0, 0, 0])\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\")\n\n\ndef eval_test(net, evaluator, mode=\"pre\", mode2=\"pre\"):\n with torch.no_grad():\n Test_time = time()\n sr = evaluator(net) # NxCxHxW\n Test_time = time() - Test_time\n\n # skip connection to add low resolution ms and residual(np version)\n sr = sr.cpu().detach().numpy() + evaluator.test_I_in[:, :-1, blk:-blk,\n blk:-blk].cpu().detach().numpy() # NxCxHxW\n\n # convert to numpy type with permute and squeeze: HxWxC (go to cpu for easy saving)\n sr = torch.from_numpy(sr) # convert to tensor version\n sr = sr.permute(0, 2, 3, 1).cpu().detach().numpy() # to: NxHxWxC\n sr = np.clip(sr, 0, 1)\n\n # print('------> [PNN+]: Fine-tuning (%d it) time = %0.4f // Prediction time = %0.4f' % (\n # FT_epochs, Train_time, Test_time))\n\n num_exm = sr.shape[0]\n if num_exm == 1:\n if evaluator.mode == \"RR\":\n file_name = \"apnn_qb_rs\" + '_ik_' + mode2 + \".mat\"\n if evaluator.mode == \"FR\":\n file_name = \"apnn_qb_os\" + '_ik_' + mode2 + \".mat\"\n # file_name2 = \"E:/01-DL-Pansharpening-Toolbox/03-Comparisons(Matlab)/2_DL_Result/WV4/APNN\"\n file_name2 = \"../results\"\n save_name = os.path.join(file_name2, file_name)\n sio.savemat(save_name, {'apnn_qb': sr[0, :, :, :]})\n else:\n for index in range(num_exm): # save the DL results to the 03-Comparisons(Matlab)\n file_name = \"apnn_qb_rs\" + str(index) + mode2 + \".mat\"\n # file_name2 = \"E:/01-DL-Pansharpening-Toolbox/03-Comparisons(Matlab)/2_DL_Result/WV4/APNN\"\n file_name2 = \"../results\"\n save_name = os.path.join(file_name2, file_name)\n sio.savemat(save_name, {'apnn_qb_rs': sr[index, :, :, :]})\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == '__main__':\n # file_path = \"E:/01-DL-Pansharpening-Toolbox/01-Data-Simulation(Matlab)/Dataset_Gemine/01-DataSimu/QB/TestData_qb.h5\"\n # file_path = \"../test_data/imgs/San_Francisco_QB_RR.mat\"\n # file_path = \"../test_data/TestData_qb.h5\"\n file_path = \"../test_data/imgs/Toulouse_IKONOS_RR.mat\"\n \"SELECT SENSOR AND TESTING IMAGE\"\n sensor_model = 'QB'\n available_models = ['IKONOS', 'GeoEye1', 'WV2', 'WV3', 'WV4', 'QB']\n if sensor_model in available_models:\n sensor_model = sensor_model + '_PNNplus_model.pth.tar'\n\n test(file_path, sensor_model)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/main_test_wv2.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.autograd import Variable\nfrom torch.utils.data import DataLoader\nfrom data_wv2 import Dataset_Pro\nimport h5py\nfrom data_single_read import load_set\nfrom evaluate import compute_index\nfrom model_wv2 import APNN, summaries, weights_init, loss_with_l2_regularization\nimport numpy as np\nimport scipy.io as sio\nfrom time import time\nfrom evaluate import analysis_accu\n\n\n###################################################################\n# ------------------- Sub-Functions (will be used) -------------------\n###################################################################\n\ndef load_set(file_path, blk):\n\n suffix = file_path.split('.')\n\n if suffix[-1] == 'h5':\n data = h5py.File(file_path) # NxCxHxW = 0x1x2x3\n ms1 = data[\"ms\"][...] # NxCxHxW=0,1,2,3\n shape_size = len(ms1.shape)\n elif suffix[-1] == 'mat':\n # ===== case2: HxWxC\n data = sio.loadmat(file_path) #\n print(data.keys())\n ms1 = data[\"I_MS_LR\"][...] # NxCxHxW=0,1,2,3\n shape_size = len(ms1.shape)\n else:\n print(\"file format is not supporetd\")\n raise NotImplemented\n\n if suffix[-2][-2:] == 'FR':\n data['I_GT'] = data['I_MS_LR'] #exception\n\n if shape_size == 4: # NxCxHxW\n # tensor type:\n lms1 = data['lms'][...] # NxCxHxW = 4x8x512x512\n lms1 = np.array(lms1, dtype=np.float32) / 2047.0\n lms = torch.from_numpy(lms1)\n\n pan1 = data['pan'][...] # NxCxHxW = 4x8x512x512\n pan1 = np.array(pan1, dtype=np.float32) / 2047.0\n pan = torch.from_numpy(pan1)\n\n test_I_in1 = np.concatenate([lms1, pan1], axis=1) # NxCxHxW = Nx9xHxW\n test_I_in1 = np.pad(test_I_in1, ((0, 0), (0, 0), (blk, blk), (blk, blk)), mode='edge') # pading\n test_I_in = torch.from_numpy(test_I_in1) # NxCxHxW = Nx9xHxW\n\n ms1 = data['ms'][...] # NxCxHxW = 4x8x512x512\n ms1 = np.array(ms1, dtype=np.float32) / 2047.0\n ms = torch.from_numpy(ms1)\n\n gt1 = data['gt'][...] # NxCxHxW = 4x8x512x512\n gt1 = np.array(gt1, dtype=np.float32) / 2047.0\n gt = torch.from_numpy(gt1)\n\n return test_I_in, ms, pan, gt\n\n if shape_size == 3: # HxWxC\n\n # tensor type:\n lms1 = data['I_MS'][...] # HxWxC=0,1,2\n lms1 = np.expand_dims(lms1, axis=0) # 1xHxWxC\n lms1 = np.array(lms1, dtype=np.float32) / 2047.0 # 1xHxWxC\n lms = torch.from_numpy(lms1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n pan1 = data['I_PAN'][...] # HxW\n pan1 = np.expand_dims(pan1, axis=0) # 1xHxW\n pan1 = np.expand_dims(pan1, axis=3) # 1xHxWx1\n pan1 = np.array(pan1, dtype=np.float32) / 2047. # 1xHxWx1\n pan = torch.from_numpy(pan1).permute(0, 3, 1, 2) # Nx1xHxW:\n\n test_I_in1 = np.concatenate([lms1, pan1], axis=3) # 1xHxWx(C+1) = Nx9xHxW\n test_I_in1 = np.transpose(test_I_in1, (0, 3, 1, 2)) # 1x(C+1)xHxW\n test_I_in1 = np.pad(test_I_in1, ((0, 0), (0, 0), (blk, blk), (blk, blk)), mode='edge') # NCHW\n test_I_in = torch.from_numpy(test_I_in1) # NxCxHxW = Nx9xHxW\n\n ms1 = data['I_MS_LR'][...] # HxWxC=0,1,2\n ms1 = np.expand_dims(ms1, axis=0) # 1xHxWxC\n ms1 = np.array(ms1, dtype=np.float32) / 2047.0 # 1xHxWxC\n ms = torch.from_numpy(ms1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n gt1 = data['I_GT'][...] # HxWxC=0,1,2\n gt1 = np.expand_dims(gt1, axis=0) # 1xHxWxC\n gt1 = np.array(gt1, dtype=np.float32) / 2047.0 # 1xHxWxC\n gt = torch.from_numpy(gt1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n return test_I_in, ms, pan, gt\n\n\n###################################################################\n# ------------------- Main Test (Run second) -------------------\n###################################################################\n\n## 1) initial test by model ##\nblk = 8\n\n\nclass Tester():\n def __init__(self, file_path, mode):\n\n test_I_in, test_ms, test_pan, test_gt = load_set(file_path, blk)\n self.test_I_in = test_I_in\n self.test_ms = test_ms\n self.test_pan = test_pan\n self.test_gt = test_gt\n self.mode = mode\n self.file_path = file_path\n \"the fine tuning phase requires downgraded input resolution\"\n if mode == 'ft':\n from wald_utilities import wald_protocol\n ms_lr, pan_lr = wald_protocol(test_ms, test_pan, 4., 'WV2', channels=8)\n self.test_I_in = torch.cat((ms_lr, pan_lr), dim=1)\n self.test_I_in = torch.nn.functional.pad(self.test_I_in, (8, 8, 8, 8), mode='reflect') # NCHW\n self.test_gt = self.test_ms\n\n def __call__(self, model):\n x = self.test_I_in # send to cuda, important!\n x = x.cuda().float() # convert to tensor type:\n out2 = model(x)\n\n if self.mode == 'test' or self.mode == 'RR':\n sr = out2 + self.test_I_in[:, :-1, blk:-blk, blk:-blk].cuda() # NxCxHxW\n sr = sr.permute(0, 2, 3, 1)\n gt = self.test_gt.permute(0, 2, 3, 1).cuda()\n our_CC, our_PSNR, our_SSIM, our_SAM, our_ERGAS = analysis_accu(gt[0, ...], sr[0, ...], 4)\n print(f'[{self.file_path}]: our_CC: {our_CC}, our_PSNR: {our_PSNR}, '\n f'our_SSIM: {our_SSIM},\\n'\n f'our_SAM: {our_SAM} our_ERGAS: {our_ERGAS}')\n\n return out2\n\n\n## 2) target-adative's fine_tune_training, i.e., PNNplus##\n\ndef test(file_path, sensor_model):\n\n suffix = file_path.split('.')[-2][-2:]\n if suffix== 'RR' or suffix == 'FR':\n simulated = suffix\n else:\n simulated = 'test'\n\n tester = Tester(file_path, mode='ft') # call initial model\n evaluator = Tester(file_path, mode=simulated)\n criterion = nn.L1Loss(reduction='mean').cuda()\n regularization = loss_with_l2_regularization().cuda()\n \" LOAD PRETRAINED MODEL\"\n init_loss = 0\n model_path = \"../pretrained_models/1WV2_PNNplus_model.pth.tar\"\n if os.path.isfile(model_path):\n print(\"loading model\")\n checkpoint = torch.load(model_path)\n # checkpoint = torch.load('./pretrained_models/' + sensor_model)\n print(checkpoint.keys())\n net = checkpoint['model']\n print(net.conv1.weight.data[0, 0, 0, 0])\n net.load_state_dict(checkpoint['model_state'])\n lr_ = 1e-6\n FT_epochs = 200 # number of fine tuning epochs\n # init_loss = checkpoint[\"loss1\"]\n else:\n nr_bands = 8 # selected by user or taken from data?\n lr_ = 0.0001 * 17 * 17 * nr_bands\n FT_epochs = 5000\n net = APNN().cuda()\n net.apply(weights_init)\n print(net.conv1.weight.data[0, 0, 0, 0])\n\n print(net.conv1.weight.data[0, 0, 0, 0])\n print(net)\n\n test_gt = tester.test_gt - tester.test_I_in[:, :-1, blk:-blk, blk:-blk]\n pretrain_inIt_loss = criterion(net(tester.test_I_in.cuda()), test_gt.cuda()).item()\n print(\"init loss: {:.20f} pretrain_inIt loss: {:.20f}\".format(init_loss, pretrain_inIt_loss))\n eval_test(net, evaluator, mode=\"eval\", mode2=\"pre\")\n print('-' * 100)\n \"scaling learning rate on last layer\"\n target_layerParam = list(map(id, net.conv3.parameters()))\n base_layerParam = filter(lambda p: id(p) not in target_layerParam, net.parameters())\n\n training_parameters = [{'params': net.conv3.parameters(), 'lr': lr_ / 10},\n {'params': base_layerParam}]\n\n optimizer = optim.SGD(training_parameters, lr=lr_, momentum=0.9)\n\n device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n v_min = 10000\n ft_loss = np.zeros(FT_epochs)\n Train_time = time()\n print(tester.test_gt.shape)\n ## 2.1) \"FINE TUNING\"--training\n for epoch in range(FT_epochs): # loop over the testing image multiple times\n net.train()\n # running_loss = 0.0\n # loading testing image\n test_I_in = tester.test_I_in\n test_gt = tester.test_gt\n\n # residual\n test_gt = test_gt - test_I_in[:, :-1, blk:-blk, blk:-blk]\n\n # zero the parameter gradients\n optimizer.zero_grad()\n\n # forward + backward + optimize\n x1 = test_I_in # send to cuda, important!\n x2 = test_gt\n x1 = x1.cuda().float() # convert to tensor type:\n x2 = x2.cuda().float() # convert to tensor type:\n\n outputs = net(x1)\n\n loss = criterion(outputs, x2) # compute loss\n new_loss = regularization(loss, net, flag=False)\n\n new_loss.backward()\n optimizer.step()\n\n running_loss = loss.item()\n ft_loss[epoch] = running_loss\n\n if running_loss < v_min:\n\n PATH = '../ft_network/WV2'\n if not os.path.exists(PATH):\n os.makedirs(PATH)\n torch.save(dict(model=net,\n model_state=net.state_dict(),\n loss=ft_loss),\n PATH + '/net.pth.tar')\n if np.abs(running_loss - v_min) > 1e-3:\n net.eval()\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\")\n v_min = running_loss\n\n print('[%d] loss: %.20f' % (epoch + 1, running_loss))\n\n Train_time = time() - Train_time\n\n ## 2.2) \"FINE TUNING\"--testing\n \" LOAD BEST MODEL\"\n checkpoint = torch.load('../ft_network/WV2/net.pth.tar')\n net = checkpoint['model']\n net.load_state_dict(checkpoint['model_state'])\n\n \" PANSHARPENING \"\n net.to(device)\n net.eval()\n print(net.conv3.weight.data[0, 0, 0, 0], net.conv2.weight.data[0, 0, 0, 0], net.conv1.weight.data[0, 0, 0, 0])\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\")\n\n\ndef eval_test(net, evaluator, mode=\"pre\", mode2=\"pre\"):\n with torch.no_grad():\n Test_time = time()\n sr = evaluator(net) # NxCxHxW\n Test_time = time() - Test_time\n\n # skip connection to add low resolution ms and residual(np version)\n sr = sr.cpu().detach().numpy() + evaluator.test_I_in[:, :-1, blk:-blk,\n blk:-blk].cpu().detach().numpy() # NxCxHxW\n\n # convert to numpy type with permute and squeeze: HxWxC (go to cpu for easy saving)\n sr = torch.from_numpy(sr) # convert to tensor version\n sr = sr.permute(0, 2, 3, 1).cpu().detach().numpy() # to: NxHxWxC\n sr = np.clip(sr, 0, 1)\n\n # print('------> [PNN+]: Fine-tuning (%d it) time = %0.4f // Prediction time = %0.4f' % (\n # FT_epochs, Train_time, Test_time))\n\n num_exm = sr.shape[0]\n if num_exm == 1:\n if evaluator.mode == \"RR\":\n file_name = \"apnn_wv2_rs\" + '_rio_' + mode2 + \".mat\"\n if evaluator.mode == \"FR\":\n file_name = \"apnn_wv2_os\" + '_rio_' + mode2 + \".mat\"\n\n # file_name2 = \"E:/01-DL-Pansharpening-Toolbox/03-Comparisons(Matlab)/2_DL_Result/WV4/APNN\"\n file_name2 = \"../results\"\n save_name = os.path.join(file_name2, file_name)\n sio.savemat(save_name, {'apnn_wv2': sr[0, :, :, :]})\n else:\n for index in range(num_exm): # save the DL results to the 03-Comparisons(Matlab)\n file_name = \"apnn_wv2_rs\" + str(index) + mode2 + \".mat\"\n # file_name2 = \"E:/01-DL-Pansharpening-Toolbox/03-Comparisons(Matlab)/2_DL_Result/WV4/APNN\"\n file_name2 = \"../results\"\n save_name = os.path.join(file_name2, file_name)\n sio.savemat(save_name, {'apnn_wv2_rs': sr[index, :, :, :]})\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == '__main__':\n file_path = \"../test_data/imgs/Rio_WV2_FR.mat\"\n # file_path = \"../test_data/TestData_wv2.h5\"\n \"SELECT SENSOR AND TESTING IMAGE\"\n sensor_model = 'WV2'\n available_models = ['IKONOS', 'GeoEye1', 'WV2', 'WV3', 'WV4', 'QB']\n if sensor_model in available_models:\n sensor_model = sensor_model + '_PNNplus_model.pth.tar'\n\n test(file_path, sensor_model)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/main_test_wv3.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.autograd import Variable\nfrom torch.utils.data import DataLoader\nfrom data_wv3 import Dataset_Pro\nimport h5py\nfrom data_single_read import load_set\nfrom evaluate import compute_index\nfrom model_wv3 import APNN, summaries, weights_init, loss_with_l2_regularization\nimport numpy as np\nimport scipy.io as sio\nfrom time import time\nfrom evaluate import analysis_accu\n\nimport math\n\nclass L1_Loss_clip(nn.Module):\n def __init__(self, multiple=2048.0, clip_flag=True, inter=False):\n super(L1_Loss_clip, self).__init__()\n self.clip_flag = clip_flag\n self.inter = inter\n self.multiple = multiple\n self.criterion = nn.L1Loss(reduction='mean')\n\n\n def forward(self, x, target, lms):\n if self.clip_flag:\n # x = torch.clamp(x * self.multiple, 0, 2048) / self.multiple\n x = x - lms\n if self.inter:\n loss = torch.mean((x - target) ** 2)#self.criterion(x, target)#torch.mean(torch.abs(x - target))\n return loss\n else:\n l1_loss = self.criterion(x, target)\n\n return l1_loss\n\n\n\n###################################################################\n# ------------------- Sub-Functions (will be used) -------------------\n###################################################################\ndef load_set(file_path, blk):\n\n suffix = file_path.split('.')\n\n if suffix[-1] == 'h5':\n ## ===== case1: NxCxHxW\n data = h5py.File(file_path)\n ms1 = data[\"ms\"][...] # NxCxHxW=0,1,2,3\n shape_size = len(ms1.shape)\n elif suffix[-1] == 'mat':\n # ===== case2: HxWxC\n data = sio.loadmat(file_path) #\n ms1 = data[\"I_MS_LR\"][...] # NxCxHxW=0,1,2,3\n shape_size = len(ms1.shape)\n else:\n raise NotImplemented(\"file format is not suppoted\")\n \"\"\"this is en exception to be addressed: RR data has four fields, FR has no GT\n For the finetuing the GT is necessary both in RR and FR testing case.\n the I_MS_LR has taken as GT in both cases\n \"\"\"\n\n if suffix[-2][-2:] == 'FR':\n data['I_GT'] = data['I_MS_LR'] #exception\n\n if shape_size == 4: # NxCxHxW\n\n # tensor type:\n lms1 = data['lms'][...] # NxCxHxW = 4x8x512x512\n lms1 = np.array(lms1, dtype=np.float32) / 2047.0\n lms = torch.from_numpy(lms1)\n\n pan1 = data['pan'][...] # NxCxHxW = 4x8x512x512\n pan1 = np.array(pan1, dtype=np.float32) / 2047.0\n pan = torch.from_numpy(pan1)\n\n test_I_in1 = np.concatenate([lms1, pan1], axis=1) # NxCxHxW = Nx9xHxW\n test_I_in1 = np.pad(test_I_in1, ((0, 0), (0, 0), (blk, blk), (blk, blk)), mode='edge') # pading\n test_I_in = torch.from_numpy(test_I_in1) # NxCxHxW = Nx9xHxW\n\n ms1 = data['ms'][...] # NxCxHxW = 4x8x512x512\n ms1 = np.array(ms1, dtype=np.float32) / 2047.0\n ms = torch.from_numpy(ms1)\n\n gt1 = data['gt'][...] # NxCxHxW = 4x8x512x512\n gt1 = np.array(gt1, dtype=np.float32) / 2047.0\n gt = torch.from_numpy(gt1)\n\n return test_I_in, ms, pan, gt\n\n if shape_size == 3: # HxWxC\n\n # tensor type:\n lms1 = data['I_MS'][...] # HxWxC=0,1,2\n lms1 = np.expand_dims(lms1, axis=0) # 1xHxWxC\n lms1 = np.array(lms1, dtype=np.float32) / 2047.0 # 1xHxWxC\n lms = torch.from_numpy(lms1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n pan1 = data['I_PAN'][...] # HxW\n pan1 = np.expand_dims(pan1, axis=0) # 1xHxW\n pan1 = np.expand_dims(pan1, axis=3) # 1xHxWx1\n pan1 = np.array(pan1, dtype=np.float32) / 2047. # 1xHxWx1\n pan = torch.from_numpy(pan1).permute(0, 3, 1, 2) # Nx1xHxW:\n\n test_I_in1 = np.concatenate([lms1, pan1], axis=3) # 1xHxWx(C+1) = Nx9xHxW\n test_I_in1 = np.transpose(test_I_in1, (0, 3, 1, 2)) # 1x(C+1)xHxW\n test_I_in1 = np.pad(test_I_in1, ((0, 0), (0, 0), (blk, blk), (blk, blk)), mode='edge') # NCHW\n test_I_in = torch.from_numpy(test_I_in1) # NxCxHxW = Nx9xHxW\n\n #预先wald仿真的\n ms1 = data['I_MS_LR'][...] # HxWxC=0,1,2\n ms1 = np.expand_dims(ms1, axis=0) # 1xHxWxC\n ms1 = np.array(ms1, dtype=np.float32) / 2047.0 # 1xHxWxC\n ms_lr = torch.from_numpy(ms1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC 1,8,128,128\n\n gt1 = data['I_GT'][...] # HxWxC=0,1,2\n gt1 = np.expand_dims(gt1, axis=0) # 1xHxWxC\n gt1 = np.array(gt1, dtype=np.float32) / 2047.0 # 1xHxWxC\n gt = torch.from_numpy(gt1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n return test_I_in, ms_lr, pan, gt\n\n\n###################################################################\n# ------------------- Main Test (Run second) -------------------\n###################################################################\n\n## 1) initial test by model ##\nblk = 8\n\n\nclass Tester():\n def __init__(self, file_path, mode):\n \"if mode==ft the input data need to be downgraded as in row 134\"\n test_I_in, test_ms_lr, test_pan, test_gt = load_set(file_path, blk)\n self.test_I_in = test_I_in #ms 1,9,528,528\n # self.test_lms = test_lms #lms - wald 1,8,512,512\n self.test_pan = test_pan #pan 1,1,512,512\n self.test_gt = test_gt #ms^ 1,8,512,512\n self.test_ms = test_ms_lr #1,8,128,128\n self.mode = mode\n self.file_path = file_path\n self.loss_clip = L1_Loss_clip(inter=True, clip_flag=True).cuda()\n \"the fine tuning phase requires downgraded input resolution\"\n if mode == 'ft':\n from wald_utilities import wald_protocol\n ms_lr, pan_lr = wald_protocol(test_ms_lr, test_pan, 4., 'WV3')\n self.test_I_in = torch.cat((test_ms_lr, pan_lr), dim=1)\n self.test_I_in = torch.nn.functional.pad(self.test_I_in, (8, 8, 8, 8), mode='reflect') # NCHW\n self.test_gt = self.test_ms\n #假如数据在matlab里仿真了，则有ms_lr: 512, pan 512, ms: 512, pan 2048，那应该不再需要再处理一次\n # lms是128 but pan是512, pan没有仿真数据，即pan_lr 128, 又没有gt是128\n #所以pan应该是仿真过的,即原始是2048,现在是512, 那ms还要wald处理一下吗\n # from wald_utilities import wald_protocol_v2\n #\n # pan_lr = wald_protocol_v2(None, test_pan, 4., 'WV3')\n # self.test_I_in = torch.cat((test_lms, pan_lr), dim=1)\n # self.test_I_in = torch.nn.functional.pad(self.test_I_in, (8, 8, 8, 8), mode='reflect') # NCHW\n\n # self.test_ms = test_ms.unsqueeze(dim=0).float() # convert to tensor type: 1xCxHxW (unsqueeze(dim=0))\n # self.test_pan = test_pan.unsqueeze(dim=0).float() # convert to tensor type: 1x1xHxW\n # self.test_I_in = test_I_in.unsqueeze(dim=0).float() # 1xCxHxW\n # self.test_gt = (test_gt * 2047.0).cuda()\n\n def __call__(self, model, err=None):\n x = self.test_I_in # send to cuda, important!\n x = x.cuda().float() # convert to tensor type:\n out2 = model(x)#1.4942e-03\n # result_our = out2 + x[:, :-1, blk:-blk, blk:-blk]\n # out2 = model(self.test_I_in)\n # result_our = out2 + self.test_I_in[:, :-1, blk:-blk, blk:-blk]\n # result_our = torch.squeeze(result_our).permute(1, 2, 0)\n # result_our = result_our* 2047\n # our_SAM, our_ERGAS = compute_index(self.test_gt, result_our, 4)\n # print('our_SAM: {} our_ERGAS: {}'.format(our_SAM, our_ERGAS)) # print loss for each epoch\n\n if self.mode == 'RR' or self.mode == 'test':\n test_in = self.test_I_in[:, :-1, blk:-blk, blk:-blk].cuda()\n sr = out2 + test_in#self.test_I_in[:, :-1, blk:-blk, blk:-blk].cuda() # NxCxHxW\n sr = sr.permute(0, 2, 3, 1)\n gt = self.test_gt.permute(0, 2, 3, 1).cuda()\n\n # sr = out2.permute(0, 2, 3, 1)\n # gt = self.test_gt.cuda() - test_in # NxCxHxW\n # gt = gt.permute(0, 2, 3, 1)\n our_CC, our_PSNR, our_SSIM, our_SAM, our_ERGAS, our_Q8 = analysis_accu(gt[0, ...], sr[0, ...], 4)\n print(f'[{self.file_path}]: our_CC: {our_CC}, our_PSNR: {our_PSNR}, '\n f'our_SSIM: {our_SSIM},\\n'\n f'our_SAM: {our_SAM} our_ERGAS: {our_ERGAS} our_Q8: {our_Q8}')\n\n\n return out2\n\n\n## 2) target-adative's fine_tune_training, i.e., PNNplus##\n\ndef test(file_path, sensor_model):\n\n suffix = file_path.split('.')[-2][-2:]\n if suffix== 'RR' or suffix == 'FR':\n simulated = suffix\n else:\n simulated = 'test'\n\n tester = Tester(file_path, mode='ft') # call initial model\n evaluator = Tester(file_path, mode=simulated)\n criterion = nn.MSELoss(reduction='mean').cuda()#L1_Loss_clip(clip_flag=True, inter=True).cuda()\n regularization = loss_with_l2_regularization().cuda()\n \" LOAD PRETRAINED MODEL\"\n init_loss = 0\n model_path = \"./pretrained_models/1WV3_PNNplus_model.tar\"\n if os.path.isfile(model_path):\n print(\"loading model\")\n checkpoint = torch.load(model_path)\n # checkpoint = torch.load('./pretrained_models/' + sensor_model)\n print(checkpoint.keys())\n net = checkpoint['model']\n print(net.conv1.weight.data[0, 0, 0, 0])\n net.load_state_dict(checkpoint['model_state'])\n '''\n 4-.h5\n 1e-4\n 1-.mat\n 1e-4 \n '''\n lr_ = 1e-4#0.0001 * 17 * 17 * 8#1e-4#0.0001 * 17 * 17 * 8#1e-3#2e-4#0.0001 * 17 * 17 * 8\n FT_epochs = 50 # number of fine tuning epochs\n # init_loss = checkpoint[\"loss1\"]\n else:\n nr_bands = 8 # selected by user or taken from data?\n lr_ = 0.0001 * 17 * 17 * nr_bands\n FT_epochs = 5000\n net = APNN().cuda()\n net.apply(weights_init)\n print(net.conv1.weight.data[0, 0, 0, 0])\n print(net.conv1.weight.data[0, 0, 0, 0])\n print(net)\n\n\n\n test_gt = tester.test_gt - tester.test_I_in[:, :-1, blk:-blk, blk:-blk]\n pretrain_inIt_loss = criterion(net(tester.test_I_in.cuda()), test_gt.cuda())#, tester.test_I_in[:, :-1, blk:-blk, blk:-blk].cuda())\n print(\"init loss: {:.20f} pretrain_inIt loss: {:.20f}\".format(init_loss, pretrain_inIt_loss.item()))\n eval_test(net, evaluator, mode=\"eval\", mode2=\"pre\", err=pretrain_inIt_loss)\n # print(\"-\" * 30)\n \"scaling learning rate on last layer\"\n # print(dict(net.conv3.named_parameters()).keys())\n target_layerParam = list(map(id, net.conv3.parameters()))\n base_layerParam = filter(lambda p: id(p) not in target_layerParam, net.parameters())\n\n training_parameters = [{'params': net.conv3.parameters(), 'lr': lr_/10 },\n {'params': base_layerParam}]\n\n optimizer = optim.SGD(training_parameters, lr=lr_, momentum=0.9, weight_decay=0)\n try:\n optimizer.load_state_dict(checkpoint[\"optim_state\"])\n except:\n print(\"default optim_state\")\n print(net.conv3.weight.requires_grad)\n print(\"inspect optimizer setting:\\n\", optimizer.state_dict())\n print(\"target id:\", target_layerParam)\n\n device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n\n v_min = 10000\n ft_loss = np.zeros(FT_epochs)\n Train_time = time()\n print(tester.test_gt.shape)\n ## 2.1) \"FINE TUNING\"--training\n for epoch in range(FT_epochs): # loop over the testing image multiple times\n net.train()\n # running_loss = 0.0\n # loading testing image\n test_I_in = tester.test_I_in[:, :-1, blk:-blk, blk:-blk]\n test_I_in = test_I_in.cuda()\n test_gt = tester.test_gt.cuda()\n\n # residual\n\n # zero the parameter gradients\n optimizer.zero_grad()\n\n # forward + backward + optimize\n x1 = tester.test_I_in # send to cuda, important!\n x2 = test_gt - test_I_in\n x1 = x1.cuda().float() # convert to tensor type:\n x2 = x2.cuda().float() # convert to tensor type:\n\n outputs = net(x1)\n test_I_in.requires_grad = False\n loss = criterion(outputs, x2) # compute loss\n # loss = criterion(outputs+test_I_in, x2, test_I_in) # compute loss\n # loss = criterion(outputs + test_I_in, test_gt, 0) # compute loss\n new_loss = regularization(loss, net, flag=False)\n\n new_loss.backward()\n # try:\n # scheduler.step(epoch)\n # except:\n # print()\n optimizer.step()\n\n running_loss = loss.item()\n ft_loss[epoch] = running_loss\n\n if running_loss < v_min:\n PATH = './ft_network/'\n if not os.path.exists(PATH):\n os.makedirs(PATH)\n torch.save(dict(model=net,\n model_state=net.state_dict(),\n loss=ft_loss),\n PATH + '/wv_3_net.pth.tar')\n # if np.abs(running_loss - v_min) > 1e-3:\n net.eval()\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\", err=loss)\n v_min = running_loss\n print('[%d] loss: %.20f' % (epoch + 1, running_loss))\n print('-')\n Train_time = time() - Train_time\n\n ## 2.2) \"FINE TUNING\"--testing\n \" LOAD BEST MODEL\"\n checkpoint = torch.load('./ft_network/wv_3_net.pth.tar')\n net = checkpoint['model']\n net.load_state_dict(checkpoint['model_state'])\n print(\"-\" * 100)\n print(\"pretrain_InIt_loss {:.20f}\".format(pretrain_inIt_loss))\n for loss in checkpoint['loss']:\n if loss > 0:\n print(loss, \" \");\n # print(\"-\"*100)\n # print(checkpoint['loss'])\n \" PANSHARPENING \"\n\n \"testing phase requires input at actual testing resolution\"\n\n net.to(device)\n net.eval()\n print(net.conv3.weight.data[0, 0, 0, 0], net.conv2.weight.data[0, 0, 0, 0], net.conv1.weight.data[0, 0, 0, 0])\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\")\n\n\ndef eval_test(net, evaluator, mode=\"pre\", mode2=\"pre\", err=None):\n with torch.no_grad():\n Test_time = time()\n sr = evaluator(net, err=err) # NxCxHxW\n Test_time = time() - Test_time\n\n # skip connection to add low resolution ms and residual(np version)\n sr = sr.cpu().detach().numpy() + evaluator.test_I_in[:, :-1, blk:-blk, blk:-blk].cpu().detach().numpy() # NxCxHxW\n\n # convert to numpy type with permute and squeeze: HxWxC (go to cpu for easy saving)\n sr = torch.from_numpy(sr) # convert to tensor version\n sr = sr.permute(0, 2, 3, 1).cpu().detach().numpy() # to: NxHxWxC\n\n \"clipping is not necessary\"\n sr = np.clip(sr, 0, 1)\n\n num_exm = sr.shape[0]\n if mode == \"eval\":\n if num_exm == 1:\n\n if evaluator.mode == \"RR\":\n key = \"apnn_wv3_rs\"\n file_name = key + '_ny_' + mode2 + \".mat\"\n if evaluator.mode == \"FR\":\n key = \"apnn_wv3_os\"\n file_name = key + '_ny_' + mode2 + \".mat\"\n file_name2 = './results/'\n save_name = os.path.join(file_name2, file_name)\n sio.savemat(save_name, {key: sr[0, :, :, :]})\n\n else:\n for index in range(num_exm): # save the DL results to the 03-Comparisons(Matlab)\n file_name = \"apnn_wv3_rs\" + str(index) + mode2 + \".mat\"\n file_name2 = './results/'\n save_name = os.path.join(file_name2, file_name)\n sio.savemat(save_name, {'apnn_wv3_rs': sr[index, :, :, :]})\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == '__main__':\n\n import random\n import torch.backends.cudnn as cudnn\n\n file_path = \"./test_data/imgs/NY1_WV3_RR.mat\"\n\n \"SELECT SENSOR AND TESTING IMAGE\"\n sensor_model = 'WV3'\n available_models = ['IKONOS', 'GeoEye1', 'WV2', 'WV3', 'WV4', 'QB']\n if sensor_model in available_models:\n sensor_model = sensor_model + '_PNNplus_model.pth.tar'\n\n test(file_path, sensor_model)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/main_test_wv4.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.autograd import Variable\nfrom torch.utils.data import DataLoader\nfrom data_wv4 import Dataset_Pro\nimport h5py\nfrom data_single_read import load_set\nfrom evaluate import compute_index\nfrom model_wv4 import APNN, summaries, weights_init, loss_with_l2_regularization\nimport numpy as np\nimport scipy.io as sio\nfrom time import time\nfrom evaluate import analysis_accu\nimport math\n\nclass L1_Loss_clip(nn.Module):\n def __init__(self, multiple=2048.0, clip_flag=True, inter=False):\n super(L1_Loss_clip, self).__init__()\n self.clip_flag = clip_flag\n self.inter = inter\n self.multiple = multiple\n self.criterion = nn.L1Loss(reduction='mean')\n\n\n def forward(self, x, target, lms):\n if self.clip_flag:\n # x = torch.clamp(x * self.multiple, 0, 2048) / self.multiple\n x = x - lms\n if self.inter:\n loss = torch.mean((x - target) ** 2)#self.criterion(x, target)#torch.mean(torch.abs(x - target))\n return loss\n else:\n l1_loss = self.criterion(x, target)\n\n return l1_loss\n\n###################################################################\n# ------------------- Sub-Functions (will be used) -------------------\n###################################################################\ndef load_set(file_path, blk):\n\n suffix = file_path.split('.')\n if suffix[-1] == 'h5':\n ## ===== case1: NxCxHxW\n data = h5py.File(file_path)\n ms1 = data[\"ms\"][...] # NxCxHxW=0,1,2,3\n shape_size = len(ms1.shape)\n elif suffix[-1] == 'mat':\n # ===== case2: HxWxC\n data = sio.loadmat(file_path) #\n print(data.keys())\n ms1 = data[\"I_MS_LR\"][...] # NxCxHxW=0,1,2,3\n shape_size = len(ms1.shape)\n else:\n print(\"file format is not supported\")\n raise NotImplemented\n\n if suffix[-2][-2:] == 'FR':\n data['I_GT'] = data['I_MS_LR'] #exception\n\n if shape_size == 4: # NxCxHxW\n\n # tensor type:\n lms1 = data['lms'][...] # NxCxHxW = 4x8x512x512\n lms1 = np.array(lms1, dtype=np.float32) / 2047.0\n lms = torch.from_numpy(lms1)\n\n pan1 = data['pan'][...] # NxCxHxW = 4x8x512x512\n pan1 = np.array(pan1, dtype=np.float32) / 2047.0\n pan = torch.from_numpy(pan1)\n\n test_I_in1 = np.concatenate([lms1, pan1], axis=1) # NxCxHxW = Nx9xHxW\n test_I_in1 = np.pad(test_I_in1, ((0, 0), (0, 0), (blk, blk), (blk, blk)), mode='edge') # pading\n test_I_in = torch.from_numpy(test_I_in1) # NxCxHxW = Nx9xHxW\n\n ms1 = data['ms'][...] # NxCxHxW = 4x8x512x512\n ms1 = np.array(ms1, dtype=np.float32) / 2047.0\n ms = torch.from_numpy(ms1)\n\n gt1 = data['gt'][...] # NxCxHxW = 4x8x512x512\n gt1 = np.array(gt1, dtype=np.float32) / 2047.0\n gt = torch.from_numpy(gt1)\n\n return test_I_in, ms, pan, gt\n\n if shape_size == 3: # HxWxC\n\n # tensor type:\n lms1 = data['I_MS'][...] # HxWxC=0,1,2\n lms1 = np.expand_dims(lms1, axis=0) # 1xHxWxC\n lms1 = np.array(lms1, dtype=np.float32) / 2047.0 # 1xHxWxC\n lms = torch.from_numpy(lms1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n pan1 = data['I_PAN'][...] # HxW\n pan1 = np.expand_dims(pan1, axis=0) # 1xHxW\n pan1 = np.expand_dims(pan1, axis=3) # 1xHxWx1\n pan1 = np.array(pan1, dtype=np.float32) / 2047. # 1xHxWx1\n pan = torch.from_numpy(pan1).permute(0, 3, 1, 2) # Nx1xHxW:\n\n test_I_in1 = np.concatenate([lms1, pan1], axis=3) # 1xHxWx(C+1) = Nx9xHxW\n test_I_in1 = np.transpose(test_I_in1, (0, 3, 1, 2)) # 1x(C+1)xHxW\n test_I_in1 = np.pad(test_I_in1, ((0, 0), (0, 0), (blk, blk), (blk, blk)), mode='edge') # NCHW\n test_I_in = torch.from_numpy(test_I_in1) # NxCxHxW = Nx9xHxW\n\n ms1 = data['I_MS_LR'][...] # HxWxC=0,1,2\n ms1 = np.expand_dims(ms1, axis=0) # 1xHxWxC\n ms1 = np.array(ms1, dtype=np.float32) / 2047.0 # 1xHxWxC\n ms = torch.from_numpy(ms1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n gt1 = data['I_GT'][...] # HxWxC=0,1,2\n gt1 = np.expand_dims(gt1, axis=0) # 1xHxWxC\n gt1 = np.array(gt1, dtype=np.float32) / 2047.0 # 1xHxWxC\n gt = torch.from_numpy(gt1).permute(0, 3, 1, 2) # NxCxHxW or HxWxC\n\n return test_I_in, ms, pan, gt\n\n\n###################################################################\n# ------------------- Main Test (Run second) -------------------\n###################################################################\n\n## 1) initial test by model ##\nblk = 8#4\n\n\nclass Tester():\n def __init__(self, file_path, mode):\n test_I_in, test_ms, test_pan, test_gt = load_set(file_path, blk)\n self.test_I_in = test_I_in\n self.test_ms = test_ms\n self.test_pan = test_pan\n self.test_gt = test_gt\n self.mode = mode\n self.file_path = file_path\n self.loss_clip = L1_Loss_clip(inter=True, clip_flag=True).cuda()\n \"the fine tuning phase requires downgraded input resolution\"\n if mode == 'ft':\n from wald_utilities import wald_protocol\n ms_lr, pan_lr = wald_protocol(test_ms, test_pan, 4., 'WV4', channels=4)\n self.test_I_in = torch.cat((ms_lr, pan_lr), dim=1)\n self.test_I_in = torch.nn.functional.pad(self.test_I_in, (8, 8, 8, 8), mode='reflect') # NCHW\n self.test_gt = self.test_ms\n # ...\n def __call__(self, model):\n x = self.test_I_in # send to cuda, important!\n x = x.cuda().float() # convert to tensor type:\n out2 = model(x)\n # result_our = out2 + x[:, :-1, blk:-blk, blk:-blk]\n # out2 = model(self.test_I_in)\n # result_our = out2 + self.test_I_in[:, :-1, blk:-blk, blk:-blk]\n # result_our = torch.squeeze(result_our).permute(1, 2, 0)\n # result_our = result_our* 2047\n # our_SAM, our_ERGAS = compute_index(self.test_gt, result_our, 4)\n # print('our_SAM: {} our_ERGAS: {}'.format(our_SAM, our_ERGAS)) # print loss for each epoch\n\n if self.mode == 'RR' or self.mode == 'test':\n test_in = self.test_I_in[:, :-1, blk:-blk, blk:-blk].cuda()\n sr = out2 + test_in # self.test_I_in[:, :-1, blk:-blk, blk:-blk].cuda() # NxCxHxW\n sr = sr.permute(0, 2, 3, 1)\n gt = self.test_gt.permute(0, 2, 3, 1).cuda()\n\n # sr = out2.permute(0, 2, 3, 1)\n # gt = self.test_gt.cuda() - test_in # NxCxHxW\n # gt = gt.permute(0, 2, 3, 1)\n our_CC, our_PSNR, our_SSIM, our_SAM, our_ERGAS = analysis_accu(gt[0, ...], sr[0, ...], 4)\n print(f'[{self.file_path}]: our_CC: {our_CC}, our_PSNR: {our_PSNR}, '\n f'our_SSIM: {our_SSIM},\\n'\n f'our_SAM: {our_SAM} our_ERGAS: {our_ERGAS}')\n return out2\n\n\n## 2) target-adative's fine_tune_training, i.e., PNNplus##\n\ndef test(file_path, sensor_model):\n\n suffix = file_path.split('.')[-2][-2:]\n if suffix== 'RR' or suffix == 'FR':\n simulated = suffix\n else:\n simulated = 'test'\n\n tester = Tester(file_path, mode='ft') # call initial model\n evaluator = Tester(file_path, mode=simulated)\n criterion = nn.L1Loss(reduction='mean').cuda()#L1_Loss_clip(clip_flag=True, inter=True).cuda()#\n regularization = loss_with_l2_regularization().cuda()\n \" LOAD PRETRAINED MODEL\"\n init_loss = 0\n model_path = \"../pretrained_models/1WV4_PNNplus_model.pth.tar\"\n if os.path.isfile(model_path):\n print(\"loading model\")\n checkpoint = torch.load(model_path)\n # checkpoint = torch.load('./pretrained_models/' + sensor_model)\n print(checkpoint.keys())\n net = checkpoint['model']\n print(net.conv1.weight.data[0, 0, 0, 0])\n net.load_state_dict(checkpoint['model_state'])\n lr_ = 1e-4\n FT_epochs = 50 # number of fine tuning epochs\n # init_loss = checkpoint[\"loss1\"]\n\n else:\n nr_bands = 4 # selected by user or taken from data?\n lr_ = 0.0001 * 17 * 17 * nr_bands\n FT_epochs = 5000\n net = APNN().cuda()\n net.apply(weights_init)\n print(net.conv1.weight.data[0, 0, 0, 0])\n\n print(net.conv1.weight.data[0, 0, 0, 0])\n print(net)\n\n test_gt = tester.test_gt - tester.test_I_in[:, :-1, blk:-blk, blk:-blk]\n pretrain_inIt_loss = criterion(net(tester.test_I_in.cuda()), test_gt.cuda()).item()#, tester.test_I_in[:, :-1, blk:-blk, blk:-blk].cuda()).item()\n print(\"init loss: {:.20f} pretrain_inIt loss: {:.20f}\".format(init_loss, pretrain_inIt_loss))\n eval_test(net, evaluator, mode=\"eval\", mode2=\"pre\")\n print('-' * 100)\n \"scaling learning rate on last layer\"\n target_layerParam = list(map(id, net.conv3.parameters()))\n base_layerParam = filter(lambda p: id(p) not in target_layerParam, net.parameters())\n\n training_parameters = [{'params': net.conv3.parameters(), 'lr': lr_ / 10},\n {'params': base_layerParam}]\n\n optimizer = optim.SGD(training_parameters, lr=lr_, momentum=0.9)\n\n device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n v_min = 10000\n ft_loss = np.zeros(FT_epochs)\n Train_time = time()\n print(tester.test_gt.shape)\n ## 2.1) \"FINE TUNING\"--training\n for epoch in range(FT_epochs): # loop over the testing image multiple times\n net.train()\n # running_loss = 0.0\n # loading testing image\n test_I_in = tester.test_I_in[:, :-1, blk:-blk, blk:-blk]\n test_I_in = test_I_in.cuda()\n test_gt = tester.test_gt.cuda()\n\n # residual\n\n # zero the parameter gradients\n optimizer.zero_grad()\n\n # forward + backward + optimize\n x1 = tester.test_I_in # send to cuda, important!\n x2 = test_gt - test_I_in\n x1 = x1.cuda().float() # convert to tensor type:\n x2 = x2.cuda().float() # convert to tensor type:\n\n outputs = net(x1)\n test_I_in.requires_grad = False\n loss = criterion(outputs, x2) # compute loss\n # loss = criterion(outputs + test_I_in, x2, test_I_in) # compute loss\n # loss = criterion(outputs + test_I_in, test_gt, 0) # compute loss\n\n new_loss = regularization(loss, net, flag=False)\n\n new_loss.backward()\n optimizer.step()\n\n running_loss = loss.item()\n ft_loss[epoch] = running_loss\n\n\n if running_loss < v_min:\n PATH = '../ft_network/WV4'\n if not os.path.exists(PATH):\n os.makedirs(PATH)\n torch.save(dict(model=net,\n model_state=net.state_dict(),\n loss=ft_loss),\n PATH + '/net.pth.tar')\n # if np.abs(running_loss - v_min) > 1e-3:\n net.eval()\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\")\n v_min = running_loss\n\n print('[%d] loss: %.20f' % (epoch + 1, running_loss))\n\n Train_time = time() - Train_time\n\n ## 2.2) \"FINE TUNING\"--testing\n \" LOAD BEST MODEL\"\n checkpoint = torch.load('../ft_network/WV4/net.pth.tar')\n net = checkpoint['model']\n net.load_state_dict(checkpoint['model_state'])\n print(\"-\" * 100)\n print(\"pretrain_InIt_loss {:.20f}\".format(pretrain_inIt_loss))\n for loss in checkpoint['loss']:\n if loss > 0:\n print(loss, \" \");\n \" PANSHARPENING \"\n \"testing phase requires input at actual testing resolution\"\n\n net.to(device)\n net.eval()\n print(net.conv3.weight.data[0, 0, 0, 0], net.conv2.weight.data[0, 0, 0, 0], net.conv1.weight.data[0, 0, 0, 0])\n eval_test(net, evaluator, mode=\"eval\", mode2=\"ft\")\n\ndef eval_test(net, evaluator, mode=\"pre\", mode2=\"pre\"):\n with torch.no_grad():\n Test_time = time()\n sr = evaluator(net) # NxCxHxW\n Test_time = time() - Test_time\n\n # skip connection to add low resolution ms and residual(np version)\n sr = sr.cpu().detach().numpy() + evaluator.test_I_in[:, :-1, blk:-blk, blk:-blk].cpu().detach().numpy() # NxCxHxW\n\n # convert to numpy type with permute and squeeze: HxWxC (go to cpu for easy saving)\n sr = torch.from_numpy(sr) # convert to tensor version\n sr = sr.permute(0, 2, 3, 1).cpu().detach().numpy() # to: NxHxWxC\n sr = np.clip(sr, 0, 1)\n\n # print('------> [PNN+]: Fine-tuning (%d it) time = %0.4f // Prediction time = %0.4f' % (\n # FT_epochs, Train_time, Test_time))\n\n num_exm = sr.shape[0]\n\n if num_exm == 1:\n if evaluator.mode == \"RR\":\n key = 'apnn_wv4_rs_alice'\n file_name = \"apnn_wv4_rs\" + '_alice_' + mode2 + \".mat\"\n if evaluator.mode == \"FR\":\n key = 'apnn_wv4_os_alice'\n file_name = \"apnn_wv4_os\" + '_alice_' + mode2 + \".mat\"\n\n\n # file_name2 = \"E:/01-DL-Pansharpening-Toolbox/03-Comparisons(Matlab)/2_DL_Result/WV4/APNN\"\n file_name2 = \"../results\"\n save_name = os.path.join(file_name2, file_name)\n sio.savemat(save_name, {key: sr[0, :, :, :]})\n else:\n for index in range(num_exm): # save the DL results to the 03-Comparisons(Matlab)\n file_name = \"apnn_wv4_rs\" + str(index)+ mode2 + \".mat\"\n # file_name2 = \"E:/01-DL-Pansharpening-Toolbox/03-Comparisons(Matlab)/2_DL_Result/WV4/APNN\"\n file_name2 = \"../results\"\n save_name = os.path.join(file_name2, file_name)\n sio.savemat(save_name, {'apnn_wv4_rs': sr[index, :, :, :]})\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == '__main__':\n # file_path = \"../test_data/TestData_wv4.h5\"\n file_path = \"../test_data/imgs/Alice_WV4_RR.mat\"\n ## case2: test on single image with the size of HxWxC\n # file_path = \"E:/01-DL-Pansharpening-Toolbox/03-Comparisons(Matlab)/1_TestData/Datasets Testing/NY1_WV3_RR.mat\"\n\n # file_path = \"E:/01-DL-Pansharpening-Toolbox/03-Comparisons(Matlab)/1_TestData/Datasets Testing/NY1_WV3_FR.mat\"\n\n \"SELECT SENSOR AND TESTING IMAGE\"\n sensor_model = 'WV4'\n available_models = ['IKONOS', 'GeoEye1', 'WV2', 'WV3', 'WV4', 'QB']\n if sensor_model in available_models:\n sensor_model = sensor_model + '_PNNplus_model.pth.tar'\n\n test(file_path, sensor_model)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/model_qb.py", "content": "import torch\nimport torch.nn as nn\nimport math\n\nclass loss_with_l2_regularization(nn.Module):\n def __init__(self):\n super(loss_with_l2_regularization, self).__init__()\n\n def forward(self, criterion, model, weight_decay=1e-5, flag=True):\n regularizations = []\n for k, v in model.named_parameters():\n if 'conv' in k and 'weight' in k:\n # print(k)\n penality = weight_decay * ((v.data ** 2).sum() / 2)\n regularizations.append(penality)\n if flag:\n print(\"{} : {}\".format(k, penality))\n # r = torch.sum(regularizations)\n\n loss = criterion + sum(regularizations)\n return loss\n\n# def weights_init(m): # 1\n# classname = m.__class__.__name__ # 2\n# if classname.find('Conv') != -1: # 3\n# variance_scaling_initializer(m.weight.data)\n\n# -------------Initialization----------------------------------------\ndef init_weights(*modules):\n for module in modules:\n for m in module.modules():\n if isinstance(m, nn.Conv2d): ## initialization for Conv2d\n variance_scaling_initializer(m.weight) # method 1: initialization\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.BatchNorm2d): ## initialization for BN\n nn.init.constant_(m.weight, 1.0)\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.Linear): ## initialization for nn.Linear\n # variance_scaling_initializer(m.weight)\n nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu')\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n\n\nclass APNN(nn.Module):\n def __init__(self):\n super(APNN, self).__init__()\n\n channel = 48\n spectral_num = 4\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n\n '''\n C.Using deeper network\n Finally, during training, we stabilize the layers’\n inputs by means of batch normalization\n \n '''\n\n self.conv1 = nn.Conv2d(in_channels=spectral_num + 1, out_channels=channel, kernel_size=9, stride=1,\n bias=True)\n self.conv2 = nn.Conv2d(in_channels=channel, out_channels=32, kernel_size=5, stride=1,\n bias=True)\n self.conv3 = nn.Conv2d(in_channels=32, out_channels=spectral_num, kernel_size=5, stride=1,\n bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n init_weights(self.conv1, self.conv2, self.conv3)\n\n def forward(self, x): # x= lms; y = pan\n\n #input1 = torch.cat((x, y), 1) # Bsx9x64x64\n\n # input1 = self.bn(input1)\n rs = self.relu(self.conv1(x))\n rs = self.relu(self.conv2(rs))\n \n\n output = self.conv3(rs)\n\n return output\n\n\n# ----------------- End-Main-Part ------------------------------------\n# QB\ndef variance_scaling_initializer(tensor):\n from scipy.stats import truncnorm\n\n def truncated_normal_(tensor, mean=0, std=1):\n with torch.no_grad():\n size = tensor.shape\n tmp = tensor.new_empty(size + (4,)).normal_()\n valid = (tmp < 2) & (tmp > -2)\n ind = valid.max(-1, keepdim=True)[1]\n tensor.data.copy_(tmp.gather(-1, ind).squeeze(-1))\n tensor.data.mul_(std).add_(mean)\n return tensor\n\n def variance_scaling(x, scale=1.0, mode=\"fan_in\", distribution=\"truncated_normal\", seed=None):\n fan_in, fan_out = torch.nn.init._calculate_fan_in_and_fan_out(x)\n if mode == \"fan_in\":\n scale /= max(1., fan_in)\n elif mode == \"fan_out\":\n scale /= max(1., fan_out)\n else:\n scale /= max(1., (fan_in + fan_out) / 2.)\n if distribution == \"normal\" or distribution == \"truncated_normal\":\n # constant taken from scipy.stats.truncnorm.std(a=-2, b=2, loc=0., scale=1.)\n stddev = math.sqrt(scale) / .87962566103423978\n # print(fan_in,fan_out,scale,stddev)#100,100,0.01,0.1136\n truncated_normal_(x, 0.0, 0.001)\n return x/10*1.28\n\n variance_scaling(tensor)\n\n return tensor\n\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/model_wv2.py", "content": "import torch\nimport torch.nn as nn\nimport math\n\nclass loss_with_l2_regularization(nn.Module):\n def __init__(self):\n super(loss_with_l2_regularization, self).__init__()\n\n def forward(self, criterion, model, weight_decay=1e-5, flag=True):\n regularizations = []\n for k, v in model.named_parameters():\n if 'conv' in k and 'weight' in k:\n # print(k)\n penality = weight_decay * ((v.data ** 2).sum() / 2)\n regularizations.append(penality)\n if flag:\n print(\"{} : {}\".format(k, penality))\n # r = torch.sum(regularizations)\n\n loss = criterion + sum(regularizations)\n return loss\n\ndef weights_init(m): # 1\n classname = m.__class__.__name__ # 2\n if classname.find('Conv') != -1: # 3\n variance_scaling_initializer(m.weight.data)\n\n# -------------Initialization----------------------------------------\ndef init_weights(*modules):\n for module in modules:\n for m in module.modules():\n if isinstance(m, nn.Conv2d): ## initialization for Conv2d\n # try:\n # import tensorflow as tf\n # tensor = tf.get_variable(shape=m.weight.shape, initializer=tf.variance_scaling_initializer(seed=1))\n # m.weight.data = tensor.eval()\n # except:\n # print(\"try error, run variance_scaling_initializer\")\n # variance_scaling_initializer(m.weight)\n variance_scaling_initializer(m.weight) # method 1: initialization\n #nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu') # method 2: initialization\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.BatchNorm2d): ## initialization for BN\n nn.init.constant_(m.weight, 1.0)\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.Linear): ## initialization for nn.Linear\n # variance_scaling_initializer(m.weight)\n nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu')\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n\n\nclass APNN(nn.Module):\n def __init__(self):\n super(APNN, self).__init__()\n\n channel = 48\n spectral_num = 8\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n\n '''\n C.Using deeper network\n Finally, during training, we stabilize the layers’\n inputs by means of batch normalization\n \n '''\n\n self.conv1 = nn.Conv2d(in_channels=spectral_num + 1, out_channels=channel, kernel_size=9, stride=1,\n bias=True)\n self.conv2 = nn.Conv2d(in_channels=channel, out_channels=32, kernel_size=5, stride=1,\n bias=True)\n self.conv3 = nn.Conv2d(in_channels=32, out_channels=spectral_num, kernel_size=5, stride=1,\n bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n # init_weights(self.conv1, self.conv2, self.conv3)\n\n def forward(self, x): # x= lms; y = pan\n\n rs = self.relu(self.conv1(x))\n rs = self.relu(self.conv2(rs))\n \n\n output = self.conv3(rs)\n\n return output\n\n\n# ----------------- End-Main-Part ------------------------------------\ndef variance_scaling_initializer(tensor):\n from scipy.stats import truncnorm\n\n def truncated_normal_(tensor, mean=0, std=1):\n with torch.no_grad():\n size = tensor.shape\n tmp = tensor.new_empty(size + (4,)).normal_()\n valid = (tmp < 2) & (tmp > -2)\n ind = valid.max(-1, keepdim=True)[1]\n tensor.data.copy_(tmp.gather(-1, ind).squeeze(-1))\n tensor.data.mul_(std).add_(mean)\n return tensor\n\n def variance_scaling(x, scale=1.0, mode=\"fan_in\", distribution=\"truncated_normal\", seed=None):\n fan_in, fan_out = torch.nn.init._calculate_fan_in_and_fan_out(x)\n if mode == \"fan_in\":\n scale /= max(1., fan_in)\n elif mode == \"fan_out\":\n scale /= max(1., fan_out)\n else:\n scale /= max(1., (fan_in + fan_out) / 2.)\n if distribution == \"normal\" or distribution == \"truncated_normal\":\n # constant taken from scipy.stats.truncnorm.std(a=-2, b=2, loc=0., scale=1.)\n stddev = math.sqrt(scale) / .87962566103423978\n # print(fan_in,fan_out,scale,stddev)#100,100,0.01,0.1136\n truncated_normal_(x, 0.0, 0.001)\n return x/10*1.28\n\n variance_scaling(tensor)\n\n return tensor" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/model_wv3.py", "content": "import torch\nimport torch.nn as nn\nimport math\nfrom variance_sacling_initializer import variance_scaling_initializer\nclass loss_with_l2_regularization(nn.Module):\n def __init__(self):\n super(loss_with_l2_regularization, self).__init__()\n\n def forward(self, criterion, model, weight_decay=1e-5, flag=True):\n regularizations = []\n for k, v in model.named_parameters():\n if 'conv' in k and 'weight' in k:\n # print(k)\n penality = weight_decay * ((v.data ** 2).sum() / 2)\n regularizations.append(penality)\n if flag:\n print(\"{} : {}\".format(k, penality))\n # r = torch.sum(regularizations)\n\n loss = criterion + sum(regularizations)\n return loss\n\ndef weights_init(m): # 1\n classname = m.__class__.__name__ # 2\n if classname.find('Conv') != -1: # 3\n variance_scaling_initializer(m.weight.data)\n\n# netG.apply(weights_init) # 8\n\n\nclass APNN(nn.Module):\n def __init__(self):\n super(APNN, self).__init__()\n\n channel = 48\n spectral_num = 8\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n\n '''\n C.Using deeper network\n Finally, during training, we stabilize the layers’\n inputs by means of batch normalization\n \n '''\n\n self.conv1 = nn.Conv2d(in_channels=spectral_num + 1, out_channels=channel, kernel_size=9, stride=1,\n bias=True)\n self.conv2 = nn.Conv2d(in_channels=channel, out_channels=32, kernel_size=5, stride=1,\n bias=True)\n self.conv3 = nn.Conv2d(in_channels=32, out_channels=spectral_num, kernel_size=5, stride=1,\n bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n # init_weights(self.conv1, self.conv2, self.conv3)\n\n def forward(self, x): # x= lms; y = pan\n\n #input1 = torch.cat((x, y), 1) # Bsx9x64x64\n\n # input1 = self.bn(input1)\n rs = self.relu(self.conv1(x))\n rs = self.relu(self.conv2(rs))\n \n\n output = self.conv3(rs)\n\n return output\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/model_wv4.py", "content": "import torch\nimport torch.nn as nn\nimport math\nclass loss_with_l2_regularization(nn.Module):\n def __init__(self):\n super(loss_with_l2_regularization, self).__init__()\n\n def forward(self, criterion, model, weight_decay=1e-5, flag=True):\n regularizations = []\n for k, v in model.named_parameters():\n if 'conv' in k and 'weight' in k:\n # print(k)\n penality = weight_decay * ((v.data ** 2).sum() / 2)\n regularizations.append(penality)\n if flag:\n print(\"{} : {}\".format(k, penality))\n # r = torch.sum(regularizations)\n\n loss = criterion + sum(regularizations)\n return loss\n\n\nclass APNN(nn.Module):\n def __init__(self):\n super(APNN, self).__init__()\n\n channel = 48\n spectral_num = 4\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n\n '''\n C.Using deeper network\n Finally, during training, we stabilize the layers’\n inputs by means of batch normalization\n \n '''\n\n self.conv1 = nn.Conv2d(in_channels=spectral_num + 1, out_channels=channel, kernel_size=9, stride=1,\n bias=True)\n self.conv2 = nn.Conv2d(in_channels=channel, out_channels=32, kernel_size=5, stride=1,\n bias=True)\n self.conv3 = nn.Conv2d(in_channels=32, out_channels=spectral_num, kernel_size=5, stride=1,\n bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n # init_weights(self.conv1, self.conv2, self.conv3)\n\n def forward(self, x): # x= lms; y = pan\n\n rs = self.relu(self.conv1(x))\n rs = self.relu(self.conv2(rs))\n \n\n output = self.conv3(rs)\n\n return output\n\n\n# ----------------- End-Main-Part ------------------------------------\ndef variance_scaling_initializer(tensor):\n from scipy.stats import truncnorm\n\n def truncated_normal_(tensor, mean=0, std=1):\n with torch.no_grad():\n size = tensor.shape\n tmp = tensor.new_empty(size + (4,)).normal_()\n valid = (tmp < 2) & (tmp > -2)\n ind = valid.max(-1, keepdim=True)[1]\n tensor.data.copy_(tmp.gather(-1, ind).squeeze(-1))\n tensor.data.mul_(std).add_(mean)\n return tensor\n\n def variance_scaling(x, scale=1.0, mode=\"fan_in\", distribution=\"truncated_normal\", seed=None):\n fan_in, fan_out = torch.nn.init._calculate_fan_in_and_fan_out(x)\n if mode == \"fan_in\":\n scale /= max(1., fan_in)\n elif mode == \"fan_out\":\n scale /= max(1., fan_out)\n else:\n scale /= max(1., (fan_in + fan_out) / 2.)\n if distribution == \"normal\" or distribution == \"truncated_normal\":\n # constant taken from scipy.stats.truncnorm.std(a=-2, b=2, loc=0., scale=1.)\n stddev = math.sqrt(scale) / .87962566103423978\n # print(fan_in,fan_out,scale,stddev)#100,100,0.01,0.1136\n truncated_normal_(x, 0.0, 0.001)\n return x/10*1.28\n\n variance_scaling(tensor)\n\n return tensor\n\n\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/variance_sacling_initializer.py", "content": "import torch\nimport torch.nn as nn\nimport math\n\n\ndef truncated_normal_(tensor, mean=0.0, std=1.0):\n with torch.no_grad():\n size = tensor.shape\n tmp = tensor.new_empty(size + (4,)).normal_()\n valid = (tmp < 2) & (tmp > -2)\n ind = valid.max(-1, keepdim=True)[1]\n tensor.data.copy_(tmp.gather(-1, ind).squeeze(-1))\n tensor.data.mul_(std).add_(mean)\n return tensor\n\n\ndef variance_scaling_initializer(tensor):\n from scipy.stats import truncnorm\n def calculate_fan(shape, factor=2.0, mode='FAN_IN', uniform=False):\n # 64 9 3 3 -> 3 3 9 64\n # 64 64 3 3 -> 3 3 64 64\n if shape:\n # fan_in = float(shape[1]) if len(shape) > 1 else float(shape[0])\n # fan_out = float(shape[0])\n fan_in = float(shape[-2]) if len(shape) > 1 else float(shape[-1])\n fan_out = float(shape[-1])\n else:\n fan_in = 1.0\n fan_out = 1.0\n for dim in shape[:-2]:\n fan_in *= float(dim)\n fan_out *= float(dim)\n if mode == 'FAN_IN':\n # Count only number of input connections.\n n = fan_in\n elif mode == 'FAN_OUT':\n # Count only number of output connections.\n n = fan_out\n elif mode == 'FAN_AVG':\n # Average number of inputs and output connections.\n n = (fan_in + fan_out) / 2.0\n if uniform:\n raise NotImplemented\n # # To get stddev = math.sqrt(factor / n) need to adjust for uniform.\n # limit = math.sqrt(3.0 * factor / n)\n # return random_ops.random_uniform(shape, -limit, limit,\n # dtype, seed=seed)\n else:\n # To get stddev = math.sqrt(factor / n) need to adjust for truncated.\n trunc_stddev = math.sqrt(1.3 * factor / n)\n return fan_in, fan_out, trunc_stddev\n\n def variance_scaling(x, scale=1.0, mode=\"fan_in\", distribution=\"truncated_normal\", seed=None):\n # fan_in, fan_out = torch.nn.init._calculate_fan_in_and_fan_out(x)\n x = x.permute(3, 2, 1, 0) # .permute(2, 3, 1, 0)\n fan_in, fan_out, trunc_stddev = calculate_fan(x.shape)\n print(trunc_stddev)\n # if mode == \"fan_in\":\n # scale /= max(1., fan_in)\n # elif mode == \"fan_out\":\n # scale /= max(1., fan_out)\n # else:\n # scale /= max(1., (fan_in + fan_out) / 2.)\n # if distribution == \"normal\" or distribution == \"truncated_normal\":\n # # constant taken from scipy.stats.truncnorm.std(a=-2, b=2, loc=0., scale=1.)\n # stddev = math.sqrt(scale) / .87962566103423978\n # print(fan_in,fan_out,scale,stddev)#100,100,0.01,0.1136\n truncated_normal_(x, 0.0, trunc_stddev) # 0.001)\n x = x.permute(3, 2, 0, 1)\n print(x.min(), x.max())\n return x # /10*1.28\n\n variance_scaling(tensor)\n\n return tensor\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/APNN/wald_utilities.py", "content": "import numpy as np\nimport torch\nimport torch.nn as nn\nimport math\nimport scipy.ndimage.filters as ft\n\ndef fspecial_gauss(size, sigma):\n # Function to mimic the 'fspecial' gaussian MATLAB function\n m, n = [(ss-1.)/2. for ss in size]\n y, x = np.ogrid[-m:m+1, -n:n+1]\n h = np.exp( -(x*x + y*y) / (2.*sigma*sigma) )\n h[ h < np.finfo(h.dtype).eps*h.max() ] = 0\n sumh = h.sum()\n if sumh != 0:\n h /= sumh\n #h = np.round(h, 4)\n return h\ndef fir_filter_wind(Hd, w):\n \"\"\"\n compute fir filter with window method\n Hd: desired freqeuncy response (2D)\n w: window (2D)\n \"\"\"\n hd = np.rot90(np.fft.fftshift(np.rot90(Hd, 2)), 2)\n h = np.fft.fftshift(np.fft.ifft2(hd))\n h = np.rot90(h, 2)\n h = h * w\n h = np.clip(h, a_min=0, a_max=np.max(h))\n h = h / np.sum(h)\n return h\n\ndef NyquistFilterGenerator(Gnyq, ratio, N):\n assert isinstance(Gnyq, (np.ndarray, list)), 'Error: GNyq must be a list or a ndarray'\n if isinstance(Gnyq, list):\n Gnyq = np.asarray(Gnyq)\n nbands = Gnyq.shape[0]\n\n kernel = np.zeros((N, N, nbands)) # generic kerenel (for normalization purpose)\n fcut = 1 / np.double(ratio)\n for j in range(nbands):\n alpha = np.sqrt(((N - 1) * (fcut / 2)) ** 2 / (-2 * np.log(Gnyq[j])))\n H = fspecial_gauss((N,N), alpha)\n Hd = H / np.max(H)\n h = np.kaiser(N, 0.5)\n kernel[:, :, j] = np.real(fir_filter_wind(Hd, h))\n #kernel = np.round(kernel, 4)\n return kernel\n\ndef MTF(ratio, sensor, N=41):\n if (sensor=='QB'):\n GNyq = np.asarray([0.34, 0.32, 0.30, 0.22]) #Bands Order: B,G,R,NIR\n elif ((sensor=='Ikonos') or (sensor=='IKONOS')):\n GNyq = np.asarray([0.26, 0.28, 0.29, 0.28]) #Bands Order: B,G,R,NIR\n elif (sensor=='GeoEye1') or (sensor == 'WV4'):\n GNyq = np.asarray([0.23, 0.23, 0.23, 0.23]) #Bands Order: B, G, R, NIR\n elif (sensor=='WV2'):\n GNyq = 0.35 * np.ones((1, 7)); GNyq = np.append(GNyq, 0.27)\n elif (sensor=='WV3'):\n GNyq = [0.325, 0.355, 0.360, 0.350, 0.365, 0.360, 0.335, 0.315]\n\n\n\n h = NyquistFilterGenerator(GNyq,ratio, N)\n return h\n\n\ndef MTF_PAN(ratio, sensor, N=41):\n if (sensor=='QB'):\n GNyq = np.array([0.15])\n elif ((sensor=='Ikonos') or (sensor=='IKONOS')):\n GNyq = np.array([0.17])\n elif (sensor=='GeoEye1') or (sensor == 'WV4'):\n GNyq = np.array([0.16])\n elif (sensor=='WV2'):\n GNyq = np.array([0.11])\n elif (sensor=='WV3'):\n GNyq = np.array([0.14])\n else:\n GNyq = np.array([0.15])\n return NyquistFilterGenerator(GNyq, ratio, N)\n\n\ndef interp23tap(img, ratio):\n\n assert((2**(round(math.log(ratio, 2)))) == ratio), 'Error: Only resize factors power of 2'\n\n r,c,b = img.shape\n\n CDF23 = np.asarray([0.5, 0.305334091185, 0, -0.072698593239, 0, 0.021809577942, 0, -0.005192756653, 0, 0.000807762146, 0, -0.000060081482])\n CDF23 = [element * 2 for element in CDF23]\n BaseCoeff = np.expand_dims(np.concatenate([np.flip(CDF23[1:]), CDF23]), axis=-1)\n\n\n for z in range(int(ratio/2)):\n\n I1LRU = np.zeros(((2 ** (z+1)) * r, (2 ** (z+1)) * c, b))\n\n if z == 0:\n I1LRU[1::2, 1::2,:] = img\n else:\n I1LRU [::2,::2,:] = img\n\n for i in range(b):\n temp = ft.convolve(np.transpose(I1LRU[:,:,i]), BaseCoeff, mode='wrap')\n I1LRU[:, :, i] = ft.convolve(np.transpose(temp), BaseCoeff, mode='wrap')\n\n img = I1LRU\n\n return img\n\ndef interp23tap_GPU(img, ratio):\n\n assert((2**(round(math.log(ratio, 2)))) == ratio), 'Error: Only resize factors power of 2'\n\n r,c,b = img.shape\n\n CDF23 = np.asarray([0.5, 0.305334091185, 0, -0.072698593239, 0, 0.021809577942, 0, -0.005192756653, 0, 0.000807762146, 0, -0.000060081482])\n CDF23 = [element * 2 for element in CDF23]\n BaseCoeff = np.expand_dims(np.concatenate([np.flip(CDF23[1:]), CDF23]), axis=-1)\n BaseCoeff = np.expand_dims(BaseCoeff, axis=(0,1))\n BaseCoeff = np.concatenate([BaseCoeff]*b, axis=0)\n\n\n BaseCoeff = torch.from_numpy(BaseCoeff)\n img = img.astype(np.float32)\n img = np.moveaxis(img, -1, 0)\n\n\n for z in range(int(ratio/2)):\n\n I1LRU = np.zeros((b, (2 ** (z+1)) * r, (2 ** (z+1)) * c))\n\n if z == 0:\n I1LRU[:,1::2, 1::2] = img\n else:\n I1LRU [:,::2,::2] = img\n\n I1LRU = np.expand_dims(I1LRU, axis=0)\n conv = nn.Conv2d(in_channels=b, out_channels=b, padding=(11,0),\n kernel_size=BaseCoeff.shape, groups=b, bias=False, padding_mode='circular')\n\n conv.weight.data = BaseCoeff\n conv.weight.requires_grad = False\n\n t = conv(torch.transpose(torch.from_numpy(I1LRU), 2, 3))\n img = conv(torch.transpose(t, 2,3)).numpy()\n img = np.squeeze(img)\n\n img = np.moveaxis(img, 0,-1)\n\n\n return img\n\ndef wald_protocol(ms,pan,ratio,sensor, channels=8):\n \n mtf_kernel = MTF(ratio, sensor)\n\n\n MTF_kern = np.moveaxis(mtf_kernel, -1, 0)\n MTF_kern = np.expand_dims(MTF_kern, axis = 1)\n MTF_kern = torch.from_numpy(MTF_kern).type(torch.float32)\n\n # DepthWise-Conv2d definition\n depthconv = nn.Conv2d(in_channels=channels,\n out_channels=channels,\n kernel_size=MTF_kern.shape,\n groups=channels,\n padding=20,\n padding_mode='replicate',\n bias=False)\n \n depthconv.weight.data = MTF_kern\n depthconv.weight.requires_grad = False\n\n ms_down = depthconv(ms)\n ms_wald_ = nn.functional.interpolate(ms_down, scale_factor=0.25, mode='bicubic')\n ms_lr = torch.zeros(ms.shape)\n for i in range(ms_wald_.shape[0]):\n temp = np.copy(np.asarray(torch.squeeze(torch.squeeze(ms_wald_[i,:,:,:]).permute((1,2,0))).detach().cpu()))\n ms_lr[i, :, :, :]= torch.from_numpy(interp23tap_GPU(temp,ratio)).permute((2,0,1))\n pan_lr = nn.functional.interpolate(pan, scale_factor=0.25, mode='bicubic')\n\n \n\n return ms_lr, pan_lr\n\n\ndef wald_protocol_v2(ms, pan, ratio, sensor, channels=8):\n\n def genMTF_MS():\n mtf_kernel = MTF(ratio, sensor)\n\n MTF_kern = np.moveaxis(mtf_kernel, -1, 0)\n MTF_kern = np.expand_dims(MTF_kern, axis=1)\n MTF_kern = torch.from_numpy(MTF_kern).type(torch.float32)\n\n # DepthWise-Conv2d definition\n depthconv = nn.Conv2d(in_channels=channels,\n out_channels=channels,\n kernel_size=MTF_kern.shape,\n groups=channels,\n padding=20,\n padding_mode='replicate',\n bias=False)\n\n depthconv.weight.data = MTF_kern\n depthconv.weight.requires_grad = False\n\n ms_down = depthconv(ms)\n ms_wald_ = nn.functional.interpolate(ms_down, scale_factor=0.25, mode='bicubic')\n ms_lr = torch.zeros(ms.shape)\n for i in range(ms_wald_.shape[0]):\n temp = np.copy(np.asarray(torch.squeeze(torch.squeeze(ms_wald_[i, :, :, :]).permute((1, 2, 0))).detach().cpu()))\n ms_lr[i, :, :, :] = torch.from_numpy(interp23tap_GPU(temp, ratio)).permute((2, 0, 1))\n return ms_lr\n\n def genMTF_PAN():\n channels = 1\n mtf_kernel = MTF_PAN(ratio, sensor)\n\n MTF_kern = np.moveaxis(mtf_kernel, -1, 0)\n MTF_kern = np.expand_dims(MTF_kern, axis=1)\n MTF_kern = torch.from_numpy(MTF_kern).type(torch.float32)\n\n # DepthWise-Conv2d definition\n depthconv = nn.Conv2d(in_channels=channels,\n out_channels=channels,\n kernel_size=MTF_kern.shape,\n groups=channels,\n padding=20,\n padding_mode='replicate',\n bias=False)\n\n depthconv.weight.data = MTF_kern\n depthconv.weight.requires_grad = False\n\n pan_down = depthconv(pan)\n pan_lr = nn.functional.interpolate(pan_down, scale_factor=0.25, mode='bicubic')\n\n return pan_lr\n\n return genMTF_PAN()#ms_lr, pan_lr" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/BDPN/bdpn_main.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, Ran Ran, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom .model_bdpn import BDPN\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n\n def __init__(self, losses, weight_dict):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: n able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relatiumber of object categories, omitting the special no-object category\n matcher: moduleve classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.weight_dict = weight_dict\n self.losses = losses\n self.loss_dicts = {}\n\n def forward(self, outputs, targets, *args, **kwargs):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n # Compute all the requested losses\n\n for k in self.losses.keys():\n # k, loss = loss_dict\n if k == 'Loss':\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets))\n else:\n self.loss_dicts.update({k: loss(outputs, targets)})\n else:\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets, *args)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets, *args))\n else:\n self.loss_dicts.update({k: loss(outputs, targets, *args)})\n\n return self.loss_dicts\n\nfrom UDL.pansharpening.models import PanSharpeningModel\nclass build_bdpn(PanSharpeningModel, name='BDPN'):\n def __call__(self, cfg):\n\n # important for Pansharpening models, which are from tensorflow code\n self.reg = cfg.reg\n\n scheduler = None\n\n if any([\"wv\" in v for v in cfg.dataset.values()]):\n spectral_num = 8\n else:\n spectral_num = 4\n\n loss = nn.MSELoss(size_average=True).cuda() ## Define the Loss function\n weight_dict = {'loss': 1}\n losses = {'loss': loss}\n criterion = SetCriterion(losses, weight_dict)\n model = BDPN(spectral_num, criterion).cuda()\n optimizer = optim.Adam(model.parameters(), lr=cfg.lr, betas=(0.9, 0.999), weight_decay=1e-5) ## optimizer 1: Adam\n scheduler = torch.optim.lr_scheduler.StepLR(optimizer=optimizer, step_size=100,\n gamma=0.8) # lr = lr* gamma for each step_size = 180\n\n return model, criterion, optimizer, scheduler\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/BDPN/loss_utils.py", "content": "import torch\nimport math\nimport numpy as np\nimport torch.nn.functional as F\nfrom pytorch_msssim import ssim, ms_ssim, SSIM, MS_SSIM\n\n# X: (N,3,H,W) a batch of RGB images with values ranging from 0 to 255.\n# Y: (N,3,H,W) ssim_val=ssim(X,Y,data_range=255,size_average=False)\n# return (N,) ms_ssim_val=ms_ssim(X,Y,data_range=255,size_average=False)\n# #(N,)# or set 'size_average=True' to get a scalar value as loss.ssim_loss=ssim(X,Y,data_range=255,size_average=True)\n# return a scalar valuems_ssim_loss=ms_ssim(X,Y,data_range=255,size_average=True)\n# or reuse windows with SSIM & MS_SSIM. ssim_module=SSIM(win_size=11,win_sigma=1.5,data_range=255,size_average=True,channel=3)\n# ms_ssim_module=MS_SSIM(win_size=11,win_sigma=1.5,data_range=255,size_average=True,channel=3)\n# ssim_loss=ssim_module(X,Y)ms_ssim_loss=ms_ssim_module(X,Y)\n\n\n\n# def compute_charbonnier_loss(tensor1, tensor2, is_mean=True):\n# epsilon = 1e-6\n# if is_mean:\n# loss = tf.reduce_mean(tf.reduce_mean(tf.sqrt(tf.square(tf.subtract(tensor1,tensor2))+epsilon), [1, 2, 3]))\n# else:\n# loss = tf.reduce_mean(tf.reduce_sum(tf.sqrt(tf.square(tf.subtract(tensor1,tensor2))+epsilon), [1, 2, 3]))\n#\n# return loss\n\ndef compute_charbonnier_loss(tensor1, tensor2, is_mean=True):\n epsilon = 1e-6\n if is_mean:\n loss = torch.mean(torch.mean(torch.sqrt(torch.square(torch.sub(tensor1, tensor2))+epsilon), [2, 3, 1]))\n else:\n loss = torch.mean(torch.sum(torch.sqrt(torch.square(torch.sub(tensor1, tensor2))+epsilon), [2, 3, 1]))\n return loss\n\n\n# def compute_ergas_loss(tensor1, tensor2):\n# epsilon = 1e-8\n# rmse = tf.sqrt(tf.reduce_mean(tf.square(tf.subtract(tensor1,tensor2)),[1,2])+epsilon)\n# mean = tf.reduce_mean(tensor2, [1, 2])\n# mean = tf.exp(mean)\n# loss = tf.sqrt(tf.reduce_mean(tf.square(tf.divide(rmse,mean)))+epsilon)\n# return loss\n\ndef compute_ergas_loss(tensor1, tensor2):\n epsilon = 1e-8\n rmse = torch.sqrt(torch.mean(torch.square(torch.subtract(tensor1, tensor2)), [2, 3])+epsilon)\n mean = torch.mean(tensor2, [2, 3])\n mean = torch.exp(mean)\n loss = torch.sqrt(torch.mean(torch.square(torch.divide(rmse, mean)))+epsilon)\n return loss\n\n# def compute_spetral_shift_loss(tensor1, tensor2):\n# epsilon = 1e-8\n# size = (int(int(tensor1.get_shape()[1])/4), int(int(tensor1.get_shape()[2])/4))\n# tensor_lr1 = tf.image.resize_images(tensor1, size)\n# tensor_lr2 = tf.image.resize_images(tensor2, size)\n# loss = compute_ergas_loss(tensor_lr1, tensor_lr2)\n# #tf.reduce_mean(tf.sqrt(tf.reduce_mean(tf.square(tf.subtract(tensor_lr1,tensor_lr2)),[1,2])+epsilon))\n# return loss\n\ndef compute_spetral_shift_loss(tensor1, tensor2):\n epsilon = 1e-8\n size = (int(int(tensor1.get_shape()[2])/4), int(int(tensor1.get_shape()[3])/4))\n tensor_lr1 = F.interpolate(tensor1, size)\n tensor_lr2 = F.interpolate(tensor2, size)\n loss = compute_ergas_loss(tensor_lr1, tensor_lr2)\n #tf.reduce_mean(tf.sqrt(tf.reduce_mean(tf.square(tf.subtract(tensor_lr1,tensor_lr2)),[1,2])+epsilon))\n return loss\n\n# def compute_ssim_loss(tensor1, tensor2):\n# ssim = tf.image.ssim_multiscale(tensor1, tensor2, np.float32(2.0))\n# loss = 1 - tf.reduce_mean(ssim)\n# return loss\n\ndef compute_ssim_loss(tensor1, tensor2, channel = 8):\n ssim = MS_SSIM(win_size=11, win_sigma=1.5, data_range=1, size_average=True, channel=channel)\n loss = 1 - torch.mean(ssim)\n return loss\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/BDPN/main_train_wv3.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, Ran Ran, LiangJian Deng\n# @reference:\nimport os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom torch.utils.data import DataLoader\nfrom data_wv3 import Dataset_Pro\nfrom model_wv3 import BDPN\nfrom torchstat import stat\nimport numpy as np\nfrom tensorboardX import SummaryWriter\nimport shutil\nfrom loss_utils import compute_charbonnier_loss, compute_ergas_loss\n\nos.environ['CUDA_VISIBLE_DEVICES'] = '0'\n\n###################################################################\n# ------------------- Pre-Define Part----------------------\n###################################################################\n# ============= 1) Pre-Define =================== #\nSEED = 10\ntorch.manual_seed(SEED)\ntorch.cuda.manual_seed(SEED)\ntorch.cuda.manual_seed_all(SEED)\n# cudnn.benchmark = True ###自动寻找最优算法\ncudnn.deterministic = True\n\n# ============= 2) HYPER PARAMS(Pre-Defined) ==========#\nlr = 0.0001\nepochs = 1000\nckpt = 50\nbatch_size = 8\nlambda_v = 1.0\nlambda_init = 0.05\nlambda_declay = 5\nmodel_path = \"Weights/wv3/.pth\"\n\n# ============= 3) Load Model + Loss + Optimizer + Learn_rate_update ==========#\nmodel = BDPN().cuda()\nif os.path.isfile(model_path):\n model.load_state_dict(torch.load(model_path)) ## Load the pretrained Encoder\n print('PANnet is Successfully Loaded from %s' % (model_path))\n\nstat(model, input_size=[(8, 16, 16), (1, 64, 64)])\n#criterion = nn.MSELoss(size_average=True).cuda()\n\n#optimizer = optim.SGD(model.parameters(), lr=lr, momentum=0.9, weight_decay=1e-7) # optimizer 2\noptimizer = optim.Adam(model.parameters(), lr=lr, betas=(0.9, 0.999), weight_decay=1e-5) # optimizer 1\nlr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer=optimizer, step_size=100,\n gamma=0.8) # lr = lr* gamma for every step_size(epochs) = 180\n\n# ============= 4) Tensorboard_show + Save_model ==========#\n#if os.path.exists('train_logs'): # for tensorboard: copy dir of train_logs ## Tensorboard_show: case 1\n# shutil.rmtree('train_logs') # ---> console (see tensorboard): tensorboard --logdir = dir of train_logs\n\nwriter = SummaryWriter('./train_logs') ## Tensorboard_show: case 2\n\ndef save_checkpoint(model, epoch): # save model function\n model_out_path = 'Weights' + '/' + \"{}.pth\".format(epoch)\n torch.save(model.state_dict(), model_out_path)\n\n###################################################################\n# ------------------- Main Train (Run second)----------------------\n###################################################################\ndef train(training_data_loader, validate_data_loader,start_epoch=0):\n global lambda_v\n print('Start training...')\n\n for epoch in range(start_epoch, epochs, 1):\n\n epoch += 1\n epoch_train_loss, epoch_val_loss = [], []\n\n if epoch <= 100:\n lambda_v = 1.0 - lambda_init*(epoch//lambda_declay) # decrease lambda_v for every lambda_declay epochs\n\n # ============Epoch Train=============== #\n model.train()\n\n for iteration, batch in enumerate(training_data_loader, 1):\n gt, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda()\n\n optimizer.zero_grad() # fixed\n\n sr, sr_down = model(ms, pan) # call model: sr=4x64x64; sr_down=4x32x32\n\n gt_down = F.interpolate(gt, scale_factor=0.5, mode='nearest') # nearest down 2\n loss1 = compute_charbonnier_loss(sr_down, gt_down) # compute loss1; orig: loss = criterion(sr, gt)\n loss2 = compute_charbonnier_loss(sr, gt) # compute loss2\n\n loss = lambda_v*loss1 + (1.0 - lambda_v)*loss2 # total loss:\n epoch_train_loss.append(loss.item()) # save all losses into a vector for one epoch\n\n loss.backward() # fixed\n optimizer.step() # fixed\n\n # for name, layer in model.named_parameters():\n # writer.add_histogram('torch/'+name + '_grad_weight_decay', layer.grad, epoch*iteration)\n # writer.add_histogram('net/'+name + '_data_weight_decay', layer, epoch*iteration)\n\n lr_scheduler.step() # if update_lr, activate here!\n\n t_loss = np.nanmean(np.array(epoch_train_loss)) # compute the mean value of all losses, as one epoch loss\n writer.add_scalar('mse_loss/t_loss', t_loss, epoch) # write to tensorboard to check\n print('Epoch: {}/{} training loss (lr={}, lam_v={}): {:.7f}'.format(epochs, epoch, lr_scheduler.get_last_lr(), lambda_v, t_loss)) # print loss for each epoch\n\n if epoch % ckpt == 0: # if each ckpt epochs, then start to save model\n save_checkpoint(model, epoch)\n\n # ============Epoch Validate=============== #\n model.eval()\n with torch.no_grad():\n for iteration, batch in enumerate(validate_data_loader, 1):\n gt, ms, pan = batch[0].cuda(), batch[1].cuda(), batch[2].cuda()\n\n sr, sr_down = model(ms, pan) # call model\n\n gt_down = F.interpolate(gt, scale_factor=0.5, mode='nearest') # nearest down 2\n loss1 = compute_charbonnier_loss(sr_down, gt_down) # compute loss1; orig: loss = criterion(sr, gt)\n loss2 = compute_charbonnier_loss(sr, gt) # compute loss2\n\n loss = lambda_v * loss1 + (1.0 - lambda_v) * loss2\n\n epoch_val_loss.append(loss.item())\n\n if epoch % 10 == 0:\n v_loss = np.nanmean(np.array(epoch_val_loss))\n writer.add_scalar('val/v_loss', v_loss, epoch)\n print(' validate loss: {:.7f}'.format(v_loss))\n\n writer.close() # close tensorboard\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\nif __name__ == \"__main__\":\n train_set = Dataset_Pro('./training_data/train_wv3_10000.h5') # creat data for training\n training_data_loader = DataLoader(dataset=train_set, num_workers=0, batch_size=batch_size, shuffle=True,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n validate_set = Dataset_Pro('./training_data/valid_wv3_10000.h5') # creat data for validation\n validate_data_loader = DataLoader(dataset=validate_set, num_workers=0, batch_size=batch_size, shuffle=False,\n pin_memory=True, drop_last=True) # put training data to DataLoader for batches\n\n train(training_data_loader, validate_data_loader) # call train function (call: Line 53)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/BDPN/model_bdpn.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, Ran Ran, LiangJian Deng\n# @reference:\n\nimport torch\nimport torch.nn as nn\nimport numpy as np\nimport math\nimport torch.nn.init as int\nimport sys\n\n# print(sys.path)\nimport torch\nimport torch.nn as nn\nimport math\nfrom UDL.Basis.variance_sacling_initializer import variance_scaling_initializer\n\n# -------------Initialization----------------------------------------\ndef init_weights(*modules):\n for module in modules:\n for m in module.modules():\n if isinstance(m, nn.Conv2d): ## initialization for Conv2d\n # try:\n # import tensorflow as tf\n # tensor = tf.get_variable(shape=m.weight.shape, initializer=tf.variance_scaling_initializer(seed=1))\n # m.weight.data = tensor.eval()\n # except:\n # print(\"try error, run variance_scaling_initializer\")\n # variance_scaling_initializer(m.weight)\n variance_scaling_initializer(m.weight) # method 1: initialization\n #nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu') # method 2: initialization\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.BatchNorm2d): ## initialization for BN\n nn.init.constant_(m.weight, 1.0)\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.Linear): ## initialization for nn.Linear\n # variance_scaling_initializer(m.weight)\n nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu')\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n\n# ----------------------------------------------------\nclass Resblock(nn.Module):\n def __init__(self):\n super(Resblock, self).__init__()\n\n channel = 64\n self.conv20 = nn.Conv2d(in_channels=channel, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.conv21 = nn.Conv2d(in_channels=channel, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.prelu = nn.PReLU(num_parameters = 1, init = 0.2)\n\n def forward(self, x):\n rs1 = self.prelu(self.conv20(x)) # Bsx32x64x64\n rs1 = self.conv21(rs1) # Bsx32x64x64\n rs = torch.add(x, rs1) # Bsx32x64x64\n\n return rs\n\n# -----------------------------------------------------\nclass BDPN(nn.Module):\n def __init__(self, spectral_num, criterion, channel=64):\n super(BDPN, self).__init__()\n\n channel1 = channel\n spectral_num = spectral_num\n channel2 = 4*spectral_num\n self.criterion = criterion\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n # Conv2d: padding = kernel_size//2\n self.conv1 = nn.Conv2d(in_channels=1, out_channels=channel1, kernel_size=3, stride=1, padding=1,\n bias=True)\n #self.conv2 = nn.Conv2d(in_channels=channel1, out_channels=channel1, kernel_size=3, stride=1, padding=1,\n # bias=True)\n self.res1 = Resblock()\n self.res2 = Resblock()\n self.res3 = Resblock()\n self.res4 = Resblock()\n self.res5 = Resblock()\n self.res6 = Resblock()\n self.res7 = Resblock()\n self.res8 = Resblock()\n self.res9 = Resblock()\n self.res10 = Resblock()\n\n\n self.rres1 = Resblock()\n self.rres2 = Resblock()\n self.rres3 = Resblock()\n self.rres4 = Resblock()\n self.rres5 = Resblock()\n self.rres6 = Resblock()\n self.rres7 = Resblock()\n self.rres8 = Resblock()\n self.rres9 = Resblock()\n self.rres10 = Resblock()\n\n\n self.conv3 = nn.Conv2d(in_channels=channel1, out_channels=spectral_num, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.conv4 = nn.Conv2d(in_channels=spectral_num, out_channels=channel2, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.conv5 = nn.Conv2d(in_channels=spectral_num, out_channels=channel2, kernel_size=3, stride=1, padding=1,\n bias=True)\n\n self.maxpool = nn.MaxPool2d(kernel_size=2, stride=2)\n self.pixshuf = nn.PixelShuffle(upscale_factor=2) # out = ps(img)\n self.prelu = nn.PReLU(num_parameters = 1, init = 0.2)\n\n\n self.backbone = nn.Sequential( # method 2: 4 resnet repeated blocks\n self.res1,\n self.res2,\n self.res3,\n self.res4,\n self.res5,\n self.res6,\n self.res7,\n self.res8,\n self.res9,\n self.res10\n )\n\n self.backbone2 = nn.Sequential( # method 2: 4 resnet repeated blocks\n self.rres1,\n self.rres2,\n self.rres3,\n self.rres4,\n self.rres5,\n self.rres6,\n self.rres7,\n self.rres8,\n self.rres9,\n self.rres10\n )\n\n\n init_weights(self.backbone, self.backbone2, self.conv1, self.conv3, self.conv4, self.conv5, self.maxpool, self.pixshuf) # state initialization, important!\n\n\n def forward(self, x, y): # x= ms(Nx8x16x16); y = pan(Nx1x64x64)\n\n # ========A): pan feature (extraction)===========\n # --------pan feature (stage 1:)------------\n pan_feature = self.conv1(y) # Nx64x64x64\n rs = pan_feature # Nx64x64x64\n\n rs = self.backbone(rs) # Nx64x64x64\n\n pan_feature1 = torch.add(pan_feature, rs) # Bsx64x64x64\n pan_feature_level1 = self.conv3(pan_feature1) # Bsx8x64x64\n pan_feature1_out = self.maxpool(pan_feature1) # Bsx64x32x32\n\n # --------pan feature (stage 2:)------------\n rs = pan_feature1_out # Bsx64x32x32\n\n rs = self.backbone2(rs) # Nx64x32x32, ????\n\n pan_feature2 = torch.add(pan_feature1_out, rs) # Bsx64x32x32\n pan_feature_level2 = self.conv3(pan_feature2) # Bsx8x32x32\n\n # ========B): ms feature (extraction)===========\n # --------ms feature (stage 1:)------------\n ms_feature1 = self.conv4(x) # x= ms(Nx8x16x16); ms_feature1 =Nx32x16x16\n ms_feature_up1 = self.pixshuf(ms_feature1) # Nx8x32x32\n ms_feature_level1 = torch.add(pan_feature_level2, ms_feature_up1) # Nx8x32x32\n\n # --------ms feature (stage 2:)------------\n ms_feature2 = self.conv5(ms_feature_level1) # Nx32x32x32\n ms_feature_up2 = self.pixshuf(ms_feature2) # Nx8x64x64\n output = torch.add(pan_feature_level1, ms_feature_up2) # Nx8x64x64\n\n return output, ms_feature_level1\n\n def train_step(self, data, *args, **kwargs):\n log_vars = {}\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n sr, _ = self(ms, pan)\n\n loss = self.criterion(sr, gt, *args, **kwargs)\n\n # return sr, loss\n log_vars.update(loss=loss['loss'])\n return {'loss': loss['loss'], 'log_vars': log_vars}\n\n def val_step(self, data, *args, **kwargs):\n\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n sr, _ = self(ms, pan)\n\n return sr, gt\n\n\n\n\nif __name__ == '__main__':\n lms = torch.randn([1, 8, 64, 64])\n pan = torch.randn([1, 1, 64, 64])\n ms = torch.randn([1, 8, 16, 16])\n model = BDPN(8, None)\n x,_ = model(ms, pan)\n print(x.shape)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/DRPNN/drpnn_main.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, Ran Ran, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom .model_drpnn import DRPNN\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n\n def __init__(self, losses, weight_dict):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: n able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relatiumber of object categories, omitting the special no-object category\n matcher: moduleve classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.weight_dict = weight_dict\n self.losses = losses\n self.loss_dicts = {}\n\n def forward(self, outputs, targets, *args, **kwargs):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n # Compute all the requested losses\n\n for k in self.losses.keys():\n # k, loss = loss_dict\n if k == 'Loss':\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets))\n else:\n self.loss_dicts.update({k: loss(outputs, targets)})\n else:\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets, *args)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets, *args))\n else:\n self.loss_dicts.update({k: loss(outputs, targets, *args)})\n\n return self.loss_dicts\n\nfrom UDL.pansharpening.models import PanSharpeningModel\nclass build_drpnn(PanSharpeningModel, name='DRPNN'):\n def __call__(self, cfg):\n\n # important for Pansharpening models, which are from tensorflow code\n self.reg = cfg.reg\n\n scheduler = None\n\n if any([\"wv\" in v for v in cfg.dataset.values()]):\n spectral_num = 8\n else:\n spectral_num = 4\n loss = nn.MSELoss(size_average=True).cuda() ## Define the Loss function\n weight_dict = {'loss': 1}\n losses = {'loss': loss}\n criterion = SetCriterion(losses, weight_dict)\n model = DRPNN(spectral_num, criterion).cuda()\n optimizer = optim.Adam(model.parameters(), lr=cfg.lr, weight_decay=0) ## optimizer 1: Adam\n scheduler = torch.optim.lr_scheduler.StepLR(optimizer=optimizer, step_size=30,\n gamma=0.5) # lr = lr* gamma for each step_size = 180\n\n return model, criterion, optimizer, scheduler\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/DRPNN/model_drpnn.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, Ran Ran, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport numpy as np\nimport math\nimport torch.nn.init as int\n\nimport torch\nimport torch.nn as nn\nimport math\nfrom UDL.Basis.variance_sacling_initializer import variance_scaling_initializer\nfrom UDL.pansharpening.models import PanSharpeningModel\n\n# -------------Initialization----------------------------------------\n\nclass Repeatblock(nn.Module):\n def __init__(self):\n super(Repeatblock, self).__init__()\n\n channel = 32 # input_channel =\n self.conv2 = nn.Conv2d(in_channels=channel, out_channels=channel, kernel_size=7, stride=1, padding=3,\n bias=True)\n self.relu = nn.ReLU(inplace=True)\n\n def forward(self, x):\n rs = self.relu(self.conv2(x))\n\n return rs\n\nclass DRPNN(nn.Module):\n def __init__(self, spectral_num, criterion, channel=32):\n super(DRPNN, self).__init__()\n\n self.criterion = criterion\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n self.conv1 = nn.Conv2d(in_channels=spectral_num+1, out_channels=channel, kernel_size=7, stride=1, padding=3,\n bias=True)\n\n self.conv2 = nn.Conv2d(in_channels=channel, out_channels=spectral_num+1, kernel_size=7, stride=1, padding=3,\n bias=True)\n self.conv3 = nn.Conv2d(in_channels=spectral_num+1, out_channels=spectral_num, kernel_size=7, stride=1, padding=3,\n bias=True)\n self.relu = nn.ReLU(inplace=True)\n\n self.backbone = nn.Sequential( # method 2: 4 resnet repeated blocks\n Repeatblock(),\n Repeatblock(),\n Repeatblock(),\n Repeatblock(),\n Repeatblock(),\n Repeatblock(),\n Repeatblock(),\n Repeatblock(),\n )\n\n def forward(self, x, y): # x= lms; y = pan\n\n input = torch.cat([x, y], 1) # Bsx9x64x64\n rs = self.relu(self.conv1(input)) # Bsx64x64x64\n\n rs = self.backbone(rs) # backbone! Bsx64x64x64\n\n out_res = self.conv2(rs) # Bsx9x64x64\n output1 = torch.add(input, out_res) # Bsx9x64x64\n output = self.conv3(output1) # Bsx8x64x64\n\n return output\n\n def train_step(self, data, *args, **kwargs):\n log_vars = {}\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n sr = self(lms, pan)\n\n loss = self.criterion(sr, gt, *args, **kwargs)\n\n # return sr, loss\n log_vars.update(loss=loss['loss'])\n return {'loss': loss['loss'], 'log_vars': log_vars}\n\n def val_step(self, data, *args, **kwargs):\n\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n sr = self(lms, pan)\n\n return sr, gt\n\n# ----------------- End-Main-Part ------------------------------------\n\n\n\n\n\nif __name__ == '__main__':\n lms = torch.randn([1, 8, 64, 64])\n pan = torch.randn([1, 8, 64, 64])\n model = DRPNN(8, None)\n x = model(lms, pan)\n print(x.shape)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/DiCNN/dicnn_main.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom .model_dicnn import DiCNN\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n\n def __init__(self, losses, weight_dict):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: n able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relatiumber of object categories, omitting the special no-object category\n matcher: moduleve classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.weight_dict = weight_dict\n self.losses = losses\n self.loss_dicts = {}\n\n def forward(self, outputs, targets, *args, **kwargs):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n # Compute all the requested losses\n\n for k in self.losses.keys():\n # k, loss = loss_dict\n if k == 'Loss':\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets))\n else:\n self.loss_dicts.update({k: loss(outputs, targets)})\n else:\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets, *args)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets, *args))\n else:\n self.loss_dicts.update({k: loss(outputs, targets, *args)})\n\n return self.loss_dicts\n\nfrom UDL.pansharpening.models import PanSharpeningModel\nclass build_dicnn(PanSharpeningModel, name='DiCNN1'):\n def __call__(self, cfg):\n\n # important for Pansharpening models, which are from tensorflow code\n self.reg = cfg.reg\n\n scheduler = None\n\n if any([\"wv\" in v for v in cfg.dataset.values()]):\n spectral_num = 8\n else:\n spectral_num = 4\n loss = nn.MSELoss(size_average=True).cuda() ## Define the Loss function\n weight_dict = {'loss': 1}\n losses = {'loss': loss}\n criterion = SetCriterion(losses, weight_dict)\n model = DiCNN(spectral_num, criterion).cuda()\n optimizer = optim.Adam(model.parameters(), lr=cfg.lr, weight_decay=0) ## optimizer 1: Adam\n scheduler = torch.optim.lr_scheduler.StepLR(optimizer=optimizer, step_size=1500,\n gamma=0.5) # lr = lr* gamma for each step_size = 180\n\n return model, criterion, optimizer, scheduler\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/DiCNN/model_dicnn.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport math\nfrom UDL.Basis.variance_sacling_initializer import variance_scaling_initializer\nfrom UDL.pansharpening.models import PanSharpeningModel\n\n# -------------Initialization----------------------------------------\ndef init_weights(*modules):\n for module in modules:\n for m in module.modules():\n if isinstance(m, nn.Conv2d):\n print(\"nn.Conv2D is initialized by variance_scaling_initializer\")\n variance_scaling_initializer(m.weight)\n\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.BatchNorm2d):\n nn.init.constant_(m.weight, 1.0)\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.Linear):\n nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu')\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n\nclass DiCNN(nn.Module):\n def __init__(self, spectral_num, criterion, channel=64, reg=True):\n super(DiCNN, self).__init__()\n self.criterion = criterion\n self.reg = reg\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n self.conv1 = nn.Conv2d(in_channels=spectral_num + 1, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.conv2 = nn.Conv2d(in_channels=channel, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.conv3 = nn.Conv2d(in_channels=channel, out_channels=spectral_num, kernel_size=3, stride=1, padding=1,\n bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n self.apply(init_weights)\n\n def forward(self, x, y):\n # x= lms; y = pan\n input1 = torch.cat([x, y], 1) # Bsx9x64x64\n\n rs = self.relu(self.conv1(input1))\n rs = self.relu(self.conv2(rs))\n out = self.conv3(rs)\n output = x + out\n\n return output\n\n def train_step(self, data, *args, **kwargs):\n log_vars = {}\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n sr = self(lms, pan)\n\n loss = self.criterion(sr, gt, *args, **kwargs)\n\n # return sr, loss\n log_vars.update(loss=loss['loss'])\n return {'loss': loss['loss'], 'log_vars': log_vars}\n\n def val_step(self, data, *args, **kwargs):\n\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n sr = self(lms, pan)\n\n return sr, gt\n\n\nif __name__ == '__main__':\n lms = torch.randn([1, 8, 64, 64])\n pan = torch.randn([1, 8, 64, 64])\n model = DiCNN(8, None)\n x = model(lms, pan)\n print(x.shape)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/FusionNet/fusionnet_main.py", "content": "import os\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.nn as nn\nimport torch.optim as optim\nfrom .model_fusionnet import FusionNet\nimport numpy as np\n\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n\n def __init__(self, losses, weight_dict):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: n able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relatiumber of object categories, omitting the special no-object category\n matcher: moduleve classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.weight_dict = weight_dict\n self.losses = losses\n self.loss_dicts = {}\n\n def forward(self, outputs, targets, *args, **kwargs):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n # lms = kwargs.get('lms')\n # outputs = outputs + lms # outputs: hp_sr\n # Compute all the requested losses\n for k in self.losses.keys():\n # k, loss = loss_dict\n if k == 'loss':\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets))\n else:\n self.loss_dicts.update({k: loss(outputs, targets)})\n else:\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets, *args)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets, *args))\n else:\n self.loss_dicts.update({k: loss(outputs, targets, *args)})\n\n return self.loss_dicts\n\nfrom UDL.pansharpening.models import PanSharpeningModel\nclass build_fusionnet(PanSharpeningModel, name='FusionNet'):\n def __call__(self, args):\n scheduler = None\n if any([\"wv\" in v for v in args.dataset.values()]):\n spectral_num = 8\n else:\n spectral_num = 4\n\n\n loss = nn.MSELoss(size_average=True).cuda() ## Define the Loss function\n weight_dict = {'loss': 1}\n losses = {'loss': loss}\n criterion = SetCriterion(losses, weight_dict)\n model = FusionNet(spectral_num, criterion).cuda()\n optimizer = optim.Adam(model.parameters(), lr=args.lr, weight_decay=0) ## optimizer 1: Adam\n\n return model, criterion, optimizer, scheduler\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/FusionNet/model_fusionnet.py", "content": "# This is a pytorch version for the work of PanNet\n# YW Jin, X Wu, LJ Deng(UESTC);\n# 2020-09;\n\nimport torch\nimport torch.nn as nn\nimport numpy as np\nimport math\nimport torch.nn.init as int\nfrom UDL.Basis.variance_sacling_initializer import variance_scaling_initializer\n\n\nclass loss_with_l2_regularization(nn.Module):\n def __init__(self):\n super(loss_with_l2_regularization, self).__init__()\n\n def forward(self, criterion, model, weight_decay=1e-5, flag=False):\n regularizations = []\n for k, v in model.named_parameters():\n if 'conv' in k and 'weight' in k:\n # print(k)\n penality = weight_decay * ((v.data ** 2).sum() / 2)\n regularizations.append(penality)\n if flag:\n print(\"{} : {}\".format(k, penality))\n # r = torch.sum(regularizations)\n\n loss = criterion + sum(regularizations)\n return loss\n\n\n# -------------Initialization----------------------------------------\ndef init_weights(*modules):\n for module in modules:\n for m in module.modules():\n if isinstance(m, nn.Conv2d): ## initialization for Conv2d\n print(\"initial nn.Conv2d with var_scale_new: \", m)\n # try:\n # import tensorflow as tf\n # tensor = tf.get_variable(shape=m.weight.shape, initializer=tf.variance_scaling_initializer(seed=1))\n # m.weight.data = tensor.eval()\n # except:\n # print(\"try error, run variance_scaling_initializer\")\n # variance_scaling_initializer(m.weight)\n variance_scaling_initializer(m.weight) # method 1: initialization\n # nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu') # method 2: initialization\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.BatchNorm2d): ## initialization for BN\n nn.init.constant_(m.weight, 1.0)\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.Linear): ## initialization for nn.Linear\n # variance_scaling_initializer(m.weight)\n nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu')\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n\n\n# -------------ResNet Block (One)----------------------------------------\nclass Resblock(nn.Module):\n def __init__(self):\n super(Resblock, self).__init__()\n\n channel = 32\n self.conv20 = nn.Conv2d(in_channels=channel, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.conv21 = nn.Conv2d(in_channels=channel, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.relu = nn.ReLU(inplace=True)\n\n def forward(self, x): # x= hp of ms; y = hp of pan\n rs1 = self.relu(self.conv20(x)) # Bsx32x64x64\n rs1 = self.conv21(rs1) # Bsx32x64x64\n rs = torch.add(x, rs1) # Bsx32x64x64\n return rs\n\nclass FusionNet(nn.Module):\n def __init__(self, spectral_num, criterion, channel=32):\n super(FusionNet, self).__init__()\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n self.spectral_num = spectral_num\n self.criterion = criterion\n\n self.conv1 = nn.Conv2d(in_channels=spectral_num, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.res1 = Resblock()\n self.res2 = Resblock()\n self.res3 = Resblock()\n self.res4 = Resblock()\n\n self.conv3 = nn.Conv2d(in_channels=channel, out_channels=spectral_num, kernel_size=3, stride=1, padding=1,\n bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n self.backbone = nn.Sequential( # method 2: 4 resnet repeated blocks\n self.res1,\n self.res2,\n self.res3,\n self.res4\n )\n\n # init_weights(self.backbone, self.conv1, self.conv3) # state initialization, important!\n # self.apply(init_weights)\n\n def forward(self, x, y): # x= lms; y = pan\n\n pan_concat = y.repeat(1, self.spectral_num, 1, 1) # Bsx8x64x64\n input = torch.sub(pan_concat, x) # Bsx8x64x64\n rs = self.relu(self.conv1(input)) # Bsx32x64x64\n\n rs = self.backbone(rs) # ResNet's backbone!\n output = self.conv3(rs) # Bsx8x64x64\n\n return output # lms + outs\n\n def train_step(self, data, *args, **kwargs):\n log_vars = {}\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n res = self(lms, pan)\n sr = lms + res # output:= lms + hp_sr\n loss = self.criterion(sr, gt, *args, **kwargs)['loss']\n # outputs = loss\n # return loss\n log_vars.update(pan2ms=loss.item(), loss=loss.item())\n metrics = {'loss': loss, 'log_vars': log_vars}\n return metrics\n\n def val_step(self, data, *args, **kwargs):\n # gt, lms, ms, pan = data\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n res = self(lms, pan)\n sr = lms + res # output:= lms + hp_sr\n\n return sr, gt" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/FusionNet/run_fusionnet.py", "content": "from UDL.Basis.config import Config\nfrom UDL.pansharpening.common.main_pansharpening import main\nfrom UDL.Basis.auxiliary import set_random_seed\nfrom UDL.pansharpening.models.FusionNet.option_fusionnet import cfg as args\nfrom UDL.pansharpening.models.FusionNet.fusionnet_main import build_fusionnet as builder\n\nif __name__ == '__main__':\n # cfg = Config.fromfile(\"../pansharpening/DCFNet/option_DCFNet.py\")\n set_random_seed(args.seed)\n # print(cfg.builder)\n args.builder = builder\n main(args)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/MSDCNN/model_msdcnn.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, Ran Ran, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport numpy as np\nimport math\nimport torch.nn.init as int\n# import sys\n# sys.path.append('/home/office-401-remote/桌面/Machine Learning/RanRan')\n# print(sys.path)\nimport torch\nimport torch.nn as nn\nimport math\nfrom UDL.Basis.variance_sacling_initializer import variance_scaling_initializer\nfrom UDL.pansharpening.models import PanSharpeningModel\n\nclass MSDCNN(nn.Module):\n def __init__(self, spectral_num, criterion, channel=64):\n super(MSDCNN, self).__init__()\n\n self.criterion = criterion\n\n input_channel = spectral_num + 1\n output_channel = spectral_num\n\n self.conv1 = nn.Conv2d(in_channels=input_channel, out_channels=60, kernel_size=7, stride=1, padding=3, bias=True)\n\n self.conv2_1 = nn.Conv2d(in_channels=60, out_channels=20, kernel_size=3, stride=1, padding=1, bias=True)\n self.conv2_2 = nn.Conv2d(in_channels=60, out_channels=20, kernel_size=5, stride=1, padding=2, bias=True)\n self.conv2_3 = nn.Conv2d(in_channels=60, out_channels=20, kernel_size=7, stride=1, padding=3, bias=True)\n\n self.conv3 = nn.Conv2d(in_channels=60, out_channels=30, kernel_size=3, stride=1, padding=1, bias=True)\n\n self.conv4_1 = nn.Conv2d(in_channels=30, out_channels=10, kernel_size=3, stride=1, padding=1, bias=True)\n self.conv4_2 = nn.Conv2d(in_channels=30, out_channels=10, kernel_size=5, stride=1, padding=2, bias=True)\n self.conv4_3 = nn.Conv2d(in_channels=30, out_channels=10, kernel_size=7, stride=1, padding=3, bias=True)\n\n self.conv5 = nn.Conv2d(in_channels=30, out_channels=output_channel, kernel_size=5, stride=1, padding=2, bias=True)\n\n self.shallow1 = nn.Conv2d(in_channels=input_channel, out_channels=64, kernel_size=9, stride=1, padding=4, bias=True)\n self.shallow2 = nn.Conv2d(in_channels=64, out_channels=32, kernel_size=1, stride=1, padding=0, bias=True)\n self.shallow3 = nn.Conv2d(in_channels=32, out_channels=output_channel, kernel_size=5, stride=1, padding=2, bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n\n def forward(self, x, y): # x: lms; y: pan\n\n concat = torch.cat([x, y], 1) # Bsx9x64x64\n\n out1 = self.relu(self.conv1(concat)) # Bsx60x64x64\n out21 = self.conv2_1(out1) # Bsx20x64x64\n out22 = self.conv2_2(out1) # Bsx20x64x64\n out23 = self.conv2_3(out1) # Bsx20x64x64\n out2 = torch.cat([out21, out22, out23], 1) # Bsx60x64x64\n\n out2 = self.relu(torch.add(out2, out1)) # Bsx60x64x64\n\n out3 = self.relu(self.conv3(out2)) # Bsx30x64x64\n out41 = self.conv4_1(out3) # Bsx10x64x64\n out42 = self.conv4_2(out3) # Bsx10x64x64\n out43 = self.conv4_3(out3) # Bsx10x64x64\n out4 = torch.cat([out41, out42, out43], 1) # Bsx30x64x64\n\n out4 = self.relu(torch.add(out4, out3)) # Bsx30x64x64\n\n out5 = self.conv5(out4) # Bsx8x64x64\n\n shallow1 = self.relu(self.shallow1(concat)) # Bsx64x64x64\n shallow2 = self.relu(self.shallow2(shallow1)) # Bsx32x64x64\n shallow3 = self.shallow3(shallow2) # Bsx8x64x64\n\n out = torch.add(out5, shallow3) # Bsx8x64x64\n out = self.relu(out) # Bsx8x64x64\n\n return out\n\n def train_step(self, data, *args, **kwargs):\n log_vars = {}\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n sr = self(lms, pan)\n\n loss = self.criterion(sr, gt, *args, **kwargs)\n\n # return sr, loss\n log_vars.update(loss=loss['loss'])\n return {'loss': loss['loss'], 'log_vars': log_vars}\n\n def val_step(self, data, *args, **kwargs):\n\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n sr = self(lms, pan)\n\n return sr, gt\n\n\n\n\nif __name__ == '__main__':\n lms = torch.randn([1, 8, 64, 64])\n pan = torch.randn([1, 1, 64, 64])\n ms = torch.randn([1, 8, 16, 16])\n model = BDPN(8, None)\n x,_ = model(ms, pan)\n print(x.shape)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/MSDCNN/msdcnn_main.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, Ran Ran, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom .model_msdcnn import MSDCNN\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n\n def __init__(self, losses, weight_dict):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: n able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relatiumber of object categories, omitting the special no-object category\n matcher: moduleve classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.weight_dict = weight_dict\n self.losses = losses\n self.loss_dicts = {}\n\n def forward(self, outputs, targets, *args, **kwargs):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n # Compute all the requested losses\n\n for k in self.losses.keys():\n # k, loss = loss_dict\n if k == 'Loss':\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets))\n else:\n self.loss_dicts.update({k: loss(outputs, targets)})\n else:\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets, *args)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets, *args))\n else:\n self.loss_dicts.update({k: loss(outputs, targets, *args)})\n\n return self.loss_dicts\n\nfrom UDL.pansharpening.models import PanSharpeningModel\nclass build_msdcnn(PanSharpeningModel, name='MSDCNN'):\n def __call__(self, cfg):\n\n # important for Pansharpening models, which are from tensorflow code\n self.reg = cfg.reg\n\n scheduler = None\n\n if any([\"wv\" in v for v in cfg.dataset.values()]):\n spectral_num = 8\n else:\n spectral_num = 4\n\n loss = nn.MSELoss(size_average=True).cuda() ## Define the Loss function\n weight_dict = {'loss': 1}\n losses = {'loss': loss}\n criterion = SetCriterion(losses, weight_dict)\n model = MSDCNN(spectral_num, criterion).cuda()\n optimizer = optim.Adam(model.parameters(), lr=cfg.lr, weight_decay=1e-5) ## optimizer 1: Adam\n scheduler = torch.optim.lr_scheduler.StepLR(optimizer=optimizer, step_size=40,\n gamma=0.5) # <=> lr = opt.lr * (0.5 ** (epoch // opt.step))\n\n return model, criterion, optimizer, scheduler\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/PNN/model_pnn.py", "content": "import torch\nimport torch.nn as nn\nfrom torch.nn import functional as F\nimport math\n# from UDL.Basis.variance_sacling_initializer import variance_scaling_initializer\n\nclass PNN(nn.Module):\n def __init__(self, spectral_num, criterion, channel=64):\n super(PNN, self).__init__()\n\n self.criterion = criterion\n\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n self.conv1 = nn.Conv2d(in_channels=spectral_num + 1, out_channels=channel, kernel_size=9, stride=1,\n bias=True)\n self.conv2 = nn.Conv2d(in_channels=channel, out_channels=32, kernel_size=5, stride=1,\n bias=True)\n self.conv3 = nn.Conv2d(in_channels=32, out_channels=spectral_num, kernel_size=5, stride=1,\n bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n # init_weights(self.conv1, self.conv2, self.conv3)\n\n def forward(self, x): # x = cat(lms,pan)\n input1 = x # Bsx9x64x64\n\n rs = self.relu(self.conv1(input1))\n rs = self.relu(self.conv2(rs))\n output = self.conv3(rs)\n\n return output\n\n def train_step(self, data, *args, **kwargs):\n log_vars = {}\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n blk = self.blk\n\n gt = gt[:, :, blk:-blk, blk:-blk]\n lms = torch.cat([lms, pan], dim=1)\n\n sr = self(lms)\n\n loss = self.criterion(sr, gt, *args, **kwargs)\n\n # return sr, loss\n log_vars.update(loss=loss['loss'])\n return {'loss': loss['loss'], 'log_vars': log_vars}\n\n def val_step(self, data, *args, **kwargs):\n blk = self.blk\n gt, lms, ms, pan = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n test_I_in1 = torch.cat([lms, pan], dim=1)\n test_I_in1 = F.pad(test_I_in1, (blk, blk, blk, blk), mode='replicate')\n sr = self(test_I_in1)\n\n return sr, gt\n\n @classmethod\n def set_blk(cls, blk):\n cls.blk = blk\n\n# ----------------- End-Main-Part ------------------------------------\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/PNN/pnn_main.py", "content": "import torch.nn as nn\nimport torch.optim as optim\nfrom .model_pnn import PNN\nimport numpy as np\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n\n def __init__(self, losses, weight_dict):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: n able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relatiumber of object categories, omitting the special no-object category\n matcher: moduleve classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.weight_dict = weight_dict\n self.losses = losses\n self.loss_dicts = {}\n\n def forward(self, outputs, targets, *args, **kwargs):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n # Compute all the requested losses\n\n for k in self.losses.keys():\n # k, loss = loss_dict\n if k == 'Loss':\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets))\n else:\n self.loss_dicts.update({k: loss(outputs, targets)})\n else:\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets, *args)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets, *args))\n else:\n self.loss_dicts.update({k: loss(outputs, targets, *args)})\n\n return self.loss_dicts\n\n\nfrom UDL.pansharpening.models import PanSharpeningModel\nclass build_pnn(PanSharpeningModel, name='PNN'):\n def __call__(self, cfg):\n\n # important for Pansharpening models, which are from tensorflow code\n self.reg = cfg.reg\n\n\n scheduler = None\n\n if any([\"wv\" in v for v in cfg.dataset.values()]):\n spectral_num = 8\n else:\n spectral_num = 4\n lr = 0.0001 * 17 * 17 * spectral_num\n cfg.lr = lr\n print(f\"PNN adopted another lr: {lr} in \\\"build_pnn in pnn_main.py\\\" \")\n\n\n loss = nn.MSELoss(size_average=True).cuda() ## Define the Loss function\n weight_dict = {'loss': 1}\n losses = {'loss': loss}\n criterion = SetCriterion(losses, weight_dict)\n model = PNN(spectral_num, criterion).cuda()\n target_layerParam = list(map(id, model.conv3.parameters()))\n base_layerParam = filter(lambda p: id(p) not in target_layerParam, model.parameters())\n\n training_parameters = [{'params': model.conv3.parameters(), 'lr': lr / 10},\n {'params': base_layerParam}]\n\n optimizer = optim.SGD(model.parameters(), lr=lr, momentum=0.9) ## optimizer 2: SGD\n\n net_scope = 0\n for name, layer in model.named_parameters():\n if 'conv' in name and 'bias' not in name:\n net_scope += layer.shape[-1] - 1\n\n net_scope = np.sum(net_scope) + 1\n blk = net_scope // 2 # 8\n model.set_blk(blk)\n\n return model, criterion, optimizer, scheduler\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/PanNet/model_pannet.py", "content": "# GPL License\n# Copyright (C) 2021 , UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\nimport torch\nimport torch.nn as nn\nimport numpy as np\nimport math\nimport torch.nn.init as int\nfrom UDL.Basis.variance_sacling_initializer import variance_scaling_initializer\nfrom UDL.pansharpening.models import PanSharpeningModel\n\n# -------------Initialization----------------------------------------\ndef init_weights(*modules):\n for module in modules:\n for m in module.modules():\n if isinstance(m, nn.Conv2d): ## initialization for Conv2d\n print(\"nn.Conv2D is initialized by variance_scaling_initializer\")\n variance_scaling_initializer(m.weight) # method 1: initialization\n # nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu') # method 2: initialization\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.BatchNorm2d): ## initialization for BN\n nn.init.constant_(m.weight, 1.0)\n nn.init.constant_(m.bias, 0.0)\n elif isinstance(m, nn.Linear): ## initialization for nn.Linear\n # variance_scaling_initializer(m.weight)\n nn.init.kaiming_normal_(m.weight, mode='fan_in', nonlinearity='relu')\n if m.bias is not None:\n nn.init.constant_(m.bias, 0.0)\n\n\n# -------------ResNet Block (One)----------------------------------------\nclass Resblock(nn.Module):\n def __init__(self):\n super(Resblock, self).__init__()\n\n channel = 32\n self.conv20 = nn.Conv2d(in_channels=channel, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.conv21 = nn.Conv2d(in_channels=channel, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.relu = nn.ReLU(inplace=True)\n\n def forward(self, x): # x= hp of ms; y = hp of pan\n rs1 = self.relu(self.conv20(x)) # Bsx32x64x64\n rs1 = self.conv21(rs1) # Bsx32x64x64\n rs = torch.add(x, rs1) # Bsx32x64x64\n return rs\n\n\n# -----------------------------------------------------\nclass PanNet(nn.Module):\n def __init__(self, spectral_num, criterion, channel=32, reg=True):\n super(PanNet, self).__init__()\n self.criterion = criterion\n self.reg = reg\n\n # ConvTranspose2d: output = (input - 1)*stride + outpading - 2*padding + kernelsize\n self.deconv = nn.ConvTranspose2d(in_channels=spectral_num, out_channels=spectral_num, kernel_size=8, stride=4,\n padding=2, bias=True)\n self.conv1 = nn.Conv2d(in_channels=spectral_num + 1, out_channels=channel, kernel_size=3, stride=1, padding=1,\n bias=True)\n self.res1 = Resblock()\n self.res2 = Resblock()\n self.res3 = Resblock()\n self.res4 = Resblock()\n\n self.conv3 = nn.Conv2d(in_channels=channel, out_channels=spectral_num, kernel_size=3, stride=1, padding=1,\n bias=True)\n\n self.relu = nn.ReLU(inplace=True)\n\n self.backbone = nn.Sequential( # method 2: 4 resnet repeated blocks\n self.res1,\n self.res2,\n self.res3,\n self.res4\n )\n\n self.apply(init_weights)\n # init_weights(self.backbone, self.deconv, self.conv1, self.conv3) # state initialization, important!\n\n def forward(self, x, y):# x= hp of ms; y = hp of pan\n\n output_deconv = self.deconv(x)\n input = torch.cat([output_deconv, y], 1) # Bsx9x64x64\n rs = self.relu(self.conv1(input)) # Bsx32x64x64\n\n rs = self.backbone(rs) # ResNet's backbone!\n\n output = self.conv3(rs) # Bsx8x64x64\n return output\n\n def train_step(self, data, *args, **kwargs):\n log_vars = {}\n gt, lms, ms_hp, pan_hp = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms_hp'].cuda(), data['pan_hp'].cuda()\n hp_sr = self(ms_hp, pan_hp)\n sr = lms + hp_sr # output:= lms + hp_sr\n loss = self.criterion(sr, gt, *args, **kwargs)\n # return sr, loss\n log_vars.update(loss=loss['loss'])\n return {'loss': loss['loss'], 'log_vars': log_vars}\n\n def val_step(self, data, *args, **kwargs):\n # gt, lms, ms, pan = data\n gt, lms, ms_hp, pan_hp = data['gt'].cuda(), data['lms'].cuda(), \\\n data['ms'].cuda(), data['pan'].cuda()\n hp_sr = self(ms_hp, pan_hp)\n sr = lms + hp_sr # output:= lms + hp_sr\n return sr, gt\n\n\n# ----------------- End-Main-Part ------------------------------------\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/PanNet/pannet_main.py", "content": "'''\n[Flops]: ConvTranspose2d is not supported!\n[Memory]: ConvTranspose2d is not supported!\n===============================================================================================================================================\nTotal params: 83,024\n-----------------------------------------------------------------------------------------------------------------------------------------------\nTotal memory: 7.25MB\nTotal MAdd: 646.84MMAdd\nTotal Flops: 323.91MFlops\nTotal MemR+W: 14.57MB\n'''\nimport torch.nn as nn\nimport torch.optim as optim\nfrom .model_pannet import PanNet\n\nclass SetCriterion(nn.Module):\n \"\"\" This class computes the loss for DETR.\n The process happens in two steps:\n 1) we compute hungarian assignment between ground truth boxes and the outputs of the model\n 2) we supervise each pair of matched ground-truth / prediction (supervise class and box)\n \"\"\"\n\n def __init__(self, losses, weight_dict):\n \"\"\" Create the criterion.\n Parameters:\n num_classes: n able to compute a matching between targets and proposals\n weight_dict: dict containing as key the names of the losses and as values their relative weight.\n eos_coef: relatiumber of object categories, omitting the special no-object category\n matcher: moduleve classification weight applied to the no-object category\n losses: list of all the losses to be applied. See get_loss for list of available losses.\n \"\"\"\n super().__init__()\n self.weight_dict = weight_dict\n self.losses = losses\n self.loss_dicts = {}\n\n def forward(self, outputs, targets, *args, **kwargs):\n \"\"\" This performs the loss computation.\n Parameters:\n outputs: dict of tensors, see the output specification of the model for the format\n targets: list of dicts, such that len(targets) == batch_size.\n The expected keys in each dict depends on the losses applied, see each loss' doc\n \"\"\"\n # Compute all the requested losses\n\n for k in self.losses.keys():\n # k, loss = loss_dict\n if k == 'Loss':\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets))\n else:\n self.loss_dicts.update({k: loss(outputs, targets)})\n else:\n loss = self.losses[k]\n loss_dicts = loss(outputs, targets, *args)\n if isinstance(loss_dicts, dict):\n self.loss_dicts.update(loss(outputs, targets, *args))\n else:\n self.loss_dicts.update({k: loss(outputs, targets, *args)})\n\n return self.loss_dicts\n\nfrom UDL.pansharpening.models import PanSharpeningModel\nclass build_pannet(PanSharpeningModel, name='PanNet'):\n def __call__(self, cfg):\n\n if not all(['hp' in name for name in list(cfg.dataset.values())]):\n raise ValueError(f\"{cfg.dataset} is wrong for PanNet, you need high-pass filter dataset.\")\n\n # important for Pansharpening models, which are from tensorflow code\n self.reg = cfg.reg\n\n scheduler = None\n\n if any([\"wv\" in v for v in cfg.dataset.values()]):\n spectral_num = 8\n else:\n spectral_num = 4\n loss = nn.MSELoss(size_average=True).cuda() ## Define the Loss function\n weight_dict = {'loss': 1}\n losses = {'loss': loss}\n criterion = SetCriterion(losses, weight_dict)\n model = PanNet(spectral_num, criterion).cuda()\n optimizer = optim.Adam(model.parameters(), lr=cfg.lr, weight_decay=0) ## optimizer 1: Adam\n\n return model, criterion, optimizer, scheduler\n\n\n###################################################################\n# ------------------- Main Function (Run first) -------------------\n###################################################################\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/models/__init__.py", "content": "from UDL.AutoDL import PanSharpeningModel\nfrom .DiCNN.dicnn_main import build_dicnn, DiCNN\nfrom .FusionNet.fusionnet_main import build_fusionnet, FusionNet\nfrom .PNN.pnn_main import build_pnn, PNN\nfrom .PanNet.pannet_main import build_pannet, PanNet\nfrom .DRPNN.drpnn_main import build_drpnn, DRPNN\nfrom .BDPN.bdpn_main import build_bdpn, BDPN\nfrom .MSDCNN.msdcnn_main import build_msdcnn, MSDCNN" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/run_pansharpening.py", "content": "# GPL License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\n\nimport sys\nsys.path.append('../..')\nfrom UDL.AutoDL import TaskDispatcher\nfrom UDL.AutoDL.trainer import main\n\nif __name__ == '__main__':\n cfg = TaskDispatcher.new(task='pansharpening', mode='entrypoint', arch='FusionNet')\n print(TaskDispatcher._task.keys())\n main(cfg)" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pansharpening/run_test_pansharpening.py", "content": "# GPL License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Author : Xiao Wu, LiangJian Deng\n# @reference:\n\nimport sys\nsys.path.append('../..')\nfrom UDL.AutoDL import TaskDispatcher\nfrom UDL.AutoDL.trainer import main\n\nif __name__ == '__main__':\n cfg = TaskDispatcher.new(task='pansharpening', mode='entrypoint', arch='MSDCNN')\n # cfg.resume_from = \"../pretrained-model/WV3/pannet.pth\"\n cfg.eval = True\n cfg.workflow = [('val', 1)]\n print(TaskDispatcher._task.keys())\n main(cfg)\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pretrained-model/QB/readme.txt", "content": "none" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pretrained-model/WV2/readme.txt", "content": "none" }, { "path": "01-DL-toolbox(Pytorch)/UDL/pretrained-model/WV4/readme.txt", "content": "none" }, { "path": "01-DL-toolbox(Pytorch)/UDL/readme.md", "content": "test\n" }, { "path": "01-DL-toolbox(Pytorch)/UDL/results/readme.txt", "content": "" }, { "path": "01-DL-toolbox(Pytorch)/readme.md", "content": "# DL toolbox\n\"DL toolbox\" for Remote Sensing Pansharpening\n\n[English]([https://github.com/XiaoXiao-Woo/PanCollection/edit/dev/README.md](https://github.com/liangjiandeng/DLPan-Toolbox/edit/main/01-DL-toolbox(Pytorch)/readme.md)) | [简体中文](https://github.com.md)\n\nThis repository is the official PyTorch implementation of our IEEE GRSM paper “Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks”, 2022 ([paper](https://github.com/liangjiandeng/liangjiandeng.github.io/tree/master/papers/2022/review-grsm2022.pdf) | [homepage](https://github.com/liangjiandeng/DLPan-Toolbox)).\n\n\n\n## Features\n\n\n## Requirements\n* Python3.7+, Pytorch>=1.6.0\n* NVIDIA GPU + CUDA\n* Run `python setup.py develop`\n\nNote: Our project is based on MMCV, but you needn't to install it currently.\n\n## Quick Start\n**Step0. Set your Python environment.**\n\n>git clone https://github.com/liangjiandeng/DLPan-Toolbox/tree/main/01-DL-toolbox(Pytorch)\n\nThen, \n\n> python setup.py develop\n\n**Step1. Put datasets and set path**\n* Put datasets (WorldView-3, QuickBird, GaoFen2, WorldView2) into the `UDL/Data/pansharpening`, see following path structure. \n\n```\n|-$ROOT/Data\n├── pansharpening\n│   ├── training_data\n│   │   ├── train_wv3.h5\n│   │   ├── ...\n│   ├── validation_data\n│   │   │   ├── valid_wv3.h5\n│   │   │   ├── ...\n│   ├── test_data\n│   │   ├── WV3\n│   │   │   ├── NY1_WV3_RR.mat\n│   │   │   ├── ...\n│   │   │   ├── ...\n```\n\n* Check and revise your dataset path in `01-DL-toolbox(Pytorch)/UDL/Basis/option.py` (line 100 or line 102, may not need to revise); Or, you can print the output of `run_pansharpening.py`, then set __cfg.data_dir__ (also line 100 or line 102) to your dataset path.\n\n\n\n**Step2. How to train?**\n\n> open `01-DL-toolbox(Pytorch)/UDL/pansharpening`\n\n> run `python run_pansharpening.py` for training\n\n> if you want to change the network, you could: \n\n1) revise arch='BDPN' in the following codes to other network's name, e.g., arch='xxx'; \n\n\t```python\n\t import sys\n sys.path.append('../..')\n from UDL.AutoDL import TaskDispatcher\n from UDL.AutoDL.trainer import main\n\n if __name__ == '__main__':\n cfg = TaskDispatcher.new(task='pansharpening', mode='entrypoint', arch='BDPN')\n print(TaskDispatcher._task.keys())\n main(cfg)\n\t ```\n2) revise the corresponding setting in `pansharpening/configs/option_bdpn.py`, e.g., hyperparameters, validation data\n\n\t```python\n\t cfg.eval = False, \n \n cfg.workflow = [('train', 50), ('val', 1)], \n\t\n\t cfg.dataset = {'train': 'wv3', 'val': 'valid_wv3.h5'}\n\t```\n\t\n\n**Step3. How to test?**\n\n> open `01-DL-toolbox(Pytorch)/UDL/pansharpening`\n\n> run `run_test_pansharpening.py` for testing\n\n> Note you need to ensure `cfg.eval = True` or `cfg.workflow = [('val', 1)]` in the following `run_test_pansharpening.py` to run\n\t \n\n```python\n\timport sys\n\tsys.path.append('../..')\n\tfrom UDL.AutoDL import TaskDispatcher\n\tfrom UDL.AutoDL.trainer import main\n\n\tif __name__ == '__main__':\n\tcfg = TaskDispatcher.new(task='pansharpening', mode='entrypoint', arch='MSDCNN')\n\tcfg.eval = True\n\tcfg.workflow = [('val', 1)]\n\tprint(TaskDispatcher._task.keys())\n\tmain(cfg)\n```\n\n> How to get test outcome using the pretrained models?\n\n1) find the given one example (i.e., `NY1_WV3_RR.mat`) in the path `UDL/Data/pansharpening/test_data`; \n\n2) load pretrained model by setting __model_path__ = \"your_model_path\" located in the folder of `pansharpening/configs/option_bdpn.py` (line 15)； Or __cfg.resume_from__ = \"your_model_path\" (line 31).\n\n3) run `run_test_pansharpening.py`, then you may find the test results in the folder of `UDL/results`\n\n\n\n## FAQ\n**Q1.** How to customize your new network/model in this framework?\n\n> 1) Construct your model, loss, optimizer, scheduler in `UDL/pansharpening/models/modelName/modelName_main.py` (you need to create your modelName in `modelName_main.py`, i.e., the similar operation as other methods in the path).\n\n> 2) Update `UDL/pansharpening/models/__init__.py` \n\n> 3) Add `option_modelName.py` in `UDL/pansharpening/configs/Option_modelName.py`, and configure your hyperparameters in this file (see other methods' configuration in `UDL/pansharpening/configs` for easy usage).\n\n> 4) train your model and infer your results, see __step2__ and __step3__ for details.\n\n> 5) save your model early, add or change `cfg.save_freq_print` and `cfg.save_top_k` in `UDL/pansharpening/configs/Option_modelName.py`. We set the default for it starting at epoch 5 and save models every 10 epochs.\n\n\n**Q2.** How to customize your datasets?\n\nYou need to update: `UDL/pansharpening/common/psdata.py` (revise/add lines 24-29 to customize your datasets).\n\n\n\n**Q3.** How to customized training settings, such as saving model, recording logs, etc.?\n\nYou need to update: `UDL/mmcv/mmcv/runner/hooks` (generally, it does not need to revise if you do not require more complicated training settings).\n\n\n**Q4.** How to know more details of runner about how to train/test in `UDL/AutoDL/trainer.py`?\n\nPlease see `UDL/mmcv/mmcv/runner/epoch_based_runner.py`.\n\n\n**Note:** Don't put any files into the folder of AutoDL. \n\n\n\n\n## Citation\n* If you use this toolbox, please kindly cite our paper:\n\n```bibtex\n@ARTICLE{deng2022grsm,\nauthor={L.-J. Deng, G. Vivone, M. E. Paoletti, G. Scarpa, J. He, Y. Zhang, J. Chanussot, and A. Plaza},\nbooktitle={IEEE Geoscience and Remote Sensing Magazine},\ntitle={Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks},\nyear={2022},\npages={},\n}\n```\n\n\n* Also, the codes of traditional methods are from the \"pansharpening toolbox for distribution\", thus please cite the corresponding paper:\n```bibtex\n@ARTICLE{vivone2021grsm,\n author={Vivone, Gemine and Dalla Mura, Mauro and Garzelli, Andrea and Restaino, Rocco and Scarpa, Giuseppe and Ulfarsson, Magnus O. and Alparone, Luciano and Chanussot, Jocelyn},\n journal={IEEE Geoscience and Remote Sensing Magazine}, \n title={A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods}, \n year={2021},\n volume={9},\n number={1},\n pages={53-81},\n doi={10.1109/MGRS.2020.3019315}\n}\n```\n\n\n## Acknowledgement\n- [MMCV](https://github.com/open-mmlab/mmcv): OpenMMLab foundational library for computer vision.\n- We appreciate the great contribution of [Xiao Wu](https://xiaoxiao-woo.github.io/) who is a graduate student in [UESTC](https://www.uestc.edu.cn/) to this toolbox.\n\n## Contribution\nWe appreciate all contributions to improving '01-DL-toolbox(Pytorch)'. Looking forward to your contribution to DLPan-Toolbox.\n\n\n## License & Copyright\nThis project is open sourced under GNU General Public License v3.0.\n\n" }, { "path": "01-DL-toolbox(Pytorch)/setup.py", "content": "from setuptools import setup, find_packages\n\nsetup(\n classifiers=\n ['Programming Language :: Python :: 3.7+', ],\n name='udl',\n description=\"unified pytorch framework for vision task\",\n author=\"XiaoXiao-Woo\",\n author_email=\"wxwsx1997@gmail.com\",\n url='https://github.com/XiaoXiao-Woo/PanCollection',\n version='0.1',\n packages=find_packages(),\n license='GPLv3',\n python_requires='>=3.7',\n install_requires=[\n \"psutil\",\n \"opencv-python\",\n \"numpy\",\n \"matplotlib\",\n \"tensorboard\",\n \"addict\",\n \"yapf\",\n \"imageio\",\n \"colorlog\",\n \"scipy\",\n \"timm\"\n ],\n)" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/1_TestData/Datasets Testing/Download link for WV3-NewYork test data.txt", "content": "This folders contain the testing examples, including:\n\n1) \"Datasets Testing\": A full-resolution WV3-NewYork example + A reduced-resolution WV3-NewYork example\n\n2) \"QB\", \"WV2\", \"WV3\" and \"WV4\": Save the test datasets for different sensors" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/1_TestData/QB/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/1_TestData/WV2/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/1_TestData/WV3/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/1_TestData/WV4/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/1_TestData/readme.txt", "content": "This folders contain the testing examples, including:\n\n1) \"Datasets Testing\": A full-resolution WV3-NewYork example + A reduced-resolution WV3-NewYork example\n\n2) \"QB\", \"WV2\", \"WV3\" and \"WV4\": Save the test datasets for different sensors" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/QB/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV2/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/APNN/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/BDPN/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/DRPNN/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/DiCNN1/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/Download link for the 8 DL methods on WV3 dataset.txt", "content": "This folders contain the testing examples, including:\n\n1) \"Datasets Testing\": A full-resolution WV3-NewYork example + A reduced-resolution WV3-NewYork example\n\n2) \"QB\", \"WV2\", \"WV3\" and \"WV4\": Save the test datasets for different sensors" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/FusionNet/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/MSDCNN/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/PNN/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV3/PanNet/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/WV4/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/2_DL_Result/readme.txt", "content": "This folder cotains the outcomes of the 8 DL methods on QB, WV2, WV3 and WV4 sensors." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/3_EPS/QB/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/3_EPS/WV2/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/3_EPS/WV3/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/3_EPS/WV4/readme.txt", "content": "none" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/3_EPS/readme.txt", "content": "This folder stores the visual ouput with .eps format, which can be used in your latex editing." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/AWLP/AWLP.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% AWLP fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Additive Wavelet Luminance Proportional (AWLP) algorithm.\n% \n% Interface:\n% I_Fus_AWLP = AWLP(I_MS,I_PAN,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_AWLP: AWLP pasharpened image.\n% \n% References:\n% [Otazu05] X. Otazu, M. Gonzalez-Audcana, O. Fors, and J. Nunez, Introduction of sensor spectral response into image fusion methods.\n% Application to wavelet-based methods, IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 10, pp. 23762385,\n% October 2005.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Alparone17] L. Alparone, A. Garzelli, and G. Vivone, \"Intersensor statistical matching for pansharpening: Theoretical issues and practical solutions\",\n% IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 8, pp. 4682-4695, 2017.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_AWLP = AWLP(I_MS,I_PAN,ratio)\n\n[Height,Width,Bands]=size(I_MS);\nI_Fus_AWLP=zeros(Height,Width,Bands,'double');\n\nSumImage=sum(I_MS,3)/Bands;\n\nIntensityRatio = zeros(size(I_MS));\nfor i=1:Bands\n IntensityRatio(:,:,i)=I_MS(:,:,i)./(SumImage+eps);\nend\n\nI_PAN = repmat(I_PAN,[1 1 size(I_MS,3)]);\n\n% for ii = 1 : size(I_MS,3) \n% I_PAN(:,:,ii) = (I_PAN(:,:,ii) - mean2(I_PAN(:,:,ii))).*(std2(I_MS(:,:,ii))./std2(I_PAN(:,:,ii))) + mean2(I_MS(:,:,ii)); \n% end\nimageHR_LR=imresize(imresize(I_PAN,1/ratio),ratio);\nfor ii = 1 : size(I_MS,3)\n I_PAN(:,:,ii) = (I_PAN(:,:,ii) - mean2(I_PAN(:,:,ii))).*(std2(I_MS(:,:,ii))./std2(imageHR_LR(:,:,ii))) + mean2(I_MS(:,:,ii));\nend\n\nh=[1 4 6 4 1 ]/16;\ng=[0 0 1 0 0 ]-h;\nhtilde=[ 1 4 6 4 1]/16;\ngtilde=[ 0 0 1 0 0 ]+htilde;\nh=sqrt(2)*h;\ng=sqrt(2)*g;\nhtilde=sqrt(2)*htilde;\ngtilde=sqrt(2)*gtilde;\nWF={h,g,htilde,gtilde};\n\nLevels = ceil(log2(ratio));\n\nfor i=1:Bands\n WT = ndwt2_working(I_PAN(:,:,i),Levels,WF); \n for ii = 2 : numel(WT.dec), WT.dec{ii} = zeros(size(WT.dec{ii})); end\n StepDetails = I_PAN(:,:,i) - indwt2_working(WT,'c');\n%%%%%%%%% OLD [as in the article Otazu05]\n% sINI = WT.sizeINI;\n% \n% StepDetails = zeros(sINI);\n% \n% for ii = 2 : numel(WT.dec)\n% h = WT.dec{ii};\n% h = imcrop(h,[(size(h,1) - sINI(1))/2 + 1,(size(h,2) - sINI(2))/2 + 1, sINI(1) - 1, sINI(2) - 1]);\n% StepDetails = StepDetails + h; \n% end\n%%%%%%%%%%%%%%%%%%%\n I_Fus_AWLP(:,:,i) = StepDetails .* IntensityRatio(:,:,i)+I_MS(:,:,i);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Avg_RR_Assessment.tex", "content": "" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/BDSD/BDSD.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% BDSD fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Band-Dependent Spatial-Detail (BDSD) algorithm. \n% \n% Interface:\n% I_Fus_BDSD = BDSD(I_MS,I_PAN,ratio,S,sensor)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% S: Local estimation on SxS distinct blocks (typically 128x128); \n% sensor: String for type of sensor (e.g. 'WV2', 'IKONOS').\n%\n% Output:\n% I_Fus_BDSD: BDSD pansharpened image.\n% \n% References:\n% [Garzelli08] A. Garzelli, F. Nencini, and L. Capobianco, Optimal MMSE pan sharpening of very high resolution multispectral images, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 1, pp. 228236, January 2008.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_BDSD = BDSD(I_MS,I_PAN,ratio,S,sensor)\n\n%%%\n% Control of input parameters and initialization\n%%%\nif (S > 1)\n if(rem(S,2) && S >1)\n fprintf(1,'\\n\\n ');\n error('block size for local estimation must be even')\n end\n\n if(rem(S,ratio))\n fprintf(1,'\\n\\n ');\n error('block size must be multiple of ratio')\n end\n\n [N,M] = size(I_PAN);\n\n if(rem(N,S)||rem(M,S))\n fprintf(1,'\\n\\n ');\n error('x and y dims of pan must be multiple of the block size')\n end\nend\n\nI_MS = double(I_MS);\nI_PAN = double(I_PAN);\n\n%%%\n% Reduced resolution\n%%%\n\npan_LP = MTF_PAN(I_PAN,sensor,ratio);\npan_LP_d = pan_LP(3:ratio:end,3:ratio:end);\n\nms_orig = imresize(I_MS,1/ratio);\n\nms_LP_d = MTF(ms_orig,sensor,ratio);\n\n%%%\n% Parameter estimation at reduced resolution\n%%%\nin3 = cat(3,ms_LP_d,ms_orig,pan_LP_d);\nfun_eg = @(bs) estimate_gamma_cube(bs.data,S,ratio);\ngamma = blockproc(in3,[S/ratio S/ratio],fun_eg);\n\n%%%\n% Fusion\n%%%\nin3 = cat(3,I_MS,I_PAN,gamma);\nfun_Hi = @(bs) compH_inject(bs.data,S);\n\nI_Fus_BDSD = blockproc(in3,[S S],fun_Hi);\n\n%%%_______________________________________________________________\n%%%\nfunction gamma = estimate_gamma_cube(in3,S,ratio)\nNb = (size(in3,3)-1)/2;\nhs_LP_d = in3(:,:,1:Nb);\nhs_orig = in3(:,:,Nb+1:2*Nb);\npan_LP_d = in3(:,:,2*Nb+1);\n% Compute Hd\nHd = zeros(S*S/ratio/ratio,Nb+1);\nfor k=1:Nb\n b = hs_LP_d(:,:,k);\n Hd(:,k) = b(:);\nend\nHd(:,Nb+1) = pan_LP_d(:);\n% Estimate gamma\nB = (Hd'*Hd)\\Hd';\ngamma = zeros(Nb+1,Nb);\nfor k=1:Nb\n b = hs_orig(:,:,k);\n bd = hs_LP_d(:,:,k);\n gamma(:,k) = B *(b(:)-bd(:));\nend\ngamma = padarray(gamma,[S-Nb-1 S-Nb],0,'post');\n\n\n\n%%%_______________________________________________________________\n%%%\nfunction ms_en = compH_inject(in3,S)\nNb = size(in3,3)-2;\nhs = in3(:,:,1:Nb);\npan = in3(:,:,Nb+1);\ngamma = in3(:,:,Nb+2); \n% Compute H\n[N,M,Nb] = size(hs);\nH = zeros(S*S,Nb+1);\nfor k=1:Nb\n b = hs(:,:,k);\n H(:,k) = b(:);\nend\nH(:,Nb+1) = pan(:);\n% Inject\ng = gamma(1:Nb+1,1:Nb);\nms_en = zeros(N,M,Nb);\nfor k=1:Nb\n b = hs(:,:,k);\n b_en = b(:) + H * g(:,k);\n ms_en(:,:,k) = reshape(b_en,N,M);\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/BDSD/BDSD_PC.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% BDSD_PC fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Band-Dependent Spatial-Detail (BDSD) model solving an optimization constrained problem. \n% \n% Interface:\n% I_Fus_BDSD = BDSD_PC(I_MS,I_PAN,ratio,S,sensor)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% sensor: String for type of sensor (e.g. 'WV2', 'IKONOS').\n%\n% Output:\n% I_Fus_BDSD: BDSD_PC pansharpened image.\n% \n% Reference:\n% [Vivone19] G. Vivone, Robust Band-Dependent Spatial-Detail Approaches for Panchromatic Sharpening, \n% IEEE Transactions on Geoscience and Remote Sensing, 2019.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_BDSD = BDSD_PC(I_MS,I_PAN,ratio,sensor)\n\nI_MS = double(I_MS);\nI_PAN = double(I_PAN);\n\nopts1 = optimset('display','off');\n\nI_GT = imresize(I_MS,1/ratio);%,'nearest');\nI_MS_LR = MTF(I_GT,sensor,ratio);\nI_PAN_LR = imresize(MTF_PAN(I_PAN,sensor,ratio),1/ratio,'nearest');\n\nI_Fus_BDSD = zeros(size(I_MS));\ngamma = zeros(size(I_MS,3)+1,size(I_MS,3));\nfor ii = 1 : size(I_MS,3)\n h1 = I_GT(:,:,ii);\n h2 = I_MS_LR(:,:,ii);\n H = [I_PAN_LR(:), reshape(I_MS_LR,[size(I_MS_LR,1)*size(I_MS_LR,2), size(I_MS_LR,3)])];\n A = eye(size(I_MS,3)+1);\n A(1,1) = -1;\n\n gamma(:,ii) = lsqlin(H,h1(:)-h2(:),A,zeros(1,size(I_MS,3)+1),[],[],[],[],[],opts1);\n I_Fus_BDSD(:,:,ii) = I_MS(:,:,ii) + reshape([I_PAN(:),reshape(I_MS,[size(I_MS,1)*size(I_MS,2), size(I_MS,3)])]*gamma(:,ii),[size(I_MS,1) size(I_MS,2)]);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/BDSD/C_BDSD.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% C_BDSD fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images \n% through the Clustered Band-Dependent Spatial-Detail (C-BDSD) algorithm. \n% \n% Interface:\n% I_Fus_C_BDSD = C_BDSD(I_MS,I_PAN,ratio,sensor,K)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% sensor: String for type of sensor (e.g. 'WV2', 'IKONOS').\n% K: Number of clusters (K>1) (Optional: default value K=30); \n%\n% Outputs:\n% I_Fus_C_BDSD: C_BDSD pansharpened image.\n% \n% Reference:\n% [Garzelli15] A. Garzelli, Pansharpening of Multispectral Images Based on Nonlocal Parameter Optimization, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 4, pp. 2096-2107, April 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_C_BDSD = C_BDSD(I_MS,I_PAN,ratio,sensor,K)\n\n%%%\n% Control of input parameters and initialization\n%%%\n[N,M,Nb] = size(I_MS);\n\nif nargin == 5\n if K < 2\n fprintf(1,'Required number of clusters K>1.\\n\\n'); \n return\n end\nend\nif nargin < 5\n K = 30; \nend\nif nargin < 4\n fprintf(1,'\\nI_Fus_C_BDSD = C_BDSD(I_MS,I_PAN,ratio,sensor,K)\\n\\n');\n error('At least four input arguments required')\nend\n\nI_MS = double(I_MS);\nI_PAN = double(I_PAN);\n\n%%%\n% Reduced resolution\n%%%\n\npan_LP = MTF_PAN(I_PAN,sensor,ratio);\npan_LP_d = pan_LP(3:ratio:end,3:ratio:end);\n\nms_orig = imresize(I_MS,1/ratio);\nms_LP_d = MTF(ms_orig,sensor,ratio);\n\n\n% CLUSTER MAPS AT FULL RESOLUTION AND REDUCED RESOLUTION\n%\nSa = stdfilt(I_PAN,ones(51)); \nSa = Sa/max(Sa(:));\nSb = I_PAN; \nSb = Sb/max(Sb(:));\n \nopts = statset('TolX',1e-5);\n\nfeatures = zeros(N/ratio,M/ratio,2,ratio*ratio);\nfor i = 1:ratio\n for j = 1:ratio\n features(:,:,1,(i-1)*ratio+j) = Sa(1+(i-1):ratio:end,1+(j-1):ratio:end);\n features(:,:,2,(i-1)*ratio+j) = Sb(1+(i-1):ratio:end,1+(j-1):ratio:end);\n end\nend\nC_stack = zeros(N/ratio,M/ratio,ratio*ratio);\n\nf = features(:,:,:,(3-1)*ratio+3);\nwarning off\n[aux, centers] = kmeans(reshape(f,[N/ratio*M/ratio,2]),K,'replicates',2,'start','cluster','options',opts);\nC = reshape(aux,[N/ratio M/ratio]);\nC_stack(:,:,(3-1)*ratio+3) = C;\nfor i = 1:ratio\n for j = 1:ratio\n if(i*j~=9)\n f = features(:,:,:,(i-1)*ratio+j);\n aux = kmeans(reshape(f,[N/ratio*M/ratio,2]),K,'start',centers,'MaxIter',1);\n C_stack(:,:,(i-1)*ratio+j) = reshape(aux,[N/ratio M/ratio]);\n end\n end\nend\n\nC4 = zeros(size(I_PAN));\nfor i = 1:ratio\n for j = 1:ratio\n C4(i:ratio:end,j:ratio:end) = C_stack(:,:,(i-1)*ratio+j);\n end\nend\n\n% ESTIMATE PARAMETERS AT REDUCED RESOLUTION AND INJECT (CLUSTER BY CLUSTER)\n%\ng = zeros(K,Nb);\nalpha = zeros(Nb,Nb,K);\noffset = zeros(Nb,K);\nms_ps_stack = zeros(N,M,Nb,K);\n\n% Estimate for K=1\n[~,g_global,alpha_global,offset_global] = parm_est(ms_LP_d(:,:,:),pan_LP_d,ms_orig,find(C>0));\n\nfor j=1:K\n [~,g(j,:),alpha(:,:,j),offset(:,j)] = parm_est(ms_LP_d(:,:,:),pan_LP_d,ms_orig,find(C==j));\n if(size(find(g<0)>0))\n g(j,:) = g_global;\n alpha(:,:,j) = alpha_global;\n offset(:,j) = offset_global;\n end\n H = H_comp(I_PAN,I_MS,find(C4==j));\n ms_ps_stack(:,:,:,j) = bdsd_injection(I_PAN,I_MS,H,g(j,:),squeeze(alpha(:,:,j)),offset(:,j),find(C4==j));\nend\n\n% FORM PANSHARPENED IMAGE\nI_Fus_C_BDSD = sum(ms_ps_stack,4);\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\nfunction [gamma,g,alpha,offset] = parm_est(hs_LP_d,pan_LP_d,hs_orig,ind)\n\nNb = size(hs_orig,3);\n\nfor i=1:Nb\n % compute Hd\n Hd = zeros(size(ind,1),Nb+2);\n gamma = zeros((Nb+2),Nb);\n for k=1:Nb\n bfull = hs_LP_d(:,:,k);\n Hd(:,k) = bfull(ind);\n end\n Hd(:,Nb+1) = ones(size(ind));\n Hd(:,Nb+2) = pan_LP_d(ind);\n \n % estimate gamma\n \n for k=1:Nb\n Z = (Hd'*Hd)\\Hd';\n bfull = hs_orig(:,:,k);\n b = bfull(ind);\n bdfull = hs_LP_d(:,:,k);\n bd = bdfull(ind);\n gamma(:,k) = Z *(b(:)-bd(:));\n end\n \n g = gamma(Nb+2,:);\n \n alpha = zeros(Nb);\n for k = 1:Nb\n alpha(:,k) = -gamma(1:Nb,k)/gamma(Nb+2,k);\n end\n \n offset = zeros(Nb,1);\n for k = 1:Nb\n offset(k) = gamma(Nb+1,k)/gamma(Nb+2,k);\n end\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nfunction hs_en = bdsd_injection(pan,msexp,H,g,alpha,offset,ind)\n\n[N,M,Nb] = size(msexp);\nIntensity = zeros(length(ind),Nb);\nfor k = 1:Nb\n Intensity(:,k) = H(:,1:Nb) * alpha(:,k) - offset(k);\nend\npfull = pan;\np = pfull(ind);\n\nhs_en = zeros(N,M,Nb); \nfor k=1:Nb\n bfull = msexp(:,:,k);\n b = bfull(ind);\n b_en = b(:) + (p - Intensity(:,k)) * g(k);\n hs_enfull = hs_en(:,:,k);\n hs_enfull(ind) = b_en;\n hs_en(:,:,k) = reshape(hs_enfull,N,M);\nend\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\nfunction H = H_comp(pan,hs,ind)\n\nNb = size(hs,3);\nH = zeros(length(ind),Nb+2);\n\nfor k=1:Nb\n bfull = hs(:,:,k);\n H(:,k) = bfull(ind);\nend\nH(:,Nb+1) = ones(size(ind));\nH(:,Nb+2) = pan(ind);\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/BT-H/BroveyRegHazeMin.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Brovey data fusion with haze correction\n% \n% Interface:\n% I_Fus_Brovey_Reg = BroveyRegHazeMin(I_MS,I_PAN,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_Brovey_Reg: Pansharpened image.\n% \n% References:\n% [Lolli17] S. Lolli, L. Alparone, A. Garzelli, and G. Vivone, \"Haze correction for contrast-based multispectral pansharpening\",\n% IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 12, pp. 2255-2259, 2017.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_Brovey_Reg = BroveyRegHazeMin(I_MS,I_PAN,ratio)\n\nif size(I_MS,3) == 4\n prc = 1;\n minMS = zeros(1,1,4);\n B = I_MS(:,:,1);\n G = I_MS(:,:,2);\n R = I_MS(:,:,3);\n NIR = I_MS(:,:,4);\n minMS(1,1,1) = 0.95 * prctile(B(:),prc);\n minMS(1,1,2) = 0.45 * prctile(G(:),prc);\n minMS(1,1,3) = 0.40 * prctile(R(:),prc);\n minMS(1,1,4) = 0.05 * prctile(NIR(:),prc);\nelse\n minMS = zeros(1,1,size(I_MS,3));\n for ii = 1 : size(I_MS, 3)\n minMS(1,1,ii) = min(min(I_MS(:,:,ii))); \n end\nend\n\nL = repmat(minMS, [size(I_MS,1) size(I_MS,2)]);\n\nimageLR = double(I_MS);\nimageHR = double(I_PAN);\n\nimageHR_LR = LPfilterGauss(imageHR,ratio);\n\nh = estimation_alpha(imageLR,imageHR_LR,'global');\n\nalpha(1,1,:) = h;\n\nI = sum((imageLR - L) .* repmat(alpha,[size(I_MS,1) size(I_MS,2) 1]),3); \n\nimageHR = (imageHR - mean2(imageHR_LR)).*(std2(I)./std2(imageHR_LR)) + mean2(I); \n\nI_MS_L = imageLR - L;\nI_MS_L(I_MS_L < 0) = 0;\n\nI_Fus_Brovey_Reg = I_MS_L .* repmat(imageHR./(I+eps),[1 1 size(imageLR,3)]) + L;\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Demo_Full_Resolution.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%For FUll-Resolution%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% 1) This is a test demo to show all full-resolution results of traditional and DL methods\n% Here, we take WV3 test dataset as example. Readers can change the corresponding director \n% and setting to test other/your datasets\n% 2) The codes of traditional methods are from the \"pansharpening toolbox for distribution\",\n% thus please cite the paper:\n% [1] G. Vivone, et al., A new benchmark based on recent advances in multispectral pansharpening: Revisiting\n% pansharpening with classical and emerging pansharpening methods, IEEE Geosci. Remote Sens. Mag., \n% 9(1): 53C81, 2021\n% 3) Also, if you use this toolbox, please cite our paper:\n% [2] L.-J. Deng, et al., Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks, \n% IEEE Geosci. Remote Sens. Mag., 2022\n\n% LJ Deng (UESTC), 2020-02-27\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Note: the test dataset of full-resolution are too huge to upload to\n% GitHub, thus we provide cloud links to readers to download them to\n% successfully run this demo, including:\n\n% i) Download link for full-resolution WV3-NewYork example (named \"NY1_WV3_FR.mat\"):\n% http:******** (put into the folder of \"1_TestData/Datasets Testing\")\n\n% ii) Download link of DL's results for full-resolution WV3-NewYork example:\n% http:******** (put into the folder of \"'2_DL_Result/WV3\")\n\n% Once you have above datasets, you can run this demo successfully, then\n% understand how this demo run!\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nclear; close all;\n%% =======load directors========\n% Tools\naddpath([pwd,'/Tools']);\n\n% Select algorithms to run\nalgorithms = {'EXP','BT-H','BDSD-PC','C-GSA','SR-D',...\n 'MTF-GLP-HPM-R','MTF-GLP-FS','TV','PanNet','DRPNN','MSDCNN','BDPN','DiCNN','PNN','APNN','FusionNet'};\n\n% director to save EPS figures for latex editing; if other dataset, please\n% change the director correspondingly\ndata_name = '3_EPS/WV3/wv3_os_ny'; \n\n%% ==========Read Data and sensors' info====================\n%% read the test dataset; if use your test dataset, please update in this folder\nfile_test = '1_TestData/Datasets Testing/NY1_WV3_FR.mat';\n\n% get I_MS_LR, I_MS, I_PAN and sensors' info; \nload(file_test) \n\n% (Note: If there is no sensor's info in your dataset, \n% please find and update these info in the following commented lines):\n\n%------ following are sensor's info for WV3 (an example for WV3)----\n% sensor = 'WV3';\n% Qblocks_size = 32;\n% bicubic = 0;% Interpolator\n% flag_cut_bounds = 1;% Cut Final Image\n% dim_cut = 21;% Cut Final Image\n% thvalues = 0;% Threshold values out of dynamic range\n% printEPS = 0;% Print Eps\n% ratio = 4;% Resize Factor\n% L = 11;% Radiometric Resolution\n\n%% Initialization of the Matrix of Results\nNumIndexes = 3;\nMatrixResults = zeros(numel(algorithms),NumIndexes);\nalg = 0;\nflagQNR = 0; %% Flag QNR/HQNR, 1: QNR otherwise HQNR\n\n% zoom-in interesting two regions of figure; you may change them\n% according to your requirment\nlocation1 = [500 700 100 300]; %default: data6: [10 50 1 60]; data7:[140 180 5 60]\nlocation2 = [200 380 1000 1250]; %default: data6: [190 240 5 60]; data7:[190 235 120 150]\n\nclear print\n\n%% show I_MS_LR, I_GT, PAN Imgs:\nif size(I_MS,3) == 4\n showImage4LR(I_MS_LR,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\nelse\n showImage8LR(I_MS_LR,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\nend\n\n% (Note: If you only want to show pan image without region zoom-in, use showPan;\n% otherwise, use showPan_zoomin)\n\n%showPan(I_PAN,printEPS,2,flag_cut_bounds,dim_cut);\nshowPan_zoomin(I_PAN,printEPS,2,flag_cut_bounds,dim_cut, location1, location2);\n\n% Note: eps figure is saved in \"data_name\" for latex editing\nprint('-depsc', strcat(data_name, '_pan', '.eps')) \n\n%% ======EXP ===================\nif ismember('EXP',algorithms)\n alg = alg + 1;\n [D_lambda_EXP,D_S_EXP,QNRI_EXP] = indexes_evaluation_FS(I_MS,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_EXP,D_S_EXP,QNRI_EXP];\n MatrixImage(:,:,:,alg) = I_MS;\n \n % (Note: You may use following \"showImage8LR\" without region zoom-in; otherwise, you can\n % use \"showImage8_zoomin\" for zoom-in visualization.) \n \n %showImage8LR(I_MS,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_MS,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_exp.eps')) \nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%% CS-based Methods %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ====== 1) BT-H Method ======\nif ismember('BT-H',algorithms)\n alg = alg + 1;\n \n cd BT-H\n t2=tic;\n I_BT_H = BroveyRegHazeMin(I_MS,I_PAN,ratio);\n time_BT_H = toc(t2);\n fprintf('Elaboration time BT-H: %.2f [sec]\\n',time_BT_H);\n cd ..\n \n %%% Quality indexes computation\n [D_lambda_BT_H,D_S_BT_H,QNRI_BT_H] = indexes_evaluation_FS(I_BT_H,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_BT_H,D_S_BT_H,QNRI_BT_H];\n MatrixImage(:,:,:,alg) = I_BT_H;\n \n %showImage8LR(I_BT_H,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_BT_H,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_bth.eps'))\nend\n\n%% ====== 2) BDSD-PC Method ======\nif ismember('BDSD-PC',algorithms)\n alg = alg + 1;\n \n cd BDSD\n t2=tic;\n I_BDSD_PC = BDSD_PC(I_MS,I_PAN,ratio,sensor);\n time_BDSD_PC = toc(t2);\n fprintf('Elaboration time BDSD-PC: %.2f [sec]\\n',time_BDSD_PC);\n cd ..\n \n [D_lambda_BDSD_PC,D_S_BDSD_PC,QNRI_BDSD_PC] = indexes_evaluation_FS(I_BDSD_PC,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_BDSD_PC,D_S_BDSD_PC,QNRI_BDSD_PC];\n MatrixImage(:,:,:,alg) = I_BDSD_PC;\n \n %showImage8LR(I_BDSD_PC,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_BDSD_PC,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_bdsd_pc.eps'))\nend\n\n%% ====== 3) C-GSA Method ======\nif ismember('C-GSA',algorithms)\n alg = alg + 1;\n \n PS_algorithm = 'GSA'; % Pansharpening algorithm\n n_segm = 5; % Number of segments\n \n cd GS\n \n t2=tic;\n I_C_GSA = GS_Segm(I_MS,I_PAN,gen_LP_image(PS_algorithm,I_MS,I_PAN,I_MS_LR,ratio,sensor), k_means_clustering(I_MS,n_segm));\n time_C_GSA = toc(t2);\n fprintf('Elaboration time GSA: %.2f [sec]\\n',time_C_GSA);\n cd ..\n \n %%% Quality indexes computation\n [D_lambda_C_GSA,D_S_C_GSA,QNRI_C_GSA] = indexes_evaluation_FS(I_C_GSA,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_C_GSA,D_S_C_GSA,QNRI_C_GSA];\n MatrixImage(:,:,:,alg) = I_C_GSA;\n \n %showImage8LR(I_C_GSA,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_C_GSA,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_c_gsa.eps'))\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%% MRA-based Methods %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ====== 1) SR-D Method ======\nif ismember('SR-D',algorithms)\n alg = alg + 1;\n \n %%%%%%%%%%%%%%%%%%%%%%%%%% Parameters setting %%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n TS = 7; % Tiling (dimensions of the patches are TS x TS)\n ol = 4; % Overlap (in pixels) between contiguous tile\n n_atoms = 10; % Max number of representation atoms (default value = 10)\n %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n \n cd SR-D\n t2=tic;\n I_SR_D = CS(I_MS,I_PAN,I_MS_LR,ratio,sensor,TS,ol,n_atoms);\n time_SR_D = toc(t2);\n fprintf('Elaboration time SR_D: %.2f [sec]\\n',time_SR_D);\n cd ..\n \n [D_lambda_SR_D,D_S_SR_D,QNRI_SR_D] = indexes_evaluation_FS(I_SR_D,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_SR_D,D_S_SR_D,QNRI_SR_D];\n MatrixImage(:,:,:,alg) = I_SR_D;\n \n %showImage8LR(I_SR_D,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_SR_D,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_sr_d.eps'))\nend\n\n%% ====== 2) MTF-GLP-HPM-R Method ======\nif ismember('MTF-GLP-HPM-R',algorithms)\n alg = alg + 1;\n \n cd GLP\n t2=tic;\n I_MTF_GLP_HPM_R = MTF_GLP_HPM_R(I_MS,I_PAN,sensor,ratio);\n time_MTF_GLP_HPM_R = toc(t2);\n fprintf('Elaboration time MTF-GLP-HPM-R: %.2f [sec]\\n',time_MTF_GLP_HPM_R);\n cd ..\n \n [D_lambda_MTF_GLP_HPM_R,D_S_MTF_GLP_HPM_R,QNRI_MTF_GLP_HPM_R] = indexes_evaluation_FS(I_MTF_GLP_HPM_R,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_MTF_GLP_HPM_R,D_S_MTF_GLP_HPM_R,QNRI_MTF_GLP_HPM_R];\n MatrixImage(:,:,:,alg) = I_MTF_GLP_HPM_R;\n \n %showImage8LR(I_MTF_GLP_HPM_R,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_MTF_GLP_HPM_R,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_mtfglp_hpm_r.eps'))\nend\n\n%% ====== 3) MTF-GLP-FS Method ======\nif ismember('MTF-GLP-FS',algorithms)\n alg = alg + 1;\n \n cd GLP\n t2=tic;\n I_MTF_GLP_FS = MTF_GLP_FS(I_MS,I_PAN,sensor,ratio);\n time_MTF_GLP_FS = toc(t2);\n fprintf('Elaboration time MTF-GLP-FS: %.2f [sec]\\n',time_MTF_GLP_FS);\n cd ..\n \n %%% Quality indexes computation\n [D_lambda_MTF_GLP_FS,D_S_MTF_GLP_FS,QNRI_MTF_GLP_FS] = indexes_evaluation_FS(I_MTF_GLP_FS,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_MTF_GLP_FS,D_S_MTF_GLP_FS,QNRI_MTF_GLP_FS];\n MatrixImage(:,:,:,alg) = I_MTF_GLP_FS;\n \n %showImage8LR(I_MTF_GLP_FS,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_MTF_GLP_FS,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_mtfglpfs.eps'))\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%% VO-based Methods %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ====== 1) TV Method ======\nif ismember('TV',algorithms)\n alg = alg + 1;\n \n %%%%%%%%%%%%%%%%%%%%%%%%%% Parameters setting %%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n switch sensor\n case 'IKONOS'\n w=[0.1091 0.2127 0.2928 0.3854];\n c = 8;\n alpha=1.064;\n maxiter=10;\n lambda = 0.47106;\n case {'GeoEye1','WV4'}\n w=[0.1552, 0.3959, 0.2902, 0.1587];\n c = 8;\n alpha=0.75;\n maxiter=50;\n lambda = 157.8954;\n case 'WV3'\n w=[0.0657 0.1012 0.1537 0.1473 0.1245 0.1545 0.1338 0.1192];\n c = 8;\n alpha=0.75;\n maxiter=50;\n lambda = 1.0000e-03;\n end\n %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n cd TV\n t2 = tic;\n I_TV = TV_pansharpen(I_MS_LR,I_PAN,alpha,lambda,c,maxiter,w);\n time_TV = toc(t2);\n fprintf('Elaboration time TV: %.2f [sec]\\n',time_TV);\n cd ..\n \n %%% Quality indexes computation\n [D_lambda_TV,D_S_TV,QNRI_TV] = indexes_evaluation_FS(I_TV,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_TV,D_S_TV,QNRI_TV];\n MatrixImage(:,:,:,alg) = I_TV;\n \n %showImage8LR(I_TV,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_TV,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_tv.eps'))\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%% DL-based Methods %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ====== 1) PanNet Method ======\n% if you use other sensor's data, please update the following director and\n% DL result. Note that the DL results here are obtained from our \"01-DL toolbox (Pytorch)\" folder, please check it.\n% Similar operation for following other DL methods.\nfile_pannet = 'pannet_wv3_os_ny';\nload(strcat('2_DL_Result/WV3/PanNet/', file_pannet, '.mat')) \n\n% (Note: val_bit = 2047 for 11-bit WV3, WV4 and QB data; val_bit = 1023 for 10-bit GF2 data)\nval_bit = 2047;\nI_pannet = val_bit*double(pannet_wv3_os_ny); \n\nif ismember('PanNet',algorithms)\n alg = alg + 1;\n %%% Quality indexes computation\n [D_lambda_pannet,D_S_pannet,QNRI_pannet] = indexes_evaluation_FS(I_pannet,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_pannet,D_S_pannet,QNRI_pannet];\n MatrixImage(:,:,:,alg) = I_pannet;\n \n %showImage8LR(I_pannet,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_pannet,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_pannet.eps'))\nend\n\n%% ====== 2) DRPNN Method ======\nfile_drpnn = 'drpnn_wv3_os_ny';\nload(strcat('2_DL_Result/WV3/DRPNN/', file_drpnn, '.mat')) % load i-th image for DiCNN\nI_drpnn = val_bit*double(drpnn_wv3_os_ny);\n\nif ismember('DRPNN',algorithms)\n alg = alg + 1;\n %%% Quality indexes computation\n [D_lambda_drpnn,D_S_drpnn,QNRI_drpnn] = indexes_evaluation_FS(I_drpnn,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_drpnn,D_S_drpnn,QNRI_drpnn];\n MatrixImage(:,:,:,alg) = I_drpnn;\n \n %showImage8LR(I_drpnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_drpnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_drpnn.eps'))\nend\n\n%% ====== 3) MSDCNN Method ======\nfile_msdcnn = 'msdcnn_wv3_os_ny';\nload(strcat('2_DL_Result/WV3/MSDCNN/', file_msdcnn, '.mat')) % load i-th image for DiCNN\nI_msdcnn = val_bit*double(msdcnn_wv3_os_ny);\n\nif ismember('MSDCNN',algorithms)\n alg = alg + 1;\n %%% Quality indexes computation\n [D_lambda_msdcnn,D_S_msdcnn,QNRI_msdcnn] = indexes_evaluation_FS(I_msdcnn,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_msdcnn,D_S_msdcnn,QNRI_msdcnn];\n MatrixImage(:,:,:,alg) = I_msdcnn;\n \n %showImage8LR(I_msdcnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_msdcnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_msdcnn.eps'))\nend\n\n%% ====== 4) BDPN Method ======\nfile_bdpn = 'bdpn_wv3_os_ny';\nload(strcat('2_DL_Result/WV3/BDPN/', file_bdpn , '.mat')) % load i-th image for DiCNN\nI_bdpn = val_bit*double(bdpn_wv3_os_ny);\n\nif ismember('BDPN',algorithms)\n alg = alg + 1;\n %%% Quality indexes computation\n [D_lambda_bdpn,D_S_bdpn,QNRI_bdpn] = indexes_evaluation_FS(I_bdpn,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_bdpn,D_S_bdpn,QNRI_bdpn];\n MatrixImage(:,:,:,alg) = I_bdpn;\n \n %showImage8LR(I_bdpn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_bdpn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_bdpn.eps'))\nend\n\n%% ====== 5) DiCNN Method ======\nfile_dicnn = 'dicnn_wv3_os_ny';\nload(strcat('2_DL_Result/WV3/DiCNN/', file_dicnn, '.mat')) % load i-th image for DiCNN\nI_dicnn = val_bit*double(dicnn_wv3_os_ny);\n\nif ismember('DiCNN',algorithms)\n alg = alg + 1;\n %%% Quality indexes computation\n [D_lambda_dicnn,D_S_dicnn,QNRI_dicnn] = indexes_evaluation_FS(I_dicnn,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_dicnn,D_S_dicnn,QNRI_dicnn];\n MatrixImage(:,:,:,alg) = I_dicnn;\n \n %showImage8LR(I_dicnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_dicnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_dicnn.eps'))\nend\n\n%% ====== 6) PNN Method ======\nfile_pnn = 'pnn_wv3_os_ny';\nload(strcat('2_DL_Result/WV3/PNN/', file_pnn, '.mat')) % load i-th image for DiCNN\nI_pnn = val_bit*double(pnn_wv3_os_ny);\n\nif ismember('PNN',algorithms)\n alg = alg + 1;\n %%% Quality indexes computation\n [D_lambda_pnn,D_S_pnn,QNRI_pnn] = indexes_evaluation_FS(I_pnn,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_pnn,D_S_pnn,QNRI_pnn];\n MatrixImage(:,:,:,alg) = I_pnn;\n \n %showImage8LR(I_pnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_pnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_pnn.eps'))\nend\n\n%% ====== 7) APNN Method ====== (not true APNN, just a replacement!!)\n file_apnn = 'apnn_wv3_os_ny';\n load(strcat('2_DL_Result/WV3/APNN/', file_apnn, '.mat')) % load i-th image for DiCNN\n I_apnn = val_bit*double(apnn_wv3_os_ny); % not right answer, just a replacement!\n\nif ismember('APNN',algorithms)\n alg = alg + 1;\n %%% Quality indexes computation\n [D_lambda_apnn,D_S_apnn,QNRI_apnn] = indexes_evaluation_FS(I_apnn,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_apnn,D_S_apnn,QNRI_apnn];\n MatrixImage(:,:,:,alg) = I_apnn;\n \n %showImage8LR(I_apnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_apnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_apnn.eps'))\nend\n\n%% ====== 8) FusionNet Method ======\nfile_fusionnet = 'fusionnet_wv3_os_ny';\nload(strcat('2_DL_Result/WV3/FusionNet/', file_fusionnet, '.mat')) % load i-th image for DiCNN\nI_fusionnet = val_bit*double(fusionnet_wv3_os_ny);\n\nif ismember('FusionNet',algorithms)\n alg = alg + 1;\n %%% Quality indexes computation\n [D_lambda_fusionnet,D_S_fusionnet,QNRI_fusionnet] = indexes_evaluation_FS(I_fusionnet,I_MS_LR,I_PAN,L,thvalues,I_MS,sensor,ratio,flagQNR);\n MatrixResults(alg,:) = [D_lambda_fusionnet,D_S_fusionnet,QNRI_fusionnet];\n MatrixImage(:,:,:,alg) = I_fusionnet;\n \n %showImage8LR(I_fusionnet,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_fusionnet,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_fusionnet.eps'))\nend\n\n%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% %%%%%%%%%%% Show and Save Results %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Print in LATEX\nif flagQNR == 1\n matrix2latex(MatrixResults,'FR_Assessment.tex', 'rowLabels',algorithms,'columnLabels',[{'DL'},{'DS'},{'QNR'}],'alignment','c','format', '%.4f');\nelse\n matrix2latex(MatrixResults,'FR_Assessment.tex', 'rowLabels',algorithms,'columnLabels',[{'DL'},{'DS'},{'HQNR'}],'alignment','c','format', '%.4f');\nend\n\n%% View All\nif size(I_MS,3) == 4\n vect_index_RGB = [3,2,1];\nelse\n vect_index_RGB = [5,3,2];\nend\n\ntitleImages = algorithms;\nfigure, showImagesAll(MatrixImage,titleImages,vect_index_RGB,flag_cut_bounds,dim_cut,0);\n\n%% ======Display the final average performance =======\nfprintf('\\n')\ndisp('#######################################################')\ndisp(['Display the performance for:'])\ndisp('#######################################################')\ndisp(' |====Q====|===Q_avg===|=====SAM=====|======ERGAS=======|=======SCC=======')\nMatrixResults\n\n%% %%%%%%%%%%% End %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Demo_Reduced_Resolution.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%For Reduced-Resolution%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% 1) This is a test demo to show all reduced-resolution results of traditional and DL methods\n% Here, we take WV3 test dataset as example. Readers can change the corresponding director \n% and setting to test other/your datasets\n% 2) The codes of traditional methods are from the \"pansharpening toolbox for distribution\",\n% thus please cite the paper:\n% [1] G. Vivone, et al., A new benchmark based on recent advances in multispectral pansharpening: Revisiting\n% pansharpening with classical and emerging pansharpening methods, IEEE Geosci. Remote Sens. Mag., \n% 9(1): 53�C81, 2021\n% 3) Also, if you use this toolbox, please cite our paper:\n% [2] L.-J. Deng, et al., Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks, \n% IEEE Geosci. Remote Sens. Mag., 2022\n\n% LJ Deng (UESTC), 2020-02-27\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Note: the test datasets of reduced-resolution are still too huge to upload to\n% GitHub, thus we provide cloud links to readers to download them to\n% successfully run this demo, including:\n\n% i) Download link for reduced-resolution WV3-NewYork example (named \"NY1_WV3_RR.mat\"):\n% http:******** (put into the folder of \"1_TestData/Datasets Testing\")\n\n% ii) Download link of DL's results for reduced-resolution WV3-NewYork example:\n% http:******** (put into the folder of \"'2_DL_Result/WV3\")\n\n% Once you have above datasets, you can run this demo successfully, then\n% understand how this demo run!\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nclear; close all;\n%% =======load directors========\n% Tools\naddpath([pwd,'/Tools']);\n\n% Select algorithms to run\nalgorithms = {'GT','EXP','BT-H','BDSD-PC','C-GSA','SR-D',...\n 'MTF-GLP-HPM-R','MTF-GLP-FS','TV','PanNet','DRPNN','MSDCNN','BDPN','DiCNN1','PNN','APNN','FusionNet'};\n% director to save EPS figures for latex editing; if other dataset, please\n% change the director correspondingly\nsatellite = 'WV3';\nmat_name = 'NY1_WV3_RR';\ndata_name = strcat('3_EPS/', satellite, '/', mat_name); \n\n%% ==========Read Data and sensors' info====================\n%% read the test dataset; if use your test dataset, please update in this folder\nfile_test = '1_TestData/Datasets Testing/NY1_WV3_RR.mat';\n\n% get I_MS_LR, I_MS, I_PAN and sensors' info; \nload(file_test) \n\n% (Note: If there is no sensor's info in your dataset, \n% please find and update these info in the following commented lines):\n\n%------ following are sensor's info for WV3 (an example for WV3)----\n% sensor = 'WV3';\n% Qblocks_size = 32;\n% bicubic = 0;% Interpolator\n% flag_cut_bounds = 1;% Cut Final Image\n% dim_cut = 21;% Cut Final Image\n% thvalues = 0;% Threshold values out of dynamic range\n% printEPS = 0;% Print Eps\n% ratio = 4;% Resize Factor\n% L = 11;% Radiometric Resolution\n\n%% Initialization of the Matrix of Results\nNumIndexes = 5;\nMatrixResults = zeros(numel(algorithms),NumIndexes);\nalg = 0;\n\n% zoom-in interesting two regions of figure; you may change them\n% according to your requirment\nlocation1 = [50 70 10 30]; %default: data6: [10 50 1 60]; data7:[140 180 5 60]\nlocation2 = [20 38 10 50]; %default: data6: [190 240 5 60]; data7:[190 235 120 150]\n\nclear print\n\n%% show I_MS_LR, I_GT, PAN Imgs:\nif size(I_MS,3) == 4\n showImage4LR(I_MS_LR,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\nelse\n showImage8LR(I_MS_LR,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\nend\n\n% (Note: You may use following \"showPan\" without region zoom-in; otherwise, you can\n% use \"showPan_zoomin\" for zoom-in visualization.)\n\n%showPan(I_PAN,printEPS,2,flag_cut_bounds,dim_cut);\nshowPan_zoomin(I_PAN,printEPS,2,flag_cut_bounds,dim_cut, location1, location2);\n\n% Note: eps figure is saved in \"data_name\" for latex editing\nprint('-depsc', strcat(data_name, '_pan', '.eps'))\n\n%% ======GT ===================\nif ismember('GT',algorithms)\n alg = alg + 1;\n [Q_avg_GT, SAM_GT, ERGAS_GT, SCC_GT_GT, Q_GT] = indexes_evaluation(I_GT,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_GT,Q_avg_GT,SAM_GT,ERGAS_GT,SCC_GT_GT];\n MatrixImage(:,:,:,alg) = I_GT;\n \n % (Note: You may use following \"showImage8LR\" without region zoom-in; otherwise, you can\n % use \"showImage8_zoomin\" for zoom-in visualization.) \n \n %showImage8LR(I_GT,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_GT,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_gt', '.eps'))\nend\n\n%% ======EXP ===================\nif ismember('EXP',algorithms)\n alg = alg + 1;\n [Q_avg_EXP, SAM_EXP, ERGAS_EXP, SCC_GT_EXP, Q_EXP] = indexes_evaluation(I_MS,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_EXP,Q_avg_EXP,SAM_EXP,ERGAS_EXP,SCC_GT_EXP];\n MatrixImage(:,:,:,alg) = I_MS;\n \n %showImage8LR(I_MS,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_MS,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_exp.eps'))\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%% CS-based Methods %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ====== 1) BT-H Method ======\nif ismember('BT-H',algorithms)\n alg = alg + 1;\n \n cd BT-H\n t2=tic;\n I_BT_H = BroveyRegHazeMin(I_MS,I_PAN,ratio);\n time_BT_H = toc(t2);\n fprintf('Elaboration time BT-H: %.2f [sec]\\n',time_BT_H);\n cd ..\n \n %%% Quality indexes computation\n [Q_avg_BT_H, SAM_BT_H, ERGAS_BT_H, SCC_GT_BT_H, Q_BT_H] = indexes_evaluation(I_BT_H,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_BT_H,Q_avg_BT_H,SAM_BT_H,ERGAS_BT_H,SCC_GT_BT_H];\n MatrixImage(:,:,:,alg) = I_BT_H;\n\n %showImage8LR(I_BT_H,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_BT_H,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_bth.eps'))\nend\n\n%% ====== 2) BDSD-PC Method ======\nif ismember('BDSD-PC',algorithms)\n alg = alg + 1;\n \n cd BDSD\n t2=tic;\n I_BDSD_PC = BDSD_PC(I_MS,I_PAN,ratio,sensor);\n time_BDSD_PC = toc(t2);\n fprintf('Elaboration time BDSD-PC: %.2f [sec]\\n',time_BDSD_PC);\n cd ..\n \n [Q_avg_BDSD_PC, SAM_BDSD_PC, ERGAS_BDSD_PC, SCC_GT_BDSD_PC, Q_BDSD_PC] = indexes_evaluation(I_BDSD_PC,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n \n MatrixResults(alg,:) = [Q_BDSD_PC,Q_avg_BDSD_PC,SAM_BDSD_PC,ERGAS_BDSD_PC,SCC_GT_BDSD_PC];\n MatrixImage(:,:,:,alg) = I_BDSD_PC;\n \n %showImage8LR(I_BDSD_PC,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_BDSD_PC,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_bdsd_pc.eps'))\nend\n\n%% ====== 3) C-GSA Method ======\nif ismember('C-GSA',algorithms)\n alg = alg + 1;\n \n PS_algorithm = 'GSA'; % Pansharpening algorithm\n n_segm = 5; % Number of segments\n \n cd GS\n \n t2=tic;\n I_C_GSA = GS_Segm(I_MS,I_PAN,gen_LP_image(PS_algorithm,I_MS,I_PAN,I_MS_LR,ratio,sensor), k_means_clustering(I_MS,n_segm));\n time_C_GSA = toc(t2);\n fprintf('Elaboration time GSA: %.2f [sec]\\n',time_C_GSA);\n cd ..\n \n [Q_avg_C_GSA, SAM_C_GSA, ERGAS_C_GSA, SCC_GT_C_GSA, Q_C_GSA] = indexes_evaluation(I_C_GSA,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_C_GSA,Q_avg_C_GSA,SAM_C_GSA,ERGAS_C_GSA,SCC_GT_C_GSA];\n MatrixImage(:,:,:,alg) = I_C_GSA;\n\n %showImage8LR(I_C_GSA,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_C_GSA,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_C_gsa.eps'))\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%% MRA-based Methods %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ====== 1) SR-D Method ======\nif ismember('SR-D',algorithms)\n alg = alg + 1;\n \n %%%%%%%%%%%%%%%%%%%%%%%%%% Parameters setting %%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n TS = 7; % Tiling (dimensions of the patches are TS x TS)\n ol = 4; % Overlap (in pixels) between contiguous tile\n n_atoms = 10; % Max number of representation atoms (default value = 10)\n %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n \n cd SR-D\n t2=tic;\n I_SR_D = CS(I_MS,I_PAN,I_MS_LR,ratio,sensor,TS,ol,n_atoms);\n time_SR_D = toc(t2);\n fprintf('Elaboration time SR_D: %.2f [sec]\\n',time_SR_D);\n cd ..\n \n [Q_avg_SR_D, SAM_SR_D, ERGAS_SR_D, SCC_GT_SR_D, Q_SR_D] = indexes_evaluation(I_SR_D,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_SR_D,Q_avg_SR_D,SAM_SR_D,ERGAS_SR_D,SCC_GT_SR_D];\n MatrixImage(:,:,:,alg) = I_SR_D;\n\n %showImage8LR(I_SR_D,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_SR_D,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_SR_D.eps'))\nend\n\n\n%% ====== 2) MTF-GLP Method ======\nif ismember('MTF-GLP-HPM-R',algorithms)\n alg = alg + 1;\n \n cd GLP\n t2=tic;\n I_MTF_GLP_HPM_R = MTF_GLP_HPM_R(I_MS,I_PAN,sensor,ratio);\n time_MTF_GLP_HPM_R = toc(t2);\n fprintf('Elaboration time MTF-GLP-HPM-R: %.2f [sec]\\n',time_MTF_GLP_HPM_R);\n cd ..\n \n [Q_avg_MTF_GLP_HPM_R, SAM_MTF_GLP_HPM_R, ERGAS_MTF_GLP_HPM_R, SCC_GT_MTF_GLP_HPM_R, Q_MTF_GLP_HPM_R] = indexes_evaluation(I_MTF_GLP_HPM_R,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_MTF_GLP_HPM_R,Q_avg_MTF_GLP_HPM_R,SAM_MTF_GLP_HPM_R,ERGAS_MTF_GLP_HPM_R,SCC_GT_MTF_GLP_HPM_R];\n MatrixImage(:,:,:,alg) = I_MTF_GLP_HPM_R;\n \n %showImage8LR(I_MTF_GLP_HPM_R,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_MTF_GLP_HPM_R,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_mtfglp_hpm_r.eps'))\nend\n\n%% ====== 3) MTF-GLP-FS Method ======\nif ismember('MTF-GLP-FS',algorithms)\n alg = alg + 1;\n \n cd GLP\n t2=tic;\n I_MTF_GLP_FS = MTF_GLP_FS(I_MS,I_PAN,sensor,ratio);\n time_MTF_GLP_FS = toc(t2);\n fprintf('Elaboration time MTF-GLP-FS: %.2f [sec]\\n',time_MTF_GLP_FS);\n cd ..\n \n [Q_avg_MTF_GLP_FS, SAM_MTF_GLP_FS, ERGAS_MTF_GLP_FS, SCC_GT_MTF_GLP_FS, Q_MTF_GLP_FS] = indexes_evaluation(I_MTF_GLP_FS,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_MTF_GLP_FS,Q_avg_MTF_GLP_FS,SAM_MTF_GLP_FS,ERGAS_MTF_GLP_FS,SCC_GT_MTF_GLP_FS];\n MatrixImage(:,:,:,alg) = I_MTF_GLP_FS;\n \n %showImage8LR(I_MTF_GLP_FS,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_MTF_GLP_FS,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_mtfglpfs.eps'))\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%% VO-based Methods %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ====== 1) TV Method ======\nif ismember('TV',algorithms)\n alg = alg + 1;\n \n %%%%%%%%%%%%%%%%%%%%%%%%%% Parameters setting %%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n switch sensor\n case 'IKONOS'\n w=[0.1091 0.2127 0.2928 0.3854];\n c = 8;\n alpha=1.064;\n maxiter=10;\n lambda = 0.47106;\n case {'GeoEye1','WV4'}\n w=[0.1552, 0.3959, 0.2902, 0.1587];\n c = 8;\n alpha=0.75;\n maxiter=50;\n lambda = 157.8954;\n case 'WV3'\n w=[0.0657 0.1012 0.1537 0.1473 0.1245 0.1545 0.1338 0.1192];\n c = 8;\n alpha=0.75;\n maxiter=50;\n lambda = 1.0000e-03;\n end\n %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n \n cd TV\n t2 = tic;\n I_TV = TV_pansharpen(I_MS_LR,I_PAN,alpha,lambda,c,maxiter,w);\n time_TV = toc(t2);\n fprintf('Elaboration time TV: %.2f [sec]\\n',time_TV);\n cd ..\n \n [Q_avg_TV, SAM_TV, ERGAS_TV, SCC_GT_TV, Q_TV] = indexes_evaluation(I_TV,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_TV,Q_avg_TV,SAM_TV,ERGAS_TV,SCC_GT_TV];\n MatrixImage(:,:,:,alg) = I_TV;\n\n %showImage8LR(I_TV,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_TV,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_tv.eps'))\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%% DL-based Methods %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ====== 1) PanNet Method ======\n% if you use other sensor's data, please update the following director and\n% DL result. Note that the DL results here are obtained from our \"01-DL toolbox (Pytorch)\" folder, please check it.\n% Similar operation for following other DL methods.\n\n\n% (Note: val_bit = 2047 for 11-bit WV3, WV4 and QB data; val_bit = 1023 for 10-bit GF2 data)\nval_bit = 2047;\n\n\nif ismember('PanNet',algorithms)\n% file_pannet = 'output';\n load(strcat('2_DL_Result/', satellite, '/PanNet/', 'output_', mat_name, '.mat')) \n I_pannet = double(sr);\n alg = alg + 1;\n [Q_avg_pannet, SAM_pannet, ERGAS_pannet, SCC_pannet, Q_pannet] = indexes_evaluation(I_pannet,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_pannet,Q_avg_pannet,SAM_pannet,ERGAS_pannet,SCC_pannet];\n MatrixImage(:,:,:,alg) = I_pannet;\n \n %showImage8LR(I_pannet,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_pannet,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_pannet.eps'))\nend\n\n%% ====== 2) DRPNN Method ======\nif ismember('DRPNN',algorithms)\n% file_drpnn = 'drpnn_wv3_rs_ny';\n load(strcat('2_DL_Result/', satellite, '/DRPNN/', 'output_', mat_name, '.mat')) \n I_drpnn = double(sr);\n alg = alg + 1;\n [Q_avg_drpnn, SAM_drpnn, ERGAS_drpnn, SCC_drpnn, Q_drpnn] = indexes_evaluation(I_drpnn,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_drpnn,Q_avg_drpnn,SAM_drpnn,ERGAS_drpnn,SCC_drpnn];\n MatrixImage(:,:,:,alg) = I_drpnn;\n \n %showImage8LR(I_drpnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_drpnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_drpnn.eps'))\nend\n\n%% ====== 3) MSDCNN Method ======\n\nif ismember('MSDCNN',algorithms)\n% file_msdcnn = 'msdcnn_wv3_rs_ny';\n load(strcat('2_DL_Result/', satellite, '/MSDCNN/', 'output_', mat_name, '.mat')) \n I_msdcnn = double(sr);\n alg = alg + 1;\n [Q_avg_msdcnn, SAM_msdcnn, ERGAS_msdcnn, SCC_msdcnn, Q_msdcnn] = indexes_evaluation(I_msdcnn,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_msdcnn,Q_avg_msdcnn,SAM_msdcnn,ERGAS_msdcnn,SCC_msdcnn];\n MatrixImage(:,:,:,alg) = I_msdcnn;\n \n %showImage8LR(I_msdcnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_msdcnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_msdcnn.eps'))\nend\n\n%% ====== 4) BDPN Method ======\nif ismember('BDPN',algorithms)\n% file_bdpn = 'bdpn_wv3_rs_ny';\n load(strcat('2_DL_Result/', satellite, '/BDPN/', 'output_', mat_name, '.mat')) \n I_bdpn = double(sr);\n alg = alg + 1;\n [Q_avg_bdpn, SAM_bdpn, ERGAS_bdpn, SCC_bdpn, Q_bdpn] = indexes_evaluation(I_bdpn,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_bdpn,Q_avg_bdpn,SAM_bdpn,ERGAS_bdpn,SCC_bdpn];\n MatrixImage(:,:,:,alg) = I_bdpn;\n \n %showImage8LR(I_bdpn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_bdpn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_bdpn.eps'))\nend\n\n%% ====== 5) DiCNN Method ======\n\n\nif ismember('DiCNN1',algorithms)\n% file_dicnn = 'dicnn_wv3_rs_ny';\n load(strcat('2_DL_Result/', satellite, '/DiCNN1/', 'output_', mat_name, '.mat')) \n I_dicnn = double(sr);\n alg = alg + 1;\n [Q_avg_dicnn, SAM_dicnn, ERGAS_dicnn, SCC_dicnn, Q_dicnn] = indexes_evaluation(I_dicnn,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_dicnn,Q_avg_dicnn,SAM_dicnn,ERGAS_dicnn,SCC_dicnn];\n MatrixImage(:,:,:,alg) = I_dicnn;\n \n %showImage8LR(I_dicnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_dicnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_dicnn.eps'))\nend\n\n%% ====== 6) PNN Method ======\nif ismember('PNN',algorithms)\n% file_pnn = 'pnn_wv3_rs_ny';\n load(strcat('2_DL_Result/', satellite ,'/PNN/', 'output_', mat_name, '.mat')) \n I_pnn = double(sr);\n alg = alg + 1;\n [Q_avg_pnn, SAM_pnn, ERGAS_pnn, SCC_pnn, Q_pnn] = indexes_evaluation(I_pnn,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_pnn,Q_avg_pnn,SAM_pnn,ERGAS_pnn,SCC_pnn];\n MatrixImage(:,:,:,alg) = I_pnn;\n \n %showImage8LR(I_pnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_pnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_pnn.eps'))\nend\n\n%% ====== 7) APNN Method ======\nif ismember('APNN',algorithms)\n% file_apnn = 'apnn_wv3_rs_ny';\n load(strcat('2_DL_Result/', satellite, '/APNN/', 'output_', mat_name, '.mat')) \n I_apnn = double(sr);\n alg = alg + 1;\n [Q_avg_apnn, SAM_apnn, ERGAS_apnn, SCC_apnn, Q_apnn] = indexes_evaluation(I_apnn,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_apnn,Q_avg_apnn,SAM_apnn,ERGAS_apnn,SCC_apnn];\n MatrixImage(:,:,:,alg) = I_apnn;\n \n %showImage8LR(I_apnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_apnn,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_apnn.eps'))\nend\n\n%% ====== 8) FusionNet Method ======\n\nif ismember('FusionNet',algorithms)\n% file_fusionnet = 'fusionnet_wv3_rs_ny';\n% load(strcat('2_DL_Result/', satellite ,'/FusionNet/', 'output_',mat_name, '.mat'));\n load(strcat('2_DL_Result/', satellite ,'/FusionNet/', 'fusionnet_wv3_rs_ny', '.mat'));\n I_fusionnet = val_bit * double(sr);\n alg = alg + 1;\n [Q_avg_fusionnet, SAM_fusionnet, ERGAS_fusionnet, SCC_fusionnet, Q_fusionnet] = indexes_evaluation(I_fusionnet,I_GT,ratio,L,Qblocks_size,flag_cut_bounds,dim_cut,thvalues);\n MatrixResults(alg,:) = [Q_fusionnet,Q_avg_fusionnet,SAM_fusionnet,ERGAS_fusionnet,SCC_fusionnet];\n MatrixImage(:,:,:,alg) = I_fusionnet;\n \n %showImage8LR(I_fusionnet,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L,ratio);\n showImage8_zoomin(I_fusionnet,printEPS,1,flag_cut_bounds,dim_cut,thvalues,L, location1, location2);\n print('-depsc', strcat(data_name, '_fusionnet.eps'))\nend\n\n%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% %%%%%%%%%%% Show and Save Results %%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Print in LATEX\nmatrix2latex(MatrixResults(:,[1,3,4]),'RR_Assessment.tex', 'rowLabels',algorithms,'columnLabels',[{'Q2n'},{'SAM'},{'ERGAS'}],'alignment','c','format', '%.4f');\n\n%% View All\nif size(I_GT,3) == 4\n vect_index_RGB = [3,2,1];\nelse\n vect_index_RGB = [5,3,2];\nend\n\ntitleImages = algorithms;\nfigure, showImagesAll(MatrixImage,titleImages,vect_index_RGB,flag_cut_bounds,dim_cut,0);\n\n%% ======Display the final average performance =======\nfprintf('\\n')\ndisp('#######################################################')\ndisp(['Display the performance for:'])\ndisp('#######################################################')\ndisp(' |====Q====|===Q_avg===|=====SAM=====|======ERGAS=======|=======SCC=======')\nMatrixResults\n\n%% %%%%%%%%%%% End %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/FE-HPM/FE.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% FE estimates the estraction detail filter via deconvolution. \n% \n% Interface:\n% PSF_l = FE(I_MS,I_PAN,ratio,tap,lambda,mu,th,num_iter,filtername)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n% tap: Filter support;\n% lambda: Coefficient for weighting the energy regularization term;\n% mu: Coefficient for weighting the derivative regularization terms;\n% th: Threshold on the kernel (it cuts to 0 values below threshold);\n% num_iter: Max number of iteration (at least 3; not sensitive); \n% filtername: Kind of derivative (default: 'Basic') \n%\n% Output:\n% PSF_l: Estimated point spread function.\n% \n% Reference:\n% [Vivone15] G. Vivone, M. Simoes, M. Dalla Mura, R. Restaino, J. Bioucas-Dias, G. A. Licciardi, and J. Chanussot, \"Pansharpening based on semiblind deconvolution\", \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 4, pp. 1997-2010, 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction PSF_l = FE(I_MS,I_PAN,ratio,tap,lambda,mu,th,num_iter,filtername)\n\nif rem(tap,2) == 0\n sum_tap = 0;\nelse\n sum_tap = 1;\nend\ntap = floor(tap/2);\n\n[R_SIZE,C_SIZE] = size(I_PAN);\n\nswitch filtername\n case 'Naive2'\n gv = zeros(2,1);\n gv(1,1) = -1;\n gv(2,1) = 1;\n \n gh = zeros(1,2);\n gh(1,1) = -1;\n gh(1,2) = 1;\n case 'Naive3'\n gv = zeros(3,1);\n gv(1,1) = -1;\n gv(3,1) = 1;\n \n gh = zeros(1,3);\n gh(1,1) = -1;\n gh(1,3) = 1;\n case 'Basic'\n gv = zeros(2,2);\n gv(1,:) = -1;\n gv(2,:) = 1;\n \n gh = zeros(2,2);\n gh(:,1) = -1;\n gh(:,2) = 1;\n case 'Prewitt'\n gv = zeros(3,3);\n gv(1,:) = -1;\n gv(3,:) = 1;\n \n gh = zeros(3,3);\n gh(:,1) = -1;\n gh(:,3) = 1;\n case 'Sobel'\n gv = zeros(3,3);\n gv(1,1) = -1;gv(1,2) = -2;gv(1,3) = -1;\n gv(3,1) = +1;gv(3,2) = +2;gv(3,3) = +1;\n \n gh = zeros(3,3);\n gh(1,1) = -1;gh(2,1) = -2;gh(3,1) = -1;\n gh(1,3) = +1;gh(2,3) = +2;gh(3,3) = +1;\n otherwise\n gv = zeros(2,2);\n gv(1,:) = -1;\n gv(2,:) = 1;\n \n gh = zeros(2,2);\n gh(:,1) = -1;\n gh(:,2) = 1;\nend\n\ngvf = fft2(gv,R_SIZE,C_SIZE);\nghf = fft2(gh,R_SIZE,C_SIZE);\n\ngvfc = conj(gvf);\nghfc = conj(ghf);\n\ngvf2 = gvfc .* gvf;\nghf2 = ghfc .* ghf;\n\ngf2sum = gvf2 + ghf2;\n\nH_E = double(I_PAN);\n\nfor jj = 1 : num_iter\n \n %%% Filter PAN to estimate alpha set\n if jj == 1\n PAN_LP = LPfilter(H_E,ratio);\n else\n PAN_LP = imfilter(H_E,PSF_l,'replicate');\n end\n \n %%% Estimate alpha\n alpha(1,1,:) = estimation_alpha(cat(3,I_MS,ones(size(I_MS,1),size(I_MS,2))),PAN_LP,'global');\n \n It_E = sum(cat(3,I_MS,ones(size(I_MS,1),size(I_MS,2))) .* repmat(alpha,[size(I_MS,1) size(I_MS,2) 1]),3); \n\n %%% Edge taper\n H_E = edgetaper(H_E,ones(tap,tap)./((tap)^2));\n It_E = edgetaper(It_E,ones(tap,tap)./((tap)^2));\n\n %%% Filter Estimation\n PSF = real(fftshift(ifft2(conj(fft2(H_E)).* fft2(It_E)./(abs(fft2(H_E)).^2 + lambda + mu * gf2sum ))));\n \n %%% Thresholding\n PSF(PSF < th) = 0;\n \n %%% Cut using the support dimension and center\n [~, maxIndex] = max(PSF(:));\n [rm, cm] = ind2sub(size(PSF), maxIndex);\n PSF_l = PSF(rm - tap : rm + tap - 1 + sum_tap, cm - tap : cm + tap - 1 + sum_tap);\n PSF_l = PSF_l ./ sum(PSF_l(:));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/FE-HPM/FE_HPM.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% FE_HPM fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the high pass modulation injection model and the estimated filter via deconvolution. \n% \n% Interface:\n% [I_Fus,D,PSF_l] = FE_HPM(I_MS,I_PAN,ratio,tap,lambda,mu,th,num_iter,filtername)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n% tap: Filter support;\n% lambda: Coefficient for weighting the energy regularization term;\n% mu: Coefficient for weighting the derivative regularization terms;\n% th: Threshold on the kernel (it cuts to 0 values below threshold);\n% num_iter_max: Max number of iteration (at least 3; not sensitive); \n% filtername: Kind of derivative (default: 'Basic') \n%\n% Outputs:\n% I_Fus,D: Pansharpened image;\n% PSF_l: Estimated point spread function.\n% \n% Reference:\n% [Vivone15] G. Vivone, M. Simoes, M. Dalla Mura, R. Restaino, J. Bioucas-Dias, G. A. Licciardi, and J. Chanussot, \"Pansharpening based on semiblind deconvolution\", \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 4, pp. 1997-2010, 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [I_Fus,PSF_l] = FE_HPM(I_MS,I_PAN,ratio,tap,lambda,mu,th,num_iter,filtername)\n\nimageHR = double(I_PAN);\nI_MS = double(I_MS);\nnBands = size(I_MS,3);\n\n%%% Equalization\nimageHR = repmat(imageHR,[1 1 size(I_MS,3)]);\nfor ii = 1 : size(I_MS,3) \n imageHR(:,:,ii) = (imageHR(:,:,ii) - mean2(imageHR(:,:,ii))).*(std2(I_MS(:,:,ii))./std2(imageHR(:,:,ii))) + mean2(I_MS(:,:,ii)); \nend\n\nPSF_l = FE(I_MS,I_PAN,ratio,tap,lambda,mu,th,num_iter,filtername);\n\nPAN_LP = zeros(size(imageHR));\nfor ii = 1 : nBands\n PAN_LP(:,:,ii) = imfilter(imageHR(:,:,ii),PSF_l,'replicate');\n t = imresize(PAN_LP(:,:,ii),1/ratio,'nearest');\n PAN_LP(:,:,ii) = interp23tap(t,ratio);\nend\n\nPAN_LP = double(PAN_LP);\n\nI_Fus = I_MS .* (imageHR ./ (PAN_LP + eps));\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/FR_Assessment.tex", "content": "" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GLP/GS2_GLP.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% GS2_GLP fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Gram-Schmidt (GS) mode 2 algorithm with Generalized Laplacian Pyramid (GLP) decomposition.\n% \n% Interface:\n% I_Fus_GS2_GLP = GS2_GLP(I_MS,I_PAN,ratio,sensor)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS').\n%\n% Outputs:\n% I_Fus_GS2_GLP: GS2_GLP pasharpened image.\n% \n% References:\n% [Aiazzi06] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTF-tailored multiscale fusion of high-resolution MS and Pan imagery,\n% Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 591596, May 2006.\n% [Alparone07] L. Alparone, L. Wald, J. Chanussot, C. Thomas, P. Gamba, and L. M. Bruce, Comparison of pansharpening algorithms: Outcome\n% of the 2006 GRS-S Data Fusion Contest, IEEE Transactions on Geoscience and Remote Sensing, vol. 45, no. 10, pp. 30123021,\n% October 2007.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_GS2_GLP = GS2_GLP(I_MS,I_PAN,ratio,sensor)\n\nimageLR = double(I_MS);\nimageHR = double(I_PAN);\n\nimageHR = repmat(imageHR,[1 1 size(imageLR,3)]);\n\nh = genMTF(ratio, sensor, size(I_MS,3));\n\nPAN_LP = zeros(size(I_MS));\nfor ii = 1 : size(I_MS,3)\n PAN_LP(:,:,ii) = imfilter(imageHR(:,:,ii),real(h(:,:,ii)),'replicate');\n t = imresize(PAN_LP(:,:,ii),1/ratio,'nearest');\n PAN_LP(:,:,ii) = interp23tap(t,ratio);\nend\n\nPAN_LP = double(PAN_LP);\n\n%%% Coefficients\ng = ones(1,size(I_MS,3));\nfor ii = 1 : size(I_MS,3)\n h = imageLR(:,:,ii);\n h2 = PAN_LP(:,:,ii);\n c = cov(h2(:),h(:));\n g(ii) = c(1,2)/var(h2(:));\nend\n\n%%% Detail Extraction\ndelta = imageHR - PAN_LP;\n\nI_Fus_GS2_GLP = zeros(size(imageLR));\n\nfor ii = 1 : size(imageLR,3)\n I_Fus_GS2_GLP(:,:,ii) = imageLR(:,:,ii) + delta(:,:,ii) .* g(ii);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GLP/MTF_GLP.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% MTF_GLP fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Modulation Transfer Function - Generalized Laplacian Pyramid (MTF-GLP) algorithm. \n% \n% Interface:\n% I_Fus_MTF_GLP = MTF_GLP(I_MS,I_PAN,sensor,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_MTF_GLP: MTF_GLP pansharpened image.\n% \n% References:\n% [Aiazzi02] B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, Context-driven fusion of high spatial and spectral resolution images based on\n% oversampled multiresolution analysis, IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 10, pp. 23002312, October\n% 2002.\n% [Aiazzi06] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTF-tailored multiscale fusion of high-resolution MS and Pan imagery,\n% Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 591596, May 2006.\n% [Vivone14a] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening, IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930934, May 2014.\n% [Vivone15a] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Alparone17] L. Alparone, A. Garzelli, and G. Vivone, \"Intersensor statistical matching for pansharpening: Theoretical issues and practical solutions\",\n% IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 8, pp. 4682-4695, 2017.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 2\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_MTF_GLP = MTF_GLP(I_MS,I_PAN,sensor,ratio)\n\nimageHR = double(I_PAN);\nI_MS = double(I_MS);\n\n%%% Equalization\nimageHR = repmat(imageHR,[1 1 size(I_MS,3)]);\n\nfor ii = 1 : size(I_MS,3) \n imageHR(:,:,ii) = (imageHR(:,:,ii) - mean2(imageHR(:,:,ii))).*(std2(I_MS(:,:,ii))./std2(LPfilterGauss(imageHR(:,:,ii),ratio))) + mean2(I_MS(:,:,ii));\nend\n\nh = genMTF(ratio, sensor, size(I_MS,3));\n\nPAN_LP = zeros(size(I_MS));\nfor ii = 1 : size(I_MS,3)\n PAN_LP(:,:,ii) = imfilter(imageHR(:,:,ii),real(h(:,:,ii)),'replicate');\n t = imresize(PAN_LP(:,:,ii),1/ratio,'nearest'); \n PAN_LP(:,:,ii) = interp23tap(t,ratio);\nend\n\nI_Fus_MTF_GLP = I_MS + imageHR - PAN_LP;\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GLP/MTF_GLP_FS.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% MTF_GLP_FS fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Modulation Transfer Function - Generalized Laplacian Pyramid (MTF-GLP) and a new Full Resolution Regression-based injection model. \n% \n% Interface:\n% I_Fus_MTF_GLP_FS = MTF_GLP_FS(I_MS,I_PAN,sensor,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_MTF_GLP_FS:\tPansharpened image.\n% \n% Reference:\n% [Vivone18] G. Vivone, R. Restaino,and J. Chanussot, \"Full scale regression-based injection coefficients for panchromatic sharpening,\" \n% IEEE Transactions on Image Processing, vol. 27, no. 7, pp. 3418-3431, Jul. 2018.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_MTF_GLP_FS = MTF_GLP_FS(I_MS,I_PAN,sensor,ratio)\n\nimageHR = double(I_PAN);\nI_MS = double(I_MS);\n\nh = genMTF(ratio, sensor, size(I_MS,3));\n\nI_Fus_MTF_GLP_FS = zeros(size(I_MS));\nfor ii = 1 : size(I_MS,3)\n %%% Low resolution PAN image\n PAN_LP = imfilter(imageHR,real(h(:,:,ii)),'replicate');\n t = imresize(PAN_LP,1/ratio,'nearest'); \n PAN_LP = interp23tap(t,ratio);\n \n %%% Injection coefficient for band ii \n MSB = I_MS(:,:,ii);\n CMSPAN = cov(MSB(:), imageHR(:)); \n CPANPANLR = cov(PAN_LP(:), imageHR(:));\n gFS = CMSPAN(1,2)./CPANPANLR(1,2);\n \n %%% Fusion rule\n I_Fus_MTF_GLP_FS(:,:,ii) = I_MS(:,:,ii) + gFS .* (imageHR - PAN_LP);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GLP/MTF_GLP_HPM.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% MTF_GLP_HPM fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Modulation Transfer Function - Generalized Laplacian Pyramid (MTF-GLP) with High Pass Modulation (HPM) injection model algorithm. \n% \n% Interface:\n% I_Fus_MTF_GLP_HPM = MTF_GLP_HPM(I_MS,I_PAN,sensor,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_MTF_GLP_HPM: MTF_GLP_HPM pansharpened image.\n% \n% References:\n% [Aiazzi03] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, An MTF-based spectral distortion minimizing model for Pan-sharpening\n% of very high resolution multispectral images of urban areas, in Proceedings of URBAN 2003: 2nd GRSS/ISPRS Joint Workshop on\n% Remote Sensing and Data Fusion over Urban Areas, 2003, pp. 9094.\n% [Aiazzi06] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTF-tailored multiscale fusion of high-resolution MS and Pan imagery,\n% Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 591596, May 2006.\n% [Vivone14a] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening, IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930934, May 2014.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Alparone17] L. Alparone, A. Garzelli, and G. Vivone, \"Intersensor statistical matching for pansharpening: Theoretical issues and practical solutions\",\n% IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 8, pp. 4682-4695, 2017.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_MTF_GLP_HPM = MTF_GLP_HPM(I_MS,I_PAN,sensor,ratio)\n\nimageHR = double(I_PAN);\nI_MS = double(I_MS);\n\n%%% Equalization\nimageHR = repmat(imageHR,[1 1 size(I_MS,3)]);\n\nfor ii = 1 : size(I_MS,3) \n imageHR(:,:,ii) = (imageHR(:,:,ii) - mean2(imageHR(:,:,ii))).*(std2(I_MS(:,:,ii))./std2(LPfilterGauss(imageHR(:,:,ii),ratio))) + mean2(I_MS(:,:,ii)); \nend\n\n\nh = genMTF(ratio, sensor, size(I_MS,3));\n\nPAN_LP = zeros(size(I_MS));\nfor ii = 1 : size(I_MS,3)\n PAN_LP(:,:,ii) = imfilter(imageHR(:,:,ii),real(h(:,:,ii)),'replicate');\n t = imresize(PAN_LP(:,:,ii),1/ratio,'nearest');\n PAN_LP(:,:,ii) = interp23tap(t,ratio);\nend\n\nI_Fus_MTF_GLP_HPM = I_MS .* (imageHR ./ (PAN_LP + eps));\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GLP/MTF_GLP_HPM_Haze_min.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Gaussian Laplacian Pyramid with high pass modulation injection model haze corrected.\n% \n% Interface:\n% I_Fus_MTF_GLP_HPM = MTF_GLP_HPM_Haze_min(I_PAN,I_MS,sensor,ratio,decimation)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% decimation: Flag decimation (1: decimated PAN_LP).\n%\n% Outputs:\n% I_Fus_MTF_GLP_HPM: Pansharpened image.\n% \n% References:\n% [Lolli17] S. Lolli, L. Alparone, A. Garzelli, and G. Vivone, \"Haze correction for contrast-based multispectral pansharpening\",\n% IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 12, pp. 2255-2259, 2017.\n% [Garzelli18] A. Garzelli, B. Aiazzi, L. Alparone, S. Lolli, and G. Vivone, \n% \"Multispectral Pansharpening with Radiative Transfer-Based Detail-Injection Modeling for Preserving Changes in Vegetation Cover\",\n% MDPI Remote Sensing, vol. 10, no. 8, pp. 1 - 18, 2018.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_MTF_GLP_HPM = MTF_GLP_HPM_Haze_min(I_MS,I_PAN,sensor,ratio,decimation)\n\nif size(I_MS,3) == 4\n prc = 1;\n minMS = zeros(1,1,4);\n B = I_MS(:,:,1);\n G = I_MS(:,:,2);\n R = I_MS(:,:,3);\n NIR = I_MS(:,:,4);\n minMS(1,1,1) = 0.95 * prctile(B(:),prc);\n minMS(1,1,2) = 0.45 * prctile(G(:),prc);\n minMS(1,1,3) = 0.40 * prctile(R(:),prc);\n minMS(1,1,4) = 0.05 * prctile(NIR(:),prc);\nelse\n minMS = zeros(1,1,size(I_MS,3));\n for ii = 1 : size(I_MS, 3)\n minMS(1,1,ii) = min(min(I_MS(:,:,ii))); \n end\nend\n\nI_PAN_LR = LPfilterGauss(I_PAN,ratio);\nw = estimation_alpha(cat(3,ones(size(I_PAN_LR)),I_MS),I_PAN_LR,'global');\nwp = w' * [1;squeeze(minMS)]; \n\nL = repmat(minMS, [size(I_MS,1) size(I_MS,2)]);\nLp = wp .* ones([size(I_MS,1) size(I_MS,2)]);\n\nimageHR = double(I_PAN);\nI_MS = double(I_MS);\n\n%%% Equalization\nimageHR = repmat(imageHR,[1 1 size(I_MS,3)]);\n\nPAN_LP = MTF(imageHR,sensor,ratio);\n\nif decimation\n for ii = 1 : size(I_MS,3)\n t = imresize(PAN_LP(:,:,ii),1/ratio,'nearest');\n PAN_LP(:,:,ii) = interp23tap(t,ratio);\n end\nend\n\nP_PL = (imageHR - Lp) ./ (PAN_LP - Lp + eps);\n\nMS_L = I_MS - L;\n\nI_Fus_MTF_GLP_HPM = MS_L .* P_PL + L;\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GLP/MTF_GLP_HPM_R.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% A Regression-Based High-Pass Modulation Pansharpening Approach (Global Version) \n% \n% Interface:\n% I_Fus_MTF_GLP_HPM_R = MTF_GLP_HPM_R(I_MS,I_PAN,sensor,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_MTF_GLP_HPM_R: Pansharpened image.\n% \n% Reference:\n% [Vivone18] G. Vivone, R. Restaino, and J. Chanussot, \"A regression-based high-pass modulation pansharpening approach,\" \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 2, pp. 984-996, Feb. 2018.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_MTF_GLP_HPM_R = MTF_GLP_HPM_R(I_MS,I_PAN,sensor,ratio)\n\nimageHR = double(I_PAN);\nI_MS = double(I_MS);\n\nh = genMTF(ratio, sensor, size(I_MS,3));\n\nI_Fus_MTF_GLP_HPM_R = zeros(size(I_MS));\nfor ii = 1 : size(I_MS,3)\n %%% Low resolution PAN image\n PAN_LP = imfilter(imageHR,real(h(:,:,ii)),'replicate');\n t = imresize(PAN_LP,1/ratio,'nearest');\n PAN_LP = interp23tap(t,ratio);\n \n %%%% Regression coefficients\n MSB = I_MS(:,:,ii);\n\tC = cov(MSB(:),PAN_LP(:));\n g = C(1,2)./C(2,2);\n cb = mean(MSB(:))./g - mean(imageHR(:));\n \n %%% Fusion rule\n I_Fus_MTF_GLP_HPM_R(:,:,ii) = I_MS(:,:,ii) .* (imageHR + cb) ./ (PAN_LP + cb + eps);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GS/GS.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% GS fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Gram-Schmidt (GS) transformation.\n% \n% Interface:\n% I_Fus_GS = GS(I_MS,I_PAN)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image.\n%\n% Outputs:\n% I_Fus_GS: GS pasharpened image.\n% \n% References:\n% [Laben00] C. A. Laben and B. V. Brower, Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening, Eastman\n% Kodak Company, Tech. Rep. US Patent # 6,011,875, 2000.\n% [Aiazzi07] B. Aiazzi, S. Baronti, and M. Selva, Improving component substitution Pansharpening through multivariate regression of MS+Pan\n% data, IEEE Transactions on Geoscience and Remote Sensing, vol. 45, no. 10, pp. 32303239, October 2007.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_GS = GS(I_MS,I_PAN)\n\nimageLR = double(I_MS);\nimageHR = double(I_PAN);\n\n%%% Remove means from imageLR\nimageLR0 = zeros(size(I_MS));\nfor ii = 1 : size(I_MS,3), imageLR0(:,:,ii) = imageLR(:,:,ii) - mean2(imageLR(:,:,ii)); end\n\n%%% Intensity\nI = mean(imageLR,3); \n\n%%% Remove mean from I\nI0 = I - mean2(I);\n\nimageHR = (imageHR - mean2(imageHR)) .* (std2(I0)./std2(imageHR)) + mean2(I0);\n\n%%% Coefficients\ng = ones(1,1,size(I_MS,3)+1);\nfor ii = 1 : size(I_MS,3)\n h = imageLR0(:,:,ii);\n c = cov(I0(:),h(:));\n g(1,1,ii+1) = c(1,2)/var(I0(:));\nend\n\n%%% Detail Extraction\ndelta = imageHR - I0;\ndeltam = repmat(delta(:),[1 size(I_MS,3)+1]);\n\n%%% Fusion\nV = I0(:);\nfor ii = 1 : size(I_MS,3)\n h = imageLR0(:,:,ii);\n V = cat(2,V,h(:));\nend\n\ngm = zeros(size(V));\nfor ii = 1 : size(g,3)\n gm(:,ii) = squeeze(g(1,1,ii)) .* ones(size(I_MS,1).*size(I_MS,2),1);\nend\n\nV_hat = V + deltam .* gm;\n\n%%% Reshape fusion result\nI_Fus_GS = reshape(V_hat(:,2:end),[size(I_MS,1) size(I_MS,2) size(I_MS,3)]);\n\n% Final Mean Equalization\nfor ii = 1 : size(I_MS,3)\n h = I_Fus_GS(:,:,ii);\n I_Fus_GS(:,:,ii) = h - mean2(h) + mean2(imageLR(:,:,ii));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GS/GSA.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% GSA fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Gram-Schmidt Adaptive (GSA) algorithm.\n% \n% Interface:\n% I_Fus_GSA = GSA(I_MS,I_PAN,I_MS_LR,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% I_MS_LR: MS image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_GSA: GSA pasharpened image.\n% \n% References:\n% [Aiazzi07] B. Aiazzi, S. Baronti, and M. Selva, Improving component substitution Pansharpening through multivariate regression of MS+Pan\n% data, IEEE Transactions on Geoscience and Remote Sensing, vol. 45, no. 10, pp. 32303239, October 2007.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nfunction I_Fus_GSA = GSA(I_MS,I_PAN,I_MS_LR,ratio)\n\nimageLR = double(I_MS);\nimageHR = double(I_PAN);\nimageLR_LP = double(I_MS_LR);\n\n%%% Remove means from imageLR\nimageLR0 = zeros(size(I_MS));\nfor ii = 1 : size(I_MS,3), imageLR0(:,:,ii) = imageLR(:,:,ii) - mean2(imageLR(:,:,ii)); end\n\n%%% Remove means from imageLR_LP\nimageLR_LP0 = zeros(size(I_MS_LR));\nfor ii = 1 : size(I_MS_LR,3), imageLR_LP0(:,:,ii) = imageLR_LP(:,:,ii) - mean2(imageLR_LP(:,:,ii)); end\n\n\n%%% Intensity\nimageHR0 = imageHR - mean2(imageHR);\nimageHR0 = LPfilterPlusDec(imageHR0,ratio);\nalpha(1,1,:) = estimation_alpha(cat(3,imageLR_LP0,ones(size(I_MS_LR,1),size(I_MS_LR,2))),imageHR0,'global');\nI = sum(cat(3,imageLR0,ones(size(I_MS,1),size(I_MS,2))) .* repmat(alpha,[size(I_MS,1) size(I_MS,2) 1]),3); \n\n%%% Remove mean from I\nI0 = I - mean2(I);\n\n%%% Coefficients\ng = ones(1,1,size(I_MS,3)+1);\nfor ii = 1 : size(I_MS,3)\n h = imageLR0(:,:,ii);\n c = cov(I0(:),h(:));\n g(1,1,ii+1) = c(1,2)/var(I0(:));\nend\n\nimageHR = imageHR - mean2(imageHR);\n\n%%% Detail Extraction\ndelta = imageHR - I0;\ndeltam = repmat(delta(:),[1 size(I_MS,3)+1]);\n\n%%% Fusion\nV = I0(:);\nfor ii = 1 : size(I_MS,3)\n h = imageLR0(:,:,ii);\n V = cat(2,V,h(:));\nend\n\ngm = zeros(size(V));\nfor ii = 1 : size(g,3)\n gm(:,ii) = squeeze(g(1,1,ii)) .* ones(size(I_MS,1).*size(I_MS,2),1);\nend\n\nV_hat = V + deltam .* gm;\n\n%%% Reshape fusion result\nI_Fus_GSA = reshape(V_hat(:,2:end),[size(I_MS,1) size(I_MS,2) size(I_MS,3)]);\n\n%%% Final Mean Equalization\nfor ii = 1 : size(I_MS,3)\n h = I_Fus_GSA(:,:,ii);\n I_Fus_GSA(:,:,ii) = h - mean2(h) + mean2(imageLR(:,:,ii));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/GS/GS_Segm.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% GS_Segm fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the segmentation-based version of the Gram-Schmidt algorithm.\n% \n% Interface:\n% PanSharpenedImage = GS_Segm(I_MS,I_PAN,I_LR_input,S)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale\n% I_PAN: PAN image\n% I_LR_input: Low Resolution PAN Image \n% S: Segmentation\n%\n% Outputs:\n% PanSharpenedImage: Pasharpened image\n% \n% Reference:\n% [Restaino17] R. Restaino, M. Dalla Mura, G. Vivone, J. Chanussot, Context-Adaptive Pansharpening Based on Image Segmentation, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 2, pp. 753766, February 2017.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction PanSharpenedImage = GS_Segm(I_MS,I_PAN,I_LR_input,S)\nI_MS = double(I_MS);\n\nI_PAN = repmat(double(I_PAN), [1, 1, size(I_MS,3)]);\nI_LR_input = double(I_LR_input);\nif size(I_LR_input, 3) == 1\n I_LR_input = repmat(I_LR_input, [1, 1, size(I_MS,3)]);\nend\nif size(I_LR_input, 3) ~= size(I_PAN, 3)\n error('I_LP should have the same number of bands as PAN');\nend\n\nDetailsHRPan = I_PAN - I_LR_input;\n\nCoeff = zeros(size(I_MS));\nlabels = unique(S);\n\nfor ii = 1: size(I_MS,3)\n MS_Band = squeeze(I_MS(:,:,ii));\n I_LR_Band = squeeze(I_LR_input(:,:,ii));\n Coeff_Band = zeros(size(I_LR_Band));\n for il=1:length(labels)\n idx = S==labels(il);\n c = cov(I_LR_Band(idx),MS_Band(idx));\n Coeff_Band(idx) = c(1,2)/var(I_LR_Band(idx));\n end\n Coeff(:,:,ii) = Coeff_Band;\nend\n\nPanSharpenedImage = Coeff .* DetailsHRPan + I_MS;\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/MF/MF_HG_Pansharpen.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Morphological Pyramid Decomposition using Half-Gradient. \n% \n% Interface:\n% I_Fus_MF_HG = MF_HG_Pansharpen(I_MS,I_PAN,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_MF_HG: Morphological Half Gradient (HG) pansharpened image.\n% \n% Reference:\n% [Restaino16] R. Restaino, G. Vivone, M. Dalla Mura, and J. Chanussot, Fusion of Multispectral and Panchromatic Images Based on Morphological Operators, \n% IEEE Transactions on Image Processing, vol. 25, no. 6, pp. 2882-2895, Jun. 2016.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_MF_HG = MF_HG_Pansharpen(I_MS,I_PAN,ratio)\n\nimageLR = double(I_MS);\nimageHR = double(I_PAN);\n\n% Equalization\nimageHR = repmat(imageHR,[1 1 size(imageLR,3)]);\nfor ii = 1 : size(imageLR,3)\n imageHR(:,:,ii) = (imageHR(:,:,ii) - mean2(imageHR(:,:,ii))).*(std2(imageLR(:,:,ii))./std2(imageHR(:,:,ii))) + mean2(imageLR(:,:,ii));\nend\n\n% Structuring Element choice\ntextse= [0 1 0; 1 1 1; 0 1 0];\n\n% Interpolation Method\nint_meth='bilinear';\n\n% Number of levels\nlev=ceil(log2(ratio))+1;\n\n% Image Construction\nP = Pyr_Dec(imageHR,textse,lev,int_meth);\n\n% Fusion \nP_LP = P(:,:,:,lev);\nI_Fus_MF_HG = imageLR .* (P(:,:,:,1)./(P_LP+eps));\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/MF/Pyr_Dec.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Morphological Pyramid Decomposition using Half-Gradient. \n% \n% Interface:\n% P = Pyr_Dec(Im,textse,lev,int_meth)\n%\n% Inputs:\n% Im: Image to decompose;\n% textse: Structuring Element;\n% lev: Number of decomposition levels;\n% int_meth: Interpolation method.\n%\n% Outputs:\n% P: Morphological Pyramid using Half-Gradient.\n% \n% References:\n% [Vivone14] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening, IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930934, May 2014.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2565-2586, May 2015.\n% [Restaino16] R. Restaino, G. Vivone, M. Dalla Mura, and J. Chanussot, Fusion of Multispectral and Panchromatic Images Based on Morphological Operators, \n% IEEE Transactions on Image Processing, vol. 25, no. 6, pp. 2882-2895, Jun. 2016.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nfunction P = Pyr_Dec(Im,textse,lev,int_meth)\n\nP(:,:,:,1) = Im;\nSizes(1,:)=[size(Im,1), size(Im,2)];\nimageI_new=P(:,:,:,1);\nfirst=1;\n\nfor ii = 2 : lev\n \n imageI_old = imageI_new;\n clear imageI_new\n\n % Half Gradient\n PD = imdilate(imageI_old,textse);\n PE= imerode(imageI_old,textse);\n rho_minus=imageI_old-PE;\n rho_plus=PD-imageI_old;\n D=rho_minus-rho_plus;\n PS = imageI_old -0.5*D;\n % PS = 0.5*squeeze(PD+PE); %equivalently\n \n % Downsampling\n if first\n for il=1:size(imageI_old,3)\n imageI_new(:,:,il)=PS(2:2:end,2:2:end,il);\n end\n first=0;\n else\n for il=1:size(imageI_old,3)\n imageI_new(:,:,il)=PS(1:2:end,1:2:end,il);\n end\n end\n Sizes(ii,:)=[size(imageI_new,1) size(imageI_new,1)];\n imageI_resized_old=imageI_new;\n for ir=ii:-1:2,\n for il=1:size(Im,3)\n imageI_resized_new(:,:,il) = imresize(imageI_resized_old(:,:,il),[Sizes(ir-1,1) Sizes(ir-1,2)],int_meth);\n end\n imageI_resized_old=imageI_resized_new;\n clear imageI_resized_new\n end\n \n if sum(isfinite(imageI_resized_old(:)))~=numel(imageI_resized_old)\n P(:,:,:,1:lev) =repmat(P(:,:,:,1),1,1,1,lev);\n break\n else\n P(:,:,:,ii) = imageI_resized_old;\n end\n \n clear imageI_resized_old\nend\n\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PRACS/PRACS.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% PRACS fuses the upsampled MultiSpectral (MS) and PANchromatic (PAN) images by \n% exploiting the Partial Replacement Adaptive CS (PRACS) algorithm. \n% \n% Interface:\n% I_Fus_PRACS = PRACS(I_MS,I_PAN,ratio)\n%\n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n%\n% Outputs:\n% I_Fus_PRACS: PRACS pansharpened image.\n% \n% References:\n% [Choi11] J. Choi, K. Yu, and Y. Kim, A new adaptive component-substitution-based satellite image fusion by using partial replacement, IEEE\n% Transactions on Geoscience and Remote Sensing, vol. 49, no. 1, pp. 295309, January 2011.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % % \n% \n% Version: 1\n% \n% % % % % % % % % % % % % \n% \n% Copyright (C) 2019\n% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_PRACS = PRACS(I_MS,I_PAN,ratio)\n\nbeta = 0.95; % for 11-bit data\n% beta = 1.95; % for 8-bit data\n\nI_MS = double(I_MS);\nI_PAN = double(I_PAN);\n[N,M,L] = size(I_MS);\n\n%%% Histogram matching of each MS band to Pan\nmsexp_hm = zeros(N,M,L);\nfor k=1:L\n b = I_MS(:,:,k);\n b = (b - mean2(b) + mean2(I_PAN)/std2(I_PAN)*std2(b)) * std2(I_PAN)/std2(b);\n b(b<0) = 0;\n msexp_hm(:,:,k) = b;\nend\n\n%%% Computing low-resolution Pan by bicubic decimation/interpolation\naux = imresize(I_PAN,1/ratio);\npan_l = imresize(aux,ratio);\nclear aux\n\n%%% Regression of Pan_low vs MS (with offset)\nbb = zeros(N*M,L);\nfor k = 1:L\n bb(:,k) = reshape(squeeze(msexp_hm(:,:,k)),[N*M,1]);\nend\nbb = [ones(N*M,1),bb];\nalpha = regress(pan_l(:),bb); \n\n%%% Initial estimate of intensity\naux = bb * alpha;\nI_l = reshape(aux,[N,M]);\nclear aux\n\nclear bb\n\n%%% Partial Replacement\nI_h = zeros(N,M,L);\ncc = zeros(1,L);\nfor k=1:L\n b = msexp_hm(:,:,k);\n cc(k) = corr2(I_l(:),b(:));\n aux = cc(k)*I_PAN(:)+(1-cc(k))*b(:);\n I_h(:,:,k) = reshape(aux,[N,M]);\nend\nclear aux\n\n%%% Band-dependent intensity\n%%% For each band, compute low-resolution I_h by bicubic decimation/interpolation\nI_h_low = zeros(N,M,L);\nfor k=1:L\n aux = imresize(I_h(:,:,k),1/ratio);\n I_h_low(:,:,k) = imresize(aux,ratio);\nend\nclear aux\n%%%\n\n%%% Regression of I_h_low_k vs MS (with offset)\n\nalpha = zeros(L+1,L);\nfor k = 1:L\n bb(:,k) = reshape(squeeze(msexp_hm(:,:,k)),[N*M,1]);\nend\nbb = [ones(N*M,1),bb];\nfor k=1:L\n aux = I_h_low(:,:,k);\n alpha(:,k) = regress(aux(:),bb);\nend\nclear aux\n\n%%% Intensities\nI_l_prime = zeros(N,M,L);\nfor k=1:L\n aux = bb * alpha(:,k);\n I_l_prime(:,:,k) = reshape(aux,[N,M]);\nend\nclear aux\n\n%%% Computing detail images\ndelta = zeros(N,M,L);\nfor k=1:L\n delta(:,:,k)= I_h(:,:,k)-I_l_prime(:,:,k)-(mean2(I_h(:,:,k))-mean2(I_l_prime(:,:,k)));\nend\n\n%%% Computing mean of std. devs.\naux3 = zeros(1,L);\nfor k=1:L\n aux3(k) = std2(I_MS(:,:,k));\nend\naux3 = mean(aux3);\n\n%%% Computing weights\nw = zeros(1,L);\nfor k=1:L\n aux1 = I_l_prime(:,:,k);\n b = I_MS(:,:,k);\n w(k) = beta .* corr2(aux1(:),b(:))*std(b(:))/aux3;%std(aux2(:));\nend\n\n%%% Computing local instability adjustment parameter\nL_I = zeros(N,M,L);\nfor k=1:L\n b = I_MS(:,:,k);\n I = I_l_prime(:,:,k);\n aux = 1-abs(1-corr2(I_l(:),b(:))*b(:)./I(:));\n L_I(:,:,k) = reshape(aux,[N,M]);\nend\n\n%%% Computing pansharpened image\ndet = zeros(N,M,L);\nI_Fus_PRACS = zeros(N,M,L);\nfor k=1:L\n det(:,:,k) = w(k) * L_I(:,:,k) .* delta(:,:,k);\n I_Fus_PRACS(:,:,k) = I_MS(:,:,k) + det(:,:,k);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/PWMBF.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Model-based fusion using PCA and wavelets.\n% \n% Interface:\n% Z = PWMBF(Pan,Low,ratio,r,wavelet,degrade,reduced,whiten)\n% \n% Inputs:\n% Pan : Panchromatic image;\n% Low: Low spatial resolution MS image;\n% ratio: Scale ratio between Pan and Low;\n% r: Number of principal components;\n% wavelet: flag;\n% degrade: flag.\n% \n% Output:\n% Z: Pansharpened image;\n% \n% References:\n% [Palsson15] F. Palsson, J.R. Sveinsson, M.O. Ulfarsson, J.A. Benediktsson, \"Model-based fusion of multi-and hyperspectral images using PCA and wavelets\", \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2652-2663, May 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction Z = PWMBF(Pan,Low,ratio,r,wavelet,degrade)\n\naddpath(sprintf('%s/rwt/bin',pwd))\n\n% Wavelet parameters\nL=4;\ntype='rwt';\n\nLow=double(Low);\nPan=double(Pan);\n\nN=size(Pan,1);\nQ=size(Pan,3);\nnb=size(Low,3);\n\nif(r>nb)\n error('Number of PCs greater than number of bands');\nend\n\nX=Pan;\nYlow=Low;\n\nif(degrade)\n X=imresize(Pan,1/ratio);\n Ylow=imresize(Low,1/ratio);\n N=N/4;\nend\n\n% Upsample Y\nY=imresize(Ylow,ratio,'bicubic');\n\n% Degrade X\nXtilde=imresize(imresize(X,1/ratio,'bilinear'),ratio,'bicubic');\n\nX=reshape(X,[N^2 Q]);\nXtilde=reshape(Xtilde,[N^2 Q]);\nY=reshape(Y,[N^2 nb]);\n\n% PCA transform\n[F, D, R]=svd(Y,'econ');\nG=F*D;\nU=R;\n\nwfilter=daubcqf(4,'min');\n\nif wavelet\n x=compute_PhiTX(Xtilde,L,wfilter,type);\n x0=compute_PhiTX(X,L,wfilter,type);\n y=compute_PhiTX(G(:,1:r),L,wfilter,type);\n yl=y(1:N^2,:);\n zh=zeros(3*L*N^2,r);\n for p=1:r\n for j=1:3*L\n xh=x(j*N^2+1:(j+1)*N^2,:);\n xh0=x0(j*N^2+1:(j+1)*N^2,:);\n yh=y(j*N^2+1:(j+1)*N^2,p);\n Cyy=yh'*yh/N^2;\n Cyx=yh'*xh/N^2;\n Cxx=xh'*xh/N^2;\n Cn=diag(mad(abs(yh))/0.6745).^2;\n inv_Cxx=inv(Cxx);\n Cy_x=Cyy-Cyx*inv_Cxx*Cyx';\n if Q>1\n CyxiCxx=Cyx*inv_Cxx;\n mu_zx=xh*CyxiCxx';\n mu_zx0=xh0*CyxiCxx';\n else\n mu_zx=repmat((Cyx*inv_Cxx)',[N^2 1]).*xh;\n mu_zx0=repmat((Cyx*inv_Cxx)',[N^2 1]).*xh0;\n end\n ymu=yh-mu_zx;\n CC=Cy_x*inv(Cy_x+Cn);\n zh((j-1)*N^2+1:N^2+(j-1)*N^2,p)=mu_zx0+ymu*CC;\n end\n end\n z=[yl;zh];\n B=compute_PhiX(z,L,wfilter,type);\n deg=0;\n if deg == 1\n U = U(:,1:r);\n Zhat=B*U';\n else\n G(:,1:r)=B;\n Zhat=G*U';\n end\nelse\n Cn=0;\n yh=G(:,1:r);\n xh=Xtilde;\n xh0=X;\n Cyy=yh'*yh/N^2;\n Cyx=yh'*xh/N^2;\n Cxx=xh'*xh/N^2;\n inv_Cxx=inv(Cxx);\n Cy_x=Cyy-Cyx*inv_Cxx*Cyx';\n CyxiCxx=Cyx*inv_Cxx;\n mu_zx=xh*CyxiCxx';\n mu_zx0=xh0*CyxiCxx';\n ymu=yh-mu_zx;\n CC=Cy_x/(Cy_x+Cn);\n B=mu_zx0+ymu*CC;\n G(:,1:r)=B;\n Zhat=G*U';\nend\n\nZ=reshape(Zhat,[N N nb]);\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/compute_PhiTX.m", "content": "function PhiTX=compute_PhiTX(X,L,h,type)\nvec = @(x) x(:);\n[M,T]=size(X);\n\nswitch lower(type)\n case 'dwt2'\n PhiTX=zeros(size(X));\n for k=1:T\n PhiTX(:,k)=vec(FWT2_PO(reshape(X(:,k),[sqrt(M) sqrt(M)]),log2(sqrt(M))-L,h));\n end\n case 'dwt'\n PhiTX=zeros(size(X));\n for k=1:T\n PhiTX(:,k)=vec(mdwt(reshape(X(:,k),[sqrt(M) sqrt(M)]),h,L));\n end\n case 'rwt'\n PhiTX=zeros((3*L+1)*M,T);\n for k=1:T\n [xl xh L]=mrdwt(reshape(X(:,k),[sqrt(M) sqrt(M)]),h,L);\n PhiTX(:,k)=vec([xl xh])/2;\n end\n case 'swt'\n PhiTX=zeros((3*L+1)*M,T);\n for k=1:T\n PhiTX(:,k)=vec(myswt2(reshape(X(:,k),[sqrt(M) sqrt(M)]),L,'db4'));\n end\n case 'iso'\n PhiTX=zeros((L+1)*M,T);\n for k=1:T\n PhiTX(:,k)=vec(cell2mat(atrousdec(reshape(X(:,k),[sqrt(M) sqrt(M)]),'maxflat',L)));\n end\n case 'cwt'\n J=L;\n [Faf, Fsf] = FSfarras; % 1st stage anal. & synth. filters\n [af, sf] = dualfilt1;\n for k=1:T\n w=dualtree2D(reshape(X(:,k),[sqrt(M) sqrt(M)]),J,Faf,af);\n W=[];\n for j=1:J\n W=[W' vec(w{j}{1}{1})']';\n W=[W' vec(w{j}{1}{2})']';\n W=[W' vec(w{j}{1}{3})']';\n end\n W=[W' vec(w{J+1}{1})']';\n for j=1:J\n W=[W' vec(w{j}{2}{1})']';\n W=[W' vec(w{j}{2}{2})']';\n W=[W' vec(w{j}{2}{3})']';\n end\n W=[W' vec(w{J+1}{2})']';\n PhiTX(:,k)=W;\n end\n case 'cplxdt'\n J=L;\n [Faf, Fsf] = FSfarras; % 1st stage anal. & synth. filters\n [af, sf] = dualfilt1;\n for k=1:T\n w=cplxdual2D(reshape(X(:,k),[sqrt(M) sqrt(M)]),J,Faf,af);\n W=[];\n for j=1:J\n W=[W' vec(w{j}{1}{1}{1})']';\n W=[W' vec(w{j}{1}{1}{2})']';\n W=[W' vec(w{j}{1}{1}{3})']';\n W=[W' vec(w{j}{1}{2}{1})']';\n W=[W' vec(w{j}{1}{2}{2})']';\n W=[W' vec(w{j}{1}{2}{3})']';\n end\n W=[W' vec(w{J+1}{1}{1})']';\n W=[W' vec(w{J+1}{1}{2})']';\n for j=1:J\n W=[W' vec(w{j}{2}{1}{1})']';\n W=[W' vec(w{j}{2}{1}{2})']';\n W=[W' vec(w{j}{2}{1}{3})']';\n W=[W' vec(w{j}{2}{2}{1})']';\n W=[W' vec(w{j}{2}{2}{2})']';\n W=[W' vec(w{j}{2}{2}{3})']';\n end\n W=[W' vec(w{J+1}{2}{1})']';\n W=[W' vec(w{J+1}{2}{2})']';\n PhiTX(:,k)=W;\n end\n otherwise\n error(['Unknown method ' type]);\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/compute_PhiX.m", "content": "function PhiX=compute_PhiX(X,L,h,type)\nvec = @(x) x(:);\n[N,r]=size(X);\n\nswitch lower(type)\n case 'dwt'\n M=N;\n PhiX=zeros(size(X));\n for k=1:r\n PhiX(:,k)=vec(midwt(reshape(X(:,k),[sqrt(M) sqrt(M)]),h,L));\n end\n case 'dwt2'\n M=N;\n PhiX=zeros(size(X));\n for k=1:r\n PhiX(:,k)=vec(IWT2_PO(reshape(X(:,k),[sqrt(M) sqrt(M)]),log2(sqrt(M))-L,h));\n end\n case 'rwt'\n M=N/(3*L+1);\n PhiX=zeros(M,r);\n for k=1:r\n PhiX(:,k)=vec(mirdwt(reshape(X(1:M,k),[sqrt(M) sqrt(M)]),reshape(X(M+1:end,k),[sqrt(M) 3*L*sqrt(M)]),h,L))*2;\n end\n case 'swt'\n M=N/(3*L+1);\n PhiX=zeros(M,r);\n for k=1:r\n PhiX(:,k)=vec(iswt2(reshape(X(:,k),[sqrt(M) sqrt(M) 3*L+1]),'db4'));\n end\n case 'iso'\n M=N/(L+1);\n PhiX=zeros(M,r);\n for k=1:r\n xc=reshape(X(:,k),[sqrt(M) (L+1)*sqrt(M)]);\n xc=mat2cell(xc,[sqrt(M)],repmat(sqrt(M),[1 L+1]));\n PhiX(:,k)=vec(atrousrec(xc,'maxflat'));\n end\n case 'cwt'\n J=L;\n [Faf, Fsf] = FSfarras; % 1st stage anal. & synth. filters\n [af, sf] = dualfilt1;\n \n PhiX=zeros(size(X,1)/2,size(X,2));\n n=sqrt(size(X,1)/2);\n for c=1:r\n \n W=X(:,c);\n j_offset=0;\n for j=1:J\n for k=1:3\n w2{j}{1}{k}=reshape(W(j_offset+1+(k-1)*(n/2^j)^2:j_offset+k*(n/2^j)^2),[n/2^j n/2^j]);\n w2{j}{2}{k}=reshape(W(j_offset+n^2+1+(k-1)*(n/2^j)^2:j_offset+n^2+k*(n/2^j)^2),[n/2^j n/2^j]);\n end\n j_offset=j_offset+3*(n/2^j)^2;\n end\n w2{J+1}{1}=reshape(W(n^2-(n/2^(J))^2+1:n^2),[n/2^J n/2^J]);\n w2{J+1}{2}=reshape(W(2*n^2-(n/2^(J))^2+1:2*n^2),[n/2^J n/2^J]);\n PhiX(:,c)=vec(idualtree2D(w2,J,Fsf,sf));\n end\n case 'cplxdt'\n J=L;\n [Faf, Fsf] = FSfarras; % 1st stage anal. & synth. filters\n [af, sf] = dualfilt1;\n \n PhiX=zeros(size(X,1)/4,size(X,2));\n n=sqrt(size(X,1)/4);\n for c=1:r\n \n W=X(:,c);\n j_offset=0;\n for j=1:J\n l_offset=0;\n for l=1:2\n for k=1:3\n w2{j}{1}{l}{k}=reshape(W(j_offset+l_offset+1+(k-1)*(n/2^j)^2:j_offset+l_offset+k*(n/2^j)^2),[n/2^j n/2^j]);\n w2{j}{2}{l}{k}=reshape(W(j_offset+l_offset+2*n^2+1+(k-1)*(n/2^j)^2:j_offset+l_offset+2*n^2+k*(n/2^j)^2),[n/2^j n/2^j]);\n end\n l_offset=l_offset+3*(n/2^j)^2;\n end\n j_offset=j_offset+6*(n/2^j)^2;\n end\n w2{J+1}{1}{1}=reshape(W(2*n^2-2*(n/2^(J))^2+1:2*n^2-(n/2^(J))^2),[n/2^J n/2^J]);\n w2{J+1}{1}{2}=reshape(W(2*n^2-(n/2^(J))^2+1:2*n^2),[n/2^J n/2^J]);\n w2{J+1}{2}{1}=reshape(W(4*n^2-2*(n/2^(J))^2+1:4*n^2-(n/2^(J))^2),[n/2^J n/2^J]);\n w2{J+1}{2}{2}=reshape(W(4*n^2-(n/2^(J))^2+1:4*n^2),[n/2^J n/2^J]);\n PhiX(:,c)=vec(icplxdual2D(w2,J,Fsf,sf));\n end\n otherwise\n error(['Unknown method ' type]);\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/readme", "content": "test\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/AUTHORS", "content": "The primary authors of Rice Wavelet Toolbox are and/or have been:\n * Richard Baraniuk\n * Hyeokho Choi\n * Ramesh Neelamani\n * Vinay Ribeiro\n * Rebecca Hindman\n * Justin Romberg\n * Haitao Guo\n * Felix Fernandes\n * Brent Hendricks\n * Ramesh Gopinath\n * Markus Lang\n * Jan Erik Odegard\n * Dong Wei \n * Joshua Jackson\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/CMakeLists.txt", "content": "cmake_minimum_required (VERSION 2.6)\nproject (rwt)\nsubdirs(lib/src)\nsubdirs(doc)\nsubdirs(python)\n\n#set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} \"${CMAKE_SOURCE_DIR}/cmake/\")\n#FIND_PACKAGE(MatlabMex REQUIRED)\n\n# This section based on http://www.cmake.org/pipermail/cmake/2003-June/003953.html\nIF (UNIX)\n ADD_CUSTOM_TARGET (distclean @echo cleaning for source distribution)\n SET(DISTCLEANED\n CMakeFiles\n cmake.depends\n cmake.check_depends\n CMakeCache.txt\n cmake.check_cache\n Makefile\n *.cmake\n */CMakeCache.txt\n */CMakeFiles\n */Makefile\n */*.cmake\n */*/CMakeCache.txt\n */*/CMakeFiles\n */*/*.cmake\n */*/Makefile\n lib/src/*.a\n doc/Doxyfile\n doc/html\n doc/latex\n core core.*\n gmon.out\n */*.mex*\n */*.o\n lib/src/*.o\n python/rwtPYTHON_wrap.cxx\n python/rwt.py\n python/rwt.pyc\n python/_rwt.so\n *~\n )\n \n ADD_CUSTOM_COMMAND(\n DEPENDS clean\n COMMENT \"distribution clean\"\n COMMAND rm\n ARGS -Rf CMakeTmp ${DISTCLEANED}\n TARGET distclean\n )\nENDIF(UNIX)\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/HACKING", "content": "PHREAK\nLook, you wanna be elite? You gotta do a\nrighteous hack. None of this accidental shit.\n\nCEREAL\nOh yeah, you want a seriously righteous hack,\nyou score one of those Gibsons man. You know,\nsupercomputers they use to like, do physics,\nand look for oil and stuff?\n\n-Hackers (1995)\n\n================================================================================\n= INTRODUCTION\n================================================================================\n\nThere are a number of ways in which wavelet toolbox might be expanded or\nimproved. Arbitrary dimension handling or just 3d, non-orthogonal wavelets,\nsupport for other environments, and so on. If you are seeking to implement these\nor any other changes, this document will be your launching point. All of this \napplies to the state of the code as of the 3.0 release - if this sentence \nhasn't been updated but the code has then you may assume that some of the rest \nof this text may be outdated.\n\n================================================================================\n= DEVELOPER DOCUMENTATION\n================================================================================\n\nYou will likely want to begin by building the documentation files. You'll need\nCMake, Doxygen, and GraphViz. Hopefully you are on Linux, OSX, or some other \nUnix flavor. If you are using Windows as your primary development platform, take \nthis time to stop and think about your life and where it's going. Ok, so to \nbuild the documentation run:\n cmake .\n make doc\nAfter this you should see HTML documentation in doc/html and a pdf with the\nsame content at doc/latex/refman.pdf\n\nIf you haven't used Doxygen before, here's what you need to know: put a ! after\nthe opening /* in a comment to include that comment in the generated docs, look\nat the top of some existing functions to see how function parameters are shown,\nand note that you can use latex formulas.\n\n================================================================================\n= UNIT TESTS\n================================================================================\n\nBefore changing anything you should make your way over to the tests directory\nfrom the MATLAB prompt and run:\n runtests\nWitness how all of these tests pass. Take care that this is still true after\nany changes you make. Now of course you have an IQ that must be measured using\n2-byte integers and have never once introduced a bug into computer software,\nbut these tests are important so that mortals can come along later and know\nwhether a change they made broke some function of the software or not. \n\nThere are also Python tests at python/test_rwt.py and these are mostly the same\nas the MATLAB tests. If you are feeling adventurous you might look into unifying\nthese into a single script that generates both sets of tests. You should run\nboth sets of tests before publishing any commit that could concievably affect\nthem. If you don't know every line of code in both platforms intimately then you\nshould take the safe route and run both test scripts, and possibly the Octave\ntests as well. Unfortunately Octave lacks a lot of things that would be needed\nto run MATLAB xUnit.\n\n================================================================================\n= TOUR OF THE C CODE\n================================================================================\n\nAs of version 3.0, all MATLAB-specific C code has been isolated to a few places.\nThe files in the mex/ directory are MATLAB MEX wrappers for the transforms and\nthese files are intended to be as short as possible. All the initialization\ncode common to the different transforms is found in lib/src/init.c and some of\nthat code is also shared with Python.\n\nThe real magic of making things work across different environments is in \nlib/inc/rwt_platform.h - in particular the mat() macro abstracts away memory \naddressing so you don't have to worry about row major order and column major \norder. The rwt_printf, mat_offset, offset_row, and offset_col macros will be\nvery useful if you need to change any of the code that uses the mat() macro.\n\nTo understand the code for the transforms themselves, start with lib/src/dwt.c\nwhich is the best documented of the transforms. The rest of them are written\nand structured in a very similar fashion.\n\nThe flow of the code is as follows. One of the transforms is called from MATLAB.\nThis invokes one of the wrappers from the mex directory. The function here calls\nrwt_matlab_init in lib/src/init.c which calls other init functions. From here \nthe mex wrapper calls the transform in lib/src. For example, the mdwt function\nfor the discrete wavlet transform calls dwt() in the lib/src/dwt.c file. This\nfunction has a few helpers in the same file. It allocates memory necessary for\nthe transform in dwt_allocate(), calculates the high and low pass coefficients\nin dwt_coefficients(), performs the convolution in dwt_convolution, and frees\nthe allocated memory in the dwt_free() function.\n\nIn the case of Python, a python wrapper function in python/rwt.i calls some of\nthe same initialization code in lib/src/init.c then decides if the input is 1D\nor 2D and calls a matching C wrapper function, also found in the python/rwt.i\nfile. Finally, this wrapper function calls the transform function found in the\nlib/src directory.\n\n================================================================================\n= PYTHON / NUMPY\n================================================================================\n\nThe HardTh, SoftTh, daubcqf, denoise, and makesig functions are implemeted twice \n- once in MATLAB and once in Python. This was simpler than rewriting them in C.\nIf you change any of these you will be glad to see that MATLAB and numpy are \nextremely similar.\n\nHere follows the differences you may need to know. MATLAB indexes start at 1 and\nnumpy starts at 0. For three part indexes the order changes - a[b:c:d] in MATLAB\ncode corresponds to a[b-1:d:c] in Python/numpy. You must use the ddof=1 argument\nto the std() function in Python. The size() function in MATLAB returns 2 numbers\nfor 1D inputs and the same function in Python returns 1 number. MATLAB assumes\nadditional return values beyond the 1st should be dropped if not assigned to\na separate variable - Python does not.\n\nA quick look over the SWIG/numpy documentation might lead you to think that you\ncould use OUTPUT_ARRAY or INPLACE_FARRAY or some other macro to change how the\npython bindings work to be more reasonable. You are probably wrong. Probably.\n\n================================================================================\n= THE BUILD SYSTEM\n================================================================================\n\nThe CMake build system was selected for its license similarity to Wavelet \nToolbox itself, though this is not strictly necessary. CMake also allows for\nsophisticated results with relatively little work. Anything you want to do will\nlikely require some searching and playing. You may be tempted to switch to some\nother more common build system, but this would probably only make things worse.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/INSTALL", "content": "================================================================================\n= MATLAB Installation Instructions =\n================================================================================\n\n1. Make sure you have the latest source code. See the GitHub page at\n https://github.com/ricedsp/rwt\n If you have the command line git tool installed you should be able to run:\n git clone https://github.com/ricedsp/rwt.git\n\n2. Properly set up your system to create MEX-files. Refer to the MATLAB \n documentation section \"Build MEX-Files\" at \n http://www.mathworks.com/help/matlab/matlab_external/building-mex-files.html\n\n3. Run MATLAB and change to the \"bin\" subdirectory containing the .m files\n\n4. Compile the toolbox by executing the Matlab command: compile\n\n5. Add the toolbox \"bin\" subdirectory to your Matlab path.\n\n================================================================================\n= Octave Installation Instructions =\n================================================================================\n\nOctave installation is similar to the procedure for MATLAB above. On Linux you\nwill need the octave-dev (Debian/Ubuntu) or octave-devel (RedHat, etc.) package\ninstalled.\n\n================================================================================\n= Python Installation Instructions =\n================================================================================\n\nPython installation requires SWIG version 2.0.11 or greater and CMake. Also you\nshould have numpy and scipy installed. To install the python bindings, execute\nthe following commands:\n cd python\n cmake .\n sudo make install\n\nOn OSX, CMake is available from Macports http://www.macports.org/\nFor Redhat Enterprise Linux, Scientific Linux, CentOS, etc. there is a package\navailable on RepoForge http://repoforge.org/use/\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/LICENSE", "content": "Copyright (c) 2000 RICE UNIVERSITY. All rights reserved.\n\nThis software is distributed and licensed to you on a non-exclusive \nbasis, free-of-charge. Redistribution and use in source and binary forms, \nwith or without modification, are permitted provided that the following \nconditions are met:\n\n1. Redistribution of source code must retain the above copyright notice, \n this list of conditions and the following disclaimer.\n2. Redistribution in binary form must reproduce the above copyright notice, \n this list of conditions and the following disclaimer in the \n documentation and/or other materials provided with the distribution.\n3. Neither the name of the University nor the names of its contributors \n may be used to endorse or promote products derived from this software \n without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY WILLIAM MARSH RICE UNIVERSITY, HOUSTON, TEXAS, \nAND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, \nBUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS \nFOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RICE UNIVERSITY \nOR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, \nEXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, \nPROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; \nOR BUSINESS INTERRUPTIONS) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, \nWHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR \nOTHERWISE), PRODUCT LIABILITY, OR OTHERWISE ARISING IN ANY WAY OUT OF THE \nUSE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\nFor information on commercial licenses, contact Rice University's Office of \nTechnology Transfer at techtran@rice.edu or (713) 348-6173\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/HardTh.m", "content": "function x = HardTh(y,thld)\n% x = HardTh(y,thld); \n%\n% HARDTH hard thresholds the input signal y with the threshold value\n% thld.\n%\n% Input: \n% y : 1D or 2D signal to be thresholded\n% thld : threshold value\n%\n% Output: \n% x : Hard thresholded output (x = (abs(y)>thld).*y)\n%\n% HERE'S AN EASY WAY TO RUN THE EXAMPLES:\n% Cut-and-paste the example you want to run to a new file \n% called ex.m, for example. Delete out the % at the beginning \n% of each line in ex.m (Can use search-and-replace in your editor\n% to replace it with a space). Type 'ex' in matlab and hit return.\n%\n%\n% Example:\n% y = makesig('WernerSorrows',8);\n% thld = 1;\n% x = HardTh(y,thld)\n% x = 1.5545 5.3175 0 1.6956 -1.2678 0 1.7332 0\n%\n% See also: SoftTh\n%\n%Author: Haitao Guo \n\nx = (abs(y) > thld).*y; \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/SoftTh.m", "content": "function x = SoftTh(y,thld)\n% x = SoftTh(y,thld); \n%\n% SOFTTH soft thresholds the input signal y with the threshold value\n% thld.\n%\n% Input: \n% y : 1D or 2D signal to be thresholded\n% thld : Threshold value\n%\n% Output: \n% x : Soft thresholded output (x = sign(y)(|y|-thld)_+)\n%\n% HERE'S AN EASY WAY TO RUN THE EXAMPLES:\n% Cut-and-paste the example you want to run to a new file \n% called ex.m, for example. Delete out the % at the beginning \n% of each line in ex.m (Can use search-and-replace in your editor\n% to replace it with a space). Type 'ex' in matlab and hit return.\n%\n%\n% Example:\n% y = makesig('Doppler',8);\n% thld = 0.2;\n% x = SoftTh(y,thld)\n% x = 0 0 0 -0.0703 0 0.2001 0.0483 0 \n%\n% See also: HardTh\n%\n% Reference: \n% \"De-noising via Soft-Thresholding\" Tech. Rept. Statistics,\n% Stanford, 1992. D.L. Donoho.\n%\n%Author: Haitao Guo \n\nx = abs(y);\nx = sign(y).*(x >= thld).*(x - thld); \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/compile.m", "content": "% COMPILE compiles the c files and generates mex files.\n%\n\nif exist('OCTAVE_VERSION', 'builtin')\n mkoctfile --mex -v -DOCTAVE_MEX_FILE ../mex/mdwt.c ../lib/src/dwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -o omdwt.mex\n mkoctfile --mex -v -DOCTAVE_MEX_FILE ../mex/midwt.c ../lib/src/idwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -o omidwt.mex\n mkoctfile --mex -v -DOCTAVE_MEX_FILE ../mex/mrdwt.c ../lib/src/rdwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -o omrdwt.mex\n mkoctfile --mex -v -DOCTAVE_MEX_FILE ../mex/mirdwt.c ../lib/src/irdwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -o omirdwt.mex\nelse\n x = computer();\n if (x(length(x)-1:length(x)) == '64')\n mex -v -largeArrayDims ../mex/mdwt.c ../lib/src/dwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -outdir ../bin\n mex -v -largeArrayDims ../mex/midwt.c ../lib/src/idwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -outdir ../bin\n mex -v -largeArrayDims ../mex/mrdwt.c ../lib/src/rdwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -outdir ../bin\n mex -v -largeArrayDims ../mex/mirdwt.c ../lib/src/irdwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -outdir ../bin\n else\n mex -v -compatibleArrayDims ../mex/mdwt.c ../lib/src/dwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -outdir ../bin\n mex -v -compatibleArrayDims ../mex/midwt.c ../lib/src/idwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -outdir ../bin\n mex -v -compatibleArrayDims ../mex/mrdwt.c ../lib/src/rdwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -outdir ../bin\n mex -v -compatibleArrayDims ../mex/mirdwt.c ../lib/src/irdwt.c ../lib/src/init.c ../lib/src/platform.c -I../lib/inc -outdir ../bin\n end\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/daubcqf.m", "content": "function [h_0,h_1] = daubcqf(N,TYPE)\n% [h_0,h_1] = daubcqf(N,TYPE); \n%\n% Function computes the Daubechies' scaling and wavelet filters\n% (normalized to sqrt(2)).\n%\n% Input: \n% N : Length of filter (must be even)\n% TYPE : Optional parameter that distinguishes the minimum phase,\n% maximum phase and mid-phase solutions ('min', 'max', or\n% 'mid'). If no argument is specified, the minimum phase\n% solution is used.\n%\n% Output: \n% h_0 : Minimal phase Daubechies' scaling filter \n% h_1 : Minimal phase Daubechies' wavelet filter \n%\n% Example:\n% N = 4;\n% TYPE = 'min';\n% [h_0,h_1] = daubcqf(N,TYPE)\n% h_0 = 0.4830 0.8365 0.2241 -0.1294\n% h_1 = 0.1294 0.2241 -0.8365 0.4830\n%\n% Reference: \"Orthonormal Bases of Compactly Supported Wavelets\",\n% CPAM, Oct.89 \n%\n%Author: Ramesh Gopinath \n\nif(nargin < 2),\n TYPE = 'min';\nend;\nif(rem(N,2) ~= 0),\n error('No Daubechies filter exists for ODD length');\nend;\nK = N/2;\na = 1;\np = 1;\nq = 1;\nh_0 = [1 1];\nfor j = 1:K-1,\n a = -a * 0.25 * (j + K - 1)/j;\n h_0 = [0 h_0] + [h_0 0];\n p = [0 -p] + [p 0];\n p = [0 -p] + [p 0];\n q = [0 q 0] + a*p;\nend;\nq = sort(roots(q));\nqt = q(1:K-1);\nif TYPE=='mid',\n if rem(K,2)==1, \n qt = q([1:4:N-2 2:4:N-2]);\n else\n qt = q([1 4:4:K-1 5:4:K-1 N-3:-4:K N-4:-4:K]);\n end;\nend;\nh_0 = conv(h_0,real(poly(qt)));\nh_0 = sqrt(2)*h_0/sum(h_0); \t%Normalize to sqrt(2);\nif(TYPE=='max'),\n h_0 = fliplr(h_0);\nend;\nif(abs(sum(h_0 .^ 2))-1 > 1e-4) \n error('Numerically unstable for this value of \"N\".');\nend;\nh_1 = rot90(h_0,2);\nh_1(1:2:N)=-h_1(1:2:N);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/denoise.m", "content": "function [xd,xn,option] = denoise(x,h,type,option)\n% [xd,xn,option] = denoise(x,h,type,option); \n%\n% DENOISE is a generic program for wavelet based denoising.\n% The program will denoise the signal x using the 2-band wavelet\n% system described by the filter h using either the traditional \n% discrete wavelet transform (DWT) or the linear shift invariant \n% discrete wavelet transform (also known as the undecimated DWT\n% (UDWT)). \n%\n% Input: \n% x : 1D or 2D signal to be denoised\n% h : Scaling filter to be applied\n% type : Type of transform (Default: type = 0)\n% 0 --> Discrete wavelet transform (DWT)\n% 1 --> Undecimated DWT (UDWT)\n% option : Default settings is marked with '*':\n% *type = 0 --> option = [0 3.0 0 0 0 0]\n% type = 1 --> option = [0 3.6 0 1 0 0]\n% option(1) : Whether to threshold low-pass part\n% 0 --> Don't threshold low pass component \n% 1 --> Threshold low pass component\n% option(2) : Threshold multiplier, c. The threshold is\n% computed as: \n% thld = c*MAD(noise_estimate)). \n% The default values are:\n% c = 3.0 for the DWT based denoising\n% c = 3.6 for the UDWT based denoising\n% option(3) : Type of variance estimator\n% 0 --> MAD (mean absolute deviation)\n% 1 --> STD (classical numerical std estimate)\n% option(4) : Type of thresholding\n% 2 --> Soft thresholding\n% 1 --> Hard thresholding\n% option(5) : Number of levels, L, in wavelet decomposition. By\n% setting this to the default value '0' a maximal\n% decomposition is used.\n% option(6) : Actual threshold to use (setting this to\n% anything but 0 will mean that option(3)\n% is ignored)\n%\n% Output: \n% xd : Estimate of noise free signal \n% xn : The estimated noise signal (x-xd)\n% option : A vector of actual parameters used by the\n% program. The vector is configured the same way as\n% the input option vector with one added element\n% option(7) = type.\n%\n% HERE'S AN EASY WAY TO RUN THE EXAMPLES:\n% Cut-and-paste the example you want to run to a new file \n% called ex.m, for example. Delete out the % at the beginning \n% of each line in ex.m (Can use search-and-replace in your editor\n% to replace it with a space). Type 'ex' in matlab and hit return.\n%\n% Example 1: \n% h = daubcqf(6); [s,N] = makesig('Doppler'); n = randn(1,N);\n% x = s + n/10; % (approximately 10dB SNR)\n% figure;plot(x);hold on;plot(s,'r');\n%\n% %Denoise x with the default method based on the DWT\n% [xd,xn,opt1] = denoise(x,h);\n% figure;plot(xd);hold on;plot(s,'r');\n%\n% %Denoise x using the undecimated (LSI) wavelet transform\n% [yd,yn,opt2] = denoise(x,h,1);\n% figure;plot(yd);hold on;plot(s,'r');\n%\n% Example 2: (on an image) \n% h = daubcqf(6); load lena; \n% noisyLena = lena + 25 * randn(size(lena));\n% figure; colormap(gray); imagesc(lena); title('Original Image');\n% figure; colormap(gray); imagesc(noisyLena); title('Noisy Image'); \n% Denoise lena with the default method based on the DWT\n% [denoisedLena,xn,opt1] = denoise(noisyLena,h);\n% figure; colormap(gray); imagesc(denoisedLena); title('denoised Image');\n% \n%\n% See also: mdwt, midwt, mrdwt, mirdwt, SoftTh, HardTh, setopt\n%\n%Author: Jan Erik Odegard \n\nif(nargin < 2)\n error('You need to provide at least 2 inputs: x and h');\nend;\nif(nargin < 3),\n type = 0;\n option = [];\nelseif(nargin < 4)\n option = [];\nend;\nif(isempty(type)),\n type = 0;\nend;\nif(type == 0),\n default_opt = [0 3.0 0 2 0 0];\nelseif(type == 1),\n default_opt = [0 3.6 0 1 0 0];\nelse\n error(['Unknown denoising method',10,...\n\t 'If it is any good we need to have a serious talk :-)']);\nend;\noption = setopt(option,default_opt);\n[mx,nx] = size(x);\ndim = min(mx,nx);\nif(dim == 1),\n n = max(mx,nx);\nelse\n n = dim;\nend;\nif(option(5) == 0),\n L = floor(log2(n));\nelse\n L = option(5);\nend;\nif(type == 0), \t\t\t% Denoising by DWT\n xd = mdwt(x,h,L);\n if (option(6) == 0),\n tmp = xd(floor(mx/2)+1:mx,floor(nx/2)+1:nx);\n if(option(3) == 0),\n thld = option(2)*median(abs(tmp(:)))/.67;\n elseif(option(3) == 1),\n thld = option(2)*std(tmp(:));\n else\n error('Unknown threshold estimator, Use either MAD or STD');\n end;\n else\n thld = option(6);\n end;\n if(dim == 1)\n ix = 1:n/(2^L);\n ykeep = xd(ix);\n else\n ix = 1:mx/(2^L);\n jx = 1:nx/(2^L);\n ykeep = xd(ix,jx);\n end;\n if(option(4) == 2),\n xd = SoftTh(xd,thld);\n elseif(option(4) == 1),\n xd = HardTh(xd,thld);\n else\n error('Unknown threshold rule. Use either Soft (2) or Hard (1)');\n end;\n if (option(1) == 0),\n if(dim == 1),\n xd(ix) = ykeep;\n else\n xd(ix,jx) = ykeep;\n end;\n end;\n xd = midwt(xd,h,L);\nelseif(type == 1), \t\t\t% Denoising by UDWT\n [xl,xh] = mrdwt(x,h,L);\n if(dim == 1),\n c_offset = 1;\n else\n c_offset = 2*nx + 1;\n end;\n if (option(6) == 0),\n tmp = xh(:,c_offset:c_offset+nx-1);\n if(option(3) == 0),\n thld = option(2)*median(abs(tmp(:)))/.67;\n elseif(option(3) == 1),\n thld = option(2)*std(tmp(:));\n else\n error('Unknown threshold estimator, Use either MAD or STD');\n end;\n else\n thld = option(6);\n end;\n if(option(4) == 2),\n xh = SoftTh(xh,thld);\n if(option(1) == 1),\n xl = SoftTh(xl,thld);\n end;\n elseif(option(4) == 1),\n xh = HardTh(xh,thld);\n if(option(1) == 1),\n xl = HardTh(xl,thld);\n end;\n else\n error('Unknown threshold rule. Use either Soft (2) or Hard (1)');\n end;\n xd = mirdwt(xl,xh,h,L);\nelse \t\t\t\t\t% Denoising by unknown method\n error(['Unknown denoising method',10,...\n 'If it is any good we need to have a serious talk :-)']);\nend;\noption(6) = thld;\noption(7) = type;\nxn = x - xd; \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/makesig.m", "content": "function [x,N] = makesig(SigName,N)\n% [x,N] = makesig(SigName,N) Creates artificial test signal identical to the\n% standard test signals proposed and used by D. Donoho and I. Johnstone\n% in WaveLab (- a matlab toolbox developed by Donoho et al. the statistics\n% department at Stanford University).\n%\n% Input: SigName - Name of the desired signal (Default 'all')\n% 'AllSig' (Returns a matrix with all the signals)\n% 'HeaviSine'\n% 'Bumps'\n% 'Blocks'\n% 'Doppler'\n% 'Ramp'\n% 'Cusp'\n% 'Sing'\n% 'HiSine'\n% 'LoSine'\n% 'LinChirp'\n% 'TwoChirp'\n% 'QuadChirp'\n% 'MishMash'\n% 'WernerSorrows' (Heisenberg)\n% 'Leopold' (Kronecker)\n% N - Length in samples of the desired signal (Default 512)\n%\n% Output: x - vector/matrix of test signals\n% N - length of signal returned\n%\n% See also: \n%\n% References:\n% WaveLab can be accessed at\n% www_url: http://playfair.stanford.edu/~wavelab/\n% Also see various articles by D.L. Donoho et al. at\n% web_url: http://playfair.stanford.edu/\n%\n%Author: Jan Erik Odegard \n%This m-file is a copy of the code provided with WaveLab\n%customized to be consistent with RWT.\n\nif(nargin < 1)\n SigName = 'AllSig';\n N = 512;\nelseif(nargin == 1)\n N = 512;\nend;\nt = (1:N) ./N;\nx = [];\ny = [];\nif(strcmp(SigName,'HeaviSine') | strcmp(SigName,'AllSig')),\n y = 4.*sin(4*pi.*t);\n y = y - sign(t - .3) - sign(.72 - t);\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'Bumps') | strcmp(SigName,'AllSig')),\n pos = [ .1 .13 .15 .23 .25 .40 .44 .65 .76 .78 .81];\n hgt = [ 4 5 3 4 5 4.2 2.1 4.3 3.1 5.1 4.2];\n wth = [.005 .005 .006 .01 .01 .03 .01 .01 .005 .008 .005];\n y = zeros(size(t));\n for j =1:length(pos)\n y = y + hgt(j)./( 1 + abs((t - pos(j))./wth(j))).^4;\n end \nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'Blocks') | strcmp(SigName,'AllSig')),\n pos = [ .1 .13 .15 .23 .25 .40 .44 .65 .76 .78 .81];\n hgt = [4 (-5) 3 (-4) 5 (-4.2) 2.1 4.3 (-3.1) 2.1 (-4.2)];\n y = zeros(size(t));\n for j=1:length(pos)\n y = y + (1 + sign(t-pos(j))).*(hgt(j)/2) ;\n end\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'Doppler') | strcmp(SigName,'AllSig')),\n y = sqrt(t.*(1-t)).*sin((2*pi*1.05) ./(t+.05));\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'Ramp') | strcmp(SigName,'AllSig')),\n y = t - (t >= .37);\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'Cusp') | strcmp(SigName,'AllSig')),\n y = sqrt(abs(t - .37));\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'Sing') | strcmp(SigName,'AllSig')),\n k = floor(N * .37);\n y = 1 ./abs(t - (k+.5)/N);\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'HiSine') | strcmp(SigName,'AllSig')),\n y = sin( pi * (N * .6902) .* t);\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'LoSine') | strcmp(SigName,'AllSig')),\n y = sin( pi * (N * .3333) .* t);\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'LinChirp') | strcmp(SigName,'AllSig')),\n y = sin(pi .* t .* ((N .* .125) .* t));\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'TwoChirp') | strcmp(SigName,'AllSig')),\n y = sin(pi .* t .* (N .* t)) + sin((pi/3) .* t .* (N .* t));\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'QuadChirp') | strcmp(SigName,'AllSig')),\n y = sin( (pi/3) .* t .* (N .* t.^2));\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'MishMash') | strcmp(SigName,'AllSig')), \n % QuadChirp + LinChirp + HiSine\n y = sin( (pi/3) .* t .* (N .* t.^2)) ;\n y = y + sin( pi * (N * .6902) .* t);\n y = y + sin(pi .* t .* (N .* .125 .* t));\nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'WernerSorrows') | strcmp(SigName,'AllSig')),\n y = sin( pi .* t .* (N/2 .* t.^2)) ;\n y = y + sin( pi * (N * .6902) .* t);\n y = y + sin(pi .* t .* (N .* t));\n pos = [ .1 .13 .15 .23 .25 .40 .44 .65 .76 .78 .81];\n hgt = [ 4 5 3 4 5 4.2 2.1 4.3 3.1 5.1 4.2];\n wth = [.005 .005 .006 .01 .01 .03 .01 .01 .005 .008 .005];\n for j =1:length(pos)\n y = y + hgt(j)./( 1 + abs((t - pos(j))./wth(j))).^4;\n end \nend;\nx = [x;y];\ny = [];\nif(strcmp(SigName,'Leopold') | strcmp(SigName,'AllSig')),\n y = (t == floor(.37 * N)/N); \t\t% Kronecker\nend;\nx = [x;y];\ny = [];\n\n% disp(sprintf('MakeSignal: I don*t recognize << %s>>',SigName))\n% disp('Allowable SigNames are:')\n% disp('AllSig'),\n% disp('HeaviSine'),\n% disp('Bumps'),\n% disp('Blocks'),\n% disp('Doppler'),\n% disp('Ramp'),\n% disp('Cusp'),\n% disp('Crease'),\n% disp('Sing'),\n% disp('HiSine'),\n% disp('LoSine'),\n% disp('LinChirp'),\n% disp('TwoChirp'),\n% disp('QuadChirp'),\n% disp('MishMash'),\n% disp('WernerSorrows'),\n% disp('Leopold'),\n%end\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/mdwt.m", "content": "function [y,L] = mdwt(x,h,L)\n% [y,L] = mdwt(x,h,L);\n%\n% Function computes the discrete wavelet transform y for a 1D or 2D input\n% signal x using the scaling filter h.\n%\n% Input:\n%\tx : finite length 1D or 2D signal (implicitly periodized)\n% h : scaling filter\n% L : number of levels. In the case of a 1D signal, length(x) must be\n% divisible by 2^L; in the case of a 2D signal, the row and the\n% column dimension must be divisible by 2^L. If no argument is\n% specified, a full DWT is returned for maximal possible L.\n%\n% Output:\n% y : the wavelet transform of the signal \n% (see example to understand the coefficients)\n% L : number of decomposition levels\n%\n% 1D Example:\n% x = makesig('LinChirp',8);\n% h = daubcqf(4,'min');\n% L = 2;\n% [y,L] = mdwt(x,h,L)\n%\n% 1D Example's output and explanation:\n%\n% y = [1.1097 0.8767 0.8204 -0.5201 -0.0339 0.1001 0.2201 -0.1401]\n% L = 2\n%\n% The coefficients in output y are arranged as follows\n%\n% y(1) and y(2) : Scaling coefficients (lowest frequency)\n% y(3) and y(4) : Band pass wavelet coefficients\n% y(5) to y(8) : Finest scale wavelet coefficients (highest frequency)\n%\n% 2D Example:\n%\n% load test_image \n% h = daubcqf(4,'min');\n% L = 1;\n% [y,L] = mdwt(test_image,h,L);\n%\n% 2D Example's output and explanation:\n%\n% The coefficients in y are arranged as follows.\n%\n% .------------------.\n% | | |\n% | 4 | 2 |\n% | | |\n% | L,L | H,L |\n% | | |\n% --------------------\n% | | |\n% | 3 | 1 |\n% | | |\n% | L,H | H,H |\n% | | |\n% `------------------'\n% \n% where \n% 1 : High pass vertically and high pass horizontally\n% 2 : Low pass vertically and high pass horizontally\n% 3 : High pass vertically and low pass horizontally\n% 4 : Low pass vertically and Low pass horizontally \n% (scaling coefficients)\n%\n%\n%\n%\n% See also: midwt, mrdwt, mirdwt\n%\n%Author: Markus Lang \nif exist('OCTAVE_VERSION', 'builtin')\n x = x * 1.0;\n if (exist('L'))\n [y,L] = omdwt(x,h,L);\n else \n [y,L] = omdwt(x,h);\n end\nelse\n error('You must compile wavelet toolbox before use')\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/midwt.m", "content": "function [y,L] = midwt(x,h,L)\n% [x,L] = midwt(y,h,L);\n% \n% Function computes the inverse discrete wavelet transform x for a 1D or\n% 2D input signal y using the scaling filter h.\n%\n% Input:\n%\ty : finite length 1D or 2D input signal (implicitly periodized)\n% (see function mdwt to find the structure of y)\n% h : scaling filter\n% L : number of levels. In the case of a 1D signal, length(x) must be\n% divisible by 2^L; in the case of a 2D signal, the row and the\n% column dimension must be divisible by 2^L. If no argument is\n% specified, a full inverse DWT is returned for maximal possible\n% L.\n%\n% Output:\n% x : periodic reconstructed signal\n% L : number of decomposition levels\n%\n% 1D Example:\n% xin = makesig('LinChirp',8);\n% h = daubcqf(4,'min');\n% L = 1;\n% [y,L] = mdwt(xin,h,L);\n% [x,L] = midwt(y,h,L)\n%\n% 1D Example's output:\n%\n% x = 0.0491 0.1951 0.4276 0.7071 0.9415 0.9808 0.6716 0.0000\n% L = 1\n%\n% See also: mdwt, mrdwt, mirdwt\n%\n%Author: Markus Lang \nif exist('OCTAVE_VERSION', 'builtin')\n if (exist('L'))\n [y,L] = omidwt(x,h,L);\n else \n [y,L] = omidwt(x,h);\n end\nelse\n error('You must compile wavelet toolbox before use')\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/mirdwt.m", "content": "function [x,L] = mirdwt(yl,yh,h,L)\n% function [x,L] = mirdwt(yl,yh,h,L);\n% \n% Function computes the inverse redundant discrete wavelet\n% transform x for a 1D or 2D input signal. (Redundant means here\n% that the sub-sampling after each stage of the forward transform\n% has been omitted.) yl contains the lowpass and yl the highpass\n% components as computed, e.g., by mrdwt. In the case of a 2D\n% signal, the ordering in\n% yh is [lh hl hh lh hl ... ] (first letter refers to row, second\n% to column filtering). \n%\n% Input:\n% yl : lowpass component\n% yh : highpass components\n% h : scaling filter\n% L : number of levels. In the case of a 1D signal, \n% length(yl) must be divisible by 2^L;\n% in the case of a 2D signal, the row and\n% the column dimension must be divisible by 2^L.\n% \n% Output:\n%\t x : finite length 1D or 2D signal\n%\t L : number of levels\n%\n% HERE'S AN EASY WAY TO RUN THE EXAMPLES:\n% Cut-and-paste the example you want to run to a new file \n% called ex.m, for example. Delete out the % at the beginning \n% of each line in ex.m (Can use search-and-replace in your editor\n% to replace it with a space). Type 'ex' in matlab and hit return.\n%\n%\n% Example 1:\n% xin = makesig('Leopold',8);\n% h = daubcqf(4,'min');\n% L = 1;\n% [yl,yh,L] = mrdwt(xin,h,L);\n% [x,L] = mirdwt(yl,yh,h,L)\n% x = 0.0000 1.0000 0.0000 -0.0000 0 0 0 -0.0000\n% L = 1\n% \n% Example 2: \n% load lena;\n% h = daubcqf(4,'min');\n% L = 2;\n% [ll_lev2,yh,L] = mrdwt(lena,h,L); % lena is a 256x256 matrix\n% N = 256;\n% lh_lev1 = yh(:,1:N); \n% hl_lev1 = yh(:,N+1:2*N); \n% hh_lev1 = yh(:,2*N+1:3*N);\n% lh_lev2 = yh(:,3*N+1:4*N); \n% hl_lev2 = yh(:,4*N+1:5*N); \n% hh_lev2 = yh(:,5*N+1:6*N);\n% figure; colormap(gray); imagesc(lena); title('Original Image');\n% figure; colormap(gray); imagesc(ll_lev2); title('LL Level 2');\n% figure; colormap(gray); imagesc(hh_lev2); title('HH Level 2');\n% figure; colormap(gray); imagesc(hl_lev2); title('HL Level 2');\n% figure; colormap(gray); imagesc(lh_lev2); title('LH Level 2');\n% figure; colormap(gray); imagesc(hh_lev1); title('HH Level 1');\n% figure; colormap(gray); imagesc(hl_lev2); title('HL Level 1');\n% figure; colormap(gray); imagesc(lh_lev2); title('LH Level 1');\n% [lena_Hat,L] = mirdwt(ll_lev2,yh,h,L);\n% figure; colormap(gray); imagesc(lena_Hat); \n% title('Reconstructed Image');\n%\n% See also: mdwt, midwt, mrdwt\n%\n% Warning! min(size(yl))/2^L should be greater than length(h)\n%\n%Author: Markus Lang \nif exist('OCTAVE_VERSION', 'builtin')\n yl = yl * 1.0;\n yh = yh * 1.0;\n if (exist('L'))\n [x,L] = omirdwt(yl,yh,h,L);\n else \n [x,L] = omirdwt(yl,yh,h);\n end\nelse\n error('You must compile wavelet toolbox before use')\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/mrdwt.m", "content": "function [yl,yh,L] = mrdwt(x,h,L)\n% [yl,yh,L] = mrdwt(x,h,L);\n% \n% Function computes the redundant discrete wavelet transform y\n% for a 1D or 2D input signal. (Redundant means here that the\n% sub-sampling after each stage is omitted.) yl contains the\n% lowpass and yh the highpass components. In the case of a 2D\n% signal, the ordering in yh is \n% [lh hl hh lh hl ... ] (first letter refers to row, second to\n% column filtering). \n%\n% Input:\n%\t x : finite length 1D or 2D signal (implicitly periodized)\n% h : scaling filter\n% L : number of levels. In the case of a 1D \n% length(x) must be divisible by 2^L;\n% in the case of a 2D signal, the row and the\n% column dimension must be divisible by 2^L.\n% If no argument is\n% specified, a full DWT is returned for maximal possible L.\n% \n% Output:\n% yl : lowpass component\n% yh : highpass components\n% L : number of levels\n%\n% HERE'S AN EASY WAY TO RUN THE EXAMPLES:\n% Cut-and-paste the example you want to run to a new file \n% called ex.m, for example. Delete out the % at the beginning \n% of each line in ex.m (Can use search-and-replace in your editor\n% to replace it with a space). Type 'ex' in matlab and hit return.\n%\n%\n% Example 1::\n% x = makesig('Leopold',8);\n% h = daubcqf(4,'min');\n% L = 1;\n% [yl,yh,L] = mrdwt(x,h,L)\n% yl = 0.8365 0.4830 0 0 0 0 -0.1294 0.2241\n% yh = -0.2241 -0.1294 0 0 0 0 -0.4830 0.8365\n% L = 1\n% Example 2:\n% load lena;\n% h = daubcqf(4,'min');\n% L = 2;\n% [ll_lev2,yh,L] = mrdwt(lena,h,L); % lena is a 256x256 matrix\n% N = 256;\n% lh_lev1 = yh(:,1:N); \n% hl_lev1 = yh(:,N+1:2*N); \n% hh_lev1 = yh(:,2*N+1:3*N);\n% lh_lev2 = yh(:,3*N+1:4*N); \n% hl_lev2 = yh(:,4*N+1:5*N); \n% hh_lev2 = yh(:,5*N+1:6*N);\n% figure; colormap(gray); imagesc(lena); title('Original Image');\n% figure; colormap(gray); imagesc(ll_lev2); title('LL Level 2');\n% figure; colormap(gray); imagesc(hh_lev2); title('HH Level 2');\n% figure; colormap(gray); imagesc(hl_lev2); title('HL Level 2');\n% figure; colormap(gray); imagesc(lh_lev2); title('LH Level 2');\n% figure; colormap(gray); imagesc(hh_lev1); title('HH Level 1');\n% figure; colormap(gray); imagesc(hl_lev2); title('HL Level 1');\n% figure; colormap(gray); imagesc(lh_lev2); title('LH Level 1');\n% \n% See also: mdwt, midwt, mirdwt\n%\n% Warning! min(size(x))/2^L should be greater than length(h)\n%\n%Author: Markus Lang \nif exist('OCTAVE_VERSION', 'builtin')\n x = x * 1.0;\n if (exist('L'))\n [yl,yh,L] = omrdwt(x,h,L);\n else \n [yl,yh,L] = omrdwt(x,h);\n end\nelse\n error('You must compile wavelet toolbox before use')\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/bin/setopt.m", "content": "function option = setopt(opt_par,default);\n% option = setopt(opt_par,default); \n%\n% SETOPT can modify a default option vector with user specified options.\n%\n% Input: \n% opt_par : Users desired option vector\n% default : Program default option vector\n%\n% Output: \n% option : New option vector\n%\n% Example:\n% opt_par = [1 2 3 4];\n% default = [1 1 1 1];\n% option = setopt(opt_par,default)\n% option = 1 2 3 4\n%\n%Author: Jan Erik Odegard \n\nif (nargin < 2) \n error('You need two inputs');\nend;\nlen = length(opt_par);\noption = zeros(size(default));\noption(1:len) = opt_par(1:len);\noption = option + (option == 0).*default; \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/dist/2.01/INSTALL", "content": "#################################################################################\n#File Name: INSTALL \n#Last Modification Date: 11/16/95 10:30:38\n#Current Version: INSTALL 1.13\n#File Creation Date: Wed Aug 25 09:25:06 1993 \n#Author: Ramesh Gopinath \n#\n#Copyright: All software, documentation, and related files in this distribution\n# are Copyright (c) 1993-1995 Rice University\n#\n#Permission is granted for use and non-profit distribution providing that this\n#notice be clearly maintained. The right to distribute any portion for profit\n#or as part of any commercial product is specifically reserved for the author.\n#\n#Change History:\n#\n#################################################################################\nIn order to install this distribution of wlet-tools:\n\n1. Uncompress and extract the tar archive in the desired directory. \n\n uncompress RWT.tar.Z\n tar xvf RWT.tar\n\n NOTE: New subdirectories (rice-wlet-tools and rice-atr-tools) will\n be generated in directory where you extract the archive.\n\n2. cd rice-wlet-tools\n\n3. make all\n\n4. make install\n\n5. Append the paths to the mex, mfile and wdemo directories. That is,\n\tin .cshrc add the following lines at the end:\n\n setenv RWT_HOME YOUR/LOCAL/PATH/TO\n setenv RWT_PATH $RWT_HOME/rice-wlet-tools/mex:$RWT_HOME/rice-wlet-tools/mfiles:\\\n\t$RWT_HOME/rice-wlet-tools/wdemos:$RWT_HOME/rice-atr-tools/mex:\\\n\t$RWT_HOME/rice-atr-tools/mfiles:$RWT_HOME/rice-atr-tools/sardemo:\\\n\t$RWT_HOME\n setenv MATLABPATH $RWT_PATH':'$MATLABPATH\n\n\twhere \n\n\tYOUR/LOCAL/PATH/TO\n\n\t\t is replaced with\n\t\n\tthe actual path to the directory where rice-wlet-tools and\n\trice-atr-tools are located on your system\n\n NOTE: If you do not have the environment variable MATLABPATH previously defined\n change the line\n\n setenv MATLABPATH $RWT_PATH':'$MATLABPATH\n\n to\n\n setenv MATLABPATH $RWT_PATH\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/dist/2.01/README", "content": "#################################################################################\n#File Name: README \n#Last Modification Date: 9/1/94 10:11:28\n#Current Version: README 1.5\n#File Creation Date: Wed Aug 25 09:25:06 1993 \n#Author: Ramesh Gopinath \n#\n#Copyright: All software, documentation, and related files in this distribution\n# are Copyright (c) 1993 Rice University\n#\n#Permission is granted for use and non-profit distribution providing that this\n#notice be clearly maintained. The right to distribute any portion for profit\n#or as part of any commercial product is specifically reserved for the author.\n#\n#Change History:\n#\n#################################################################################\nThis \"rice-wlet-tools\", version 2.01\nReleased - \n\nINSTALLATION: \nTo install this distribution of wlet-tools see INSTALL.\n\nSOURCE:\n ftp: cml.rice.edu (128.42.62.23) /pub/software\n mosaic: URL http://jazz.rice.edu\n\nAssociated references can be obtained from directory\n/pub/dsp/papers and /pub/reports\n\nEMAIL: \nFor bug reports and questions send email to webmaster-dsp@ece.rice.edu\n\nCONDITIONS FOR USE:\n This software is Copyright (C) Rice University 1993.\n\tYou have the right to use, free of charge, with the following terms \n and conditions:\n\n (1) You can redistribute this software in source form. If you\n redistribute this software in compiled form you will include\n the source code.\n (2) You can distribute your own applications that link this software\n if you include the source code for this software.\n (3) You own full rights to any output files you generate with this\n software.\n (4) You can make modifications to this software and use it for\n in-house use only. Under no circumstances can modified software\n be redistributed.\n (5) If you make any modifications to this software you will send\n the changes by email to webmaster-dsp@ece.rice.edu\n (6) The DSP group at Rice University shall be credited should\n this software be used in in any form or written about in any\n publication.\n (7) This software is provided \"as is\", without warranty by Rice University.\n \t In no event shall Rice University be liable for any loss or for any\n indirect, special, punitive, exemplary, incidental, or\n consequential damages arising from the use, possession or\n performance of this software.\n\n---------\nALTERNATIVE WAY OF GETTING TO SOFTWARE AND REPORTS\n(THIS MIGHT BE DISCONTINUED SINCE IT IS NOT ROBUST):\n It can also be obtained (usually) with the following command on unix systems:\n\n %telnet dsp.rice.edu 5555 |sed '1,3d' | csh -fbs software\n OR %telnet 128.42.4.62 5555 |sed '1,3d' | csh -fbs software\n\nYou probably want to add\n\n\talias riceget \"telnet 128.42.4.62 5555 |sed '1,3d' | csh -fsb\"\n\nso that you can access the distribution (which will hopefully be updated periodically)\n\n\t%riceget OPTIONS\n\nwhere options is a list of options. \n\n\t%riceget help\n\nwould return all options currently available.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/dist/2.01/doc/index.html", "content": " \n Rice Wavelet Toolbox Documentation\n \n \n\n
\n\n\t

Rice Wavelet Toolbox Documentation Version\n\t2.01

\n\n\t

\n\n\t \n\n\t \n\t\t \n\t\t \n\t\t \n\t\t\t \n\t\t\n\t\t\n\t\t \n\t\t \n\t\t \n\t\t \n\t\t \n\t\t\n\n\t\t\n\t\t \n\t\t \n\t\t \n\t\t \n\t\t \n\t\t\n\n\t\t\n\t\t \n\t\t \n\t\t \n\t\t \n\t\t \n\t\t\n\n\t\t\n\t\t \n\t\t \n\t\t \n\t\t \n\t\t \n\t\t\n\t
A Y in the M-file and/or the\n\t MEX-file column indicates whether the given\n\t function is implemented as a matlab M-file, a MEX-file\n\t or both. An X in the Depend column will\n\t indicate that although the function itself is not a\n\t MEX-file it depends on subroutines written in C and\n\t compiled as MEX-files for significant speedup on 2D\n\t problems in particular.\n\n\t\t
Function\n\t\t Description M-fileMEX-fileDepend
\n\t\t denoise Nonlinear wavelet denoising Y X
dwt\n\t\t Computes the 1D and 2D discrete wavelet\n\t\t transform Y Y
\n\t\t hoeleder Estimate of the Hoelder exponent for a given\n\t\t scaling function Y
\n\t\t makesig Generates the 'Donoho' test signals Y
\n\t \n\t

\n\n\t

\n\n\t\tThe lastest version of the Rice Wavelet Toolbox is\n\t\tavailable in Version\n\t\t2.3\n\n\t\t \n\n\t
\n\t\t \n \n \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/dist/2.3/INSTALL", "content": "Installation instructions\n-------------------------\n\nIn order to install this distribution of Rice Wavelet Tools version 2.3\nreleased - \n\n1. Properly set up your system to create MEX-files. Please refer to the\n \"Matlab Application Program Guide\" to properly set up of your matlab\n and C-compiler to be able to compile C-mex files on your system.\n All reference documentations are available on the MathWorks web page:\n www.mathworks.com\n\n2. Make a toolbox directory and uncompress/extract all the files.\n For example, in the unix environment,\n\n \tgunzip rwt.tar.gz\n \ttar xvf rwt.tar\n\n3. Run MATLAB and change to the temporary directory containing the files.\n\n4. Compile the toolbox by executing the Matlab command \n\n \tcompile\n\n5. Add the toolbox directory to your Matlab path.\n\n6. For further instructions, please refer to the README file.\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/dist/2.3/INSTALL_PRECOMPILED", "content": "Installation instructions\n-------------------------\n\nIn order to install this distribution of Rice Wavelet Tools version 2.3\nreleased - \n\n1. Make a toolbox directory and uncompress/extract all the files.\n For example, in the unix environment,\n\n \tgunzip rwt.tar.gz\n \ttar xvf rwt.tar\n\n2. Add the toolbox directory to your Matlab path.\n\n3. For further instructions, please refer to the README file.\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/dist/2.3/LICENSE", "content": "This \"rice-wlet-tools\", version 2.3\nReleased - \n\nCONDITIONS FOR USE:\nCopyright (c) 2000 RICE UNIVERSITY. All rights reserved.\n\nThis software is distributed and licensed to you on a non-exclusive \nbasis, free-of-charge. Redistribution and use in source and binary forms, \nwith or without modification, are permitted provided that the following \nconditions are met:\n\n1. Redistribution of source code must retain the above copyright notice, \n this list of conditions and the following disclaimer.\n2. Redistribution in binary form must reproduce the above copyright notice, \n this list of conditions and the following disclaimer in the \n documentation and/or other materials provided with the distribution.\n3. All advertising materials mentioning features or use of this software \n must display the following acknowledgment: This product includes \n software developed by Rice University, Houston, Texas and its contributors.\n4. Neither the name of the University nor the names of its contributors \n may be used to endorse or promote products derived from this software \n without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY WILLIAM MARSH RICE UNIVERSITY, HOUSTON, TEXAS, \nAND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, \nBUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS \nFOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RICE UNIVERSITY \nOR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, \nEXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, \nPROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; \nOR BUSINESS INTERRUPTIONS) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, \nWHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR \nOTHERWISE), PRODUCT LIABILITY, OR OTHERWISE ARISING IN ANY WAY OUT OF THE \nUSE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\nFor information on commercial licenses, contact Rice University's Office of \nTechnology Transfer at techtran@rice.edu or (713) 348-6173\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/dist/2.3/README", "content": "Rice Wavelet Tools version 2.3\nReleased - \n\nNEWER VERSION:\n A newer version of this toolbox is available at \n http://www.dsp.rice.edu/software/rwt.shtml\n\nINSTALLATION: \nTo install this distribution of Rice Wavelet Tools see the INSTALL file.\n\nSOURCE:\n www.dsp.rice.edu/software/rwt.shtml\n\nEMAIL: \nFor bug reports and questions, send email to webmaster-dsp@ece.rice.edu\n\nCONDITIONS FOR USE:\nSee the LICENSE file\n\nTOOLBOX FUNCTIONS:\n\n Wavelet Transforms\n mdwt - Discrete orthogonal wavelet transform using the Mallat algorithm (1D and 2D)\n midwt - Inverse discrete orthogonal wavelet transform\n mrdwt - Undecimated (redundant) discrete wavelet transform (1D and 2D)\n mirdwt - Inverse undecimated discrete wavelet transform\n daubcqf - Daubechies filter coefficients\n\n Wavelet Domain Processing\n denoise - Denoise signals and images by thresholding wavelet coefficients\n HardTh - Hard thresholding\n SoftTh - Soft thresholding\n\n Other\n makesig - Create Donoho-Johnstone test signals\n compile - Compile the Rice Wavelet Toolbox\n\nFunctions omitted in this version of toolbox can be found in \nversion 2.01 at www.dsp.rice.edu/software/RWT2.01/RWT-2.01.tar.Z\n\nThis version may not compile with Matlab 6.0 (Release 12) and above. \nThis problem has been fixed in version 2.4 at\nwww.dsp.rice.edu/software/rwt.shtml\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/doc/CMakeLists.txt", "content": "# add a target to generate API documentation with Doxygen\nfind_package(Doxygen)\nif(DOXYGEN_FOUND)\nconfigure_file(${CMAKE_CURRENT_SOURCE_DIR}/Doxyfile.in ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile @ONLY)\nadd_custom_target(doc\n${DOXYGEN_EXECUTABLE} ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile\nWORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}\nCOMMENT \"Generating API documentation with Doxygen\" VERBATIM\n)\n\nfind_package(LATEX)\nfind_program(DOXYFILE_MAKE make)\nmark_as_advanced(DOXYFILE_MAKE)\nadd_custom_command(TARGET doc\n POST_BUILD\n COMMAND \"${DOXYFILE_MAKE}\"\n COMMENT \"Running LaTeX for Doxygen documentation in ${CMAKE_CURRENT_SOURCE_DIR}/latex...\"\n WORKING_DIRECTORY \"${CMAKE_CURRENT_SOURCE_DIR}/latex\")\n\n\nendif(DOXYGEN_FOUND)\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/doc/Doxyfile.in", "content": "# Doxyfile 1.8.3.1\n\n# This file describes the settings to be used by the documentation system\n# doxygen (www.doxygen.org) for a project.\n#\n# All text after a hash (#) is considered a comment and will be ignored.\n# The format is:\n# TAG = value [value, ...]\n# For lists items can also be appended using:\n# TAG += value [value, ...]\n# Values that contain spaces should be placed between quotes (\" \").\n\n#---------------------------------------------------------------------------\n# Project related configuration options\n#---------------------------------------------------------------------------\n\n# This tag specifies the encoding used for all characters in the config file\n# that follow. The default is UTF-8 which is also the encoding used for all\n# text before the first occurrence of this tag. Doxygen uses libiconv (or the\n# iconv built into libc) for the transcoding. See\n# http://www.gnu.org/software/libiconv for the list of possible encodings.\n\nDOXYFILE_ENCODING = UTF-8\n\n# The PROJECT_NAME tag is a single word (or sequence of words) that should\n# identify the project. Note that if you do not use Doxywizard you need\n# to put quotes around the project name if it contains spaces.\n\nPROJECT_NAME = \"Rice Wavelet Toolbox\"\n\n# The PROJECT_NUMBER tag can be used to enter a project or revision number.\n# This could be handy for archiving the generated documentation or\n# if some version control system is used.\n\nPROJECT_NUMBER =\n\n# Using the PROJECT_BRIEF tag one can provide an optional one line description\n# for a project that appears at the top of each page and should give viewer\n# a quick idea about the purpose of the project. Keep the description short.\n\nPROJECT_BRIEF =\n\n# With the PROJECT_LOGO tag one can specify an logo or icon that is\n# included in the documentation. The maximum height of the logo should not\n# exceed 55 pixels and the maximum width should not exceed 200 pixels.\n# Doxygen will copy the logo to the output directory.\n\nPROJECT_LOGO =\n\n# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute)\n# base path where the generated documentation will be put.\n# If a relative path is entered, it will be relative to the location\n# where doxygen was started. If left blank the current directory will be used.\n\nOUTPUT_DIRECTORY =\n\n# If the CREATE_SUBDIRS tag is set to YES, then doxygen will create\n# 4096 sub-directories (in 2 levels) under the output directory of each output\n# format and will distribute the generated files over these directories.\n# Enabling this option can be useful when feeding doxygen a huge amount of\n# source files, where putting all generated files in the same directory would\n# otherwise cause performance problems for the file system.\n\nCREATE_SUBDIRS = NO\n\n# The OUTPUT_LANGUAGE tag is used to specify the language in which all\n# documentation generated by doxygen is written. Doxygen will use this\n# information to generate all constant output in the proper language.\n# The default language is English, other supported languages are:\n# Afrikaans, Arabic, Brazilian, Catalan, Chinese, Chinese-Traditional,\n# Croatian, Czech, Danish, Dutch, Esperanto, Farsi, Finnish, French, German,\n# Greek, Hungarian, Italian, Japanese, Japanese-en (Japanese with English\n# messages), Korean, Korean-en, Lithuanian, Norwegian, Macedonian, Persian,\n# Polish, Portuguese, Romanian, Russian, Serbian, Serbian-Cyrillic, Slovak,\n# Slovene, Spanish, Swedish, Ukrainian, and Vietnamese.\n\nOUTPUT_LANGUAGE = English\n\n# If the BRIEF_MEMBER_DESC tag is set to YES (the default) Doxygen will\n# include brief member descriptions after the members that are listed in\n# the file and class documentation (similar to JavaDoc).\n# Set to NO to disable this.\n\nBRIEF_MEMBER_DESC = YES\n\n# If the REPEAT_BRIEF tag is set to YES (the default) Doxygen will prepend\n# the brief description of a member or function before the detailed description.\n# Note: if both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the\n# brief descriptions will be completely suppressed.\n\nREPEAT_BRIEF = YES\n\n# This tag implements a quasi-intelligent brief description abbreviator\n# that is used to form the text in various listings. Each string\n# in this list, if found as the leading text of the brief description, will be\n# stripped from the text and the result after processing the whole list, is\n# used as the annotated text. Otherwise, the brief description is used as-is.\n# If left blank, the following values are used (\"$name\" is automatically\n# replaced with the name of the entity): \"The $name class\" \"The $name widget\"\n# \"The $name file\" \"is\" \"provides\" \"specifies\" \"contains\"\n# \"represents\" \"a\" \"an\" \"the\"\n\nABBREVIATE_BRIEF =\n\n# If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then\n# Doxygen will generate a detailed section even if there is only a brief\n# description.\n\nALWAYS_DETAILED_SEC = NO\n\n# If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all\n# inherited members of a class in the documentation of that class as if those\n# members were ordinary class members. Constructors, destructors and assignment\n# operators of the base classes will not be shown.\n\nINLINE_INHERITED_MEMB = NO\n\n# If the FULL_PATH_NAMES tag is set to YES then Doxygen will prepend the full\n# path before files name in the file list and in the header files. If set\n# to NO the shortest path that makes the file name unique will be used.\n\nFULL_PATH_NAMES = YES\n\n# If the FULL_PATH_NAMES tag is set to YES then the STRIP_FROM_PATH tag\n# can be used to strip a user-defined part of the path. Stripping is\n# only done if one of the specified strings matches the left-hand part of\n# the path. The tag can be used to show relative paths in the file list.\n# If left blank the directory from which doxygen is run is used as the\n# path to strip. Note that you specify absolute paths here, but also\n# relative paths, which will be relative from the directory where doxygen is\n# started.\n\nSTRIP_FROM_PATH = @CMAKE_CURRENT_SOURCE_DIR@/..\n\n# The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of\n# the path mentioned in the documentation of a class, which tells\n# the reader which header file to include in order to use a class.\n# If left blank only the name of the header file containing the class\n# definition is used. Otherwise one should specify the include paths that\n# are normally passed to the compiler using the -I flag.\n\nSTRIP_FROM_INC_PATH =\n\n# If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter\n# (but less readable) file names. This can be useful if your file system\n# doesn't support long names like on DOS, Mac, or CD-ROM.\n\nSHORT_NAMES = NO\n\n# If the JAVADOC_AUTOBRIEF tag is set to YES then Doxygen\n# will interpret the first line (until the first dot) of a JavaDoc-style\n# comment as the brief description. If set to NO, the JavaDoc\n# comments will behave just like regular Qt-style comments\n# (thus requiring an explicit @brief command for a brief description.)\n\nJAVADOC_AUTOBRIEF = NO\n\n# If the QT_AUTOBRIEF tag is set to YES then Doxygen will\n# interpret the first line (until the first dot) of a Qt-style\n# comment as the brief description. If set to NO, the comments\n# will behave just like regular Qt-style comments (thus requiring\n# an explicit \\brief command for a brief description.)\n\nQT_AUTOBRIEF = NO\n\n# The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make Doxygen\n# treat a multi-line C++ special comment block (i.e. a block of //! or ///\n# comments) as a brief description. This used to be the default behaviour.\n# The new default is to treat a multi-line C++ comment block as a detailed\n# description. Set this tag to YES if you prefer the old behaviour instead.\n\nMULTILINE_CPP_IS_BRIEF = NO\n\n# If the INHERIT_DOCS tag is set to YES (the default) then an undocumented\n# member inherits the documentation from any documented member that it\n# re-implements.\n\nINHERIT_DOCS = YES\n\n# If the SEPARATE_MEMBER_PAGES tag is set to YES, then doxygen will produce\n# a new page for each member. If set to NO, the documentation of a member will\n# be part of the file/class/namespace that contains it.\n\nSEPARATE_MEMBER_PAGES = NO\n\n# The TAB_SIZE tag can be used to set the number of spaces in a tab.\n# Doxygen uses this value to replace tabs by spaces in code fragments.\n\nTAB_SIZE = 4\n\n# This tag can be used to specify a number of aliases that acts\n# as commands in the documentation. An alias has the form \"name=value\".\n# For example adding \"sideeffect=\\par Side Effects:\\n\" will allow you to\n# put the command \\sideeffect (or @sideeffect) in the documentation, which\n# will result in a user-defined paragraph with heading \"Side Effects:\".\n# You can put \\n's in the value part of an alias to insert newlines.\n\nALIASES =\n\n# This tag can be used to specify a number of word-keyword mappings (TCL only).\n# A mapping has the form \"name=value\". For example adding\n# \"class=itcl::class\" will allow you to use the command class in the\n# itcl::class meaning.\n\nTCL_SUBST =\n\n# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C\n# sources only. Doxygen will then generate output that is more tailored for C.\n# For instance, some of the names that are used will be different. The list\n# of all members will be omitted, etc.\n\nOPTIMIZE_OUTPUT_FOR_C = YES\n\n# Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java\n# sources only. Doxygen will then generate output that is more tailored for\n# Java. For instance, namespaces will be presented as packages, qualified\n# scopes will look different, etc.\n\nOPTIMIZE_OUTPUT_JAVA = NO\n\n# Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran\n# sources only. Doxygen will then generate output that is more tailored for\n# Fortran.\n\nOPTIMIZE_FOR_FORTRAN = NO\n\n# Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL\n# sources. Doxygen will then generate output that is tailored for\n# VHDL.\n\nOPTIMIZE_OUTPUT_VHDL = NO\n\n# Doxygen selects the parser to use depending on the extension of the files it\n# parses. With this tag you can assign which parser to use for a given\n# extension. Doxygen has a built-in mapping, but you can override or extend it\n# using this tag. The format is ext=language, where ext is a file extension,\n# and language is one of the parsers supported by doxygen: IDL, Java,\n# Javascript, CSharp, C, C++, D, PHP, Objective-C, Python, Fortran, VHDL, C,\n# C++. For instance to make doxygen treat .inc files as Fortran files (default\n# is PHP), and .f files as C (default is Fortran), use: inc=Fortran f=C. Note\n# that for custom extensions you also need to set FILE_PATTERNS otherwise the\n# files are not read by doxygen.\n\nEXTENSION_MAPPING =\n\n# If MARKDOWN_SUPPORT is enabled (the default) then doxygen pre-processes all\n# comments according to the Markdown format, which allows for more readable\n# documentation. See http://daringfireball.net/projects/markdown/ for details.\n# The output of markdown processing is further processed by doxygen, so you\n# can mix doxygen, HTML, and XML commands with Markdown formatting.\n# Disable only in case of backward compatibilities issues.\n\nMARKDOWN_SUPPORT = YES\n\n# When enabled doxygen tries to link words that correspond to documented classes,\n# or namespaces to their corresponding documentation. Such a link can be\n# prevented in individual cases by by putting a % sign in front of the word or\n# globally by setting AUTOLINK_SUPPORT to NO.\n\nAUTOLINK_SUPPORT = YES\n\n# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want\n# to include (a tag file for) the STL sources as input, then you should\n# set this tag to YES in order to let doxygen match functions declarations and\n# definitions whose arguments contain STL classes (e.g. func(std::string); v.s.\n# func(std::string) {}). This also makes the inheritance and collaboration\n# diagrams that involve STL classes more complete and accurate.\n\nBUILTIN_STL_SUPPORT = NO\n\n# If you use Microsoft's C++/CLI language, you should set this option to YES to\n# enable parsing support.\n\nCPP_CLI_SUPPORT = NO\n\n# Set the SIP_SUPPORT tag to YES if your project consists of sip sources only.\n# Doxygen will parse them like normal C++ but will assume all classes use public\n# instead of private inheritance when no explicit protection keyword is present.\n\nSIP_SUPPORT = NO\n\n# For Microsoft's IDL there are propget and propput attributes to indicate\n# getter and setter methods for a property. Setting this option to YES (the\n# default) will make doxygen replace the get and set methods by a property in\n# the documentation. This will only work if the methods are indeed getting or\n# setting a simple type. If this is not the case, or you want to show the\n# methods anyway, you should set this option to NO.\n\nIDL_PROPERTY_SUPPORT = YES\n\n# If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC\n# tag is set to YES, then doxygen will reuse the documentation of the first\n# member in the group (if any) for the other members of the group. By default\n# all members of a group must be documented explicitly.\n\nDISTRIBUTE_GROUP_DOC = NO\n\n# Set the SUBGROUPING tag to YES (the default) to allow class member groups of\n# the same type (for instance a group of public functions) to be put as a\n# subgroup of that type (e.g. under the Public Functions section). Set it to\n# NO to prevent subgrouping. Alternatively, this can be done per class using\n# the \\nosubgrouping command.\n\nSUBGROUPING = YES\n\n# When the INLINE_GROUPED_CLASSES tag is set to YES, classes, structs and\n# unions are shown inside the group in which they are included (e.g. using\n# @ingroup) instead of on a separate page (for HTML and Man pages) or\n# section (for LaTeX and RTF).\n\nINLINE_GROUPED_CLASSES = NO\n\n# When the INLINE_SIMPLE_STRUCTS tag is set to YES, structs, classes, and\n# unions with only public data fields will be shown inline in the documentation\n# of the scope in which they are defined (i.e. file, namespace, or group\n# documentation), provided this scope is documented. If set to NO (the default),\n# structs, classes, and unions are shown on a separate page (for HTML and Man\n# pages) or section (for LaTeX and RTF).\n\nINLINE_SIMPLE_STRUCTS = NO\n\n# When TYPEDEF_HIDES_STRUCT is enabled, a typedef of a struct, union, or enum\n# is documented as struct, union, or enum with the name of the typedef. So\n# typedef struct TypeS {} TypeT, will appear in the documentation as a struct\n# with name TypeT. When disabled the typedef will appear as a member of a file,\n# namespace, or class. And the struct will be named TypeS. This can typically\n# be useful for C code in case the coding convention dictates that all compound\n# types are typedef'ed and only the typedef is referenced, never the tag name.\n\nTYPEDEF_HIDES_STRUCT = NO\n\n# The SYMBOL_CACHE_SIZE determines the size of the internal cache use to\n# determine which symbols to keep in memory and which to flush to disk.\n# When the cache is full, less often used symbols will be written to disk.\n# For small to medium size projects (<1000 input files) the default value is\n# probably good enough. For larger projects a too small cache size can cause\n# doxygen to be busy swapping symbols to and from disk most of the time\n# causing a significant performance penalty.\n# If the system has enough physical memory increasing the cache will improve the\n# performance by keeping more symbols in memory. Note that the value works on\n# a logarithmic scale so increasing the size by one will roughly double the\n# memory usage. The cache size is given by this formula:\n# 2^(16+SYMBOL_CACHE_SIZE). The valid range is 0..9, the default is 0,\n# corresponding to a cache size of 2^16 = 65536 symbols.\n\nSYMBOL_CACHE_SIZE = 0\n\n# Similar to the SYMBOL_CACHE_SIZE the size of the symbol lookup cache can be\n# set using LOOKUP_CACHE_SIZE. This cache is used to resolve symbols given\n# their name and scope. Since this can be an expensive process and often the\n# same symbol appear multiple times in the code, doxygen keeps a cache of\n# pre-resolved symbols. If the cache is too small doxygen will become slower.\n# If the cache is too large, memory is wasted. The cache size is given by this\n# formula: 2^(16+LOOKUP_CACHE_SIZE). The valid range is 0..9, the default is 0,\n# corresponding to a cache size of 2^16 = 65536 symbols.\n\nLOOKUP_CACHE_SIZE = 0\n\n#---------------------------------------------------------------------------\n# Build related configuration options\n#---------------------------------------------------------------------------\n\n# If the EXTRACT_ALL tag is set to YES doxygen will assume all entities in\n# documentation are documented, even if no documentation was available.\n# Private class members and static file members will be hidden unless\n# the EXTRACT_PRIVATE and EXTRACT_STATIC tags are set to YES\n\nEXTRACT_ALL = YES\n\n# If the EXTRACT_PRIVATE tag is set to YES all private members of a class\n# will be included in the documentation.\n\nEXTRACT_PRIVATE = NO\n\n# If the EXTRACT_PACKAGE tag is set to YES all members with package or internal\n# scope will be included in the documentation.\n\nEXTRACT_PACKAGE = NO\n\n# If the EXTRACT_STATIC tag is set to YES all static members of a file\n# will be included in the documentation.\n\nEXTRACT_STATIC = NO\n\n# If the EXTRACT_LOCAL_CLASSES tag is set to YES classes (and structs)\n# defined locally in source files will be included in the documentation.\n# If set to NO only classes defined in header files are included.\n\nEXTRACT_LOCAL_CLASSES = YES\n\n# This flag is only useful for Objective-C code. When set to YES local\n# methods, which are defined in the implementation section but not in\n# the interface are included in the documentation.\n# If set to NO (the default) only methods in the interface are included.\n\nEXTRACT_LOCAL_METHODS = NO\n\n# If this flag is set to YES, the members of anonymous namespaces will be\n# extracted and appear in the documentation as a namespace called\n# 'anonymous_namespace{file}', where file will be replaced with the base\n# name of the file that contains the anonymous namespace. By default\n# anonymous namespaces are hidden.\n\nEXTRACT_ANON_NSPACES = NO\n\n# If the HIDE_UNDOC_MEMBERS tag is set to YES, Doxygen will hide all\n# undocumented members of documented classes, files or namespaces.\n# If set to NO (the default) these members will be included in the\n# various overviews, but no documentation section is generated.\n# This option has no effect if EXTRACT_ALL is enabled.\n\nHIDE_UNDOC_MEMBERS = NO\n\n# If the HIDE_UNDOC_CLASSES tag is set to YES, Doxygen will hide all\n# undocumented classes that are normally visible in the class hierarchy.\n# If set to NO (the default) these classes will be included in the various\n# overviews. This option has no effect if EXTRACT_ALL is enabled.\n\nHIDE_UNDOC_CLASSES = NO\n\n# If the HIDE_FRIEND_COMPOUNDS tag is set to YES, Doxygen will hide all\n# friend (class|struct|union) declarations.\n# If set to NO (the default) these declarations will be included in the\n# documentation.\n\nHIDE_FRIEND_COMPOUNDS = NO\n\n# If the HIDE_IN_BODY_DOCS tag is set to YES, Doxygen will hide any\n# documentation blocks found inside the body of a function.\n# If set to NO (the default) these blocks will be appended to the\n# function's detailed documentation block.\n\nHIDE_IN_BODY_DOCS = NO\n\n# The INTERNAL_DOCS tag determines if documentation\n# that is typed after a \\internal command is included. If the tag is set\n# to NO (the default) then the documentation will be excluded.\n# Set it to YES to include the internal documentation.\n\nINTERNAL_DOCS = NO\n\n# If the CASE_SENSE_NAMES tag is set to NO then Doxygen will only generate\n# file names in lower-case letters. If set to YES upper-case letters are also\n# allowed. This is useful if you have classes or files whose names only differ\n# in case and if your file system supports case sensitive file names. Windows\n# and Mac users are advised to set this option to NO.\n\nCASE_SENSE_NAMES = NO\n\n# If the HIDE_SCOPE_NAMES tag is set to NO (the default) then Doxygen\n# will show members with their full class and namespace scopes in the\n# documentation. If set to YES the scope will be hidden.\n\nHIDE_SCOPE_NAMES = NO\n\n# If the SHOW_INCLUDE_FILES tag is set to YES (the default) then Doxygen\n# will put a list of the files that are included by a file in the documentation\n# of that file.\n\nSHOW_INCLUDE_FILES = YES\n\n# If the FORCE_LOCAL_INCLUDES tag is set to YES then Doxygen\n# will list include files with double quotes in the documentation\n# rather than with sharp brackets.\n\nFORCE_LOCAL_INCLUDES = NO\n\n# If the INLINE_INFO tag is set to YES (the default) then a tag [inline]\n# is inserted in the documentation for inline members.\n\nINLINE_INFO = YES\n\n# If the SORT_MEMBER_DOCS tag is set to YES (the default) then doxygen\n# will sort the (detailed) documentation of file and class members\n# alphabetically by member name. If set to NO the members will appear in\n# declaration order.\n\nSORT_MEMBER_DOCS = YES\n\n# If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the\n# brief documentation of file, namespace and class members alphabetically\n# by member name. If set to NO (the default) the members will appear in\n# declaration order.\n\nSORT_BRIEF_DOCS = NO\n\n# If the SORT_MEMBERS_CTORS_1ST tag is set to YES then doxygen\n# will sort the (brief and detailed) documentation of class members so that\n# constructors and destructors are listed first. If set to NO (the default)\n# the constructors will appear in the respective orders defined by\n# SORT_MEMBER_DOCS and SORT_BRIEF_DOCS.\n# This tag will be ignored for brief docs if SORT_BRIEF_DOCS is set to NO\n# and ignored for detailed docs if SORT_MEMBER_DOCS is set to NO.\n\nSORT_MEMBERS_CTORS_1ST = NO\n\n# If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the\n# hierarchy of group names into alphabetical order. If set to NO (the default)\n# the group names will appear in their defined order.\n\nSORT_GROUP_NAMES = NO\n\n# If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be\n# sorted by fully-qualified names, including namespaces. If set to\n# NO (the default), the class list will be sorted only by class name,\n# not including the namespace part.\n# Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES.\n# Note: This option applies only to the class list, not to the\n# alphabetical list.\n\nSORT_BY_SCOPE_NAME = NO\n\n# If the STRICT_PROTO_MATCHING option is enabled and doxygen fails to\n# do proper type resolution of all parameters of a function it will reject a\n# match between the prototype and the implementation of a member function even\n# if there is only one candidate or it is obvious which candidate to choose\n# by doing a simple string match. By disabling STRICT_PROTO_MATCHING doxygen\n# will still accept a match between prototype and implementation in such cases.\n\nSTRICT_PROTO_MATCHING = NO\n\n# The GENERATE_TODOLIST tag can be used to enable (YES) or\n# disable (NO) the todo list. This list is created by putting \\todo\n# commands in the documentation.\n\nGENERATE_TODOLIST = YES\n\n# The GENERATE_TESTLIST tag can be used to enable (YES) or\n# disable (NO) the test list. This list is created by putting \\test\n# commands in the documentation.\n\nGENERATE_TESTLIST = YES\n\n# The GENERATE_BUGLIST tag can be used to enable (YES) or\n# disable (NO) the bug list. This list is created by putting \\bug\n# commands in the documentation.\n\nGENERATE_BUGLIST = YES\n\n# The GENERATE_DEPRECATEDLIST tag can be used to enable (YES) or\n# disable (NO) the deprecated list. This list is created by putting\n# \\deprecated commands in the documentation.\n\nGENERATE_DEPRECATEDLIST= YES\n\n# The ENABLED_SECTIONS tag can be used to enable conditional\n# documentation sections, marked by \\if section-label ... \\endif\n# and \\cond section-label ... \\endcond blocks.\n\nENABLED_SECTIONS =\n\n# The MAX_INITIALIZER_LINES tag determines the maximum number of lines\n# the initial value of a variable or macro consists of for it to appear in\n# the documentation. If the initializer consists of more lines than specified\n# here it will be hidden. Use a value of 0 to hide initializers completely.\n# The appearance of the initializer of individual variables and macros in the\n# documentation can be controlled using \\showinitializer or \\hideinitializer\n# command in the documentation regardless of this setting.\n\nMAX_INITIALIZER_LINES = 30\n\n# Set the SHOW_USED_FILES tag to NO to disable the list of files generated\n# at the bottom of the documentation of classes and structs. If set to YES the\n# list will mention the files that were used to generate the documentation.\n\nSHOW_USED_FILES = YES\n\n# Set the SHOW_FILES tag to NO to disable the generation of the Files page.\n# This will remove the Files entry from the Quick Index and from the\n# Folder Tree View (if specified). The default is YES.\n\nSHOW_FILES = YES\n\n# Set the SHOW_NAMESPACES tag to NO to disable the generation of the\n# Namespaces page.\n# This will remove the Namespaces entry from the Quick Index\n# and from the Folder Tree View (if specified). The default is YES.\n\nSHOW_NAMESPACES = YES\n\n# The FILE_VERSION_FILTER tag can be used to specify a program or script that\n# doxygen should invoke to get the current version for each file (typically from\n# the version control system). Doxygen will invoke the program by executing (via\n# popen()) the command , where is the value of\n# the FILE_VERSION_FILTER tag, and is the name of an input file\n# provided by doxygen. Whatever the program writes to standard output\n# is used as the file version. See the manual for examples.\n\nFILE_VERSION_FILTER =\n\n# The LAYOUT_FILE tag can be used to specify a layout file which will be parsed\n# by doxygen. The layout file controls the global structure of the generated\n# output files in an output format independent way. To create the layout file\n# that represents doxygen's defaults, run doxygen with the -l option.\n# You can optionally specify a file name after the option, if omitted\n# DoxygenLayout.xml will be used as the name of the layout file.\n\nLAYOUT_FILE =\n\n# The CITE_BIB_FILES tag can be used to specify one or more bib files\n# containing the references data. This must be a list of .bib files. The\n# .bib extension is automatically appended if omitted. Using this command\n# requires the bibtex tool to be installed. See also\n# http://en.wikipedia.org/wiki/BibTeX for more info. For LaTeX the style\n# of the bibliography can be controlled using LATEX_BIB_STYLE. To use this\n# feature you need bibtex and perl available in the search path. Do not use\n# file names with spaces, bibtex cannot handle them.\n\nCITE_BIB_FILES =\n\n#---------------------------------------------------------------------------\n# configuration options related to warning and progress messages\n#---------------------------------------------------------------------------\n\n# The QUIET tag can be used to turn on/off the messages that are generated\n# by doxygen. Possible values are YES and NO. If left blank NO is used.\n\nQUIET = NO\n\n# The WARNINGS tag can be used to turn on/off the warning messages that are\n# generated by doxygen. Possible values are YES and NO. If left blank\n# NO is used.\n\nWARNINGS = YES\n\n# If WARN_IF_UNDOCUMENTED is set to YES, then doxygen will generate warnings\n# for undocumented members. If EXTRACT_ALL is set to YES then this flag will\n# automatically be disabled.\n\nWARN_IF_UNDOCUMENTED = YES\n\n# If WARN_IF_DOC_ERROR is set to YES, doxygen will generate warnings for\n# potential errors in the documentation, such as not documenting some\n# parameters in a documented function, or documenting parameters that\n# don't exist or using markup commands wrongly.\n\nWARN_IF_DOC_ERROR = YES\n\n# The WARN_NO_PARAMDOC option can be enabled to get warnings for\n# functions that are documented, but have no documentation for their parameters\n# or return value. If set to NO (the default) doxygen will only warn about\n# wrong or incomplete parameter documentation, but not about the absence of\n# documentation.\n\nWARN_NO_PARAMDOC = NO\n\n# The WARN_FORMAT tag determines the format of the warning messages that\n# doxygen can produce. The string should contain the $file, $line, and $text\n# tags, which will be replaced by the file and line number from which the\n# warning originated and the warning text. Optionally the format may contain\n# $version, which will be replaced by the version of the file (if it could\n# be obtained via FILE_VERSION_FILTER)\n\nWARN_FORMAT = \"$file:$line: $text\"\n\n# The WARN_LOGFILE tag can be used to specify a file to which warning\n# and error messages should be written. If left blank the output is written\n# to stderr.\n\nWARN_LOGFILE =\n\n#---------------------------------------------------------------------------\n# configuration options related to the input files\n#---------------------------------------------------------------------------\n\n# The INPUT tag can be used to specify the files and/or directories that contain\n# documented source files. You may enter file names like \"myfile.cpp\" or\n# directories like \"/usr/src/myproject\". Separate the files or directories\n# with spaces.\n\nINPUT = @CMAKE_CURRENT_SOURCE_DIR@/../src @CMAKE_CURRENT_SOURCE_DIR@/../lib/src @CMAKE_CURRENT_SOURCE_DIR@/../lib/inc @CMAKE_CURRENT_SOURCE_DIR@/../mex\n\n\n# This tag can be used to specify the character encoding of the source files\n# that doxygen parses. Internally doxygen uses the UTF-8 encoding, which is\n# also the default input encoding. Doxygen uses libiconv (or the iconv built\n# into libc) for the transcoding. See http://www.gnu.org/software/libiconv for\n# the list of possible encodings.\n\nINPUT_ENCODING = UTF-8\n\n# If the value of the INPUT tag contains directories, you can use the\n# FILE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp\n# and *.h) to filter out the source-files in the directories. If left\n# blank the following patterns are tested:\n# *.c *.cc *.cxx *.cpp *.c++ *.d *.java *.ii *.ixx *.ipp *.i++ *.inl *.h *.hh\n# *.hxx *.hpp *.h++ *.idl *.odl *.cs *.php *.php3 *.inc *.m *.mm *.dox *.py\n# *.f90 *.f *.for *.vhd *.vhdl\n\nFILE_PATTERNS =\n\n# The RECURSIVE tag can be used to turn specify whether or not subdirectories\n# should be searched for input files as well. Possible values are YES and NO.\n# If left blank NO is used.\n\nRECURSIVE = NO\n\n# The EXCLUDE tag can be used to specify files and/or directories that should be\n# excluded from the INPUT source files. This way you can easily exclude a\n# subdirectory from a directory tree whose root is specified with the INPUT tag.\n# Note that relative paths are relative to the directory from which doxygen is\n# run.\n\nEXCLUDE =\n\n# The EXCLUDE_SYMLINKS tag can be used to select whether or not files or\n# directories that are symbolic links (a Unix file system feature) are excluded\n# from the input.\n\nEXCLUDE_SYMLINKS = NO\n\n# If the value of the INPUT tag contains directories, you can use the\n# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude\n# certain files from those directories. Note that the wildcards are matched\n# against the file with absolute path, so to exclude all test directories\n# for example use the pattern */test/*\n\nEXCLUDE_PATTERNS =\n\n# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names\n# (namespaces, classes, functions, etc.) that should be excluded from the\n# output. The symbol name can be a fully qualified name, a word, or if the\n# wildcard * is used, a substring. Examples: ANamespace, AClass,\n# AClass::ANamespace, ANamespace::*Test\n\nEXCLUDE_SYMBOLS =\n\n# The EXAMPLE_PATH tag can be used to specify one or more files or\n# directories that contain example code fragments that are included (see\n# the \\include command).\n\nEXAMPLE_PATH =\n\n# If the value of the EXAMPLE_PATH tag contains directories, you can use the\n# EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp\n# and *.h) to filter out the source-files in the directories. If left\n# blank all files are included.\n\nEXAMPLE_PATTERNS =\n\n# If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be\n# searched for input files to be used with the \\include or \\dontinclude\n# commands irrespective of the value of the RECURSIVE tag.\n# Possible values are YES and NO. If left blank NO is used.\n\nEXAMPLE_RECURSIVE = NO\n\n# The IMAGE_PATH tag can be used to specify one or more files or\n# directories that contain image that are included in the documentation (see\n# the \\image command).\n\nIMAGE_PATH =\n\n# The INPUT_FILTER tag can be used to specify a program that doxygen should\n# invoke to filter for each input file. Doxygen will invoke the filter program\n# by executing (via popen()) the command , where \n# is the value of the INPUT_FILTER tag, and is the name of an\n# input file. Doxygen will then use the output that the filter program writes\n# to standard output.\n# If FILTER_PATTERNS is specified, this tag will be\n# ignored.\n\nINPUT_FILTER =\n\n# The FILTER_PATTERNS tag can be used to specify filters on a per file pattern\n# basis.\n# Doxygen will compare the file name with each pattern and apply the\n# filter if there is a match.\n# The filters are a list of the form:\n# pattern=filter (like *.cpp=my_cpp_filter). See INPUT_FILTER for further\n# info on how filters are used. If FILTER_PATTERNS is empty or if\n# non of the patterns match the file name, INPUT_FILTER is applied.\n\nFILTER_PATTERNS =\n\n# If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using\n# INPUT_FILTER) will be used to filter the input files when producing source\n# files to browse (i.e. when SOURCE_BROWSER is set to YES).\n\nFILTER_SOURCE_FILES = NO\n\n# The FILTER_SOURCE_PATTERNS tag can be used to specify source filters per file\n# pattern. A pattern will override the setting for FILTER_PATTERN (if any)\n# and it is also possible to disable source filtering for a specific pattern\n# using *.ext= (so without naming a filter). This option only has effect when\n# FILTER_SOURCE_FILES is enabled.\n\nFILTER_SOURCE_PATTERNS =\n\n# If the USE_MD_FILE_AS_MAINPAGE tag refers to the name of a markdown file that\n# is part of the input, its contents will be placed on the main page (index.html).\n# This can be useful if you have a project on for instance GitHub and want reuse\n# the introduction page also for the doxygen output.\n\nUSE_MDFILE_AS_MAINPAGE =\n\n#---------------------------------------------------------------------------\n# configuration options related to source browsing\n#---------------------------------------------------------------------------\n\n# If the SOURCE_BROWSER tag is set to YES then a list of source files will\n# be generated. Documented entities will be cross-referenced with these sources.\n# Note: To get rid of all source code in the generated output, make sure also\n# VERBATIM_HEADERS is set to NO.\n\nSOURCE_BROWSER = YES\n\n# Setting the INLINE_SOURCES tag to YES will include the body\n# of functions and classes directly in the documentation.\n\nINLINE_SOURCES = YES\n\n# Setting the STRIP_CODE_COMMENTS tag to YES (the default) will instruct\n# doxygen to hide any special comment blocks from generated source code\n# fragments. Normal C, C++ and Fortran comments will always remain visible.\n\nSTRIP_CODE_COMMENTS = YES\n\n# If the REFERENCED_BY_RELATION tag is set to YES\n# then for each documented function all documented\n# functions referencing it will be listed.\n\nREFERENCED_BY_RELATION = NO\n\n# If the REFERENCES_RELATION tag is set to YES\n# then for each documented function all documented entities\n# called/used by that function will be listed.\n\nREFERENCES_RELATION = NO\n\n# If the REFERENCES_LINK_SOURCE tag is set to YES (the default)\n# and SOURCE_BROWSER tag is set to YES, then the hyperlinks from\n# functions in REFERENCES_RELATION and REFERENCED_BY_RELATION lists will\n# link to the source code.\n# Otherwise they will link to the documentation.\n\nREFERENCES_LINK_SOURCE = YES\n\n# If the USE_HTAGS tag is set to YES then the references to source code\n# will point to the HTML generated by the htags(1) tool instead of doxygen\n# built-in source browser. The htags tool is part of GNU's global source\n# tagging system (see http://www.gnu.org/software/global/global.html). You\n# will need version 4.8.6 or higher.\n\nUSE_HTAGS = NO\n\n# If the VERBATIM_HEADERS tag is set to YES (the default) then Doxygen\n# will generate a verbatim copy of the header file for each class for\n# which an include is specified. Set to NO to disable this.\n\nVERBATIM_HEADERS = YES\n\n#---------------------------------------------------------------------------\n# configuration options related to the alphabetical class index\n#---------------------------------------------------------------------------\n\n# If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index\n# of all compounds will be generated. Enable this if the project\n# contains a lot of classes, structs, unions or interfaces.\n\nALPHABETICAL_INDEX = YES\n\n# If the alphabetical index is enabled (see ALPHABETICAL_INDEX) then\n# the COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns\n# in which this list will be split (can be a number in the range [1..20])\n\nCOLS_IN_ALPHA_INDEX = 5\n\n# In case all classes in a project start with a common prefix, all\n# classes will be put under the same header in the alphabetical index.\n# The IGNORE_PREFIX tag can be used to specify one or more prefixes that\n# should be ignored while generating the index headers.\n\nIGNORE_PREFIX =\n\n#---------------------------------------------------------------------------\n# configuration options related to the HTML output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_HTML tag is set to YES (the default) Doxygen will\n# generate HTML output.\n\nGENERATE_HTML = YES\n\n# The HTML_OUTPUT tag is used to specify where the HTML docs will be put.\n# If a relative path is entered the value of OUTPUT_DIRECTORY will be\n# put in front of it. If left blank `html' will be used as the default path.\n\nHTML_OUTPUT = html\n\n# The HTML_FILE_EXTENSION tag can be used to specify the file extension for\n# each generated HTML page (for example: .htm,.php,.asp). If it is left blank\n# doxygen will generate files with .html extension.\n\nHTML_FILE_EXTENSION = .html\n\n# The HTML_HEADER tag can be used to specify a personal HTML header for\n# each generated HTML page. If it is left blank doxygen will generate a\n# standard header. Note that when using a custom header you are responsible\n# for the proper inclusion of any scripts and style sheets that doxygen\n# needs, which is dependent on the configuration options used.\n# It is advised to generate a default header using \"doxygen -w html\n# header.html footer.html stylesheet.css YourConfigFile\" and then modify\n# that header. Note that the header is subject to change so you typically\n# have to redo this when upgrading to a newer version of doxygen or when\n# changing the value of configuration settings such as GENERATE_TREEVIEW!\n\nHTML_HEADER =\n\n# The HTML_FOOTER tag can be used to specify a personal HTML footer for\n# each generated HTML page. If it is left blank doxygen will generate a\n# standard footer.\n\nHTML_FOOTER =\n\n# The HTML_STYLESHEET tag can be used to specify a user-defined cascading\n# style sheet that is used by each HTML page. It can be used to\n# fine-tune the look of the HTML output. If left blank doxygen will\n# generate a default style sheet. Note that it is recommended to use\n# HTML_EXTRA_STYLESHEET instead of this one, as it is more robust and this\n# tag will in the future become obsolete.\n\nHTML_STYLESHEET =\n\n# The HTML_EXTRA_STYLESHEET tag can be used to specify an additional\n# user-defined cascading style sheet that is included after the standard\n# style sheets created by doxygen. Using this option one can overrule\n# certain style aspects. This is preferred over using HTML_STYLESHEET\n# since it does not replace the standard style sheet and is therefor more\n# robust against future updates. Doxygen will copy the style sheet file to\n# the output directory.\n\nHTML_EXTRA_STYLESHEET =\n\n# The HTML_EXTRA_FILES tag can be used to specify one or more extra images or\n# other source files which should be copied to the HTML output directory. Note\n# that these files will be copied to the base HTML output directory. Use the\n# $relpath$ marker in the HTML_HEADER and/or HTML_FOOTER files to load these\n# files. In the HTML_STYLESHEET file, use the file name only. Also note that\n# the files will be copied as-is; there are no commands or markers available.\n\nHTML_EXTRA_FILES =\n\n# The HTML_COLORSTYLE_HUE tag controls the color of the HTML output.\n# Doxygen will adjust the colors in the style sheet and background images\n# according to this color. Hue is specified as an angle on a colorwheel,\n# see http://en.wikipedia.org/wiki/Hue for more information.\n# For instance the value 0 represents red, 60 is yellow, 120 is green,\n# 180 is cyan, 240 is blue, 300 purple, and 360 is red again.\n# The allowed range is 0 to 359.\n\nHTML_COLORSTYLE_HUE = 220\n\n# The HTML_COLORSTYLE_SAT tag controls the purity (or saturation) of\n# the colors in the HTML output. For a value of 0 the output will use\n# grayscales only. A value of 255 will produce the most vivid colors.\n\nHTML_COLORSTYLE_SAT = 100\n\n# The HTML_COLORSTYLE_GAMMA tag controls the gamma correction applied to\n# the luminance component of the colors in the HTML output. Values below\n# 100 gradually make the output lighter, whereas values above 100 make\n# the output darker. The value divided by 100 is the actual gamma applied,\n# so 80 represents a gamma of 0.8, The value 220 represents a gamma of 2.2,\n# and 100 does not change the gamma.\n\nHTML_COLORSTYLE_GAMMA = 80\n\n# If the HTML_TIMESTAMP tag is set to YES then the footer of each generated HTML\n# page will contain the date and time when the page was generated. Setting\n# this to NO can help when comparing the output of multiple runs.\n\nHTML_TIMESTAMP = YES\n\n# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML\n# documentation will contain sections that can be hidden and shown after the\n# page has loaded.\n\nHTML_DYNAMIC_SECTIONS = NO\n\n# With HTML_INDEX_NUM_ENTRIES one can control the preferred number of\n# entries shown in the various tree structured indices initially; the user\n# can expand and collapse entries dynamically later on. Doxygen will expand\n# the tree to such a level that at most the specified number of entries are\n# visible (unless a fully collapsed tree already exceeds this amount).\n# So setting the number of entries 1 will produce a full collapsed tree by\n# default. 0 is a special value representing an infinite number of entries\n# and will result in a full expanded tree by default.\n\nHTML_INDEX_NUM_ENTRIES = 100\n\n# If the GENERATE_DOCSET tag is set to YES, additional index files\n# will be generated that can be used as input for Apple's Xcode 3\n# integrated development environment, introduced with OSX 10.5 (Leopard).\n# To create a documentation set, doxygen will generate a Makefile in the\n# HTML output directory. Running make will produce the docset in that\n# directory and running \"make install\" will install the docset in\n# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find\n# it at startup.\n# See http://developer.apple.com/tools/creatingdocsetswithdoxygen.html\n# for more information.\n\nGENERATE_DOCSET = NO\n\n# When GENERATE_DOCSET tag is set to YES, this tag determines the name of the\n# feed. A documentation feed provides an umbrella under which multiple\n# documentation sets from a single provider (such as a company or product suite)\n# can be grouped.\n\nDOCSET_FEEDNAME = \"Doxygen generated docs\"\n\n# When GENERATE_DOCSET tag is set to YES, this tag specifies a string that\n# should uniquely identify the documentation set bundle. This should be a\n# reverse domain-name style string, e.g. com.mycompany.MyDocSet. Doxygen\n# will append .docset to the name.\n\nDOCSET_BUNDLE_ID = org.doxygen.Project\n\n# When GENERATE_PUBLISHER_ID tag specifies a string that should uniquely\n# identify the documentation publisher. This should be a reverse domain-name\n# style string, e.g. com.mycompany.MyDocSet.documentation.\n\nDOCSET_PUBLISHER_ID = org.doxygen.Publisher\n\n# The GENERATE_PUBLISHER_NAME tag identifies the documentation publisher.\n\nDOCSET_PUBLISHER_NAME = Publisher\n\n# If the GENERATE_HTMLHELP tag is set to YES, additional index files\n# will be generated that can be used as input for tools like the\n# Microsoft HTML help workshop to generate a compiled HTML help file (.chm)\n# of the generated HTML documentation.\n\nGENERATE_HTMLHELP = NO\n\n# If the GENERATE_HTMLHELP tag is set to YES, the CHM_FILE tag can\n# be used to specify the file name of the resulting .chm file. You\n# can add a path in front of the file if the result should not be\n# written to the html output directory.\n\nCHM_FILE =\n\n# If the GENERATE_HTMLHELP tag is set to YES, the HHC_LOCATION tag can\n# be used to specify the location (absolute path including file name) of\n# the HTML help compiler (hhc.exe). If non-empty doxygen will try to run\n# the HTML help compiler on the generated index.hhp.\n\nHHC_LOCATION =\n\n# If the GENERATE_HTMLHELP tag is set to YES, the GENERATE_CHI flag\n# controls if a separate .chi index file is generated (YES) or that\n# it should be included in the master .chm file (NO).\n\nGENERATE_CHI = NO\n\n# If the GENERATE_HTMLHELP tag is set to YES, the CHM_INDEX_ENCODING\n# is used to encode HtmlHelp index (hhk), content (hhc) and project file\n# content.\n\nCHM_INDEX_ENCODING =\n\n# If the GENERATE_HTMLHELP tag is set to YES, the BINARY_TOC flag\n# controls whether a binary table of contents is generated (YES) or a\n# normal table of contents (NO) in the .chm file.\n\nBINARY_TOC = NO\n\n# The TOC_EXPAND flag can be set to YES to add extra items for group members\n# to the contents of the HTML help documentation and to the tree view.\n\nTOC_EXPAND = NO\n\n# If the GENERATE_QHP tag is set to YES and both QHP_NAMESPACE and\n# QHP_VIRTUAL_FOLDER are set, an additional index file will be generated\n# that can be used as input for Qt's qhelpgenerator to generate a\n# Qt Compressed Help (.qch) of the generated HTML documentation.\n\nGENERATE_QHP = NO\n\n# If the QHG_LOCATION tag is specified, the QCH_FILE tag can\n# be used to specify the file name of the resulting .qch file.\n# The path specified is relative to the HTML output folder.\n\nQCH_FILE =\n\n# The QHP_NAMESPACE tag specifies the namespace to use when generating\n# Qt Help Project output. For more information please see\n# http://doc.trolltech.com/qthelpproject.html#namespace\n\nQHP_NAMESPACE = org.doxygen.Project\n\n# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating\n# Qt Help Project output. For more information please see\n# http://doc.trolltech.com/qthelpproject.html#virtual-folders\n\nQHP_VIRTUAL_FOLDER = doc\n\n# If QHP_CUST_FILTER_NAME is set, it specifies the name of a custom filter to\n# add. For more information please see\n# http://doc.trolltech.com/qthelpproject.html#custom-filters\n\nQHP_CUST_FILTER_NAME =\n\n# The QHP_CUST_FILT_ATTRS tag specifies the list of the attributes of the\n# custom filter to add. For more information please see\n# \n# Qt Help Project / Custom Filters.\n\nQHP_CUST_FILTER_ATTRS =\n\n# The QHP_SECT_FILTER_ATTRS tag specifies the list of the attributes this\n# project's\n# filter section matches.\n# \n# Qt Help Project / Filter Attributes.\n\nQHP_SECT_FILTER_ATTRS =\n\n# If the GENERATE_QHP tag is set to YES, the QHG_LOCATION tag can\n# be used to specify the location of Qt's qhelpgenerator.\n# If non-empty doxygen will try to run qhelpgenerator on the generated\n# .qhp file.\n\nQHG_LOCATION =\n\n# If the GENERATE_ECLIPSEHELP tag is set to YES, additional index files\n# will be generated, which together with the HTML files, form an Eclipse help\n# plugin. To install this plugin and make it available under the help contents\n# menu in Eclipse, the contents of the directory containing the HTML and XML\n# files needs to be copied into the plugins directory of eclipse. The name of\n# the directory within the plugins directory should be the same as\n# the ECLIPSE_DOC_ID value. After copying Eclipse needs to be restarted before\n# the help appears.\n\nGENERATE_ECLIPSEHELP = NO\n\n# A unique identifier for the eclipse help plugin. When installing the plugin\n# the directory name containing the HTML and XML files should also have\n# this name.\n\nECLIPSE_DOC_ID = org.doxygen.Project\n\n# The DISABLE_INDEX tag can be used to turn on/off the condensed index (tabs)\n# at top of each HTML page. The value NO (the default) enables the index and\n# the value YES disables it. Since the tabs have the same information as the\n# navigation tree you can set this option to NO if you already set\n# GENERATE_TREEVIEW to YES.\n\nDISABLE_INDEX = NO\n\n# The GENERATE_TREEVIEW tag is used to specify whether a tree-like index\n# structure should be generated to display hierarchical information.\n# If the tag value is set to YES, a side panel will be generated\n# containing a tree-like index structure (just like the one that\n# is generated for HTML Help). For this to work a browser that supports\n# JavaScript, DHTML, CSS and frames is required (i.e. any modern browser).\n# Windows users are probably better off using the HTML help feature.\n# Since the tree basically has the same information as the tab index you\n# could consider to set DISABLE_INDEX to NO when enabling this option.\n\nGENERATE_TREEVIEW = NO\n\n# The ENUM_VALUES_PER_LINE tag can be used to set the number of enum values\n# (range [0,1..20]) that doxygen will group on one line in the generated HTML\n# documentation. Note that a value of 0 will completely suppress the enum\n# values from appearing in the overview section.\n\nENUM_VALUES_PER_LINE = 4\n\n# If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be\n# used to set the initial width (in pixels) of the frame in which the tree\n# is shown.\n\nTREEVIEW_WIDTH = 250\n\n# When the EXT_LINKS_IN_WINDOW option is set to YES doxygen will open\n# links to external symbols imported via tag files in a separate window.\n\nEXT_LINKS_IN_WINDOW = NO\n\n# Use this tag to change the font size of Latex formulas included\n# as images in the HTML documentation. The default is 10. Note that\n# when you change the font size after a successful doxygen run you need\n# to manually remove any form_*.png images from the HTML output directory\n# to force them to be regenerated.\n\nFORMULA_FONTSIZE = 14\n\n# Use the FORMULA_TRANPARENT tag to determine whether or not the images\n# generated for formulas are transparent PNGs. Transparent PNGs are\n# not supported properly for IE 6.0, but are supported on all modern browsers.\n# Note that when changing this option you need to delete any form_*.png files\n# in the HTML output before the changes have effect.\n\nFORMULA_TRANSPARENT = YES\n\n# Enable the USE_MATHJAX option to render LaTeX formulas using MathJax\n# (see http://www.mathjax.org) which uses client side Javascript for the\n# rendering instead of using prerendered bitmaps. Use this if you do not\n# have LaTeX installed or if you want to formulas look prettier in the HTML\n# output. When enabled you may also need to install MathJax separately and\n# configure the path to it using the MATHJAX_RELPATH option.\n\nUSE_MATHJAX = NO\n\n# When MathJax is enabled you can set the default output format to be used for\n# thA MathJax output. Supported types are HTML-CSS, NativeMML (i.e. MathML) and\n# SVG. The default value is HTML-CSS, which is slower, but has the best\n# compatibility.\n\nMATHJAX_FORMAT = HTML-CSS\n\n# When MathJax is enabled you need to specify the location relative to the\n# HTML output directory using the MATHJAX_RELPATH option. The destination\n# directory should contain the MathJax.js script. For instance, if the mathjax\n# directory is located at the same level as the HTML output directory, then\n# MATHJAX_RELPATH should be ../mathjax. The default value points to\n# the MathJax Content Delivery Network so you can quickly see the result without\n# installing MathJax.\n# However, it is strongly recommended to install a local\n# copy of MathJax from http://www.mathjax.org before deployment.\n\nMATHJAX_RELPATH = http://cdn.mathjax.org/mathjax/latest\n\n# The MATHJAX_EXTENSIONS tag can be used to specify one or MathJax extension\n# names that should be enabled during MathJax rendering.\n\nMATHJAX_EXTENSIONS =\n\n# When the SEARCHENGINE tag is enabled doxygen will generate a search box\n# for the HTML output. The underlying search engine uses javascript\n# and DHTML and should work on any modern browser. Note that when using\n# HTML help (GENERATE_HTMLHELP), Qt help (GENERATE_QHP), or docsets\n# (GENERATE_DOCSET) there is already a search function so this one should\n# typically be disabled. For large projects the javascript based search engine\n# can be slow, then enabling SERVER_BASED_SEARCH may provide a better solution.\n\nSEARCHENGINE = YES\n\n# When the SERVER_BASED_SEARCH tag is enabled the search engine will be\n# implemented using a web server instead of a web client using Javascript.\n# There are two flavours of web server based search depending on the\n# EXTERNAL_SEARCH setting. When disabled, doxygen will generate a PHP script for\n# searching and an index file used by the script. When EXTERNAL_SEARCH is\n# enabled the indexing and searching needs to be provided by external tools.\n# See the manual for details.\n\nSERVER_BASED_SEARCH = NO\n\n# When EXTERNAL_SEARCH is enabled doxygen will no longer generate the PHP\n# script for searching. Instead the search results are written to an XML file\n# which needs to be processed by an external indexer. Doxygen will invoke an\n# external search engine pointed to by the SEARCHENGINE_URL option to obtain\n# the search results. Doxygen ships with an example indexer (doxyindexer) and\n# search engine (doxysearch.cgi) which are based on the open source search engine\n# library Xapian. See the manual for configuration details.\n\nEXTERNAL_SEARCH = NO\n\n# The SEARCHENGINE_URL should point to a search engine hosted by a web server\n# which will returned the search results when EXTERNAL_SEARCH is enabled.\n# Doxygen ships with an example search engine (doxysearch) which is based on\n# the open source search engine library Xapian. See the manual for configuration\n# details.\n\nSEARCHENGINE_URL =\n\n# When SERVER_BASED_SEARCH and EXTERNAL_SEARCH are both enabled the unindexed\n# search data is written to a file for indexing by an external tool. With the\n# SEARCHDATA_FILE tag the name of this file can be specified.\n\nSEARCHDATA_FILE = searchdata.xml\n\n# When SERVER_BASED_SEARCH AND EXTERNAL_SEARCH are both enabled the\n# EXTERNAL_SEARCH_ID tag can be used as an identifier for the project. This is\n# useful in combination with EXTRA_SEARCH_MAPPINGS to search through multiple\n# projects and redirect the results back to the right project.\n\nEXTERNAL_SEARCH_ID =\n\n# The EXTRA_SEARCH_MAPPINGS tag can be used to enable searching through doxygen\n# projects other than the one defined by this configuration file, but that are\n# all added to the same external search index. Each project needs to have a\n# unique id set via EXTERNAL_SEARCH_ID. The search mapping then maps the id\n# of to a relative location where the documentation can be found.\n# The format is: EXTRA_SEARCH_MAPPINGS = id1=loc1 id2=loc2 ...\n\nEXTRA_SEARCH_MAPPINGS =\n\n#---------------------------------------------------------------------------\n# configuration options related to the LaTeX output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_LATEX tag is set to YES (the default) Doxygen will\n# generate Latex output.\n\nGENERATE_LATEX = YES\n\n# The LATEX_OUTPUT tag is used to specify where the LaTeX docs will be put.\n# If a relative path is entered the value of OUTPUT_DIRECTORY will be\n# put in front of it. If left blank `latex' will be used as the default path.\n\nLATEX_OUTPUT = latex\n\n# The LATEX_CMD_NAME tag can be used to specify the LaTeX command name to be\n# invoked. If left blank `latex' will be used as the default command name.\n# Note that when enabling USE_PDFLATEX this option is only used for\n# generating bitmaps for formulas in the HTML output, but not in the\n# Makefile that is written to the output directory.\n\nLATEX_CMD_NAME = latex\n\n# The MAKEINDEX_CMD_NAME tag can be used to specify the command name to\n# generate index for LaTeX. If left blank `makeindex' will be used as the\n# default command name.\n\nMAKEINDEX_CMD_NAME = makeindex\n\n# If the COMPACT_LATEX tag is set to YES Doxygen generates more compact\n# LaTeX documents. This may be useful for small projects and may help to\n# save some trees in general.\n\nCOMPACT_LATEX = NO\n\n# The PAPER_TYPE tag can be used to set the paper type that is used\n# by the printer. Possible values are: a4, letter, legal and\n# executive. If left blank a4wide will be used.\n\nPAPER_TYPE = a4\n\n# The EXTRA_PACKAGES tag can be to specify one or more names of LaTeX\n# packages that should be included in the LaTeX output.\n\nEXTRA_PACKAGES =\n\n# The LATEX_HEADER tag can be used to specify a personal LaTeX header for\n# the generated latex document. The header should contain everything until\n# the first chapter. If it is left blank doxygen will generate a\n# standard header. Notice: only use this tag if you know what you are doing!\n\nLATEX_HEADER =\n\n# The LATEX_FOOTER tag can be used to specify a personal LaTeX footer for\n# the generated latex document. The footer should contain everything after\n# the last chapter. If it is left blank doxygen will generate a\n# standard footer. Notice: only use this tag if you know what you are doing!\n\nLATEX_FOOTER =\n\n# If the PDF_HYPERLINKS tag is set to YES, the LaTeX that is generated\n# is prepared for conversion to pdf (using ps2pdf). The pdf file will\n# contain links (just like the HTML output) instead of page references\n# This makes the output suitable for online browsing using a pdf viewer.\n\nPDF_HYPERLINKS = YES\n\n# If the USE_PDFLATEX tag is set to YES, pdflatex will be used instead of\n# plain latex in the generated Makefile. Set this option to YES to get a\n# higher quality PDF documentation.\n\nUSE_PDFLATEX = YES\n\n# If the LATEX_BATCHMODE tag is set to YES, doxygen will add the \\\\batchmode.\n# command to the generated LaTeX files. This will instruct LaTeX to keep\n# running if errors occur, instead of asking the user for help.\n# This option is also used when generating formulas in HTML.\n\nLATEX_BATCHMODE = NO\n\n# If LATEX_HIDE_INDICES is set to YES then doxygen will not\n# include the index chapters (such as File Index, Compound Index, etc.)\n# in the output.\n\nLATEX_HIDE_INDICES = NO\n\n# If LATEX_SOURCE_CODE is set to YES then doxygen will include\n# source code with syntax highlighting in the LaTeX output.\n# Note that which sources are shown also depends on other settings\n# such as SOURCE_BROWSER.\n\nLATEX_SOURCE_CODE = NO\n\n# The LATEX_BIB_STYLE tag can be used to specify the style to use for the\n# bibliography, e.g. plainnat, or ieeetr. The default style is \"plain\". See\n# http://en.wikipedia.org/wiki/BibTeX for more info.\n\nLATEX_BIB_STYLE = plain\n\n#---------------------------------------------------------------------------\n# configuration options related to the RTF output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_RTF tag is set to YES Doxygen will generate RTF output\n# The RTF output is optimized for Word 97 and may not look very pretty with\n# other RTF readers or editors.\n\nGENERATE_RTF = NO\n\n# The RTF_OUTPUT tag is used to specify where the RTF docs will be put.\n# If a relative path is entered the value of OUTPUT_DIRECTORY will be\n# put in front of it. If left blank `rtf' will be used as the default path.\n\nRTF_OUTPUT = rtf\n\n# If the COMPACT_RTF tag is set to YES Doxygen generates more compact\n# RTF documents. This may be useful for small projects and may help to\n# save some trees in general.\n\nCOMPACT_RTF = NO\n\n# If the RTF_HYPERLINKS tag is set to YES, the RTF that is generated\n# will contain hyperlink fields. The RTF file will\n# contain links (just like the HTML output) instead of page references.\n# This makes the output suitable for online browsing using WORD or other\n# programs which support those fields.\n# Note: wordpad (write) and others do not support links.\n\nRTF_HYPERLINKS = NO\n\n# Load style sheet definitions from file. Syntax is similar to doxygen's\n# config file, i.e. a series of assignments. You only have to provide\n# replacements, missing definitions are set to their default value.\n\nRTF_STYLESHEET_FILE =\n\n# Set optional variables used in the generation of an rtf document.\n# Syntax is similar to doxygen's config file.\n\nRTF_EXTENSIONS_FILE =\n\n#---------------------------------------------------------------------------\n# configuration options related to the man page output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_MAN tag is set to YES (the default) Doxygen will\n# generate man pages\n\nGENERATE_MAN = NO\n\n# The MAN_OUTPUT tag is used to specify where the man pages will be put.\n# If a relative path is entered the value of OUTPUT_DIRECTORY will be\n# put in front of it. If left blank `man' will be used as the default path.\n\nMAN_OUTPUT = man\n\n# The MAN_EXTENSION tag determines the extension that is added to\n# the generated man pages (default is the subroutine's section .3)\n\nMAN_EXTENSION = .3\n\n# If the MAN_LINKS tag is set to YES and Doxygen generates man output,\n# then it will generate one additional man file for each entity\n# documented in the real man page(s). These additional files\n# only source the real man page, but without them the man command\n# would be unable to find the correct page. The default is NO.\n\nMAN_LINKS = NO\n\n#---------------------------------------------------------------------------\n# configuration options related to the XML output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_XML tag is set to YES Doxygen will\n# generate an XML file that captures the structure of\n# the code including all documentation.\n\nGENERATE_XML = NO\n\n# The XML_OUTPUT tag is used to specify where the XML pages will be put.\n# If a relative path is entered the value of OUTPUT_DIRECTORY will be\n# put in front of it. If left blank `xml' will be used as the default path.\n\nXML_OUTPUT = xml\n\n# The XML_SCHEMA tag can be used to specify an XML schema,\n# which can be used by a validating XML parser to check the\n# syntax of the XML files.\n\nXML_SCHEMA =\n\n# The XML_DTD tag can be used to specify an XML DTD,\n# which can be used by a validating XML parser to check the\n# syntax of the XML files.\n\nXML_DTD =\n\n# If the XML_PROGRAMLISTING tag is set to YES Doxygen will\n# dump the program listings (including syntax highlighting\n# and cross-referencing information) to the XML output. Note that\n# enabling this will significantly increase the size of the XML output.\n\nXML_PROGRAMLISTING = YES\n\n#---------------------------------------------------------------------------\n# configuration options for the AutoGen Definitions output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_AUTOGEN_DEF tag is set to YES Doxygen will\n# generate an AutoGen Definitions (see autogen.sf.net) file\n# that captures the structure of the code including all\n# documentation. Note that this feature is still experimental\n# and incomplete at the moment.\n\nGENERATE_AUTOGEN_DEF = NO\n\n#---------------------------------------------------------------------------\n# configuration options related to the Perl module output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_PERLMOD tag is set to YES Doxygen will\n# generate a Perl module file that captures the structure of\n# the code including all documentation. Note that this\n# feature is still experimental and incomplete at the\n# moment.\n\nGENERATE_PERLMOD = NO\n\n# If the PERLMOD_LATEX tag is set to YES Doxygen will generate\n# the necessary Makefile rules, Perl scripts and LaTeX code to be able\n# to generate PDF and DVI output from the Perl module output.\n\nPERLMOD_LATEX = NO\n\n# If the PERLMOD_PRETTY tag is set to YES the Perl module output will be\n# nicely formatted so it can be parsed by a human reader.\n# This is useful\n# if you want to understand what is going on.\n# On the other hand, if this\n# tag is set to NO the size of the Perl module output will be much smaller\n# and Perl will parse it just the same.\n\nPERLMOD_PRETTY = YES\n\n# The names of the make variables in the generated doxyrules.make file\n# are prefixed with the string contained in PERLMOD_MAKEVAR_PREFIX.\n# This is useful so different doxyrules.make files included by the same\n# Makefile don't overwrite each other's variables.\n\nPERLMOD_MAKEVAR_PREFIX =\n\n#---------------------------------------------------------------------------\n# Configuration options related to the preprocessor\n#---------------------------------------------------------------------------\n\n# If the ENABLE_PREPROCESSING tag is set to YES (the default) Doxygen will\n# evaluate all C-preprocessor directives found in the sources and include\n# files.\n\nENABLE_PREPROCESSING = YES\n\n# If the MACRO_EXPANSION tag is set to YES Doxygen will expand all macro\n# names in the source code. If set to NO (the default) only conditional\n# compilation will be performed. Macro expansion can be done in a controlled\n# way by setting EXPAND_ONLY_PREDEF to YES.\n\nMACRO_EXPANSION = NO\n\n# If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES\n# then the macro expansion is limited to the macros specified with the\n# PREDEFINED and EXPAND_AS_DEFINED tags.\n\nEXPAND_ONLY_PREDEF = NO\n\n# If the SEARCH_INCLUDES tag is set to YES (the default) the includes files\n# pointed to by INCLUDE_PATH will be searched when a #include is found.\n\nSEARCH_INCLUDES = YES\n\n# The INCLUDE_PATH tag can be used to specify one or more directories that\n# contain include files that are not input files but should be processed by\n# the preprocessor.\n\nINCLUDE_PATH =\n\n# You can use the INCLUDE_FILE_PATTERNS tag to specify one or more wildcard\n# patterns (like *.h and *.hpp) to filter out the header-files in the\n# directories. If left blank, the patterns specified with FILE_PATTERNS will\n# be used.\n\nINCLUDE_FILE_PATTERNS =\n\n# The PREDEFINED tag can be used to specify one or more macro names that\n# are defined before the preprocessor is started (similar to the -D option of\n# gcc). The argument of the tag is a list of macros of the form: name\n# or name=definition (no spaces). If the definition and the = are\n# omitted =1 is assumed. To prevent a macro definition from being\n# undefined via #undef or recursively expanded use the := operator\n# instead of the = operator.\n\nPREDEFINED =\n\n# If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then\n# this tag can be used to specify a list of macro names that should be expanded.\n# The macro definition that is found in the sources will be used.\n# Use the PREDEFINED tag if you want to use a different macro definition that\n# overrules the definition found in the source code.\n\nEXPAND_AS_DEFINED =\n\n# If the SKIP_FUNCTION_MACROS tag is set to YES (the default) then\n# doxygen's preprocessor will remove all references to function-like macros\n# that are alone on a line, have an all uppercase name, and do not end with a\n# semicolon, because these will confuse the parser if not removed.\n\nSKIP_FUNCTION_MACROS = YES\n\n#---------------------------------------------------------------------------\n# Configuration::additions related to external references\n#---------------------------------------------------------------------------\n\n# The TAGFILES option can be used to specify one or more tagfiles. For each\n# tag file the location of the external documentation should be added. The\n# format of a tag file without this location is as follows:\n#\n# TAGFILES = file1 file2 ...\n# Adding location for the tag files is done as follows:\n#\n# TAGFILES = file1=loc1 \"file2 = loc2\" ...\n# where \"loc1\" and \"loc2\" can be relative or absolute paths\n# or URLs. Note that each tag file must have a unique name (where the name does\n# NOT include the path). If a tag file is not located in the directory in which\n# doxygen is run, you must also specify the path to the tagfile here.\n\nTAGFILES =\n\n# When a file name is specified after GENERATE_TAGFILE, doxygen will create\n# a tag file that is based on the input files it reads.\n\nGENERATE_TAGFILE =\n\n# If the ALLEXTERNALS tag is set to YES all external classes will be listed\n# in the class index. If set to NO only the inherited external classes\n# will be listed.\n\nALLEXTERNALS = NO\n\n# If the EXTERNAL_GROUPS tag is set to YES all external groups will be listed\n# in the modules index. If set to NO, only the current project's groups will\n# be listed.\n\nEXTERNAL_GROUPS = YES\n\n# The PERL_PATH should be the absolute path and name of the perl script\n# interpreter (i.e. the result of `which perl').\n\nPERL_PATH = /usr/bin/perl\n\n#---------------------------------------------------------------------------\n# Configuration options related to the dot tool\n#---------------------------------------------------------------------------\n\n# If the CLASS_DIAGRAMS tag is set to YES (the default) Doxygen will\n# generate a inheritance diagram (in HTML, RTF and LaTeX) for classes with base\n# or super classes. Setting the tag to NO turns the diagrams off. Note that\n# this option also works with HAVE_DOT disabled, but it is recommended to\n# install and use dot, since it yields more powerful graphs.\n\nCLASS_DIAGRAMS = YES\n\n# You can define message sequence charts within doxygen comments using the \\msc\n# command. Doxygen will then run the mscgen tool (see\n# http://www.mcternan.me.uk/mscgen/) to produce the chart and insert it in the\n# documentation. The MSCGEN_PATH tag allows you to specify the directory where\n# the mscgen tool resides. If left empty the tool is assumed to be found in the\n# default search path.\n\nMSCGEN_PATH =\n\n# If set to YES, the inheritance and collaboration graphs will hide\n# inheritance and usage relations if the target is undocumented\n# or is not a class.\n\nHIDE_UNDOC_RELATIONS = YES\n\n# If you set the HAVE_DOT tag to YES then doxygen will assume the dot tool is\n# available from the path. This tool is part of Graphviz, a graph visualization\n# toolkit from AT&T and Lucent Bell Labs. The other options in this section\n# have no effect if this option is set to NO (the default)\n\nHAVE_DOT = YES\n\n# The DOT_NUM_THREADS specifies the number of dot invocations doxygen is\n# allowed to run in parallel. When set to 0 (the default) doxygen will\n# base this on the number of processors available in the system. You can set it\n# explicitly to a value larger than 0 to get control over the balance\n# between CPU load and processing speed.\n\nDOT_NUM_THREADS = 0\n\n# By default doxygen will use the Helvetica font for all dot files that\n# doxygen generates. When you want a differently looking font you can specify\n# the font name using DOT_FONTNAME. You need to make sure dot is able to find\n# the font, which can be done by putting it in a standard location or by setting\n# the DOTFONTPATH environment variable or by setting DOT_FONTPATH to the\n# directory containing the font.\n\nDOT_FONTNAME = Helvetica\n\n# The DOT_FONTSIZE tag can be used to set the size of the font of dot graphs.\n# The default size is 10pt.\n\nDOT_FONTSIZE = 10\n\n# By default doxygen will tell dot to use the Helvetica font.\n# If you specify a different font using DOT_FONTNAME you can use DOT_FONTPATH to\n# set the path where dot can find it.\n\nDOT_FONTPATH =\n\n# If the CLASS_GRAPH and HAVE_DOT tags are set to YES then doxygen\n# will generate a graph for each documented class showing the direct and\n# indirect inheritance relations. Setting this tag to YES will force the\n# CLASS_DIAGRAMS tag to NO.\n\nCLASS_GRAPH = YES\n\n# If the COLLABORATION_GRAPH and HAVE_DOT tags are set to YES then doxygen\n# will generate a graph for each documented class showing the direct and\n# indirect implementation dependencies (inheritance, containment, and\n# class references variables) of the class with other documented classes.\n\nCOLLABORATION_GRAPH = YES\n\n# If the GROUP_GRAPHS and HAVE_DOT tags are set to YES then doxygen\n# will generate a graph for groups, showing the direct groups dependencies\n\nGROUP_GRAPHS = YES\n\n# If the UML_LOOK tag is set to YES doxygen will generate inheritance and\n# collaboration diagrams in a style similar to the OMG's Unified Modeling\n# Language.\n\nUML_LOOK = NO\n\n# If the UML_LOOK tag is enabled, the fields and methods are shown inside\n# the class node. If there are many fields or methods and many nodes the\n# graph may become too big to be useful. The UML_LIMIT_NUM_FIELDS\n# threshold limits the number of items for each type to make the size more\n# managable. Set this to 0 for no limit. Note that the threshold may be\n# exceeded by 50% before the limit is enforced.\n\nUML_LIMIT_NUM_FIELDS = 10\n\n# If set to YES, the inheritance and collaboration graphs will show the\n# relations between templates and their instances.\n\nTEMPLATE_RELATIONS = NO\n\n# If the ENABLE_PREPROCESSING, SEARCH_INCLUDES, INCLUDE_GRAPH, and HAVE_DOT\n# tags are set to YES then doxygen will generate a graph for each documented\n# file showing the direct and indirect include dependencies of the file with\n# other documented files.\n\nINCLUDE_GRAPH = YES\n\n# If the ENABLE_PREPROCESSING, SEARCH_INCLUDES, INCLUDED_BY_GRAPH, and\n# HAVE_DOT tags are set to YES then doxygen will generate a graph for each\n# documented header file showing the documented files that directly or\n# indirectly include this file.\n\nINCLUDED_BY_GRAPH = YES\n\n# If the CALL_GRAPH and HAVE_DOT options are set to YES then\n# doxygen will generate a call dependency graph for every global function\n# or class method. Note that enabling this option will significantly increase\n# the time of a run. So in most cases it will be better to enable call graphs\n# for selected functions only using the \\callgraph command.\n\nCALL_GRAPH = YES\n\n# If the CALLER_GRAPH and HAVE_DOT tags are set to YES then\n# doxygen will generate a caller dependency graph for every global function\n# or class method. Note that enabling this option will significantly increase\n# the time of a run. So in most cases it will be better to enable caller\n# graphs for selected functions only using the \\callergraph command.\n\nCALLER_GRAPH = YES\n\n# If the GRAPHICAL_HIERARCHY and HAVE_DOT tags are set to YES then doxygen\n# will generate a graphical hierarchy of all classes instead of a textual one.\n\nGRAPHICAL_HIERARCHY = YES\n\n# If the DIRECTORY_GRAPH and HAVE_DOT tags are set to YES\n# then doxygen will show the dependencies a directory has on other directories\n# in a graphical way. The dependency relations are determined by the #include\n# relations between the files in the directories.\n\nDIRECTORY_GRAPH = YES\n\n# The DOT_IMAGE_FORMAT tag can be used to set the image format of the images\n# generated by dot. Possible values are svg, png, jpg, or gif.\n# If left blank png will be used. If you choose svg you need to set\n# HTML_FILE_EXTENSION to xhtml in order to make the SVG files\n# visible in IE 9+ (other browsers do not have this requirement).\n\nDOT_IMAGE_FORMAT = png\n\n# If DOT_IMAGE_FORMAT is set to svg, then this option can be set to YES to\n# enable generation of interactive SVG images that allow zooming and panning.\n# Note that this requires a modern browser other than Internet Explorer.\n# Tested and working are Firefox, Chrome, Safari, and Opera. For IE 9+ you\n# need to set HTML_FILE_EXTENSION to xhtml in order to make the SVG files\n# visible. Older versions of IE do not have SVG support.\n\nINTERACTIVE_SVG = NO\n\n# The tag DOT_PATH can be used to specify the path where the dot tool can be\n# found. If left blank, it is assumed the dot tool can be found in the path.\n\nDOT_PATH =\n\n# The DOTFILE_DIRS tag can be used to specify one or more directories that\n# contain dot files that are included in the documentation (see the\n# \\dotfile command).\n\nDOTFILE_DIRS =\n\n# The MSCFILE_DIRS tag can be used to specify one or more directories that\n# contain msc files that are included in the documentation (see the\n# \\mscfile command).\n\nMSCFILE_DIRS =\n\n# The DOT_GRAPH_MAX_NODES tag can be used to set the maximum number of\n# nodes that will be shown in the graph. If the number of nodes in a graph\n# becomes larger than this value, doxygen will truncate the graph, which is\n# visualized by representing a node as a red box. Note that doxygen if the\n# number of direct children of the root node in a graph is already larger than\n# DOT_GRAPH_MAX_NODES then the graph will not be shown at all. Also note\n# that the size of a graph can be further restricted by MAX_DOT_GRAPH_DEPTH.\n\nDOT_GRAPH_MAX_NODES = 50\n\n# The MAX_DOT_GRAPH_DEPTH tag can be used to set the maximum depth of the\n# graphs generated by dot. A depth value of 3 means that only nodes reachable\n# from the root by following a path via at most 3 edges will be shown. Nodes\n# that lay further from the root node will be omitted. Note that setting this\n# option to 1 or 2 may greatly reduce the computation time needed for large\n# code bases. Also note that the size of a graph can be further restricted by\n# DOT_GRAPH_MAX_NODES. Using a depth of 0 means no depth restriction.\n\nMAX_DOT_GRAPH_DEPTH = 0\n\n# Set the DOT_TRANSPARENT tag to YES to generate images with a transparent\n# background. This is disabled by default, because dot on Windows does not\n# seem to support this out of the box. Warning: Depending on the platform used,\n# enabling this option may lead to badly anti-aliased labels on the edges of\n# a graph (i.e. they become hard to read).\n\nDOT_TRANSPARENT = NO\n\n# Set the DOT_MULTI_TARGETS tag to YES allow dot to generate multiple output\n# files in one run (i.e. multiple -o and -T options on the command line). This\n# makes dot run faster, but since only newer versions of dot (>1.8.10)\n# support this, this feature is disabled by default.\n\nDOT_MULTI_TARGETS = NO\n\n# If the GENERATE_LEGEND tag is set to YES (the default) Doxygen will\n# generate a legend page explaining the meaning of the various boxes and\n# arrows in the dot generated graphs.\n\nGENERATE_LEGEND = YES\n\n# If the DOT_CLEANUP tag is set to YES (the default) Doxygen will\n# remove the intermediate dot files that are used to generate\n# the various graphs.\n\nDOT_CLEANUP = YES\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/lib/inc/rwt_init.h", "content": "/*! \\file rwt_init.h\n \\brief Header for matlab init functions in init.c\n*/\n#ifndef RWT_INIT_H_\n#define RWT_INIT_H_\n\n#include \"rwt_platform.h\"\n\n#if defined(MATLAB_MEX_FILE) || defined(OCTAVE_MEX_FILE)\n #include \"mex.h\"\n #ifndef OCTAVE_MEX_FILE\n #include \"matrix.h\"\n #endif\n typedef struct {\n size_t nrows; /*!< The number of rows in the input matrix. Output matrix will match. */\n size_t ncols; /*!< The number of columns in the input matrix. Output matrix will match. */\n int levels; /*!< L, the number of levels for the transform. */\n int ncoeff; /*!< Length of h / the number of scaling coefficients */\n double *scalings; /*!< Wavelet scaling coefficients */\n } rwt_init_params;\n typedef enum {NORMAL_DWT, REDUNDANT_DWT, INVERSE_DWT, INVERSE_REDUNDANT_DWT} transform_t;\n#endif\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n#if defined(MATLAB_MEX_FILE) || defined(OCTAVE_MEX_FILE)\n rwt_init_params rwt_matlab_init(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[], transform_t dwtType);\n#else\n int rwt_find_levels(size_t m, size_t n);\n int rwt_check_levels(int levels, size_t rows, size_t cols);\n#endif\n\n#ifdef __cplusplus\n}\n#endif\n\n#endif /* RWT_INIT_H_ */\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/lib/inc/rwt_platform.h", "content": "/*! \\file rwt_platform.h\n \\brief Abstract away environment differences and provide some common macros\n*/\n#ifndef RWT_PLATFORM_H\n#define RWT_PLATFORM_H\n\n#include \n#include \n#include \n\n/*! For MATLAB we address 2d inputs and outputs in column-major order */\n/*! For Python we address 2d inputs and outputs in row-major order */\n/*! The offset macros are for debugging */\n/*! The parameters for the mat() macro are:\n * a - the base pointer to the matrix of values\n * i - index of the target row\n * j - index of the target column\n * m - the number of rows\n * n - the number of columns\n */\n#if defined(MATLAB_MEX_FILE) || defined(OCTAVE_MEX_FILE)\n #define COLUMN_MAJOR_ORDER 1\n #include \"mex.h\"\n #ifndef OCTAVE_MEX_FILE\n #include \"matrix.h\"\n #endif\n #define mat(a, i, j, m, n) (*(a + (m*(j)+i)))\n #define mat_offset(a, i, j, m, n) (m*(j)+i)\n #define offset_row(offset, m, n) (offset % m)\n #define offset_col(offset, m, n) ((offset - (offset % m)) / m)\n #define rwt_printf(fmt, ...) mexPrintf(fmt, ##__VA_ARGS__)\n #define rwt_errormsg(msg) mexErrMsgTxt(msg)\n#else\n #define ROW_MAJOR_ORDER 1\n #define mat(a, i, j, m, n) (*(a + (n*(i)+j)))\n #define mat_offset(a, i, j, m, n) (n*(i)+j)\n #define offset_row(offset, m, n) ((offset - (offset % n)) / n)\n #define offset_col(offset, m, n) (offset % n)\n #define rwt_printf(fmt, ...) printf(fmt, ##__VA_ARGS__)\n #define rwt_errormsg(msg) printf(\"\\033[91m%s\\033[0m\\n\", msg);\n#endif\n\n#ifndef max\n #define max(A,B) (A > B ? A : B)\n#endif\n#ifndef min\n #define min(A,B) (A < B ? A : B)\n#endif\n#define even(x) ((x & 1) ? 0 : 1)\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\nvoid *rwt_malloc(size_t size);\nvoid *rwt_calloc(size_t num, size_t size);\nvoid rwt_free(void *ptr);\n\n#ifdef __cplusplus\n}\n#endif\n\n#endif\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/lib/inc/rwt_transforms.h", "content": "/*! \\file rwt_transforms.h\n \\brief Function prototypes for the transform implementations\n*/\n#ifndef TRANSFORMS_H_\n#define TRANSFORMS_H_\n\n#include \n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\n/*! dwt and rdwt take an input x and store the result in y or yl and yh\n * idwt and irdwt take an input y or yl and yh and store the result in x\n * In all cases it is expected that the output array has already been\n * allocated prior to calling the transform function.\n */\nvoid dwt(double *x, size_t nrows, size_t ncols, double *h, int ncoeff, int levels, double *y);\nvoid idwt(double *x, size_t nrows, size_t ncols, double *h, int ncoeff, int levels, double *y);\nvoid rdwt(double *x, size_t nrows, size_t ncols, double *h, int ncoeff, int levels, double *yl, double *yh);\nvoid irdwt(double *x, size_t nrows, size_t ncols, double *h, int ncoeff, int levels, double *yl, double *yh);\n\n#ifdef __cplusplus\n}\n#endif\n\n#endif /* TRANSFORMS_H_ */\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/lib/src/CMakeLists.txt", "content": "include_directories (\"${PROJECT_SOURCE_DIR}/lib/inc\")\nadd_library(dwt dwt.c)\nadd_library(idwt idwt.c)\nadd_library(irdwt irdwt.c)\nadd_library(rdwt rdwt.c)\nadd_library(platform platform.c)\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/lib/src/dwt.c", "content": "/*! \\file dwt.c\n \\brief Implementation of the discrete wavelet transform\n\n*/\n\n#include \"rwt_platform.h\"\n\n/*!\n * Perform convolution for dwt\n *\n * @param x_in input signal values\n * @param lx the length of x_in\n * @param coeff_low the low pass coefficients\n * @param coeff_high the high pass coefficients\n * @param ncoeff_minus_one one less than the number of scaling coefficients\n * @param x_out_low low pass results\n * @param x_out_high high pass results\n * \n * For the convolution we will calculate the output of the lowpass and highpass filters in parallel\n *\n * Normally we can describe the calculation of a convolution as\n * \\f$ (\\textbf{w} * \\textbf{z})_k = \\frac{1}{N} \\sum\\limits_{l=0}^{2N-1} w_{k-l} \\cdot z_{l} \\f$\n *\n * Our actual implementation resembles this\n *\n */\nvoid dwt_convolution(double *x_in, size_t lx, double *coeff_low, double *coeff_high, int ncoeff_minus_one, double *x_out_low, double *x_out_high) {\n size_t i, j, ind;\n double x0, x1;\n for (i=lx; i1) {\n for (idx_columns=0; idx_columns -1; k--) {\n x_in_low[k] = x_in_low[lx+k];\n x_in_high[k] = x_in_high[lx+k];\n }\n\n ind = 0;\n for (i=0; i<(lx); i++) {\n x0 = 0;\n x1 = 0;\n tj = 0;\n for (j=0; j<=ncoeff_halved_minus_one; j++) {\n x0 = x0 + (x_in_low[i+j] * coeff_low[ncoeff_minus_one-1-tj]) + (x_in_high[i+j] * coeff_high[ncoeff_minus_one-1-tj]);\n x1 = x1 + (x_in_low[i+j] * coeff_low[ncoeff_minus_one-tj]) + (x_in_high[i+j] * coeff_high[ncoeff_minus_one-tj]);\n tj += 2;\n }\n x_out[ind++] = x0;\n x_out[ind++] = x1;\n }\n}\n\n\n/*!\n * Allocate memory for idwt\n *\n * @param m the number of rows of the input matrix\n * @param n the number of columns of the input matrix\n * @param ncoeff the number of scaling coefficients\n * @param x_dummy\n * @param y_dummy_low\n * @param y_dummy_high\n * @param coeff_low\n * @param coeff_high\n *\n */\nvoid idwt_allocate(size_t m, size_t n, int ncoeff, double **x_dummy, double **y_dummy_low, double **y_dummy_high, double **coeff_low, double **coeff_high) {\n *x_dummy = (double *) rwt_calloc(max(m,n), sizeof(double));\n *y_dummy_low = (double *) rwt_calloc(max(m,n)+ncoeff/2-1, sizeof(double));\n *y_dummy_high = (double *) rwt_calloc(max(m,n)+ncoeff/2-1, sizeof(double));\n *coeff_low = (double *) rwt_calloc(ncoeff, sizeof(double));\n *coeff_high = (double *) rwt_calloc(ncoeff, sizeof(double));\n}\n\n\n/*!\n * Free memory we allocated for idwt\n *\n * @param x_dummy\n * @param y_dummy_low\n * @param y_dummy_high\n * @param coeff_low\n * @param coeff_high\n *\n */\nvoid idwt_free(double **x_dummy, double **y_dummy_low, double **y_dummy_high, double **coeff_low, double **coeff_high) {\n rwt_free(*x_dummy);\n rwt_free(*y_dummy_low);\n rwt_free(*y_dummy_high);\n rwt_free(*coeff_low);\n rwt_free(*coeff_high);\n}\n\n\n/*!\n * Put the scaling coeffients into a form ready for use in the convolution function\n *\n * @param ncoeff length of h / the number of scaling coefficients\n * @param h the wavelet scaling coefficients\n * @param coeff_low same as h\n * @param coeff_high reversed h, even values are sign flipped\n *\n */\nvoid idwt_coefficients(int ncoeff, double *h, double **coeff_low, double **coeff_high) {\n int i;\n for (i=0; i1)\n current_rows = nrows/sample_f;\n else \n current_rows = 1;\n current_cols = ncols/sample_f;\n\n for (i=0; i<(nrows*ncols); i++)\n x[i] = y[i];\n \n /* main loop */\n for (current_level=levels; current_level >= 1; current_level--) {\n row_cursor = current_rows/2;\n column_cursor = current_cols/2;\n \n /* go by columns in case of a 2D signal*/\n if (nrows>1) {\n for (idx_cols=0; idx_cols\n\n#if defined(MATLAB_MEX_FILE) || defined(OCTAVE_MEX_FILE)\n/*!\n * Checks for correct # of input variables based on type of transform.\n *\n * @param nrhs number of items on right hand side of matlab call\n * @param transform_type \n *\n */\nint rwt_check_parameter_count(int nrhs, transform_t transform_type) {\n if (transform_type == INVERSE_REDUNDANT_DWT) {\n if (nrhs > 4) {\n rwt_errormsg(\"There are at most 4 input parameters allowed!\");\n return 1;\n }\n if (nrhs < 3) {\n rwt_errormsg(\"There are at least 3 input parameters required!\");\n return 1;\n }\n }\n else {\n if (nrhs > 3) {\n rwt_errormsg(\"There are at most 3 input parameters allowed!\");\n return 1;\n }\n if (nrhs < 2) {\n rwt_errormsg(\"There are at least 2 input parameters required!\");\n return 1;\n }\n }\n return 0;\n}\n\n\n/*!\n * For the inverse redundant transform check that the dimensions of the low and high inputs match\n *\n * @param prhs\n * @param params\n *\n */\nint rwt_check_yl_matches_yh(const mxArray *prhs[], size_t nrows, size_t ncols, int levels) {\n size_t mh = mxGetM(prhs[1]);\n size_t nh = mxGetN(prhs[1]);\n if (min(nrows, ncols) > 1) {\n if ((nrows != mh) | (3 * ncols * levels != nh)) {\n return 0;\n }\n }\n else {\n if ((nrows != mh) | (ncols * levels != nh)) {\n return 0;\n }\n }\n return 1;\n}\n#endif\n\n\n/*!\n * Find L, the number of levels\n *\n * @param m the number of rows in the input\n * @param n the number of columns in the input\n *\n * L is the exponent of the largest power of 2 that is a factor of all input dimensions\n * \n */\nint rwt_find_levels(size_t m, size_t n) {\n size_t i, j, L;\n i = n ; j = 0;\n while (even(i)) {\n i = (i >> 1);\n j++;\n }\n L = m; i = 0;\n while (even(L)) {\n L = (L >> 1);\n i++;\n }\n if (min(m, n) == 1)\n L = max(i, j);\n else\n L = min(i, j);\n if (L == 0) {\n rwt_errormsg(\"Maximum number of levels is zero; no decomposition can be performed!\");\n return -1;\n }\n else return L;\n}\n\n\n/*!\n * Check that length is divisble by 2^L\n *\n * @param length the number of rows or number of columns\n * @param L the number of levels\n *\n */\nint rwt_check_dimensions(size_t length, int L) {\n double test = (double) length / pow(2.0, (double) L);\n if ((test - floor(test)) > 0.0) {\n return -1;\n }\n return 0;\n}\n\n\n/*!\n * Sanity check the levels parameter\n *\n * @param levels the number of levels specified or calculated for the input\n * @param rows the number of rows of input\n * @param cols the number of columns of input\n *\n */\nint rwt_check_levels(int levels, size_t rows, size_t cols) {\n if (levels < 1) {\n rwt_errormsg(\"The number of levels, L, must be a positive integer\");\n return -1;\n }\n\n /*! Check that both the rows and columns are divisible by 2^L */\n if ((rows > 1 && rwt_check_dimensions(rows, levels)) || (cols > 1 && rwt_check_dimensions(cols, levels))) {\n rwt_errormsg(\"All dimensions must be divisible by 2^L\");\n return -1;\n }\n\n return 0;\n}\n\n\n#if defined(MATLAB_MEX_FILE) || defined(OCTAVE_MEX_FILE)\n/*!\n * Parse input from MATLAB and do some sanity checking\n *\n * @param nlhs number of items on left hand side of matlab call\n * @param plhs pointer to left hand side data structure\n * @param nrhs number of items on right hand side of matlab call\n * @param prhs pointer to right hand side data structure\n * @param transform_type which transform are we setting up to do\n *\n */\nrwt_init_params rwt_matlab_init(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[], transform_t transform_type) {\n rwt_init_params params;\n int argNumL;\n\n /*! Check for correct # of input parameters */\n if (rwt_check_parameter_count(nrhs, transform_type) != 0) return params;\n /*! Check that we don't have more than two dimensions in the input since that is currently unsupported. */\n if (mxGetNumberOfDimensions(prhs[0]) > 2) {\n rwt_errormsg(\"Matrix must have fewer than 3 dimensions!\");\n return params;\n }\n /*! Get the number of rows and columns in the input matrix. */\n params.nrows = mxGetM(prhs[0]);\n params.ncols = mxGetN(prhs[0]);\n\n if (params.nrows == 0 && params.ncols == 0) {\n rwt_errormsg(\"The input matrix cannot be empty\");\n return params;\n }\n\n /*! Read the number of levels, L, from the input values if it was given, otherwise calculate L. Sanity check L */\n argNumL = (transform_type == INVERSE_REDUNDANT_DWT) ? 3 : 2;\n if ((argNumL + 1) == nrhs)\n params.levels = (int) *mxGetPr(prhs[argNumL]);\n else\n params.levels = rwt_find_levels(params.nrows, params.ncols);\n\n if (rwt_check_levels(params.levels, params.nrows, params.ncols)) {\n return params;\n }\n\n /*! Read the scaling coefficients, h, from the input and find their length, ncoeff. \n * In the case of the redundant transform, the scalings are found one further position to the right, \n * and also we check for matching dimensions in the low and high inputs\n */\n if (transform_type == INVERSE_REDUNDANT_DWT) {\n params.scalings = mxGetPr(prhs[2]);\n params.ncoeff = max(mxGetM(prhs[2]), mxGetN(prhs[2]));\n if (!rwt_check_yl_matches_yh(prhs, params.nrows, params.ncols, params.levels)) {\n rwt_errormsg(\"Dimensions of first two input matrices not consistent!\");\n return params;\n }\n }\n else {\n params.scalings = mxGetPr(prhs[1]);\n params.ncoeff = max(mxGetM(prhs[1]), mxGetN(prhs[1]));\n }\n /*! Create the first item in the output array as a double matrix with the same dimensions as the input. */\n plhs[0] = mxCreateDoubleMatrix(params.nrows, params.ncols, mxREAL);\n return params;\n}\n#endif\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/lib/src/irdwt.c", "content": "/*! \\file irdwt.c\n \\brief Implementation of the inverse redundant discrete wavelet transform\n\n*/\n\n#include \"rwt_platform.h\"\n\nvoid irdwt_convolution(double *x_out, size_t lx, double *coeff_low, double *coeff_high, int ncoeff, double *x_in_low, double *x_in_high) {\n int k;\n size_t i, j;\n double x0;\n\n for (k=ncoeff-2; k > -1; k--) {\n x_in_low[k] = x_in_low[lx+k];\n x_in_high[k] = x_in_high[lx+k];\n }\n for (i=0; i= 1; current_level--) {\n /* actual (level dependent) column offset */\n if (nrows==1)\n column_cursor = ncols*(current_level-1);\n else\n column_cursor = 3*ncols*(current_level-1);\n column_cursor_plus_n = column_cursor + ncols;\n column_cursor_plus_double_n = column_cursor_plus_n + ncols;\n \n /* go by columns in case of a 2D signal*/\n if (nrows>1) {\n row_blocks_per_column = nrows/current_rows; /* # of row blocks per column */\n for (idx_cols=0; idx_cols1)\n\t y_dummy_high_high[i+ncoeff_minus_one] = mat(x_high, idx_rows, idx_cols, nrows, ncols);\n\t else\n y_dummy_high_high[i+ncoeff_minus_one] = mat(y_high, idx_rows, idx_cols + column_cursor, nrows, three_n_L);\n\t} \n\t/* perform filtering lowpass/highpass */\n\tirdwt_convolution(x_dummy_low, current_cols, coeff_low, coeff_high, ncoeff, y_dummy_low_low, y_dummy_high_high); \n\t/* restore dummy variables in matrices */\n\tidx_cols = -sample_f + n_c;\n\tfor (i=0; i1) {\n n_rb = nrows/current_rows; /* # of row blocks per column */\n for (idx_cols=0; idx_cols\n%}\n\n/**********************************************************************/\n\n/* The following code originally appeared in\n * enthought/kiva/agg/src/numeric.i written by Eric Jones. It was\n * translated from C++ to C by John Hunter. Bill Spotz has modified\n * it to fix some minor bugs, upgrade from Numeric to numpy (all\n * versions), add some comments and functionality, and convert from\n * direct code insertion to SWIG fragments.\n */\n\n%fragment(\"NumPy_Macros\", \"header\")\n{\n/* Macros to extract array attributes.\n */\n%#define is_array(a) ((a) && PyArray_Check((PyArrayObject *)a))\n%#define array_type(a) (int)(PyArray_TYPE(a))\n%#define array_numdims(a) (((PyArrayObject *)a)->nd)\n%#define array_dimensions(a) (((PyArrayObject *)a)->dimensions)\n%#define array_size(a,i) (((PyArrayObject *)a)->dimensions[i])\n%#define array_data(a) (((PyArrayObject *)a)->data)\n%#define array_is_contiguous(a) (PyArray_ISCONTIGUOUS(a))\n%#define array_is_native(a) (PyArray_ISNOTSWAPPED(a))\n%#define array_is_fortran(a) (PyArray_ISFORTRAN(a))\n}\n\n/**********************************************************************/\n\n%fragment(\"NumPy_Utilities\", \"header\")\n{\n /* Given a PyObject, return a string describing its type.\n */\n const char* pytype_string(PyObject* py_obj) {\n if (py_obj == NULL ) return \"C NULL value\";\n if (py_obj == Py_None ) return \"Python None\" ;\n if (PyCallable_Check(py_obj)) return \"callable\" ;\n if (PyString_Check( py_obj)) return \"string\" ;\n if (PyInt_Check( py_obj)) return \"int\" ;\n if (PyFloat_Check( py_obj)) return \"float\" ;\n if (PyDict_Check( py_obj)) return \"dict\" ;\n if (PyList_Check( py_obj)) return \"list\" ;\n if (PyTuple_Check( py_obj)) return \"tuple\" ;\n if (PyModule_Check( py_obj)) return \"module\" ;\n%#if PY_MAJOR_VERSION < 3\n if (PyFile_Check( py_obj)) return \"file\" ;\n if (PyInstance_Check(py_obj)) return \"instance\" ;\n%#endif\n\n return \"unkown type\";\n }\n\n /* Given a NumPy typecode, return a string describing the type.\n */\n const char* typecode_string(int typecode) {\n static const char* type_names[25] = {\"bool\", \"byte\", \"unsigned byte\",\n \"short\", \"unsigned short\", \"int\",\n \"unsigned int\", \"long\", \"unsigned long\",\n \"long long\", \"unsigned long long\",\n \"float\", \"double\", \"long double\",\n \"complex float\", \"complex double\",\n \"complex long double\", \"object\",\n \"string\", \"unicode\", \"void\", \"ntypes\",\n \"notype\", \"char\", \"unknown\"};\n return typecode < 24 ? type_names[typecode] : type_names[24];\n }\n\n /* Make sure input has correct numpy type. Allow character and byte\n * to match. Also allow int and long to match. This is deprecated.\n * You should use PyArray_EquivTypenums() instead.\n */\n int type_match(int actual_type, int desired_type) {\n return PyArray_EquivTypenums(actual_type, desired_type);\n }\n}\n\n/**********************************************************************/\n\n%fragment(\"NumPy_Object_to_Array\", \"header\",\n fragment=\"NumPy_Backward_Compatibility\",\n fragment=\"NumPy_Macros\",\n fragment=\"NumPy_Utilities\")\n{\n /* Given a PyObject pointer, cast it to a PyArrayObject pointer if\n * legal. If not, set the python error string appropriately and\n * return NULL.\n */\n PyArrayObject* obj_to_array_no_conversion(PyObject* input, int typecode)\n {\n PyArrayObject* ary = NULL;\n if (is_array(input) && (typecode == NPY_NOTYPE ||\n PyArray_EquivTypenums(array_type(input), typecode)))\n {\n ary = (PyArrayObject*) input;\n }\n else if is_array(input)\n {\n const char* desired_type = typecode_string(typecode);\n const char* actual_type = typecode_string(array_type(input));\n PyErr_Format(PyExc_TypeError,\n \"Array of type '%s' required. Array of type '%s' given\",\n desired_type, actual_type);\n ary = NULL;\n }\n else\n {\n const char * desired_type = typecode_string(typecode);\n const char * actual_type = pytype_string(input);\n PyErr_Format(PyExc_TypeError,\n \"Array of type '%s' required. A '%s' was given\",\n desired_type, actual_type);\n ary = NULL;\n }\n return ary;\n }\n\n /* Convert the given PyObject to a NumPy array with the given\n * typecode. On success, return a valid PyArrayObject* with the\n * correct type. On failure, the python error string will be set and\n * the routine returns NULL.\n */\n PyArrayObject* obj_to_array_allow_conversion(PyObject* input, int typecode,\n int* is_new_object)\n {\n PyArrayObject* ary = NULL;\n PyObject* py_obj;\n if (is_array(input) && (typecode == NPY_NOTYPE ||\n PyArray_EquivTypenums(array_type(input),typecode)))\n {\n ary = (PyArrayObject*) input;\n *is_new_object = 0;\n }\n else\n {\n py_obj = PyArray_FROMANY(input, typecode, 0, 0, NPY_DEFAULT);\n /* If NULL, PyArray_FromObject will have set python error value.*/\n ary = (PyArrayObject*) py_obj;\n *is_new_object = 1;\n }\n return ary;\n }\n\n /* Given a PyArrayObject, check to see if it is contiguous. If so,\n * return the input pointer and flag it as not a new object. If it is\n * not contiguous, create a new PyArrayObject using the original data,\n * flag it as a new object and return the pointer.\n */\n PyArrayObject* make_contiguous(PyArrayObject* ary, int* is_new_object,\n int min_dims, int max_dims)\n {\n PyArrayObject* result;\n if (array_is_contiguous(ary))\n {\n result = ary;\n *is_new_object = 0;\n }\n else\n {\n result = (PyArrayObject*) PyArray_ContiguousFromObject((PyObject*)ary,\n array_type(ary),\n min_dims,\n max_dims);\n *is_new_object = 1;\n }\n return result;\n }\n\n /* Given a PyArrayObject, check to see if it is Fortran-contiguous.\n * If so, return the input pointer, but do not flag it as not a new\n * object. If it is not Fortran-contiguous, create a new\n * PyArrayObject using the original data, flag it as a new object\n * and return the pointer.\n */\n PyArrayObject* make_fortran(PyArrayObject* ary, int* is_new_object,\n int min_dims, int max_dims)\n {\n PyArrayObject* result;\n if (array_is_fortran(ary))\n {\n result = ary;\n *is_new_object = 0;\n }\n else\n {\n Py_INCREF(ary->descr);\n result = (PyArrayObject*) PyArray_FromArray(ary, ary->descr, NPY_FORTRAN);\n *is_new_object = 1;\n }\n return result;\n }\n\n /* Convert a given PyObject to a contiguous PyArrayObject of the\n * specified type. If the input object is not a contiguous\n * PyArrayObject, a new one will be created and the new object flag\n * will be set.\n */\n PyArrayObject* obj_to_array_contiguous_allow_conversion(PyObject* input,\n int typecode,\n int* is_new_object)\n {\n int is_new1 = 0;\n int is_new2 = 0;\n PyArrayObject* ary2;\n PyArrayObject* ary1 = obj_to_array_allow_conversion(input, typecode,\n &is_new1);\n if (ary1)\n {\n ary2 = make_contiguous(ary1, &is_new2, 0, 0);\n if ( is_new1 && is_new2)\n {\n Py_DECREF(ary1);\n }\n ary1 = ary2;\n }\n *is_new_object = is_new1 || is_new2;\n return ary1;\n }\n\n /* Convert a given PyObject to a Fortran-ordered PyArrayObject of the\n * specified type. If the input object is not a Fortran-ordered\n * PyArrayObject, a new one will be created and the new object flag\n * will be set.\n */\n PyArrayObject* obj_to_array_fortran_allow_conversion(PyObject* input,\n int typecode,\n int* is_new_object)\n {\n int is_new1 = 0;\n int is_new2 = 0;\n PyArrayObject* ary2;\n PyArrayObject* ary1 = obj_to_array_allow_conversion(input, typecode,\n &is_new1);\n if (ary1)\n {\n ary2 = make_fortran(ary1, &is_new2, 0, 0);\n if (is_new1 && is_new2)\n {\n Py_DECREF(ary1);\n }\n ary1 = ary2;\n }\n *is_new_object = is_new1 || is_new2;\n return ary1;\n }\n\n} /* end fragment */\n\n\n/**********************************************************************/\n\n%fragment(\"NumPy_Array_Requirements\", \"header\",\n fragment=\"NumPy_Backward_Compatibility\",\n fragment=\"NumPy_Macros\")\n{\n /* Test whether a python object is contiguous. If array is\n * contiguous, return 1. Otherwise, set the python error string and\n * return 0.\n */\n int require_contiguous(PyArrayObject* ary)\n {\n int contiguous = 1;\n if (!array_is_contiguous(ary))\n {\n PyErr_SetString(PyExc_TypeError,\n \"Array must be contiguous. A non-contiguous array was given\");\n contiguous = 0;\n }\n return contiguous;\n }\n\n /* Require that a numpy array is not byte-swapped. If the array is\n * not byte-swapped, return 1. Otherwise, set the python error string\n * and return 0.\n */\n int require_native(PyArrayObject* ary)\n {\n int native = 1;\n if (!array_is_native(ary))\n {\n PyErr_SetString(PyExc_TypeError,\n \"Array must have native byteorder. \"\n \"A byte-swapped array was given\");\n native = 0;\n }\n return native;\n }\n\n /* Require the given PyArrayObject to have a specified number of\n * dimensions. If the array has the specified number of dimensions,\n * return 1. Otherwise, set the python error string and return 0.\n */\n int require_dimensions(PyArrayObject* ary, int exact_dimensions)\n {\n int success = 1;\n if (array_numdims(ary) != exact_dimensions)\n {\n PyErr_Format(PyExc_TypeError,\n \"Array must have %d dimensions. Given array has %d dimensions\",\n exact_dimensions, array_numdims(ary));\n success = 0;\n }\n return success;\n }\n\n /* Require the given PyArrayObject to have one of a list of specified\n * number of dimensions. If the array has one of the specified number\n * of dimensions, return 1. Otherwise, set the python error string\n * and return 0.\n */\n int require_dimensions_n(PyArrayObject* ary, int* exact_dimensions, int n)\n {\n int success = 0;\n int i;\n char dims_str[255] = \"\";\n char s[255];\n for (i = 0; i < n && !success; i++)\n {\n if (array_numdims(ary) == exact_dimensions[i])\n {\n success = 1;\n }\n }\n if (!success)\n {\n for (i = 0; i < n-1; i++)\n {\n sprintf(s, \"%d, \", exact_dimensions[i]);\n strcat(dims_str,s);\n }\n sprintf(s, \" or %d\", exact_dimensions[n-1]);\n strcat(dims_str,s);\n PyErr_Format(PyExc_TypeError,\n \"Array must have %s dimensions. Given array has %d dimensions\",\n dims_str, array_numdims(ary));\n }\n return success;\n }\n\n /* Require the given PyArrayObject to have a specified shape. If the\n * array has the specified shape, return 1. Otherwise, set the python\n * error string and return 0.\n */\n int require_size(PyArrayObject* ary, npy_intp* size, int n)\n {\n int i;\n int success = 1;\n int len;\n char desired_dims[255] = \"[\";\n char s[255];\n char actual_dims[255] = \"[\";\n for(i=0; i < n;i++)\n {\n if (size[i] != -1 && size[i] != array_size(ary,i))\n {\n success = 0;\n }\n }\n if (!success)\n {\n for (i = 0; i < n; i++)\n {\n if (size[i] == -1)\n {\n sprintf(s, \"*,\");\n }\n else\n {\n sprintf(s, \"%ld,\", (long int)size[i]);\n }\n strcat(desired_dims,s);\n }\n len = strlen(desired_dims);\n desired_dims[len-1] = ']';\n for (i = 0; i < n; i++)\n {\n sprintf(s, \"%ld,\", (long int)array_size(ary,i));\n strcat(actual_dims,s);\n }\n len = strlen(actual_dims);\n actual_dims[len-1] = ']';\n PyErr_Format(PyExc_TypeError,\n \"Array must have shape of %s. Given array has shape of %s\",\n desired_dims, actual_dims);\n }\n return success;\n }\n\n /* Require the given PyArrayObject to to be FORTRAN ordered. If the\n * the PyArrayObject is already FORTRAN ordered, do nothing. Else,\n * set the FORTRAN ordering flag and recompute the strides.\n */\n int require_fortran(PyArrayObject* ary)\n {\n int success = 1;\n int nd = array_numdims(ary);\n int i;\n if (array_is_fortran(ary)) return success;\n /* Set the FORTRAN ordered flag */\n ary->flags = NPY_FARRAY;\n /* Recompute the strides */\n ary->strides[0] = ary->strides[nd-1];\n for (i=1; i < nd; ++i)\n ary->strides[i] = ary->strides[i-1] * array_size(ary,i-1);\n return success;\n }\n}\n\n/* Combine all NumPy fragments into one for convenience */\n%fragment(\"NumPy_Fragments\", \"header\",\n fragment=\"NumPy_Backward_Compatibility\",\n fragment=\"NumPy_Macros\",\n fragment=\"NumPy_Utilities\",\n fragment=\"NumPy_Object_to_Array\",\n fragment=\"NumPy_Array_Requirements\") { }\n\n/* End John Hunter translation (with modifications by Bill Spotz)\n */\n\n/* %numpy_typemaps() macro\n *\n * This macro defines a family of 41 typemaps that allow C arguments\n * of the form\n *\n * (DATA_TYPE IN_ARRAY1[ANY])\n * (DATA_TYPE* IN_ARRAY1, DIM_TYPE DIM1)\n * (DIM_TYPE DIM1, DATA_TYPE* IN_ARRAY1)\n *\n * (DATA_TYPE IN_ARRAY2[ANY][ANY])\n * (DATA_TYPE* IN_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n * (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_ARRAY2)\n * (DATA_TYPE* IN_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n * (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_FARRAY2)\n *\n * (DATA_TYPE IN_ARRAY3[ANY][ANY][ANY])\n * (DATA_TYPE* IN_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n * (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* IN_ARRAY3)\n * (DATA_TYPE* IN_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n * (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* IN_FARRAY3)\n *\n * (DATA_TYPE INPLACE_ARRAY1[ANY])\n * (DATA_TYPE* INPLACE_ARRAY1, DIM_TYPE DIM1)\n * (DIM_TYPE DIM1, DATA_TYPE* INPLACE_ARRAY1)\n *\n * (DATA_TYPE INPLACE_ARRAY2[ANY][ANY])\n * (DATA_TYPE* INPLACE_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n * (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* INPLACE_ARRAY2)\n * (DATA_TYPE* INPLACE_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n * (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* INPLACE_FARRAY2)\n *\n * (DATA_TYPE INPLACE_ARRAY3[ANY][ANY][ANY])\n * (DATA_TYPE* INPLACE_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n * (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* INPLACE_ARRAY3)\n * (DATA_TYPE* INPLACE_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n * (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* INPLACE_FARRAY3)\n *\n * (DATA_TYPE ARGOUT_ARRAY1[ANY])\n * (DATA_TYPE* ARGOUT_ARRAY1, DIM_TYPE DIM1)\n * (DIM_TYPE DIM1, DATA_TYPE* ARGOUT_ARRAY1)\n *\n * (DATA_TYPE ARGOUT_ARRAY2[ANY][ANY])\n *\n * (DATA_TYPE ARGOUT_ARRAY3[ANY][ANY][ANY])\n *\n * (DATA_TYPE** ARGOUTVIEW_ARRAY1, DIM_TYPE* DIM1)\n * (DIM_TYPE* DIM1, DATA_TYPE** ARGOUTVIEW_ARRAY1)\n *\n * (DATA_TYPE** ARGOUTVIEW_ARRAY2, DIM_TYPE* DIM1, DIM_TYPE* DIM2)\n * (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DATA_TYPE** ARGOUTVIEW_ARRAY2)\n * (DATA_TYPE** ARGOUTVIEW_FARRAY2, DIM_TYPE* DIM1, DIM_TYPE* DIM2)\n * (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DATA_TYPE** ARGOUTVIEW_FARRAY2)\n *\n * (DATA_TYPE** ARGOUTVIEW_ARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3)\n * (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DATA_TYPE** ARGOUTVIEW_ARRAY3)\n * (DATA_TYPE** ARGOUTVIEW_FARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3)\n * (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DATA_TYPE** ARGOUTVIEW_FARRAY3)\n *\n * where \"DATA_TYPE\" is any type supported by the NumPy module, and\n * \"DIM_TYPE\" is any int-like type suitable for specifying dimensions.\n * The difference between \"ARRAY\" typemaps and \"FARRAY\" typemaps is\n * that the \"FARRAY\" typemaps expect FORTRAN ordering of\n * multidimensional arrays. In python, the dimensions will not need\n * to be specified (except for the \"DATA_TYPE* ARGOUT_ARRAY1\"\n * typemaps). The IN_ARRAYs can be a numpy array or any sequence that\n * can be converted to a numpy array of the specified type. The\n * INPLACE_ARRAYs must be numpy arrays of the appropriate type. The\n * ARGOUT_ARRAYs will be returned as new numpy arrays of the\n * appropriate type.\n *\n * These typemaps can be applied to existing functions using the\n * %apply directive. For example:\n *\n * %apply (double* IN_ARRAY1, int DIM1) {(double* series, int length)};\n * double prod(double* series, int length);\n *\n * %apply (int DIM1, int DIM2, double* INPLACE_ARRAY2)\n * {(int rows, int cols, double* matrix )};\n * void floor(int rows, int cols, double* matrix, double f);\n *\n * %apply (double IN_ARRAY3[ANY][ANY][ANY])\n * {(double tensor[2][2][2] )};\n * %apply (double ARGOUT_ARRAY3[ANY][ANY][ANY])\n * {(double low[2][2][2] )};\n * %apply (double ARGOUT_ARRAY3[ANY][ANY][ANY])\n * {(double upp[2][2][2] )};\n * void luSplit(double tensor[2][2][2],\n * double low[2][2][2],\n * double upp[2][2][2] );\n *\n * or directly with\n *\n * double prod(double* IN_ARRAY1, int DIM1);\n *\n * void floor(int DIM1, int DIM2, double* INPLACE_ARRAY2, double f);\n *\n * void luSplit(double IN_ARRAY3[ANY][ANY][ANY],\n * double ARGOUT_ARRAY3[ANY][ANY][ANY],\n * double ARGOUT_ARRAY3[ANY][ANY][ANY]);\n */\n\n%define %numpy_typemaps(DATA_TYPE, DATA_TYPECODE, DIM_TYPE)\n\n/************************/\n/* Input Array Typemaps */\n/************************/\n\n/* Typemap suite for (DATA_TYPE IN_ARRAY1[ANY])\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE IN_ARRAY1[ANY])\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE IN_ARRAY1[ANY])\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[1] = { $1_dim0 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 1) ||\n !require_size(array, size, 1)) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n%typemap(freearg)\n (DATA_TYPE IN_ARRAY1[ANY])\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DATA_TYPE* IN_ARRAY1, DIM_TYPE DIM1)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* IN_ARRAY1, DIM_TYPE DIM1)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* IN_ARRAY1, DIM_TYPE DIM1)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[1] = { -1 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 1) ||\n !require_size(array, size, 1)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n}\n%typemap(freearg)\n (DATA_TYPE* IN_ARRAY1, DIM_TYPE DIM1)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DATA_TYPE* IN_ARRAY1)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DATA_TYPE* IN_ARRAY1)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DATA_TYPE* IN_ARRAY1)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[1] = {-1};\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 1) ||\n !require_size(array, size, 1)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DATA_TYPE*) array_data(array);\n}\n%typemap(freearg)\n (DIM_TYPE DIM1, DATA_TYPE* IN_ARRAY1)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DATA_TYPE IN_ARRAY2[ANY][ANY])\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE IN_ARRAY2[ANY][ANY])\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE IN_ARRAY2[ANY][ANY])\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[2] = { $1_dim0, $1_dim1 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 2) ||\n !require_size(array, size, 2)) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n%typemap(freearg)\n (DATA_TYPE IN_ARRAY2[ANY][ANY])\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DATA_TYPE* IN_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* IN_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* IN_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[2] = { -1, -1 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 2) ||\n !require_size(array, size, 2)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n $3 = (DIM_TYPE) array_size(array,1);\n}\n%typemap(freearg)\n (DATA_TYPE* IN_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_ARRAY2)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_ARRAY2)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_ARRAY2)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[2] = { -1, -1 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 2) ||\n !require_size(array, size, 2)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DIM_TYPE) array_size(array,1);\n $3 = (DATA_TYPE*) array_data(array);\n}\n%typemap(freearg)\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_ARRAY2)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DATA_TYPE* IN_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* IN_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* IN_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[2] = { -1, -1 };\n array = obj_to_array_fortran_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 2) ||\n !require_size(array, size, 2) || !require_fortran(array)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n $3 = (DIM_TYPE) array_size(array,1);\n}\n%typemap(freearg)\n (DATA_TYPE* IN_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_FARRAY2)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_FARRAY2)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_FARRAY2)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[2] = { -1, -1 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 2) ||\n !require_size(array, size, 2) || !require_fortran(array)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DIM_TYPE) array_size(array,1);\n $3 = (DATA_TYPE*) array_data(array);\n}\n%typemap(freearg)\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* IN_FARRAY2)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DATA_TYPE IN_ARRAY3[ANY][ANY][ANY])\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE IN_ARRAY3[ANY][ANY][ANY])\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE IN_ARRAY3[ANY][ANY][ANY])\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[3] = { $1_dim0, $1_dim1, $1_dim2 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 3) ||\n !require_size(array, size, 3)) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n%typemap(freearg)\n (DATA_TYPE IN_ARRAY3[ANY][ANY][ANY])\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DATA_TYPE* IN_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2,\n * DIM_TYPE DIM3)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* IN_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* IN_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[3] = { -1, -1, -1 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 3) ||\n !require_size(array, size, 3)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n $3 = (DIM_TYPE) array_size(array,1);\n $4 = (DIM_TYPE) array_size(array,2);\n}\n%typemap(freearg)\n (DATA_TYPE* IN_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3,\n * DATA_TYPE* IN_ARRAY3)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* IN_ARRAY3)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* IN_ARRAY3)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[3] = { -1, -1, -1 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 3) ||\n !require_size(array, size, 3)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DIM_TYPE) array_size(array,1);\n $3 = (DIM_TYPE) array_size(array,2);\n $4 = (DATA_TYPE*) array_data(array);\n}\n%typemap(freearg)\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* IN_ARRAY3)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DATA_TYPE* IN_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2,\n * DIM_TYPE DIM3)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* IN_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* IN_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[3] = { -1, -1, -1 };\n array = obj_to_array_fortran_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 3) ||\n !require_size(array, size, 3) | !require_fortran(array)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n $3 = (DIM_TYPE) array_size(array,1);\n $4 = (DIM_TYPE) array_size(array,2);\n}\n%typemap(freearg)\n (DATA_TYPE* IN_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3,\n * DATA_TYPE* IN_FARRAY3)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* IN_FARRAY3)\n{\n $1 = is_array($input) || PySequence_Check($input);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* IN_FARRAY3)\n (PyArrayObject* array=NULL, int is_new_object=0)\n{\n npy_intp size[3] = { -1, -1, -1 };\n array = obj_to_array_contiguous_allow_conversion($input, DATA_TYPECODE,\n &is_new_object);\n if (!array || !require_dimensions(array, 3) ||\n !require_size(array, size, 3) || !require_fortran(array)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DIM_TYPE) array_size(array,1);\n $3 = (DIM_TYPE) array_size(array,2);\n $4 = (DATA_TYPE*) array_data(array);\n}\n%typemap(freearg)\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* IN_FARRAY3)\n{\n if (is_new_object$argnum && array$argnum)\n { Py_DECREF(array$argnum); }\n}\n\n/***************************/\n/* In-Place Array Typemaps */\n/***************************/\n\n/* Typemap suite for (DATA_TYPE INPLACE_ARRAY1[ANY])\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE INPLACE_ARRAY1[ANY])\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE INPLACE_ARRAY1[ANY])\n (PyArrayObject* array=NULL)\n{\n npy_intp size[1] = { $1_dim0 };\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,1) || !require_size(array, size, 1) ||\n !require_contiguous(array) || !require_native(array)) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n\n/* Typemap suite for (DATA_TYPE* INPLACE_ARRAY1, DIM_TYPE DIM1)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* INPLACE_ARRAY1, DIM_TYPE DIM1)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* INPLACE_ARRAY1, DIM_TYPE DIM1)\n (PyArrayObject* array=NULL, int i=1)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,1) || !require_contiguous(array)\n || !require_native(array)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = 1;\n for (i=0; i < array_numdims(array); ++i) $2 *= array_size(array,i);\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DATA_TYPE* INPLACE_ARRAY1)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DATA_TYPE* INPLACE_ARRAY1)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DATA_TYPE* INPLACE_ARRAY1)\n (PyArrayObject* array=NULL, int i=0)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,1) || !require_contiguous(array)\n || !require_native(array)) SWIG_fail;\n $1 = 1;\n for (i=0; i < array_numdims(array); ++i) $1 *= array_size(array,i);\n $2 = (DATA_TYPE*) array_data(array);\n}\n\n/* Typemap suite for (DATA_TYPE INPLACE_ARRAY2[ANY][ANY])\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE INPLACE_ARRAY2[ANY][ANY])\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE INPLACE_ARRAY2[ANY][ANY])\n (PyArrayObject* array=NULL)\n{\n npy_intp size[2] = { $1_dim0, $1_dim1 };\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,2) || !require_size(array, size, 2) ||\n !require_contiguous(array) || !require_native(array)) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n\n/* Typemap suite for (DATA_TYPE* INPLACE_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* INPLACE_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* INPLACE_ARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n (PyArrayObject* array=NULL)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,2) || !require_contiguous(array)\n || !require_native(array)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n $3 = (DIM_TYPE) array_size(array,1);\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* INPLACE_ARRAY2)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* INPLACE_ARRAY2)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* INPLACE_ARRAY2)\n (PyArrayObject* array=NULL)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,2) || !require_contiguous(array) ||\n !require_native(array)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DIM_TYPE) array_size(array,1);\n $3 = (DATA_TYPE*) array_data(array);\n}\n\n/* Typemap suite for (DATA_TYPE* INPLACE_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* INPLACE_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* INPLACE_FARRAY2, DIM_TYPE DIM1, DIM_TYPE DIM2)\n (PyArrayObject* array=NULL)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,2) || !require_contiguous(array)\n || !require_native(array) || !require_fortran(array)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n $3 = (DIM_TYPE) array_size(array,1);\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* INPLACE_FARRAY2)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* INPLACE_FARRAY2)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DATA_TYPE* INPLACE_FARRAY2)\n (PyArrayObject* array=NULL)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,2) || !require_contiguous(array) ||\n !require_native(array) || !require_fortran(array)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DIM_TYPE) array_size(array,1);\n $3 = (DATA_TYPE*) array_data(array);\n}\n\n/* Typemap suite for (DATA_TYPE INPLACE_ARRAY3[ANY][ANY][ANY])\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE INPLACE_ARRAY3[ANY][ANY][ANY])\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE INPLACE_ARRAY3[ANY][ANY][ANY])\n (PyArrayObject* array=NULL)\n{\n npy_intp size[3] = { $1_dim0, $1_dim1, $1_dim2 };\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,3) || !require_size(array, size, 3) ||\n !require_contiguous(array) || !require_native(array)) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n\n/* Typemap suite for (DATA_TYPE* INPLACE_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2,\n * DIM_TYPE DIM3)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* INPLACE_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* INPLACE_ARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n (PyArrayObject* array=NULL)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,3) || !require_contiguous(array) ||\n !require_native(array)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n $3 = (DIM_TYPE) array_size(array,1);\n $4 = (DIM_TYPE) array_size(array,2);\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3,\n * DATA_TYPE* INPLACE_ARRAY3)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* INPLACE_ARRAY3)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* INPLACE_ARRAY3)\n (PyArrayObject* array=NULL)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,3) || !require_contiguous(array)\n || !require_native(array)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DIM_TYPE) array_size(array,1);\n $3 = (DIM_TYPE) array_size(array,2);\n $4 = (DATA_TYPE*) array_data(array);\n}\n\n/* Typemap suite for (DATA_TYPE* INPLACE_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2,\n * DIM_TYPE DIM3)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DATA_TYPE* INPLACE_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* INPLACE_FARRAY3, DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3)\n (PyArrayObject* array=NULL)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,3) || !require_contiguous(array) ||\n !require_native(array) || !require_fortran(array)) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n $2 = (DIM_TYPE) array_size(array,0);\n $3 = (DIM_TYPE) array_size(array,1);\n $4 = (DIM_TYPE) array_size(array,2);\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3,\n * DATA_TYPE* INPLACE_FARRAY3)\n */\n%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,\n fragment=\"NumPy_Macros\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* INPLACE_FARRAY3)\n{\n $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),\n DATA_TYPECODE);\n}\n%typemap(in,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DIM_TYPE DIM2, DIM_TYPE DIM3, DATA_TYPE* INPLACE_FARRAY3)\n (PyArrayObject* array=NULL)\n{\n array = obj_to_array_no_conversion($input, DATA_TYPECODE);\n if (!array || !require_dimensions(array,3) || !require_contiguous(array)\n || !require_native(array) || !require_fortran(array)) SWIG_fail;\n $1 = (DIM_TYPE) array_size(array,0);\n $2 = (DIM_TYPE) array_size(array,1);\n $3 = (DIM_TYPE) array_size(array,2);\n $4 = (DATA_TYPE*) array_data(array);\n}\n\n/*************************/\n/* Argout Array Typemaps */\n/*************************/\n\n/* Typemap suite for (DATA_TYPE ARGOUT_ARRAY1[ANY])\n */\n%typemap(in,numinputs=0,\n fragment=\"NumPy_Backward_Compatibility,NumPy_Macros\")\n (DATA_TYPE ARGOUT_ARRAY1[ANY])\n (PyObject * array = NULL)\n{\n npy_intp dims[1] = { $1_dim0 };\n array = PyArray_SimpleNew(1, dims, DATA_TYPECODE);\n if (!array) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n%typemap(argout)\n (DATA_TYPE ARGOUT_ARRAY1[ANY])\n{\n $result = SWIG_Python_AppendOutput($result,array$argnum);\n}\n\n/* Typemap suite for (DATA_TYPE* ARGOUT_ARRAY1, DIM_TYPE DIM1)\n */\n%typemap(in,numinputs=1,\n fragment=\"NumPy_Fragments\")\n (DATA_TYPE* ARGOUT_ARRAY1, DIM_TYPE DIM1)\n (PyObject * array = NULL)\n{\n npy_intp dims[1];\n if (!PyInt_Check($input))\n {\n const char* typestring = pytype_string($input);\n PyErr_Format(PyExc_TypeError,\n \"Int dimension expected. '%s' given.\",\n typestring);\n SWIG_fail;\n }\n $2 = (DIM_TYPE) PyInt_AsLong($input);\n dims[0] = (npy_intp) $2;\n array = PyArray_SimpleNew(1, dims, DATA_TYPECODE);\n if (!array) SWIG_fail;\n $1 = (DATA_TYPE*) array_data(array);\n}\n%typemap(argout)\n (DATA_TYPE* ARGOUT_ARRAY1, DIM_TYPE DIM1)\n{\n $result = SWIG_Python_AppendOutput($result,array$argnum);\n}\n\n/* Typemap suite for (DIM_TYPE DIM1, DATA_TYPE* ARGOUT_ARRAY1)\n */\n%typemap(in,numinputs=1,\n fragment=\"NumPy_Fragments\")\n (DIM_TYPE DIM1, DATA_TYPE* ARGOUT_ARRAY1)\n (PyObject * array = NULL)\n{\n npy_intp dims[1];\n if (!PyInt_Check($input))\n {\n const char* typestring = pytype_string($input);\n PyErr_Format(PyExc_TypeError,\n \"Int dimension expected. '%s' given.\",\n typestring);\n SWIG_fail;\n }\n $1 = (DIM_TYPE) PyInt_AsLong($input);\n dims[0] = (npy_intp) $1;\n array = PyArray_SimpleNew(1, dims, DATA_TYPECODE);\n if (!array) SWIG_fail;\n $2 = (DATA_TYPE*) array_data(array);\n}\n%typemap(argout)\n (DIM_TYPE DIM1, DATA_TYPE* ARGOUT_ARRAY1)\n{\n $result = SWIG_Python_AppendOutput($result,array$argnum);\n}\n\n/* Typemap suite for (DATA_TYPE ARGOUT_ARRAY2[ANY][ANY])\n */\n%typemap(in,numinputs=0,\n fragment=\"NumPy_Backward_Compatibility,NumPy_Macros\")\n (DATA_TYPE ARGOUT_ARRAY2[ANY][ANY])\n (PyObject * array = NULL)\n{\n npy_intp dims[2] = { $1_dim0, $1_dim1 };\n array = PyArray_SimpleNew(2, dims, DATA_TYPECODE);\n if (!array) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n%typemap(argout)\n (DATA_TYPE ARGOUT_ARRAY2[ANY][ANY])\n{\n $result = SWIG_Python_AppendOutput($result,array$argnum);\n}\n\n/* Typemap suite for (DATA_TYPE ARGOUT_ARRAY3[ANY][ANY][ANY])\n */\n%typemap(in,numinputs=0,\n fragment=\"NumPy_Backward_Compatibility,NumPy_Macros\")\n (DATA_TYPE ARGOUT_ARRAY3[ANY][ANY][ANY])\n (PyObject * array = NULL)\n{\n npy_intp dims[3] = { $1_dim0, $1_dim1, $1_dim2 };\n array = PyArray_SimpleNew(3, dims, DATA_TYPECODE);\n if (!array) SWIG_fail;\n $1 = ($1_ltype) array_data(array);\n}\n%typemap(argout)\n (DATA_TYPE ARGOUT_ARRAY3[ANY][ANY][ANY])\n{\n $result = SWIG_Python_AppendOutput($result,array$argnum);\n}\n\n/*****************************/\n/* Argoutview Array Typemaps */\n/*****************************/\n\n/* Typemap suite for (DATA_TYPE** ARGOUTVIEW_ARRAY1, DIM_TYPE* DIM1)\n */\n%typemap(in,numinputs=0)\n (DATA_TYPE** ARGOUTVIEW_ARRAY1, DIM_TYPE* DIM1 )\n (DATA_TYPE* data_temp , DIM_TYPE dim_temp)\n{\n $1 = &data_temp;\n $2 = &dim_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility\")\n (DATA_TYPE** ARGOUTVIEW_ARRAY1, DIM_TYPE* DIM1)\n{\n npy_intp dims[1] = { *$2 };\n PyObject * array = PyArray_SimpleNewFromData(1, dims, DATA_TYPECODE, (void*)(*$1));\n if (!array) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,array);\n}\n\n/* Typemap suite for (DIM_TYPE* DIM1, DATA_TYPE** ARGOUTVIEW_ARRAY1)\n */\n%typemap(in,numinputs=0)\n (DIM_TYPE* DIM1 , DATA_TYPE** ARGOUTVIEW_ARRAY1)\n (DIM_TYPE dim_temp, DATA_TYPE* data_temp )\n{\n $1 = &dim_temp;\n $2 = &data_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility\")\n (DIM_TYPE* DIM1, DATA_TYPE** ARGOUTVIEW_ARRAY1)\n{\n npy_intp dims[1] = { *$1 };\n PyObject * array = PyArray_SimpleNewFromData(1, dims, DATA_TYPECODE, (void*)(*$2));\n if (!array) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,array);\n}\n\n/* Typemap suite for (DATA_TYPE** ARGOUTVIEW_ARRAY2, DIM_TYPE* DIM1, DIM_TYPE* DIM2)\n */\n%typemap(in,numinputs=0)\n (DATA_TYPE** ARGOUTVIEW_ARRAY2, DIM_TYPE* DIM1 , DIM_TYPE* DIM2 )\n (DATA_TYPE* data_temp , DIM_TYPE dim1_temp, DIM_TYPE dim2_temp)\n{\n $1 = &data_temp;\n $2 = &dim1_temp;\n $3 = &dim2_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility\")\n (DATA_TYPE** ARGOUTVIEW_ARRAY2, DIM_TYPE* DIM1, DIM_TYPE* DIM2)\n{\n npy_intp dims[2] = { *$2, *$3 };\n PyObject * array = PyArray_SimpleNewFromData(2, dims, DATA_TYPECODE, (void*)(*$1));\n if (!array) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,array);\n}\n\n/* Typemap suite for (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DATA_TYPE** ARGOUTVIEW_ARRAY2)\n */\n%typemap(in,numinputs=0)\n (DIM_TYPE* DIM1 , DIM_TYPE* DIM2 , DATA_TYPE** ARGOUTVIEW_ARRAY2)\n (DIM_TYPE dim1_temp, DIM_TYPE dim2_temp, DATA_TYPE* data_temp )\n{\n $1 = &dim1_temp;\n $2 = &dim2_temp;\n $3 = &data_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility\")\n (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DATA_TYPE** ARGOUTVIEW_ARRAY2)\n{\n npy_intp dims[2] = { *$1, *$2 };\n PyObject * array = PyArray_SimpleNewFromData(2, dims, DATA_TYPECODE, (void*)(*$3));\n if (!array) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,array);\n}\n\n/* Typemap suite for (DATA_TYPE** ARGOUTVIEW_FARRAY2, DIM_TYPE* DIM1, DIM_TYPE* DIM2)\n */\n%typemap(in,numinputs=0)\n (DATA_TYPE** ARGOUTVIEW_FARRAY2, DIM_TYPE* DIM1 , DIM_TYPE* DIM2 )\n (DATA_TYPE* data_temp , DIM_TYPE dim1_temp, DIM_TYPE dim2_temp)\n{\n $1 = &data_temp;\n $2 = &dim1_temp;\n $3 = &dim2_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility,NumPy_Array_Requirements\")\n (DATA_TYPE** ARGOUTVIEW_FARRAY2, DIM_TYPE* DIM1, DIM_TYPE* DIM2)\n{\n npy_intp dims[2] = { *$2, *$3 };\n PyObject * obj = PyArray_SimpleNewFromData(2, dims, DATA_TYPECODE, (void*)(*$1));\n PyArrayObject * array = (PyArrayObject*) obj;\n if (!array || !require_fortran(array)) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,obj);\n}\n\n/* Typemap suite for (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DATA_TYPE** ARGOUTVIEW_FARRAY2)\n */\n%typemap(in,numinputs=0)\n (DIM_TYPE* DIM1 , DIM_TYPE* DIM2 , DATA_TYPE** ARGOUTVIEW_FARRAY2)\n (DIM_TYPE dim1_temp, DIM_TYPE dim2_temp, DATA_TYPE* data_temp )\n{\n $1 = &dim1_temp;\n $2 = &dim2_temp;\n $3 = &data_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility,NumPy_Array_Requirements\")\n (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DATA_TYPE** ARGOUTVIEW_FARRAY2)\n{\n npy_intp dims[2] = { *$1, *$2 };\n PyObject * obj = PyArray_SimpleNewFromData(2, dims, DATA_TYPECODE, (void*)(*$3));\n PyArrayObject * array = (PyArrayObject*) obj;\n if (!array || !require_fortran(array)) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,obj);\n}\n\n/* Typemap suite for (DATA_TYPE** ARGOUTVIEW_ARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2,\n DIM_TYPE* DIM3)\n */\n%typemap(in,numinputs=0)\n (DATA_TYPE** ARGOUTVIEW_ARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3)\n (DATA_TYPE* data_temp, DIM_TYPE dim1_temp, DIM_TYPE dim2_temp, DIM_TYPE dim3_temp)\n{\n $1 = &data_temp;\n $2 = &dim1_temp;\n $3 = &dim2_temp;\n $4 = &dim3_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility\")\n (DATA_TYPE** ARGOUTVIEW_ARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3)\n{\n npy_intp dims[3] = { *$2, *$3, *$4 };\n PyObject * array = PyArray_SimpleNewFromData(3, dims, DATA_TYPECODE, (void*)(*$1));\n if (!array) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,array);\n}\n\n/* Typemap suite for (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3,\n DATA_TYPE** ARGOUTVIEW_ARRAY3)\n */\n%typemap(in,numinputs=0)\n (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DATA_TYPE** ARGOUTVIEW_ARRAY3)\n (DIM_TYPE dim1_temp, DIM_TYPE dim2_temp, DIM_TYPE dim3_temp, DATA_TYPE* data_temp)\n{\n $1 = &dim1_temp;\n $2 = &dim2_temp;\n $3 = &dim3_temp;\n $4 = &data_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility\")\n (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DATA_TYPE** ARGOUTVIEW_ARRAY3)\n{\n npy_intp dims[3] = { *$1, *$2, *$3 };\n PyObject * array = PyArray_SimpleNewFromData(3, dims, DATA_TYPECODE, (void*)(*$3));\n if (!array) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,array);\n}\n\n/* Typemap suite for (DATA_TYPE** ARGOUTVIEW_FARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2,\n DIM_TYPE* DIM3)\n */\n%typemap(in,numinputs=0)\n (DATA_TYPE** ARGOUTVIEW_FARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3)\n (DATA_TYPE* data_temp, DIM_TYPE dim1_temp, DIM_TYPE dim2_temp, DIM_TYPE dim3_temp)\n{\n $1 = &data_temp;\n $2 = &dim1_temp;\n $3 = &dim2_temp;\n $4 = &dim3_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility,NumPy_Array_Requirements\")\n (DATA_TYPE** ARGOUTVIEW_FARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3)\n{\n npy_intp dims[3] = { *$2, *$3, *$4 };\n PyObject * obj = PyArray_SimpleNewFromData(3, dims, DATA_TYPECODE, (void*)(*$1));\n PyArrayObject * array = (PyArrayObject*) obj;\n if (!array || require_fortran(array)) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,obj);\n}\n\n/* Typemap suite for (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3,\n DATA_TYPE** ARGOUTVIEW_FARRAY3)\n */\n%typemap(in,numinputs=0)\n (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DATA_TYPE** ARGOUTVIEW_FARRAY3)\n (DIM_TYPE dim1_temp, DIM_TYPE dim2_temp, DIM_TYPE dim3_temp, DATA_TYPE* data_temp)\n{\n $1 = &dim1_temp;\n $2 = &dim2_temp;\n $3 = &dim3_temp;\n $4 = &data_temp;\n}\n%typemap(argout,\n fragment=\"NumPy_Backward_Compatibility,NumPy_Array_Requirements\")\n (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DATA_TYPE** ARGOUTVIEW_FARRAY3)\n{\n npy_intp dims[3] = { *$1, *$2, *$3 };\n PyObject * obj = PyArray_SimpleNewFromData(3, dims, DATA_TYPECODE, (void*)(*$3));\n PyArrayObject * array = (PyArrayObject*) obj;\n if (!array || require_fortran(array)) SWIG_fail;\n $result = SWIG_Python_AppendOutput($result,obj);\n}\n\n%enddef /* %numpy_typemaps() macro */\n/* *************************************************************** */\n\n/* Concrete instances of the %numpy_typemaps() macro: Each invocation\n * below applies all of the typemaps above to the specified data type.\n */\n%numpy_typemaps(signed char , NPY_BYTE , int)\n%numpy_typemaps(unsigned char , NPY_UBYTE , int)\n%numpy_typemaps(short , NPY_SHORT , int)\n%numpy_typemaps(unsigned short , NPY_USHORT , int)\n%numpy_typemaps(int , NPY_INT , int)\n%numpy_typemaps(unsigned int , NPY_UINT , int)\n%numpy_typemaps(long , NPY_LONG , int)\n%numpy_typemaps(unsigned long , NPY_ULONG , int)\n%numpy_typemaps(long long , NPY_LONGLONG , int)\n%numpy_typemaps(unsigned long long, NPY_ULONGLONG, int)\n%numpy_typemaps(float , NPY_FLOAT , int)\n%numpy_typemaps(double , NPY_DOUBLE , int)\n\n/* ***************************************************************\n * The follow macro expansion does not work, because C++ bool is 4\n * bytes and NPY_BOOL is 1 byte\n *\n * %numpy_typemaps(bool, NPY_BOOL, int)\n */\n\n/* ***************************************************************\n * On my Mac, I get the following warning for this macro expansion:\n * 'swig/python detected a memory leak of type 'long double *', no destructor found.'\n *\n * %numpy_typemaps(long double, NPY_LONGDOUBLE, int)\n */\n\n/* ***************************************************************\n * Swig complains about a syntax error for the following macro\n * expansions:\n *\n * %numpy_typemaps(complex float, NPY_CFLOAT , int)\n *\n * %numpy_typemaps(complex double, NPY_CDOUBLE, int)\n *\n * %numpy_typemaps(complex long double, NPY_CLONGDOUBLE, int)\n */\n\n#endif /* SWIGPYTHON */\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/python/rwt.i", "content": "%pythonbegin %{\nfrom __future__ import division\n%}\n\n%define MODDOCSTRING\n\"The Rice Wavelet Toolbox (RWT) is a collection of functions for 1D and 2D\nwavelet and filter bank design, analysis, and processing.\"\n%enddef\n\n%module(docstring=MODDOCSTRING) rwt\n\n/* The C functions for the transforms are not suitable for direct use from python so let's rename them. */\n\n%rename(_c_dwt) dwt;\n%rename(_c_idwt) idwt;\n%rename(_c_rdwt) rdwt;\n%rename(_c_irdwt) irdwt;\n\n%rename(_find_levels) rwt_find_levels;\n%rename(_check_levels) rwt_check_levels;\n\n%{\n #define SWIG_FILE_WITH_INIT\n #include \"../lib/inc/rwt_transforms.h\"\n #include \"../lib/inc/rwt_init.h\"\n%}\n\n%include \"../lib/inc/rwt_init.h\"\n%include \"numpy.i\"\n\n%init %{\n import_array();\n%}\n\n/* Building on the numpy SWIG macros we make wrapper functions for 1D and 2D for each transform */\n\nvoid _c_dwt_1( double* INPLACE_ARRAY1, int DIM1, double* INPLACE_ARRAY1, int DIM1, int levels, double* INPLACE_ARRAY1, int DIM1);\nvoid _c_dwt_2( double* INPLACE_ARRAY2, int DIM1, int DIM2, double* INPLACE_ARRAY1, int DIM1, int levels, double* INPLACE_ARRAY2, int DIM1, int DIM2);\nvoid _c_idwt_1( double* INPLACE_ARRAY1, int DIM1, double* INPLACE_ARRAY1, int DIM1, int levels, double* INPLACE_ARRAY1, int DIM1);\nvoid _c_idwt_2( double* INPLACE_ARRAY2, int DIM1, int DIM2, double* INPLACE_ARRAY1, int DIM1, int levels, double* INPLACE_ARRAY2, int DIM1, int DIM2);\nvoid _c_rdwt_1( double* INPLACE_ARRAY1, int DIM1, double* INPLACE_ARRAY1, int DIM1, int levels, double* INPLACE_ARRAY1, int DIM1, double* INPLACE_ARRAY1, int DIM1);\nvoid _c_rdwt_2( double* INPLACE_ARRAY2, int DIM1, int DIM2, double* INPLACE_ARRAY1, int DIM1, int levels, double* INPLACE_ARRAY2, int DIM1, int DIM2, double* INPLACE_ARRAY2, int DIM1, int DIM2);\nvoid _c_irdwt_1(double* INPLACE_ARRAY1, int DIM1, double* INPLACE_ARRAY1, int DIM1, int levels, double* INPLACE_ARRAY1, int DIM1, double* INPLACE_ARRAY1, int DIM1);\nvoid _c_irdwt_2(double* INPLACE_ARRAY2, int DIM1, int DIM2, double* INPLACE_ARRAY1, int DIM1, int levels, double* INPLACE_ARRAY2, int DIM1, int DIM2, double* INPLACE_ARRAY2, int DIM1, int DIM2);\n\n%inline %{\n\nvoid _c_dwt_1(double *x, int nrows, double *h, int ncoeff, int levels, double *y, int toss1) {\n dwt(x, nrows, 1, h, ncoeff, levels, y);\n}\n\nvoid _c_idwt_1(double *x, int nrows, double *h, int ncoeff, int levels, double *y, int toss1) {\n idwt(x, nrows, 1, h, ncoeff, levels, y);\n}\n\nvoid _c_rdwt_1(double *x, int nrows, double *h, int ncoeff, int levels, double *yl, int toss1, double *yh, int toss2) {\n rdwt(x, nrows, 1, h, ncoeff, levels, yl, yh);\n}\n\nvoid _c_irdwt_1(double *x, int nrows, double *h, int ncoeff, int levels, double *yl, int toss1, double *yh, int toss2) {\n irdwt(x, nrows, 1, h, ncoeff, levels, yl, yh);\n}\n\nvoid _c_dwt_2(double *x, int nrows, int ncols, double *h, int ncoeff, int levels, double *y, int toss1, int toss2) {\n dwt(x, nrows, ncols, h, ncoeff, levels, y);\n}\n\nvoid _c_idwt_2(double *x, int nrows, int ncols, double *h, int ncoeff, int levels, double *y, int toss1, int toss2) {\n idwt(x, nrows, ncols, h, ncoeff, levels, y);\n}\n\nvoid _c_rdwt_2(double *x, int nrows, int ncols, double *h, int ncoeff, int levels, double *yl, int toss1, int toss2, double *yh, int toss3, int toss4) {\n rdwt(x, nrows, ncols, h, ncoeff, levels, yl, yh);\n}\n\nvoid _c_irdwt_2(double *x, int nrows, int ncols, double *h, int ncoeff, int levels, double *yl, int toss1, int toss2, double *yh, int toss3, int toss4) {\n irdwt(x, nrows, ncols, h, ncoeff, levels, yl, yh);\n}\n\n%}\n\n%pythoncode %{\n\nimport numpy as np\n\ndef _levels(x, L):\n dim = len(x.shape) # Determine the dimensions of our input\n m = x.shape[0]\n if (dim == 2):\n n = x.shape[1]\n else:\n n = 1\n if (L == 0): # If the number of levels was not specified then use the max\n L = _find_levels(m, n)\n _check_levels(L, m, n) # Sanity check the number of levels\n return L\n\ndef dwt(x, h, L = 0):\n \"\"\"\nFunction computes the discrete wavelet transform y for a 1D or 2D input\nsignal x using the scaling filter h.\n\nInput:\n x : finite length 1D or 2D signal (implicitly periodized)\n h : scaling filter\n L : number of levels. In the case of a 1D signal, length(x) must be\n divisible by 2^L; in the case of a 2D signal, the row and the\n column dimension must be divisible by 2^L. If no argument is\n specified, a full DWT is returned for maximal possible L.\n\nOutput:\n y : the wavelet transform of the signal \n (see example to understand the coefficients)\n L : number of decomposition levels\n\n1D Example:\n x = makesig('LinChirp', 8)\n h = daubcqf(4, 'min')[0]\n L = 2\n y,L = dwt(x,h,L)\n\n1D Example's output and explanation:\n\n y = array([1.1097,0.8767,0.8204,-0.5201,-0.0339,0.1001,0.2201,-0.1401])\n L = 2\n\nThe coefficients in output y are arranged as follows\n\n y(0) and y(1) : Scaling coefficients (lowest frequency)\n y(2) and y(3) : Band pass wavelet coefficients\n y(4) to y(7) : Finest scale wavelet coefficients (highest frequency)\n\n2D Example:\n\n load test_image \n h = daubcqf(4,'min')[0]\n L = 1\n y,L = dwt(test_image,h,L)\n\n2D Example's output and explanation:\n\n The coefficients in y are arranged as follows.\n\n .------------------.\n | | |\n | 4 | 2 |\n | | |\n | L,L | H,L |\n | | |\n --------------------\n | | |\n | 3 | 1 |\n | | |\n | L,H | H,H |\n | | |\n `------------------'\n \n where \n 1 : High pass vertically and high pass horizontally\n 2 : Low pass vertically and high pass horizontally\n 3 : High pass vertically and low pass horizontally\n 4 : Low pass vertically and Low pass horizontally \n (scaling coefficients)\n \"\"\"\n if (x.dtype != 'float'):\n x = x * 1.0\n L = _levels(x, L)\n y = np.ascontiguousarray(np.zeros(x.shape))\n dim = len(x.shape)\n x = np.ascontiguousarray(x)\n if (dim == 1):\n _rwt._c_dwt_1(x, h, L, y)\n if (dim == 2):\n _rwt._c_dwt_2(x, h, L, y)\n return y, L\n\ndef idwt(y, h, L = 0):\n \"\"\"\nFunction computes the inverse discrete wavelet transform x for a 1D or\n2D input signal y using the scaling filter h.\n\nInput:\n y : finite length 1D or 2D input signal (implicitly periodized)\n (see function mdwt to find the structure of y)\n h : scaling filter\n L : number of levels. In the case of a 1D signal, length(x) must be\n divisible by 2^L; in the case of a 2D signal, the row and the\n column dimension must be divisible by 2^L. If no argument is\n specified, a full inverse DWT is returned for maximal possible\n L.\n\nOutput:\n x : periodic reconstructed signal\n L : number of decomposition levels\n\n1D Example:\n xin = makesig('LinChirp', 8)\n h = daubcqf(4, 'min')[0]\n L = 1\n y, L = mdwt(xin, h, L)\n x, L = midwt(y, h, L)\n\n1D Example's output:\n\n x = array([0.0491,0.1951,0.4276,0.7071,0.9415,0.9808,0.6716,0.0000])\n L = 1\n \"\"\"\n if (y.dtype != 'float'):\n y = y * 1.0\n L = _levels(y, L)\n x = np.ascontiguousarray(np.zeros(y.shape))\n y = np.ascontiguousarray(y)\n dim = len(x.shape)\n if (dim == 1):\n _rwt._c_idwt_1(x, h, L, y)\n if (dim == 2):\n _rwt._c_idwt_2(x, h, L, y)\n return x, L\n\ndef rdwt(x, h, L = 0):\n \"\"\"\nFunction computes the redundant discrete wavelet transform y\nfor a 1D or 2D input signal. (Redundant means here that the\nsub-sampling after each stage is omitted.) yl contains the\nlowpass and yh the highpass components. In the case of a 2D\nsignal, the ordering in yh is \n[lh hl hh lh hl ... ] (first letter refers to row, second to\ncolumn filtering). \n\nInput:\n x : finite length 1D or 2D signal (implicitly periodized)\n h : scaling filter\n L : number of levels. In the case of a 1D \n length(x) must be divisible by 2^L;\n in the case of a 2D signal, the row and the\n column dimension must be divisible by 2^L.\n If no argument is\n specified, a full DWT is returned for maximal possible L.\n\nOutput:\n yl : lowpass component\n yh : highpass components\n L : number of levels\n\nExample:\n x = makesig('Leopold', 8)\n h = daubcqf(4, 'min')[0]\n L = 1\n yl, yh, L = mrdwt(x,h,L)\n\nExample's output:\n yl = 0.8365 0.4830 0 0 0 0 -0.1294 0.2241\n yh = -0.2241 -0.1294 0 0 0 0 -0.4830 0.8365\n L = 1\n \"\"\"\n if (x.dtype != 'float'):\n x = x * 1.0\n L = _levels(x, L)\n yl = np.ascontiguousarray(np.zeros(x.shape))\n dim = len(x.shape)\n x = np.ascontiguousarray(x)\n if (dim == 1):\n yh = np.ascontiguousarray(np.zeros(x.shape[0] * L))\n _rwt._c_rdwt_1(x, h, L, yl, yh)\n if (dim == 2):\n yh = np.ascontiguousarray(np.zeros((x.shape[0], x.shape[1] * L * 3)))\n _rwt._c_rdwt_2(x, h, L, yl, yh)\n return yl, yh, L\n\ndef irdwt(yl, yh, h, L = 0):\n \"\"\"\nFunction computes the inverse redundant discrete wavelet\ntransform x for a 1D or 2D input signal. (Redundant means here\nthat the sub-sampling after each stage of the forward transform\nhas been omitted.) yl contains the lowpass and yl the highpass\ncomponents as computed, e.g., by mrdwt. In the case of a 2D\nsignal, the ordering in\nyh is [lh hl hh lh hl ... ] (first letter refers to row, second\nto column filtering). \n\nInput:\n yl : lowpass component\n yh : highpass components\n h : scaling filter\n L : number of levels. In the case of a 1D signal, \n length(yl) must be divisible by 2^L;\n in the case of a 2D signal, the row and\n the column dimension must be divisible by 2^L.\n\nOutput:\n x : finite length 1D or 2D signal\n L : number of levels\n\nExample:\n xin = makesig('Leopold', 8)\n h = daubcqf(4, 'min')[0]\n L = 1\n yl, yh, L = mrdwt(xin, h, L)\n x, L = mirdwt(yl, yh, h, L)\n\nExample Output:\n x = array([0.0000,1.0000,0.0000,-0.0000,0,0,0,-0.0000])\n L = 1\n \"\"\"\n if (yl.dtype != 'float'):\n yl = yl * 1.0\n if (yh.dtype != 'float'):\n yh = yh * 1.0\n L = _levels(yl, L)\n x = np.ascontiguousarray(np.zeros(yl.shape))\n yl = np.ascontiguousarray(yl)\n yh = np.ascontiguousarray(yh)\n dim = len(x.shape)\n if (dim == 1):\n _rwt._c_irdwt_1(x, h, L, yl, yh)\n if (dim == 2):\n _rwt._c_irdwt_2(x, h, L, yl, yh)\n return x, L\n\ndef daubcqf(n, dtype = 'min'):\n \"\"\"\nFunction computes the Daubechies' scaling and wavelet filters\n(normalized to sqrt(2)).\n\nInput: \n n : Length of filter (must be even)\n dtype : Optional parameter that distinguishes the minimum phase,\n maximum phase and mid-phase solutions ('min', 'max', or\n 'mid'). If no argument is specified, the minimum phase\n solution is used.\n\nOutput: \n h_0 : Minimal phase Daubechies' scaling filter \n h_1 : Minimal phase Daubechies' wavelet filter \n\nExample:\n n = 4\n dtype = 'min'\n h_0, h_1 = daubcqf(n, dtype)\n\nExample Result:\n h_0 = array([0.4830, 0.8365, 0.2241, -0.1294])\n h_1 = array([0.1294, 0.2241, -0.8365, 0.4830])\n\nReference: \\\"Orthonormal Bases of Compactly Supported Wavelets\\\",\n CPAM, Oct.89 \n \"\"\"\n if (n % 2 != 0):\n raise Exception(\"No Daubechies filter exists for ODD length\")\n k = n // 2\n a = p = q = 1\n h_0 = np.array([1, 1])\n for j in range(1, k):\n a = -a * 0.25 * (j + k - 1) / j\n h_0 = np.hstack((0, h_0)) + np.hstack((h_0, 0))\n p = np.hstack((0, -p)) + np.hstack((p, 0))\n p = np.hstack((0, -p)) + np.hstack((p, 0))\n q = np.hstack((0, q, 0)) + a*p\n q = np.sort(np.roots(q))\n qt = q[0:k-1]\n if (dtype == 'mid'):\n if (k % 2 == 1):\n qt = np.hstack((q[0:n-2:4], q[1:n-2:4]))\n else:\n qt = np.hstack((q[0], q[3:k-1:4], q[4:k-1:4], q[n-4:k:-4],\n q[n-5:k:-4]))\n h_0 = np.convolve(h_0, np.real(np.poly(qt)))\n h_0 = np.sqrt(2)*h_0 / sum(h_0)\n if (dtype == 'max'):\n h_0 = np.flipud(h_0)\n if (np.abs(sum(np.power(h_0, 2))) -1 > 1e-4):\n raise Exception(\"Numerically unstable for this value of n\")\n h_1 = np.copy(np.flipud(h_0))\n h_1[0:n-1:2] = -h_1[0:n-1:2]\n return h_0, h_1\n\ndef hard_th(y, thld):\n \"\"\"\nHARDTH hard thresholds the input signal y with the threshold value\nthld.\n\nInput: \n y : 1D or 2D signal to be thresholded\n thld : threshold value\n\nOutput: \n x : Hard thresholded output (x = (abs(y)>thld) * y)\n\nExample:\n y = makesig('WernerSorrows', 8)\n thld = 1\n x = HardTh(y, thld)\n\nExample Output:\n x = array([1.5545, 5.3175, 0, 1.6956, -1.2678, 0, 1.7332, 0])\n \"\"\"\n return (np.abs(y) > thld) * y\n\ndef soft_th(y, thld):\n \"\"\"\nSoft thresholds the input signal y with the threshold value thld.\n\nInput: \n y : 1D or 2D signal to be thresholded\n thld : Threshold value\n\nOutput: \n x : Soft thresholded output (sign(y) * (x >= thld) * (x - thld))\n\nExample:\n y = makesig('Doppler', 8)\n thld = 0.2\n x = soft_th(y, thld)\n\nExample Output:\n x = array([0, 0, 0, -0.0703, 0, 0.2001, 0.0483, 0])\n\nReference: \n \\\"De-noising via Soft-Thresholding\\\" Tech. Rept. Statistics,\n Stanford, 1992. D.L. Donoho.\n \"\"\"\n x = np.abs(y)\n return np.sign(y) * (x >= thld) * (x - thld)\n\ndef makesig(signame, n = 512):\n \"\"\"\nCreates artificial test signal identical to the standard test \nsignals proposed and used by D. Donoho and I. Johnstone in\nWaveLab (- a matlab toolbox developed by Donoho et al. the statistics\ndepartment at Stanford University).\n\nInput: signame - Name of the desired signal\n 'HeaviSine'\n 'Bumps'\n 'Blocks'\n 'Doppler'\n 'Ramp'\n 'Cusp'\n 'Sing'\n 'HiSine'\n 'LoSine'\n 'LinChirp'\n 'TwoChirp'\n 'QuadChirp'\n 'MishMash'\n 'WernerSorrows' (Heisenberg)\n 'Leopold' (Kronecker)\n n - Length in samples of the desired signal (Default 512)\n\nOutput: x - resulting test signal\n\nReferences:\n WaveLab can be accessed at\n www_url: http://playfair.stanford.edu/~wavelab/\n Also see various articles by D.L. Donoho et al. at\n web_url: http://playfair.stanford.edu/\n \"\"\"\n t = np.array(range(1, n + 1)) / float(n)\n if (signame == 'HeaviSine'):\n y = 4 * np.sin(4 * np.pi * t)\n return y - np.sign(t - .3) - np.sign(.72 - t)\n if (signame == 'Bumps'):\n pos = np.array([.1, .13, .15, .23, .25, .40, .44, .65, .76, .78, .81])\n hgt = np.array([4, 5, 3, 4, 5, 4.2, 2.1, 4.3, 3.1, 5.1, 4.2])\n wth = np.array(\n [.005, .005, .006, .01, .01, .03, .01, .01, .005, .008, .005])\n y = np.zeros(n)\n for j in range(0, pos.size):\n y = y + hgt[j] / pow((1 + np.abs((t - pos[j]) / wth[j])), 4)\n return y\n if (signame == 'Blocks'):\n pos = np.array([.1, .13, .15, .23, .25, .40, .44, .65, .76, .78, .81])\n hgt = np.array([4, (-5), 3, (-4), 5, (-4.2), 2.1, 4.3, (-3.1),\n 2.1, (-4.2)])\n y = np.zeros(n)\n for j in range(0, pos.size):\n y = y + (1 + np.sign(t - pos[j])) * (hgt[j]/2)\n return y\n if (signame == 'Doppler'):\n return np.sqrt(t * (1-t)) * np.sin((2 * np.pi * 1.05) / (t+.05))\n if (signame == 'Ramp'):\n return t - (t >= .37)\n if (signame == 'Cusp'):\n return np.sqrt(np.abs(t - .37))\n if (signame == 'Sing'):\n k = np.floor(n * .37)\n return 1 / np.abs(t - (k + .5)/n)\n if (signame == 'HiSine'):\n return np.sin(np.pi * (n * .6902) * t)\n if (signame == 'LoSine'):\n return np.sin(np.pi * (n * .3333) * t)\n if (signame == 'LinChirp'):\n return np.sin(np.pi * t * ((n * .125) * t))\n if (signame == 'TwoChirp'):\n return np.sin(np.pi * t * (n * t)) + np.sin((np.pi / 3) * t * (n * t))\n if (signame == 'QuadChirp'):\n return np.sin((np.pi/3) * t * (n * pow(t,2)))\n if (signame == 'MishMash'):\n y = np.sin((np.pi/3) * t * (n * pow(t,2)))\n y = y + np.sin(np.pi * (n * .6902) * t)\n return y + np.sin(np.pi * t * (n * .125 * t))\n if (signame == 'WernerSorrows'):\n y = np.sin(np.pi * t * (n/2 * pow(t, 2)))\n y = y + np.sin(np.pi * (n * .6902) * t)\n y = y + np.sin(np.pi * t * (n * t))\n pos = np.array([.1, .13, .15, .23, .25, .40, .44, .65, .76, .78, .81])\n hgt = np.array([4, 5, 3, 4, 5, 4.2, 2.1, 4.3, 3.1, 5.1, 4.2])\n wth = np.array(\n [.005, .005, .006, .01, .01, .03, .01, .01, .005, .008, .005])\n for j in range(0, pos.size):\n y = y + hgt[j] / pow((1 + np.abs((t - pos[j]) / wth[j])), 4)\n return y\n if (signame == 'Leopold'):\n return (t == np.floor(.37 * n)/n) * 1.0\n\ndef denoise(x, h, denoise_type = 0, option = None):\n \"\"\"\nDENOISE is a generic routine for wavelet based denoising.\nThe routine will denoise the signal x using the 2-band wavelet\nsystem described by the filter h using either the traditional \ndiscrete wavelet transform (DWT) or the linear shift invariant \ndiscrete wavelet transform (also known as the undecimated DWT\n(UDWT)). \n\nInput: \n x : 1D or 2D signal to be denoised\n h : Scaling filter to be applied\n denoise_type : Type of transform (Default: type = 0)\n 0 --> Discrete wavelet transform (DWT)\n 1 --> Undecimated DWT (UDWT)\n option : Default settings is marked with '*':\n *type = 0 --> option = [0 3.0 0 0 0 0]\n type = 1 --> option = [0 3.6 0 1 0 0]\n option(1) : Whether to threshold low-pass part\n 0 --> Don't threshold low pass component \n 1 --> Threshold low pass component\n option(2) : Threshold multiplier, c. The threshold is\n computed as: \n thld = c*MAD(noise_estimate)). \n The default values are:\n c = 3.0 for the DWT based denoising\n c = 3.6 for the UDWT based denoising\n option(3) : Type of variance estimator\n 0 --> MAD (mean absolute deviation)\n 1 --> STD (classical numerical std estimate)\n option(4) : Type of thresholding\n 2 --> Soft thresholding\n 1 --> Hard thresholding\n option(5) : Number of levels, L, in wavelet decomposition. By\n setting this to the default value '0' a maximal\n decomposition is used.\n option(6) : Actual threshold to use (setting this to\n anything but 0 will mean that option(3)\n is ignored)\n\nOutput: \n xd : Estimate of noise free signal \n xn : The estimated noise signal (x-xd)\n option : A vector of actual parameters used by the\n routine. The vector is configured the same way as\n the input option vector with one added element\n option(7) = type.\n\nExample 1: \n from numpy.random import randn\n N = 16\n h = daubcqf(6)[0]\n s = makesig('Doppler', N)\n n = randn(1,N)\n x = s + n/10 # (approximately 10dB SNR)\n %Denoise x with the default method based on the DWT\n xd, xn, opt1 = denoise(x,h)\n %Denoise x using the undecimated (LSI) wavelet transform\n yd, yn, opt2 = denoise(x,h,1)\n\nExample 2: (on an image) \n from scipy.io import loadmat\n from numpy.random import random_sample\n lena = loadmat('../tests/lena512.mat')['lena512']\n h = daubcqf(6)[0]\n noisyLena = lena + 25 * random_sample(lena.shape)\n denoisedLena, xn, opt1 = denoise(noisyLena, h)\n \"\"\"\n if (option is None and denoise_type == 0):\n option = [0, 3.0, 0, 2, 0, 0]\n if (option is None and denoise_type == 1):\n option = [0, 3.6, 0, 1, 0, 0]\n if (not isinstance(option, list)):\n option = list(option)\n mx = x.shape[0]\n nx = 1\n if (len(x.shape) > 1):\n nx = x.shape[1]\n dim = min(mx, nx)\n n = dim\n if (dim == 1):\n n = max(mx, nx)\n if (option[4] == 0):\n L = np.int(np.floor(np.log2(n)))\n else:\n L = option[4]\n if (denoise_type == 0):\n xd = dwt(x, h, L)[0]\n if (option[5] == 0):\n if (nx > 1):\n tmp = xd[mx // 2:mx, nx // 2:nx]\n else:\n tmp = xd[mx // 2:mx]\n if (option[2] == 0):\n thld = option[1] * np.median(np.abs(tmp)) / .67\n elif (option[2] == 1):\n thld = option[1] * np.std(tmp, ddof=1)\n else:\n thld = option[5]\n if (dim == 1):\n ix = np.array(range(0, (n // (np.power(2, L)))))\n if (ix.size == 1):\n ix = ix[0]\n ykeep = xd[ix]\n else:\n ix = np.array(range(0, (mx // (np.power(2, L)))))\n jx = np.array(range(0, (nx // (np.power(2, L)))))\n if (ix.size == 1):\n ix = ix[0]\n if (jx.size == 1):\n jx = jx[0]\n ykeep = xd[ix, jx]\n if (option[3] == 2):\n xd = soft_th(xd, thld)\n elif (option[3] == 1):\n xd = hard_th(xd, thld)\n if (option[0] == 0):\n if (dim == 1):\n xd[ix] = ykeep\n else:\n xd[ix, jx] = ykeep\n xd = idwt(xd, h, L)[0]\n elif (denoise_type == 1):\n (xl, xh, L) = rdwt(x, h, L)\n easter_egg = 23\n if (dim == 1):\n c_offset = 0\n else:\n c_offset = 2 * nx\n if (option[5] == 0):\n if (nx > 1):\n tmp = xh[:,c_offset:c_offset+mx] \n else:\n tmp = xh[c_offset:c_offset+mx:1] \n if (option[2] == 0):\n thld = option[1] * np.median(np.abs(tmp)) / .67\n elif (option[2] == 1):\n thld = option[1] * np.std(tmp, ddof=1)\n else:\n thld = option[5]\n if (option[3] == 2):\n xh = soft_th(xh, thld)\n if (option[0] == 1):\n xl = soft_th(xl, thld)\n elif (option[3] == 1):\n xh = hard_th(xh, thld)\n if (option[0] == 1):\n xl = hard_th(xl, thld)\n xd = irdwt(xl, xh, h, L)[0]\n option[5] = (thld)\n option.append(denoise_type)\n xn = x - xd\n return xd, xn, option\n\n%}\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/python/test_rwt.py", "content": "#!/usr/bin/python\n\nimport unittest\nfrom numpy import *\nfrom scipy.io import loadmat\nfrom rwt import *\n\nclass TestRWT(unittest.TestCase):\n\n def setUp(self):\n pass\n\n def test_dwt(self):\n x = makesig('LinChirp', 8)\n h = daubcqf(4, 'min')[0]\n L = 2\n y, L = dwt(x, h, L)\n y_corr = array([1.109692262737501,0.876661822959323,0.820391852106669,-0.520074093642583,-0.033927668247206,0.100110695461285,0.220088240246095,-0.140081604397608])\n self.assertTrue(allclose(y, y_corr, 0.0001))\n\n def test_dwt_2d(self):\n x = array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16.0]])\n h = daubcqf(4)[0]\n L = 2\n y, L = dwt(x, h, L)\n y_corr = array([[34.0000, -3.4641, 0.0000, -2.0000], [-13.8564, 0.0000, 0.0000, -2.0000], [-0.0000, 0.0000, -0.0000, -0.0000], [-8.0000, -8.0000, 0.0000, -0.0000]])\n self.assertTrue(allclose(y, y_corr, 0.0000005))\n\n def test_idwt(self):\n x = makesig('LinChirp', 8)\n h = daubcqf(4, 'min')[0]\n L = 2\n y, L = dwt(x, h, L)\n x_new, L = idwt(y, h, L)\n self.assertTrue(allclose(x, x_new, 0.0005))\n\n def test_idwt_2d(self):\n x = loadmat('../tests/lena512.mat')['lena512'] * 1.0\n h = daubcqf(6)[0]\n L = 9\n y, L = dwt(x, h, L)\n x_new, L = idwt(y, h, L)\n self.assertTrue(allclose(x, x_new, 0.0005))\n\n def test_rdwt(self):\n x = makesig('Leopold', 8)\n h = daubcqf(4, 'min')[0]\n L = 1\n (yl, yh, L) = rdwt(x, h, L)\n yl_corr = [0.8365, 0.4830, 0, 0, 0, 0, -0.1294, 0.2241]\n yh_corr = [-0.2241, -0.1294, 0, 0, 0, 0, -0.4830, 0.8365]\n L_corr = 1\n self.assertTrue(allclose(yl, yl_corr, 0.0005))\n self.assertTrue(allclose(yh, yh_corr, 0.0005))\n self.assertTrue(allclose(L, L_corr, 0.0005))\n\n def test_rdwt_2(self):\n x = array([[1.0,3,5,2],[3,4,8,1],[3,9,2,0],[1,2,3,0]])\n h = daubcqf(4, 'min')[0]\n yl, yh, L = rdwt(x, h, 1)\n yl_corr = array([\n [9.0111, 10.7799, 5.8795, 4.1107],\n [11.1393, 8.7766, 2.5502, 4.9130],\n [6.9465, 5.7578, 1.6630, 2.8517],\n [4.8182, 7.7611, 4.9922, 2.0494]])\n yh_corr = array([\n [4.5724, 0.4285, -1.8828, 2.2611, 4.8714, -3.1026, -1.7978, 0.0290, -2.9620, -1.1818, -1.1295, 5.2733],\n [-2.4441, -2.4318, -1.4465, -1.4587, 1.8861, -4.2488, -1.9776, 4.3403, -0.0233, 0.0356, 0.9498, -0.9620],\n [-1.7488, -0.5870, 0.5592, -0.6026, 1.1663, -2.3550, -1.7398, 2.9285, -0.6965, 1.8583, -0.7120, -0.4498],\n [-0.3795, 2.5903, 2.7700, -0.1998, 4.1516, -1.2087, -1.5601, -1.3828, 3.6818, -0.7120, 0.8917, -3.8615]])\n self.assertTrue(allclose(yl, yl_corr, 0.001))\n self.assertTrue(allclose(yh, yh_corr, 0.001))\n\n def test_rdwt_2L2(self):\n x = array([[1.0,3,5,2],[3,4,8,1],[3,9,2,0],[1,2,3,0]])\n h = daubcqf(4, 'min')[0]\n yl, yh, L = rdwt(x, h, 2)\n yl_corr = array([\n [11.7500, 11.7500, 11.7500, 11.7500],\n [11.7500, 11.7500, 11.7500, 11.7500],\n [11.7500, 11.7500, 11.7500, 11.7500],\n [11.7500, 11.7500, 11.7500, 11.7500]])\n yh_corr = array([\n [4.5724, 0.4285, -1.8828, 2.2611, 4.8714, -3.1026, -1.7978, 0.0290, -2.9620, -1.1818, -1.1295, 5.2733,\n 3.1405, 3.1405, 3.1405, 3.1405, 4.2075, 4.7877, -4.2075, -4.7877, -1.0760, 1.8816, 1.0760, -1.8816],\n [-2.4441, -2.4318, -1.4465, -1.4587, 1.8861, -4.2488, -1.9776, 4.3403, -0.0233, 0.0356, 0.9498, -0.9620,\n 1.9396, 1.9396, 1.9396, 1.9396, 4.2075, 4.7877, -4.2075, -4.7877, 4.3816, -0.9240, -4.3816, 0.9240],\n [-1.7488, -0.5870, 0.5592, -0.6026, 1.1663, -2.3550, -1.7398, 2.9285, -0.6965, 1.8583, -0.7120, -0.4498,\n -3.1405, -3.1405, -3.1405, -3.1405, 4.2075, 4.7877, -4.2075, -4.7877, 1.0760, -1.8816, -1.0760, 1.8816],\n [-0.3795, 2.5903, 2.7700, -0.1998, 4.1516, -1.2087, -1.5601, -1.3828, 3.6818, -0.7120, 0.8917, -3.8615,\n -1.9396, -1.9396, -1.9396, -1.9396, 4.2075, 4.7877, -4.2075, -4.7877, -4.3816, 0.9240, 4.3816, -0.9240]])\n self.assertTrue(allclose(yl, yl_corr, 0.001))\n self.assertTrue(allclose(yh, yh_corr, 0.001))\n\n def test_irdwt(self):\n xin = makesig('Leopold',8)\n h = daubcqf(4, 'min')[0]\n Lin = 1\n (yl, yh, L) = rdwt(xin, h, Lin) \n (x, L) = irdwt(yl, yh, h, L)\n self.assertTrue(allclose(x, xin, 0.0005))\n \n def test_irdwt_2d(self):\n x = loadmat('../tests/lena512.mat')['lena512'] * 1.0\n h = daubcqf(6)[0]\n L = 9 \n yl, yh, L = rdwt(x, h, L)\n x_new, L = irdwt(yl, yh, h, L)\n self.assertTrue(allclose(x, x_new, 0.0005))\n\n def test_makesig_heavisine(self):\n x = makesig('HeaviSine', 8)\n y = array([4.0000, 0.0000, -6.0000, -2.0000, 2.0000, 0.0000, -4.0000, -0.0000])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_bumps(self):\n x = around(makesig('Bumps', 8), 4)\n y = array([0.3206, 5.0527, 0.3727, 0.0129, 0.0295, 0.0489, 0.0004, 0.0000])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_blocks(self):\n x = makesig('Blocks', 8)\n y = array([4.0000, 0.5000, 3.0000, 0.9000, 0.9000, 5.2000, -0.0000, -0.0000])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_doppler(self):\n x = makesig('Doppler', 12)\n y = array([-0.1954, -0.3067, 0.0000, -0.4703, 0.4930, -0.2703, -0.4127, 0.1025, 0.4001, 0.3454, 0.1425, 0])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_ramp(self):\n x = makesig('Ramp', 8)\n y = array([0.1250, 0.2500, -0.6250, -0.5000, -0.3750, -0.2500, -0.1250, 0])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_cusp(self):\n x = makesig('Cusp', 8)\n y = array([0.4950, 0.3464, 0.0707, 0.3606, 0.5050, 0.6164, 0.7106, 0.7937])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_sing(self):\n x = makesig('Sing', 8)\n y = array([5.3333, 16.0000, 16.0000, 5.3333, 3.2000, 2.2857, 1.7778, 1.4545])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_hisine(self):\n x = makesig('HiSine', 8)\n y = array([0.8267, -0.9302, 0.2200, 0.6827, -0.9882, 0.4292, 0.5053, -0.9977])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_losine(self):\n x = makesig('LoSine', 8)\n y = array([0.865973039158459,0.866130104544730,0.000314159260191,-0.865815888304075,-0.866287084447387,-0.000628318489377,0.865658651997088,0.866443978850937])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_linchirp(self):\n x = makesig('LinChirp', 8)\n y = array([0.049067674327418,0.195090322016128,0.427555093430282,0.707106781186547,0.941544065183021,0.980785280403230,0.671558954847019,0.000000000000000])\n self.assertTrue(allclose(x, y, 0.0001))\n \n def test_makesig_twochirp(self):\n x = makesig('TwoChirp', 8)\n y = array([0.5132, 1.5000, 0.5412, 0.8660, -0.5132, 0, 0.5132, 0.8660])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_quadchirp(self):\n x = makesig('QuadChirp', 8)\n y = array([0.016361731626487,0.130526192220052,0.427555093430282,0.866025403784439,0.889516075421856,-0.382683432365090,-0.621660573370077,0.866025403784439])\n self.assertTrue(allclose(x, y, 0.0001))\n \n def test_makesig_mishmash(self):\n x = makesig('MishMash', 8)\n y = array([0.8922, -0.6046, 1.0751, 2.2558, 0.8429, 1.0273, 0.5551, -0.1317])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_wernersorrows(self):\n x = makesig('WernerSorrows', 8)\n y = array([1.5545, 5.3175, 0.8252, 1.6956, -1.2678, 0.6466, 1.7332, -0.9977])\n self.assertTrue(allclose(x, y, 0.0005))\n \n def test_makesig_leopold(self):\n x = makesig('Leopold', 8)\n y = array([0, 1, 0, 0, 0, 0, 0, 0])\n self.assertTrue(allclose(x, y, 0.0005))\n\n def test_denoise_default(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h)\n signal_denoised_corr = array([0.0741827688375062,0.0791701902526268,0.0760842615272340,0.0750476831774179,0.111279774779568,0.163475053283544,-0.0498263815350539,0.0946073088237311,0.135126562486911,-0.0186090620958193,-0.0748812479991294,-0.103470206059426,0.0234254843251780,0.239772540836257,0.0920583398962312,-0.152180640366891,-0.116682073306156,-0.0459389850762785,-0.00245240039778375,0.0755739164104836,0.102548333512214,0.121099911744184,0.177390507921620,0.240386041553093,0.231105933317157,0.198210924493273,0.175672812990725,0.138822049613034,0.127491615387826,0.121409597186325,0.0994935320130783,0.0760019340865427])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n\n def test_denoise_2d(self):\n x = array([[1,2,3,4],[5,6,7,8],[9,10.09,11,12],[13,13.91,15,16]])\n h = daubcqf(4)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(x, h)\n signal_denoised_corr = array([[1.093495801587334,2.052784169768518,3.036985129109070,4.014510779767102],[5.037416383975946,6.006178652683398,6.994963120759174,7.978382656683513],[9.047593546684929,10.003998510025589,10.977825887256145,11.94698494275469],[13.009489364401729,13.937038667522501,14.939852728547271,15.9224996584731398]])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n\n def test_denoise_udwt(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 1)\n signal_denoised_corr = array([0.126244615385152,0.0952319712425300,0.0671343607152503,0.0513902979722585,0.0430402732682634,0.0586932575131794,0.0861069751902698,0.0989949047763016,0.0908418658128637,-0.0141454670119059,-0.144791527437026,-0.0185533166035902,0.278351613782131,0.279033706376659,-0.0205012032054263,-0.212367658407976,-0.241484343697995,-0.248582298831059,-0.213374214781743,-0.101963712141109,0.0454248851310567,0.181104333949749,0.275294407293259,0.309076259882059,0.298600450385073,0.259080737796607,0.211123535801718,0.183021783525739,0.171966340866576,0.171616812586097,0.168720006300193,0.151066428184072])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n\n def test_denoise_udwt_2d(self):\n x = array([[1,2,3,4],[5,6,7,8],[9,10.09,11,12],[13,13.91,15,16]])\n h = daubcqf(4)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(x, h, 1)\n signal_denoised_corr = array([[1.007040488866197,1.993405274521765,3.006268404030089,3.996424654030090],[4.995935171857875,6.002401216530091,7.001252328142127,8.005847881693983],[9.009508189685661,10.059981743374523,11.001190131625481,11.999030274521770],[12.987516149590270,13.944211765573623,14.991289136202310,15.998697189754166]])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_threshold_low(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 0, [1,3.0,0,2,0,0])\n signal_denoised_corr = array([0.0187742354278351,0.0237616568429558,0.0206757281175629,0.0196391497677469,0.0558712413698966,0.108066519873873,-0.105234914944725,0.0391987754140600,0.0797180290772401,-0.0740175955054904,-0.130289781408801,-0.158878739469097,-0.0319830490844931,0.184364007426586,0.0366498064865601,-0.207589173776562,-0.172090606715827,-0.101347518485950,-0.0578609338074549,0.0201653830008125,0.0471398001025425,0.0656913783345127,0.121981974511949,0.184977508143422,0.175697399907486,0.142802391083602,0.120264279581054,0.0834135162033633,0.0720830819781554,0.0660010637766539,0.0440849986034073,0.0205934006768717])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_thresh_multiplier(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 0, [1,3.5,0,2,0,0])\n signal_denoised_corr = array([0.00563527074803461,0.0110853052404048,0.0101590193471916,0.0116789518546074,0.0354625658443208,0.0691904606426981,-0.0647010252187970,0.0393485097012034,0.0302297746478269,-0.0658230296401878,-0.0947938063374137,-0.147943151851009,-0.0355607514547514,0.143027827800490,0.0126752977970079,-0.200577663821584,-0.149059259007655,-0.0564432101940217,-0.0281365070661950,0.0201021371871464,0.0438412772787373,0.0596866399869512,0.0967101937989458,0.136451641917565,0.130716307107088,0.109146914388131,0.0925200849653435,0.0657607417363412,0.0550584910898860,0.0469636231448182,0.0277268486177313,0.00667135407398081])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_std(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 0, [0,3.0,1,2,0,0])\n signal_denoised_corr = array([0.0686926069658060,0.0706216045196474,0.0719769032529757,0.0743568305131058,0.0754251996534692,0.0763549103855611,0.0783972750744446,0.0807092136475563,0.0763109954998047,0.0693017683604205,0.0628697537191382,0.0547492531677562,0.0755519478401559,0.107931256046656,0.0859959791464885,0.0494376118339224,0.0602059364595448,0.0785077229738383,0.0791999606842265,0.0809410605777517,0.0844652184548917,0.0873749084881920,0.0911535278085727,0.0952027332951270,0.0936316016468421,0.0898878427420561,0.0866734185917041,0.0820709685744921,0.0793481432323076,0.0768306965269240,0.0727995727792393,0.0684196591566048])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_hard(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 0, [0,3.0,0,1,0,0])\n signal_denoised_corr = array([0.0977394160103721,0.0994161560983385,0.0832447407807381,0.0666983311697188,0.177420971595413,0.340230583897110,-0.354597069671295,0.0250017872275015,0.394418485343238,-0.0595745304374512,-0.452401570793399,-0.175707560852101,-0.00622320325130765,0.437867065411816,0.187485346584306,-0.241060664687049,-0.306285896120773,-0.373946536466370,-0.246165924475657,0.00210496326791051,0.0528629966064817,0.0967383656953347,0.275410693617439,0.487298926169970,0.454985253718689,0.348603331393631,0.288205743942248,0.186806596496260,0.172147260405660,0.180050851714681,0.142136445826288,0.104484725401481])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_levels(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 0, [0,3.0,0,2,4,0])\n signal_denoised_corr = array([0.164259992817262,0.156379071218712,0.142212685671703,0.125038963573761,0.150297815252073,0.191536767978636,-0.0381639580765735,0.0881092032192094,0.119629284458486,-0.0406090725365491,-0.105645426731493,-0.141820831994602,-0.0280318977202704,0.173171960129832,0.0117537437282443,-0.247115729957293,-0.206759297285911,-0.123147866042363,-0.0685808245422524,0.0255826360141400,0.0635302930397082,0.0930381970490923,0.165728084463140,0.246884147157615,0.246603211345582,0.220210934934003,0.206436991723089,0.177172675548210,0.178948997433275,0.188010177892750,0.179798128181065,0.170937023676945])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_actual_thresh(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 0, [0,3.0,0,2,0,0.5])\n signal_denoised_corr = array([0.0607099183942295,0.0654351521193524,0.0684154759800610,0.0742018934148454,0.0758845005390013,0.0769511530643110,0.0810856606730252,0.0858023375316036,0.0704706443350518,0.0472060906047587,0.0254329679518446,-0.00154590940405266,0.0598455182579352,0.156556707841878,0.0864272987162393,-0.0287835335280487,0.00606017120154721,0.0659592575432934,0.0713958080495586,0.0812891735076492,0.0953701981347179,0.107554576791239,0.123739146895592,0.141180422640726,0.137085044622601,0.124838366760086,0.114852957437233,0.0997294000571788,0.0922174665178409,0.0857758976557685,0.0737052631031342,0.0605470542090229])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n\n def test_denoise_udwt_threshold_low(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 1, [1,3,0,1,0,0])\n signal_denoised_corr = array([0.135039400483741,0.117805175604609,0.0967709584177031,0.0142060292567307,-0.0239840294603812,0.323425861331697,-0.212285200125643,0.166066657685731,0.136653739821785,-0.0361708285655289,-0.244622217319313,-0.0751486112344819,0.279128997196628,0.299915294672821,0.00822389077239383,-0.232180770499244,-0.330137263335199,-0.293955318206172,-0.175538926380835,-0.0733568677543535,0.049241196655251,0.200165899490694,0.304615650610263,0.337325376378116,0.325593984310807,0.282048956150932,0.228861081870546,0.196656880842149,0.180959366486141,0.175210410022406,0.169828050229736,0.155033256209497])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_udwt_thresh_multiplier(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 1, [1,3.5,0,1,0,0])\n signal_denoised_corr = array([0.0479478506866607,0.0160653046305043,-0.012660890293452,-0.0292521383561941,-0.0383355043751224,-0.0239494802109215,0.00200042536526626,0.0135636610003902,0.00399637041195728,-0.100521378500944,-0.229923524965501,-0.102614225576592,0.195850596270724,0.197593413336102,-0.100882406775293,-0.291163630119251,-0.318524834100706,-0.324752887320235,-0.288916218874243,-0.176658530913858,-0.028536592326759,0.108409816572649,0.204063702017061,0.239170248556769,0.230108690684778,0.190119394184444,0.14091827822899,0.11174543739754,0.0991301032767805,0.0977198505254529,0.0937639547688583,0.0745251447941448])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_udwt_std(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 1, [0,3.0,1,1,0,0])\n signal_denoised_corr = array([0.0847626939447046,0.0648669375488877,0.0505127048998841,0.0431477690668965,0.0443458995091662,0.0638361516754724,0.0926698200065443,0.122716357496751,0.135591683864019,0.0377466753027189,-0.0889166586897228,-0.0310700016943258,0.16530654803759,0.237349858169585,0.0577692051497442,-0.137751577705709,-0.18354744395111,-0.188205427540335,-0.157902857480421,-0.055391323576937,0.0791892398460303,0.198068185997372,0.271471422836112,0.282275886815228,0.246689293630916,0.205546705496588,0.16546007731141,0.145130898382968,0.1471329636038,0.142472749823065,0.132163448290946,0.111958195551385])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_udwt_soft(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 1, [0,3.0,0,2,0,0])\n signal_denoised_corr = array([0.086668016749428,0.078090652632278,0.070455842749544,0.062824684205684,0.064249795534642,0.086899924318641,0.053549539548214,0.100644175366308,0.100726560037458,0.051479406046214,-0.011299945211104,0.036115394710961,0.147624998547612,0.159516308766960,0.059119062682569,-0.020817294484415,-0.042170912413038,-0.046825168298822,-0.027179285827824,0.017379645805457,0.071225126011476,0.123532780238470,0.153926034241219,0.160138755049699,0.153562168658336,0.138748019440599,0.123707805352361,0.115223425612607,0.110890877355381,0.107909648973443,0.103630954238181,0.095849084980685])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_udwt_levels(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 1, [0,3.0,0,1,4,0])\n signal_denoised_corr = array([0.137633389000662,0.120676804147327,0.0997827582151432,0.0156985740202669,-0.0251180988153785,0.319788331991522,-0.217919217670089,0.160238201773756,0.131270340429534,-0.0414158027972923,-0.249853610380694,-0.0801267408837784,0.275034335985338,0.296982831400265,0.00620014657281041,-0.234309647934845,-0.33273125185212,-0.296826946748889,-0.178550726178275,-0.0748494125178897,0.0503752660102483,0.203803428830869,0.310249668154709,0.343153832290091,0.330977383703058,0.287293930382695,0.234092474931927,0.201635010491445,0.185054027697432,0.178142873294961,0.171851794429319,0.157162133645098])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n \n def test_denoise_udwt_actual_thresh(self):\n signal = makesig('Doppler', 32)\n noise = array([1.54421189550395,0.0859311331754255,-1.49159031063761,-0.742301837259857,-1.06158173331999,2.35045722400204,-0.615601881466894,0.748076783703985,-0.192418510588264,0.888610425420721,-0.764849236567874,-1.40226896933876,-1.42237592509150,0.488193909859941,-0.177375156618825,-0.196053487807333,1.41931015064255,0.291584373984183,0.197811053464361,1.58769908997406,-0.804465956349547,0.696624415849607,0.835088165072682,-0.243715140377952,0.215670086403744,-1.16584393148205,-1.14795277889859,0.104874716016494,0.722254032225002,2.58549125261624,-0.666890670701386,0.187331024578940])\n with_noise = signal + noise / 10\n h = daubcqf(6)[0]\n signal_denoised, subtracted_noise, actual_options = denoise(with_noise, h, 1, [0,3.0,0,1,0,0.5])\n signal_denoised_corr = array([0.126244615385152,0.09523197124253,0.0671343607152503,0.0513902979722585,0.0430402732682634,0.0586932575131794,0.0861069751902698,0.0989949047763016,0.0908418658128637,-0.0141454670119059,-0.144791527437026,-0.0185533166035902,0.278351613782131,0.279033706376659,-0.0205012032054263,-0.212367658407976,-0.241484343697995,-0.248582298831059,-0.213374214781743,-0.101963712141109,0.0454248851310567,0.181104333949749,0.275294407293258,0.309076259882059,0.298600450385073,0.259080737796607,0.211123535801717,0.183021783525739,0.171966340866576,0.171616812586097,0.168720006300193,0.151066428184072])\n self.assertTrue(allclose(signal_denoised, signal_denoised_corr, 0.01))\n\n def test_daubcqf_min(self):\n (a, b) = daubcqf(4)\n ax = [0.482962913144534,0.836516303737808,0.224143868042013,-0.129409522551260]\n bx = [0.129409522551260,0.224143868042013,-0.836516303737808,0.482962913144534]\n self.assertTrue(allclose(a, ax, 0.000001))\n self.assertTrue(allclose(b, bx, 0.000001))\n \n def test_daubcqf_max(self):\n (a, b) = daubcqf(4, 'max')\n ax = [-0.129409522551260,0.224143868042013,0.836516303737808,0.482962913144534]\n bx = [-0.482962913144534,0.836516303737808,-0.224143868042013,-0.129409522551260]\n self.assertTrue(allclose(a, ax, 0.000001))\n self.assertTrue(allclose(b, bx, 0.000001))\n \n def test_daubcqf_mid_even_k(self):\n (a, b) = daubcqf(4, 'mid')\n ax = [0.482962913144534,0.836516303737808,0.224143868042013,-0.129409522551260]\n bx = [0.129409522551260,0.224143868042013,-0.836516303737808,0.482962913144534]\n self.assertTrue(allclose(a, ax, 0.000001))\n self.assertTrue(allclose(b, bx, 0.000001))\n \n def test_daubcqf_mid_odd_k(self):\n (a, b) = daubcqf(6, 'mid')\n ax = [0.332670552950083,0.806891509311093,0.459877502118491,-0.135011020010255,-0.085441273882027,0.035226291885710]\n bx = [-0.035226291885710,-0.085441273882027,0.135011020010255,0.459877502118491,-0.806891509311093,0.332670552950083]\n self.assertTrue(allclose(a, ax, 0.000001))\n self.assertTrue(allclose(b, bx, 0.000001))\n \nif __name__ == '__main__':\n unittest.main()\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/readme", "content": "test\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/Readme.html", "content": "\n \n \n \n \n \n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/architecture/html/matlab_xunit_architecture.html", "content": "\n\n\n \n \n \n \n MATLAB xUnit Test Framework: Architectural Notes\n \n \n \n \n
\n

MATLAB xUnit Test Framework: Architectural Notes

\n \n

This document summarizes the key classes and design choices for MATLAB xUnit, a MATLAB unit testing framework based on xUnit\n patterns.\n

\n

Note: Testing pattern and smell terminology in this document is drawn from xUnit Test Patterns: Refactoring Test Code, by Gerard Meszaros, Addison-Wesley, 2007.\n

\n \n

Contents

\n \n

TestComponent, TestCase, and TestSuite

\n

\"\"

\n

The abstract TestComponent class defines an object that has a description (a name and a location) and that can be run.\n

\n

A TestCase object is a test component that defines an individual test case that can be run with a pass or fail result.\n

\n

A TestSuite object is a test component that contains a collection of other test components. Note the hierarchical nature of test suites;\n they can contain both individual test case objects as well as other test suites. Running a test suite means invoking the run method on each test component in its collection.\n

\n

TestCase: The Four-Phase Test

\n

The TestCase class provides the standard xUnit Four-Phase Test, using a Fresh Fixture, Implicit Setup, and Implicit Teardown. These all elements can all be seen in the run method of TestCase:\n 

       function did_pass = run(self, monitor)\n           %run Execute the test case\n           %    test_case.run(monitor) calls the TestCase object's setUp()\n           %    method, then the test method, then the tearDown() method.\n           %    observer is a TestRunObserver object.  The testStarted(),\n           %    testFailure(), testError(), and testFinished() methods of\n           %    observer are called at the appropriate times.  monitor is a\n           %    TestRunMonitor object.  Typically it is either a TestRunLogger\n           %    subclass or a CommandWindowTestRunDisplay subclass.\n           %\n           %    test_case.run() automatically uses a\n           %    CommandWindowTestRunDisplay object in order to print test\n           %    suite execution information to the Command Window.
           if nargin < 2\n               monitor = CommandWindowTestRunDisplay();\n           end
           did_pass = true;\n           monitor.testComponentStarted(self);
           try\n               self.setUp();\n               f = str2func(self.MethodName);
               try\n                   % Call the test method.\n                   f(self);\n               catch failureException\n                   monitor.testCaseFailure(self, failureException);\n                   did_pass = false;\n               end
               self.tearDown();
           catch errorException\n               monitor.testCaseError(self, errorException);\n               did_pass = false;\n           end
           monitor.testComponentFinished(self, did_pass);\n       end

Phase 1 sets up the test fixture via the Implicit Setup call, self.setUp(). The base class setUp() method does nothing.\n

\n

Phases 2 and 3 (exercising the system under test and verifying the expected outcome) are handled by the test method, which\n is invoked by f(self).\n

\n

Phase 4 tears down the test fixture via the Implicit Teardown call, self.tearDown(). The base class tearDown() method does nothing.\n

\n

Test failure and test error exceptions are caught and handled by the run() method, so test methods do not need to use try-catch. This facilitates simple, straight-line test-method code.\n

\n

Note: The monitor object will be discussed later.

\n

Test Case Discovery

\n

The static method TestSuite.fromName constructs a test suite based on the name of an M-file. If the M-file defines a TestCase subclass, then fromName inspects the methods of the class and constructs a TestCase object for each method whose name begins with \"[tT]est\". If the M-file does not define a TestCase subclass, then fromName attempts to construct either a simple procedural test case or a set of subfunction-based test cases. (See the next section).\n

\n

The static method TestSuite.fromPwd constructs a test suite by discovering all the test cases in the present working directory. It discovers all TestCase subclasses in the directory. In addition, it constructs test suites from all the procedural M-files in the directory beginning\n with \"[tT]est\".\n

\n

The File System Test Runner, runtests, provides convenient syntaxes for performing test case discovery automatically.\n

\n

FunctionHandleTestCase: For the Procedural World

\n

Most MATLAB users are much more comfortable with procedural programming. An important design goal for MATLAB xUnit is to\n make it as easy as possible for MATLAB users with little object-oriented programming experience to create and run their own\n tests. The FunctionHandleTestCase supplies the plumbing necessary to support procedural test functions:\n

\n

\"\"

\n

Private properties SetupFcn, TestFcn, and TeardownFcn are procedural function handles (similar to function pointers or function references in other languages).\n

\n

runTestCase() is the test method used for constructing a TestCase object.\n

\n

Managing test fixtures requires special consideration, because procedural function handles don't have access to object instance\n data in order to access a test fixture.\n

\n

The overridden setUp() method looks at the number of outputs of the function handle SetupFcn. If it has an output argument, then the argument is saved in the private TestData property, and TestData is then passed to both TestFcn and TeardownFcn for their use.\n

\n

Writing Procedural Test Cases

\n

Procedural test cases can be written in two ways:

\n
\n
    \n
  • A simple M-file function that is treated as a single test case
    • \n
  • An M-file containing multiple subfunctions that are each treated as a test case.
    • \n
\n
\n

In either case, the test case is considered to pass if it executes without error.

\n

Writing one test case per file is not ideal; it would lead to either zillions of tiny little test files, or long test methods\n exhibiting various bad test smells (Multiple Test Conditions, Flexible Test, Conditional Test Logic, Eager Test, Obscure Test, etc.) So we need a way to write multiple test cases in a single procedural M-file. The natural MATLAB way would be to\n use subfunctions.\n

\n

However, subfunction-based test cases require special consideration. Consider the following M-file structure:

  === File A.m ===\n  function A\n     ...
  function B\n     ...
  function C\n     ...
  function D\n     ...

The first function in the file, A, has the same name as the file. When other code outside this function calls A, it is this first function that gets called. Functions B, C, and D are called subfunctions. Normally, these subfunctions are only visible to and can only be called by A. The only way that code elsewhere might be able to call B, C, or D is if function A forms handles to them and passes those handles out of its scope. Normally this would be done by returning the function handles\n as output arguments.\n

\n

Note that no code executing outside the scope of a function in A.m can form function handles to B, C, or D, or can even determine that these functions exist.\n

\n

This obviously poses a problem for test discovery!

\n

The MATLAB xUnit solution is to establish the following convention for subfunction-based tests. The first function in a test\n M-file containing subfunction tests has to begin with these lines:\n 

  === File A.m ===\n  function test_suite = A\n  initTestSuite;\n  ...

initTestSuite is a script that runs in the scope of the function A. initTestSuite determines which subfunctions are test functions, as well as setup or teardown functions. It forms handles to these functions\n and constructs a set of FunctionHandleTestCase objects, which function A returns as the output argument test_suite.\n

\n

TestRunMonitor

\n

The abstract TestRunMonitor class defines the interface for an object that \"observe\" the in-progress execution of a test suite. MATLAB xUnit provides\n two subclasses of TestRunMonitor:\n

\n
\n
    \n
  • TestRunLogger silently logs test suite events and captures the details of any test failures or test errors.\n
    • \n
  • CommandWindowTestRunDisplay prints the progress of an executing test suite to the Command Window.\n
    • \n
\n
\n

\"\"

\n

A TestRunMonitor is passed to the run() method of a TestComponent object. The run() method calls the appropriate notification methods of the monitor.\n

\n

Here is the output when using the CommandWindowTestRunDisplay object on the MATLAB xUnit's own test suite:

  runtests\n  Starting test run with 92 test cases.\n  ....................\n  ....................\n  ....................\n  ....................\n  ............\n  PASSED in 7.040 seconds.

File System Test Runner

\n

MATLAB xUnit provides a command-line File System Test Runner called runtests. When called with no input arguments, runtests gathers all the test cases from the current directory and runs them, summarizing the results to the Command Window. runtests can also take a string argument specifying which test file, and optionally which specific test case, to run.\n

\n

Test Selection

\n

Test selection is supported in runtests by passing in a string of the form:\n 

   'Location:Name'

or just:

   'Location'

Both of these forms are handled by runtests and by TestSuite.fromName.\n

\n

'Location' is the name of the M-file containing test cases. 'Name' is the name of a specific test case. Normally, the name\n of the test case is the name of the corresponding TestCase method. For FunctionHandleTestCase objects, though, 'Name' is\n the subfunction name.\n

\n

Assertion Methods

\n

MATLAB xUnit provides the following assertion methods:

\n
\n
    \n
  • Stated Outcome Assertion (assertTrue, assertFalse)\n
    • \n
  • Equality Assertion (assertEqual)\n
    • \n
  • Fuzzy Equality Assertion (assertElementsAlmostEqual, assertVectorsAlmostEqual)\n
    • \n
  • Expected Exception Assertion (assertExceptionRaised)\n
    • \n
\n
\n

Assertion functions are provided via globally accessible names (e.g., assertEqual). The assertion functions could be moved to the xunit package, but MATLAB users are not accustomed yet to packages and package name-scoping syntax.\n

\n

'message' is the last input to the assertion functions and is optional. (See below for discussion of Assertion Roulette.)\n

\n

The Expected Exception Assertion, assertExceptionRaised is used by forming an anonymous function handle from an expression that is expected to error, and then passing that function\n handle to assertExceptionRaised along with the expected exception identifier. For example:\n 

  f = @() sin(1,2,3);\n  assertExceptionRaised(f, 'MATLAB:maxrhs')

By using this mechanism, test writers can verify exceptions without using try-catch logic in their test code.

\n

Stack Traces and \"Assertion Roulette\"

\n

xUnit Test Patterns explains the smell Assertion Roulette this way: \"It is hard to tell which of several assertions within the same test method caused a test failure.\n

\n

MATLAB xUnit mitigates against Assertion Roulette by capturing the entire stack trace, including line numbers, for every test failure and test error. (The MATLAB MException\n object, which you obtain via the catch clause, contains the stack trace.) The stack trace is displayed to the Command Window, with clickable links that load the\n corresponding M-file into editor at the appropriate line number.\n

\n

Stack traces can be pretty long, though. Also, test framework plumbing tends to occupy the trace in between the assertion\n and the user's test code, thus making the trace hard to interpret for less-experienced users. MATLAB xUnit, therefore, uses\n a stack filtering heuristic for displaying test fault traces: Starting at the deepest call level, once the trace leaves MATLAB\n xUnit framework functions, all further framework functions are filtered out of the stack trace.\n

\n

Here's an example of stack trace display in the output of runtests:\n

\n

>> runtests testSample
Starting test run with 1 test case.
F
FAILED in 0.081 seconds.
\n
===== Test Case Failure =====
Location: c:\\work\\matlab_xunit\\architecture\\testSample.m
Name: testMyCode
c:\\work\\matlab_xunit\\architecture\\testSample.m at line 6
Input elements are not all equal within relative tolerance: 1.49012e-008

First input:
1
Second input:
1.1000
 \n

\n

Clicking on the blue, underlined link above loads the corresponding file into the editor, positioned at the appropriate line.

\n

Extending the Framework

\n

The MATLAB xUnit framework can be extended primarily by subclassing TestCase, TestSuite, and TestMonitor.\n

\n

TestCase can be subclassed to enable a new set of test cases that all share some particular behavior. The MATLAB xUnit Test Framework\n contains three examples of extending TestCase behavior in this way:\n

\n
\n
    \n
  • FunctionHandleTestCase provides the ability to define test cases based on procedural function handles.\n
    • \n
  • TestCaseInDir defines a test case that must be run inside a particular directory. The setUp and tearDown functions are overridden to change the MATLAB working directory before running the test case, and then to restore the original\n working directory when the test case finished. The class is used by the framework's own test suite.\n
    • \n
  • TestCaseInPath defines a test case that must be run with a particular directory temporarily added to the MATLAB path. Its implementation\n is similar to TestCaseInDir, and it is also used by the framework's own test suite.\n
    • \n
\n
\n

TestSuite could be similarly extended by subclassing. This might a provide a way in the future to define a test suite containing collections\n of test components in separate directories, which is not currently supported.\n

\n

Finally TestRunMonitor could be subclassed to support a variety of test monitoring mechanisms, such as what might be required by a Graphical Test Runner.\n

\n


\n Published with MATLAB® 7.8

\n
\n \n \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/architecture/matlab_xunit_architecture.m", "content": "%% MATLAB xUnit Test Framework: Architectural Notes\n% This document summarizes the key classes and design choices for MATLAB xUnit,\n% a MATLAB unit testing framework based on xUnit patterns.\n%\n% Note: Testing pattern and smell terminology in this document is drawn from\n% _xUnit Test Patterns: Refactoring Test Code_, by Gerard Meszaros,\n% Addison-Wesley, 2007.\n\n%% TestComponent, TestCase, and TestSuite\n%\n% <>\n%\n% The abstract |TestComponent| class defines an object that has a description (a\n% name and a location) and that can be run.\n%\n% A |TestCase| object is a test component that defines an individual test case\n% that can be run with a pass or fail result.\n%\n% A |TestSuite| object is a test component that contains a collection of other\n% test components. Note the hierarchical nature of test suites; they can\n% contain both individual test case objects as well as other test suites.\n% Running a test suite means invoking the |run| method on each test component in\n% its collection.\n\n%% TestCase: The Four-Phase Test\n%\n% The TestCase class provides the standard xUnit _Four-Phase Test_, using\n% a _Fresh Fixture_, _Implicit Setup_, and _Implicit Teardown_. These all\n% elements can all be seen in the |run| method of TestCase:\n%\n% function did_pass = run(self, monitor)\n% %run Execute the test case\n% % test_case.run(monitor) calls the TestCase object's setUp()\n% % method, then the test method, then the tearDown() method.\n% % observer is a TestRunObserver object. The testStarted(),\n% % testFailure(), testError(), and testFinished() methods of\n% % observer are called at the appropriate times. monitor is a\n% % TestRunMonitor object. Typically it is either a TestRunLogger\n% % subclass or a CommandWindowTestRunDisplay subclass.\n% %\n% % test_case.run() automatically uses a\n% % CommandWindowTestRunDisplay object in order to print test\n% % suite execution information to the Command Window.\n% \n% if nargin < 2\n% monitor = CommandWindowTestRunDisplay();\n% end\n% \n% did_pass = true;\n% monitor.testComponentStarted(self);\n% \n% try\n% self.setUp();\n% f = str2func(self.MethodName);\n% \n% try\n% % Call the test method.\n% f(self);\n% catch failureException\n% monitor.testCaseFailure(self, failureException);\n% did_pass = false;\n% end\n% \n% self.tearDown();\n% \n% catch errorException\n% monitor.testCaseError(self, errorException);\n% did_pass = false;\n% end\n% \n% monitor.testComponentFinished(self, did_pass);\n% end\n%\n% Phase 1 sets up the test fixture via the _Implicit Setup_ call, |self.setUp()|.\n% The base class |setUp()| method does nothing.\n%\n% Phases 2 and 3 (exercising the system under test and verifying the expected\n% outcome) are handled by the test method, which is invoked by |f(self)|.\n%\n% Phase 4 tears down the test fixture via the _Implicit Teardown_ call,\n% |self.tearDown()|. The base class |tearDown()| method does nothing.\n%\n% Test failure and test error exceptions are caught and handled by the |run()|\n% method, so test methods do not need to use try-catch. This facilitates\n% simple, straight-line test-method code.\n%\n% _Note: The |monitor| object will be discussed later._\n\n%% Test Case Discovery\n% The static method |TestSuite.fromName| constructs a test suite based on the\n% name of an M-file. If the M-file defines a |TestCase| subclass, then |fromName|\n% inspects the methods of the class and constructs a |TestCase| object for each\n% method whose name begins with \"[tT]est\". If the M-file does not define a\n% |TestCase| subclass, then |fromName| attempts to construct either a simple\n% procedural test case or a set of subfunction-based test cases. (See the next\n% section).\n%\n% The static method |TestSuite.fromPwd| constructs a test suite by discovering\n% all the test cases in the present working directory. It discovers all\n% |TestCase| subclasses in the directory. In addition, it constructs test suites\n% from all the procedural M-files in the directory beginning with \"[tT]est\".\n%\n% The _File System Test Runner_, |runtests|, provides convenient syntaxes for\n% performing test case discovery automatically.\n\n%% FunctionHandleTestCase: For the Procedural World\n% Most MATLAB users are much more comfortable with procedural programming. An\n% important design goal for MATLAB xUnit is to make it as easy as possible for MATLAB\n% users with little object-oriented programming experience to create and run\n% their own tests. The FunctionHandleTestCase supplies the plumbing necessary\n% to support procedural test functions:\n%\n% <>\n%\n% Private properties |SetupFcn|, |TestFcn|, and |TeardownFcn| are procedural \n% _function handles_ (similar to function pointers or function references in\n% other languages).\n%\n% |runTestCase()| is the test method used for constructing a TestCase object.\n%\n% Managing test fixtures requires special consideration, because procedural\n% function handles don't have access to object instance data in order to access\n% a test fixture.\n%\n% The overridden |setUp()| method looks at the number of outputs of the function\n% handle |SetupFcn|. If it has an output argument, then the argument is saved\n% in the private |TestData| property, and |TestData| is then passed to both\n% |TestFcn| and |TeardownFcn| for their use.\n\n%% Writing Procedural Test Cases\n% Procedural test cases can be written in two ways: \n%\n% * A simple M-file function that is treated as a single test case\n% * An M-file containing multiple subfunctions that are each treated as a test case. \n%\n% In either case, the test\n% case is considered to pass if it executes without error.\n%\n% Writing one test case per file is not ideal; it would lead to either zillions\n% of tiny little test files, or long test methods exhibiting various bad test\n% smells (_Multiple Test Conditions_, _Flexible Test_, _Conditional Test Logic_,\n% _Eager Test_, _Obscure Test_, etc.) So we need a way to write multiple test\n% cases in a single procedural M-file. The natural MATLAB way would be to use\n% subfunctions.\n%\n% However, subfunction-based test cases require special consideration. Consider\n% the following M-file structure:\n%\n% === File A.m ===\n% function A\n% ...\n% \n% function B\n% ...\n% \n% function C\n% ...\n% \n% function D\n% ...\n%\n% The first function in the file, |A|, has the same name as the file. When\n% other code outside this function calls |A|, it is this first function that\n% gets called. Functions |B|, |C|, and |D| are called _subfunctions_.\n% Normally, these subfunctions are only visible to and can only be called by\n% |A|. The only way that code elsewhere might be able to call |B|, |C|, or |D|\n% is if function |A| forms handles to them and passes those handles out of its\n% scope. Normally this would be done by returning the function handles as\n% output arguments.\n%\n% Note that no code executing outside the scope of a function in A.m can form\n% function handles to |B|, |C|, or |D|, or can even determine that these\n% functions exist.\n%\n% This obviously poses a problem for test discovery!\n%\n% The MATLAB xUnit solution is to establish the following convention for\n% subfunction-based tests. The first function in a test M-file containing\n% subfunction tests has to begin with these lines:\n%\n% === File A.m ===\n% function test_suite = A\n% initTestSuite;\n% ...\n%\n% |initTestSuite| is a _script_ that runs in the scope of the function |A|.\n% |initTestSuite| determines which subfunctions are test functions, as well as setup\n% or teardown functions. It forms handles to these functions and constructs a\n% set of FunctionHandleTestCase objects, which function |A| returns as the\n% output argument |test_suite|.\n\n%% TestRunMonitor\n% The abstract |TestRunMonitor| class defines the interface for an object that\n% \"observe\" the in-progress execution of a test suite. MATLAB xUnit provides two\n% subclasses of |TestRunMonitor|:\n%\n% * |TestRunLogger| silently logs test suite events and captures the details of\n% any test failures or test errors.\n% * |CommandWindowTestRunDisplay| prints the progress of an executing test suite\n% to the Command Window.\n%\n% <>\n%\n% A TestRunMonitor is passed to the |run()| method of a TestComponent object.\n% The |run()| method calls the appropriate notification methods of the\n% monitor.\n%\n% Here is the output when using the CommandWindowTestRunDisplay object on the\n% MATLAB xUnit's own test suite:\n%\n% runtests \n% Starting test run with 92 test cases.\n% ....................\n% ....................\n% ....................\n% ....................\n% ............\n% PASSED in 7.040 seconds.\n\n%% File System Test Runner\n% MATLAB xUnit provides a command-line _File System Test Runner_ called\n% |runtests|. When called with no input arguments, |runtests| gathers all the \n% test cases from the current directory and runs them, summarizing the results\n% to the Command Window. |runtests| can also take a string argument specifying\n% which test file, and optionally which specific test case, to run.\n\n%% Test Selection\n% Test selection is supported in |runtests| by passing in a string of the form:\n%\n% 'Location:Name'\n%\n% or just:\n%\n% 'Location'\n%\n% Both of these forms are handled by |runtests| and by |TestSuite.fromName|.\n%\n% 'Location' is the name of the M-file containing test cases. 'Name' is the\n% name of a specific test case. Normally, the name of the test case is the name\n% of the corresponding TestCase method. For FunctionHandleTestCase objects,\n% though, 'Name' is the subfunction name.\n\n%% Assertion Methods\n% MATLAB xUnit provides the following assertion methods:\n%\n% * _Stated Outcome Assertion_ (|assertTrue|, |assertFalse|)\n% * _Equality Assertion_ (|assertEqual|)\n% * _Fuzzy Equality Assertion_ (|assertElementsAlmostEqual|, |assertVectorsAlmostEqual|)\n% * _Expected Exception Assertion_ (|assertExceptionRaised|)\n%\n% Assertion functions are provided via globally accessible names (e.g.,\n% |assertEqual|). The assertion functions could be moved to the |xunit|\n% package, but MATLAB users are not accustomed yet to packages and package\n% name-scoping syntax.\n%\n% 'message' is the last input to the assertion functions and is optional. (See\n% below for discussion of _Assertion Roulette_.)\n%\n% The _Expected Exception Assertion_, |assertExceptionRaised| is used by forming\n% an anonymous function handle from an expression that is expected to error, and\n% then passing that function handle to |assertExceptionRaised| along with the\n% expected exception identifier. For example:\n%\n% f = @() sin(1,2,3);\n% assertExceptionRaised(f, 'MATLAB:maxrhs')\n%\n% By using this mechanism, test writers can verify exceptions without using\n% try-catch logic in their test code.\n\n%% Stack Traces and \"Assertion Roulette\"\n% _xUnit Test Patterns_ explains the smell _Assertion Roulette_ this way: \"It is\n% hard to tell which of several assertions within the same test method caused a\n% test failure.\n%\n% MATLAB xUnit mitigates against _Assertion Roulette_ by capturing the entire stack\n% trace, including line numbers, for every test failure and test error. (The\n% MATLAB MException object, which you obtain via the |catch| clause, contains\n% the stack trace.) The stack trace is displayed to the Command Window, with\n% clickable links that load the corresponding M-file into editor at the\n% appropriate line number.\n%\n% Stack traces can be pretty long, though. Also, test framework plumbing tends\n% to occupy the trace in between the assertion and the user's test code, thus\n% making the trace hard to interpret for less-experienced users. MATLAB xUnit,\n% therefore, uses a stack filtering heuristic for displaying test fault traces:\n% Starting at the deepest call level, once the trace leaves MATLAB xUnit framework\n% functions, all further framework functions are filtered out of the stack\n% trace.\n%\n% Here's an example of stack trace display in the output of |runtests|:\n%\n% \n% \n% >> runtests testSample
\n% Starting test run with 1 test case.
\n% F
\n% FAILED in 0.081 seconds.
\n%
\n% ===== Test Case Failure =====
\n% Location: c:\\work\\matlab_xunit\\architecture\\testSample.m
\n% Name: testMyCode
\n%
\n% c:\\work\\matlab_xunit\\architecture\\testSample.m at line 6
\n%
\n% Input elements are not all equal within relative tolerance: 1.49012e-008
\n%
\n% First input:
\n% 1
\n%
\n% Second input:
\n% 1.1000
\n% \n% \n%\n% Clicking on the blue, underlined link above loads the corresponding file into\n% the editor, positioned at the appropriate line.\n\n%% Extending the Framework\n% The MATLAB xUnit framework can be extended primarily by subclassing |TestCase|,\n% |TestSuite|, and |TestMonitor|.\n%\n% |TestCase| can be subclassed to enable a new set of test cases that all share\n% some particular behavior. The MATLAB xUnit Test Framework contains three\n% examples of extending |TestCase| behavior in this way:\n%\n% * |FunctionHandleTestCase| provides the ability to define test cases based on\n% procedural function handles.\n% * |TestCaseInDir| defines a test case that must be run inside a particular\n% directory. The |setUp| and |tearDown| functions are overridden to change the\n% MATLAB working directory before running the test case, and then to restore the\n% original working directory when the test case finished. The class is used by\n% the framework's own test suite.\n% * |TestCaseInPath| defines a test case that must be run with a particular\n% directory temporarily added to the MATLAB path. Its implementation is similar\n% to |TestCaseInDir|, and it is also used by the framework's own test suite.\n%\n% |TestSuite| could be similarly extended by subclassing. This might a provide a\n% way in the future to define a test suite containing collections of test\n% components in separate directories, which is not currently supported.\n%\n% Finally |TestRunMonitor| could be subclassed to support a variety of test\n% monitoring mechanisms, such as what might be required by a _Graphical Test\n% Runner_." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/architecture/testSample.m", "content": "function test_suite = testSample\ninitTestSuite;\n\nfunction testMyCode\nassertEqual(1, 1);\nassertElementsAlmostEqual(1, 1.1);\nassertTrue(10 == 10);" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/+abc/+tests/test_that.m", "content": "% Do-nothing test used in the examples for organizing tests inside packages.\n%\n% Steven L. Eddins\n% Copyright 2010 The MathWorks, Inc.\n\nfunction test_that\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/+abc/+tests/test_this.m", "content": "% Do-nothing test used in the examples for organizing tests inside packages.\n%\n% Steven L. Eddins\n% Copyright 2010 The MathWorks, Inc.\n\nfunction test_this\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/+abc_tests/test_that.m", "content": "% Do-nothing test used in the examples for organizing tests inside packages.\n%\n% Steven L. Eddins\n% Copyright 2010 The MathWorks, Inc.\n\nfunction test_that\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/+abc_tests/test_this.m", "content": "% Do-nothing test used in the examples for organizing tests inside packages.\n%\n% Steven L. Eddins\n% Copyright 2010 The MathWorks, Inc.\n\nfunction test_this\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exException.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Test an Error Message\n% It's surprising to most people (but not quality engineers) how\n% often programmers make errors in error-handling code. Because of\n% this unfortunate truth, it is useful to write unit tests that\n% verify that your MATLAB code throws the proper error, at the\n% proper time.\n%\n% The assertion function that makes this task easy is\n% |assertExceptionThrown|. This example shows how to write a unit\n% test that verifies the \"Too many input arguments\" error for the\n% |cos| function.\n%\n% Your first step is to determine the _error identifier_ associated\n% with the error message. You can find out the error identifier by\n% using the |lasterror| function.\n%\n% If you call |cos| with two input arguments, like this:\n%\n% cos(1, 2)\n%\n% you get this error message:\n%\n% Error using ==> cos\n% Too many input arguments. \n%\n% Then if you call |lasterror|, you get this output:\n%\n% ans = \n% \n% message: [1x45 char]\n% identifier: 'MATLAB:maxrhs'\n% stack: [0x1 struct]\n%\n% So the _identifier_ associated with this error message is\n% |'MATLAB:maxrhs'|.\n%\n% When you write your test function, you'll form an anonymous\n% function handle that calls |cos| with the erroneous additional\n% input argument.\n\nf = @() cos(1, 2)\n\n%%\n% You then pass this function to |assertExceptionThrown|, along with\n% the expected error identifier.\n\nassertExceptionThrown(f, 'MATLAB:maxrhs');\n\n%%\n% |assertExceptionThrown| verifies that when |f()| is called, an\n% error results with the specified error identifier.\n%\n% Here's our error condition test for the |cos| function.\n\ncd examples_general\ntype testCos\n\n%%\n% Run the test using |runtests|.\n\nruntests testCos\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exQuickStart.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Write and Run Tests \n% This example shows how to write and run a couple of test cases for the MATLAB\n% |fliplr| function.\n\n%% Make a folder for your tests\n% To get started, create a folder (directory) that will contain your tests, and\n% then make that your working folder. The test directory in this example is\n% example_quick_start.\n\ncd example_quick_start\n\n%% Write each test case as a simple M-file\n% Write each test case as an M-file function that returns no output arguments.\n% The function name should start or end with \"test\" or \"Test\". The test case\n% passes if the function runs with no error.\n%\n% Here's a test-case M-file that verifies the correct output for a vector input.\n\ntype testFliplrVector\n\n%%\n% The function |testFliplrVector| calls the function being tested and checks the\n% output against the expected output. If the output is different than expected,\n% the function calls |error|.\n%\n% Here's another test-case M-file that verifies the correct |fliplr| output for\n% a matrix input.\n\ntype testFliplrMatrix\n\n%%\n% This function is simpler than |testFliplrVector| because it uses the utility\n% testing function |assertEqual|. |assertEqual| checks to see whether its two\n% inputs are equal. If they are equal, |assertEqual| simply returns silently.\n% If they are not equal, |assertEqual| calls |error|.\n\n%% Run all the tests using |runtests|\n% To run all your test cases, simply call |runtests|. |runtests| automatically finds\n% all the test cases in the current directory, runs them, and reports the\n% results to the Command Window.\n\nruntests\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exRunSpecificTest.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Run a Specific Test\n% To run all the test cases in just one M-file, ignoring other test\n% cases that might be in other files in the same directory, give\n% the name of the file (without the \".m\" extension) as an argument\n% to |runtests|.\n%\n% For example\n\ncd example_subfunction_tests\n\nruntests testFliplr\n\n%%\n% To run a single test case, add the name of the test case using a\n% colon (\":\"), like this:\n\nruntests testFliplr:testFliplrVector\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exRunTestsInADirectory.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Run Tests in Specific Directories\n% To run all the test cases in a specific directory, give the name of the\n% directory as an argument to |runtests|.\n%\n% For example\n\nruntests example_subfunction_tests\n\n%%\n% To run tests in multiple directories, give each directory name as a separate\n% argument to |runtests|.\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exRunTestsInPackage.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Run Tests in a Package\n% To run all the test cases in a package, give the name of the\n% package as an argument to |runtests|. *Note:* Running tests in a package\n% requires MATLAB R2009a or later.\n%\n% For example, suppose you are distributing a set of MATLAB files called the\n% \"ABC Toolbox.\" Then you could put your tests inside a package called abc_tests\n% and run them like this:\n\nruntests abc_tests\n\n%%\n% (Note that the initial \"+\" character in the name of the package folder on disk\n% is not part of the package name.)\n%\n% Or you could put your tests inside a subpackage called abc.tests and run them\n% like this:\n\nruntests abc.tests\n\n%%\n% You should not use a generic top-level package name such \"tests\" because then\n% your package might be unintentionally combined with packages with the same\n% name created by other people. \n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2010 The MathWorks, Inc.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exSilentRunning.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Run Tests Silently and Query the Results\n% When you run a test suite using |runtests|, the results are\n% summarized in the Command Window. This example shows you how to\n% run a test suite so that nothing prints to the Command Window, and\n% it shows you how to write a program to automatically determine the\n% results of running the test suite.\n%\n% There are four steps to follow.\n%\n% 1. Construct a |TestSuite| object. In this example we'll use the |fromPwd|\n% method of the |TestSuite| class to construct a test suite using all the test\n% cases found in the |examples_general| directory.\n\ncd examples_general\nsuite = TestSuite.fromPwd();\n\n%%\n% You can look up information about the individual test cases.\n\nsuite.TestComponents{1}\n\n%%\n% You can see above that the first test component in the test suite is itself\n% another test suite, which contains the test cases defined by the M-file named\n% TestUsingTestCase. Here's what one of these individual test cases looks like:\n\nsuite.TestComponents{1}.TestComponents{1}\n\n%%\n% 2. Construct a TestLogger object. This object can receive\n% notifications about what happens when a test suite is executed.\n\nlogger = TestRunLogger;\n\n%%\n% 3. Call the |run| method of the |TestSuite| object, passing it the\n% logger.\n\nsuite.run(logger);\n\n%%\n% The |TestLogger| object can now be queried to determine what\n% happened during the test.\n\nlogger\n\n%%\n% There were eight test cases run (logger.NumTestCases), resulting in\n% one test failure and one test error. Detailed information about\n% what went wrong can be found in |logger.Faults|.\n\nlogger.Faults(1)\n\n%%\n\nlogger.Faults(2)\n\n%%\n% You can drill further to determine the names of the failing tests,\n% as well as the complete stack trace associated with each failure.\n\nlogger.Faults(1).TestCase\n\n%%\n\nlogger.Faults(1).Exception.stack(1)\n\n%%\n\nlogger.Faults(1).Exception.stack(2)\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exSubfunctionTests.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Put Multiple Test Cases in One M-file\n% The Quick Start example showed how you can write a simple M-file\n% to be a single test case. This example shows you how to put multiple\n% test cases in one M-file.\n%\n% Name your M-file beginning or ending with \"test\", like\n% \"testMyFunc\". Start by putting the following two lines at the\n% beginning of the file. It's important that the output variable\n% name on line 1 be |test_suite|.\n%\n% function test_suite = testMyFunc\n% initTestSuite;\n%\n% Next, add subfunctions to the file. Each subfunction beginning\n% or ending with \"test\" becomes an individual test case.\n%\n% The directory example_subfunction_tests contains a test M-file\n% containing subfunction test cases for the |fliplr| function.\n\ncd example_subfunction_tests\n\ntype testFliplr\n\n%%\n% As usual, run the test cases using |runtests|:\n\nruntests\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exTestCase.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Write xUnit-Style Tests by Subclassing TestCase\n% The MATLAB xUnit architecture is based closely on the xUnit style, in\n% which each test case is an instance of a subclass of the base\n% TestCase class. Programmers who are familiar with this style may\n% want to write their own TestCase subclasses instead of using\n% <./exSubfunctionTests.html subfunction-based tests>.\n%\n% This example shows a TestCase subclass containing test case\n% methods and test fixture methods. If you are not familiar with\n% defining your own classes in MATLAB, you might want to review the\n% MATLAB documentation on \n% ,\n% or you can simply stick to using subfunction-based tests.\n%\n% The sample M-file begins with the |classdef| statement, which sets\n% the name of the class and indicates that it is a subclass of\n% |TestCase|.\n\ncd examples_general\ndbtype TestUsingTestCase 1\n\n%%\n% The properties block contains a field that is initialized by the\n% setup method and is used by the two test methods.\n\ndbtype TestUsingTestCase 3:5\n\n%%\n% The first method in the methods block is the constructor. It\n% takes the desired test method name as its input argument, and it\n% passes that input along to the base class constructor.\n\ndbtype TestUsingTestCase 7:10\n\n%%\n% The |setUp| method creates a figure window and stores its handle in\n% the field |fh|.\n\ndbtype TestUsingTestCase 12:14\n\n%%\n% Test methods are those beginning with \"test\".\n\ndbtype TestUsingTestCase 20:26\n\n%%\n% The |tearDown| method cleans up by deleting the figure window.\n\ndbtype TestUsingTestCase 16:18\n\n%%\n% Run the test cases in the class by calling |runtests| with the name\n% of the class.\n\nruntests TestUsingTestCase\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exTestCaseSearching.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How RUNTESTS Searches for Test Cases\n% When you call |runtests| with no input arguments:\n%\n% >> runtests\n%\n% it automatically searches for all the test cases in the current directory. It\n% looks for test cases in three types of M-files:\n%\n% 1. An M-file function whose name begins or ends with \"test\" or \"Test\" and that does\n% not return an output argument. Such a function is considered to be a single\n% test case. \n%\n% 2. An M-file function whose name begins or ends with \"test\" or \"Test\" and that returns\n% an output argument that is a test suite. Such a function is considered to contain\n% subfunction-style test cases. Each subfunction whose name begins or ends with \"test\"\n% or \"Test\" is a test case. \n%\n% 3. An M-file that defines a subclass of TestCase. Each method beginning or ending with\n% \"test\" or \"Test\" is a test case.\n%\n% |runtests| uses the |TestSuite| static methods |fromName| and |fromPwd| to\n% automatically construct the test suites.\n%\n% Here are a couple of examples.\n%\n% |TestSuite.fromName| takes an M-file name, determines what\n% kind of test file it is, and returns a cell array of test case objects.\n\ncd examples_general\ntest_suite_1 = TestSuite.fromName('testSetupExample')\n\n%%\n% |TestSuite.fromPwd| returns a test suite based on all the test files in the\n% current directory.\n\ntest_suite_2 = TestSuite.fromPwd()\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exTestFixtures.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Write Tests That Share Common Set-Up Code\n% Sometimes you want to write a set of test cases in which the same\n% set of initialization steps is performed before each test case, or\n% in which the same set of cleanup steps is performed after each\n% test case. This set of common _setup_ and _teardown_ code is\n% called a _test fixture_.\n%\n% In subfunction-based test files, you can add subfunctions whose\n% names begin with \"setup\" and \"teardown\". These functions will be\n% called before and after every test-case subfunction is called. If\n% the setup function returns an output argument, that value is saved\n% and passed to every test-case subfunction and also to the teardown\n% function.\n%\n% This example shows a setup function that creates a figure and \n% returns its handle. The figure handle is passed to each test-case\n% subfunction. The figure handle is also passed to the teardown\n% function, which cleans up after each test case by deleting the\n% figure.\n\ncd examples_general\ntype testSetupExample\n\n%%\n% Run the tests using |runtests|.\n\nruntests testSetupExample\n\n%%\n% You might also want to see the \n% <./exTestCase.html example on writing test cases by\n% subclassing TestCase>.\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/exTolerance.m", "content": "%% <../index.html MATLAB xUnit Test Framework>: How to Test Using a Floating-Point Tolerance\n% MATLAB performs arithmetic operations using the floating-point\n% hardware instructions on your processor. Because\n% almost all floating-point operations are subject to round-off\n% error, arithmetic operations can sometimes produce surprising\n% results. Here's an example.\n\na = 1 + 0.1 + 0.1 + 0.1\n\n%%\na == 1.3\n\n%%\n% So why doesn't |a| equal 1.3? Because 0.1, 1.3, and most other\n% decimal fractions do not have exact representations in the binary\n% floating-point number representation your computer uses. The\n% first line above is doing an approximate addition of 1 plus an\n% approximation of 0.1, plus an approximation of 0.1, plus an\n% approximation of 0.1. The second line compares the result of all\n% that with an approximation of 1.3.\n%\n% If you subtract 1.3 from |a|, you can see that the computed result\n% for |a| is _extremely close_ to the floating-point approximation\n% of 1.3, but it is not exactly the same.\n\na - 1.3\n\n%%\n% As a general rule, when comparing the results of floating-point\n% calculations for equality, it is necessary to use a tolerance\n% value. Two types of tolerance comparisons are commonly used: absolute\n% tolerance and relative tolerance. An absolute tolerance comparison of _a_ and _b_ \n% looks like:\n%\n% $$|a-b| \\leq T$$\n%\n% A relative tolerance comparison looks like:\n%\n% $$|a-b| \\leq T\\max(|a|,|b|) + T_f$$\n%\n% where _Tf_ is called the _floor tolerance_. It acts as an absolute tolerance\n% when _a_ and _b_ are very close to 0.\n%\n% For example, suppose that _a_ is 100, _b_ is 101, and T is 0.1. Then _a_ and\n% _b_ would not be considered equal using an absolute tolerance, because 1 >\n% 0.1. However, _a_ and _b_ would be considered equal using a relative\n% tolerance, because they differ by only 1 part in 100.\n%\n% MATLAB xUnit provides the utility assertion functions called\n% |assertElementsAlmostEqual| and |assertVectorAlmostEqual|. These functions\n% make it easy to write tests involving floating-point tolerances.\n%\n% |assertElementsAlmostEqual(A,B)| applies the tolerance test independently to\n% every element of |A| and |B|. The function uses a relative tolerance test by\n% default, but you make it use an absolute tolerance test, or change the\n% tolerance values used, by passing additional arguments to it.\n%\n% |assertVectorsAlmostEqual(A,B)| applies the tolerance test to the vectors |A|\n% and |B| in the L2-norm sense. For example, suppose |A| is |[1 1e10|], |B|\n% is |[2 1e10]|, and the tolerance is 1e-8. Then |A| and |B| would fail an\n% elementwise relative tolerance comparison, because the relative difference\n% between the first elements is 0.5. However, they would pass a vector relative\n% tolerance comparison, because the relative vector difference between |A| and\n% |B| is only about 1 part in 1e10.\n%\n% The |examples_general| directory contains a portion of a unit test for the\n% |sin| function. The output of |sin| can sometimes be a bit surprising because\n% of floating-point issues. For example:\n\nsin(pi)\n\n%%\n% That's very close but not exactly equal to 0. Here's how the\n% |sin| unit test uses |assertElementsAlmostEqual| to write the |sin(pi)|\n% test with a minimum of fuss.\n\ncd examples_general\ntype testSin\n\n%%\n% Run the test using |runtests|.\n\nruntests testSin\n\n%%\n% <../index.html Back to MATLAB xUnit Test Framework>\n\n%%\n% Copyright 2008-2010 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/example_quick_start/testFliplrMatrix.m", "content": "function testFliplrMatrix\n%testFliplrMatrix Unit test for fliplr with matrix input\n\nin = magic(3);\nassertEqual(fliplr(in), in(:, [3 2 1]));\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/example_quick_start/testFliplrVector.m", "content": "function testFliplrVector\n%testFliplrVector Unit test for fliplr with vector input\n\nin = [1 4 10];\nout = fliplr(in);\nexpected_out = [10 4 1];\n\nif ~isequal(out, expected_out)\n error('testFliplrVector:notEqual', 'Incorrect output for vector.');\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/example_subfunction_tests/testFliplr.m", "content": "function test_suite = testFliplr\ninitTestSuite;\n\nfunction testFliplrMatrix\nin = magic(3);\nassertEqual(fliplr(in), in(:, [3 2 1]));\n\nfunction testFliplrVector\nassertEqual(fliplr([1 4 10]), [10 4 1]);\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/examples_general/TestUsingTestCase.m", "content": "classdef TestUsingTestCase < TestCase\n\n properties\n fh\n end\n\n methods\n function self = TestUsingTestCase(name)\n self = self@TestCase(name);\n end\n\n function setUp(self)\n self.fh = figure;\n end\n\n function tearDown(self)\n delete(self.fh);\n end\n\n function testColormapColumns(self)\n assertEqual(size(get(self.fh, 'Colormap'), 2), 3);\n end\n\n function testPointer(self)\n assertEqual(get(self.fh, 'Pointer'), 'arrow');\n end\n end\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/examples_general/testBadSinTest.m", "content": "function test_suite = testBadSinTest\ninitTestSuite;\n\nfunction testSinPi\n% Example of a failing test case. The test writer should have used\n% assertAlmostEqual here.\nassertEqual(sin(pi), 0);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/examples_general/testCos.m", "content": "function test_suite = testCos\ninitTestSuite;\n\nfunction testTooManyInputs\nassertExceptionThrown(@() cos(1, 2), 'MATLAB:maxrhs');" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/examples_general/testSetupExample.m", "content": "function test_suite = testSetupExample\ninitTestSuite;\n\nfunction fh = setup\nfh = figure;\n\nfunction teardown(fh)\ndelete(fh);\n\nfunction testColormapColumns(fh)\nassertEqual(size(get(fh, 'Colormap'), 2), 3);\n\nfunction testPointer(fh)\nassertEqual(get(fh, 'Pointer'), 'arrow');\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/examples_general/testSin.m", "content": "function testSin\n\nassertElementsAlmostEqual(sin(pi), 0);" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/examples_general/testWithSetupError.m", "content": "function test_suite = testWithSetupError\n%Example of a test with an error. The setup function calls cos with\n%too many input arguments.\n\ninitTestSuite;\n\nfunction testData = setup\ntestData = cos(1, 2);\n\nfunction testMyFeature(testData)\nassertEqual(1, 1);\n\nfunction teardown(testData)\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/file_exchange_description.txt", "content": "MATLAB xUnit Test Framework is a unit test framework for MATLAB code.\n\nMATLAB xUnit is designed to be easy to use for MATLAB users with a wide range of experience. Users can write tests using ordinary M-files that are very simple in structure.\n\nMATLAB xUnit comes with extensive documentation that ranges in scope from a \"Getting Started\" section to advanced techniques and architectural notes. You can view this documentation online without downloading the package. For example, scroll down to the \"Published M Files\" section on this page and click on \"MATLAB xUnit Quick Start - How to write and run tests.\" To see all the MATLAB xUnit documentation online, scroll down to the \"HTML Files\" section on this page and click on \"Readme.html.\"\n\nOnly the \"xunit\" directory is needed to use the framework. The \"tests\" directory contains the framework's own test suite. The \"architecture\" directory contains architectural notes on the framework's design and how it might be extended.\n\nMATLAB xUnit can be used with MATLAB releases R2008a and later. MATLAB xUnit relies heavily on object-oriented language features introduced in R2008a and will not work with earlier releases.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exException.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Test an Error Message

MATLAB xUnit Test Framework: How to Test an Error Message

It's surprising to most people (but not quality engineers) how often programmers make errors in error-handling code. Because of this unfortunate truth, it is useful to write unit tests that verify that your MATLAB code throws the proper error, at the proper time.

The assertion function that makes this task easy is assertExceptionThrown. This example shows how to write a unit test that verifies the \"Too many input arguments\" error for the cos function.

Your first step is to determine the error identifier associated with the error message. You can find out the error identifier by using the lasterror function.

If you call cos with two input arguments, like this:

 cos(1, 2)

you get this error message:

 Error using ==> cos\n Too many input arguments.

Then if you call lasterror, you get this output:

 ans =
        message: [1x45 char]\n     identifier: 'MATLAB:maxrhs'\n          stack: [0x1 struct]

So the identifier associated with this error message is 'MATLAB:maxrhs'.

When you write your test function, you'll form an anonymous function handle that calls cos with the erroneous additional input argument.

f = @() cos(1, 2)\n
\nf = \n\n    @()cos(1,2)\n\n

You then pass this function to assertExceptionThrown, along with the expected error identifier.

assertExceptionThrown(f, 'MATLAB:maxrhs');\n

assertExceptionThrown verifies that when f() is called, an error results with the specified error identifier.

Here's our error condition test for the cos function.

cd examples_general\ntype testCos\n
\nfunction test_suite = testCos\ninitTestSuite;\n\nfunction testTooManyInputs\nassertExceptionThrown(@() cos(1, 2), 'MATLAB:maxrhs');\n

Run the test using runtests.

runtests testCos\n
Starting test run with 1 test case.\n.\nPASSED in 0.018 seconds.\n

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exQuickStart.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Write and Run Tests

MATLAB xUnit Test Framework: How to Write and Run Tests

This example shows how to write and run a couple of test cases for the MATLAB fliplr function.

Contents

Make a folder for your tests

To get started, create a folder (directory) that will contain your tests, and then make that your working folder. The test directory in this example is example_quick_start.

cd example_quick_start\n

Write each test case as a simple M-file

Write each test case as an M-file function that returns no output arguments. The function name should start or end with \"test\" or \"Test\". The test case passes if the function runs with no error.

Here's a test-case M-file that verifies the correct output for a vector input.

type testFliplrVector\n
\nfunction testFliplrVector\n%testFliplrVector Unit test for fliplr with vector input\n\nin = [1 4 10];\nout = fliplr(in);\nexpected_out = [10 4 1];\n\nif ~isequal(out, expected_out)\n    error('testFliplrVector:notEqual', 'Incorrect output for vector.');\nend\n\n

The function testFliplrVector calls the function being tested and checks the output against the expected output. If the output is different than expected, the function calls error.

Here's another test-case M-file that verifies the correct fliplr output for a matrix input.

type testFliplrMatrix\n
\nfunction testFliplrMatrix\n%testFliplrMatrix Unit test for fliplr with matrix input\n\nin = magic(3);\nassertEqual(fliplr(in), in(:, [3 2 1]));\n\n

This function is simpler than testFliplrVector because it uses the utility testing function assertEqual. assertEqual checks to see whether its two inputs are equal. If they are equal, assertEqual simply returns silently. If they are not equal, assertEqual calls error.

Run all the tests using runtests

To run all your test cases, simply call runtests. runtests automatically finds all the test cases in the current directory, runs them, and reports the results to the Command Window.

runtests\n
Starting test run with 2 test cases.\n..\nPASSED in 0.002 seconds.\n

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exRunSpecificTest.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Run a Specific Test

MATLAB xUnit Test Framework: How to Run a Specific Test

To run all the test cases in just one M-file, ignoring other test cases that might be in other files in the same directory, give the name of the file (without the \".m\" extension) as an argument to runtests.

For example

cd example_subfunction_tests\n\nruntests testFliplr\n
Starting test run with 2 test cases.\n..\nPASSED in 0.023 seconds.\n

To run a single test case, add the name of the test case using a colon (\":\"), like this:

runtests testFliplr:testFliplrVector\n
Starting test run with 1 test case.\n.\nPASSED in 0.001 seconds.\n

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exRunTestsInADirectory.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Run Tests in Specific Directories

MATLAB xUnit Test Framework: How to Run Tests in Specific Directories

To run all the test cases in a specific directory, give the name of the directory as an argument to runtests.

For example

runtests example_subfunction_tests\n
Starting test run with 2 test cases.\n..\nPASSED in 0.062 seconds.\n

To run tests in multiple directories, give each directory name as a separate argument to runtests.

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exRunTestsInPackage.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Run Tests in a Package

MATLAB xUnit Test Framework: How to Run Tests in a Package

To run all the test cases in a package, give the name of the package as an argument to runtests. Note: Running tests in a package requires MATLAB R2009a or later.

For example, suppose you are distributing a set of MATLAB files called the \"ABC Toolbox.\" Then you could put your tests inside a package called abc_tests and run them like this:

runtests abc_tests\n
Test suite: abc_tests\nTest suite location: Package\n19-Nov-2010 14:14:36\n\nStarting test run with 2 test cases.\n..\nPASSED in 0.028 seconds.\n

(Note that the initial \"+\" character in the name of the package folder on disk is not part of the package name.)

Or you could put your tests inside a subpackage called abc.tests and run them like this:

runtests abc.tests\n
Test suite: abc.tests\nTest suite location: Package\n19-Nov-2010 14:14:36\n\nStarting test run with 2 test cases.\n..\nPASSED in 0.001 seconds.\n

You should not use a generic top-level package name such \"tests\" because then your package might be unintentionally combined with packages with the same name created by other people.

Back to MATLAB xUnit Test Framework

Copyright 2010 The MathWorks, Inc.
\n Published with MATLAB® 7.11

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exSilentRunning.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Run Tests Silently and Query the Results

MATLAB xUnit Test Framework: How to Run Tests Silently and Query the Results

When you run a test suite using runtests, the results are summarized in the Command Window. This example shows you how to run a test suite so that nothing prints to the Command Window, and it shows you how to write a program to automatically determine the results of running the test suite.

There are four steps to follow.

1. Construct a TestSuite object. In this example we'll use the fromPwd method of the TestSuite class to construct a test suite using all the test cases found in the examples_general directory.

cd examples_general\nsuite = TestSuite.fromPwd();\n

You can look up information about the individual test cases.

suite.TestComponents{1}\n
\nans = \n\n  TestSuite handle\n\n  Properties:\n    TestComponents: {[1x1 TestUsingTestCase]  [1x1 TestUsingTestCase]}\n              Name: 'TestUsingTestCase'\n          Location: [1x80 char]\n\n\n

You can see above that the first test component in the test suite is itself another test suite, which contains the test cases defined by the M-file named TestUsingTestCase. Here's what one of these individual test cases looks like:

suite.TestComponents{1}.TestComponents{1}\n
\nans = \n\n  TestUsingTestCase handle\n\n  Properties:\n            fh: []\n    MethodName: 'testPointer'\n          Name: 'testPointer'\n      Location: 'C:\\Users\\eddins\\local-work\\matlab_xunit\\doc\\examples_general\\TestUsingTestCase.m'\n\n\n

2. Construct a TestLogger object. This object can receive notifications about what happens when a test suite is executed.

logger = TestRunLogger;\n

3. Call the run method of the TestSuite object, passing it the logger.

suite.run(logger);\n

The TestLogger object can now be queried to determine what happened during the test.

logger\n
\nlogger = \n\n  TestRunLogger handle\n\n  Properties:\n             Log: {1x34 cell}\n     NumFailures: 1\n       NumErrors: 1\n    NumTestCases: 8\n          Faults: [1x2 struct]\n\n\n

There were eight test cases run (logger.NumTestCases), resulting in one test failure and one test error. Detailed information about what went wrong can be found in logger.Faults.

logger.Faults(1)\n
\nans = \n\n         Type: 'failure'\n     TestCase: [1x1 FunctionHandleTestCase]\n    Exception: [1x1 MException]\n\n
logger.Faults(2)\n
\nans = \n\n         Type: 'error'\n     TestCase: [1x1 FunctionHandleTestCase]\n    Exception: [1x1 MException]\n\n

You can drill further to determine the names of the failing tests, as well as the complete stack trace associated with each failure.

logger.Faults(1).TestCase\n
\nans = \n\n  FunctionHandleTestCase handle\n\n  Properties:\n    MethodName: 'runTestCase'\n          Name: 'testSinPi'\n      Location: 'C:\\Users\\eddins\\local-work\\matlab_xunit\\doc\\examples_general\\testBadSinTest.m'\n\n\n
logger.Faults(1).Exception.stack(1)\n
\nans = \n\n    file: 'C:\\Users\\eddins\\local-work\\matlab_xunit\\doc\\examples_general\\testBadSinTest.m'\n    name: 'testSinPi'\n    line: 7\n\n
logger.Faults(1).Exception.stack(2)\n
\nans = \n\n    file: 'C:\\Users\\eddins\\local-work\\matlab_xunit\\xunit\\FunctionHandleTestCase.m'\n    name: 'FunctionHandleTestCase.runTestCase'\n    line: 112\n\n

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exSubfunctionTests.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Put Multiple Test Cases in One M-file

MATLAB xUnit Test Framework: How to Put Multiple Test Cases in One M-file

The Quick Start example showed how you can write a simple M-file to be a single test case. This example shows you how to put multiple test cases in one M-file.

Name your M-file beginning or ending with \"test\", like \"testMyFunc\". Start by putting the following two lines at the beginning of the file. It's important that the output variable name on line 1 be test_suite.

  function test_suite = testMyFunc\n  initTestSuite;

Next, add subfunctions to the file. Each subfunction beginning or ending with \"test\" becomes an individual test case.

The directory example_subfunction_tests contains a test M-file containing subfunction test cases for the fliplr function.

cd example_subfunction_tests\n\ntype testFliplr\n
\nfunction test_suite = testFliplr\ninitTestSuite;\n\nfunction testFliplrMatrix\nin = magic(3);\nassertEqual(fliplr(in), in(:, [3 2 1]));\n\nfunction testFliplrVector\nassertEqual(fliplr([1 4 10]), [10 4 1]);\n\n\n

As usual, run the test cases using runtests:

runtests\n
Starting test run with 2 test cases.\n..\nPASSED in 0.027 seconds.\n

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exTestCase.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Write xUnit-Style Tests by Subclassing TestCase

MATLAB xUnit Test Framework: How to Write xUnit-Style Tests by Subclassing TestCase

The MATLAB xUnit architecture is based closely on the xUnit style, in which each test case is an instance of a subclass of the base TestCase class. Programmers who are familiar with this style may want to write their own TestCase subclasses instead of using subfunction-based tests.

This example shows a TestCase subclass containing test case methods and test fixture methods. If you are not familiar with defining your own classes in MATLAB, you might want to review the MATLAB documentation on classes and object-oriented programming, or you can simply stick to using subfunction-based tests.

The sample M-file begins with the classdef statement, which sets the name of the class and indicates that it is a subclass of TestCase.

cd examples_general\ndbtype TestUsingTestCase 1\n
\n1     classdef TestUsingTestCase < TestCase\n\n

The properties block contains a field that is initialized by the setup method and is used by the two test methods.

dbtype TestUsingTestCase 3:5\n
\n3         properties\n4             fh\n5         end\n\n

The first method in the methods block is the constructor. It takes the desired test method name as its input argument, and it passes that input along to the base class constructor.

dbtype TestUsingTestCase 7:10\n
\n7         methods\n8             function self = TestUsingTestCase(name)\n9                 self = self@TestCase(name);\n10            end\n\n

The setUp method creates a figure window and stores its handle in the field fh.

dbtype TestUsingTestCase 12:14\n
\n12            function setUp(self)\n13                self.fh = figure;\n14            end\n\n

Test methods are those beginning with \"test\".

dbtype TestUsingTestCase 20:26\n
\n20            function testColormapColumns(self)\n21                assertEqual(size(get(self.fh, 'Colormap'), 2), 3);\n22            end\n23    \n24            function testPointer(self)\n25                assertEqual(get(self.fh, 'Pointer'), 'arrow');\n26            end\n\n

The tearDown method cleans up by deleting the figure window.

dbtype TestUsingTestCase 16:18\n
\n16            function tearDown(self)\n17                delete(self.fh);\n18            end\n\n

Run the test cases in the class by calling runtests with the name of the class.

runtests TestUsingTestCase\n
Starting test run with 2 test cases.\n..\nPASSED in 0.095 seconds.\n

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exTestCaseSearching.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How RUNTESTS Searches for Test Cases

MATLAB xUnit Test Framework: How RUNTESTS Searches for Test Cases

When you call runtests with no input arguments:

 >> runtests

it automatically searches for all the test cases in the current directory. It looks for test cases in three types of M-files:

1. An M-file function whose name begins or ends with \"test\" or \"Test\" and that does not return an output argument. Such a function is considered to be a single test case.

2. An M-file function whose name begins or ends with \"test\" or \"Test\" and that returns an output argument that is a test suite. Such a function is considered to contain subfunction-style test cases. Each subfunction whose name begins or ends with \"test\" or \"Test\" is a test case.

3. An M-file that defines a subclass of TestCase. Each method beginning or ending with \"test\" or \"Test\" is a test case.

runtests uses the TestSuite static methods fromName and fromPwd to automatically construct the test suites.

Here are a couple of examples.

TestSuite.fromName takes an M-file name, determines what kind of test file it is, and returns a cell array of test case objects.

cd examples_general\ntest_suite_1 = TestSuite.fromName('testSetupExample')\n
\ntest_suite_1 = \n\n  TestSuite handle\n\n  Properties:\n    TestComponents: {[1x1 FunctionHandleTestCase]  [1x1 FunctionHandleTestCase]}\n              Name: 'testSetupExample'\n          Location: [1x79 char]\n\n\n

TestSuite.fromPwd returns a test suite based on all the test files in the current directory.

test_suite_2 = TestSuite.fromPwd()\n
\ntest_suite_2 = \n\n  TestSuite handle\n\n  Properties:\n    TestComponents: {1x6 cell}\n              Name: 'C:\\Users\\eddins\\local-work\\matlab_xunit\\doc\\examples_general'\n          Location: 'C:\\Users\\eddins\\local-work\\matlab_xunit\\doc\\examples_general'\n\n\n

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.11

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exTestFixtures.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Write Tests That Share Common Set-Up Code

MATLAB xUnit Test Framework: How to Write Tests That Share Common Set-Up Code

Sometimes you want to write a set of test cases in which the same set of initialization steps is performed before each test case, or in which the same set of cleanup steps is performed after each test case. This set of common setup and teardown code is called a test fixture.

In subfunction-based test files, you can add subfunctions whose names begin with \"setup\" and \"teardown\". These functions will be called before and after every test-case subfunction is called. If the setup function returns an output argument, that value is saved and passed to every test-case subfunction and also to the teardown function.

This example shows a setup function that creates a figure and returns its handle. The figure handle is passed to each test-case subfunction. The figure handle is also passed to the teardown function, which cleans up after each test case by deleting the figure.

cd examples_general\ntype testSetupExample\n
\nfunction test_suite = testSetupExample\ninitTestSuite;\n\nfunction fh = setup\nfh = figure;\n\nfunction teardown(fh)\ndelete(fh);\n\nfunction testColormapColumns(fh)\nassertEqual(size(get(fh, 'Colormap'), 2), 3);\n\nfunction testPointer(fh)\nassertEqual(get(fh, 'Pointer'), 'arrow');\n\n

Run the tests using runtests.

runtests testSetupExample\n
Starting test run with 2 test cases.\n..\nPASSED in 0.095 seconds.\n

You might also want to see the example on writing test cases by subclassing TestCase.

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/html/exTolerance.html", "content": "\n\n\n \n MATLAB xUnit Test Framework: How to Test Using a Floating-Point Tolerance

MATLAB xUnit Test Framework: How to Test Using a Floating-Point Tolerance

MATLAB performs arithmetic operations using the floating-point hardware instructions on your processor. Because almost all floating-point operations are subject to round-off error, arithmetic operations can sometimes produce surprising results. Here's an example.

a = 1 + 0.1 + 0.1 + 0.1\n
\na =\n\n    1.3000\n\n
a == 1.3\n
\nans =\n\n     0\n\n

So why doesn't a equal 1.3? Because 0.1, 1.3, and most other decimal fractions do not have exact representations in the binary floating-point number representation your computer uses. The first line above is doing an approximate addition of 1 plus an approximation of 0.1, plus an approximation of 0.1, plus an approximation of 0.1. The second line compares the result of all that with an approximation of 1.3.

If you subtract 1.3 from a, you can see that the computed result for a is extremely close to the floating-point approximation of 1.3, but it is not exactly the same.

a - 1.3\n
\nans =\n\n  2.2204e-016\n\n

As a general rule, when comparing the results of floating-point calculations for equality, it is necessary to use a tolerance value. Two types of tolerance comparisons are commonly used: absolute tolerance and relative tolerance. An absolute tolerance comparison of a and b looks like:

\"$$|a-b|

A relative tolerance comparison looks like:

\"$$|a-b|

where Tf is called the floor tolerance. It acts as an absolute tolerance when a and b are very close to 0.

For example, suppose that a is 100, b is 101, and T is 0.1. Then a and b would not be considered equal using an absolute tolerance, because 1 > 0.1. However, a and b would be considered equal using a relative tolerance, because they differ by only 1 part in 100.

MATLAB xUnit provides the utility assertion functions called assertElementsAlmostEqual and assertVectorAlmostEqual. These functions make it easy to write tests involving floating-point tolerances.

assertElementsAlmostEqual(A,B) applies the tolerance test independently to every element of A and B. The function uses a relative tolerance test by default, but you make it use an absolute tolerance test, or change the tolerance values used, by passing additional arguments to it.

assertVectorsAlmostEqual(A,B) applies the tolerance test to the vectors A and B in the L2-norm sense. For example, suppose A is [1 1e10], B is [2 1e10], and the tolerance is 1e-8. Then A and B would fail an elementwise relative tolerance comparison, because the relative difference between the first elements is 0.5. However, they would pass a vector relative tolerance comparison, because the relative vector difference between A and B is only about 1 part in 1e10.

The examples_general directory contains a portion of a unit test for the sin function. The output of sin can sometimes be a bit surprising because of floating-point issues. For example:

sin(pi)\n
\nans =\n\n  1.2246e-016\n\n

That's very close but not exactly equal to 0. Here's how the sin unit test uses assertElementsAlmostEqual to write the sin(pi) test with a minimum of fuss.

cd examples_general\ntype testSin\n
\nfunction testSin\n\nassertElementsAlmostEqual(sin(pi), 0);\n

Run the test using runtests.

runtests testSin\n
Starting test run with 1 test case.\n.\nPASSED in 0.023 seconds.\n

Back to MATLAB xUnit Test Framework

Copyright 2008-2010 The MathWorks, Inc.
\n Published with MATLAB® 7.10

" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/index.html", "content": "\n \n \n \n \n \n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/release-history.html", "content": "\n \n MATLAB xUnit Release History\n \n \n \n

MATLAB xUnit Release History

\n \n

3.1 19-Nov-2010

\n
    \n
  • \n Add -logfile option to runtests.\n
    • \n
  • \n Allow test names to be passed to runtests as a cell array of strings.\n
    • \n
  • \n Add test suite name and execution date to output of runtests.\n
    • \n
  • \n Added warning message if function-handle-based test file has a setup function\n that returns more than one output argument.\n
    • \n
  • \n Fix bug related to handling subfunction test files in packages.\n
    • \n
  • \n Fix TestSuite.fromPackageName to find tests inside subpackages.\n
    • \n
  • \n Correct text in exTestCaseSearching.m to show that test files and functions\n can begin or end with \"test\" or \"Test\".\n
    • \n
\n \n

3.0.2 30-Jul-2010

\n
    \n
  • \n Fixed bug that caused TestCase subclasses in a test package to\n be executed twice in some versions of MATLAB.\n
    • \n
  • \n Documented the out = runsuite(...) syntax.\n
    • \n
  • \n Added home doc links to the various doc pages.\n
    • \n
\n \n

3.0.1 16-Jun-2010

\n

\n Fixed handling of TestCase subclasses in a test package.\n

\n \n

3.0 12-Jun-2010

\n
    \n
  • Added capability to run tests stored inside packages.
    • \n
  • runtests errors if no test cases are found instead\n of silently passing.
    • \n
  • Accept function names that end in \"test\" or \"Test\" as test functions.
    • \n
  • In assertElementsAlmostEqual and assertVectorsAlmostEqual, change the \n default floor_tol value to sqrt(eps) instead of eps. This makes the assertion\n a bit more forgiving when comparing numbers very close to 0.
    • \n
  • Added -verbose option to runtests.
    • \n
  • Fixed handling of message strings containing sprintf-style control characters\n in the assert*.m functions.
    • \n
\n \n

2.0.1 04-Aug-2009

\n

\n Corrected errors in assertElementsAlmostEqual and assertVectorsAlmostEqual\n related to NaN and Inf inputs. assertElementsAlmostEqual now properly\n asserts when one input is finite and the other is infinite.\n assertVectorsAlmostEqual now asserts whenever any input element is NaN\n or infinite. The behavior of xunit.utils.compareFloats has been changed\n to match.\n

\n \n

2.0 05-June-2009

\n
    \n
  • \n The name of the package has been changed to \"MATLAB xUnit Test Framework.\"\n The command-line test runner is now called runtests. The utilities\n package is now called xunit.utils. If you want to continue using\n the old command-line test runner (mtest) and utilities package\n (mtest.utils), then put the obsolete directory on the\n MATLAB path.\n
    • \n
  • \n The assertion functions assertEqual, assertElementsAlmostEqual,\n and assertVectorsAlmostEqual now print more information about the input\n values in the case of an assertion failure.\n
    • \n
  • \n A new assertion function, assertFilesEqual, has been added.\n
    • \n
  • \n The command-line test runner, runtests, now supports multiple directory\n names.\n
    • \n
  • \n The assertion function assertAlmostEqual has been deprecated. Use\n assertElementsAlmostEqual and assertVectorsAlmostEqual\n instead. If you want to continue using assertAlmostEqual, then\n put the obsolete directory on the MATLAB path.\n
    • \n
\n \n

1.1.3 20-May-2009

\n

\n Remove the LICENSE.txt file because the open source BSD license is\n now supplied automatically by the MATLAB Central File Exchange.\n There are no functional changes in this version.\n

\n \n

1.1.2 02-Apr-2009

\n

\n This release fixes a bug with assertVectorsAlmostEqual when\n the caller provided a custom message. When the function errored out\n because the tolerance was exceeded, the function would issue a\n different error message than expected.\n

\n \n

1.1.1 16-Mar-2009

\n

\n This release fixes a problem when calling mtest with no input\n arguments. Previously, it was not limiting its test-case discovery\n to TestCase subclasses and ordinary M-files beginning with \"test\" \n or \"Test\" as documented.\n

\n \n

\n This release also integrates the MTEST documentation with the MATLAB\n Help Browser.\n

\n\n

1.1 11-Mar-2009

\n

\n This release adds new tools for performing floating-point\n comparisons. Using the new assertion functions\n assertElementsAlmostEqual and \n assertVectorsAlmostEqual, you can perform both\n absolute and relative tolerance comparisons, either elementwise\n or in a vector L2-norm fashion.\n

\n\n

\n The previous floating-point comparison function,\n assertAlmostEqual, is still available, but its\n use is discouraged.\n

\n \n

1.0 30-Jan-2009

\n

\n This release, the first to be posted on the MATLAB Central File\n Exchange, includes a refactoring of TestCase and TestSuite to use\n the composite design pattern. Both classes now subclass the \n abstract class TestComponent, and the individual items contained\n in a TestSuite object are TestComponent objects. That means\n a TestSuite object can contain both TestSuite objects and \n TestCase objects. \n

\n \n

\n TestSuites are now built up hierarchically. All the test cases \n in a subfunction-based test M-file become a test suite, which in\n turn can be part of a test suite for an entire test directory.\n

\n \n

\n The mtest driver function can now take the name of a directory,\n in which case it will automatically discover and run all the test\n cases in that directory.\n

\n \n

\n The old TestRunObserver class has become the abstract TestRunMonitor\n class, with subclasses TestRunLogger and CommandWindowTestRunDisplay.\n

\n \n

\n TestCaseInDir has been modified to do a directory change before\n executing the test case. The new class TestCaseWithAddPath makes\n a temporary path addition before executing the test case.\n

\n \n

\n Subfunction-based test M-files written for one of the alpha versions\n of MTEST need to be revised so that the output variable name is \n \"test_suite\", and so that the first line of code calls the script\n \"initTestSuite\".\n

\n \n

0.9 12-Sep-2008

\n

\n This release is an extensive update that provides simpler ways of \n writing and running test cases.\n

\n
    \n
  • \n The new function mtest automatically finds and runs all test\n cases in the current directory.\n
    • \n
  • \n Test cases can be written as simple M-file functions.\n
    • \n
  • \n Multiple test cases can be defined in a single M-file by using\n subfunctions.\n
    • \n
  • \n Many new documentation examples have been provided, including a\n \"Quick Start\" example intended to enable users to write and\n run their first tests in just a few minutes.\n
    • \n
\n \n \n

0.8.1 17-Mar-2008

\n
    \n
  • \n Some of the sample TestCase classes were missing classdef lines. \n FIXED\n
    • \n
  • \n Now using dot method invocation syntax in examples and doc.\n
    • \n
  • \n Minor edits to HTML doc (munit_doc.html).\n
    • \n
  • \n Edited munit_doc.html by hand to clean up command-window links.\n
    • \n
\n \n

0.8 15-Mar-2008

\n
    \n
  • \n Limited initial distribution for review and comment.\n
    • \n
\n \n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/doc/xunit_product_page.html", "content": "\n \n MATLAB® xUnit Test Framework\n \n \n

MATLAB® xUnit Test Framework

\n\n

\n MATLAB xUnit is a unit test framework for MATLAB code. \n MATLAB xUnit is designed to be easy to use for MATLAB users with a \n wide range of experience. Users can write tests using \n ordinary M-files that are very simple in structure. \n

\n\n

\n Important Note: MATLAB xUnit \n relies heavily on object-oriented language features\n introduced in MATLAB 7.6 (R2008a), which was released in March 2008. \n MATLAB xUnit functions and classes will not work in earlier MATLAB releases.\n In addition, writing and running tests inside packages requires MATLAB\n 7.7 (R008b) or later.\n

\n \n\n \n \n

Installation

\n \n

\n To use MATLAB xUnit in MATLAB, add the \"xunit\" folder (directory) to the MATLAB path. See the \n MATLAB documentation for setting\n the search path. (The \"tests\" directory contains the \n framework's own self-tests, and the \"architecture\" directory contains\n information about the framework architecture; these directories are not\n needed for using MATLAB xUnit.)\n

\n \n

\n Note for users of earlier versions of MATLAB xUnit: If you have already\n written unit tests based on MTEST, an earlier version of MATLAB xUnit, you may also want \n to add the \"obsolete\" folder to the MATLAB path. This folder contains the old command-line\n test runner, mtest, as well as the deprecated function assertAlmostEqual.\n

\n \n

Getting Started

\n \n

\n Quick Start: How to Write and\n Run Tests\n

\n \n

\n How to Put Multiple Test\n Cases in One M-file\n

\n \n

\n How to Run a Specific Test\n

\n \n

\n How to Run Tests in Specific Directories\n

\n \n

\n How to Run Tests in a Package\n

\n \n

Advanced Usage

\n \n

\n How to Test Using a Floating-Point Tolerance\n

\n \n

\n How to Test an Error Message\n

\n \n

\n How to Run Tests Silently and Query the Results\n

\n \n

\n How to Write Tests That Share Common\n Set-Up Code\n

\n \n

\n How to Write xUnit-Style Tests by \n Subclassing TestCase\n

\n \n

\n How MATLAB xUnit Searches for Test \n Cases\n

\n \n

Key Functions and Classes

\n \n

\n Main test driver function:\n

\n

\n \n

\n Assertion functions you can use in your tests:\n

\n

\n \n

\n The key xUnit-style classes that make everything work:\n

\n

\n\n

\n Release History\n

\n \n

\n Copyright 2008-2010 The MathWorks, Inc.\n

\n \n \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/license.txt", "content": "Copyright (c) 2010, The MathWorks, Inc.\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without \nmodification, are permitted provided that the following conditions are \nmet:\n\n * Redistributions of source code must retain the above copyright \n notice, this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above copyright \n notice, this list of conditions and the following disclaimer in \n the documentation and/or other materials provided with the distribution\n * Neither the name of the The MathWorks, Inc. nor the names \n of its contributors may be used to endorse or promote products derived \n from this software without specific prior written permission.\n \nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" \nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE \nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE \nARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE \nLIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR \nCONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF \nSUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS \nINTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN \nCONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) \nARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE \nPOSSIBILITY OF SUCH DAMAGE.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/Contents.m", "content": "% UTILS Utility package for MTEST unit testing framework\n%\n% Array Comparison\n% compareFloats - Compare floating-point arrays using tolerance\n%\n% Test Case Discovery Functions\n% isTestCaseSubclass - True for name of TestCase subclass\n%\n% String Functions\n% containsRegexp - True if string contains regular expression\n% isSetUpString - True for string that looks like a setup function\n% isTearDownString - True for string that looks like teardown function\n% isTestString - True for string that looks like a test function\n%\n% Miscellaneous Functions\n% generateDoc - Publish test scripts in mtest/doc\n% parseFloatAssertInputs - Common input-parsing logic for several functions\n\n% Undocumented Functions\n% isAlmostEqual - Floating-point equality test using relative tolerance\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/compareFloats.m", "content": "function result = compareFloats(varargin)\n%compareFloats Compare floating-point arrays using tolerance.\n% result = compareFloats(A, B, compare_type, tol_type, tol, floor_tol)\n% compares the floating-point arrays A and B using a tolerance. compare_type\n% is either 'elementwise' or 'vector'. tol_type is either 'relative' or\n% 'absolute'. tol and floor_tol are the scalar tolerance values.\n%\n% There are four different tolerance tests used, depending on the comparison\n% type and the tolerance type:\n%\n% 1. Comparison type: 'elementwise' Tolerance type: 'relative'\n%\n% all( abs(A(:) - B(:)) <= tol * max(abs(A(:)), abs(B(:))) + floor_tol )\n%\n% 2. Comparison type: 'elementwise' Tolerance type: 'absolute'\n%\n% all( abs(A(:) - B(:) <= tol )\n%\n% 3. Comparison type: 'vector' Tolerance type: 'relative'\n%\n% norm(A(:) - B(:) <= tol * max(norm(A(:)), norm(B(:))) + floor_tol\n%\n% 4. Comparison type: 'vector' Tolerance type: 'absolute'\n%\n% norm(A(:) - B(:)) <= tol\n%\n% Note that floor_tol is not used when the tolerance type is 'absolute'.\n%\n% compare_type, tol_type, tol, and floor_tol are all optional inputs. The\n% default value for compare_type is 'elementwise'. The default value for\n% tol_type is 'relative'. If both A and B are double, then the default value\n% for tol is sqrt(eps), and the default value for floor_tol is eps. If either\n% A or B is single, then the default value for tol is sqrt(eps('single')), and\n% the default value for floor_tol is eps('single').\n%\n% If A or B is complex, then the tolerance test is applied independently to\n% the real and imaginary parts.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nif nargin >= 3\n % compare_type specified. Grab it and then use parseFloatAssertInputs to\n % process the remaining input arguments.\n compare_type = varargin{3};\n varargin(3) = [];\n if isempty(strcmp(compare_type, {'elementwise', 'vector'}))\n error('MTEST:compareFloats:unrecognizedCompareType', ...\n 'COMPARE_TYPE must be ''elementwise'' or ''vector''.');\n end\nelse\n compare_type = 'elementwise';\nend\n\nparams = mtest.utils.parseFloatAssertInputs(varargin{:});\n\nA = params.A(:);\nB = params.B(:);\n\n[A, B] = preprocessNanInf(A, B);\n\nswitch compare_type\n case 'elementwise'\n magFcn = @abs;\n \n case 'vector'\n magFcn = @norm;\n \n otherwise\n error('MTEST:compareFloats:unrecognizedCompareType', ...\n 'COMPARE_TYPE must be ''elementwise'' or ''vector''.');\nend\n\nswitch params.ToleranceType\n case 'relative'\n compareFcn = @(A, B) magFcn(A - B) <= ...\n params.Tolerance * max(magFcn(A), magFcn(B)) + ...\n params.FloorTolerance;\n \n case 'absolute'\n compareFcn = @(A, B) magFcn(A - B) <= params.Tolerance;\n \n otherwise\n error('MTEST:compareFloats:unrecognizedToleranceType', ...\n 'TOL_TYPE must be ''relative'' or ''absolute''.');\nend\n\nif isreal(A) && isreal(B)\n result = compareFcn(A, B);\nelse\n result = compareFcn(real(A), real(B)) & compareFcn(imag(A), imag(B));\nend\n\nresult = all(result);\n\n%===============================================================================\nfunction [A, B] = preprocessNanInf(A, B)\n\nmake_zero = isnan(A) & isnan(B);\nmake_zero = make_zero | ((A == Inf) & (B == Inf));\nmake_zero = make_zero | ((A == -Inf) & (B == -Inf));\n\nA(make_zero) = 0;\nB(make_zero) = 0;\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/containsRegexp.m", "content": "function tf = containsRegexp(str, exp)\n%containsRegexp True if string contains regular expression\n% TF = containsRegexp(str, exp) returns true if the string str contains the\n% regular expression exp. If str is a cell array of strings, then\n% containsRegexp tests each string in the cell array, returning the results in\n% a logical array with the same size as str.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\n% Convert to canonical input form: A cell array of strings.\nif ~iscell(str)\n str = {str};\nend\n\nmatches = regexp(str, exp);\ntf = ~cellfun('isempty', matches);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/generateDoc.m", "content": "function generateDoc\n%generateDoc Publish the example scripts in the doc directory\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\ndoc_dir = fullfile(fileparts(which('mtest')), '..', 'doc');\naddpath(doc_dir);\ncd(doc_dir)\nmfiles = dir('*.m');\nfor k = 1:numel(mfiles)\n publish(mfiles(k).name);\n cd(doc_dir)\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/isAlmostEqual.m", "content": "function same = isAlmostEqual(A, B, reltol)\n%isAlmostEqual Equality test using relative tolerance\n% same = isAlmostEqual(A, B, reltol), for two floating-point arrays A and B,\n% tests A and B for equality using the specified relative tolerance.\n% isAlmostEqual returns true if the following relationship is satisfied for\n% all values in A and B:\n%\n% abs(A - B) ./ max(abs(A), abs(B)) <= reltol\n%\n% same = isAlmostEqual(A, B) uses the following value for the relative\n% tolerance:\n%\n% 100 * max(eps(class(A)), eps(class(B)))\n%\n% If either A or B is not a floating-point array, then isAlmostEqual returns\n% the result of isequal(A, B).\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nif ~isfloat(A) || ~isfloat(B)\n same = isequal(A, B);\n return\nend\n\nif nargin < 3\n reltol = 100 * max(eps(class(A)), eps(class(B)));\nend\n\nif ~isequal(size(A), size(B))\n same = false;\n return\nend\n\nA = A(:);\nB = B(:);\n\ndelta = abs(A - B) ./ max(max(abs(A), abs(B)), 1);\n\n% Some floating-point values require special handling.\ndelta((A == 0) & (B == 0)) = 0;\ndelta(isnan(A) & isnan(B)) = 0;\ndelta((A == Inf) & (B == Inf)) = 0;\ndelta((A == -Inf) & (B == -Inf)) = 0;\n\nsame = all(delta <= reltol);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/isSetUpString.m", "content": "function tf = isSetUpString(str)\n%isSetUpString True if string looks like the name of a setup function\n% tf = isSetUpString(str) returns true if the string str looks like the name\n% of a setup function. If str is a cell array of strings, then isSetUpString\n% tests each string in the cell array, returning the results in a logical\n% array with the same size as str.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nsetup_exp = '^[sS]et[uU]p';\ntf = mtest.utils.containsRegexp(str, setup_exp);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/isTearDownString.m", "content": "function tf = isTearDownString(str)\n%isTearDownString True if string looks like the name of a teardown function\n% tf = isTearDownString(str) returns true if the string str looks like the\n% name of a teardown function. If str is a cell array of strings, then\n% isTearDownString tests each string in the cell array, returning the results\n% in a logical array with the same size as str.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nsetup_exp = '^[tT]ear[dD]own';\ntf = mtest.utils.containsRegexp(str, setup_exp);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/isTestCaseSubclass.m", "content": "function tf = isTestCaseSubclass(name)\n%isTestCaseSubclass True for name of a TestCase subclass\n% tf = isTestCaseSubclass(name) returns true if the string name is the name of\n% a TestCase subclass on the MATLAB path.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\ntf = false;\n\nclass_meta = meta.class.fromName(name);\nif isempty(class_meta)\n % Not the name of a class\n return;\nend\n\nif strcmp(class_meta.Name, 'TestCase')\n tf = true;\nelse\n tf = isMetaTestCaseSubclass(class_meta);\nend\n\nfunction tf = isMetaTestCaseSubclass(class_meta)\n\ntf = false;\n\nif strcmp(class_meta.Name, 'TestCase')\n tf = true;\nelse\n % Invoke function recursively on parent classes.\n super_classes = class_meta.SuperClasses;\n for k = 1:numel(super_classes)\n if isMetaTestCaseSubclass(super_classes{k})\n tf = true;\n break;\n end\n end\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/isTestString.m", "content": "function tf = isTestString(str)\n%isTestString True if string looks like the name of a test\n% tf = isTestString(str) returns true if the string str looks like the name of\n% a test. If str is a cell array of strings, then isTestString tests each\n% string in the cell array, returning the results in a logical array with the\n% same size as str.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\ntest_exp = '^[tT]est';\ntf = mtest.utils.containsRegexp(str, test_exp);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/+mtest/+utils/parseFloatAssertInputs.m", "content": "function params = parseFloatAssertInputs(varargin)\n%parseFloatAssertInputs Parse inputs for floating-point assertion functions.\n% params = parseFloatAssertInputs(varargin) parses the input arguments for\n% assertElementsAlmostEqual, assertVectorsAlmostEqual, and compareFcn. It\n% returns a parameter struct containing the fields:\n%\n% A B ToleranceType Tolerance FloorTolerance\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nerror(nargchk(2, 6, nargin, 'struct'));\n\nparams = struct('A', {[]}, 'B', {[]}, 'ToleranceType', {[]}, ...\n 'Tolerance', {[]}, 'FloorTolerance', {[]}, 'Message', {''});\n\n% The first two input arguments are always A and B.\nparams.A = varargin{1};\nparams.B = varargin{2};\nvarargin(1:2) = [];\n\n% If the last argument is a message string, process it and remove it from the list.\nif (numel(varargin) >= 1) && ischar(varargin{end}) && ...\n ~any(strcmp(varargin{end}, {'relative', 'absolute'}))\n params.Message = varargin{end};\n varargin(end) = [];\nend\n\ncheckAB(params.A, params.B);\n\nepsilon = max(eps(class(params.A)), eps(class(params.B)));\n\nif numel(varargin) < 3\n % floor_tol not specified; set default.\n params.FloorTolerance = epsilon;\nelse\n params.FloorTolerance = varargin{3};\nend\n\nif numel(varargin) < 2\n % tol not specified; set default.\n params.Tolerance = sqrt(epsilon);\nelse\n params.Tolerance = varargin{2};\nend\n\nif numel(varargin) < 1\n % tol_type not specified; set default.\n params.ToleranceType = 'relative';\nelse\n params.ToleranceType = varargin{1};\nend\n\n%===============================================================================\nfunction checkAB(A, B)\nif ~isfloat(A) || ~isfloat(B)\n error('MTEST:parseFloatAssertInputs:inputsNotFloat', ...\n 'A and B must be floating-point arrays.');\nend\n\nif ~isequal(size(A), size(B))\n error('MTEST:parseFloatAssertInputs:sizeMismatch', ...\n 'A and B must have the same size.');\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/assertAlmostEqual.m", "content": "function assertAlmostEqual(A, B, reltol, message)\n%assertEqual Assert that inputs are equal within relative tolerance\n% assertEqual(A, B, RELTOL) throws an exception of any of the values in A and\n% B are not equal within the specified tolerance. NaN values are considered\n% to be equal. A and B have to have the same class and sparsity to be\n% considered equal.\n%\n% assertEqual(A, B) uses the following relative tolerance value:\n%\n% 100 * eps(class(A))\n%\n% assertEqual(A, B, RELTOL, MESSAGE) uses the specified message string when\n% throwing the exception. With this syntax, use RELTOL = [] to specify the\n% default relative tolerance.\n%\n% Note that if either A or B are not floating-point arrays, then A and B are\n% compared using ISEQUALWITHEQUALNANS and the relative tolerance value is not\n% used. \n%\n% Examples\n% --------\n% % This call returns silently.\n% assertAlmostEqual(1.0, 1.0 + eps);\n%\n% % This call throws an error.\n% assertAlmostEqual(1.0, 1.1);\n%\n% See also assertEqual, mtest.utils.isAlmostEqual\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nif ~(issparse(A) == issparse(B))\n throw(MException('assertAlmostEqual:sparsityNotEqual', message));\nend\n\nif ~strcmp(class(A), class(B))\n throw(MException('assertAlmostEqual:classNotEqual', message));\nend\n\nif nargin < 3 || isempty(reltol)\n reltol = 100 * eps(class(A));\nend\n\nif nargin < 4\n message = sprintf('Inputs are not equal within relative tolerance: %g', ...\n reltol);\nend\n\nif ~mtest.utils.isAlmostEqual(A, B, reltol)\n throw(MException('assertAlmostEqual:tolExceeded', message));\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/mtest.m", "content": "function out = mtest(name)\n%mtest Run unit tests\n% mtest runs all the test cases that can be found in the current directory and\n% summarizes the results in the Command Window.\n%\n% Test cases can be found in the following places in the current directory:\n%\n% * An M-file function whose name starts with \"test\" or \"Test\" that\n% returns no output arguments.\n%\n% * An M-file function whose name starts with \"test\" or \"Test\" that\n% contains subfunction tests and uses the initTestSuite script to\n% return a TestSuite object.\n%\n% * An M-file defining a subclass of TestCase.\n%\n% mtest(mfilename) runs test cases found in the specified function or class\n% name. The function or class needs to be in the current directory or on the\n% MATLAB path.\n%\n% mtest('mfilename:testname') runs the specific test case named 'testname'\n% found in the function or class 'name'.\n%\n% mtest(dirname) runs all the test cases that can be found in the specified\n% directory.\n%\n% Examples\n% --------\n% Find and run all the test cases in the current directory.\n%\n% mtest\n%\n% Find and run all the test cases contained in the M-file myfunc.\n%\n% mtest myfunc\n%\n% Find and run all the test cases contained in the TestCase subclass\n% MyTestCase.\n%\n% mtest MyTestCase\n%\n% Run the test case named 'testFeature' contained in the M-file myfunc.\n%\n% mtest myfunc:testFeature\n%\n% Run all the tests in a specific directory.\n%\n% mtest c:\\Work\\MyProject\\tests\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nif nargin < 1\n suite = TestSuite.fromPwd();\nelse\n suite = TestSuite.fromName(name);\n \n user_gave_a_directory_name = isempty(suite.TestComponents) && ...\n (exist(name, 'file') == 7);\n if user_gave_a_directory_name\n % Before changing directories, arrange to restore the current directory\n % safely.\n currentDir = pwd;\n c = onCleanup(@() cd(currentDir));\n \n cd(name);\n suite = TestSuite.fromPwd();\n end\nend\n\ndid_pass = suite.run(CommandWindowTestRunDisplay());\n\nif nargout > 0\n out = did_pass;\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/tests/MtestTest.m", "content": "%TestSuiteTest Unit tests for mtest command-line test runner.\n\nclassdef MtestTest < TestCaseInDir\n\n methods\n \n function self = MtestTest(name)\n self = self@TestCaseInDir(name, ...\n fullfile(fileparts(which(mfilename)), 'cwd_test'));\n end\n \n function test_noInputArgs(self)\n [T, did_pass] = evalc('mtest');\n % The cwd_test directory contains some test cases that fail,\n % so output of mtest should be false.\n assertFalse(did_pass);\n end\n \n function test_oneInputArg(self)\n [T, did_pass] = evalc('mtest(''testFoobar'')');\n % cwd_test/testFoobar.m is supposed to pass.\n assertTrue(did_pass);\n end\n \n function test_oneInputArgWithFilter_passing(self)\n [T, did_pass] = evalc('mtest(''TestCaseSubclass:testA'')');\n assertTrue(did_pass);\n end\n \n function test_oneInputArgWithFilter_failing(self)\n [T, did_pass] = evalc('mtest(''TestCaseSubclass:testB'')');\n assertFalse(did_pass);\n end\n \n end\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/tests/cwd_test/TestCaseSubclass.m", "content": "%TestCaseSubclass TestCase subclass containing two passing tests\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\nclassdef TestCaseSubclass < TestCase\n methods\n function self = TestCaseSubclass(name)\n self = self@TestCase(name);\n end\n \n function testA(self)\n end\n \n function testB(self)\n % Intentionally fail this test case.\n assertFalse(true);\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/tests/cwd_test/testFoobar.m", "content": "function testFoobar\n%testFoobar Passing M-file test\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/tests/cwd_test/testSubfunctions.m", "content": "function test_cases = testSubfunctions\n%testSubfunctions Contains two passing subfunction tests\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\nfindSubfunctionTests;\n\nfunction testSub1\n\nfunction testSub2\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/tests/testAssertAlmostEqual.m", "content": "function test_suite = testAssertAlmostEqual\n%testAssertAlmostEqual Unit tests for assertAlmostEqual\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testEqual\nassertAlmostEqual(1, 1);\n\nfunction testEqualWithThreeInputs\nassertAlmostEqual(1, 1.1, 0.2);\n\nfunction testEqualWithFourInputs\nassertExceptionThrown(@() assertAlmostEqual(1, 2, 0.1, 'checkmate'), ...\n 'assertAlmostEqual:tolExceeded');\n\nfunction testEmptyRelTol\nassertAlmostEqual(1, 1+10*eps, [], 'checkmate');\n\nfunction testNotEqual\nassertExceptionThrown(@() assertAlmostEqual(1, 1+1000*eps), ...\n 'assertAlmostEqual:tolExceeded');\n\nfunction testSingleEqual\nassertAlmostEqual(single(1), single(1 + 10*eps('single')));\n\nfunction testSingleNotEqual\nassertExceptionThrown(@() assertAlmostEqual(single(1), ...\n single(1 + 1000*eps('single'))), 'assertAlmostEqual:tolExceeded');\n\nfunction testZeros\nassertAlmostEqual(0, 0);\n\nfunction testSingleZeros\nassertAlmostEqual(single(0), single(0));\n\nfunction testSparse\nassertAlmostEqual(sparse(1), sparse(1 + 10*eps));\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/obsolete/tests/testIsAlmostEqual.m", "content": "function test_suite = testIsAlmostEqual\n%testIsAlmostEqual Unit tests for isAlmostEqual\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testExactlyEqual\nA = [1 2; 3 4];\nB = [1 2; 3 4];\nassertTrue(mtest.utils.isAlmostEqual(A, B));\n\nfunction testDefaultTolerance\nassertTrue(mtest.utils.isAlmostEqual(1, 1+10*eps));\nassertFalse(mtest.utils.isAlmostEqual(1, 1+1000*eps));\n\nfunction testDefaultToleranceSingle\nassertTrue(mtest.utils.isAlmostEqual(single(1), 1 + 10*eps('single')));\nassertFalse(mtest.utils.isAlmostEqual(single(1), 1 + 1000*eps('single')));\n\nfunction testSpecifiedTolerance\nassertTrue(mtest.utils.isAlmostEqual(1, 1.09, 0.1));\nassertFalse(mtest.utils.isAlmostEqual(1, 1.2, 0.1));\n\nfunction testSpecialValues\nA = [Inf, -Inf, NaN, 2.0];\nB = [Inf, -Inf, NaN, 2.0+10*eps];\nassertTrue(mtest.utils.isAlmostEqual(A, B));\n\nC = [Inf, -Inf, NaN, 2.0];\nD = [Inf, -Inf, 0, 2.0+10*eps];\nassertFalse(mtest.utils.isAlmostEqual(C, D));\n\nfunction testUint8\nassertTrue(mtest.utils.isAlmostEqual(uint8(1), uint8(1)));\nassertFalse(mtest.utils.isAlmostEqual(uint8(1), uint8(2)));\n\nfunction testChar\nassertTrue(mtest.utils.isAlmostEqual('foobar', 'foobar'));\nassertFalse(mtest.utils.isAlmostEqual('foo', 'bar'));" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/readme", "content": "test\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/+xunit/+mocktests/+subpkg/test_a_bit.m", "content": "function test_suite = test_a_bit\ninitTestSuite\n\nfunction test_now\n\nfunction test_later\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/+xunit/+mocktests/A.m", "content": "% Class A is a TestCase subclass containing two test cases (test_a and test_b).\nclassdef A < TestCase\n \n methods\n function self = A(name)\n self = self@TestCase(name);\n end\n \n function test_a(self)\n end\n \n function test_b(self)\n end\n end\n \nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/+xunit/+mocktests/B.m", "content": "% Class B is not a TestCase subclass.\n\nclassdef B\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/+xunit/+mocktests/FooTest.m", "content": "classdef FooTest < TestCase\n methods\n function object = FooTest(name)\n object = object@TestCase(name);\n end\n function test_sanity(object)\n assertEqual(0, 0)\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/+xunit/+mocktests/helper_that.m", "content": "% helper_that is not a test file.\n\nfunction y = helper_that(x)\ny = x;\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/+xunit/+mocktests/test_that.m", "content": "% test_that.m is a subfunction test file.\nfunction test_suite = test_this\ninitTestSuite\n\nfunction test_the_other\na = magic(3);\n\nfunction test_nifty\nb = magic(5);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/+xunit/+mocktests/test_this.m", "content": "% test_this.m is a function-file test case.\nfunction test_this" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/Readme.m", "content": "% This directory contains the test suite for the mUnit test framework. Before\n% running the test suite, do the following:\n%\n% 1. Make sure the mUnit test framework directory is on your path\n% 2. Make sure the helper_classes subdirectory of the test directory is on\n% your path.\n% 3. Make the test directory your current directory.\n%\n% To run the test suite:\n%\n% run(TestSuite())\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks\n\nhelp Readme" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/RuntestsTest.m", "content": "%TestSuiteTest Unit tests for runtests command-line test runner.\n\nclassdef RuntestsTest < TestCaseInDir\n\n methods\n \n function self = RuntestsTest(name)\n self = self@TestCaseInDir(name, ...\n fullfile(fileparts(which(mfilename)), 'cwd_test'));\n end\n \n function test_noInputArgs(self)\n [T, did_pass] = evalc('runtests');\n % The cwd_test directory contains some test cases that fail,\n % so output of runtests should be false.\n assertFalse(did_pass);\n end\n \n function test_Verbose(self)\n [T, did_pass] = evalc('runtests(''-verbose'')');\n assertFalse(did_pass);\n end\n \n function test_oneInputArg(self)\n [T, did_pass] = evalc('runtests(''testFoobar'')');\n % cwd_test/testFoobar.m is supposed to pass.\n assertTrue(did_pass);\n end\n \n function test_verboseThenTestName(self)\n [T, did_pass] = evalc('runtests(''-verbose'', ''.'')');\n assertFalse(did_pass);\n end\n \n function test_testNameThenVerbose(self)\n [T, did_pass] = evalc('runtests(''.'', ''-verbose'')');\n assertFalse(did_pass);\n end\n \n function test_oneInputArgWithFilter_passing(self)\n [T, did_pass] = evalc('runtests(''TestCaseSubclass:testA'')');\n assertTrue(did_pass);\n end\n \n function test_oneInputArgWithFilter_failing(self)\n [T, did_pass] = evalc('runtests(''TestCaseSubclass:testB'')');\n assertFalse(did_pass);\n end\n \n function test_oneDirname(self)\n [T, did_pass] = evalc('runtests(''../dir1'')');\n assertTrue(did_pass);\n \n [T, did_pass] = evalc('runtests(''../dir2'')');\n assertFalse(did_pass);\n end\n \n function test_twoDirnames(self)\n [T, did_pass] = evalc('runtests(''../dir1'', ''../dir2'')');\n assertFalse(did_pass);\n end\n \n function test_packageName(self)\n [T, did_pass] = evalc('runtests(''xunit.mocktests'')');\n assertTrue(did_pass);\n end\n \n function test_noTestCasesFound(self)\n assertExceptionThrown(@() runtests('no_such_test'), ...\n 'xunit:runtests:noTestCasesFound');\n end\n \n function test_optionStringsIgnored(self)\n % Option string at beginning.\n [T, did_pass] = evalc('runtests(''-bogus'', ''../dir1'')');\n assertTrue(did_pass);\n \n % Option string at end.\n [T, did_pass] = evalc('runtests(''../dir2'', ''-bogus'')');\n assertFalse(did_pass);\n end\n \n function test_logfile(self)\n name = tempname;\n command = sprintf('runtests(''../dir1'', ''-logfile'', ''%s'')', name);\n [T, did_pass] = evalc(command);\n assertTrue(did_pass);\n assertTrue(exist(name, 'file') ~= 0);\n delete(name);\n end\n \n function test_logfileWithNoFile(self)\n assertExceptionThrown(@() runtests('../dir1', '-logfile'), ...\n 'xunit:runtests:MissingLogfile');\n end\n \n function test_logfileWithNoWritePermission(self)\n assertExceptionThrown(@() runtests('../dir1', '-logfile', ...\n 'C:\\dir__does__not__exist\\foobar.txt'), ...\n 'xunit:runtests:FileOpenFailed');\n end\n \n function test_namesInCellArray(self)\n [T, did_pass] = evalc('runtests({''TestCaseSubclass:testA''})');\n assertTrue(did_pass);\n \n [T, did_pass] = evalc('runtests({''TestCaseSubclass:testA'', ''TestCaseSubclass:testB''})');\n assertFalse(did_pass);\n end\n \n end\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/TestCaseTest.m", "content": "%TestCaseTest Unit tests for the TestCase class\n\n% Steven L. Eddins\n% Copyright The MathWorks 2008\n\nclassdef TestCaseTest < TestCaseInDir\n\n methods\n function self = TestCaseTest(name)\n self = self@TestCaseInDir(name, ...\n fullfile(fileparts(which(mfilename)), 'helper_classes'));\n end\n\n function testConstructor(self)\n % Exercise the constructor. Verify that the Name and Location\n % properties are set correctly.\n tc = TwoPassingTests('testMethod1');\n assertEqual(tc.Name, 'testMethod1');\n assertEqual(tc.Location, which('TwoPassingTests'));\n end\n\n function testPassingTests(self)\n % Verify that the expected observer notifications are received in\n % the proper order.\n logger = TestRunLogger();\n TestSuite('TwoPassingTests').run(logger);\n assertTrue(isequal(logger.Log, ...\n {'TestRunStarted', 'TestComponentStarted', ...\n 'TestComponentStarted', 'TestComponentFinished', ...\n 'TestComponentStarted', 'TestComponentFinished', ...\n 'TestComponentFinished', 'TestRunFinished'}));\n end\n\n function testFixtureCalls(self)\n % Verify that fixture calls are made in the proper order.\n tc = LoggingTestCase('testMethod');\n tc.run(TestRunLogger());\n assertTrue(isequal(tc.log, {'setUp', 'testMethod', 'tearDown'}));\n end\n\n function testTestFailure(self)\n % Verify that a test failure is recorded.\n logger = TestRunLogger();\n TestSuite('FailingTestCase').run(logger);\n assertTrue(isequal(logger.NumFailures, 1));\n end\n\n function testTestError(self)\n % Verify that a test error is recorded.\n logger = TestRunLogger();\n TestSuite('BadFixture').run(logger);\n assertTrue(isequal(logger.NumErrors, 1));\n end\n\n end\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/TestCaseWithAddPathTest.m", "content": "%TestCaseTest Unit tests for the TestCaseWithAddPath class\n\n% Steven L. Eddins\n% Copyright The MathWorks 2008\n\nclassdef TestCaseWithAddPathTest < TestCaseWithAddPath\n\n methods\n function self = TestCaseWithAddPathTest(name)\n self = self@TestCaseWithAddPath(name, ...\n fullfile(fileparts(which(mfilename)), 'helper_classes'));\n end\n\n function testPath(self)\n % Verify that a function in helper_classes is seen on the path.\n assertEqual(exist('testFunctionHandlesA', 'file'), 2);\n end\n \n function testRunTestOnPath(self)\n % Verify that we can make a test suite and run it using a file\n % in the new path directory.\n logger = TestRunLogger();\n suite = TestSuite('testFunctionHandlesA');\n did_pass = suite.run(logger);\n assertTrue(did_pass);\n end\n end\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/TestFuncHandleTests.m", "content": "%TestFuncHandleTests TeseCase class used to test function-handle-based tests\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\nclassdef TestFuncHandleTests < TestCaseInDir\n\n methods\n function self = TestFuncHandleTests(name)\n self = self@TestCaseInDir(name, ...\n fullfile(fileparts(which(mfilename)), 'helper_classes'));\n end\n \n function testSuiteNameAndLocation(self)\n test_suite = testFunctionHandlesA();\n assertEqual(test_suite.Name, 'testFunctionHandlesA');\n assertEqual(test_suite.Location, which('testFunctionHandlesA'));\n end\n\n function testOutputs(self)\n % Exercise the function-handle test M-file. Output should be a\n % two-element cell array of TestCase objects.\n test_suite = testFunctionHandlesA();\n assertTrue(isa(test_suite, 'TestSuite'));\n assertEqual(test_suite.numTestCases(), 2);\n end\n\n function testCaseNames(self)\n % Verify that Name property of test cases is set properly.\n test_suite = testFunctionHandlesA();\n assertEqual(test_suite.TestComponents{1}.Name, 'testA');\n assertEqual(test_suite.TestComponents{2}.Name, 'testB');\n end\n\n function testCaseLocation(self)\n % Verify that the Location field of test cases is set properly.\n test_suite = testFunctionHandlesA();\n expected_location = which('testFunctionHandlesA');\n assertEqual(test_suite.TestComponents{1}.Location, expected_location);\n assertEqual(test_suite.TestComponents{2}.Location, expected_location);\n end\n\n function testPassingTests(self)\n % Verify that the expected observer notifications are received in\n % the proper order.\n logger = TestRunLogger();\n suite = testFunctionHandlesA;\n suite.run(logger);\n assertEqual(logger.Log, ...\n {'TestRunStarted', 'TestComponentStarted', ...\n 'TestComponentStarted', 'TestComponentFinished', ...\n 'TestComponentStarted', 'TestComponentFinished', ...\n 'TestComponentFinished', 'TestRunFinished'});\n end\n\n function testTestFixture(self)\n % Verify that test fixture functions that use testData run without\n % error. (See test assertions in testFunctionHandlesB.)\n logger = TestRunLogger();\n suite = testFunctionHandlesB;\n suite.run(logger);\n assertEqual(logger.NumFailures, 0);\n assertEqual(logger.NumErrors, 0);\n end\n\n function testTestFixtureError(self)\n % Verify that an exception thrown in a test fixture is recorded as a\n % test error.\n logger = TestRunLogger();\n suite = testFunctionHandlesC();\n suite.run(logger);\n assertEqual(logger.NumErrors, 2);\n end\n\n function testFixtureNoTestData(self)\n % Verify that when setupFcn returns no output argument, the test\n % functions and the teardown function are called with no inputs.\n % (See test assertions in testFunctionHandlesD.)\n logger = TestRunLogger();\n suite = testFunctionHandlesD();\n suite.run(logger);\n assertEqual(logger.NumFailures, 0);\n assertEqual(logger.NumErrors, 0);\n end\n \n function testFailingTest(self)\n % Verify that the expected observer notifications are received in\n % the proper order for a failing test.\n logger = TestRunLogger();\n suite = testFunctionHandlesE();\n suite.run(logger);\n assertEqual(logger.Log, ...\n {'TestRunStarted', 'TestComponentStarted', ...\n 'TestComponentStarted', 'TestCaseFailure', 'TestComponentFinished', ...\n 'TestComponentFinished', 'TestRunFinished'});\n end\n \n function testTeardownFcnButNoSetupFcn(self)\n % Verify that a test file works if it has a teardown function but no\n % setup function.\n logger = TestRunLogger();\n suite = testFunctionHandlesTeardownNoSetup();\n suite.run(logger);\n \n assertEqual(logger.NumTestCases, 1);\n assertEqual(logger.NumFailures, 0);\n assertEqual(logger.NumErrors, 0);\n end\n\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/TestRunLoggerTest.m", "content": "%TestSuiteTest Unit tests for TestSuite class\n\nclassdef TestRunLoggerTest < TestCaseInDir\n\n methods\n function self = TestRunLoggerTest(name)\n self = self@TestCaseInDir(name, ...\n fullfile(fileparts(which(mfilename)), 'helper_classes'));\n end\n \n function testTwoPassingTests(self)\n logger = TestRunLogger;\n suite = TestSuite('TwoPassingTests');\n suite.run(logger);\n \n assertEqual(logger.Log, ...\n {'TestRunStarted', ...\n 'TestComponentStarted', ...\n 'TestComponentStarted', 'TestComponentFinished', ...\n 'TestComponentStarted', 'TestComponentFinished', ...\n 'TestComponentFinished', ...\n 'TestRunFinished'});\n \n assertEqual(logger.NumTestCases, 2);\n assertEqual(logger.NumFailures, 0);\n assertEqual(logger.NumErrors, 0);\n assertTrue(isempty(logger.Faults));\n end\n \n function testFailingTestCase(self)\n logger = TestRunLogger;\n suite = TestSuite('FailingTestCase');\n suite.run(logger);\n \n assertEqual(logger.Log, ...\n {'TestRunStarted', ...\n 'TestComponentStarted', ...\n 'TestComponentStarted', 'TestCaseFailure', 'TestComponentFinished', ...\n 'TestComponentFinished', ...\n 'TestRunFinished'});\n \n assertEqual(logger.NumTestCases, 1);\n assertEqual(logger.NumFailures, 1);\n assertEqual(logger.NumErrors, 0);\n assertEqual(numel(logger.Faults), 1);\n assertEqual(logger.Faults(1).Type, 'failure');\n end\n \n end\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/TestSuiteTest.m", "content": "%TestSuiteTest Unit tests for TestSuite class\n\nclassdef TestSuiteTest < TestCaseInDir\n\n methods\n function self = TestSuiteTest(name)\n self = self@TestCaseInDir(name, ...\n fullfile(fileparts(which(mfilename)), 'helper_classes'));\n end\n \n function testClassNameIn(self)\n % Syntax check: TestSuite('classname')\n suite = TestSuite('TwoPassingTests');\n assertTrue(numel(suite.TestComponents) == 2, ...\n 'TestSuite finds two test methods given class name');\n end\n \n function testCurrentDirectory(self)\n % See that the no-input syntax executes without error.\n % Not sure how to test this more effectively.\n suite = TestSuite();\n end\n \n function testNoTestMethods(self)\n % TestCase class containing no test methods\n suite = TestSuite('NoTestMethods');\n assertTrue(numel(suite.TestComponents) == 0, ...\n 'No test cases when class contains no test methods');\n end\n \n function test_fromTestCaseClassName(self)\n suite = TestSuite.fromTestCaseClassName('TwoPassingTests');\n assertTrue(numel(suite.TestComponents) == 2);\n assertTrue(ismember(suite.TestComponents{1}.Name, ...\n {'testMethod1', 'testMethod2'}));\n assertTrue(ismember(suite.TestComponents{2}.Name, ...\n {'testMethod1', 'testMethod2'})); \n end\n \n function test_fromTestCaseClassName_badclass(self)\n assertExceptionThrown(@() TestSuite.fromTestCaseClassName('atan2'), ...\n 'xunit:fromTestCaseClassName');\n end\n \n function test_fromName_TestCaseSubclass(self)\n suite = TestSuite.fromName('TwoPassingTests');\n assertTrue(numel(suite.TestComponents) == 2);\n assertEqual(suite.Name, 'TwoPassingTests');\n end\n \n function test_fromName_notTestCaseSubclass(self)\n suite = TestSuite.fromName('TestRunMonitor');\n assertTrue(isempty(suite.TestComponents));\n assertEqual(suite.Name, 'TestRunMonitor');\n end\n \n function test_fromName_simpleTest(self)\n suite = TestSuite.fromName('testSimple');\n assertEqual(numel(suite.TestComponents), 1);\n assertEqual(suite.Name, 'testSimple');\n assertEqual(suite.Location, which('testSimple'));\n end\n \n function test_fromName_subfunctions(self)\n suite = TestSuite.fromName('testFunctionHandlesA');\n assertEqual(numel(suite.TestComponents), 2);\n assertEqual(suite.Name, 'testFunctionHandlesA');\n assertEqual(suite.Location, which('testFunctionHandlesA'));\n end\n \n function test_fromName_bogus_name(self)\n suite = TestSuite.fromName('atan2');\n assertTrue(isempty(suite.TestComponents));\n assertEqual(suite.Name, 'atan2');\n end\n \n function test_fromName_with_filter_string(self)\n suite = TestSuite.fromName('testFunctionHandlesA:testA');\n assertEqual(numel(suite.TestComponents), 1);\n assertEqual(suite.TestComponents{1}.Name, 'testA');\n assertEqual(suite.Name, 'testFunctionHandlesA');\n end\n \n function test_fromName_with_nonmatching_filter_string(self)\n suite = TestSuite.fromName('testFunctionHandlesA:foobar');\n assertTrue(isempty(suite.TestComponents));\n end\n \n function test_fromName_with_dirname(self)\n xunit_test_dir = which('TestSuiteTest');\n xunit_test_dir = fileparts(xunit_test_dir);\n cwd_test_dir = fullfile(xunit_test_dir, 'cwd_test');\n suite = TestSuite.fromName(cwd_test_dir);\n \n assertEqual(suite.Name, 'cwd_test');\n assertEqual(suite.Location, cwd_test_dir);\n assertEqual(numel(suite.TestComponents), 3);\n end\n \n function test_fromPwd(self)\n % Verify that the fromPwd method returns a nonempty TestSuite object\n % from the helper_classes directory, with the correct number of\n % test components.\n suite = TestSuite.fromPwd();\n assertTrue(isa(suite, 'TestSuite'));\n assertTrue(numel(suite.TestComponents) == 16);\n end\n \n end\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/ThrowsExceptionTest.m", "content": "classdef ThrowsExceptionTest < TestCaseInDir\n \n methods\n function self = ThrowsExceptionTest(methodName)\n self = self@TestCaseInDir(methodName, ...\n fullfile(fileparts(which(mfilename)), 'helper_classes'));\n end\n \n function testPassingTest(self)\n logger = TestRunLogger();\n TestSuite('PassingExceptionTest').run(logger);\n assertTrue((logger.NumTestCases == 1) && ...\n (logger.NumFailures == 0) && ...\n (logger.NumErrors == 0), ...\n 'Passing exception test should have no failures or errors');\n end\n \n function testNoExceptionTest(self)\n logger = TestRunLogger();\n TestSuite('ExceptionNotThrownTest').run(logger);\n assertTrue(strcmp(logger.Faults(1).Exception.identifier, ...\n 'assertExceptionThrown:noException'), ...\n 'Fault exception should be throwsException:noException');\n end\n \n function testWrongExceptionTest(self)\n logger = TestRunLogger();\n TestSuite('WrongExceptionThrownTest').run(logger);\n assertTrue(strcmp(logger.Faults(1).Exception.identifier, ...\n 'assertExceptionThrown:wrongException'), ...\n 'Fault exception should be throwsException:wrongException');\n end\n \n end\n \n \nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/cwd_test/TestCaseSubclass.m", "content": "%TestCaseSubclass TestCase subclass containing two passing tests\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\nclassdef TestCaseSubclass < TestCase\n methods\n function self = TestCaseSubclass(name)\n self = self@TestCase(name);\n end\n \n function testA(self)\n end\n \n function testB(self)\n % Intentionally fail this test case.\n assertFalse(true);\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/cwd_test/testFoobar.m", "content": "function testFoobar\n%testFoobar Passing M-file test\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/cwd_test/testSubfunctions.m", "content": "function test_suite = testSubfunctions\n%testSubfunctions Contains two passing subfunction tests\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testSub1\n\nfunction testSub2\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/dir1/test_thatPasses.m", "content": "function test_suite = test_thatPasses\ninitTestSuite;\n\nfunction test_case\nassertTrue(true);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/dir2/test_thatFails.m", "content": "function test_suite = test_thatFails\ninitTestSuite;\n\nfunction test_case\nassertTrue(false);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/empty_file", "content": "" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/BadFixture.m", "content": "classdef BadFixture < TestCase\n \n methods\n function self = BadFixture(name)\n self = self@TestCase(name);\n end\n \n function setUp(self)\n throw(MException('setUpError:BadFixture', ...\n 'BadFixture setUp method always throws exception'));\n end\n \n function testMethod(self)\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/Contents.m", "content": "% Helper Classes for mUnit Test Suite\n%\n% TestCase Subclasses\n% BadFixture - Contains setUp method that throws exception\n% FailingTestCase - Contains one test method that throws exception\n% LoggingTestCase - Logs calls to setUp, tearDown, and test method\n% NoTestMethods - TestCase subclass that contains no test methods\n% TestsToBeDiscovered - Used in TestSuiteTest\n% TwoPassingTests - Contains two passing test methods\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/ExceptionNotThrownTest.m", "content": "classdef ExceptionNotThrownTest < TestCase\n methods\n function self = ExceptionNotThrownTest(methodName)\n self = self@TestCase(methodName);\n end\n \n function testThrowsException(self)\n f = @() [];\n assertExceptionThrown(f, 'a:b:c');\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/FailingTestCase.m", "content": "% FailingTestCase\n% Utility class used by unit tests.\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\nclassdef FailingTestCase < TestCase\n\n methods\n function self = FailingTestCase(name)\n self = self@TestCase(name);\n end\n\n function testFail(self)\n throw(MException('testFail:FailingTestCase', ...\n 'testFail always fails'));\n end\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/LoggingTestCase.m", "content": "% LoggingTestCase\n% Utility class used by unit tests.\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\nclassdef LoggingTestCase < TestCase\n \n properties\n log = {};\n end\n \n methods\n function self = LoggingTestCase(name)\n self = self@TestCase(name);\n end\n \n function setUp(self)\n self.log{end + 1} = 'setUp';\n end\n \n function tearDown(self)\n self.log{end + 1} = 'tearDown';\n end\n \n function testMethod(self)\n self.log{end + 1} = 'testMethod';\n end\n \n function testBrokenMethod(self)\n throw(MException('brokenMethod:WasRun', ...\n 'Call to testBrokenMethod always throws exception'));\n end\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/NoTestMethods.m", "content": "classdef NoTestMethods < TestCase\n methods\n function self = NoTestMethods(name)\n self = self@TestCase(name);\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/PassingExceptionTest.m", "content": "classdef PassingExceptionTest < TestCase\n methods\n function self = PassingExceptionTest(methodName)\n self = self@TestCase(methodName);\n end\n \n function testThrowsException(self)\n f = @() error('a:b:c', 'error message');\n assertExceptionThrown(f, 'a:b:c');\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/TestsToBeDiscovered.m", "content": "classdef TestsToBeDiscovered < TestCase\n\n methods\n function self = TestsToBeDiscovered(name)\n self = self@TestCase(name);\n end\n \n function testMethodA\n end\n \n function testMethodB\n end\n \n function notATestMethod\n end\n\n end\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/TwoPassingTests.m", "content": "classdef TwoPassingTests < TestCase\n \n methods\n function self = TwoPassingTests(name)\n self = self@TestCase(name);\n end\n \n function testMethod1(self)\n end\n \n function testMethod2(self)\n end\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/WrongExceptionThrownTest.m", "content": "classdef WrongExceptionThrownTest < TestCase\n methods\n function self = WrongExceptionThrownTest(methodName)\n self = self@TestCase(methodName);\n end\n \n function testThrowsException(self)\n f = @() error('d:e:f', 'message');\n assertExceptionThrown(f, 'a:b:c');\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/notTestString.m", "content": "function suite = notTestString\n% This function exists to help test that the TestSuite.fromPwd() method does not\n% pick up function-handle test files that do not match the naming convention.\ninitTestSuite;\n\nfunction testA\n\nfunction testB\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/testFunctionHandlesA.m", "content": "function test_suite = testFunctionHandlesA\n%testFunctionHandlesA Test file used by TestFunctionHandlesTest\n% Contains two passing tests.\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testA\n\nfunction testB\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/testFunctionHandlesB.m", "content": "function test_suite = testFunctionHandlesB\n%testFunctionHandlesB Test file used by TestFunctionHandlesTest\n% Contains two passing tests that use a test fixture.\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testData = setUpFcn\ntestData = 5;\n\nfunction testA(testData)\nassertEqual(testData, 5);\n\nfunction testB(testData)\nassertEqual(testData, 5);\n\nfunction tearDownFcn(testData)\nassertEqual(testData, 5);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/testFunctionHandlesC.m", "content": "function test_suite = testFunctionHandlesC\n%testFunctionHandlesC Test file used by TestFunctionHandlesTest\n% Contains two passing tests that use a test fixture containing an intentional\n% error.\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testData = setUpFcn\ntestData = 5;\n\nfunction testA(testData)\nassertEqual(testData, 5);\n\nfunction testB(testData)\nassertEqual(testData, 5);\n\nfunction tearDownFcn(testData)\n% This assertion is expected to error.\nassertEqual(testData, 20);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/testFunctionHandlesD.m", "content": "function test_suite = testFunctionHandlesD\n%testFunctionHandlesD Test file used by TestFunctionHandlesTest\n% Contains two passing tests that use a test fixture with no test data.\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction setUpFcn\n\n\nfunction testA(varargin)\nassertTrue(isempty(varargin));\n\nfunction testB(varargin)\nassertTrue(isempty(varargin));\n\nfunction tearDownFcn(varargin)\nassertTrue(isempty(varargin));\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/testFunctionHandlesE.m", "content": "function test_suite = testFunctionHandlesA\n%testFunctionHandlesE Test file used by TestFunctionHandlesTest\n% Contains one failing test.\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testA\nerror('testFunctionHandlesA:expectedFailure', 'Bogus message');\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/testFunctionHandlesTeardownNoSetup.m", "content": "function suite = testFunctionHandlesTeardownNoSetup\n% Verify that test file works if it has a teardown function but no setup\n% function.\ninitTestSuite;\n\nfunction teardown\nclose all\n\nfunction test_normalCase\nassertEqual(1, 1);\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/helper_classes/testSimple.m", "content": "function testSimple\n%testSimple Simple M-file test that passes\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testAssertEqual.m", "content": "function test_suite = testAssertEqual\n%testAssertEqual Unit tests for assertEqual\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testAssertEqualHappyCase\nassertEqual(5, 5);\n\nfunction testAssertEqualWithThreeInputs\nassertEqual(5, 5, 'Scandinavian Defense');\n\nfunction testAssertEqualHappyCaseString\nassertEqual('foobar', 'foobar');\n\nfunction testAssertEqualHappyCaseMatrix\nassertEqual(magic(3), magic(3))\n\nfunction testInfAndInf\nassertEqual(Inf, Inf);\n\nfunction testMinusInfAndMinusInf\nassertEqual(-Inf, -Inf);\n\nfunction testOppositeSignInfs\nassertExceptionThrown(@() assertEqual(-Inf, Inf), 'assertEqual:nonEqual');\n\nfunction testFiniteAndInf\nassertExceptionThrown(@() assertEqual(1, Inf), 'assertEqual:nonEqual');\n\nfunction testFiniteAndNaN\nassertExceptionThrown(@() assertEqual(1, NaN), 'assertEqual:nonEqual');\n\nfunction testInfiniteAndNaN\nassertExceptionThrown(@() assertEqual(Inf, NaN), 'assertEqual:nonEqual');\n\nfunction testAssertEqualNotEqual\nassertExceptionThrown(@() assertEqual(5, 4), 'assertEqual:nonEqual');\n\nfunction testAssertEqualSparsity\nassertExceptionThrown(@() assertEqual(5, sparse(5)), 'assertEqual:sparsityNotEqual');\n\nfunction testAssertEqualNans\nassertEqual([1 NaN 2], [1 NaN 2]);\n\nfunction testAssertEqualClass\nassertExceptionThrown(@() assertEqual(5, uint8(5)), 'assertEqual:classNotEqual');" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testAssertExceptionThrown.m", "content": "function test_suite = testAssertExceptionThrown\n%testAssertExceptionThrown Unit tests for assertExceptionThrown\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction test_happyCase\nassertExceptionThrown(...\n @() error('MyProd:MyFun:MyId', 'my message'), 'MyProd:MyFun:MyId');\n\nfunction test_wrongException\nassertExceptionThrown(@() assertExceptionThrown(...\n @() error('MyProd:MyFun:MyId', 'my message'), ...\n 'MyProd:MyFun:DifferentId'), 'assertExceptionThrown:wrongException');\n\nfunction test_noException\nassertExceptionThrown(@() assertExceptionThrown(@() sin(pi), 'foobar'), ...\n 'assertExceptionThrown:noException');\n\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testAssertFalse.m", "content": "function test_suite = testAssertFalse\n%testAssertFalse Unit tests for assertFalse\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testAssertFalseHappyCase\nassertFalse(false);\n\nfunction testAssertFalseHappyCaseWithTwoArgs\nassertFalse(false, '1.e4 e5 2.Nf3 Nc6');\n\nfunction testAssertFalseFailed\n% Verify exception when false is passed to assertFalse.\nassertExceptionThrown(@() assertFalse(true), 'assertFalse:trueCondition');\n\nfunction testAssertFalseNonscalar\n% Verify that assertFalse doesn't like nonscalar input.\nassertExceptionThrown(@() assertFalse(logical([0 0])), 'assertFalse:invalidCondition');\n\nfunction testAssertFalseNonlogical\n% Verify that assertFalse doesn't like nonlogical input.\nassertExceptionThrown(@() assertFalse(0), 'assertFalse:invalidCondition');\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testAssertTrue.m", "content": "function test_suite = testAssertTrue\n%testAssertTrue Unit tests for assertTrue\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testAssertTrueHappyCase\nassertTrue(true);\n\nfunction testAssertTrueHappyCaseWithTwoArgs\nassertTrue(true, '1.e4 e5 2.Nf3 Nc6');\n\nfunction testAssertTrueFailed\n% Verify exception when false is passed to assertTrue.\nassertExceptionThrown(@() assertTrue(false), 'assertTrue:falseCondition');\n\nfunction testAssertTrueNonscalar\n% Verify that assertTrue doesn't like nonscalar input.\nassertExceptionThrown(@() assertTrue(logical([1 1])), 'assertTrue:invalidCondition');\n\nfunction testAssertTrueNonlogical\n% Verify that assertTrue doesn't like nonlogical input.\nassertExceptionThrown(@() assertTrue(5), 'assertTrue:invalidCondition');\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testContainsRegexp.m", "content": "function test_suite = testContainsRegexp\n%testContainsRegexp Unit tests for containsRegexp\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testOneStringContains\nassertTrue(xunit.utils.containsRegexp('MATLAB is great', '[A-Z]'));\n\nfunction testOneStringDoesntContain\nassertTrue(~ xunit.utils.containsRegexp('no upper-case letters', '[A-Z]'));\n\nfunction testCellArray\nstrs = {'MATLAB is great', 'no upper-case letters'};\nassertEqual(xunit.utils.containsRegexp(strs, '[A-Z]'), [true false]);\nassertEqual(xunit.utils.containsRegexp(strs', '[A-Z]'), [true; false]);" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testIsSetUpString.m", "content": "function test_suite = testIsSetUpString\n%testIsSetUpString Unit tests for isSetUpString\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testOneStringIs\nassertTrue(xunit.utils.isSetUpString('setup'));\nassertTrue(xunit.utils.isSetUpString('setUp_fixture'));\n\nfunction testOneStringIsNot\nassertFalse(xunit.utils.isSetUpString('bogus'));\n\nfunction testCellArray\nstrs = {'setup', 'bogus'};\nassertEqual(xunit.utils.isSetUpString(strs), [true false]);\nassertEqual(xunit.utils.isSetUpString(strs'), [true; false]);" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testIsTearDownString.m", "content": "function test_suite = testIsTearDownString\n%testIsTearDownString Unit tests for isTearDownString\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testOneStringIs\nassertTrue(xunit.utils.isTearDownString('teardownfoobar'));\nassertTrue(xunit.utils.isTearDownString('TearDown_foobar'));\n\nfunction testOneStringIsNot\nassertFalse(xunit.utils.isTearDownString('tEardown'));\n\nfunction testCellArray\nstrs = {'teardown', 'tearup'};\nassertEqual(xunit.utils.isTearDownString(strs), [true false]);\nassertEqual(xunit.utils.isTearDownString(strs'), [true; false]);" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testIsTestCaseSubclass.m", "content": "function test_suite = testIsTestCaseSubclass\n%testIsTestCaseSubclass Unit tests for isTestCaseSubclass\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testTestCase\nassertTrue(xunit.utils.isTestCaseSubclass('TestCase'));\n\nfunction testSubclass\nassertTrue(xunit.utils.isTestCaseSubclass('TestCaseInDir'));\n\nfunction testNotASubclass\nassertFalse(xunit.utils.isTestCaseSubclass('atan2'));" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testIsTestString.m", "content": "function test_suite = testIsTestString\n%testIsTestString Unit tests for isTestString\n\n% Steven L. Eddins\n% Copyright 2008 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction testOneStringIs\nassertTrue(xunit.utils.isTestString('testFoobar'));\nassertTrue(xunit.utils.isTestString('Test_foobar'));\n\nfunction testOneStringIsNot\nassertFalse(xunit.utils.isTestString('foobar'));\n\nfunction testCellArray\nstrs = {'testFoobar', 'foobar_test', 'foobar', 'foobar_Test'};\nassertEqual(xunit.utils.isTestString(strs), [true true false true]);\nassertEqual(xunit.utils.isTestString(strs'), [true; true; false; true]);" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/testRuntestsWithDirectoryName.m", "content": "function test_suite = testRuntestsWithDirectoryName\n%testRuntestsWithDirectoryName Unit test for mtest('dirname') syntax.\n\ninitTestSuite;\n\nfunction testDirName\ncurrent_dir = pwd;\ntarget_dir = fullfile(fileparts(which(mfilename)), 'cwd_test');\n[T, did_pass] = evalc('runtests(target_dir)');\nassertFalse(did_pass);\nassertEqual(current_dir, pwd);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_TestSuiteInDir.m", "content": "function test_suite = test_TestSuiteInDir\n%test_TestSuiteInDir Unit test for TestSuiteInDir class.\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction test_constructor\nthis_test_path = fileparts(which(mfilename));\ncwd_test_dir = fullfile(this_test_path, 'cwd_test');\nsuite = TestSuiteInDir(cwd_test_dir);\n\nassertEqual(suite.Name, 'cwd_test');\nassertEqual(suite.Location, cwd_test_dir);\n\nfunction test_gatherTestCases\nthis_test_path = fileparts(which(mfilename));\ncwd_test_dir = fullfile(this_test_path, 'cwd_test');\nsuite = TestSuiteInDir(cwd_test_dir);\nsuite.gatherTestCases();\n\nassertEqual(numel(suite.TestComponents), 3);\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_arrayToString.m", "content": "function test_suite = test_arrayToString\n%test_arrayToString Unit test for arrayToString.\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction test_smallInput\nA = [1 2 3];\nassertEqual(strtrim(xunit.utils.arrayToString(A)), '1 2 3');\n\nfunction test_largeInput\nA = zeros(1000, 1000);\nassertEqual(xunit.utils.arrayToString(A), '[1000x1000 double]');\n\nfunction test_emptyInput\nassertEqual(xunit.utils.arrayToString(zeros(1,0,2)), '[1x0x2 double]');\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_assertElementsAlmostEqual.m", "content": "function suite = test_assertElementsAlmostEqual\ninitTestSuite;\n\n%===============================================================================\nfunction test_happyCase\n\n% All code here should execute with no error.\nassertElementsAlmostEqual(1, 1 + sqrt(eps)/10);\nassertElementsAlmostEqual(1, 1 + sqrt(eps)/10, 'custom message');\n\n%===============================================================================\nfunction test_failedAssertion\n\nf = @() assertElementsAlmostEqual(1, 1 + 10*sqrt(eps));\nassertExceptionThrown(f, 'assertElementsAlmostEqual:tolExceeded');\n\n%===============================================================================\nfunction test_nonFloatInputs()\nassertExceptionThrown(@() assertElementsAlmostEqual('hello', 'world'), ...\n 'assertElementsAlmostEqual:notFloat');\n\n%===============================================================================\nfunction test_sizeMismatch()\nassertExceptionThrown(@() assertElementsAlmostEqual(1, [1 2]), ...\n 'assertElementsAlmostEqual:sizeMismatch');\n\nfunction test_finiteAndInfinite()\nassertExceptionThrown(@() assertElementsAlmostEqual(1, Inf), ...\n 'assertElementsAlmostEqual:tolExceeded');\n\nfunction test_infiniteAndInfinite()\nassertElementsAlmostEqual(Inf, Inf);\n\nfunction test_finiteAndNaN()\nassertExceptionThrown(@() assertElementsAlmostEqual(1, NaN), ...\n 'assertElementsAlmostEqual:tolExceeded');\n\nfunction test_nanAndNaN()\nassertElementsAlmostEqual(NaN, NaN);\n\nfunction test_plusMinusInfinity()\nassertExceptionThrown(@() assertElementsAlmostEqual(+Inf, -Inf), ...\n 'assertElementsAlmostEqual:tolExceeded');\n\nfunction test_infiniteAndNaN()\nassertExceptionThrown(@() assertElementsAlmostEqual(Inf, NaN), ...\n 'assertElementsAlmostEqual:tolExceeded');\n\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_assertFilesEqual.m", "content": "function test_suite = test_assertFilesEqual\n%test_assertFilesEqual Unit test for assertFilesEqual\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction test_equal\nassertFilesEqual('black.tif', 'black.tif');\n\nfunction test_differentSize\nassertExceptionThrown(@() assertFilesEqual('black.tif', 'black.png'), ...\n 'assertFilesEqual:sizeMismatch');\n\nfunction test_sameSizeButDifferent\nassertExceptionThrown(@() assertFilesEqual('black.tif', 'almost_black.tif'), ...\n 'assertFilesEqual:valuesDiffer');\n\nfunction test_oneFileEmpty\nassertExceptionThrown(@() assertFilesEqual('empty_file', 'black.png'), ...\n 'assertFilesEqual:sizeMismatch');\n\nfunction test_bothFilesEmpty\nassertFilesEqual('empty_file', 'empty_file');\n\nfunction test_cannotReadFirstFile\nassertExceptionThrown(@() assertFilesEqual('bogus', 'black.png'), ...\n 'assertFilesEqual:readFailure');\n\nfunction test_cannotReadSecondFile\nassertExceptionThrown(@() assertFilesEqual('black.png', 'bogus'), ...\n 'assertFilesEqual:readFailure');\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_assertVectorsAlmostEqual.m", "content": "function suite = test_assertVectorsAlmostEqual\ninitTestSuite;\n\n%===============================================================================\nfunction test_happyCase\n\nA = [1 1e10];\nB = [2 1e10];\n% All code here should execute with no error.\nassertVectorsAlmostEqual(A, B);\nassertVectorsAlmostEqual(A, B, 'custom message');\n\n%===============================================================================\nfunction test_failedAssertion\n\nA = [1 1e6];\nB = [2 1e6];\n\nf = @() assertVectorsAlmostEqual(A, B);\nassertExceptionThrown(f, 'assertVectorsAlmostEqual:tolExceeded');\n\n%===============================================================================\nfunction test_failedAssertionWithCustomMessage\n\nA = [1 1e6];\nB = [2 1e6];\nf = @() assertVectorsAlmostEqual(A, B, 'my message');\nassertExceptionThrown(f, 'assertVectorsAlmostEqual:tolExceeded');\n\n%===============================================================================\nfunction test_nonFloatInputs()\nassertExceptionThrown(@() assertVectorsAlmostEqual('hello', 'world'), ...\n 'assertVectorsAlmostEqual:notFloat');\n\n%===============================================================================\nfunction test_sizeMismatch()\nassertExceptionThrown(@() assertVectorsAlmostEqual(1, [1 2]), ...\n 'assertVectorsAlmostEqual:sizeMismatch');\n\n%===============================================================================\nfunction test_finiteAndInfinite()\nassertExceptionThrown(@() assertVectorsAlmostEqual([1 2], [1 Inf]), ...\n 'assertVectorsAlmostEqual:tolExceeded');\n\n%===============================================================================\nfunction test_infiniteAndInfinite\nassertExceptionThrown(@() assertVectorsAlmostEqual([1 Inf], [1 Inf]), ...\n 'assertVectorsAlmostEqual:tolExceeded');\n\n%===============================================================================\nfunction test_finiteAndNaN\nassertExceptionThrown(@() assertVectorsAlmostEqual([1 2], [1 NaN]), ...\n 'assertVectorsAlmostEqual:tolExceeded');\n\n%===============================================================================\nfunction test_NanAndNan\nassertExceptionThrown(@() assertVectorsAlmostEqual([1 NaN], [1 NaN]), ...\n 'assertVectorsAlmostEqual:tolExceeded');\n\n%===============================================================================\nfunction test_oppositeSignInfs\nassertExceptionThrown(@() assertVectorsAlmostEqual([1 Inf], [1 -Inf]), ...\n 'assertVectorsAlmostEqual:tolExceeded');\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_compareFloats.m", "content": "function suite = test_compareFloats\ninitTestSuite;\n\n%===============================================================================\nfunction test_elementwiseRelativeTolerance\n\ntol = 0.1;\nfloor_tol = 0.01;\n\nassertTrue(xunit.utils.compareFloats([10 20], [11 20], 'elementwise', ...\n 'relative', tol, floor_tol));\nassertFalse(xunit.utils.compareFloats([10 20], [11.2 20], 'elementwise', ...\n 'relative', tol, floor_tol));\n\n% Verify floor tolerance\nassertTrue(xunit.utils.compareFloats([0.001 1], [0.010 1], 'elementwise', ...\n 'relative', tol, floor_tol));\n\n%===============================================================================\nfunction test_elementwiseAbsoluteTolerance\n\nassertTrue(xunit.utils.compareFloats([10 20], [10.1 20], 'elementwise', ...\n 'absolute', 0.1));\nassertFalse(xunit.utils.compareFloats([10 20], [10.1001 20], 'elementwise', ...\n 'absolute', 0.1));\n\n%===============================================================================\nfunction test_vectorRelativeTolerance\n\n% The A-B pair below would fail an elementwise test.\nA = [1 10];\nB = [1.5 10];\ntol = 0.05;\n\nassertTrue(xunit.utils.compareFloats(A, B, 'vector', 'relative', tol));\n\nB = [1.6 10];\nassertFalse(xunit.utils.compareFloats(A, B, 'vector', 'relative', tol));\n\n%===============================================================================\nfunction test_vectorAbsoluteTolerance\n\nA = [1 10];\nB = [1.4 10];\n\nassertTrue(xunit.utils.compareFloats(A, B, 'vector', 'absolute', 0.5));\nassertFalse(xunit.utils.compareFloats(A, B, 'vector', 'absolute', 0.3));\n\n%===============================================================================\nfunction test_NaNs\n\n% NaNs in the same spots are OK.\nA = [1 1 1 NaN 1 1 1 NaN 1];\nB = [1 1 1 NaN 1 1 1 NaN 1];\n\nassertTrue(xunit.utils.compareFloats(A, B));\n\n% NaNs in different spots are not OK.\nB2 = [1 1 NaN NaN 1 1 1 NaN 1];\nassertFalse(xunit.utils.compareFloats(A, B2));\n\n%===============================================================================\nfunction test_Infs\n\n% Infinities in the same locations are OK if they have the same sign.\nassertTrue(xunit.utils.compareFloats([1 2 3 Inf 4 5], [1 2 3 Inf 4 5]));\nassertTrue(xunit.utils.compareFloats([1 2 3 -Inf 4 5], [1 2 3 -Inf 4 5]));\nassertFalse(xunit.utils.compareFloats([1 2 3 Inf 4 5], [1 2 3 -Inf 4 5], ...\n 'elementwise', 'absolute'));\n\n%===============================================================================\nfunction test_complexInput\n\n% Real and imaginary parts are compared separately.\nassertTrue(xunit.utils.compareFloats(1, 1+0.09i, 'elementwise', 'absolute', 0.1));\nassertFalse(xunit.utils.compareFloats(1, 1+0.11i, 'elementwise', 'absolute', 0.1));\n\n%===============================================================================\nfunction test_comparisonTypeSpecified\n\n% Verify handling of third input argument, the comparison type. The rest of the\n% input syntax is handled by parseFloatAssertInputs and tested by the unit test\n% for that function.\n\n% The A-B pair below fails using elementwise comparison but passes using vector\n% comparison.\nA = [1.5 10];\nB = [1 10];\ntol = 0.1;\n\nassertFalse(xunit.utils.compareFloats(A, B, 'elementwise', 'relative', tol));\nassertTrue(xunit.utils.compareFloats(A, B, 'vector', 'relative', tol));\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_comparisonMessage.m", "content": "function test_suite = test_comparisonMessage\n%test_comparisonMessage Unit test for comparisonMessage.\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction test_happyCase\ns = xunit.utils.comparisonMessage('user message', 'assertion message', ...\n [1 2 3], 'hello');\nc = xunit.utils.stringToCellArray(s);\n\nexpected_output = { 'user message' \n 'assertion message' \n ''\n 'First input:'\n ' 1 2 3'\n ''\n 'Second input:'\n 'hello'};\n\nassertEqual(c, expected_output);\n\n " }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_packageName.m", "content": "function test_suite = test_packageName\ninitTestSuite;\n\nfunction test_happyCase\nsuite = TestSuite.fromPackageName('xunit.mocktests');\nassertEqual(numel(suite.TestComponents), 5);\n\nassertEqual(numel(suite.TestComponents{1}.TestComponents), 1);\nassertEqual(suite.TestComponents{1}.Name, 'xunit.mocktests.subpkg');\n\nassertEqual(numel(suite.TestComponents{2}.TestComponents), 2);\nassertEqual(suite.TestComponents{2}.Name, 'xunit.mocktests.A');\n\nassertEqual(numel(suite.TestComponents{3}.TestComponents), 1);\nassertEqual(suite.TestComponents{3}.Name, 'xunit.mocktests.FooTest');\n\nassertEqual(numel(suite.TestComponents{4}.TestComponents), 2);\nassertEqual(suite.TestComponents{4}.Name, 'test_that');\n\nassertEqual(numel(suite.TestComponents{5}.TestComponents), 1);\nassertEqual(suite.TestComponents{5}.Name, 'xunit.mocktests.test_this');\n\nfunction test_badPackageName\nassertExceptionThrown(@() TestSuite.fromPackageName('bogus'), ...\n 'xunit:fromPackageName:invalidName');\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_parseFloatAssertInputs.m", "content": "function suite = test_parseFloatAssertInputs\ninitTestSuite;\n\n%===============================================================================\nfunction test_tooFewInputs()\nassertExceptionThrown(@() xunit.utils.parseFloatAssertInputs(), ...\n 'MATLAB:nargchk:notEnoughInputs');\n\n%===============================================================================\nfunction test_tooManyInputs()\nassertExceptionThrown(@() xunit.utils.parseFloatAssertInputs(1,2,3,4,5,6,7), ...\n 'MATLAB:nargchk:tooManyInputs');\n\n%===============================================================================\nfunction test_twoInputs()\nparams = xunit.utils.parseFloatAssertInputs(1, 2);\nassertEqual(params.A, 1);\nassertEqual(params.B, 2);\nassertEqual(params.ToleranceType, 'relative');\nassertEqual(params.Tolerance, sqrt(eps));\nassertEqual(params.FloorTolerance, sqrt(eps));\nassertEqual(params.Message, '');\n\n%===============================================================================\nfunction test_threeInputs()\nexpected.A = 1;\nexpected.B = 2;\nexpected.ToleranceType = 'relative';\nexpected.Tolerance = sqrt(eps);\nexpected.FloorTolerance = sqrt(eps);\nexpected.Message = '';\n\nparams = xunit.utils.parseFloatAssertInputs(1, 2, 'relative');\nassertEqual(params, expected);\n\nparams = xunit.utils.parseFloatAssertInputs(1, 2, 'absolute');\nexpected.ToleranceType = 'absolute';\nassertEqual(params, expected);\n\nparams = xunit.utils.parseFloatAssertInputs(1, 2, 'message');\nexpected.ToleranceType = 'relative';\nexpected.Message = 'message';\nassertEqual(params, expected);\n\n%===============================================================================\nfunction test_fourInputs()\nexpected.A = 1;\nexpected.B = 2;\nexpected.ToleranceType = 'absolute';\nexpected.Tolerance = sqrt(eps);\nexpected.FloorTolerance = sqrt(eps);\nexpected.Message = '';\n\nparams = xunit.utils.parseFloatAssertInputs(1, 2, 'absolute', 0.1);\nexpected.Tolerance = 0.1;\nassertEqual(params, expected);\n\nparams = xunit.utils.parseFloatAssertInputs(1, 2, 'absolute', 'message');\nexpected.Tolerance = sqrt(eps);\nexpected.Message = 'message';\nassertEqual(params, expected);\n\n%===============================================================================\nfunction test_fiveInputs()\nexpected.A = 1;\nexpected.B = 2;\nexpected.ToleranceType = 'absolute';\nexpected.Tolerance = 0.1;\nexpected.FloorTolerance = 0.05;\nexpected.Message = '';\n\nparams = xunit.utils.parseFloatAssertInputs(1, 2, 'absolute', 0.1, 0.05);\nassertEqual(params, expected);\n\nparams = xunit.utils.parseFloatAssertInputs(1, 2, 'absolute', 0.1, 'message');\nexpected.FloorTolerance = sqrt(eps);\nexpected.Message = 'message';\nassertEqual(params, expected);\n\n%===============================================================================\nfunction test_sixInputs()\nexpected.A = 1;\nexpected.B = 2;\nexpected.ToleranceType = 'absolute';\nexpected.Tolerance = 0.1;\nexpected.FloorTolerance = 0.05;\nexpected.Message = 'message';\n\nparams = xunit.utils.parseFloatAssertInputs(1, 2, 'absolute', 0.1, 0.05, 'message');\nassertEqual(params, expected);\n\n%===============================================================================\nfunction test_twoSingleInputs()\nexpected.A = 1;\nexpected.B = 2;\nexpected.ToleranceType = 'relative';\nexpected.Tolerance = sqrt(eps('single'));\nexpected.FloorTolerance = sqrt(eps('single'));\nexpected.Message = '';\n\nparams = xunit.utils.parseFloatAssertInputs(single(1), single(2));\nassertEqual(params, expected);\n\n%===============================================================================\nfunction test_twoSingleAndDoubleInputs()\nexpected.A = 1;\nexpected.B = 2;\nexpected.ToleranceType = 'relative';\nexpected.Tolerance = sqrt(eps('single'));\nexpected.FloorTolerance = sqrt(eps('single'));\nexpected.Message = '';\n\nparams = xunit.utils.parseFloatAssertInputs(single(1), double(2));\nassertEqual(params, expected);\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/tests/test_stringToCellArray.m", "content": "function test_suite = test_stringToCellArray\n%test_stringToCellArray Unit test for stringToCellArray\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\ninitTestSuite;\n\nfunction test_happyCase\ns = sprintf('Hello\\nWorld');\nassertEqual(xunit.utils.stringToCellArray(s), {'Hello' ; 'World'});\n\nfunction test_emptyInput\nassertEqual(xunit.utils.stringToCellArray(''), cell(0, 1));\n\nfunction test_spacesInFront\ns = sprintf(' Hello\\n World\\n');\nassertEqual(xunit.utils.stringToCellArray(s), {' Hello' ; ' World'});\n\nfunction test_spacesAtEnd\ns = sprintf('Hello \\nWorld ');\nassertEqual(xunit.utils.stringToCellArray(s), {'Hello ' ; 'World '});\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/Contents.m", "content": "% UTILS Utility package for MATLAB xUnit Test Framework\n%\n% Array Comparison\n% compareFloats - Compare floating-point arrays using tolerance\n%\n% Test Case Discovery Functions\n% isTestCaseSubclass - True for name of TestCase subclass\n%\n% String Functions\n% arrayToString - Convert array to string for display\n% comparisonMessage - Assertion message string for comparing two arrays\n% containsRegexp - True if string contains regular expression\n% isSetUpString - True for string that looks like a setup function\n% isTearDownString - True for string that looks like teardown function\n% isTestString - True for string that looks like a test function\n% stringToCellArray - Convert string to cell array of strings\n%\n% Miscellaneous Functions\n% generateDoc - Publish test scripts in mtest/doc\n% parseFloatAssertInputs - Common input-parsing logic for several functions\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/arrayToString.m", "content": "function s = arrayToString(A)\n%arrayToString Convert array to string for display.\n% S = arrayToString(A) converts the array A into a string suitable for\n% including in assertion messages. Small arrays are converted using disp(A).\n% Large arrays are displayed similar to the way structure field values display\n% using disp.\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\nif isTooBigToDisp(A)\n s = dispAsStructField(A);\nelse\n s = dispAsArray(A);\nend\n\n%===============================================================================\nfunction tf = isTooBigToDisp(A)\n% Use a heuristic to determine if the array is to convert to a string using\n% disp. The heuristic is based on the size of the array in bytes, as reported\n% by the whos function.\n\nwhos_output = whos('A');\nbyte_threshold = 1000;\ntf = whos_output.bytes > byte_threshold;\n\n%===============================================================================\nfunction s = dispAsArray(A)\n% Convert A to a string using disp. Remove leading and trailing blank lines.\n\ns = evalc('disp(A)');\nif isempty(s)\n % disp displays nothing for some kinds of empty arrays.\n s = dispAsStructField(A);\nelse\n s = postprocessDisp(s);\nend\n\n%===============================================================================\nfunction s = dispAsStructField(A)\n% Convert A to a string using structure field display.\n\nb.A = A;\ns = evalc('disp(b)');\ns = postprocessStructDisp(s);\n\n%===============================================================================\nfunction out = postprocessDisp(in)\n% Remove leading and trailing blank lines from input string. Don't include a\n% newline at the end.\n\nlines = xunit.utils.stringToCellArray(in);\n\n% Remove leading blank lines.\nlines = removeLeadingBlankLines(lines);\n\n% Remove trailing blank lines.\nwhile ~isempty(lines) && isBlankLine(lines{end})\n lines(end) = [];\nend\n\n% Convert cell of strings to single string with newlines. Don't put a newline\n% at the end.\nout = sprintf('%s\\n', lines{1:end-1});\nout = [out, lines{end}];\n\n%===============================================================================\nfunction out = postprocessStructDisp(in)\n% Return the portion of the display string to the right of the colon in the\n% output of the first structure field. Input is a string.\n\nlines = xunit.utils.stringToCellArray(in);\n\n% Remove leading blank lines\nlines = removeLeadingBlankLines(lines);\n\nline = lines{1};\nidx = find(line == ':');\nout = line((idx+2):end); % struct fields display with blank space following colon\n\n%===============================================================================\nfunction out = removeLeadingBlankLines(in)\n% Input and output are cell arrays of strings.\n\nout = in;\nwhile ~isempty(out) && isBlankLine(out{1})\n out(1) = [];\nend\n\n%===============================================================================\nfunction tf = isBlankLine(line)\n% Input is a string.\n\ntf = all(isspace(line));\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/compareFloats.m", "content": "function result = compareFloats(varargin)\n%compareFloats Compare floating-point arrays using tolerance.\n% result = compareFloats(A, B, compare_type, tol_type, tol, floor_tol)\n% compares the floating-point arrays A and B using a tolerance. compare_type\n% is either 'elementwise' or 'vector'. tol_type is either 'relative' or\n% 'absolute'. tol and floor_tol are the scalar tolerance values.\n%\n% There are four different tolerance tests used, depending on the comparison\n% type and the tolerance type:\n%\n% 1. Comparison type: 'elementwise' Tolerance type: 'relative'\n%\n% all( abs(A(:) - B(:)) <= tol * max(abs(A(:)), abs(B(:))) + floor_tol )\n%\n% 2. Comparison type: 'elementwise' Tolerance type: 'absolute'\n%\n% all( abs(A(:) - B(:) <= tol )\n%\n% 3. Comparison type: 'vector' Tolerance type: 'relative'\n%\n% norm(A(:) - B(:) <= tol * max(norm(A(:)), norm(B(:))) + floor_tol\n%\n% 4. Comparison type: 'vector' Tolerance type: 'absolute'\n%\n% norm(A(:) - B(:)) <= tol\n%\n% Note that floor_tol is not used when the tolerance type is 'absolute'.\n%\n% compare_type, tol_type, tol, and floor_tol are all optional inputs. The\n% default value for compare_type is 'elementwise'. The default value for\n% tol_type is 'relative'. If both A and B are double, then the default value\n% for tol is sqrt(eps), and the default value for floor_tol is eps. If either\n% A or B is single, then the default value for tol is sqrt(eps('single')), and\n% the default value for floor_tol is eps('single').\n%\n% If A or B is complex, then the tolerance test is applied independently to\n% the real and imaginary parts.\n%\n% For elementwise comparisons, compareFloats returns true for two elements\n% that are both NaN, or for two infinite elements that have the same sign.\n% For vector comparisons, compareFloats returns false if any input elements\n% are infinite or NaN.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nif nargin >= 3\n % compare_type specified. Grab it and then use parseFloatAssertInputs to\n % process the remaining input arguments.\n compare_type = varargin{3};\n varargin(3) = [];\n if isempty(strcmp(compare_type, {'elementwise', 'vector'}))\n error('compareFloats:unrecognizedCompareType', ...\n 'COMPARE_TYPE must be ''elementwise'' or ''vector''.');\n end\nelse\n compare_type = 'elementwise';\nend\n\nparams = xunit.utils.parseFloatAssertInputs(varargin{:});\n\nA = params.A(:);\nB = params.B(:);\n\nswitch compare_type\n case 'elementwise'\n magFcn = @abs;\n \n case 'vector'\n magFcn = @norm;\n \n otherwise\n error('compareFloats:unrecognizedCompareType', ...\n 'COMPARE_TYPE must be ''elementwise'' or ''vector''.');\nend\n\nswitch params.ToleranceType\n case 'relative'\n coreCompareFcn = @(A, B) magFcn(A - B) <= ...\n params.Tolerance * max(magFcn(A), magFcn(B)) + ...\n params.FloorTolerance;\n \n case 'absolute'\n coreCompareFcn = @(A, B) magFcn(A - B) <= params.Tolerance;\n \n otherwise\n error('compareFloats:unrecognizedToleranceType', ...\n 'TOL_TYPE must be ''relative'' or ''absolute''.');\nend\n\nif strcmp(compare_type, 'elementwise')\n compareFcn = @(A, B) ( coreCompareFcn(A, B) | bothNaN(A, B) | sameSignInfs(A, B) ) & ...\n ~oppositeSignInfs(A, B) & ...\n ~finiteAndInfinite(A, B);\nelse\n compareFcn = @(A, B) coreCompareFcn(A, B) & ...\n isfinite(magFcn(A)) & ...\n isfinite(magFcn(B));\nend\n\nif isreal(A) && isreal(B)\n result = compareFcn(A, B);\nelse\n result = compareFcn(real(A), real(B)) & compareFcn(imag(A), imag(B));\nend\n\nresult = all(result);\n\n%===============================================================================\nfunction out = bothNaN(A, B)\n\nout = isnan(A) & isnan(B);\n\n%===============================================================================\nfunction out = oppositeSignInfs(A, B)\n\nout = isinf(A) & isinf(B) & (sign(A) ~= sign(B));\n\n%===============================================================================\nfunction out = sameSignInfs(A, B)\n\nout = isinf(A) & isinf(B) & (sign(A) == sign(B));\n\n%===============================================================================\nfunction out = finiteAndInfinite(A, B)\n\nout = xor(isinf(A), isinf(B));\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/comparisonMessage.m", "content": "function msg = comparisonMessage(user_message, assertion_message, A, B)\n%comparisonMessage Generate assertion message when comparing two arrays.\n% msg = comparisonMessage(user_message, assertion_message, A, B) returns a\n% string appropriate to use in a call to throw inside an assertion function\n% that compares two arrays A and B.\n%\n% The string returned has the following form:\n%\n% \n% \n%\n% First input:\n% \n%\n% Second input:\n% \n%\n% user_message can be the empty string, '', in which case user_message is\n% skipped.\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\nmsg = sprintf('%s\\n\\n%s\\n%s\\n\\n%s\\n%s', ...\n assertion_message, ...\n 'First input:', ...\n xunit.utils.arrayToString(A), ...\n 'Second input:', ...\n xunit.utils.arrayToString(B));\n\nif ~isempty(user_message)\n msg = sprintf('%s\\n%s', user_message, msg);\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/containsRegexp.m", "content": "function tf = containsRegexp(str, exp)\n%containsRegexp True if string contains regular expression\n% TF = containsRegexp(str, exp) returns true if the string str contains the\n% regular expression exp. If str is a cell array of strings, then\n% containsRegexp tests each string in the cell array, returning the results in\n% a logical array with the same size as str.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\n% Convert to canonical input form: A cell array of strings.\nif ~iscell(str)\n str = {str};\nend\n\nmatches = regexp(str, exp);\ntf = ~cellfun('isempty', matches);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/generateDoc.m", "content": "function generateDoc\n%generateDoc Publish the example scripts in the doc directory\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\ndoc_dir = fullfile(fileparts(which('runtests')), '..', 'doc');\naddpath(doc_dir);\ncd(doc_dir)\nmfiles = dir('*.m');\nfor k = 1:numel(mfiles)\n publish(mfiles(k).name);\n cd(doc_dir)\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/isAlmostEqual.m", "content": "function same = isAlmostEqual(A, B, reltol)\n%isAlmostEqual Equality test using relative tolerance\n% same = isAlmostEqual(A, B, reltol), for two floating-point arrays A and B,\n% tests A and B for equality using the specified relative tolerance.\n% isAlmostEqual returns true if the following relationship is satisfied for\n% all values in A and B:\n%\n% abs(A - B) ./ max(abs(A), abs(B)) <= reltol\n%\n% same = isAlmostEqual(A, B) uses the following value for the relative\n% tolerance:\n%\n% 100 * max(eps(class(A)), eps(class(B)))\n%\n% If either A or B is not a floating-point array, then isAlmostEqual returns\n% the result of isequal(A, B).\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nif ~isfloat(A) || ~isfloat(B)\n same = isequal(A, B);\n return\nend\n\nif nargin < 3\n reltol = 100 * max(eps(class(A)), eps(class(B)));\nend\n\nif ~isequal(size(A), size(B))\n same = false;\n return\nend\n\nA = A(:);\nB = B(:);\n\ndelta = abs(A - B) ./ max(max(abs(A), abs(B)), 1);\n\n% Some floating-point values require special handling.\ndelta((A == 0) & (B == 0)) = 0;\ndelta(isnan(A) & isnan(B)) = 0;\ndelta((A == Inf) & (B == Inf)) = 0;\ndelta((A == -Inf) & (B == -Inf)) = 0;\n\nsame = all(delta <= reltol);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/isSetUpString.m", "content": "function tf = isSetUpString(str)\n%isSetUpString True if string looks like the name of a setup function\n% tf = isSetUpString(str) returns true if the string str looks like the name\n% of a setup function. If str is a cell array of strings, then isSetUpString\n% tests each string in the cell array, returning the results in a logical\n% array with the same size as str.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nsetup_exp = '^[sS]et[uU]p';\ntf = xunit.utils.containsRegexp(str, setup_exp);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/isTearDownString.m", "content": "function tf = isTearDownString(str)\n%isTearDownString True if string looks like the name of a teardown function\n% tf = isTearDownString(str) returns true if the string str looks like the\n% name of a teardown function. If str is a cell array of strings, then\n% isTearDownString tests each string in the cell array, returning the results\n% in a logical array with the same size as str.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nsetup_exp = '^[tT]ear[dD]own';\ntf = xunit.utils.containsRegexp(str, setup_exp);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/isTestCaseSubclass.m", "content": "function tf = isTestCaseSubclass(name)\n%isTestCaseSubclass True for name of a TestCase subclass\n% tf = isTestCaseSubclass(name) returns true if the string name is the name of\n% a TestCase subclass on the MATLAB path.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\ntf = false;\n\nclass_meta = meta.class.fromName(name);\nif isempty(class_meta)\n % Not the name of a class\n return;\nend\n\nif strcmp(class_meta.Name, 'TestCase')\n tf = true;\nelse\n tf = isMetaTestCaseSubclass(class_meta);\nend\n\nfunction tf = isMetaTestCaseSubclass(class_meta)\n\ntf = false;\n\nif strcmp(class_meta.Name, 'TestCase')\n tf = true;\nelse\n % Invoke function recursively on parent classes.\n super_classes = class_meta.SuperClasses;\n for k = 1:numel(super_classes)\n if isMetaTestCaseSubclass(super_classes{k})\n tf = true;\n break;\n end\n end\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/isTestString.m", "content": "function tf = isTestString(str)\n%isTestString True if string looks like the name of a test\n% tf = isTestString(str) returns true if the string str looks like the name of\n% a test. If str is a cell array of strings, then isTestString tests each\n% string in the cell array, returning the results in a logical array with the\n% same size as str.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\ntest_at_beginning = '^[tT]est';\ntest_at_end = '[tT]est$';\n\ntf = xunit.utils.containsRegexp(str, test_at_beginning) | ...\n xunit.utils.containsRegexp(str, test_at_end);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/parseFloatAssertInputs.m", "content": "function params = parseFloatAssertInputs(varargin)\n%parseFloatAssertInputs Parse inputs for floating-point assertion functions.\n% params = parseFloatAssertInputs(varargin) parses the input arguments for\n% assertElementsAlmostEqual, assertVectorsAlmostEqual, and compareFcn. It\n% returns a parameter struct containing the fields:\n%\n% A B Message ToleranceType Tolerance FloorTolerance\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nerror(nargchk(2, 6, nargin, 'struct'));\n\nparams = struct('A', {[]}, 'B', {[]}, 'ToleranceType', {[]}, ...\n 'Tolerance', {[]}, 'FloorTolerance', {[]}, 'Message', {''});\n\n% The first two input arguments are always A and B.\nparams.A = varargin{1};\nparams.B = varargin{2};\nvarargin(1:2) = [];\n\n% If the last argument is a message string, process it and remove it from the list.\nif (numel(varargin) >= 1) && ischar(varargin{end}) && ...\n ~any(strcmp(varargin{end}, {'relative', 'absolute'}))\n params.Message = varargin{end};\n varargin(end) = [];\nelse\n params.Message = '';\nend\n\ntry\n epsilon = max(eps(class(params.A)), eps(class(params.B)));\ncatch\n epsilon = eps;\nend\n\nif numel(varargin) < 3\n % floor_tol not specified; set default.\n params.FloorTolerance = sqrt(epsilon);\nelse\n params.FloorTolerance = varargin{3};\nend\n\nif numel(varargin) < 2\n % tol not specified; set default.\n params.Tolerance = sqrt(epsilon);\nelse\n params.Tolerance = varargin{2};\nend\n\nif numel(varargin) < 1\n % tol_type not specified; set default.\n params.ToleranceType = 'relative';\nelse\n params.ToleranceType = varargin{1};\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/+xunit/+utils/stringToCellArray.m", "content": "function c = stringToCellArray(s)\n%stringToCellArray Convert string with newlines to cell array of strings.\n% C = stringToCellArray(S) converts the input string S to a cell array of\n% strings, breaking up S at new lines.\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\nif isempty(s)\n c = cell(0, 1);\nelse\n c = textscan(s, '%s', 'Delimiter', '\\n', 'Whitespace', '');\n c = c{1};\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/CommandWindowTestRunDisplay.m", "content": "classdef CommandWindowTestRunDisplay < TestRunDisplay\n %CommandWindowTestRunDisplay Print test suite execution results to Command Window.\n % CommandWindowTestRunDisplay is a subclass of TestRunMonitor. If a\n % CommandWindowTestRunDisplay object is passed to the run method of a\n % TestComponent, such as a TestSuite or a TestCase, it will print information\n % to the Command Window as the test run proceeds.\n %\n % CommandWindowTestRunDisplay methods:\n % testComponentStarted - Update Command Window display\n % testComponentFinished - Update Command Window display\n % testCaseFailure - Log test failure information\n % testCaseError - Log test error information\n %\n % CommandWindowTestRunDisplay properties:\n % TestCaseCount - Number of test cases executed\n % Faults - Struct array of test fault info\n %\n % See also TestRunLogger, TestRunMonitor, TestSuite\n \n % Steven L. Eddins\n % Copyright 2008-2010 The MathWorks, Inc.\n \n methods\n function self = CommandWindowTestRunDisplay\n self = self@TestRunDisplay(1);\n end\n end\n \nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/Contents.m", "content": "% MATLAB xUnit Test Framework\n% Version 3.1 (R2010b) 19-Nov-2010\n%\n% Running Unit Tests\n% runtests - Run unit tests\n%\n% Writing Unit Tests\n% assertElementsAlmostEqual - Assert floating-point array elements almost equal\n% assertEqual - Assert that inputs are equal\n% assertFilesEqual - Assert that two files have the same content\n% assertExceptionThrown - Assert that specified exception is thrown\n% assertFalse - Assert that input condition is false\n% assertTrue - Assert that input condition is true\n% assertVectorsAlmostEqual - Assert floating-point vectors almost equal in norm sense\n% initTestSuite - Utility script used for subfunction-based tests\n%\n% Framework Classes\n% CommandWindowTestRunDisplay - Print test suite results to command window\n% FunctionHandleTestCase - Test case based on a function handle\n% TestCase - Class defining interface for test cases\n% TestCaseInDir - Test case requiring temporary directory change\n% TestCaseWithAddPath - Test case requiring temporary path modification\n% TestComponent - Abstract base class for TestCase and TestSuite\n% TestComponentInDir - Test component requiring temporary directory change\n% TestLogger - Collect data (silently) from running test suite\n% TestRunDisplay - Print test suite execution results\n% TestRunMonitor - Abstract base class for monitoring test suite\n% TestSuite - Collection of TestComponent objects\n% TestSuiteInDir - Test suite requiring temporary directory change\n% %VerboseTestRunDisplay - Print test suite execution results\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/FunctionHandleTestCase.m", "content": "classdef FunctionHandleTestCase < TestCase\n%FunctionHandleTestCase Test case based on a function handle\n% FunctionHandleTestCase is a TestCase subclass. It defines a test case object\n% that executes by running a function handle instead of by running a method of\n% the TestCase subclass. \n%\n% FunctionHandleTestCase methods:\n% FunctionHandleTestCase - Constructor\n% runTestCase - Run function handle test \n% setUp - Run test-fixture setup function\n% tearDown - Run test-fixture teardown function\n%\n% FunctionHandleTestCase properties:\n% TestFcn - Function handle of test function\n% SetupFcn - Function handle of setup function\n% TeardownFcn - Function handle of teardown function\n% TestData - Data needed by test function or teardown function\n%\n% See also TestCase, TestSuite\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\n properties (SetAccess = protected, GetAccess = protected, Hidden = true)\n %TestFcn - Function handle of test function\n % If SetupFcn has one or more output arguments, then TestFcn is\n % called with this syntax:\n %\n % TestFcn(data)\n %\n % where data is the return value from SetupFcn. Otherwise, TestFcn is\n % called with no input and no output arguments.\n TestFcn;\n \n %SetupFcn - Function handle of setup function\n % If SetupFcn has one or more output arguments, then SetupFcn is\n % called with this syntax:\n %\n % data = SetupFcn()\n %\n % and data will be saved in the TestData property. Otherwise, SetupFcn\n % is called with no input and no output arguments.\n SetupFcn;\n \n %TeardownFcn - Function handle of teardown function\n % If SetupFcn has one or more output arguments, then TeardownFcn is\n % called with this syntax:\n %\n % TeardownFcn(data)\n %\n % were data is the return value from SetupFcn. Otherwise, TeardownFcn\n % is called with no input and no output arguments.\n TeardownFcn;\n \n %TestData - Data needed by test function or teardown function.\n TestData;\n end\n\n methods\n function self = FunctionHandleTestCase(testFcn, setupFcn, teardownFcn)\n %FunctionHandleTestCase Constructor\n % FunctionHandleTestCase(testFcn, setupFcn, teardownFcn) creates a\n % TestCase object that executes by running the function handle\n % TestFcn. setupFcn is a function handle that will be executed\n % before testFcn, and teardownFcn is a function handle that will\n % be executed after TestFcn. Either setupFcn or teardownFcn can\n % be empty.\n %\n % If setupFcn is function handle that has one output argument,\n % then the three test functions will be called using these\n % syntaxes:\n %\n % testData = setupFcn();\n % testFcn(testData);\n % teardownFcn(testData);\n %\n % Otherwise, the three test functions are all called with no input\n % arguments:\n %\n % setupFcn();\n % TestFcn();\n % teardownFcn();\n \n % Call the base class constructor. Give it the name of the\n % FunctionHandleTestCase method that executes TestFcn.\n self = self@TestCase('runTestCase');\n \n self.TestFcn = testFcn;\n self.SetupFcn = setupFcn;\n self.TeardownFcn = teardownFcn;\n\n % Determine the name and M-file location of the function handle.\n functionHandleInfo = functions(testFcn);\n self.Name = functionHandleInfo.function;\n if strcmp(functionHandleInfo.type, 'anonymous')\n % Anonymous function handles don't have an M-file location.\n self.Location = '';\n else\n self.Location = functionHandleInfo.file;\n end\n end\n\n function runTestCase(self)\n %runTestCase Run function handle test\n % test_case.run() calls the test function handle. If a nonempty\n % SetupFcn was provided and it has at least one output argument,\n % pass self.TestData to the test function. Otherwise, call the\n % test function with no input arguments.\n if ~isempty(self.SetupFcn) && nargout(self.SetupFcn) > 0\n self.TestFcn(self.TestData);\n else\n self.TestFcn();\n end\n end\n\n function setUp(self)\n %setUp Run test-fixture setup function\n % If a nonempty SetupFcn was provided, run it. If the SetupFcn\n % has at least one output argument, capture the first output\n % argument in instance data (TestData).\n if ~isempty(self.SetupFcn)\n if nargout(self.SetupFcn) > 0\n if nargout(self.SetupFcn) > 1\n message = sprintf(['A test fixture setup function returns more than one output argument. ', ...\n 'The test harness only calls the setup function with one output argument. ', ...\n 'Return a struct or a cell array from your setup function if you need to bundle several parts together.', ...\n '\\nTest name: %s\\nTest location: %s'], ...\n self.Name, self.Location);\n warning('xunit:FunctionHandleTestCase:TooManySetupOutputs', ...\n '%s', message);\n end\n self.TestData = self.SetupFcn();\n else\n self.SetupFcn();\n end\n end\n end\n\n function tearDown(self)\n %tearDown Run test-fixture teardown function\n % If a nonempty TeardownFcn was provided, run it. If there is\n % TestData (the output of the SetupFcn), then pass it to \n % TeardownFcn. Otherwise, call TeardownFcn with no input\n % arguments.\n if ~isempty(self.TeardownFcn)\n if ~isempty(self.SetupFcn) && (nargout(self.SetupFcn) > 0)\n self.TeardownFcn(self.TestData);\n else\n self.TeardownFcn();\n end\n end\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestCase.m", "content": "%TestCase Class defining interface for test cases\n% The TestCase class defines an individual test case.\n%\n% Normally a test writer will create their own test class that is a subclass\n% of TestCase. Each instance of the TestCase subclass that gets created will\n% be associated with a single test method.\n%\n% If a test fixture is needed, override the setUp() and tearDown() methods.\n%\n% TestSuite(subclass_name), where subclass_name is the name of a TestCase\n% subclass, creates a test suite containing one TestCase instance per test\n% method contained in the subclass.\n%\n% A simpler test-writing alternative to use subfunction-based M-file tests.\n% See the MATLAB xUnit documentation.\n%\n% TestCase methods:\n% TestCase - Constructor\n% run - Execute the test case\n%\n% TestCase properties:\n% Location - Location of M-file containing the test case\n% Name - Name of test case\n%\n% See also TestComponent, TestSuite\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n\nclassdef TestCase < TestComponent\n \n properties\n MethodName\n end\n\n methods\n function self = TestCase(testMethod)\n %TestCase Constructor\n % TestCase(methodName) constructs a TestCase object using the\n % specified testMethod (a string).\n \n self.MethodName = testMethod;\n self.Name = testMethod;\n self.Location = which(class(self));\n end\n \n function did_pass = run(self, monitor)\n %run Execute the test case\n % test_case.run(monitor) calls the TestCase object's setUp()\n % method, then the test method, then the tearDown() method.\n % observer is a TestRunObserver object. The testStarted(),\n % testFailure(), testError(), and testFinished() methods of\n % observer are called at the appropriate times. monitor is a\n % TestRunMonitor object. Typically it is either a TestRunLogger\n % subclass or a CommandWindowTestRunDisplay subclass.\n %\n % test_case.run() automatically uses a\n % CommandWindowTestRunDisplay object in order to print test\n % suite execution information to the Command Window.\n \n if nargin < 2\n monitor = CommandWindowTestRunDisplay();\n end\n \n did_pass = true;\n monitor.testComponentStarted(self);\n \n try\n self.setUp();\n f = str2func(self.MethodName);\n \n try\n % Call the test method.\n f(self);\n catch failureException\n monitor.testCaseFailure(self, failureException);\n did_pass = false;\n end\n \n self.tearDown();\n \n catch errorException\n monitor.testCaseError(self, errorException);\n did_pass = false;\n end\n \n monitor.testComponentFinished(self, did_pass);\n end\n \n function num = numTestCases(self)\n num = 1;\n end\n \n function print(self, numLeadingBlanks)\n if nargin < 2\n numLeadingBlanks = 0;\n end\n fprintf('%s%s\\n', blanks(numLeadingBlanks), self.Name);\n end\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestCaseInDir.m", "content": "%TestCaseInDir Test case requiring temporary directory change\n% The TestCaseInDir class defines a test case that has to be run by first\n% changing to a specified directory.\n%\n% The setUp method adds the starting directory to the path and then uses cd to \n% change into the specified directory. The tearDown method restores the\n% original path and directory.\n%\n% TestCaseInDir is used by MATLAB xUnit's own test suite in order to test itself.\n%\n% TestCaseInDir methods:\n% TestCaseInDir - Constructor\n%\n% See also TestCase, TestCaseWithAddPath, TestComponent\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nclassdef TestCaseInDir < TestCase & TestComponentInDir\n\n methods\n function self = TestCaseInDir(methodName, testDirectory)\n %TestCaseInDir Constructor\n % TestCaseInDir(testName, testDirectory) constructs a test case \n % using the specified name and located in the specified directory.\n self = self@TestCase(methodName);\n self = self@TestComponentInDir(testDirectory);\n end\n end\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestCaseWithAddPath.m", "content": "%TestCaseInDir Test case requiring temporary path modification\n% The TestCaseInDir class defines a test case that has to be run by first\n% adding a specific directory to the path.\n%\n% The setUp method adds the directory to the path, and the tearDown method\n% restores the original path.\n%\n% TestCaseWithAddPath is used by MATLAB xUnit's own test suite in order to test\n% itself. \n%\n% TestCaseWithAddPath methods:\n% TestCaseWithAddPath - Constructor\n% setUp - Add test directory to MATLAB path\n% tearDown - Restore original MATLAB path\n%\n% See also TestCase, TestCaseInDir\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nclassdef TestCaseWithAddPath < TestCase\n properties (SetAccess = private, GetAccess = private)\n %TestDirectory - Directory to be added to the path\n TestDirectory\n \n %OriginalPath - Path prior to adding the test directory\n OriginalPath\n end\n\n methods\n function self = TestCaseWithAddPath(methodName, testDirectory)\n %TestCaseInDir Constructor\n % TestCaseInDir(testName, testDirectory) constructs a test case \n % using the specified name and located in the specified directory.\n self = self@TestCase(methodName);\n self.TestDirectory = testDirectory;\n end\n\n function setUp(self)\n %setUp Add test directory to MATLAB path.\n % test_case.setUp() saves the current path in the OriginalPath\n % property and then adds the TestDirectory to the MATLAB path.\n self.OriginalPath = path;\n addpath(self.TestDirectory);\n end\n\n function tearDown(self)\n %tearDown Restore original MATLAB path\n % test_case.tearDown() restores the saved MATLAB path.\n path(self.OriginalPath);\n end\n end\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestComponent.m", "content": "classdef TestComponent < handle\n%TestComponent Abstract base class for TestCase and TestSuite\n%\n% TestComponent methods:\n% run - Run all test cases in test component\n% print - Display summary of test component to Command Window\n% numTestCases - Number of test cases in test component\n% setUp - Initialize test fixture\n% tearDown - Clean up text fixture\n%\n% TestComponent properties:\n% Name - Name of test component\n% Location - Directory where test component is defined\n%\n% See TestCase, TestSuite\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\n properties\n Name = '';\n Location = '';\n end\n \n properties (Access = 'protected')\n PrintIndentationSize = 4\n end\n \n methods (Abstract)\n print()\n %print Display summary of test component to Command Window\n % obj.print() displays information about the test component to the\n % Command Window.\n \n run()\n %run Execute test cases\n % obj.run() executes all the test cases in the test component\n \n numTestCases()\n %numTestCases Number of test cases in test component\n end\n \n methods\n function setUp(self)\n %setUp Set up test fixture\n % test_component.setUp() is called at the beginning of the run()\n % method. Test writers can override setUp if necessary to\n % initialize a test fixture.\n end\n \n function tearDown(self)\n %tearDown Tear down test fixture\n % test_component.tearDown() is at the end of the method. Test\n % writers can override tearDown if necessary to clean up a test\n % fixture.\n end\n \n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestComponentInDir.m", "content": "%TestComponentInDir Test component requiring temporary directory change\n% The TestComponentInDir class defines a test component that has to be run by\n% first changing to a specified directory.\n%\n% The setUp method adds the starting directory to the path and then uses cd to \n% change into the specified directory. The tearDown method restores the\n% original path and directory.\n%\n% TestComponentInDir methods:\n% TestComponentInDir - Constructor\n% setUp - Add test directory to MATLAB path\n% tearDown - Restore original MATLAB path\n%\n% See also TestComponent\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nclassdef TestComponentInDir < TestComponent\n properties (SetAccess = private, GetAccess = protected)\n %TestDirectory - Directory to change to in the test fixture\n TestDirectory\n \n %OriginalPath - Path prior to adding the starting directory\n OriginalPath\n \n %OriginalDirectory - Starting directory\n OriginalDirectory\n end\n\n methods\n function self = TestComponentInDir(testDirectory)\n %TestCaseInDir Constructor\n % TestCaseInDir(testName, testDirectory) constructs a test case \n % using the specified name and located in the specified directory.\n self.TestDirectory = testDirectory;\n end\n\n function setUp(self)\n %setUp Add test directory to MATLAB path\n % test_case.setUp() saves the current directory in the\n % OriginalDirectory property, saves the current path in the\n % OriginalPath property, and then uses cd to change into the test\n % directory.\n self.OriginalDirectory = pwd;\n self.OriginalPath = path;\n addpath(pwd);\n cd(self.TestDirectory);\n end\n\n function tearDown(self)\n %tearDown Restore original MATLAB path and directory\n % test_case.tearDown() restores the original path and directory.\n cd(self.OriginalDirectory);\n path(self.OriginalPath);\n end\n end\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestRunDisplay.m", "content": "classdef TestRunDisplay < TestRunMonitor\n%TestRunDisplay Print test suite execution results.\n% TestRunDisplay is a subclass of TestRunMonitor. If a TestRunDisplay\n% object is passed to the run method of a TestComponent, such as a\n% TestSuite or a TestCase, it will print information to the Command\n% Window (or specified file handle) as the test run proceeds.\n%\n% TestRunDisplay methods:\n% testComponentStarted - Update Command Window display\n% testComponentFinished - Update Command Window display\n% testCaseFailure - Log test failure information\n% testCaseError - Log test error information\n%\n% TestRunDisplay properties:\n% TestCaseCount - Number of test cases executed\n% Faults - Struct array of test fault info\n%\n% See also TestRunLogger, TestRunMonitor, TestSuite\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n \n properties (SetAccess = private)\n %TestCaseCount - Number of test cases executed\n TestCaseCount\n \n %Faults - Struct array of test fault info\n % Faults is a struct array with these fields:\n % Type - either 'failure' or 'error'\n % TestCase - the TestCase object that suffered the fault\n % Exception - the MException thrown when the fault occurred\n Faults = struct('Type', {}, 'TestCase', {}, 'Exception', {});\n \n end\n \n properties (SetAccess = private, GetAccess = private)\n %InitialTic - Out of tic at beginning of test run\n InitialTic\n \n %InitialComponent First test component executed\n % InitialComponent is set to the first test component executed in the\n % test run. This component is saved so that the end of the test run\n % can be identified.\n InitialComponent = [] \n \n end\n \n properties (Access = protected)\n %FileHandle - Handle used by fprintf for displaying results.\n % Default value of 1 displays to Command Window.\n FileHandle = 1\n end\n \n \n methods\n function self = TestRunDisplay(output)\n if nargin > 0\n if ischar(output)\n self.FileHandle = fopen(output, 'w');\n if self.FileHandle < 0\n error('xunit:TestRunDisplay:FileOpenError', ...\n 'Could not open file \"%s\" for writing.', ...\n filename);\n end\n else\n self.FileHandle = output;\n end\n end\n end\n \n function testComponentStarted(self, component)\n %testComponentStarted Update Command Window display\n % If the InitialComponent property is not yet set, \n % obj.testComponentStarted(component) sets the property and calls\n % obj.testRunStarted(component).\n \n if isempty(self.InitialComponent)\n self.InitialComponent = component;\n self.testRunStarted(component);\n end\n end \n \n function testComponentFinished(self, component, did_pass)\n %testComponentFinished Update Command Window display\n % If component is a TestCase object, then \n % obj.testComponentFinished(component, did_pass) prints pass/fail\n % information to the Command Window.\n %\n % If component is the InitialComponent, then\n % obj.testRunFinished(did_pass) is called.\n \n if isa(component, 'TestCase')\n self.TestCaseCount = self.TestCaseCount + 1;\n if did_pass\n fprintf(self.FileHandle, '.');\n else\n fprintf(self.FileHandle, 'F');\n end\n line_length = 20;\n if mod(self.TestCaseCount, line_length) == 0\n fprintf(self.FileHandle, '\\n');\n end\n end\n \n if isequal(component, self.InitialComponent)\n self.testRunFinished(did_pass);\n end\n end\n \n function testCaseFailure(self, test_case, failure_exception)\n %testCaseFailure Log test failure information\n % obj.testCaseFailure(test_case, failure_exception) logs the test\n % case failure information.\n \n self.logFault('failure', test_case, ...\n failure_exception);\n end\n \n function testCaseError(self, test_case, error_exception)\n %testCaseError Log test error information\n % obj.testCaseError(test_case, error_exception) logs the test\n % case error information.\n \n self.logFault('error', test_case, ...\n error_exception);\n end\n \n end\n \n methods (Access = protected)\n function testRunStarted(self, component)\n %testRunStarted Update Command Window display\n % obj.testRunStarted(component) displays information about the test\n % run to the Command Window.\n \n self.InitialTic = tic;\n self.TestCaseCount = 0;\n num_cases = component.numTestCases();\n if num_cases == 1\n str = 'case';\n else\n str = 'cases';\n end\n fprintf(self.FileHandle, 'Starting test run with %d test %s.\\n', ...\n num_cases, str);\n end\n \n function testRunFinished(self, did_pass)\n %testRunFinished Update Command Window display\n % obj.testRunFinished(component) displays information about the test\n % run results, including any test failures, to the Command Window.\n \n if did_pass\n result = 'PASSED';\n else\n result = 'FAILED';\n end\n \n fprintf(self.FileHandle, '\\n%s in %.3f seconds.\\n', result, toc(self.InitialTic));\n \n self.displayFaults();\n end\n \n\n \n function logFault(self, type, test_case, exception)\n %logFault Log test fault information\n % obj.logFault(type, test_case, exception) logs test fault\n % information. type is either 'failure' or 'error'. test_case is a\n % TestCase object. exception is an MException object.\n \n self.Faults(end + 1).Type = type;\n self.Faults(end).TestCase = test_case;\n self.Faults(end).Exception = exception;\n end\n \n function displayFaults(self)\n %displayFaults Display test fault info to Command Window\n % obj.displayFaults() displays a summary of each test failure and\n % test error to the command window.\n for k = 1:numel(self.Faults)\n faultData = self.Faults(k);\n if strcmp(faultData.Type, 'failure')\n str = 'Failure';\n else\n str = 'Error';\n end\n fprintf(self.FileHandle, '\\n===== Test Case %s =====\\nLocation: %s\\nName: %s\\n\\n', str, ...\n faultData.TestCase.Location, faultData.TestCase.Name);\n displayStack(filterStack(faultData.Exception.stack), ...\n self.FileHandle);\n fprintf(self.FileHandle, '\\n%s\\n', faultData.Exception.message);\n\n fprintf(self.FileHandle, '\\n');\n end\n end\n \n end\n \nend\n\nfunction displayStack(stack, file_handle)\n%displayStack Display stack trace from MException instance\n% displayStack(stack) prints information about an exception stack to the\n% command window. \n\nfor k = 1:numel(stack)\n filename = stack(k).file;\n linenumber = stack(k).line;\n href = sprintf('matlab: opentoline(''%s'',%d)', filename, linenumber);\n fprintf(file_handle, '%s at line %d\\n', filename, href, linenumber);\nend\nend\n\nfunction new_stack = filterStack(stack)\n%filterStack Remove unmeaningful stack trace calls\n% new_stack = filterStack(stack) removes from the input stack trace calls\n% that are framework functions and methods that are not likely to be\n% meaningful to the user.\n\n% Testing stack traces follow this common pattern:\n%\n% 1. The first function call in the trace is often one of the assert functions\n% in the framework directory. This is useful to see.\n%\n% 2. The next function calls are in the user-written test functions/methods and\n% the user-written code under test. These calls are useful to see.\n%\n% 3. The final set of function calls are methods in the various framework\n% classes. There are usually several of these calls, which clutter up the \n% stack display without being that useful.\n%\n% The pattern above suggests the following stack filtering strategy: Once the\n% stack trace has left the framework directory, do not follow the stack trace back\n% into the framework directory.\n\nmtest_directory = fileparts(which('runtests'));\nlast_keeper = numel(stack);\nhave_left_mtest_directory = false;\nfor k = 1:numel(stack)\n directory = fileparts(stack(k).file);\n if have_left_mtest_directory\n if strcmp(directory, mtest_directory)\n % Stack trace has reentered mtest directory.\n last_keeper = k - 1;\n break;\n end\n else\n if ~strcmp(directory, mtest_directory)\n have_left_mtest_directory = true;\n end\n end\nend\n\nnew_stack = stack(1:last_keeper);\n \nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestRunLogger.m", "content": "%TestRunLogger Collect data (silently) from running test suite\n% TestRunLogger is a subclass of TestRunMonitor uses to collect information \n% from an executing test component (either a test case or a test suite).\n% It maintains a record of event notifications received, as well as any test\n% failures or test errors.\n%\n% TestRunLogger methods:\n% testComponentStarted - Log test component started\n% testComponentFinished - Log test component finished\n% testCaseFailure - Log test case failure\n% testCaseError - Log test case error\n%\n% TestRunLogger properties:\n% Log - Cell array of test notification strings\n% NumFailures - Number of test failures during execution\n% NumErrors - Number of test errors during execution\n% NumTestCases - Total number of test cases executed\n% Faults - Struct array of test fault information\n%\n% See also CommandWindowTestRunDisplay, TestRunMonitor, TestSuite\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nclassdef TestRunLogger < TestRunMonitor\n\n properties (SetAccess = protected) \n %Log Cell array of test notification strings\n % Test notification strings include 'TestRunStarted',\n % 'TestRunFinished', 'TestComponentStarted', 'TestComponentFinished',\n % 'TestCaseFailure', and 'TestCaseError'.\n Log\n \n %NumFailures Number of test failures during execution\n NumFailures = 0\n \n %NumErrors Number of test errors during execution\n NumErrors = 0\n \n %NumTestCases Total number of test cases executed\n NumTestCases = 0\n \n %Faults Struct array of test fault information\n % Faults is a struct array with the fields Type, TestCase, and\n % Exception. Type is either 'failure' or 'error'. TestCase is the\n % test case object that triggered the fault. Exception is the\n % MException object thrown during the fault.\n Faults = struct('Type', {}, 'TestCase', {}, 'Exception', {});\n end\n \n properties (SetAccess = private, GetAccess = private)\n InitialTestComponent = []\n end\n\n methods\n \n function testComponentStarted(self, component)\n if isempty(self.InitialTestComponent)\n self.InitialTestComponent = component;\n self.appendToLog('TestRunStarted');\n end\n \n self.appendToLog('TestComponentStarted');\n \n if isa(component, 'TestCase')\n self.NumTestCases = self.NumTestCases + 1;\n end\n end\n \n function testComponentFinished(self, component, did_pass)\n self.appendToLog('TestComponentFinished');\n \n if isequal(component, self.InitialTestComponent)\n self.appendToLog('TestRunFinished');\n end\n end\n \n function testCaseFailure(self, test_case, failure_exception)\n self.appendToLog('TestCaseFailure');\n self.NumFailures = self.NumFailures + 1;\n self.logFault('failure', test_case, ...\n failure_exception);\n end\n \n function testCaseError(self, test_case, error_exception)\n self.appendToLog('TestCaseError');\n self.NumErrors = self.NumErrors + 1;\n self.logFault('error', test_case, ...\n error_exception);\n end\n end\n \n methods (Access = private)\n function appendToLog(self, item)\n self.Log{end+1} = item;\n end\n \n function logFault(self, type, test_case, exception)\n self.Faults(end + 1).Type = type;\n self.Faults(end).TestCase = test_case;\n self.Faults(end).Exception = exception;\n end\n end\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestRunMonitor.m", "content": "%TestRunMonitor Abstract base class for monitoring a running test suite\n% The abstract TestRunMonitor class defines an object that can observe and\n% record the results of running a test suite. The run() method of a\n% TestComponent object takes a TestRunMonitor object as an input argument.\n%\n% Different test suite logging or reporting functionality can be achieved by\n% subclassing TestRunMonitor. For example, see the TestRunLogger and the\n% CommandWindowTestRunDisplay classes.\n%\n% TestRunMonitor methods:\n% TestRunMonitor - Constructor\n% testComponentStarted - Called at beginning of test component run\n% testComponentFinished - Called when test component run finished\n% testCaseFailure - Called when a test case fails\n% testCaseError - Called when a test case causes an error\n%\n% See also CommandWindowTestRunDisplay, TestRunLogger, TestCase, TestSuite\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\nclassdef TestRunMonitor < handle\n\n methods (Abstract)\n \n testComponentStarted(self, component)\n \n testComponentFinished(self, component, did_pass)\n \n testCaseFailure(self, test_case, failure_exception)\n \n testCaseError(self, test_case, error_exception)\n \n end\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestSuite.m", "content": "%TestSuite Collection of TestComponent objects\n% The TestSuite class defines a collection of TestComponent objects.\n%\n% TestSuite methods:\n% TestSuite - Constructor\n% add - Add test component to test suite\n% print - Display test suite summary to Command Window\n% run - Run the test suite\n% keepMatchingTestCase - Keep only the named test component\n% fromName - Construct test suite from directory or MATLAB function file name\n% fromTestCaseClassName - Construct test suite from TestCase class name\n% fromPackageName - Construct test suite from package name\n% fromPwd - Construct test suite from present directory\n%\n% TestSuite properties:\n% TestComponents - Cell array of TestComponent objects\n%\n% Examples\n% --------\n% Run all the test cases in the SampleTests1 class. Display test suite\n% progress and a summary of results in the Command Window.\n%\n% TestSuite('SampleTests1').run()\n%\n% Construct a test suite from all test components found in the current\n% directory.\n%\n% suite = TestSuite.fromPwd();\n%\n% Construct a test suite from all test components found in the package\n% 'mytool.tests'. (Note that the \"+\" character at the beginning of the package\n% folder name on disk is not part of the package name.)\n%\n% suite = TestSuite.fromPackageName('mytool.tests');\n%\n% Run all the test cases in the SampleTests class. Display no output to the\n% Command Window. Upon completion, query the number of test failures and test\n% errors.\n%\n% logger = TestRunLogger();\n% TestSuite('SampleTests1').run(logger);\n% numFailures = logger.NumFailures\n% numErrors = logger.NumErrors\n%\n% See also CommandWindowTestRunDisplay, TestCase, TestComponent, TestRunLogger\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n\nclassdef TestSuite < TestComponent\n \n properties (SetAccess = protected)\n TestComponents = {};\n end\n \n methods\n \n function self = TestSuite(name)\n %TestSuite Constructor\n % suite = TestSuite constructs an empty test suite. suite =\n % TestSuite(name) constructs a test suite by searching for test\n % cases defined in an M-file with the specified name.\n \n if nargin >= 1\n self = TestSuite.fromName(name);\n end\n end\n \n function did_pass_out = run(self, monitor)\n %run Execute test cases in test suite\n % did_pass = suite.run() executes all test cases in the test\n % suite, returning a logical value indicating whether or not all\n % test cases passed.\n \n if nargin < 2\n monitor = CommandWindowTestRunDisplay();\n end\n \n monitor.testComponentStarted(self);\n did_pass = true;\n \n self.setUp();\n \n for k = 1:numel(self.TestComponents)\n this_component_passed = self.TestComponents{k}.run(monitor);\n did_pass = did_pass && this_component_passed;\n end\n \n self.tearDown();\n \n monitor.testComponentFinished(self, did_pass);\n \n if nargout > 0\n did_pass_out = did_pass;\n end\n end\n \n function num = numTestCases(self)\n %numTestCases Number of test cases in test suite\n \n num = 0;\n for k = 1:numel(self.TestComponents)\n component_k = self.TestComponents{k};\n num = num + component_k.numTestCases();\n end\n end\n \n function print(self, numLeadingBlanks)\n %print Display test suite summary to Command Window\n % test_suite.print() displays a summary of the test suite to the\n % Command Window.\n \n if nargin < 2\n numLeadingBlanks = 0;\n end\n fprintf('%s%s\\n', blanks(numLeadingBlanks), self.Name);\n for k = 1:numel(self.TestComponents)\n self.TestComponents{k}.print(numLeadingBlanks + ...\n self.PrintIndentationSize);\n end\n end\n \n function add(self, component)\n %add Add test component to test suite\n % test_suite.add(component) adds the TestComponent object to the\n % test suite.\n \n if iscell(component)\n self.TestComponents((1:numel(component)) + end) = component;\n else\n self.TestComponents{end + 1} = component;\n end\n end\n \n function keepMatchingTestCase(self, name)\n %keepMatchingTestCase Keep only the named test component\n % test_suite.keepMatchingTestCase(name) keeps only the test\n % component with a matching name and discards the rest.\n \n idx = [];\n for k = 1:numel(self.TestComponents)\n if strcmp(self.TestComponents{k}.Name, name)\n idx = k;\n break;\n end\n end\n if isempty(idx)\n self.TestComponents = {};\n else\n self.TestComponents = self.TestComponents(idx);\n end\n end\n \n end\n \n methods (Static)\n function suite = fromTestCaseClassName(class_name)\n %fromTestCaseClassName Construct test suite from TestCase class name\n % suite = TestSuite.fromTestCaseClassName(name) constructs a\n % TestSuite object from the name of a TestCase subclass.\n \n if ~xunit.utils.isTestCaseSubclass(class_name)\n error('xunit:fromTestCaseClassName', ...\n 'Input string \"%s\" is not the name of a TestCase class.', ...\n class_name);\n end\n \n suite = TestSuite;\n suite.Name = class_name;\n suite.Location = which(class_name);\n \n methods = getClassMethods(class_name);\n for k = 1:numel(methods)\n if methodIsConstructor(methods{k})\n continue\n end\n \n method_name = methods{k}.Name;\n if xunit.utils.isTestString(method_name)\n suite.add(feval(class_name, method_name));\n end\n end\n \n end\n \n function suite = fromName(name)\n %fromName Construct test suite from M-file name\n % test_suite = TestSuite.fromName(name) constructs a TestSuite\n % object from an M-file with the given name. The name can be of a\n % directory, a TestCase subclass, or an M-file containing a simple\n % test or containing subfunction-based tests.\n %\n % Optionally, name can contain a colon (':') followed by filter\n % string. The filter string is used to select a particular named\n % test case. For example, TestSuite.fromName('MyTests:testA')\n % constructs a TestSuite object containing only the test case\n % named 'testA' found in the TestCase subclass MyTests.\n \n if isdir(name)\n suite = TestSuiteInDir(name);\n suite.gatherTestCases();\n return;\n end\n \n [name, filter_string] = strtok(name, ':');\n if ~isempty(filter_string)\n filter_string = filter_string(2:end);\n end\n \n if xunit.utils.isTestCaseSubclass(name)\n suite = TestSuite.fromTestCaseClassName(name);\n \n elseif ~isempty(meta.class.fromName(name))\n % Input is the name of a class that is not a TestCase subclass.\n % Return an empty test suite.\n suite = TestSuite();\n suite.Name = name;\n \n elseif isPackage(name)\n suite = TestSuite.fromPackageName(name);\n \n else\n \n try\n if nargout(name) == 0\n suite = TestSuite();\n suite.Name = name;\n suite.add(FunctionHandleTestCase(str2func(name), [], []));\n suite.Location = which(name);\n \n else\n suite = feval(name);\n if ~isa(suite, 'TestSuite')\n error('Function did not return a TestSuite object.');\n end\n end\n \n catch\n % Ordinary function does not appear to contain tests.\n % Return an empty test suite.\n suite = TestSuite();\n suite.Name = name;\n end\n end\n \n if ~isempty(filter_string)\n suite.keepMatchingTestCase(filter_string);\n end\n end\n \n function test_suite = fromPwd()\n %fromPwd Construct test suite from present directory\n % test_suite = TestSuite.fromPwd() constructs a TestSuite object\n % from all the test components in the present working directory.\n % all TestCase subclasses will be found, as well as simple and\n % subfunction-based M-file tests beginning with the string 'test'\n % or 'Test'.\n \n test_suite = TestSuite();\n test_suite.Name = pwd;\n test_suite.Location = pwd;\n \n mfiles = dir(fullfile('.', '*.m'));\n for k = 1:numel(mfiles)\n [path, name] = fileparts(mfiles(k).name);\n if xunit.utils.isTestCaseSubclass(name)\n test_suite.add(TestSuite.fromTestCaseClassName(name));\n elseif xunit.utils.isTestString(name)\n suite_k = TestSuite.fromName(name);\n if ~isempty(suite_k.TestComponents)\n test_suite.add(suite_k);\n end\n end\n end\n end\n \n function test_suite = fromPackageName(name)\n %fromPackageName Construct test suite from package name\n % test_suite = TestSuite.fromPackageName(name) constructs a\n % TestSuite object from all the test components found in the\n % specified package.\n\n package_info = meta.package.fromName(name);\n if isempty(package_info)\n error('xunit:fromPackageName:invalidName', ...\n 'Input string \"%s\" is not the name of a package.', ...\n name);\n end\n test_suite = TestSuite();\n test_suite.Name = name;\n test_suite.Location = 'Package';\n \n for k = 1:numel(package_info.Packages)\n pkg_name = package_info.Packages{k}.Name;\n pkg_suite = TestSuite.fromPackageName(pkg_name);\n if ~isempty(pkg_suite.TestComponents)\n test_suite.add(TestSuite.fromPackageName(pkg_name));\n end\n end\n \n class_names = cell(1, numel(package_info.Classes));\n for k = 1:numel(package_info.Classes)\n class_name = package_info.Classes{k}.Name;\n class_names{k} = class_name;\n if xunit.utils.isTestCaseSubclass(class_name)\n test_suite.add(TestSuite.fromTestCaseClassName(class_name));\n end\n end\n \n for k = 1:numel(package_info.Functions)\n function_name = package_info.Functions{k}.Name;\n if xunit.utils.isTestString(function_name)\n full_function_name = [package_info.Name '.' package_info.Functions{k}.Name];\n if ~ismember(full_function_name, class_names)\n suite_k = TestSuite.fromName(full_function_name);\n if ~isempty(suite_k.TestComponents)\n test_suite.add(suite_k);\n end\n end\n end\n end\n end\n end\nend\n\nfunction tf = isPackage(name)\ntf = ~isempty(meta.package.fromName(name));\nend\n\nfunction methods = getClassMethods(class_name)\nclass_meta = meta.class.fromName(class_name);\nmethods = class_meta.Methods;\nend\n\nfunction result = methodIsConstructor(method)\nmethod_name = method.Name;\nif ~isempty(method.DefiningClass.ContainingPackage)\n method_name = [method.DefiningClass.ContainingPackage.Name, '.', ...\n method_name];\nend\nresult = strcmp(method_name, method.DefiningClass.Name);\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/TestSuiteInDir.m", "content": "%TestSuiteInDir Test suite requiring temporary directory change\n% The TestSuiteInDir class defines a test suite that has to be run by first\n% changing to a specified directory.\n%\n% The setUp method adds the starting directory to the path and then uses cd to\n% change into the specified directory. The tearDown method restores the\n% original path and directory.\n%\n% TestSuiteInDir methods:\n% TestSuiteInDir - Constructor\n% gatherTestCases - Add test cases found in the target directory\n%\n% See also TestSuite\n\n% Steven L. Eddins\n% Copyright 2009 The MathWorks, Inc.\n\nclassdef TestSuiteInDir < TestSuite & TestComponentInDir\n \n methods\n function self = TestSuiteInDir(testDirectory)\n %TestCaseInDir Constructor\n % TestCaseInDir(testName, testDirectory) constructs a test case\n % using the specified name and located in the specified directory.\n self = self@TestComponentInDir(testDirectory);\n \n if strcmp(testDirectory, '.')\n self.Name = pwd;\n self.Location = pwd;\n else\n [pathstr, name] = fileparts(testDirectory);\n self.Name = name;\n self.Location = testDirectory;\n end\n end\n \n function gatherTestCases(self)\n %gatherTestCases Add test cases found in the target directory\n % suite.gatherTestCases() automaticall finds all the test cases in\n % the directory specified in the constructor call and adds them to\n % the suite.\n current_dir = pwd;\n c = onCleanup(@() cd(current_dir));\n \n cd(self.TestDirectory);\n tmp = TestSuite.fromPwd();\n self.TestComponents = tmp.TestComponents;\n end\n end\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/VerboseTestRunDisplay.m", "content": "classdef VerboseTestRunDisplay < TestRunDisplay\n%VerboseTestRunDisplay Print test suite execution results.\n% VerboseTestRunDisplay is a subclass of\n% TestRunDisplay. It supports the -verbose option of runtests.\n%\n% Overriddent methods:\n% testComponentStarted - Update Command Window display\n% testComponentFinished - Update Command Window display\n% testRunFinished - Update Command Window display at end of run\n%\n% See also TestRunDisplay, TestRunLogger, TestRunMonitor, TestSuite\n\n% Steven L. Eddins\n% Copyright 2010 The MathWorks, Inc. \n \n properties (SetAccess = private, GetAccess = private)\n TicStack = uint64([])\n end\n \n methods\n function self = VerboseTestRunDisplay(output)\n if nargin < 1\n output = 1;\n end\n \n self = self@TestRunDisplay(output);\n end\n \n function testComponentStarted(self, component)\n %testComponentStarted Update Command Window display\n \n self.pushTic();\n \n if ~isa(component, 'TestCase')\n fprintf(self.FileHandle, '\\n');\n end\n \n fprintf(self.FileHandle, '%s%s', self.indentationSpaces(), component.Name);\n \n if ~isa(component, 'TestCase')\n fprintf(self.FileHandle, '\\n');\n else\n fprintf(self.FileHandle, ' %s ', self.leaderDots(component.Name));\n end\n end \n \n function testComponentFinished(self, component, did_pass)\n %testComponentFinished Update Command Window display\n\n if ~isa(component, 'TestCase')\n fprintf(self.FileHandle, '%s%s %s ', self.indentationSpaces(), component.Name, ...\n self.leaderDots(component.Name));\n end\n \n component_run_time = toc(self.popTic());\n \n if did_pass\n fprintf(self.FileHandle, 'passed in %12.6f seconds\\n', component_run_time);\n else\n fprintf(self.FileHandle, 'FAILED in %12.6f seconds\\n', component_run_time);\n end\n \n if ~isa(component, 'TestCase')\n fprintf(self.FileHandle, '\\n');\n end\n \n if isempty(self.TicStack)\n self.testRunFinished();\n end\n \n end\n \n end\n \n methods (Access = protected)\n function testRunFinished(self)\n %testRunFinished Update Command Window display\n % obj.testRunFinished(component) displays information about the test\n % run results, including any test failures, to the Command\n % Window.\n \n self.displayFaults();\n end\n end\n \n methods (Access = private)\n function pushTic(self)\n self.TicStack(end+1) = tic;\n end\n \n function t1 = popTic(self)\n t1 = self.TicStack(end);\n self.TicStack(end) = [];\n end\n \n function str = indentationSpaces(self)\n str = repmat(' ', 1, self.numIndentationSpaces());\n end\n \n function n = numIndentationSpaces(self)\n indent_level = numel(self.TicStack) - 1;\n n = 3 * indent_level;\n end\n \n function str = leaderDots(self, name)\n num_dots = max(0, 60 - self.numIndentationSpaces() - numel(name));\n str = repmat('.', 1, num_dots);\n end\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/assertElementsAlmostEqual.m", "content": "function assertElementsAlmostEqual(varargin)\n%assertElementsAlmostEqual Assert floating-point array elements almost equal.\n% assertElementsAlmostEqual(A, B, tol_type, tol, floor_tol) asserts that all\n% elements of floating-point arrays A and B are equal within some tolerance.\n% tol_type can be 'relative' or 'absolute'. tol and floor_tol are scalar\n% tolerance values.\n%\n% If the tolerance type is 'relative', then the tolerance test used is:\n%\n% all( abs(A(:) - B(:)) <= tol * max(abs(A(:)), abs(B(:))) + floor_tol )\n%\n% If the tolerance type is 'absolute', then the tolerance test used is:\n%\n% all( abs(A(:) - B(:)) <= tol )\n%\n% tol_type, tol, and floor_tol are all optional. The default value for\n% tol_type is 'relative'. If both A and B are double, then the default value\n% for tol and floor_tol is sqrt(eps). If either A or B is single, then the\n% default value for tol and floor_tol is sqrt(eps('single')).\n%\n% If A or B is complex, then the tolerance test is applied independently to\n% the real and imaginary parts.\n%\n% Corresponding elements in A and B that are both NaN, or are both infinite\n% with the same sign, are considered to pass the tolerance test.\n%\n% assertElementsAlmostEqual(A, B, ..., msg) prepends the string msg to the\n% output message if A and B fail the tolerance test.\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n\nparams = xunit.utils.parseFloatAssertInputs(varargin{:});\n\nif ~isequal(size(params.A), size(params.B))\n message = xunit.utils.comparisonMessage(params.Message, ...\n 'Inputs are not the same size.', ...\n params.A, params.B);\n throwAsCaller(MException('assertElementsAlmostEqual:sizeMismatch', ...\n '%s', message));\nend\n\nif ~(isfloat(params.A) && isfloat(params.B))\n message = xunit.utils.comparisonMessage(params.Message, ...\n 'Inputs are not both floating-point.', ...\n params.A, params.B);\n throwAsCaller(MException('assertElementsAlmostEqual:notFloat', ...\n '%s', message));\nend\n\nif ~xunit.utils.compareFloats(params.A, params.B, 'elementwise', ...\n params.ToleranceType, params.Tolerance, params.FloorTolerance)\n \n tolerance_message = sprintf('Input elements are not all equal within %s tolerance: %g', ...\n params.ToleranceType, params.Tolerance);\n message = xunit.utils.comparisonMessage(params.Message, tolerance_message, ...\n params.A, params.B);\n \n throwAsCaller(MException('assertElementsAlmostEqual:tolExceeded', ...\n '%s', message));\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/assertEqual.m", "content": "function assertEqual(A, B, custom_message)\n%assertEqual Assert that inputs are equal\n% assertEqual(A, B) throws an exception if A and B are not equal. A and B\n% must have the same class and sparsity to be considered equal.\n%\n% assertEqual(A, B, MESSAGE) prepends the string MESSAGE to the assertion\n% message if A and B are not equal.\n%\n% Examples\n% --------\n% % This call returns silently.\n% assertEqual([1 NaN 2], [1 NaN 2]);\n%\n% % This call throws an error.\n% assertEqual({'A', 'B', 'C'}, {'A', 'foo', 'C'});\n%\n% See also assertElementsAlmostEqual, assertVectorsAlmostEqual\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n\nif nargin < 3\n custom_message = '';\nend\n\nif ~ (issparse(A) == issparse(B))\n message = xunit.utils.comparisonMessage(custom_message, ...\n 'One input is sparse and the other is not.', A, B);\n throwAsCaller(MException('assertEqual:sparsityNotEqual', '%s', message));\nend\n\nif ~strcmp(class(A), class(B))\n message = xunit.utils.comparisonMessage(custom_message, ...\n 'The inputs differ in class.', A, B);\n throwAsCaller(MException('assertEqual:classNotEqual', '%s', message));\nend\n\nif ~isequalwithequalnans(A, B)\n message = xunit.utils.comparisonMessage(custom_message, ...\n 'Inputs are not equal.', A, B);\n throwAsCaller(MException('assertEqual:nonEqual', '%s', message));\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/assertExceptionThrown.m", "content": "function assertExceptionThrown(f, expectedId, custom_message)\n%assertExceptionThrown Assert that specified exception is thrown\n% assertExceptionThrown(F, expectedId) calls the function handle F with no\n% input arguments. If the result is a thrown exception whose identifier is\n% expectedId, then assertExceptionThrown returns silently. If no exception is\n% thrown, then assertExceptionThrown throws an exception with identifier equal\n% to 'assertExceptionThrown:noException'. If a different exception is thrown,\n% then assertExceptionThrown throws an exception identifier equal to\n% 'assertExceptionThrown:wrongException'.\n%\n% assertExceptionThrown(F, expectedId, msg) prepends the string msg to the\n% assertion message.\n%\n% Example\n% -------\n% % This call returns silently.\n% f = @() error('a:b:c', 'error message');\n% assertExceptionThrown(f, 'a:b:c');\n%\n% % This call returns silently.\n% assertExceptionThrown(@() sin, 'MATLAB:minrhs');\n%\n% % This call throws an error because calling sin(pi) does not error.\n% assertExceptionThrown(@() sin(pi), 'MATLAB:foo');\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n\nnoException = false;\ntry\n f();\n noException = true;\n \ncatch exception\n if ~strcmp(exception.identifier, expectedId)\n message = sprintf('Expected exception %s but got exception %s.', ...\n expectedId, exception.identifier);\n if nargin >= 3\n message = sprintf('%s\\n%s', custom_message, message);\n end\n throwAsCaller(MException('assertExceptionThrown:wrongException', ...\n '%s', message));\n end\nend\n\nif noException\n message = sprintf('Expected exception \"%s\", but none thrown.', ...\n expectedId);\n if nargin >= 3\n message = sprintf('%s\\n%s', custom_message, message);\n end\n throwAsCaller(MException('assertExceptionThrown:noException', '%s', message));\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/assertFalse.m", "content": "function assertFalse(condition, message)\n%assertFalse Assert that input condition is false\n% assertFalse(CONDITION, MESSAGE) throws an exception containing the string\n% MESSAGE if CONDITION is not false.\n%\n% MESSAGE is optional.\n%\n% Examples\n% --------\n% assertFalse(isreal(sqrt(-1)))\n%\n% assertFalse(isreal(sqrt(-1)), ...\n% 'Expected isreal(sqrt(-1)) to be false.')\n%\n% See also assertTrue\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n\nif nargin < 2\n message = 'Asserted condition is not false.';\nend\n\nif ~isscalar(condition) || ~islogical(condition)\n throwAsCaller(MException('assertFalse:invalidCondition', ...\n 'CONDITION must be a scalar logical value.'));\nend\n\nif condition\n throwAsCaller(MException('assertFalse:trueCondition', '%s', message));\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/assertFilesEqual.m", "content": "function assertFilesEqual(filename1, filename2, user_message)\n%assertFilesEqual Assert that files contain the same contents.\n% assertFilesEqual(filename1, filename2) throws an exception if the two\n% specified files do not contain the same contents.\n%\n% assertFilesEqual(filename1, filename2, message) prepends the specified\n% message string to the assertion message.\n\n% Steven L. Eddins\n% Copyright 2009-2010 The MathWorks, Inc.\n\nif nargin < 3\n user_message = '';\nend\n\nfid1 = fopen(filename1, 'r');\nif (fid1 < 0)\n message = sprintf('%s\\nCould not open file for reading: %s', ...\n user_message, filename1);\n throwAsCaller(MException('assertFilesEqual:readFailure', ...\n '%s', message));\nelse\n c1 = onCleanup(@() fclose(fid1));\nend\n\nfid2 = fopen(filename2, 'r');\nif (fid2 < 0)\n message = sprintf('%s\\nCould not open file for reading: %s', ...\n user_message, filename2);\n throwAsCaller(MException('assertFilesEqual:readFailure', '%s', message));\nelse\n c2 = onCleanup(@() fclose(fid2));\nend\n\nblock_size = 100000;\nnum_blocks = 0;\ndone = false;\nwhile ~done\n block_from_file1 = fread(fid1, block_size, '*uint8');\n block_from_file2 = fread(fid2, block_size, '*uint8');\n \n if numel(block_from_file1) ~= numel(block_from_file2)\n fseek(fid1, 0, 'eof');\n fseek(fid2, 0, 'eof');\n message = sprintf('The two files are not the same size. File \"%s\" has %d bytes and file \"%s\" has %d bytes', ...\n filename1, ftell(fid1), filename2, ftell(fid2));\n if ~isempty(user_message)\n message = sprintf('%s\\n%s', user_message, message);\n end\n throwAsCaller(MException('assertFilesEqual:sizeMismatch', '%s', message));\n end\n \n if ~isequal(block_from_file1, block_from_file2)\n first_difference_in_block = find(block_from_file1 ~= block_from_file2);\n first_difference = num_blocks * block_size + first_difference_in_block;\n \n message = sprintf('Files are not equal. First difference is at byte %d, where file \"%s\" contains 0x%X and file \"%s\" contains 0x%X', ...\n first_difference, filename1, block_from_file1(first_difference_in_block), ...\n filename2, block_from_file2(first_difference_in_block));\n if ~isempty(user_message)\n message = sprintf('%s\\n%s', user_message, message);\n end\n throwAsCaller(MException('assertFilesEqual:valuesDiffer', '%s', message));\n end\n \n done = numel(block_from_file1) < block_size;\n num_blocks = num_blocks + 1;\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/assertTrue.m", "content": "function assertTrue(condition, message)\n%assertTrue Assert that input condition is true\n% assertTrue(CONDITION, MESSAGE) throws an exception containing the string\n% MESSAGE if CONDITION is not true.\n%\n% MESSAGE is optional.\n%\n% Examples\n% --------\n% % This call returns silently.\n% assertTrue(rand < 1, 'Expected output of rand to be less than 1')\n%\n% % This call throws an error.\n% assertTrue(sum(sum(magic(3))) == 0, ...\n% 'Expected sum of elements of magic(3) to be 0')\n%\n% See also assertEqual, assertFalse\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n\nif nargin < 2\n message = 'Asserted condition is not true.';\nend\n\nif ~isscalar(condition) || ~islogical(condition)\n throwAsCaller(MException('assertTrue:invalidCondition', ...\n 'CONDITION must be a scalar logical value.'));\nend\n\nif ~condition\n throwAsCaller(MException('assertTrue:falseCondition', '%s', message));\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/assertVectorsAlmostEqual.m", "content": "function assertVectorsAlmostEqual(varargin)\n%assertVectorsAlmostEqual Assert floating-point vectors almost equal in norm sense.\n% assertVectorsAlmostEqual(A, B, tol_type, tol, floor_tol) asserts that the\n% vectors A and B are equal, in the L2-norm sense and within some tolerance.\n% tol_type can be 'relative' or 'absolute'. tol and floor_tol are scalar\n% tolerance values.\n%\n% If the tolerance type is 'relative', then the tolerance test used is:\n%\n% all( norm(A - B) <= tol * max(norm(A), norm(B)) + floor_tol )\n%\n% If the tolerance type is 'absolute', then the tolerance test used is:\n%\n% all( norm(A - B) <= tol )\n%\n% tol_type, tol, and floor_tol are all optional. The default value for\n% tol_type is 'relative'. If both A and B are double, then the default value\n% for tol and floor_tol is sqrt(eps). If either A or B is single, then the\n% default value for tol and floor_tol is sqrt(eps('single')).\n%\n% If A or B is complex, then the tolerance test is applied independently to\n% the real and imaginary parts.\n%\n% Any infinite or NaN element of A or B will cause an assertion failure.\n%\n% assertVectorsAlmostEqual(A, B, ..., msg) prepends the string msg to the\n% assertion message if A and B fail the tolerance test.\n\n% Steven L. Eddins\n% Copyright 2008-2010 The MathWorks, Inc.\n\nparams = xunit.utils.parseFloatAssertInputs(varargin{:});\n\nif ~isequal(size(params.A), size(params.B))\n message = xunit.utils.comparisonMessage(params.Message, ...\n 'Inputs are not the same size.', ...\n params.A, params.B);\n throwAsCaller(MException('assertVectorsAlmostEqual:sizeMismatch', ...\n '%s', message));\nend\n\nif ~(isfloat(params.A) && isfloat(params.B))\n message = xunit.utils.comparisonMessage(params.Message, ...\n 'Inputs are not both floating-point.', ...\n params.A, params.B);\n throwAsCaller(MException('assertVectorsAlmostEqual:notFloat', ...\n '%s', message));\nend\n\nif ~xunit.utils.compareFloats(params.A, params.B, 'vector', ...\n params.ToleranceType, params.Tolerance, params.FloorTolerance)\n \n tolerance_message = sprintf('Inputs are not equal within %s vector tolerance: %g', ...\n params.ToleranceType, params.Tolerance);\n message = xunit.utils.comparisonMessage(params.Message, tolerance_message, ...\n params.A, params.B);\n throwAsCaller(MException('assertVectorsAlmostEqual:tolExceeded', ...\n '%s', message));\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/initTestSuite.m", "content": "%findSubfunctionTests Utility script used for subfunction-based tests\n% This file is a script that is called at the top of M-files containing\n% subfunction-based tests.\n%\n% The top of a typical M-file using this script looks like this:\n%\n% function test_suite = testFeatureA\n%\n% findSubfunctionTests;\n%\n% IMPORTANT NOTE\n% --------------\n% The output variable name for an M-file using this script must be test_suite.\n\n% Steven L. Eddins\n% Copyright 2008-2009 The MathWorks, Inc.\n\n[ST,I] = dbstack('-completenames');\ncaller_name = ST(I + 1).name;\ncaller_file = ST(I + 1).file;\nsubFcns = which('-subfun', caller_file);\n\nsetup_fcn_name = subFcns(xunit.utils.isSetUpString(subFcns));\nif numel(setup_fcn_name) > 1\n error('findSubfunctionTests:tooManySetupFcns', ...\n 'Found more than one setup subfunction.')\nelseif isempty(setup_fcn_name)\n setup_fcn = [];\nelse\n setup_fcn = str2func(setup_fcn_name{1});\nend\n\nteardown_fcn_name = subFcns(xunit.utils.isTearDownString(subFcns));\nif numel(teardown_fcn_name) > 1\n error('findSubfunctionTests:tooManyTeardownFcns', ...\n 'Found more than one teardown subfunction.')\nelseif isempty(teardown_fcn_name)\n teardown_fcn = [];\nelse\n teardown_fcn = str2func(teardown_fcn_name{1});\nend\n\ntest_fcns = cellfun(@str2func, subFcns(xunit.utils.isTestString(subFcns)), ...\n 'UniformOutput', false);\n\nsuite = TestSuite;\nsuite.Name = caller_name;\nsuite.Location = which(caller_file);\nfor k = 1:numel(test_fcns)\n suite.add(FunctionHandleTestCase(test_fcns{k}, setup_fcn, teardown_fcn));\nend\n\nif nargout > 0\n test_suite = suite;\nelse\n suite.run();\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/matlab_xunit/xunit/runtests.m", "content": "function out = runtests(varargin)\n%runtests Run unit tests\n% runtests runs all the test cases that can be found in the current directory\n% and summarizes the results in the Command Window.\n%\n% Test cases can be found in the following places in the current directory:\n%\n% * An M-file function whose name starts or ends with \"test\" or\n% \"Test\" and that returns no output arguments.\n%\n% * An M-file function whose name starts or ends with \"test\" or\n% \"Test\" and that contains subfunction tests and uses the\n% initTestSuite script to return a TestSuite object.\n%\n% * An M-file defining a subclass of TestCase.\n%\n% runtests(dirname) runs all the test cases found in the specified directory.\n%\n% runtests(packagename) runs all the test cases found in the specified\n% package. (This option requires R2009a or later).\n%\n% runtests(mfilename) runs test cases found in the specified function or class\n% name. The function or class needs to be in the current directory or on the\n% MATLAB path.\n%\n% runtests('mfilename:testname') runs the specific test case named 'testname'\n% found in the function or class 'name'.\n%\n% Multiple directories or file names can be specified by passing multiple\n% names to runtests, as in runtests(name1, name2, ...) or\n% runtests({name1, name2, ...}, ...)\n%\n% runtests(..., '-verbose') displays the name and result, result, and time\n% taken for each test case to the Command Window.\n%\n% runtests(..., '-logfile', filename) directs the output of runtests to\n% the specified log file instead of to the Command Window.\n%\n% out = runtests(...) returns a logical value that is true if all the\n% tests passed.\n%\n% Examples\n% --------\n% Find and run all the test cases in the current directory.\n%\n% runtests\n%\n% Find and run all the test cases in the current directory. Display more\n% detailed information to the Command Window as the test cases are run.\n%\n% runtests -verbose\n%\n% Save verbose runtests output to a log file.\n%\n% runtests -verbose -logfile my_test_log.txt\n%\n% Find and run all the test cases contained in the M-file myfunc.\n%\n% runtests myfunc\n%\n% Find and run all the test cases contained in the TestCase subclass\n% MyTestCase.\n%\n% runtests MyTestCase\n%\n% Run the test case named 'testFeature' contained in the M-file myfunc.\n%\n% runtests myfunc:testFeature\n%\n% Run all the tests in a specific directory.\n%\n% runtests c:\\Work\\MyProject\\tests\n%\n% Run all the tests in two directories.\n%\n% runtests c:\\Work\\MyProject\\tests c:\\Work\\Book\\tests\n\n% Steven L. Eddins\n% Copyright 2009-2010 The MathWorks, Inc.\n\nverbose = false;\nlogfile = '';\nif nargin < 1\n suite = TestSuite.fromPwd();\nelse\n [name_list, verbose, logfile] = getInputNames(varargin{:});\n if numel(name_list) == 0\n suite = TestSuite.fromPwd();\n elseif numel(name_list) == 1\n suite = TestSuite.fromName(name_list{1});\n else\n suite = TestSuite();\n for k = 1:numel(name_list)\n suite.add(TestSuite.fromName(name_list{k}));\n end\n end\nend\n\nif isempty(suite.TestComponents)\n error('xunit:runtests:noTestCasesFound', 'No test cases found.');\nend\n\nif isempty(logfile)\n logfile_handle = 1; % File handle corresponding to Command Window\nelse\n logfile_handle = fopen(logfile, 'w');\n if logfile_handle < 0\n error('xunit:runtests:FileOpenFailed', ...\n 'Could not open \"%s\" for writing.', logfile);\n else\n cleanup = onCleanup(@() fclose(logfile_handle));\n end\nend\n\nfprintf(logfile_handle, 'Test suite: %s\\n', suite.Name);\nif ~strcmp(suite.Name, suite.Location)\n fprintf(logfile_handle, 'Test suite location: %s\\n', suite.Location);\nend\nfprintf(logfile_handle, '%s\\n\\n', datestr(now));\n\nif verbose\n monitor = VerboseTestRunDisplay(logfile_handle);\nelse\n monitor = TestRunDisplay(logfile_handle);\nend\ndid_pass = suite.run(monitor);\n\nif nargout > 0\n out = did_pass;\nend\n\nfunction [name_list, verbose, logfile] = getInputNames(varargin)\nname_list = {};\nverbose = false;\nlogfile = '';\nk = 1;\nwhile k <= numel(varargin)\n arg = varargin{k};\n if iscell(arg)\n name_list = [name_list; arg];\n elseif ~isempty(arg) && (arg(1) == '-')\n if strcmp(arg, '-verbose')\n verbose = true;\n elseif strcmp(arg, '-logfile')\n if k == numel(varargin)\n error('xunit:runtests:MissingLogfile', ...\n 'The option -logfile must be followed by a filename.');\n else\n logfile = varargin{k+1};\n k = k + 1;\n end\n else\n warning('runtests:unrecognizedOption', 'Unrecognized option: %s', arg);\n end\n else\n name_list{end+1} = arg;\n end\n k = k + 1;\nend\n \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/assertEqual.m", "content": "function [] = assertEqual(a, b)\n if (a != b)\n testFailed;\n end\n\nfunction [] = testFailed()\n [ST, I] = dbstack(2);\n disp(strcat(\"FAILED: \", ST(1).name));\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/assertVectorsAlmostEqual.m", "content": "function [] = assertVectorsAlmostEqual(a, b, comparetype, tolerance)\n if (max(abs(reshape(a-b,[],1))) > tolerance)\n testFailed(a,b);\n elseif (min(size(a) == size(b)) < 1)\n testFailed(a,b);\n end\n\nfunction [] = testFailed(a, b)\n [ST, I] = dbstack(2);\n disp(strcat(\"FAILED: \", ST(1).name));\n disp(a)\n disp(\"--\")\n disp(b)\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/runtests.m", "content": "path('../../bin', path)\ntest_mdwt\ntest_midwt\ntest_mirdwt\ntest_mrdwt\ntest_makesig\ntest_denoise\ntest_setopt\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/test_denoise.m", "content": "function test_denoise\n disp(\"denoise\")\n test_denoise_default\n test_denoise_2d\n test_denoise_threshold_low\n test_denoise_thresh_multiplier\n test_denoise_std\n test_denoise_hard\n test_denoise_levels\n test_denoise_actual_thresh\n \nfunction test_denoise_default\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h);\n signal_denoised_corr = [0.0741827688375062 0.0791701902526268 0.0760842615272340 0.0750476831774179 0.111279774779568 0.163475053283544 -0.0498263815350539 0.0946073088237311 0.135126562486911 -0.0186090620958193 -0.0748812479991294 -0.103470206059426 0.0234254843251780 0.239772540836257 0.0920583398962312 -0.152180640366891 -0.116682073306156 -0.0459389850762785 -0.00245240039778375 0.0755739164104836 0.102548333512214 0.121099911744184 0.177390507921620 0.240386041553093 0.231105933317157 0.198210924493273 0.175672812990725 0.138822049613034 0.127491615387826 0.121409597186325 0.0994935320130783 0.0760019340865427];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_2d\n x = [1 2 3 4; 5 6 7 8 ; 9 10.09 11 12; 13 13.91 15 16];\n h = daubcqf(4);\n [signal_denoised, subtracted_noise, actual_options] = denoise(x, h);\n signal_denoised_corr = [1.093495801587334 2.052784169768518 3.036985129109070 4.014510779767102; 5.037416383975946 6.006178652683398 6.994963120759174 7.978382656683513; 9.047593546684929 10.003998510025589 10.977825887256145 11.94698494275469; 13.009489364401729 13.937038667522501 14.939852728547271 15.9224996584731398];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_threshold_low\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [1 3.0 0 2 0 0]);\n signal_denoised_corr = [0.0187742354278351 0.0237616568429558 0.0206757281175629 0.0196391497677469 0.0558712413698966 0.108066519873873 -0.105234914944725 0.0391987754140600 0.0797180290772401 -0.0740175955054904 -0.130289781408801 -0.158878739469097 -0.0319830490844931 0.184364007426586 0.0366498064865601 -0.207589173776562 -0.172090606715827 -0.101347518485950 -0.0578609338074549 0.0201653830008125 0.0471398001025425 0.0656913783345127 0.121981974511949 0.184977508143422 0.175697399907486 0.142802391083602 0.120264279581054 0.0834135162033633 0.0720830819781554 0.0660010637766539 0.0440849986034073 0.0205934006768717];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_thresh_multiplier\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [1 3.5 0 2 0 0]);\n signal_denoised_corr = [0.00563527074803461 0.0110853052404048 0.0101590193471916 0.0116789518546074 0.0354625658443208 0.0691904606426981 -0.0647010252187970 0.0393485097012034 0.0302297746478269 -0.0658230296401878 -0.0947938063374137 -0.147943151851009 -0.0355607514547514 0.143027827800490 0.0126752977970079 -0.200577663821584 -0.149059259007655 -0.0564432101940217 -0.0281365070661950 0.0201021371871464 0.0438412772787373 0.0596866399869512 0.0967101937989458 0.136451641917565 0.130716307107088 0.109146914388131 0.0925200849653435 0.0657607417363412 0.0550584910898860 0.0469636231448182 0.0277268486177313 0.00667135407398081];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_std\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [0 3.0 1 2 0 0]);\n signal_denoised_corr = [0.0686926069658060 0.0706216045196474 0.0719769032529757 0.0743568305131058 0.0754251996534692 0.0763549103855611 0.0783972750744446 0.0807092136475563 0.0763109954998047 0.0693017683604205 0.0628697537191382 0.0547492531677562 0.0755519478401559 0.107931256046656 0.0859959791464885 0.0494376118339224 0.0602059364595448 0.0785077229738383 0.0791999606842265 0.0809410605777517 0.0844652184548917 0.0873749084881920 0.0911535278085727 0.0952027332951270 0.0936316016468421 0.0898878427420561 0.0866734185917041 0.0820709685744921 0.0793481432323076 0.0768306965269240 0.0727995727792393 0.0684196591566048];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_hard\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [0 3.0 0 1 0 0]);\n signal_denoised_corr = [0.0977394160103721 0.0994161560983385 0.0832447407807381 0.0666983311697188 0.177420971595413 0.340230583897110 -0.354597069671295 0.0250017872275015 0.394418485343238 -0.0595745304374512 -0.452401570793399 -0.175707560852101 -0.00622320325130765 0.437867065411816 0.187485346584306 -0.241060664687049 -0.306285896120773 -0.373946536466370 -0.246165924475657 0.00210496326791051 0.0528629966064817 0.0967383656953347 0.275410693617439 0.487298926169970 0.454985253718689 0.348603331393631 0.288205743942248 0.186806596496260 0.172147260405660 0.180050851714681 0.142136445826288 0.104484725401481];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_levels\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [0 3.0 0 2 4 0]);\n signal_denoised_corr = [0.164259992817262 0.156379071218712 0.142212685671703 0.125038963573761 0.150297815252073 0.191536767978636 -0.0381639580765735 0.0881092032192094 0.119629284458486 -0.0406090725365491 -0.105645426731493 -0.141820831994602 -0.0280318977202704 0.173171960129832 0.0117537437282443 -0.247115729957293 -0.206759297285911 -0.123147866042363 -0.0685808245422524 0.0255826360141400 0.0635302930397082 0.0930381970490923 0.165728084463140 0.246884147157615 0.246603211345582 0.220210934934003 0.206436991723089 0.177172675548210 0.178948997433275 0.188010177892750 0.179798128181065 0.170937023676945];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_actual_thresh\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [0 3.0 0 2 0 0.5]);\n signal_denoised_corr = [0.0607099183942295 0.0654351521193524 0.0684154759800610 0.0742018934148454 0.0758845005390013 0.0769511530643110 0.0810856606730252 0.0858023375316036 0.0704706443350518 0.0472060906047587 0.0254329679518446 -0.00154590940405266 0.0598455182579352 0.156556707841878 0.0864272987162393 -0.0287835335280487 0.00606017120154721 0.0659592575432934 0.0713958080495586 0.0812891735076492 0.0953701981347179 0.107554576791239 0.123739146895592 0.141180422640726 0.137085044622601 0.124838366760086 0.114852957437233 0.0997294000571788 0.0922174665178409 0.0857758976557685 0.0737052631031342 0.0605470542090229];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/test_makesig.m", "content": "function test_suite = test_makesig\n disp(\"makesig\")\n test_makesig_heavisine\n test_makesig_bumps\n test_makesig_blocks\n test_makesig_doppler\n test_makesig_ramp\n test_makesig_cusp\n test_makesig_sing\n test_makesig_hisine\n test_makesig_losine\n test_makesig_linchirp\n test_makesig_twochirp\n test_makesig_quadchirp\n test_makesig_mishmash\n test_makesig_wernersorrows\n test_makesig_leopold\n\nfunction test_makesig_heavisine\n x = makesig('HeaviSine', 8);\n y = [4.0000 0.0000 -6.0000 -2.0000 2.0000 0.0000 -4.0000 -0.0000];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_bumps\n x = makesig('Bumps', 8);\n y = [0.3206 5.0527 0.3727 0.0129 0.0295 0.0489 0.0004 0.0000];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_blocks\n x = makesig('Blocks', 8);\n y = [4.0000 0.5000 3.0000 0.9000 0.9000 5.2000 -0.0000 -0.0000];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_doppler\n x = makesig('Doppler', 12);\n y = [-0.1954 -0.3067 0.0000 -0.4703 0.4930 -0.2703 -0.4127 0.1025 0.4001 0.3454 0.1425 0];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_ramp\n x = makesig('Ramp', 8);\n y = [0.1250 0.2500 -0.6250 -0.5000 -0.3750 -0.2500 -0.1250 0];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_cusp\n x = makesig('Cusp', 8);\n y = [0.4950 0.3464 0.0707 0.3606 0.5050 0.6164 0.7106 0.7937];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_sing\n x = makesig('Sing', 8);\n y = [5.3333 16.0000 16.0000 5.3333 3.2000 2.2857 1.7778 1.4545];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_hisine\n x = makesig('HiSine', 8);\n y = [0.8267 -0.9302 0.2200 0.6827 -0.9882 0.4292 0.5053 -0.9977];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_losine\n x = makesig('LoSine', 8);\n y = [0.8660 0.8661 0.0003 -0.8658 -0.8663 -0.0006 0.8657 0.8664];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_linchirp\n x = makesig('LinChirp', 8);\n y = [0.0491 0.1951 0.4276 0.7071 0.9415 0.9808 0.6716 0.0000];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_twochirp\n x = makesig('TwoChirp', 8);\n y = [0.5132 1.5000 0.5412 0.8660 -0.5132 0 0.5132 0.8660];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_quadchirp\n x = makesig('QuadChirp', 8);\n y = [0.0164 0.1305 0.4276 0.8660 0.8895 -0.3827 -0.6217 0.8660];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_mishmash\n x = makesig('MishMash', 8);\n y = [0.8922 -0.6046 1.0751 2.2558 0.8429 1.0273 0.5551 -0.1317];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_wernersorrows\n x = makesig('WernerSorrows', 8);\n y = [1.5545 5.3175 0.8252 1.6956 -1.2678 0.6466 1.7332 -0.9977];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_leopold\n x = makesig('Leopold', 8);\n y = [0 1 0 0 0 0 0 0];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/test_mdwt.m", "content": "function test_suite = test_mdwt\n disp(\"mdwt\")\n test_mdwt_1D\n test_mdwt_2D\n test_mdwt_compute_L1\n test_mdwt_compute_L2\n test_mdwt_compute_L3\n\nfunction test_mdwt_1D\n x = makesig('LinChirp', 8);\n h = daubcqf(4, 'min');\n L = 2; % For 8 values in x we would normally be L=2 \n [y, L] = mdwt(x, h, L);\n y_corr = [1.1097 0.8767 0.8204 -0.5201 -0.0339 0.1001 0.2201 -0.1401];\n L_corr = 2;\nassertVectorsAlmostEqual(y, y_corr, 'relative', 0.001);\nassertEqual(L, L_corr);\n\nfunction test_mdwt_2D\n x = [1 2 3 4; 5 6 7 8 ; 9 10 11 12; 13 14 15 16];\n h = daubcqf(4);\n y = mdwt(x, h);\n y_corr = [34.0000 -3.4641 0.0000 -2.0000; -13.8564 0.0000 0.0000 -2.0000; -0.0000 0.0000 -0.0000 -0.0000; -8.0000 -8.0000 0.0000 -0.0000];\nassertVectorsAlmostEqual(y, y_corr, 'relative', 0.001);\n\nfunction test_mdwt_compute_L1\n x = [1 2];\n h = daubcqf(4, 'min');\n [y, L] = mdwt(x, h);\nassertEqual(L, 1);\n\nfunction test_mdwt_compute_L2\n x = [1 2 3 4];\n h = daubcqf(4, 'min');\n [y, L] = mdwt(x, h);\nassertEqual(L, 2);\n\nfunction test_mdwt_compute_L3\n x = [1 2 3 4 5 6 7 8];\n h = daubcqf(4, 'min');\n [y, L] = mdwt(x, h);\nassertEqual(L, 3);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/test_midwt.m", "content": "function test_suite = test_midwt\n disp(\"midwt\")\n test_midwt_1D\n test_midwt_2D\n\nfunction test_midwt_1D\n x = makesig('LinChirp',8);\n h = daubcqf(4,'min');\n L = 2;\n [y,L] = mdwt(x,h,L);\n [x_new,L] = midwt(y,h,L);\nassertVectorsAlmostEqual(x, x_new,'relative',0.0001);\n\nfunction test_midwt_2D\n load ../lena512; \n x = lena512;\n h = daubcqf(6);\n [y,L] = mdwt(x,h);\n [x_new,L] = midwt(y,h);\nassertEqual(L,9);\nassertVectorsAlmostEqual(x, x_new,'relative',0.0001);\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/test_mirdwt.m", "content": "function test_suite = test_mirdwt\n disp(\"mrdwt\")\n test_mirdwt_1\n test_mirdwt_2\n\nfunction test_mirdwt_1 \n xin = makesig('Leopold',8);\n h = daubcqf(4,'min');\n Lin = 1;\n [yl,yh,L] = mrdwt(xin,h,Lin);\n [x,L] = mirdwt(yl,yh,h,L);\n\nassertEqual(L,Lin);\nassertVectorsAlmostEqual(x, xin,'relative',0.0001);\n\nfunction test_mirdwt_2\n load ../lena512; \n x = lena512;\n h = daubcqf(6);\n [yl,yh,L] = mrdwt(x,h);\nassertEqual(L,9);\n [x_new,L] = mirdwt(yl,yh,h);\nassertEqual(L,9);\nassertVectorsAlmostEqual(x, x_new,'relative',0.0001);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/test_mrdwt.m", "content": "function test_suite = test_mrdwt\n disp(\"mrdwt\")\n test_mrdwt_1\n test_mrdwt_2\n test_mrdwt_2L2\n\nfunction test_mrdwt_1\n x = makesig('Leopold',8);\n h = daubcqf(4,'min');\n L = 1;\n [yl, yh, L] = mrdwt(x, h, L);\n yl_corr = [0.8365 0.4830 0 0 0 0 -0.1294 0.2241];\n yh_corr = [-0.2241 -0.1294 0 0 0 0 -0.4830 0.8365];\n L_corr = 1;\nassertVectorsAlmostEqual(yl, yl_corr, 'relative', 0.001);\nassertVectorsAlmostEqual(yh, yh_corr, 'relative', 0.001);\nassertEqual(L, L_corr);\n\nfunction test_mrdwt_2\n x = [1 3 5 2; 3 4 8 1; 3 9 2 0; 1 2 3 0];\n h = daubcqf(4, 'min');\n [yl, yh, L] = mrdwt(x, h, 1);\n yl_corr = [\n 9.0111 10.7799 5.8795 4.1107;\n 11.1393 8.7766 2.5502 4.9130;\n 6.9465 5.7578 1.6630 2.8517;\n 4.8182 7.7611 4.9922 2.0494];\n yh_corr = [\n 4.5724 0.4285 -1.8828 2.2611 4.8714 -3.1026 -1.7978 0.0290 -2.9620 -1.1818 -1.1295 5.2733;\n -2.4441 -2.4318 -1.4465 -1.4587 1.8861 -4.2488 -1.9776 4.3403 -0.0233 0.0356 0.9498 -0.9620;\n -1.7488 -0.5870 0.5592 -0.6026 1.1663 -2.3550 -1.7398 2.9285 -0.6965 1.8583 -0.7120 -0.4498;\n -0.3795 2.5903 2.7700 -0.1998 4.1516 -1.2087 -1.5601 -1.3828 3.6818 -0.7120 0.8917 -3.8615];\nassertVectorsAlmostEqual(yl, yl_corr, 'relative', 0.001);\nassertVectorsAlmostEqual(yh, yh_corr, 'relative', 0.001);\n\nfunction test_mrdwt_2L2\n x = [1 3 5 2; 3 4 8 1; 3 9 2 0; 1 2 3 0];\n h = daubcqf(4, 'min');\n [yl, yh, L] = mrdwt(x, h, 2);\n yl_corr = [\n 11.7500 11.7500 11.7500 11.7500;\n 11.7500 11.7500 11.7500 11.7500;\n 11.7500 11.7500 11.7500 11.7500;\n 11.7500 11.7500 11.7500 11.7500];\n yh_corr = [\n 4.5724 0.4285 -1.8828 2.2611 4.8714 -3.1026 -1.7978 0.0290 -2.9620 -1.1818 -1.1295 5.2733 ...\n 3.1405 3.1405 3.1405 3.1405 4.2075 4.7877 -4.2075 -4.7877 -1.0760 1.8816 1.0760 -1.8816;\n -2.4441 -2.4318 -1.4465 -1.4587 1.8861 -4.2488 -1.9776 4.3403 -0.0233 0.0356 0.9498 -0.9620 ...\n 1.9396 1.9396 1.9396 1.9396 4.2075 4.7877 -4.2075 -4.7877 4.3816 -0.9240 -4.3816 0.9240;\n -1.7488 -0.5870 0.5592 -0.6026 1.1663 -2.3550 -1.7398 2.9285 -0.6965 1.8583 -0.7120 -0.4498 ...\n -3.1405 -3.1405 -3.1405 -3.1405 4.2075 4.7877 -4.2075 -4.7877 1.0760 -1.8816 -1.0760 1.8816;\n -0.3795 2.5903 2.7700 -0.1998 4.1516 -1.2087 -1.5601 -1.3828 3.6818 -0.7120 0.8917 -3.8615 ...\n -1.9396 -1.9396 -1.9396 -1.9396 4.2075 4.7877 -4.2075 -4.7877 -4.3816 0.9240 4.3816 -0.9240];\nassertVectorsAlmostEqual(yl, yl_corr, 'relative', 0.001);\nassertVectorsAlmostEqual(yh, yh_corr, 'relative', 0.001);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/octave/test_setopt.m", "content": "function test_suite = test_setopt\n disp(\"setopt\")\n test_setopt_all_defaults\n test_setopt_nonzero_becomes_zero\n\nfunction test_setopt_all_defaults\n x = [];\n default_opts = [5 6 7 8];\n z = setopt(x, default_opts);\n z_corr = [5 6 7 8];\nassertVectorsAlmostEqual(z, z_corr, 'relative', 0.0001);\n\nfunction test_setopt_nonzero_becomes_zero\n x = [1 0 3];\n default_opts = [5 6 7 8];\n z = setopt(x, default_opts);\n z_corr = [1 6 3 8];\n %z_corr = [1 0 3 8]; % This would be more intuitive \nassertVectorsAlmostEqual(z, z_corr, 'relative', 0.0001);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/readme", "content": "test\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/runtests.m", "content": "path(path, '../bin')\npath(path, 'matlab_xunit/xunit')\ntest_mdwt\ntest_midwt\ntest_mirdwt\ntest_mrdwt\ntest_makesig\ntest_denoise\ntest_setopt\ntest_daubcqf\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/test_daubcqf.m", "content": "function test_suite = test_daubcqf\ninitTestSuite;\n\nfunction test_daubcqf_min\n [a, b] = daubcqf(4);\n ax = [0.482962913144534 0.836516303737808 0.224143868042013 -0.129409522551260];\n bx = [0.129409522551260 0.224143868042013 -0.836516303737808 0.482962913144534];\nassertVectorsAlmostEqual(a, ax, 'relative', 0.001);\nassertVectorsAlmostEqual(b, bx, 'relative', 0.001);\n\nfunction test_daubcqf_max\n [a, b] = daubcqf(4, 'max');\n ax = [-0.129409522551260 0.224143868042013 0.836516303737808 0.482962913144534];\n bx = [-0.482962913144534 0.836516303737808 -0.224143868042013 -0.129409522551260];\nassertVectorsAlmostEqual(a, ax, 'relative', 0.001);\nassertVectorsAlmostEqual(b, bx, 'relative', 0.001);\n\nfunction test_daubcqf_mid_even_k\n [a, b] = daubcqf(4, 'mid');\n ax = [0.482962913144534 0.836516303737808 0.224143868042013 -0.129409522551260];\n bx = [0.129409522551260 0.224143868042013 -0.836516303737808 0.482962913144534];\nassertVectorsAlmostEqual(a, ax, 'relative', 0.001);\nassertVectorsAlmostEqual(b, bx, 'relative', 0.001);\n\nfunction test_daubcqf_mid_odd_k\n [a, b] = daubcqf(6, 'mid');\n ax = [0.332670552950083 0.806891509311093 0.459877502118491 -0.135011020010255 -0.085441273882027 0.035226291885710];\n bx = [-0.035226291885710 -0.085441273882027 0.135011020010255 0.459877502118491 -0.806891509311093 0.332670552950083];\nassertVectorsAlmostEqual(a, ax, 'relative', 0.001);\nassertVectorsAlmostEqual(b, bx, 'relative', 0.001);\n\nfunction test_daubcqf_odd\n handle = @() daubcqf(9);\nassertExceptionThrown(handle, '');\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/test_denoise.m", "content": "function test_suite = test_denoise\ninitTestSuite;\n\n% We could throw an exception if someone specified zero for SoftTH vs HardTH but currently we don't\n\n%function test_denoise_old_invalid_arg_dwt\n% signal = makesig('Doppler', 32);\n% h = daubcqf(6);\n% badarg_handle = @() denoise(signal, h, 0, [0 3.0 0 0 0 0]);\n%assertExceptionThrown(badarg_handle, '');\n \n%function test_denoise_old_invalid_arg_udwt\n% signal = makesig('Doppler', 32);\n% h = daubcqf(6);\n% badarg_handle = @() denoise(signal, h, 1, [0 3.6 0 0 0 0]);\n%assertExceptionThrown(badarg_handle, '');\n \nfunction test_denoise_default\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h);\n signal_denoised_corr = [0.0741827688375062 0.0791701902526268 0.0760842615272340 0.0750476831774179 0.111279774779568 0.163475053283544 -0.0498263815350539 0.0946073088237311 0.135126562486911 -0.0186090620958193 -0.0748812479991294 -0.103470206059426 0.0234254843251780 0.239772540836257 0.0920583398962312 -0.152180640366891 -0.116682073306156 -0.0459389850762785 -0.00245240039778375 0.0755739164104836 0.102548333512214 0.121099911744184 0.177390507921620 0.240386041553093 0.231105933317157 0.198210924493273 0.175672812990725 0.138822049613034 0.127491615387826 0.121409597186325 0.0994935320130783 0.0760019340865427];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_2d\n x = [1 2 3 4; 5 6 7 8 ; 9 10.09 11 12; 13 13.91 15 16];\n h = daubcqf(4);\n [signal_denoised, subtracted_noise, actual_options] = denoise(x, h);\n signal_denoised_corr = [1.093495801587334 2.052784169768518 3.036985129109070 4.014510779767102; 5.037416383975946 6.006178652683398 6.994963120759174 7.978382656683513; 9.047593546684929 10.003998510025589 10.977825887256145 11.94698494275469; 13.009489364401729 13.937038667522501 14.939852728547271 15.9224996584731398];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_udwt\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 1);\n signal_denoised_corr = [0.126244615385152 0.0952319712425300 0.0671343607152503 0.0513902979722585 0.0430402732682634 0.0586932575131794 0.0861069751902698 0.0989949047763016 0.0908418658128637 -0.0141454670119059 -0.144791527437026 -0.0185533166035902 0.278351613782131 0.279033706376659 -0.0205012032054263 -0.212367658407976 -0.241484343697995 -0.248582298831059 -0.213374214781743 -0.101963712141109 0.0454248851310567 0.181104333949749 0.275294407293259 0.309076259882059 0.298600450385073 0.259080737796607 0.211123535801718 0.183021783525739 0.171966340866576 0.171616812586097 0.168720006300193 0.151066428184072];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_udwt_2d\n x = [1 2 3 4; 5 6 7 8 ; 9 10.09 11 12; 13 13.91 15 16];\n h = daubcqf(4);\n [signal_denoised, subtracted_noise, actual_options] = denoise(x, h, 1);\n signal_denoised_corr = [\n 1.007040488866197 1.993405274521765 3.006268404030089 3.996424654030090;\n 4.995935171857875 6.002401216530091 7.001252328142127 8.005847881693983;\n 9.009508189685661 10.059981743374523 11.001190131625481 11.999030274521770;\n 12.987516149590270 13.944211765573623 14.991289136202310 15.998697189754166];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_threshold_low\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [1 3.0 0 2 0 0]);\n signal_denoised_corr = [0.0187742354278351 0.0237616568429558 0.0206757281175629 0.0196391497677469 0.0558712413698966 0.108066519873873 -0.105234914944725 0.0391987754140600 0.0797180290772401 -0.0740175955054904 -0.130289781408801 -0.158878739469097 -0.0319830490844931 0.184364007426586 0.0366498064865601 -0.207589173776562 -0.172090606715827 -0.101347518485950 -0.0578609338074549 0.0201653830008125 0.0471398001025425 0.0656913783345127 0.121981974511949 0.184977508143422 0.175697399907486 0.142802391083602 0.120264279581054 0.0834135162033633 0.0720830819781554 0.0660010637766539 0.0440849986034073 0.0205934006768717];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_thresh_multiplier\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [1 3.5 0 2 0 0]);\n signal_denoised_corr = [0.00563527074803461 0.0110853052404048 0.0101590193471916 0.0116789518546074 0.0354625658443208 0.0691904606426981 -0.0647010252187970 0.0393485097012034 0.0302297746478269 -0.0658230296401878 -0.0947938063374137 -0.147943151851009 -0.0355607514547514 0.143027827800490 0.0126752977970079 -0.200577663821584 -0.149059259007655 -0.0564432101940217 -0.0281365070661950 0.0201021371871464 0.0438412772787373 0.0596866399869512 0.0967101937989458 0.136451641917565 0.130716307107088 0.109146914388131 0.0925200849653435 0.0657607417363412 0.0550584910898860 0.0469636231448182 0.0277268486177313 0.00667135407398081];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_std\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [0 3.0 1 2 0 0]);\n signal_denoised_corr = [0.0686926069658060 0.0706216045196474 0.0719769032529757 0.0743568305131058 0.0754251996534692 0.0763549103855611 0.0783972750744446 0.0807092136475563 0.0763109954998047 0.0693017683604205 0.0628697537191382 0.0547492531677562 0.0755519478401559 0.107931256046656 0.0859959791464885 0.0494376118339224 0.0602059364595448 0.0785077229738383 0.0791999606842265 0.0809410605777517 0.0844652184548917 0.0873749084881920 0.0911535278085727 0.0952027332951270 0.0936316016468421 0.0898878427420561 0.0866734185917041 0.0820709685744921 0.0793481432323076 0.0768306965269240 0.0727995727792393 0.0684196591566048];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_hard\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [0 3.0 0 1 0 0]);\n signal_denoised_corr = [0.0977394160103721 0.0994161560983385 0.0832447407807381 0.0666983311697188 0.177420971595413 0.340230583897110 -0.354597069671295 0.0250017872275015 0.394418485343238 -0.0595745304374512 -0.452401570793399 -0.175707560852101 -0.00622320325130765 0.437867065411816 0.187485346584306 -0.241060664687049 -0.306285896120773 -0.373946536466370 -0.246165924475657 0.00210496326791051 0.0528629966064817 0.0967383656953347 0.275410693617439 0.487298926169970 0.454985253718689 0.348603331393631 0.288205743942248 0.186806596496260 0.172147260405660 0.180050851714681 0.142136445826288 0.104484725401481];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_levels\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [0 3.0 0 2 4 0]);\n signal_denoised_corr = [0.164259992817262 0.156379071218712 0.142212685671703 0.125038963573761 0.150297815252073 0.191536767978636 -0.0381639580765735 0.0881092032192094 0.119629284458486 -0.0406090725365491 -0.105645426731493 -0.141820831994602 -0.0280318977202704 0.173171960129832 0.0117537437282443 -0.247115729957293 -0.206759297285911 -0.123147866042363 -0.0685808245422524 0.0255826360141400 0.0635302930397082 0.0930381970490923 0.165728084463140 0.246884147157615 0.246603211345582 0.220210934934003 0.206436991723089 0.177172675548210 0.178948997433275 0.188010177892750 0.179798128181065 0.170937023676945];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_actual_thresh\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 0, [0 3.0 0 2 0 0.5]);\n signal_denoised_corr = [0.0607099183942295 0.0654351521193524 0.0684154759800610 0.0742018934148454 0.0758845005390013 0.0769511530643110 0.0810856606730252 0.0858023375316036 0.0704706443350518 0.0472060906047587 0.0254329679518446 -0.00154590940405266 0.0598455182579352 0.156556707841878 0.0864272987162393 -0.0287835335280487 0.00606017120154721 0.0659592575432934 0.0713958080495586 0.0812891735076492 0.0953701981347179 0.107554576791239 0.123739146895592 0.141180422640726 0.137085044622601 0.124838366760086 0.114852957437233 0.0997294000571788 0.0922174665178409 0.0857758976557685 0.0737052631031342 0.0605470542090229];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_udwt_threshold_low\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 1, [1 3.0 0 1 0 0]);\n signal_denoised_corr = [0.135039400483741 0.117805175604609 0.0967709584177031 0.0142060292567307 -0.0239840294603812 0.323425861331697 -0.212285200125643 0.166066657685731 0.136653739821785 -0.0361708285655289 -0.244622217319313 -0.0751486112344819 0.279128997196628 0.299915294672821 0.00822389077239383 -0.232180770499244 -0.330137263335199 -0.293955318206172 -0.175538926380835 -0.0733568677543535 0.049241196655251 0.200165899490694 0.304615650610263 0.337325376378116 0.325593984310807 0.282048956150932 0.228861081870546 0.196656880842149 0.180959366486141 0.175210410022406 0.169828050229736 0.155033256209497];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_udwt_thresh_multiplier\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 1, [1 3.5 0 1 0 0]);\n signal_denoised_corr = [0.0479478506866607 0.0160653046305043 -0.012660890293452 -0.0292521383561941 -0.0383355043751224 -0.0239494802109215 0.00200042536526626 0.0135636610003902 0.00399637041195728 -0.100521378500944 -0.229923524965501 -0.102614225576592 0.195850596270724 0.197593413336102 -0.100882406775293 -0.291163630119251 -0.318524834100706 -0.324752887320235 -0.288916218874243 -0.176658530913858 -0.028536592326759 0.108409816572649 0.204063702017061 0.239170248556769 0.230108690684778 0.190119394184444 0.14091827822899 0.11174543739754 0.0991301032767805 0.0977198505254529 0.0937639547688583 0.0745251447941448];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_udwt_std\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 1, [0 3.0 1 1 0 0]);\n signal_denoised_corr = [0.0847626939447046 0.0648669375488877 0.0505127048998841 0.0431477690668965 0.0443458995091662 0.0638361516754724 0.0926698200065443 0.122716357496751 0.135591683864019 0.0377466753027189 -0.0889166586897228 -0.0310700016943258 0.16530654803759 0.237349858169585 0.0577692051497442 -0.137751577705709 -0.18354744395111 -0.188205427540335 -0.157902857480421 -0.055391323576937 0.0791892398460303 0.198068185997372 0.271471422836112 0.282275886815228 0.246689293630916 0.205546705496588 0.16546007731141 0.145130898382968 0.1471329636038 0.142472749823065 0.132163448290946 0.111958195551385];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_udwt_soft\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 1, [0 3.0 0 2 0 0]);\n signal_denoised_corr = [0.086668016749428 0.078090652632278 0.070455842749544 0.062824684205684 0.064249795534642 0.086899924318641 0.053549539548214 0.100644175366308 0.100726560037458 0.051479406046214 -0.011299945211104 0.036115394710961 0.147624998547612 0.159516308766960 0.059119062682569 -0.020817294484415 -0.042170912413038 -0.046825168298822 -0.027179285827824 0.017379645805457 0.071225126011476 0.123532780238470 0.153926034241219 0.160138755049699 0.153562168658336 0.138748019440599 0.123707805352361 0.115223425612607 0.110890877355381 0.107909648973443 0.103630954238181 0.095849084980685];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_udwt_levels\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 1, [0 3.0 0 1 4 0]);\n signal_denoised_corr = [0.137633389000662 0.120676804147327 0.099782758215143 0.015698574020267 -0.025118098815379 0.319788331991522 -0.217919217670089 0.160238201773756 0.131270340429534 -0.041415802797292 -0.249853610380694 -0.080126740883778 0.275034335985338 0.296982831400265 0.006200146572810 -0.234309647934845 -0.332731251852120 -0.296826946748889 -0.178550726178275 -0.074849412517890 0.050375266010248 0.203803428830869 0.310249668154709 0.343153832290091 0.330977383703058 0.287293930382695 0.234092474931927 0.201635010491445 0.185054027697432 0.178142873294961 0.171851794429319 0.157162133645098];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n\nfunction test_denoise_udwt_actual_thresh\n signal = makesig('Doppler', 32);\n noise = [1.54421189550395 0.0859311331754255 -1.49159031063761 -0.742301837259857 -1.06158173331999 2.35045722400204 -0.615601881466894 0.748076783703985 -0.192418510588264 0.888610425420721 -0.764849236567874 -1.40226896933876 -1.42237592509150 0.488193909859941 -0.177375156618825 -0.196053487807333 1.41931015064255 0.291584373984183 0.197811053464361 1.58769908997406 -0.804465956349547 0.696624415849607 0.835088165072682 -0.243715140377952 0.215670086403744 -1.16584393148205 -1.14795277889859 0.104874716016494 0.722254032225002 2.58549125261624 -0.666890670701386 0.187331024578940];\n with_noise = signal + noise / 10; \n h = daubcqf(6);\n [signal_denoised, subtracted_noise, actual_options] = denoise(with_noise, h, 1, [0 3.0 0 1 0 0.5]);\n signal_denoised_corr = [0.126244615385152 0.09523197124253 0.0671343607152503 0.0513902979722585 0.0430402732682634 0.0586932575131794 0.0861069751902698 0.0989949047763016 0.0908418658128637 -0.0141454670119059 -0.144791527437026 -0.0185533166035902 0.278351613782131 0.279033706376659 -0.0205012032054263 -0.212367658407976 -0.241484343697995 -0.248582298831059 -0.213374214781743 -0.101963712141109 0.0454248851310567 0.181104333949749 0.275294407293258 0.309076259882059 0.298600450385073 0.259080737796607 0.211123535801717 0.183021783525739 0.171966340866576 0.171616812586097 0.168720006300193 0.151066428184072];\nassertVectorsAlmostEqual(signal_denoised, signal_denoised_corr, 'relative', 0.0001);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/test_makesig.m", "content": "function test_suite = test_makesig\ninitTestSuite;\n\nfunction test_makesig_heavisine\n x = makesig('HeaviSine', 8);\n y = [4.0000 0.0000 -6.0000 -2.0000 2.0000 0.0000 -4.0000 -0.0000];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_bumps\n x = makesig('Bumps', 8);\n y = [0.3206 5.0527 0.3727 0.0129 0.0295 0.0489 0.0004 0.0000];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_blocks\n x = makesig('Blocks', 8);\n y = [4.0000 0.5000 3.0000 0.9000 0.9000 5.2000 -0.0000 -0.0000];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_doppler\n x = makesig('Doppler', 12);\n y = [-0.1954 -0.3067 0.0000 -0.4703 0.4930 -0.2703 -0.4127 0.1025 0.4001 0.3454 0.1425 0];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_ramp\n x = makesig('Ramp', 8);\n y = [0.1250 0.2500 -0.6250 -0.5000 -0.3750 -0.2500 -0.1250 0];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_cusp\n x = makesig('Cusp', 8);\n y = [0.4950 0.3464 0.0707 0.3606 0.5050 0.6164 0.7106 0.7937];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_sing\n x = makesig('Sing', 8);\n y = [5.3333 16.0000 16.0000 5.3333 3.2000 2.2857 1.7778 1.4545];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_hisine\n x = makesig('HiSine', 8);\n y = [0.8267 -0.9302 0.2200 0.6827 -0.9882 0.4292 0.5053 -0.9977];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_losine\n x = makesig('LoSine', 8);\n y = [0.865973039158459 0.866130104544730 0.000314159260191 -0.865815888304075 -0.866287084447387 -0.000628318489377 0.865658651997088 0.866443978850937];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0000001);\n\nfunction test_makesig_linchirp\n x = makesig('LinChirp', 8);\n y = [0.0491 0.1951 0.4276 0.7071 0.9415 0.9808 0.6716 0.0000];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_twochirp\n x = makesig('TwoChirp', 8);\n y = [0.5132 1.5000 0.5412 0.8660 -0.5132 0 0.5132 0.8660];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_quadchirp\n x = makesig('QuadChirp', 8);\n y = [0.0164 0.1305 0.4276 0.8660 0.8895 -0.3827 -0.6217 0.8660];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_mishmash\n x = makesig('MishMash', 8);\n y = [0.8922 -0.6046 1.0751 2.2558 0.8429 1.0273 0.5551 -0.1317];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_wernersorrows\n x = makesig('WernerSorrows', 8);\n y = [1.5545 5.3175 0.8252 1.6956 -1.2678 0.6466 1.7332 -0.9977];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\nfunction test_makesig_leopold\n x = makesig('Leopold', 8);\n y = [0 1 0 0 0 0 0 0];\nassertVectorsAlmostEqual(x, y, 'relative', 0.0001);\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/test_mdwt.m", "content": "function test_suite = test_mdwt\ninitTestSuite;\n\nfunction test_mdwt_1D\n x = makesig('LinChirp', 8);\n h = daubcqf(4, 'min');\n L = 2; % For 8 values in x we would normally be L=2 \n [y, L] = mdwt(x, h, L);\n y_corr = [1.1097 0.8767 0.8204 -0.5201 -0.0339 0.1001 0.2201 -0.1401];\n L_corr = 2;\nassertVectorsAlmostEqual(y, y_corr, 'relative', 0.001);\nassertEqual(L, L_corr);\n\nfunction test_mdwt_2D\n x = [1 2 3 4; 5 6 7 8 ; 9 10 11 12; 13 14 15 16];\n h = daubcqf(4);\n y = mdwt(x, h);\n y_corr = [34.0000 -3.4641 0.0000 -2.0000; -13.8564 0.0000 0.0000 -2.0000; -0.0000 0.0000 -0.0000 -0.0000; -8.0000 -8.0000 0.0000 -0.0000];\nassertVectorsAlmostEqual(y, y_corr, 'relative', 0.001);\n\nfunction test_mdwt_compute_L1\n x = [1 2];\n h = daubcqf(4, 'min');\n [y, L] = mdwt(x, h);\nassertEqual(L, 1);\n\nfunction test_mdwt_compute_L2\n x = [1 2 3 4];\n h = daubcqf(4, 'min');\n [y, L] = mdwt(x, h);\nassertEqual(L, 2);\n\nfunction test_mdwt_compute_L3\n x = [1 2 3 4 5 6 7 8];\n h = daubcqf(4, 'min');\n [y, L] = mdwt(x, h);\nassertEqual(L, 3);\n\nfunction test_mdwt_compute_bad_L\n L = -1;\n x = [1 2 3 4 5 6 7 8 9];\n h = daubcqf(4, 'min');\n mdwtHandle = @() mdwt(x, h);\nassertExceptionThrown(mdwtHandle, '');\n\nfunction test_mdwt_empty_input\n mdwtHandle = @() mdwt([], [0 0 0 0]);\nassertExceptionThrown(mdwtHandle, '');\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/test_midwt.m", "content": "function test_suite = test_midwt\ninitTestSuite;\n\n\n\nfunction test_midwt_1D\n x = makesig('LinChirp',8);\n h = daubcqf(4,'min');\n L = 2;\n [y,L] = mdwt(x,h,L);\n [x_new,L] = midwt(y,h,L);\nassertVectorsAlmostEqual(x, x_new,'relative',0.0001);\n\nfunction test_midwt_2D\n load lena512; \n x = lena512;\n h = daubcqf(6);\n [y,L] = mdwt(x,h);\n [x_new,L] = midwt(y,h);\nassertEqual(L,9);\nassertVectorsAlmostEqual(x, x_new,'relative',0.0001);\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/test_mirdwt.m", "content": "function test_suite = test_mirdwt\ninitTestSuite;\n\nfunction test_mirdwt_1 \n xin = makesig('Leopold',8);\n h = daubcqf(4,'min');\n Lin = 1;\n [yl,yh,L] = mrdwt(xin,h,Lin);\n [x,L] = mirdwt(yl,yh,h,L);\n\nassertEqual(L,Lin);\nassertVectorsAlmostEqual(x, xin,'relative',0.0001);\n\nfunction test_mirdwt_2D\n load lena512; \n x = lena512;\n h = daubcqf(6);\n [yl,yh,L] = mrdwt(x,h);\nassertEqual(L,9);\n [x_new,L] = mirdwt(yl,yh,h);\nassertEqual(L,9);\nassertVectorsAlmostEqual(x, x_new,'relative',0.0001);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/test_mrdwt.m", "content": "function test_suite = test_mrdwt\ninitTestSuite;\n\nfunction test_mrdwt_1\n x = makesig('Leopold',8);\n h = daubcqf(4,'min');\n L = 1;\n [yl, yh, L] = mrdwt(x, h, L);\n yl_corr = [0.8365 0.4830 0 0 0 0 -0.1294 0.2241];\n yh_corr = [-0.2241 -0.1294 0 0 0 0 -0.4830 0.8365];\n L_corr = 1;\nassertVectorsAlmostEqual(yl, yl_corr, 'relative', 0.001);\nassertVectorsAlmostEqual(yh, yh_corr, 'relative', 0.001);\nassertEqual(L, L_corr);\n\nfunction test_mrdwt_2\n x = [1 3 5 2; 3 4 8 1; 3 9 2 0; 1 2 3 0];\n h = daubcqf(4, 'min');\n [yl, yh, L] = mrdwt(x, h, 1);\n yl_corr = [\n 9.0111 10.7799 5.8795 4.1107;\n 11.1393 8.7766 2.5502 4.9130;\n 6.9465 5.7578 1.6630 2.8517;\n 4.8182 7.7611 4.9922 2.0494];\n yh_corr = [\n 4.5724 0.4285 -1.8828 2.2611 4.8714 -3.1026 -1.7978 0.0290 -2.9620 -1.1818 -1.1295 5.2733;\n -2.4441 -2.4318 -1.4465 -1.4587 1.8861 -4.2488 -1.9776 4.3403 -0.0233 0.0356 0.9498 -0.9620;\n -1.7488 -0.5870 0.5592 -0.6026 1.1663 -2.3550 -1.7398 2.9285 -0.6965 1.8583 -0.7120 -0.4498;\n -0.3795 2.5903 2.7700 -0.1998 4.1516 -1.2087 -1.5601 -1.3828 3.6818 -0.7120 0.8917 -3.8615];\nassertVectorsAlmostEqual(yl, yl_corr, 'relative', 0.001);\nassertVectorsAlmostEqual(yh, yh_corr, 'relative', 0.001);\n\nfunction test_mrdwt_2L2\n x = [1 3 5 2; 3 4 8 1; 3 9 2 0; 1 2 3 0];\n h = daubcqf(4, 'min');\n [yl, yh, L] = mrdwt(x, h, 2);\n yl_corr = [\n 11.7500 11.7500 11.7500 11.7500;\n 11.7500 11.7500 11.7500 11.7500;\n 11.7500 11.7500 11.7500 11.7500;\n 11.7500 11.7500 11.7500 11.7500];\n yh_corr = [\n 4.5724 0.4285 -1.8828 2.2611 4.8714 -3.1026 -1.7978 0.0290 -2.9620 -1.1818 -1.1295 5.2733 ...\n 3.1405 3.1405 3.1405 3.1405 4.2075 4.7877 -4.2075 -4.7877 -1.0760 1.8816 1.0760 -1.8816;\n -2.4441 -2.4318 -1.4465 -1.4587 1.8861 -4.2488 -1.9776 4.3403 -0.0233 0.0356 0.9498 -0.9620 ...\n 1.9396 1.9396 1.9396 1.9396 4.2075 4.7877 -4.2075 -4.7877 4.3816 -0.9240 -4.3816 0.9240;\n -1.7488 -0.5870 0.5592 -0.6026 1.1663 -2.3550 -1.7398 2.9285 -0.6965 1.8583 -0.7120 -0.4498 ...\n -3.1405 -3.1405 -3.1405 -3.1405 4.2075 4.7877 -4.2075 -4.7877 1.0760 -1.8816 -1.0760 1.8816;\n -0.3795 2.5903 2.7700 -0.1998 4.1516 -1.2087 -1.5601 -1.3828 3.6818 -0.7120 0.8917 -3.8615 ...\n -1.9396 -1.9396 -1.9396 -1.9396 4.2075 4.7877 -4.2075 -4.7877 -4.3816 0.9240 4.3816 -0.9240];\nassertVectorsAlmostEqual(yl, yl_corr, 'relative', 0.001);\nassertVectorsAlmostEqual(yh, yh_corr, 'relative', 0.001);\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/PWMBF/rwt/tests/test_setopt.m", "content": "function test_suite = test_setopt\ninitTestSuite;\n\nfunction test_setopt_all_defaults\n x = [];\n default_opts = [5 6 7 8];\n z = setopt(x, default_opts);\n z_corr = [5 6 7 8];\nassertVectorsAlmostEqual(z, z_corr, 'relative', 0.0001);\n\nfunction test_setopt_nonzero_becomes_zero\n x = [1 0 3];\n default_opts = [5 6 7 8];\n z = setopt(x, default_opts);\n z_corr = [1 6 3 8];\n %z_corr = [1 0 3 8]; % This would be more intuitive \nassertVectorsAlmostEqual(z, z_corr, 'relative', 0.0001);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/D_lambda.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Quality with No Reference (QNR). Spectral distortion index. \n% \n% Interface:\n% D_lambda_index = D_lambda(I_F,I_MS,I_MS_LR,S,ratio,p)\n%\n% Inputs:\n% I_F: Pansharpened image;\n% I_MS: MS image resampled to panchromatic scale;\n% I_MS_LR: Original MS image;\n% S: Block size (optional); Default value: 32;\n% ratio: Resolution ratio;\n% p: Exponent value (optional); Default value: p = 1.\n% \n% Outputs:\n% D_lambda_index: D_lambda index.\n% \n% References:\n% [Alparone08] L. Alparone, B. Aiazzi, S. Baronti, A. Garzelli, F. Nencini, and M. Selva, \"Multispectral and panchromatic data fusion assessment without reference,\"\n% Photogrammetric Engineering and Remote Sensing, vol. 74, no. 2, pp. 193200, February 2008. \n% [Vivone14] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transaction on Geoscience and Remote Sensing, 2014. (Accepted)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction D_lambda_index = D_lambda(I_F,I_MS,I_MS_LR,S,ratio,p)\n\nflag_orig_paper = 0; % if 0, Toolbox 1.0, otherwise, original QNR paper \n\nif (size(I_F) ~= size(I_MS))\n error('The two input images must have the same dimensions')\nend\n\n[N,M,Nb] = size(I_F);\n\nif (rem(N,S) ~= 0)\n error('The number of rows must be multiple of the block size')\nend\n\nif (rem(M,S) ~= 0)\n error('The number of columns must be multiple of the block size')\nend\n\nD_lambda_index = 0;\nfor i = 1:Nb-1\n for j = i+1:Nb \n if flag_orig_paper == 0\n %%%%%%% Opt. 1 (as toolbox 1.0)\n band1 = I_MS(:,:,i);\n band2 = I_MS(:,:,j);\n fun_uqi = @(bs) uqi(bs.data,...\n band2(bs.location(1):bs.location(1)+S-1,...\n bs.location(2):bs.location(2)+S-1));\n Qmap_exp = blockproc(band1,[S S],fun_uqi);\n else\n %%%%%%% Opt. 2 (as paper QNR)\n band1 = I_MS_LR(:,:,i);\n band2 = I_MS_LR(:,:,j);\n fun_uqi = @(bs) uqi(bs.data,...\n band2(bs.location(1):bs.location(1)+S/ratio-1,...\n bs.location(2):bs.location(2)+S/ratio-1));\n Qmap_exp = blockproc(band1,[S/ratio S/ratio],fun_uqi);\n end\n Q_exp = mean2(Qmap_exp);\n \n band1 = I_F(:,:,i);\n band2 = I_F(:,:,j);\n fun_uqi = @(bs) uqi(bs.data,...\n band2(bs.location(1):bs.location(1)+S-1,...\n bs.location(2):bs.location(2)+S-1));\n Qmap_fused = blockproc(band1,[S S],fun_uqi);\n Q_fused = mean2(Qmap_fused);\n D_lambda_index = D_lambda_index + abs(Q_fused-Q_exp)^p;\n end\nend\ns = ((Nb^2)-Nb)/2;\nD_lambda_index = (D_lambda_index/s)^(1/p);\n\nend\n\n%%%%%%% Q-index on x and y images\nfunction Q = uqi(x,y)\n\nx = double(x(:));\ny = double(y(:));\nmx = mean(x);\nmy = mean(y);\nC = cov(x,y);\n\nQ = 4 * C(1,2) * mx * my / (C(1,1)+C(2,2)) / (mx^2 + my^2); \n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/D_lambda_K.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Spectral distortion index of the Hybrid Quality with No Reference (HQNR). \n% \n% Interface:\n% Dl = D_lambda_K(fused,ms,ratio,sensor,S)\n%\n% Inputs:\n% fused: Pansharpened image;\n% msexp: MS image resampled to panchromatic scale;\n% sensor: Type of sensor;\n% ratio: Resolution ratio;\n% S: Block size (optional); Default value: 32.\n% \n% Outputs:\n% Dl: D_lambda index.\n% \n% Reference:\n% [Khan09] M. M. Khan, L. Alparone, and J. Chanussot, \"Pansharpening quality assessment using the modulation transfer functions of instruments,\"\n% IEEE Transactions on Geoscience and Remote Sensing, vol. 47, no. 11, pp. 3880-3891, 2009.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction Dl = D_lambda_K(fused,msexp,ratio,sensor,S)\n\nif (size(fused,1) ~= size(msexp,1) || size(fused,2) ~= size(msexp,2))\n error('The two images must have the same dimensions')\nend\n\n[N,M,~] = size(fused);\nif (rem(N,S) ~= 0)\n error('number of rows must be multiple of the block size')\nend\nif (rem(M,S) ~= 0)\n error('number of columns must be multiple of the block size')\nend\n\nfused_degraded = MTF(fused,sensor,ratio);\n\n[Q2n_index,~] = q2n(msexp,fused_degraded,S,S);\nDl = 1-Q2n_index;\n\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/D_s.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Quality with No Reference (QNR). Spatial distortion index.\n% \n% Interface:\n% D_s_index = D_s(I_F,I_MS,I_MS_LR,I_PAN,ratio,S,q)\n%\n% Inputs:\n% I_F: Pansharpened image;\n% I_MS: MS image resampled to panchromatic scale;\n% I_MS_LR: Original MS image;\n% I_PAN: Panchromatic image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% S: Block size (optional); Default value: 32;\n% q: Exponent value (optional); Default value: q = 1.\n% \n% Outputs:\n% D_s_index: D_s index.\n% \n% References:\n% [Alparone08] L. Alparone, B. Aiazzi, S. Baronti, A. Garzelli, F. Nencini, and M. Selva, \"Multispectral and panchromatic data fusion assessment without reference,\"\n% Photogrammetric Engineering and Remote Sensing, vol. 74, no. 2, pp. 193200, February 2008. \n% [Vivone14] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transaction on Geoscience and Remote Sensing, 2014. (Accepted)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction D_s_index = D_s(I_F,I_MS,I_MS_LR,I_PAN,ratio,S,q)\n\nflag_orig_paper = 0; % if 0, Toolbox 1.0, otherwise, original QNR paper \n\nif (size(I_F) ~= size(I_MS))\n error('The two images must have the same dimensions')\nend\n\n[N, M, Nb] = size(I_F);\n\nif (rem(N,S) ~= 0)\n error('number of rows must be multiple of the block size')\nend\n\nif (rem(M,S) ~= 0)\n error('number of columns must be multiple of the block size')\nend\n\nif flag_orig_paper == 0\n %%%%%%% Opt. 1 (as toolbox 1.0) \n pan_filt = interp23tap(imresize(I_PAN,1./ratio),ratio);\nelse\n %%%%%%% Opt. 2 (as paper QNR)\n pan_filt = imresize(I_PAN,1./ratio);\nend\n\nD_s_index = 0;\nfor i = 1:Nb\n band1 = I_F(:,:,i);\n band2 = I_PAN;\n fun_uqi = @(bs) uqi(bs.data,...\n band2(bs.location(1):bs.location(1)+S-1,...\n bs.location(2):bs.location(2)+S-1));\n Qmap_high = blockproc(band1,[S S],fun_uqi);\n Q_high = mean2(Qmap_high);\n \n if flag_orig_paper == 0\n %%%%%%% Opt. 1 (as toolbox 1.0)\n band1 = I_MS(:,:,i);\n band2 = pan_filt;\n fun_uqi = @(bs) uqi(bs.data,...\n band2(bs.location(1):bs.location(1)+S-1,...\n bs.location(2):bs.location(2)+S-1));\n Qmap_low = blockproc(band1,[S S],fun_uqi);\n else\n %%%%%%% Opt. 2 (as paper QNR)\n band1 = I_MS_LR(:,:,i);\n band2 = pan_filt;\n fun_uqi = @(bs) uqi(bs.data,...\n band2(bs.location(1):bs.location(1)+S/ratio-1,...\n bs.location(2):bs.location(2)+S/ratio-1));\n Qmap_low = blockproc(band1,[S/ratio S/ratio],fun_uqi);\n end\n Q_low = mean2(Qmap_low);\n D_s_index = D_s_index + abs(Q_high-Q_low)^q;\nend\n\nD_s_index = (D_s_index/Nb)^(1/q);\n\nend\n\n%%%%%%% Q-index on x and y images\nfunction Q = uqi(x,y)\n\nx = double(x(:));\ny = double(y(:));\nmx = mean(x);\nmy = mean(y);\nC = cov(x,y);\n\nQ = 4 * C(1,2) * mx * my / (C(1,1)+C(2,2)) / (mx^2 + my^2); \n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/ERGAS.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Erreur Relative Globale Adimensionnelle de Synthse (ERGAS).\n% \n% Interface:\n% ERGAS_index = ERGAS(I1,I2,ratio)\n%\n% Inputs:\n% I1: First multispectral image;\n% I2: Second multispectral image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n% \n% Outputs:\n% ERGAS_index: ERGAS index.\n% References:\n% [Ranchin00] T. Ranchin and L. Wald, Fusion of high spatial and spectral resolution images: the ARSIS concept and its implementation,\n% Photogrammetric Engineering and Remote Sensing, vol. 66, no. 1, pp. 4961, January 2000.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction ERGAS_index = ERGAS(I1,I2,ratio)\n\nI1 = double(I1);\nI2 = double(I2);\n\nErr=I1-I2;\nERGAS_index=0;\nfor iLR=1:size(Err,3),\n ERGAS_index = ERGAS_index+mean2(Err(:,:,iLR).^2)/(mean2((I1(:,:,iLR))))^2; \nend\n\nERGAS_index = (100/ratio) * sqrt((1/size(Err,3)) * ERGAS_index);\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/HQNR.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Hybrid Quality with No Reference (HQNR) index. \n% \n% Interface:\n% [HQNR_value,Dl,Ds] = HQNR(ps_ms,ms,msexp,pan,S,sensor,ratio)\n%\n% Inputs:\n% ps_ms: Pansharpened image;\n% ms: Original MS image;\n% msexp: MS image resampled to panchromatic scale;\n% pan: Panchromatic image;\n% S: Block size (optional); Default value: 32;\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n% \n% Outputs:\n% HQNR_value: QNR index;\n% Dl: D_lambda index;\n% Ds: D_s index.\n% \n% References:\n% [Alparone08] L. Alparone, B. Aiazzi, S. Baronti, A. Garzelli, F. Nencini, and M. Selva, \"Multispectral and panchromatic data fusion assessment without reference,\"\n% Photogrammetric Engineering and Remote Sensing, vol. 74, no. 2, pp. 193200, February 2008. \n% [Khan09] M. M. Khan, L. Alparone, and J. Chanussot, \"Pansharpening quality assessment using the modulation transfer functions of instruments\", \n% IEEE Trans. Geosci. Remote Sens., vol. 11, no. 47, pp. 38803891, Nov. 2009.\n% [Aiazzi14] B. Aiazzi, L. Alparone, S. Baronti, R. Carl, A. Garzelli, and L. Santurri, \n% \"Full scale assessment of pansharpening methods and data products\", \n% in SPIE Remote Sensing, pp. 924 402 924 402, 2014.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [HQNR_value,Dl,Ds] = HQNR(ps_ms,ms,msexp,pan,S,sensor,ratio)\n\nDl = D_lambda_K(ps_ms,msexp,ratio,sensor,S);\n\nDs = D_s(ps_ms,msexp,ms,pan,ratio,S,1);\n\nHQNR_value = (1-Dl)*(1-Ds);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/Q.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Q/SSIM averaged on all bands.\n% \n% Interface:\n% Q_avg = Q(I1,I2,L)\n%\n% Inputs:\n% I1: First multispectral image;\n% I2: Second multispectral image;\n% L: Radiometric resolution.\n%\n% Outputs:\n% Q_avg: Q index averaged on all bands.\n% \n% References:\n% [Wang02] Z. Wang and A. C. Bovik, A universal image quality index, IEEE Signal Processing Letters, vol. 9, no. 3, pp. 8184, March 2002.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction Q_avg = Q(I1,I2,L)\n\nQ_orig = zeros(1,size(I1,3));\n\nfor idim=1:size(I1,3),\n% Q_orig(idim) = ssim(I_GT(:,:,idim),I1U(:,:,idim), [0.01 0.03],fspecial('gaussian', 11, 1.5), L);\n Q_orig(idim) = img_qi(I1(:,:,idim),I2(:,:,idim), 32);\nend\n\nQ_avg = mean(Q_orig);\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/QNR.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Quality with No Reference (QNR) index. \n% \n% Interface:\n% [QNR_index,D_lambda_index,D_s_index] = QNR(I_F,I_MS,I_MS_LR,I_PAN,ratio,S,p,q,alpha,beta)\n%\n% Inputs:\n% I_F: Pansharpened image;\n% I_MS: MS image resampled to panchromatic scale;\n% I_MS_LR: Original MS image;\n% I_PAN: Panchromatic image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% S: Block size (optional); Default value: 32;\n% p, q, alpha, beta: Exponent values (optional); Default values: p = q = alpha = beta = 1.\n% \n% Outputs:\n% QNR_index: QNR index;\n% D_lambda_index: D_lambda index;\n% D_s_index: D_s index.\n% \n% References:\n% [Alparone08] L. Alparone, B. Aiazzi, S. Baronti, A. Garzelli, F. Nencini, and M. Selva, \"Multispectral and panchromatic data fusion assessment without reference,\"\n% Photogrammetric Engineering and Remote Sensing, vol. 74, no. 2, pp. 193200, February 2008. \n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [QNR_index,D_lambda_index,D_s_index] = QNR(I_F,I_MS,I_MS_LR,I_PAN,ratio,S,p,q,alpha,beta)\n\nif nargin < 11, beta=1; end\nif nargin < 10, alpha=1; end\nif nargin < 9, q=1; end\nif nargin < 8, p=1; end\nif nargin < 7, S=32; end\n\nD_lambda_index = D_lambda(I_F,I_MS,I_MS_LR,S,ratio,p);\n\nD_s_index = D_s(I_F,I_MS,I_MS_LR,I_PAN,ratio,S,q);\n\nQNR_index = (1-D_lambda_index)^alpha * (1-D_s_index)^beta;\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/SAM.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Spectral Angle Mapper (SAM).\n% \n% Interface:\n% [SAM_index,SAM_map] = SAM(I1,I2)\n%\n% Inputs:\n% I1: First multispectral image;\n% I2: Second multispectral image.\n% \n% Outputs:\n% SAM_index: SAM index;\n% SAM_map: Image of SAM values.\n% \n% References:\n% [Yuhas92] R. H. Yuhas, A. F. H. Goetz, and J. W. Boardman, \"Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm,\" \n% in Proceeding Summaries 3rd Annual JPL Airborne Geoscience Workshop, 1992, pp. 147149.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [SAM_index,SAM_map] = SAM(I1,I2)\n\n[M,N,~] = size(I2);\nprod_scal = dot(I1,I2,3); \nnorm_orig = dot(I1,I1,3);\nnorm_fusa = dot(I2,I2,3);\nprod_norm = sqrt(norm_orig.*norm_fusa);\nprod_map = prod_norm;\nprod_map(prod_map==0)=eps;\nSAM_map = acos(prod_scal./prod_map);\nprod_scal = reshape(prod_scal,M*N,1);\nprod_norm = reshape(prod_norm, M*N,1);\nz=find(prod_norm==0);\nprod_scal(z)=[];prod_norm(z)=[];\nangolo = sum(sum(acos(prod_scal./prod_norm)))/(size(prod_norm,1));\nSAM_index = real(angolo)*180/pi;\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/SCC.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% spatial Correlation Coefficient (sCC).\n% \n% Interface:\n% [sCC,SCCMap] = SCC(I_F,I_GT)\n%\n% Inputs:\n% I_F: Fused image;\n% I_GT: Ground-truth image.\n% \n% Outputs:\n% sCC: spatial correlation coefficient;\n% SCCMap: Image of sCC values.\n% \n% Reference:\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transaction on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2565-2586, May 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [sCC,SCCMap]=SCC(I_F,I_GT)\n\nIm_Lap_F = zeros(size(I_F,1)-2,size(I_F,2)-2,size(I_F,3));\nfor idim=1:size(I_F,3)\n Im_Lap_F_y= imfilter(I_F(2:end-1,2:end-1,idim),fspecial('sobel'));\n Im_Lap_F_x= imfilter(I_F(2:end-1,2:end-1,idim),fspecial('sobel')');\n Im_Lap_F(:,:,idim) = sqrt(Im_Lap_F_y.^2+Im_Lap_F_x.^2);\nend\n\nIm_Lap_GT = zeros(size(I_GT,1)-2,size(I_GT,2)-2,size(I_GT,3));\nfor idim=1:size(I_GT,3)\n Im_Lap_GT_y= imfilter(I_GT(2:end-1,2:end-1,idim),fspecial('sobel'));\n Im_Lap_GT_x= imfilter(I_GT(2:end-1,2:end-1,idim),fspecial('sobel')');\n Im_Lap_GT(:,:,idim) = sqrt(Im_Lap_GT_y.^2+Im_Lap_GT_x.^2);\nend\n\nsCC=sum(sum(sum(Im_Lap_F.*Im_Lap_GT)));\nsCC = sCC/sqrt(sum(Im_Lap_F(:).^2));\nsCC = sCC/sqrt(sum(Im_Lap_GT(:).^2));\n\nSCCMap=sum(Im_Lap_F.*Im_Lap_GT,3)/sqrt(sum(Im_Lap_GT(:).^2))...\n /sqrt(sum(Im_Lap_GT(:).^2));\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/img_qi.m", "content": "function [quality, quality_map] = img_qi(img1, img2, block_size)\n\n%========================================================================\n%\n%Copyright (c) 2001 The University of Texas at Austin\n%All Rights Reserved.\n% \n%This program is free software; you can redistribute it and/or modify\n%it under the terms of the GNU General Public License as published by\n%the Free Software Foundation; either version 2 of the License, or\n%(at your option) any later version.\n% \n%This program is distributed in the hope that it will be useful,\n%but WITHOUT ANY WARRANTY; without even the implied warranty of\n%MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n%GNU General Public License for more details.\n% \n%The GNU Public License is available in the file LICENSE, or you\n%can write to the Free Software Foundation, Inc., 59 Temple Place -\n%Suite 330, Boston, MA 02111-1307, USA, or you can find it on the\n%World Wide Web at http://www.fsf.org.\n%\n%Author : Zhou Wang \n%Version : 1.0\n% \n%The authors are with the Laboratory for Image and Video Engineering\n%(LIVE), Department of Electrical and Computer Engineering, The\n%University of Texas at Austin, Austin, TX.\n%\n%Kindly report any suggestions or corrections to zwang@ece.utexas.edu\n%\n%Acknowledgement:\n%The author would like to thank Mr. Umesh Rajashekar, the Matlab master\n%in our lab, for spending his precious time and giving his kind help\n%on writing this program. Without his help, this program would not\n%achieve its current efficiency.\n%\n%========================================================================\n%\n%This is an efficient implementation of the algorithm for calculating\n%the universal image quality index proposed by Zhou Wang and Alan C. \n%Bovik. Please refer to the paper \"A Universal Image Quality Index\"\n%by Zhou Wang and Alan C. Bovik, published in IEEE Signal Processing\n%Letters, 2001. In order to run this function, you must have Matlab's\n%Image Processing Toobox.\n%\n%Input : an original image and a test image of the same size\n%Output: (1) an overall quality index of the test image, with a value\n% range of [-1, 1].\n% (2) a quality map of the test image. The map has a smaller\n% size than the input images. The actual size is\n% img_size - BLOCK_SIZE + 1.\n%\n%Usage:\n%\n%1. Load the original and the test images into two matrices\n% (say img1 and img2)\n%\n%2. Run this function in one of the two ways:\n%\n% % Choice 1 (suggested):\n% [qi qi_map] = img_qi(img1, img2);\n%\n% % Choice 2:\n% [qi qi_map] = img_qi(img1, img2, BLOCK_SIZE);\n%\n% The default BLOCK_SIZE is 8 (Choice 1). Otherwise, you can specify\n% it by yourself (Choice 2).\n%\n%3. See the results:\n%\n% qi %Gives the over quality index.\n% imshow((qi_map+1)/2) %Shows the quality map as an image.\n%\n%========================================================================\n\nif (nargin == 1 | nargin > 3)\n quality = -Inf;\n quality_map = -1*ones(size(img1));\n return;\nend\n\nif (size(img1) ~= size(img2))\n quality = -Inf;\n quality_map = -1*ones(size(img1));\n return;\nend\n\nif (nargin == 2)\n block_size = 8;\nend\n\nN = block_size.^2;\nsum2_filter = ones(block_size);\n\nimg1_sq = img1.*img1;\nimg2_sq = img2.*img2;\nimg12 = img1.*img2;\n\nimg1_sum = filter2(sum2_filter, img1, 'valid');\nimg2_sum = filter2(sum2_filter, img2, 'valid');\nimg1_sq_sum = filter2(sum2_filter, img1_sq, 'valid');\nimg2_sq_sum = filter2(sum2_filter, img2_sq, 'valid');\nimg12_sum = filter2(sum2_filter, img12, 'valid');\n\nimg12_sum_mul = img1_sum.*img2_sum;\nimg12_sq_sum_mul = img1_sum.*img1_sum + img2_sum.*img2_sum;\nnumerator = 4*(N*img12_sum - img12_sum_mul).*img12_sum_mul;\ndenominator1 = N*(img1_sq_sum + img2_sq_sum) - img12_sq_sum_mul;\ndenominator = denominator1.*img12_sq_sum_mul;\n\nquality_map = ones(size(denominator));\nindex = (denominator1 == 0) & (img12_sq_sum_mul ~= 0);\nquality_map(index) = 2*img12_sum_mul(index)./img12_sq_sum_mul(index);\nindex = (denominator ~= 0);\nquality_map(index) = numerator(index)./denominator(index);\n\nquality = mean2(quality_map);" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/norm_blocco.m", "content": "%%%%%%%%%%%%%% Q2n aux. function\nfunction [y,a,c] = norm_blocco(x)\n\na=mean2(x);\nc=std2(x);\n\nif(c==0)\n\tc = eps;\nend\n\ny=((x-a)/c)+1;\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/onion_mult.m", "content": "%%%%%%%%%%%%%% Q2n aux. function\nfunction ris=onion_mult(onion1,onion2)\n\nN=length(onion1);\n\nif N>1\n \n L=N/2;\n\n a=onion1(1:L);\n b=onion1(L+1:end);\n b=[b(1),-b(2:end)];\n c=onion2(1:L);\n d=onion2(L+1:end);\n d=[d(1),-d(2:end)];\n\n\n if N==2\n ris=[a*c-d*b,a*d+c*b];\n else\n ris1=onion_mult(a,c);\n ris2=onion_mult(d,[b(1),-b(2:end)]); %%\n ris3=onion_mult([a(1),-a(2:end)],d); %%\n ris4=onion_mult(c,b);\n\n aux1=ris1-ris2;\n aux2=ris3+ris4;\n\n ris=[aux1,aux2];\n end\n \nelse\n ris = onion1*onion2;\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/onion_mult2D.m", "content": "%%%%%%%%%%%%%% Q2n aux. function\nfunction ris = onion_mult2D(onion1,onion2)\n\n[~,~,N3]=size(onion1);\n\nif N3>1\n \n L=N3/2;\n\n a=onion1(:,:,1:L);\n b=onion1(:,:,L+1:end);\n b=cat(3,b(:,:,1),-b(:,:,2:end));\n c=onion2(:,:,1:L);\n d=onion2(:,:,L+1:end);\n d=cat(3,d(:,:,1),-d(:,:,2:end));\n\n\n if N3==2\n ris=cat(3,a.*c-d.*b,a.*d+c.*b); \n else\n ris1=onion_mult2D(a,c);\n ris2=onion_mult2D(d,cat(3,b(:,:,1),-b(:,:,2:end)));\n ris3=onion_mult2D(cat(3,a(:,:,1),-a(:,:,2:end)),d);\n ris4=onion_mult2D(c,b);\n\n aux1=ris1-ris2;\n aux2=ris3+ris4;\n\n ris=cat(3,aux1,aux2);\n end\n \nelse\n ris = onion1.*onion2; \nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/onions_quality.m", "content": "%%%%%%%%%%%%%% Q2n aux. function\nfunction q = onions_quality(dat1,dat2,size1)\n\ndat1=double(dat1);\ndat2=double(dat2);\ndat2=cat(3,dat2(:,:,1),-dat2(:,:,2:end));\n[~,~,N3]=size(dat1);\nsize2=size1;\n\n% Block normalization\nfor i=1:N3\n [a1,s,t]=norm_blocco(squeeze(dat1(:,:,i)));\n dat1(:,:,i)=a1;\n clear a1\n if s==0\n if i==1\n dat2(:,:,i)=dat2(:,:,i)-s+1;\n else\n dat2(:,:,i)=-(-dat2(:,:,i)-s+1); \n end\n else\n if i==1\n dat2(:,:,i)=((dat2(:,:,i)-s)/t)+1;\n else\n dat2(:,:,i)=-(((-dat2(:,:,i)-s)/t)+1); \n end\n end\nend\n\nm1=zeros(1,N3);\nm2=zeros(1,N3);\n\nmod_q1m=0;\nmod_q2m=0;\nmod_q1=zeros(size1,size2);\nmod_q2=zeros(size1,size2);\n\nfor i=1:N3\n m1(i)=mean2(squeeze(dat1(:,:,i)));\n m2(i)=mean2(squeeze(dat2(:,:,i)));\n mod_q1m=mod_q1m+(m1(i)^2);\n mod_q2m=mod_q2m+(m2(i)^2);\n mod_q1=mod_q1+((squeeze(dat1(:,:,i))).^2);\n mod_q2=mod_q2+((squeeze(dat2(:,:,i))).^2);\nend\n\nmod_q1m=sqrt(mod_q1m);\nmod_q2m=sqrt(mod_q2m);\nmod_q1=sqrt(mod_q1);\nmod_q2=sqrt(mod_q2);\n\ntermine2 = (mod_q1m*mod_q2m);\ntermine4 = ((mod_q1m^2)+(mod_q2m^2));\nint1=(size1*size2)/((size1*size2)-1)*mean2(mod_q1.^2);\nint2=(size1*size2)/((size1*size2)-1)*mean2(mod_q2.^2);\ntermine3=int1+int2-(size1*size2)/((size1*size2)-1)*((mod_q1m^2)+(mod_q2m^2));\n\nmean_bias=2*termine2/termine4;\nif termine3==0\n q=zeros(1,1,N3);\n q(:,:,N3)=mean_bias;\nelse\n cbm=2/termine3;\n qu=onion_mult2D(dat1,dat2);\n \n qm=onion_mult(m1,m2);\n qv=zeros(1,N3);\n for i=1:N3\n qv(i)=(size1*size2)/((size1*size2)-1)*mean2(squeeze(qu(:,:,i)));\n end\n q=qv-(size1*size2)/((size1*size2)-1)*qm;\n \n q=q*mean_bias*cbm;\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Quality_Indices/q2n.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Q2n index. \n% \n% Interface:\n% [Q2n_index, Q2n_index_map] = q2n(I_GT, I_F, Q_blocks_size, Q_shift)\n%\n% Inputs:\n% I_GT: Ground-Truth image;\n% I_F: Fused Image;\n% Q_blocks_size: Block size of the Q-index locally applied;\n% Q_shift: Block shift of the Q-index locally applied.\n%\n% Outputs:\n% Q2n_index: Q2n index;\n% Q2n_index_map: Map of Q2n values.\n%\n% References:\n% [Garzelli09] A. Garzelli and F. Nencini, \"Hypercomplex quality assessment of multi/hyper-spectral images,\" \n% IEEE Geoscience and Remote Sensing Letters, vol. 6, no. 4, pp. 662665, October 2009.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [Q2n_index, Q2n_index_map] = q2n(I_GT, I_F, Q_blocks_size, Q_shift)\n\n[N1,N2,N3]=size(I_GT);\nsize2=Q_blocks_size;\n\nstepx=ceil(N1/Q_shift);\nstepy=ceil(N2/Q_shift);\n\nif stepy<=0\n stepy=1;\n stepx=1;\nend\n\nest1=(stepx-1)*Q_shift+Q_blocks_size-N1;\nest2=(stepy-1)*Q_shift+Q_blocks_size-N2;\n\nif sum([(est1~=0),(est2~=0)])>0\n refref=[];\n fusfus=[];\n \n for i=1:N3\n a1=squeeze(I_GT(:,:,1));\n \n ia1=zeros(N1+est1,N2+est2);\n ia1(1:N1,1:N2)=a1;\n ia1(:,N2+1:N2+est2)=ia1(:,N2:-1:N2-est2+1);\n ia1(N1+1:N1+est1,:)=ia1(N1:-1:N1-est1+1,:);\n refref=cat(3,refref,ia1);\n \n if i 5)\n mssim = -Inf;\n ssim_map = -Inf;\n return;\nend\n\nif (size(img1) ~= size(img2))\n mssim = -Inf;\n ssim_map = -Inf;\n return;\nend\n\n[M N] = size(img1);\n\nif (nargin == 2)\n if ((M < 11) || (N < 11))\n\t mssim = -Inf;\n\t ssim_map = -Inf;\n return\n end\n window = fspecial('gaussian', 11, 1.5);\t%\n K(1) = 0.01;\t\t\t\t\t% default settings\n K(2) = 0.03;\t\t\t\t\t%\n L = 255; %\nend\n\nif (nargin == 3)\n if ((M < 11) || (N < 11))\n\t mssim = -Inf;\n\t ssim_map = -Inf;\n return\n end\n window = fspecial('gaussian', 11, 1.5);\n L = 255;\n if (length(K) == 2)\n if (K(1) < 0 || K(2) < 0)\n\t\t mssim = -Inf;\n \t\tssim_map = -Inf;\n\t \treturn;\n end\n else\n\t mssim = -Inf;\n \tssim_map = -Inf;\n\t return;\n end\nend\n\nif (nargin == 4)\n [H W] = size(window);\n if ((H*W) < 4 || (H > M) || (W > N))\n\t mssim = -Inf;\n\t ssim_map = -Inf;\n return\n end\n L = 255;\n if (length(K) == 2)\n if (K(1) < 0 || K(2) < 0)\n\t\t mssim = -Inf;\n \t\tssim_map = -Inf;\n\t \treturn;\n end\n else\n\t mssim = -Inf;\n \tssim_map = -Inf;\n\t return;\n end\nend\n\nif (nargin == 5)\n [H W] = size(window);\n if ((H*W) < 4 || (H > M) || (W > N))\n\t mssim = -Inf;\n\t ssim_map = -Inf;\n return\n end\n if (length(K) == 2)\n if (K(1) < 0 || K(2) < 0)\n\t\t mssim = -Inf;\n \t\tssim_map = -Inf;\n\t \treturn;\n end\n else\n\t mssim = -Inf;\n \tssim_map = -Inf;\n\t return;\n end\nend\n\n\nimg1 = double(img1);\nimg2 = double(img2);\n\n% automatic downsampling\nf = max(1,round(min(M,N)/256));\n%downsampling by f\n%use a simple low-pass filter \nif(f>1)\n lpf = ones(f,f);\n lpf = lpf/sum(lpf(:));\n img1 = imfilter(img1,lpf,'symmetric','same');\n img2 = imfilter(img2,lpf,'symmetric','same');\n\n img1 = img1(1:f:end,1:f:end);\n img2 = img2(1:f:end,1:f:end);\nend\n\nC1 = (K(1)*L)^2;\nC2 = (K(2)*L)^2;\nwindow = window/sum(sum(window));\n\nmu1 = filter2(window, img1, 'valid');\nmu2 = filter2(window, img2, 'valid');\nmu1_sq = mu1.*mu1;\nmu2_sq = mu2.*mu2;\nmu1_mu2 = mu1.*mu2;\nsigma1_sq = filter2(window, img1.*img1, 'valid') - mu1_sq;\nsigma2_sq = filter2(window, img2.*img2, 'valid') - mu2_sq;\nsigma12 = filter2(window, img1.*img2, 'valid') - mu1_mu2;\n\nif (C1 > 0 && C2 > 0)\n ssim_map = ((2*mu1_mu2 + C1).*(2*sigma12 + C2))./((mu1_sq + mu2_sq + C1).*(sigma1_sq + sigma2_sq + C2));\nelse\n numerator1 = 2*mu1_mu2 + C1;\n numerator2 = 2*sigma12 + C2;\n\tdenominator1 = mu1_sq + mu2_sq + C1;\n denominator2 = sigma1_sq + sigma2_sq + C2;\n ssim_map = ones(size(mu1));\n index = (denominator1.*denominator2 > 0);\n ssim_map(index) = (numerator1(index).*numerator2(index))./(denominator1(index).*denominator2(index));\n index = (denominator1 ~= 0) & (denominator2 == 0);\n ssim_map(index) = numerator1(index)./denominator1(index);\nend\n\nmssim = mean2(ssim_map);\n\nreturn" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/RRpansharp.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% This method performs pansharpening. We assume that\n% the noisy satellite images yi, i=1,...,L, where y1 is the PAN image\n% and yi, i=2,...,L are the observed MS images, are related to the full\n% resolution target images by\n% \n% yi = Mi*Bi*xi + ni, i=1,...,L \n% \n% where Mi is a downsampling operator, Bi is a circulant blurring matrix,\n% and ni is noise. The method solves\n% min (1/2) sum_{i=1}^L || y_i - Mi*Bi*G*fi ||^2 + lambda * phi(G)\n% F, G\n% where phi is a regularizer function.\n% The function returns Xhat=G*F'. See [1] and [2] for details.\n% \n% Interface:\n% Xhat_im = RRpansharp(Yim,varargin)\n% \n% Inputs:\n% Yim : 1xL cell array containing the observed images the first image\n% is the PAN image and the last L-1 images are the MS images;\n% CDiter: Number of cyclic descent iterations. \n% CDiter=100 is the default;\n% r: The subspace dimension;\n% lambda: The regularization parameter, lambda=0.005 is the \n% default;\n% q: penalty weights;\n% X0: Initial value for X = G * F'.\n% \n% Outputs:\n% Xhat_im: estimated image (3D) at high resolution for each \n% spectral channel.\n% \n% References:\n% [Ulfarsson19] M.O. Ulfarsson, F. Palsson, M.Dalla Mura, J.R. Sveinsson, \"Sentinel-2 Sharpening using a Reduced-Rank Method\", \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 9, pp. 6408-6420, 2019.\n% [Palsson19] F. Palsson, MO. Ulfarsson, and JR. Sveinsson, \"Model-Based Reduced-Rank Pansharpening\", \n% IEEE Geoscience and Remote Sensing Letters, 2019\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction Xhat_im = RRpansharp(Yim,varargin)\n\n % import the manopt optimizer\n addpath('./manopt')\n p1=pwd;\n cd('./manopt');\n importmanopt\n cd(p1)\n % initialization\n CDiter=10;\n r=7;\n lambda=0.005;\n X0 = '';\n tolgradnorm = 0.1;\n if(r==7)\n q = [1, 1.5, 4, 8, 15, 15, 20 ]';\n else\n q = ones(r,1);\n end\n Gstep_only=0;\n GCV = 0;\n for i=1:2:(length(varargin)-1)\n switch varargin{i}\n case 'CDiter'\n CDiter=varargin{i+1};\n case 'r'\n r=varargin{i+1};\n case 'lambda'\n lambda=varargin{i+1};\n case 'q'\n q=varargin{i+1};\n case 'X0'\n X0 = varargin{i+1};\n case 'tolgradnorm'\n tolgradnorm = varargin{i+1};\n case 'Gstep_only'\n Gstep_only = varargin{i+1};\n case 'GCV'\n GCV = varargin{i+1};\n case 'd'\n d = varargin{i+1};\n case 'mtf'\n mtf = varargin{i+1};\n end\n end\n tic;\n if(length(q)~=r), error('The length of q has to match r'); end\n % dimensions of the inputs\n L=length(Yim);\n for i=1:L, Yim{i}=double(Yim{i}); end\n [nl,nc] = size(Yim{1});\n n = nl*nc;\n [Yim2, av] = normaliseData(Yim);\n % Sequence of bands\n % [B1 B2 B3 B4 B5 B6 B7 B8 B8A B9 B11 B12]\n % subsampling factors (in pixels)\n %d = [6 1 1 1 2 2 2 1 2 6 2 2]';\n % convolution operators (Gaussian convolution filters), taken from ref [5]\n %mtf = [ .32 .26 .28 .24 .38 .34 .34 .26 .33 .26 .22 .23];\n sdf = d.*sqrt(-2*log(mtf)/pi^2)';\n % Do not sharpen high-res bands\n sdf(d==1) = 0;\n % remove border for computing the subspace and the result (because of\n % circular assumption\n limsub = 2;\n % kernel filter support\n dx = 12;\n dy = 12;\n % Define blurring operators\n FBM = createConvKernel(sdf,d,nl,nc,L,dx,dy);\n % IMPORTANT!!!\n % Note that the blur kernels are shifted to accomodate the co-registration\n % of real images with different resolutions.\n [Y,M,F]=initialization(Yim2,sdf,nl,nc,L,dx,dy,d,limsub,r);\n Mask=reshape(M,[n,L])';\n % CD\n if isempty(X0)\n Z = zeros(r,n); \n else\n [X0, ~] = normaliseData(X0);\n X0 = reshape(X0,[n,L])';\n [F,D,V]=svd(X0,'econ');\n F = F(:,1:r);\n Z = D(1:r,1:r)*V(:,1:r)';\n end\n % Operators for differences\n [FDH,FDV,FDHC,FDVC] = createDiffkernels(nl,nc,r);\n % Compute weights\n sigmas = 1;\n W = computeWeights(Y,d,sigmas,nl);\n Whalf=W.^(1/2);\n if( GCV == 1), Gstep_only=1; end\n if( Gstep_only ~= 0), CDiter=1; end\n for jCD=1:CDiter\n [Z,Jcost(jCD),options]=Zstep(Y,FBM,F,lambda,nl,nc,Z,Mask,q,FDH,FDV,FDHC,FDVC,W,Whalf,tolgradnorm); \n if(Gstep_only==0) \n F1=Fstep(F,Z,Y,FBM,nl,nc,Mask); \n F=F1;\n end\n if( GCV==1 )\n Ynoise = ( abs(Y) > 0 ) .* randn( size(Y) );\n [Znoise]=Zstep(Ynoise,FBM,F,lambda,nl,nc,Z,Mask,q,FDH,FDV,FDHC,FDVC,W,Whalf,tolgradnorm);\n HtHBXnoise = Mask.*ConvCM(F*Znoise,FBM,nl);\n Ynoise = Ynoise([2:end],:); \n HtHBXnoise = HtHBXnoise([2:end],:);\n den = trace(Ynoise*(Ynoise - HtHBXnoise)'); \n HtHBX=Mask.*ConvCM(F*Z,FBM,nl); \n num = norm( Y([2:end],:) - HtHBX([2:end],:) , 'fro')^2; \n end\n end\n \n Xhat_im = conv2im(F*Z,nl,nc,L);\n Xhat_im = unnormaliseData(Xhat_im,av);\n Xhat_im = Xhat_im(:,:,2:end);\nend\n\nfunction [Y,M,F]=initialization(Yim2,sdf,nl,nc,L,dx,dy,d,limsub,r)\n FBM2 = createConvKernelSubspace(sdf,nl,nc,L,dx,dy);\n % Generate LR MS image FOR SUBSPACE\n % Upsample image via interpolation\n for i=1:L\n Ylim(:,:,i) = imresize(Yim2{i},d(i));\n end\n Y2im=real(ifft2(fft2(Ylim).*FBM2));\n Y2tr=Y2im(limsub+1:end-limsub,limsub+1:end-limsub,:);\n Y2n = reshape(Y2tr,[(nl-4)*(nc-4),L]); \n % SVD analysis\n % Y2n is the image for subspace with the removed border\n [F,D,P] = svd(Y2n','econ');\n F=F(:,1:r);\n [M, Y] = createSubsampling(Yim2,d,nl,nc,L);\nend\n\n\nfunction [Z, xcost,options]=Zstep(Y,FBM,F,tau,nl,nc,Z,Mask,q,FDH,FDV,FDHC,FDVC,W,Whalf,tolgradnorm)\n r = size(F,2);\n n = nl*nc; \n UBTMTy=F'*ConvCM(Y,conj(FBM),nl); \n [Z] = CG(Z,F,Y,UBTMTy,FBM,Mask,nl,nc,r,tau,q,FDH,FDV,FDHC,FDVC,W);\n xcost=1;\n options=[]; \nend \n\nfunction F1=Fstep(F,Z,Y,FBM,nl,nc,Mask)\n F0=F;% U; % initialization\n BTXhat = ConvCM(F0*Z,FBM,nl);\n MBTXhat=Mask.*BTXhat;\n [L,r]=size(F);\n for ii=1:L\n MBZT(:,:,ii)=repmat(Mask(ii,:),[r,1]).*ConvCM(Z,repmat(FBM(:,:,ii),[1,1,r]),nl);\n A(:,:,ii)=MBZT(:,:,ii)*MBZT(:,:,ii)';\n ZBMTy(:,ii)=MBZT(:,:,ii)*Y(ii,:)';\n end\n ZBYT=ZBMTy';% BTY*Z';\n manifold = stiefelfactory(L,r,1); %euclideanfactory(L,r); \n problem.M = manifold;\n problem.cost = @(F) costF(F,MBZT,Y); \n problem.egrad = @(F) egrad(F,A,ZBYT); \n warning('off', 'manopt:getHessian:approx') \n options.tolgradnorm = 1e-2;\n options.verbosity=0;\n [F1, xcost, info, options] = trustregions(problem,F0,options);\n\nend\n\n% Cost functions\n\nfunction [Ju]=costF(F,MBZT,Y)\n L=size(F,1);\n Ju=0;\n for i=1:L\n fi=F(i,:)';\n yi=Y(i,:)';\n Ju=Ju+0.5*norm(MBZT(:,:,i)'*fi-yi,'fro')^2;\n end\nend\n\nfunction [Du]=egrad(F,A,ZBYT)\n p=size(A,3);\n Du=0*F;\n for ii=1:p\n Du(ii,:)=F(ii,:)*A(:,:,ii)'-ZBYT(ii,:);\n end\nend\n\n\n%%% AUXILILARY FUNCTIONS\n\nfunction [FDH,FDV,FDHC,FDVC] = createDiffkernels(nl,nc,r)\n dh = zeros(nl,nc);\n dh(1,1) = 1;\n dh(1,nc) = -1;\n dv = zeros(nl,nc);\n dv(1,1) = 1;\n dv(nl,1) = -1;\n FDH = repmat(fft2(dh),1,1,r);\n FDV = repmat(fft2(dv),1,1,r);\n FDHC = conj(FDH);\n FDVC = conj(FDV);\nend\n\n\nfunction [Yim, av] = normaliseData(Yim)\n % Normalize each cell to unit power\n if iscell(Yim)\n % mean squared power = 1\n nb = length(Yim);\n for i=1:nb\n av(i,1) = mean2(Yim{i}.^2);\n Yim{i,1} = sqrt(Yim{i}.^2/av(i,1));\n end \n else\n nb = size(Yim,3);\n for i=1:nb\n av(i,1) = mean2(Yim(:,:,i).^2);\n Yim(:,:,i) = sqrt(Yim(:,:,i).^2/av(i,1));\n end\n end\nend\n\nfunction FBM = createConvKernel(sdf,d,nl,nc,L,dx,dy)\n %--------------------------------------------------------------------------\n % Build convolution kernels\n %--------------------------------------------------------------------------\n \n middlel=((nl)/2);\n middlec=((nc)/2);\n % kernel filters expanded to size [nl,nc]\n B = zeros(nl,nc,L);\n % fft2 of kernels\n FBM = zeros(nl,nc,L);\n for i=1:L\n if d(i) > 1\n h = fspecial('gaussian',[dx,dy],sdf(i));\n B((middlel-dy/2+1:middlel+dy/2)-d(i)/2+1,(middlec-dx/2+1:middlec+dx/2)-d(i)/2+1,i) = h; %run\n % circularly center\n B(:,:,i)= fftshift(B(:,:,i));\n % normalize\n B(:,:,i) = B(:,:,i)/sum(sum(B(:,:,i)));\n FBM(:,:,i) = fft2(B(:,:,i));\n else\n B(1,1,i) = 1;\n FBM(:,:,i) = fft2(B(:,:,i));\n end\n end\nend\n\nfunction FBM2 = createConvKernelSubspace(sdf,nl,nc,L,dx,dy)\n\n %--------------------------------------------------------------------------\n % Build convolution kernels FOR SUBSPACE!!!!\n %--------------------------------------------------------------------------\n %\n middlel=round((nl+1)/2);\n middlec=round((nc+1)/2);\n\n dx = dx+1;\n dy = dy+1;\n\n % kernel filters expanded to size [nl,nc]\n B = zeros(nl,nc,L);\n % fft2 of kernels\n FBM2 = zeros(nl,nc,L);\n\n s2 = max(sdf);\n for i=1:L\n if sdf(i) < s2\n h = fspecial('gaussian',[dx,dy],sqrt(s2^2-sdf(i)^2));\n B(middlel-(dy-1)/2:middlel+(dy-1)/2,middlec-(dx-1)/2:middlec+(dx-1)/2,i) = h;\n \n %circularly center\n B(:,:,i)= fftshift(B(:,:,i));\n \n % normalize\n B(:,:,i) = B(:,:,i)/sum(sum(B(:,:,i)));\n FBM2(:,:,i) = fft2(B(:,:,i));\n else\n % unit impulse\n B(1,1,i) = 1;\n FBM2(:,:,i) = fft2(B(:,:,i));\n end\n end\nend\n\nfunction X = ConvCM(X,FKM,nl,nc,L)\n\n if nargin == 3\n [L,n] = size(X);\n nc = n/nl;\n end\n X = conv2mat(real(ifft2(fft2(conv2im(X,nl,nc,L)).*FKM)));\n\n %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n % define a circular convolution (the same for all bands) accepting a\n % matrix and returnig a matrix\n % size(X) is [no_bands_ms,n]\n % FKM is the of the cube containing the fft2 of the convolution kernels\n % ConvCM = @(X,FKM) reshape(real(ifft2(fft2(reshape(X', nl,nc,nb)).*FKM)), nl*nc,nb)';\n\n %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nend\n\nfunction X = conv2mat(X,nl,nc,L)\n if ndims(X) == 3\n [nl,nc,L] = size(X);\n X = reshape(X,nl*nc,L)';\n elseif ndims(squeeze(X)) == 2\n L = 1;\n [nl,nc] = size(X);\n X = reshape(X,nl*nc,L)';\n end\nend\n\nfunction [M, Y] = createSubsampling(Yim,d,nl,nc,L)\n\n % subsampling matrix\n M = zeros(nl,nc,L);\n indexes = cell([L 1]);\n\n for i=1:L\n im = ones(floor(nl/d(i)),floor(nc/d(i)));\n maux = zeros(d(i));\n maux(1,1) = 1;\n \n M(:,:,i) = kron(im,maux);\n indexes{i} = find(M(:,:,i) == 1);\n Y(i,indexes{i}) = conv2mat(Yim{i},nl/d(i),nc/d(i),1);\n end\nend\n\nfunction [Yim] = unnormaliseData(Yim, av)\n if iscell(Yim)\n % mean squared power = 1\n nb = length(Yim); \n for i=1:nb\n Yim{i,1} = sqrt(Yim{i}.^2*av(i,1));\n end\n else\n nb = size(Yim,3);\n for i=1:nb\n Yim(:,:,i) = sqrt(Yim(:,:,i).^2*av(i,1));\n end\n end\nend\n\n\n\nfunction W = computeWeights(Y,d,sigmas,nl)\n\n % As in eq. (14) and (15)\n % Compute weigts for each pixel based on HR bands\n hr_bands = d==1;\n hr_bands = find(hr_bands)';\n for i=hr_bands\n % grad(:,:,i) = imgradient(conv2im(Y(i,:),nl),'prewitt').^2;\n % Intermediate gives also good results compared to prewitt\n grad(:,:,i) = imgradient(conv2im(Y(i,:),nl),'intermediate').^2;\n end\n grad = sqrt(max(grad,[],3));\n grad = grad / quantile(grad(:),0.95);\n\n Wim = exp(-grad.^2/2/sigmas^2);\n Wim(Wim<0.5) = 0.5;\n\n W = conv2mat(Wim,nl);\nend\n\nfunction X = conv2im(X,nl,nc,L)\n\n if size(X,2)==1\n X = conv2mat(X,nl,nc,L);\n end\n if nargin == 2\n [L,n] = size(X);\n if n==1\n X = conv2mat(X,nl,nc,L);\n end\n nc = n/nl;\n end\n X = reshape(X',nl,nc,L);\nend\n\nfunction [J,gradJ,AtAg] = grad_cost_G(Z,F,Y,UBTMTy,FBM,Mask,nl,nc,r,tau,q,FDH,FDV,FDHC,FDVC,W)\n X=F*Z;\n BX=ConvCM(X,FBM,nl);\n HtHBX=Mask.*BX;\n ZH=ConvCM(Z,FDHC,nl);\n Zv=ConvCM(Z,FDVC,nl);\n ZHW=ZH.*W;\n ZVW=Zv.*W;\n grad_pen=ConvCM(ZHW,FDH,nl)+ConvCM(ZVW,FDV,nl);\n AtAg = F'*ConvCM(HtHBX,conj(FBM),nl)+2*tau*(q*ones(1,nl*nc)).*grad_pen;\n gradJ=AtAg-UBTMTy;\n J = 1/2 * sum( sum( Z .* AtAg ) ) - sum( sum( Z.*UBTMTy ) ); \nend\n\nfunction [ Z ] = CG(Z,F,Y,UBTMTy,FBM,Mask,nl,nc,r,tau,q,FDH,FDV,FDHC,FDVC,W)\n maxiter = 1000;\n tolgradnorm = 0.1;%1e-6; \n [cost,grad] = grad_cost_G(Z,F,Y,UBTMTy,FBM,Mask,nl,nc,r,tau,q,FDH,FDV,FDHC,FDVC,W);\n gradnorm = norm(grad(:));\n iter = 0;\n res = -grad;\n while ( gradnorm > tolgradnorm & iter < maxiter ) \n iter = iter + 1;\n % fprintf('%5d\\t%+.16e\\t%.8e\\n', iter, cost, gradnorm); \n if( iter == 1 )\n desc_dir = res;\n else\n beta = ( res(:).' * res(:) ) / ( old_res(:).' * old_res(:) );\n desc_dir = res + beta * desc_dir;\n end\n [~, ~, AtAp] = grad_cost_G(desc_dir,F,Y,UBTMTy,FBM,Mask,nl,nc,r,tau,q,FDH,FDV,FDHC,FDVC,W);\n alpha = ( res(:).' * res(:) ) / ( desc_dir(:).' * AtAp(:) );\n Z1 = Z + alpha * desc_dir;\n old_res = res;\n res = res - alpha* AtAp;\n gradnorm = norm( res(:) );\n % Transfer iterate info\n Z = Z1;\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/CLA.txt", "content": "Thank you for your interest in Manopt. The purpose of this Contributor License Agreement is to\nclarify the intellectual property license granted with contributions of software from any person or\nentity (the \"Contributor\") to the owners of Manopt. This license is for your protection as a\nContributor of software to Manopt and does not change your right to use your own contributions for\nany other purpose.\n\nThe owners of Manopt are the copyright holders of Manopt indicated in the license files distributed\nwith the software.\n\nYou and the owners of Manopt hereby accept and agree to the following terms and conditions:\n\nYour \"Contributions\" means all of your past, present and future contributions of object code, source\ncode and documentation to Manopt, however submitted to Manopt, excluding any submissions that are\nconspicuously marked or otherwise designated in writing by You as \"Not a Contribution.\"\n\nYou hereby grant to the owners of Manopt a non-exclusive, irrevocable, worldwide, no-charge,\ntransferable copyright license to use, execute, prepare derivative works of, and distribute\n(internally and externally, in object code and, if included in your Contributions, source code form)\nyour Contributions. Except for the rights granted to the owners of Manopt in this paragraph, You\nreserve all right, title and interest in and to your Contributions.\n\nYou represent that you are legally entitled to grant the above license. If your employer(s) have\nrights to intellectual property that you create, you represent that you have received permission to\nmake the Contibutions on behalf of that employer, or that your employer has waived such rights for\nyour Contributions to Manopt.\n\nYou represent that, except as disclosed in your Contribution submission(s), each of your\nContributions is your original creation. You represent that your Contribution submissions(s)\nincluded complete details of any license or other restriction (including, but not limited to,\nrelated patents and trademarks) associated with any part of your Contribution(s) (including a copy\nof any applicable license agreement). You agree to notify the owners of Manopt of any facts or\ncircumstances of which you become aware that would make Your representations in the Agreement\ninaccurate in any respect.\n\nYou are not expected to provide support for your Contributions, except to the extent you desire to\nprovide support. Your may provide support for free, for a fee, or not at all. Your Contributions are\nprovided as-is, with all faults, defects and errors, and without any warranty of any kind (either\nexpress or implied) including, without limitation, any implied warranty of merchantability and\nfitness for a particular purpose and any warranty of non-infringement.\n\n\nThis CLA is a modification of the CLA used by the UW Calendar project of the University of\nWashington: \n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/COPYING.txt", "content": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free Software Foundation, Inc. \n Everyone is permitted to copy and distribute verbatim copies\n of this license document, but changing it is not allowed.\n\n Preamble\n\n The GNU General Public License is a free, copyleft license for\nsoftware and other kinds of works.\n\n The licenses for most software and other practical works are designed\nto take away your freedom to share and change the works. By contrast,\nthe GNU General Public License is intended to guarantee your freedom to\nshare and change all versions of a program--to make sure it remains free\nsoftware for all its users. We, the Free Software Foundation, use the\nGNU General Public License for most of our software; it applies also to\nany other work released this way by its authors. You can apply it to\nyour programs, too.\n\n When we speak of free software, we are referring to freedom, not\nprice. Our General Public Licenses are designed to make sure that you\nhave the freedom to distribute copies of free software (and charge for\nthem if you wish), that you receive source code or can get it if you\nwant it, that you can change the software or use pieces of it in new\nfree programs, and that you know you can do these things.\n\n To protect your rights, we need to prevent others from denying you\nthese rights or asking you to surrender the rights. Therefore, you have\ncertain responsibilities if you distribute copies of the software, or if\nyou modify it: responsibilities to respect the freedom of others.\n\n For example, if you distribute copies of such a program, whether\ngratis or for a fee, you must pass on to the recipients the same\nfreedoms that you received. You must make sure that they, too, receive\nor can get the source code. And you must show them these terms so they\nknow their rights.\n\n Developers that use the GNU GPL protect your rights with two steps:\n(1) assert copyright on the software, and (2) offer you this License\ngiving you legal permission to copy, distribute and/or modify it.\n\n For the developers' and authors' protection, the GPL clearly explains\nthat there is no warranty for this free software. For both users' and\nauthors' sake, the GPL requires that modified versions be marked as\nchanged, so that their problems will not be attributed erroneously to\nauthors of previous versions.\n\n Some devices are designed to deny users access to install or run\nmodified versions of the software inside them, although the manufacturer\ncan do so. This is fundamentally incompatible with the aim of\nprotecting users' freedom to change the software. The systematic\npattern of such abuse occurs in the area of products for individuals to\nuse, which is precisely where it is most unacceptable. Therefore, we\nhave designed this version of the GPL to prohibit the practice for those\nproducts. If such problems arise substantially in other domains, we\nstand ready to extend this provision to those domains in future versions\nof the GPL, as needed to protect the freedom of users.\n\n Finally, every program is threatened constantly by software patents.\nStates should not allow patents to restrict development and use of\nsoftware on general-purpose computers, but in those that do, we wish to\navoid the special danger that patents applied to a free program could\nmake it effectively proprietary. To prevent this, the GPL assures that\npatents cannot be used to render the program non-free.\n\n The precise terms and conditions for copying, distribution and\nmodification follow.\n\n TERMS AND CONDITIONS\n\n 0. Definitions.\n\n \"This License\" refers to version 3 of the GNU General Public License.\n\n \"Copyright\" also means copyright-like laws that apply to other kinds of\nworks, such as semiconductor masks.\n\n \"The Program\" refers to any copyrightable work licensed under this\nLicense. Each licensee is addressed as \"you\". \"Licensees\" and\n\"recipients\" may be individuals or organizations.\n\n To \"modify\" a work means to copy from or adapt all or part of the work\nin a fashion requiring copyright permission, other than the making of an\nexact copy. The resulting work is called a \"modified version\" of the\nearlier work or a work \"based on\" the earlier work.\n\n A \"covered work\" means either the unmodified Program or a work based\non the Program.\n\n To \"propagate\" a work means to do anything with it that, without\npermission, would make you directly or secondarily liable for\ninfringement under applicable copyright law, except executing it on a\ncomputer or modifying a private copy. Propagation includes copying,\ndistribution (with or without modification), making available to the\npublic, and in some countries other activities as well.\n\n To \"convey\" a work means any kind of propagation that enables other\nparties to make or receive copies. Mere interaction with a user through\na computer network, with no transfer of a copy, is not conveying.\n\n An interactive user interface displays \"Appropriate Legal Notices\"\nto the extent that it includes a convenient and prominently visible\nfeature that (1) displays an appropriate copyright notice, and (2)\ntells the user that there is no warranty for the work (except to the\nextent that warranties are provided), that licensees may convey the\nwork under this License, and how to view a copy of this License. If\nthe interface presents a list of user commands or options, such as a\nmenu, a prominent item in the list meets this criterion.\n\n 1. Source Code.\n\n The \"source code\" for a work means the preferred form of the work\nfor making modifications to it. \"Object code\" means any non-source\nform of a work.\n\n A \"Standard Interface\" means an interface that either is an official\nstandard defined by a recognized standards body, or, in the case of\ninterfaces specified for a particular programming language, one that\nis widely used among developers working in that language.\n\n The \"System Libraries\" of an executable work include anything, other\nthan the work as a whole, that (a) is included in the normal form of\npackaging a Major Component, but which is not part of that Major\nComponent, and (b) serves only to enable use of the work with that\nMajor Component, or to implement a Standard Interface for which an\nimplementation is available to the public in source code form. A\n\"Major Component\", in this context, means a major essential component\n(kernel, window system, and so on) of the specific operating system\n(if any) on which the executable work runs, or a compiler used to\nproduce the work, or an object code interpreter used to run it.\n\n The \"Corresponding Source\" for a work in object code form means all\nthe source code needed to generate, install, and (for an executable\nwork) run the object code and to modify the work, including scripts to\ncontrol those activities. However, it does not include the work's\nSystem Libraries, or general-purpose tools or generally available free\nprograms which are used unmodified in performing those activities but\nwhich are not part of the work. For example, Corresponding Source\nincludes interface definition files associated with source files for\nthe work, and the source code for shared libraries and dynamically\nlinked subprograms that the work is specifically designed to require,\nsuch as by intimate data communication or control flow between those\nsubprograms and other parts of the work.\n\n The Corresponding Source need not include anything that users\ncan regenerate automatically from other parts of the Corresponding\nSource.\n\n The Corresponding Source for a work in source code form is that\nsame work.\n\n 2. Basic Permissions.\n\n All rights granted under this License are granted for the term of\ncopyright on the Program, and are irrevocable provided the stated\nconditions are met. This License explicitly affirms your unlimited\npermission to run the unmodified Program. The output from running a\ncovered work is covered by this License only if the output, given its\ncontent, constitutes a covered work. This License acknowledges your\nrights of fair use or other equivalent, as provided by copyright law.\n\n You may make, run and propagate covered works that you do not\nconvey, without conditions so long as your license otherwise remains\nin force. You may convey covered works to others for the sole purpose\nof having them make modifications exclusively for you, or provide you\nwith facilities for running those works, provided that you comply with\nthe terms of this License in conveying all material for which you do\nnot control copyright. Those thus making or running the covered works\nfor you must do so exclusively on your behalf, under your direction\nand control, on terms that prohibit them from making any copies of\nyour copyrighted material outside their relationship with you.\n\n Conveying under any other circumstances is permitted solely under\nthe conditions stated below. Sublicensing is not allowed; section 10\nmakes it unnecessary.\n\n 3. Protecting Users' Legal Rights From Anti-Circumvention Law.\n\n No covered work shall be deemed part of an effective technological\nmeasure under any applicable law fulfilling obligations under article\n11 of the WIPO copyright treaty adopted on 20 December 1996, or\nsimilar laws prohibiting or restricting circumvention of such\nmeasures.\n\n When you convey a covered work, you waive any legal power to forbid\ncircumvention of technological measures to the extent such circumvention\nis effected by exercising rights under this License with respect to\nthe covered work, and you disclaim any intention to limit operation or\nmodification of the work as a means of enforcing, against the work's\nusers, your or third parties' legal rights to forbid circumvention of\ntechnological measures.\n\n 4. Conveying Verbatim Copies.\n\n You may convey verbatim copies of the Program's source code as you\nreceive it, in any medium, provided that you conspicuously and\nappropriately publish on each copy an appropriate copyright notice;\nkeep intact all notices stating that this License and any\nnon-permissive terms added in accord with section 7 apply to the code;\nkeep intact all notices of the absence of any warranty; and give all\nrecipients a copy of this License along with the Program.\n\n You may charge any price or no price for each copy that you convey,\nand you may offer support or warranty protection for a fee.\n\n 5. Conveying Modified Source Versions.\n\n You may convey a work based on the Program, or the modifications to\nproduce it from the Program, in the form of source code under the\nterms of section 4, provided that you also meet all of these conditions:\n\n a) The work must carry prominent notices stating that you modified\n it, and giving a relevant date.\n\n b) The work must carry prominent notices stating that it is\n released under this License and any conditions added under section\n 7. This requirement modifies the requirement in section 4 to\n \"keep intact all notices\".\n\n c) You must license the entire work, as a whole, under this\n License to anyone who comes into possession of a copy. This\n License will therefore apply, along with any applicable section 7\n additional terms, to the whole of the work, and all its parts,\n regardless of how they are packaged. This License gives no\n permission to license the work in any other way, but it does not\n invalidate such permission if you have separately received it.\n\n d) If the work has interactive user interfaces, each must display\n Appropriate Legal Notices; however, if the Program has interactive\n interfaces that do not display Appropriate Legal Notices, your\n work need not make them do so.\n\n A compilation of a covered work with other separate and independent\nworks, which are not by their nature extensions of the covered work,\nand which are not combined with it such as to form a larger program,\nin or on a volume of a storage or distribution medium, is called an\n\"aggregate\" if the compilation and its resulting copyright are not\nused to limit the access or legal rights of the compilation's users\nbeyond what the individual works permit. Inclusion of a covered work\nin an aggregate does not cause this License to apply to the other\nparts of the aggregate.\n\n 6. Conveying Non-Source Forms.\n\n You may convey a covered work in object code form under the terms\nof sections 4 and 5, provided that you also convey the\nmachine-readable Corresponding Source under the terms of this License,\nin one of these ways:\n\n a) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by the\n Corresponding Source fixed on a durable physical medium\n customarily used for software interchange.\n\n b) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by a\n written offer, valid for at least three years and valid for as\n long as you offer spare parts or customer support for that product\n model, to give anyone who possesses the object code either (1) a\n copy of the Corresponding Source for all the software in the\n product that is covered by this License, on a durable physical\n medium customarily used for software interchange, for a price no\n more than your reasonable cost of physically performing this\n conveying of source, or (2) access to copy the\n Corresponding Source from a network server at no charge.\n\n c) Convey individual copies of the object code with a copy of the\n written offer to provide the Corresponding Source. This\n alternative is allowed only occasionally and noncommercially, and\n only if you received the object code with such an offer, in accord\n with subsection 6b.\n\n d) Convey the object code by offering access from a designated\n place (gratis or for a charge), and offer equivalent access to the\n Corresponding Source in the same way through the same place at no\n further charge. You need not require recipients to copy the\n Corresponding Source along with the object code. If the place to\n copy the object code is a network server, the Corresponding Source\n may be on a different server (operated by you or a third party)\n that supports equivalent copying facilities, provided you maintain\n clear directions next to the object code saying where to find the\n Corresponding Source. Regardless of what server hosts the\n Corresponding Source, you remain obligated to ensure that it is\n available for as long as needed to satisfy these requirements.\n\n e) Convey the object code using peer-to-peer transmission, provided\n you inform other peers where the object code and Corresponding\n Source of the work are being offered to the general public at no\n charge under subsection 6d.\n\n A separable portion of the object code, whose source code is excluded\nfrom the Corresponding Source as a System Library, need not be\nincluded in conveying the object code work.\n\n A \"User Product\" is either (1) a \"consumer product\", which means any\ntangible personal property which is normally used for personal, family,\nor household purposes, or (2) anything designed or sold for incorporation\ninto a dwelling. In determining whether a product is a consumer product,\ndoubtful cases shall be resolved in favor of coverage. For a particular\nproduct received by a particular user, \"normally used\" refers to a\ntypical or common use of that class of product, regardless of the status\nof the particular user or of the way in which the particular user\nactually uses, or expects or is expected to use, the product. A product\nis a consumer product regardless of whether the product has substantial\ncommercial, industrial or non-consumer uses, unless such uses represent\nthe only significant mode of use of the product.\n\n \"Installation Information\" for a User Product means any methods,\nprocedures, authorization keys, or other information required to install\nand execute modified versions of a covered work in that User Product from\na modified version of its Corresponding Source. The information must\nsuffice to ensure that the continued functioning of the modified object\ncode is in no case prevented or interfered with solely because\nmodification has been made.\n\n If you convey an object code work under this section in, or with, or\nspecifically for use in, a User Product, and the conveying occurs as\npart of a transaction in which the right of possession and use of the\nUser Product is transferred to the recipient in perpetuity or for a\nfixed term (regardless of how the transaction is characterized), the\nCorresponding Source conveyed under this section must be accompanied\nby the Installation Information. But this requirement does not apply\nif neither you nor any third party retains the ability to install\nmodified object code on the User Product (for example, the work has\nbeen installed in ROM).\n\n The requirement to provide Installation Information does not include a\nrequirement to continue to provide support service, warranty, or updates\nfor a work that has been modified or installed by the recipient, or for\nthe User Product in which it has been modified or installed. Access to a\nnetwork may be denied when the modification itself materially and\nadversely affects the operation of the network or violates the rules and\nprotocols for communication across the network.\n\n Corresponding Source conveyed, and Installation Information provided,\nin accord with this section must be in a format that is publicly\ndocumented (and with an implementation available to the public in\nsource code form), and must require no special password or key for\nunpacking, reading or copying.\n\n 7. Additional Terms.\n\n \"Additional permissions\" are terms that supplement the terms of this\nLicense by making exceptions from one or more of its conditions.\nAdditional permissions that are applicable to the entire Program shall\nbe treated as though they were included in this License, to the extent\nthat they are valid under applicable law. If additional permissions\napply only to part of the Program, that part may be used separately\nunder those permissions, but the entire Program remains governed by\nthis License without regard to the additional permissions.\n\n When you convey a copy of a covered work, you may at your option\nremove any additional permissions from that copy, or from any part of\nit. (Additional permissions may be written to require their own\nremoval in certain cases when you modify the work.) You may place\nadditional permissions on material, added by you to a covered work,\nfor which you have or can give appropriate copyright permission.\n\n Notwithstanding any other provision of this License, for material you\nadd to a covered work, you may (if authorized by the copyright holders of\nthat material) supplement the terms of this License with terms:\n\n a) Disclaiming warranty or limiting liability differently from the\n terms of sections 15 and 16 of this License; or\n\n b) Requiring preservation of specified reasonable legal notices or\n author attributions in that material or in the Appropriate Legal\n Notices displayed by works containing it; or\n\n c) Prohibiting misrepresentation of the origin of that material, or\n requiring that modified versions of such material be marked in\n reasonable ways as different from the original version; or\n\n d) Limiting the use for publicity purposes of names of licensors or\n authors of the material; or\n\n e) Declining to grant rights under trademark law for use of some\n trade names, trademarks, or service marks; or\n\n f) Requiring indemnification of licensors and authors of that\n material by anyone who conveys the material (or modified versions of\n it) with contractual assumptions of liability to the recipient, for\n any liability that these contractual assumptions directly impose on\n those licensors and authors.\n\n All other non-permissive additional terms are considered \"further\nrestrictions\" within the meaning of section 10. If the Program as you\nreceived it, or any part of it, contains a notice stating that it is\ngoverned by this License along with a term that is a further\nrestriction, you may remove that term. If a license document contains\na further restriction but permits relicensing or conveying under this\nLicense, you may add to a covered work material governed by the terms\nof that license document, provided that the further restriction does\nnot survive such relicensing or conveying.\n\n If you add terms to a covered work in accord with this section, you\nmust place, in the relevant source files, a statement of the\nadditional terms that apply to those files, or a notice indicating\nwhere to find the applicable terms.\n\n Additional terms, permissive or non-permissive, may be stated in the\nform of a separately written license, or stated as exceptions;\nthe above requirements apply either way.\n\n 8. Termination.\n\n You may not propagate or modify a covered work except as expressly\nprovided under this License. Any attempt otherwise to propagate or\nmodify it is void, and will automatically terminate your rights under\nthis License (including any patent licenses granted under the third\nparagraph of section 11).\n\n However, if you cease all violation of this License, then your\nlicense from a particular copyright holder is reinstated (a)\nprovisionally, unless and until the copyright holder explicitly and\nfinally terminates your license, and (b) permanently, if the copyright\nholder fails to notify you of the violation by some reasonable means\nprior to 60 days after the cessation.\n\n Moreover, your license from a particular copyright holder is\nreinstated permanently if the copyright holder notifies you of the\nviolation by some reasonable means, this is the first time you have\nreceived notice of violation of this License (for any work) from that\ncopyright holder, and you cure the violation prior to 30 days after\nyour receipt of the notice.\n\n Termination of your rights under this section does not terminate the\nlicenses of parties who have received copies or rights from you under\nthis License. If your rights have been terminated and not permanently\nreinstated, you do not qualify to receive new licenses for the same\nmaterial under section 10.\n\n 9. Acceptance Not Required for Having Copies.\n\n You are not required to accept this License in order to receive or\nrun a copy of the Program. Ancillary propagation of a covered work\noccurring solely as a consequence of using peer-to-peer transmission\nto receive a copy likewise does not require acceptance. However,\nnothing other than this License grants you permission to propagate or\nmodify any covered work. These actions infringe copyright if you do\nnot accept this License. Therefore, by modifying or propagating a\ncovered work, you indicate your acceptance of this License to do so.\n\n 10. Automatic Licensing of Downstream Recipients.\n\n Each time you convey a covered work, the recipient automatically\nreceives a license from the original licensors, to run, modify and\npropagate that work, subject to this License. You are not responsible\nfor enforcing compliance by third parties with this License.\n\n An \"entity transaction\" is a transaction transferring control of an\norganization, or substantially all assets of one, or subdividing an\norganization, or merging organizations. If propagation of a covered\nwork results from an entity transaction, each party to that\ntransaction who receives a copy of the work also receives whatever\nlicenses to the work the party's predecessor in interest had or could\ngive under the previous paragraph, plus a right to possession of the\nCorresponding Source of the work from the predecessor in interest, if\nthe predecessor has it or can get it with reasonable efforts.\n\n You may not impose any further restrictions on the exercise of the\nrights granted or affirmed under this License. For example, you may\nnot impose a license fee, royalty, or other charge for exercise of\nrights granted under this License, and you may not initiate litigation\n(including a cross-claim or counterclaim in a lawsuit) alleging that\nany patent claim is infringed by making, using, selling, offering for\nsale, or importing the Program or any portion of it.\n\n 11. Patents.\n\n A \"contributor\" is a copyright holder who authorizes use under this\nLicense of the Program or a work on which the Program is based. The\nwork thus licensed is called the contributor's \"contributor version\".\n\n A contributor's \"essential patent claims\" are all patent claims\nowned or controlled by the contributor, whether already acquired or\nhereafter acquired, that would be infringed by some manner, permitted\nby this License, of making, using, or selling its contributor version,\nbut do not include claims that would be infringed only as a\nconsequence of further modification of the contributor version. For\npurposes of this definition, \"control\" includes the right to grant\npatent sublicenses in a manner consistent with the requirements of\nthis License.\n\n Each contributor grants you a non-exclusive, worldwide, royalty-free\npatent license under the contributor's essential patent claims, to\nmake, use, sell, offer for sale, import and otherwise run, modify and\npropagate the contents of its contributor version.\n\n In the following three paragraphs, a \"patent license\" is any express\nagreement or commitment, however denominated, not to enforce a patent\n(such as an express permission to practice a patent or covenant not to\nsue for patent infringement). To \"grant\" such a patent license to a\nparty means to make such an agreement or commitment not to enforce a\npatent against the party.\n\n If you convey a covered work, knowingly relying on a patent license,\nand the Corresponding Source of the work is not available for anyone\nto copy, free of charge and under the terms of this License, through a\npublicly available network server or other readily accessible means,\nthen you must either (1) cause the Corresponding Source to be so\navailable, or (2) arrange to deprive yourself of the benefit of the\npatent license for this particular work, or (3) arrange, in a manner\nconsistent with the requirements of this License, to extend the patent\nlicense to downstream recipients. \"Knowingly relying\" means you have\nactual knowledge that, but for the patent license, your conveying the\ncovered work in a country, or your recipient's use of the covered work\nin a country, would infringe one or more identifiable patents in that\ncountry that you have reason to believe are valid.\n\n If, pursuant to or in connection with a single transaction or\narrangement, you convey, or propagate by procuring conveyance of, a\ncovered work, and grant a patent license to some of the parties\nreceiving the covered work authorizing them to use, propagate, modify\nor convey a specific copy of the covered work, then the patent license\nyou grant is automatically extended to all recipients of the covered\nwork and works based on it.\n\n A patent license is \"discriminatory\" if it does not include within\nthe scope of its coverage, prohibits the exercise of, or is\nconditioned on the non-exercise of one or more of the rights that are\nspecifically granted under this License. You may not convey a covered\nwork if you are a party to an arrangement with a third party that is\nin the business of distributing software, under which you make payment\nto the third party based on the extent of your activity of conveying\nthe work, and under which the third party grants, to any of the\nparties who would receive the covered work from you, a discriminatory\npatent license (a) in connection with copies of the covered work\nconveyed by you (or copies made from those copies), or (b) primarily\nfor and in connection with specific products or compilations that\ncontain the covered work, unless you entered into that arrangement,\nor that patent license was granted, prior to 28 March 2007.\n\n Nothing in this License shall be construed as excluding or limiting\nany implied license or other defenses to infringement that may\notherwise be available to you under applicable patent law.\n\n 12. No Surrender of Others' Freedom.\n\n If conditions are imposed on you (whether by court order, agreement or\notherwise) that contradict the conditions of this License, they do not\nexcuse you from the conditions of this License. If you cannot convey a\ncovered work so as to satisfy simultaneously your obligations under this\nLicense and any other pertinent obligations, then as a consequence you may\nnot convey it at all. For example, if you agree to terms that obligate you\nto collect a royalty for further conveying from those to whom you convey\nthe Program, the only way you could satisfy both those terms and this\nLicense would be to refrain entirely from conveying the Program.\n\n 13. Use with the GNU Affero General Public License.\n\n Notwithstanding any other provision of this License, you have\npermission to link or combine any covered work with a work licensed\nunder version 3 of the GNU Affero General Public License into a single\ncombined work, and to convey the resulting work. The terms of this\nLicense will continue to apply to the part which is the covered work,\nbut the special requirements of the GNU Affero General Public License,\nsection 13, concerning interaction through a network will apply to the\ncombination as such.\n\n 14. Revised Versions of this License.\n\n The Free Software Foundation may publish revised and/or new versions of\nthe GNU General Public License from time to time. Such new versions will\nbe similar in spirit to the present version, but may differ in detail to\naddress new problems or concerns.\n\n Each version is given a distinguishing version number. If the\nProgram specifies that a certain numbered version of the GNU General\nPublic License \"or any later version\" applies to it, you have the\noption of following the terms and conditions either of that numbered\nversion or of any later version published by the Free Software\nFoundation. If the Program does not specify a version number of the\nGNU General Public License, you may choose any version ever published\nby the Free Software Foundation.\n\n If the Program specifies that a proxy can decide which future\nversions of the GNU General Public License can be used, that proxy's\npublic statement of acceptance of a version permanently authorizes you\nto choose that version for the Program.\n\n Later license versions may give you additional or different\npermissions. However, no additional obligations are imposed on any\nauthor or copyright holder as a result of your choosing to follow a\nlater version.\n\n 15. Disclaimer of Warranty.\n\n THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY\nAPPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT\nHOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM \"AS IS\" WITHOUT WARRANTY\nOF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,\nTHE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\nPURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM\nIS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF\nALL NECESSARY SERVICING, REPAIR OR CORRECTION.\n\n 16. Limitation of Liability.\n\n IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING\nWILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS\nTHE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY\nGENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE\nUSE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF\nDATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD\nPARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),\nEVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF\nSUCH DAMAGES.\n\n 17. Interpretation of Sections 15 and 16.\n\n If the disclaimer of warranty and limitation of liability provided\nabove cannot be given local legal effect according to their terms,\nreviewing courts shall apply local law that most closely approximates\nan absolute waiver of all civil liability in connection with the\nProgram, unless a warranty or assumption of liability accompanies a\ncopy of the Program in return for a fee.\n\n END OF TERMS AND CONDITIONS\n\n How to Apply These Terms to Your New Programs\n\n If you develop a new program, and you want it to be of the greatest\npossible use to the public, the best way to achieve this is to make it\nfree software which everyone can redistribute and change under these terms.\n\n To do so, attach the following notices to the program. It is safest\nto attach them to the start of each source file to most effectively\nstate the exclusion of warranty; and each file should have at least\nthe \"copyright\" line and a pointer to where the full notice is found.\n\n \n Copyright (C) \n\n This program is free software: you can redistribute it and/or modify\n it under the terms of the GNU General Public License as published by\n the Free Software Foundation, either version 3 of the License, or\n (at your option) any later version.\n\n This program is distributed in the hope that it will be useful,\n but WITHOUT ANY WARRANTY; without even the implied warranty of\n MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n GNU General Public License for more details.\n\n You should have received a copy of the GNU General Public License\n along with this program. If not, see .\n\nAlso add information on how to contact you by electronic and paper mail.\n\n If the program does terminal interaction, make it output a short\nnotice like this when it starts in an interactive mode:\n\n Copyright (C) \n This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.\n This is free software, and you are welcome to redistribute it\n under certain conditions; type `show c' for details.\n\nThe hypothetical commands `show w' and `show c' should show the appropriate\nparts of the General Public License. Of course, your program's commands\nmight be different; for a GUI interface, you would use an \"about box\".\n\n You should also get your employer (if you work as a programmer) or school,\nif any, to sign a \"copyright disclaimer\" for the program, if necessary.\nFor more information on this, and how to apply and follow the GNU GPL, see\n.\n\n The GNU General Public License does not permit incorporating your program\ninto proprietary programs. If your program is a subroutine library, you\nmay consider it more useful to permit linking proprietary applications with\nthe library. If this is what you want to do, use the GNU Lesser General\nPublic License instead of this License. But first, please read\n.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/CREDITS.txt", "content": "The core developers of Manopt are\n\n* Nicolas Boumal\n* Bamdev Mishra\n\nThrough the RANSO group, Manopt is supported by\n\n* Pierre-Antoine Absil\n* Yurii Nesterov\n* Rodolphe Sepulchre\n\nWe are grateful for the excellent contributions of\n\n* Pierre Borckmans\n* Bart Vandereycken\n* Hiroyuki Sato\n* Roberto Tron\n* Sarod Yatawatta\n* Hiroyuki Kasai\n* Bruno Iannazzo\n* Margherita Procelli\n* Jesus Briales\n* Changshuo Liu\n\nFurthermore, code written by the following people can be found in Manopt:\n\n* Chris Baker\n* Pierre-Antoine Absil\n* Kyle Gallivan\n* Paolo de Leva\n* Wynton Moore\n* Michael Kleder\n\nEach person's contribution is marked by their name in the relevant files.\nSee http://www.manopt.org/about.html for a more precise breakdown.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/LICENSE.txt", "content": "Manopt, a Matlab toolbox for optimization on manifolds, is copyright by\nNicolas Boumal and is distributed under the terms of the GNU General Public\nLicense (GPL) version 3 (or later). See accompanying file or\n.\n\nIn short, this means that everyone is free to use Manopt, to modify it and\nto redistribute it on a free basis. Manopt is not in the public domain;\nit is copyrighted and there are restrictions on its distribution (see the\nlicense). For example, you cannot integrate this version of Manopt (in full\nor in parts) in any closed-source software you plan to distribute\n(commercially or not). Please contact us for more information.\n\nContact:\n http://www.manopt.org\n manopttoolbox@gmail.com\n\nThe documentation of Manopt (the website) is copyright by Nicolas Boumal,\nall rights reserved.\n\n\n\nTHIRD-PARTY CODE\n\nThe following files contain third-party code or extensively rely on\nthird-party code, and their specific license should be considered before\nmodifying and/or redistributing them. The license information can be found\neither in the comments in the code or in a separate text file in the same\ndirectory as the Matlab files.\n\n/manopt/solvers/trustregions/trustregions.m\n/manopt/solvers/trustregions/tCG.m\n/manopt/tools/multitransp.m\n/manopt/tools/multiprod.m\n/manopt/tools/diagsum.m\n/manopt/tools/hashmd5.m\n\n\n\nCONTRIBUTIONS\n\nContributions are licensed to the owners of Manopt under the Contributor\nLicense Agreement, see accompanying file . Be sure to check the\nheader comments of Matlab files and look for the \"original author\" tag.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/README.txt", "content": "Manopt is a Matlab toolbox for optimization on manifolds.\n\nInstallation instructions, documentation and updates are available online:\nhttp://www.manopt.org\n\nManopt is copyright by Nicolas Boumal (nicolasboumal@gmail.com)\nand is distributed under the terms of the GNU General Public License (GPL)\nversion 3 (or later). See the files LICENSE.TXT, COPYING.TXT and CREDITS.TXT.\n\nContact: manopttoolbox@gmail.com\n\n\nQuick installation guide\n------------------------\n\n* Unzip and copy the whole manopt directory you just downloaded in a\n location of your choice on disk, say, in /my/directory/.\n\n* Go to /my/directory/manopt/ at the command prompt and execute importmanopt.\n You may save this path for your next Matlab sessions: follow the menu\n File Set Path... and save.\n\n* Go to /my/directory/manopt/checkinstall/ and run the script basicexample.m.\n If there are no errors, you are done! Otherwise, feel free to contact us.\n\n\nFeedback\n--------\n\nPlease let us know how you use Manopt: it helps us develop a better toolbox.\n\nPlease cite the Manopt paper in your work (as well as relevant solvers/geometries):\nhttp://jmlr.org/papers/v15/boumal14a.html\n\n@article{manopt,\n author = {Nicolas Boumal and Bamdev Mishra and P.-A. Absil and Rodolphe Sepulchre},\n title = {{M}anopt, a {M}atlab Toolbox for Optimization on Manifolds},\n journal = {Journal of Machine Learning Research},\n year = {2014},\n volume = {15},\n pages = {1455--1459},\n url = {http://www.manopt.org}\n}\n\n\n\nFor more info or help: http://www.manopt.org -- we are active on the forum!\n\nThis version:\nManopt 4.0, released Sep. 9, 2017.\n\n\nGitHub: https://github.com/NicolasBoumal/manopt\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/checkinstall/basicexample.m", "content": "function basicexample()\n \n % Verify that Manopt was indeed added to the Matlab path.\n if isempty(which('spherefactory'))\n error(['You should first add Manopt to the Matlab path.\\n' ...\n\t\t 'Please run importmanopt.']);\n end\n \n % Generate the problem data.\n n = 1000;\n A = randn(n);\n A = .5*(A+A');\n \n % Create the problem structure.\n manifold = spherefactory(n);\n problem.M = manifold;\n \n % Define the problem cost function and its gradient.\n problem.cost = @(x) -x'*(A*x);\n problem.egrad = @(x) -2*A*x;\n problem.ehess = @(x, xdot) -2*A*xdot;\n \n % Numerically check gradient and Hessian consistency.\n figure;\n checkgradient(problem);\n figure;\n checkhessian(problem);\n \n % Solve.\n [x, xcost, info] = trustregions(problem); %#ok\n \n % Display some statistics.\n figure;\n semilogy([info.iter], [info.gradnorm], '.-');\n xlabel('Iteration #');\n ylabel('Gradient norm');\n title('Convergence of the trust-regions algorithm on the sphere');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/PCA_stochastic.m", "content": "function [X, A] = PCA_stochastic(A, k)\n% Example of stochastic gradient algorithm in Manopt on a PCA problem.\n% \n% PCA (principal component analysis) on a dataset A of size nxd consists\n% in solving\n% \n% minimize_X f(X) = -.5*norm(A*X, 'fro')^2 / n,\n% \n% where X is a matrix of dimension dxk with orthonormal columns. This\n% is equivalent to finding k dominant singular vectors of A, or k top\n% eigenvectors of A'*A.\n% \n% If n is large, this computation can be expensive. Thus, stochastic\n% gradient algorithms take the point of view that f(X) is a sum of many (n)\n% terms: each term involves only one of the n rows of A.\n%\n% To make progress, it may be sufficient to optimize with respect to a\n% subset of the terms at each iteration. This way, each individual\n% iteration can be very cheap. In particular, individual operations have\n% cost independent of n, because f or its gradient need never be evaluated\n% completely (or at all in the case of f.)\n%\n% Stochastic gradient algorithms (this implementation in particular) are\n% sensitive to proper parameter tuning. See in code.\n\n% This file is part of Manopt and is copyrighted. See the license file.\n% \n% Main author: Bamdev Mishra and Nicolas Boumal, Sept. 6, 2017\n% Contributors:\n% \n% Change log:\n% \n\n\n % If none is given, generate a random data set: n samples in R^d\n if ~exist('A', 'var') || isempty(A)\n d = 1000;\n n = 100000;\n fprintf('Generating data...');\n A = randn(n, d)*diag([[15 10 5], ones(1, d-3)]);\n fprintf(' done (size: %d x %d).\\n', size(A));\n else\n [n, d] = size(A);\n end\n\n % Pick a number of component to compute\n if ~exist('k', 'var') || isempty(k)\n k = 3;\n end\n \n % We are looking for k orthonormal vectors in R^d: Stiefel manifold.\n problem.M = stiefelfactory(d, k);\n \n % The cost function to minimize is a sum of n terms. This parameter\n % must be set for stochastic algorithms.\n problem.ncostterms = n;\n \n % We do not need to specify how to compute the value of the cost\n % function (stochastic algorithms never use this). All we need is to\n % specify how to compute the gradient of the cost function, where the\n % sum is restricted to a subset of the terms (a sample). Notice that we\n % specify a partial Euclidean gradient (hence the 'e' in partialegrad).\n % This way, Manopt will automatically convert the Euclidean vector into\n % a proper Riemannian partial gradient, in the tangent space at X.\n % In particular, if sample = 1:n, then the partial gradient corresponds\n % to the actual (complete) gradient.\n problem.partialegrad = @partialegrad;\n function G = partialegrad(X, sample)\n \n % X is an orthonormal matrix of size dxk\n % sample is a vector if indices between 1 and n: a subset\n % Extract a subset of the dataset\n Asample = A(sample, :);\n \n % Compute the gradient of f restricted to that sample\n G = -Asample'*(Asample*X);\n G = G / n;\n \n end\n\n % If one wants to use checkgradient to verify one's work, then it is\n % necessary to specify the cost function as well, as below.\n % problem.cost = @(X) -.5*norm(A*X, 'fro')^2 / n;\n % checkgradient(problem); pause;\n\n % To have the solver record statistics every x iterations, set\n % options.checkperiod to x. This will record simple quantities which\n % are almost free to compute (namely, elapsed time and step size of the\n % last step.) To record more sophisticated quantities, you can use\n % options.statsfun as usual. Time spent computing these statistics is\n % not counted in times reported in the info structure returned by the\n % solver.\n options.checkperiod = 10;\n options.statsfun = statsfunhelper('metric', @(X) norm(A*X, 'fro'));\n \n % Set the parameters for the solver: stochastic gradient algorithms\n % tend to be quite sensitive to proper tuning, especially regarding\n % step size selection. See the solver's documentation for details.\n options.maxiter = 200;\n options.batchsize = 10;\n % options.stepsize_type = 'decay';\n options.stepsize_init = 1e2;\n options.stepsize_lambda = 1e-3;\n options.verbosity = 2;\n \n % Run the solver\n [X, info] = stochasticgradient(problem, [], options);\n \n \n % Plot the special metric recorded by options.statsfun\n plot([info.iter], [info.metric], '.-');\n xlabel('Iteration #');\n ylabel('Frobenius norm of A*X');\n title('Convergence of stochasticgradient on stiefelfactory for PCA');\n \n % Add to that plot a reference: the globally optimal value attained if\n % the true dominant singular vectors are computed.\n fprintf('Running svds... ');\n t = tic();\n [V, ~] = svds(A', k);\n fprintf('done: %g [s] (note: svd may be faster)\\n', toc(t));\n hold all;\n bound = norm(A*V, 'fro');\n plot([info.iter], bound*ones(size([info.iter])), '--');\n hold off;\n \n legend('Algorithm', 'SVD bound', 'Location', 'SouthEast');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/dominant_invariant_subspace.m", "content": "function [X, info] = dominant_invariant_subspace(A, p)\n% Returns an orthonormal basis of the dominant invariant p-subspace of A.\n%\n% function X = dominant_invariant_subspace(A, p)\n%\n% Input: A real, symmetric matrix A of size nxn and an integer p < n.\n% Output: A real, orthonormal matrix X of size nxp such that trace(X'*A*X)\n% is maximized. That is, the columns of X form an orthonormal basis\n% of a dominant subspace of dimension p of A. These are thus\n% eigenvectors associated with the largest eigenvalues of A (in no\n% particular order). Sign is important: 2 is deemed a larger\n% eigenvalue than -5.\n%\n% The optimization is performed on the Grassmann manifold, since only the\n% space spanned by the columns of X matters. The implementation is short to\n% show how Manopt can be used to quickly obtain a prototype. To make the\n% implementation more efficient, one might first try to use the caching\n% system, that is, use the optional 'store' arguments in the cost, grad and\n% hess functions. Furthermore, using egrad2rgrad and ehess2rhess is quick\n% and easy, but not always efficient. Having a look at the formulas\n% implemented in these functions can help rewrite the code without them,\n% possibly more efficiently.\n%\n% See also: dominant_invariant_subspace_complex\n\n% This file is part of Manopt and is copyrighted. See the license file.\n%\n% Main author: Nicolas Boumal, July 5, 2013\n% Contributors:\n%\n% Change log:\n%\n% NB Dec. 6, 2013:\n% We specify a max and initial trust region radius in the options.\n \n % Generate some random data to test the function\n if ~exist('A', 'var') || isempty(A)\n A = randn(128);\n A = (A+A')/2;\n end\n if ~exist('p', 'var') || isempty(p)\n p = 3;\n end\n \n % Make sure the input matrix is square and symmetric\n n = size(A, 1);\n\tassert(isreal(A), 'A must be real.')\n assert(size(A, 2) == n, 'A must be square.');\n assert(norm(A-A', 'fro') < n*eps, 'A must be symmetric.');\n\tassert(p<=n, 'p must be smaller than n.');\n \n % Define the cost and its derivatives on the Grassmann manifold\n Gr = grassmannfactory(n, p);\n problem.M = Gr;\n problem.cost = @(X) -trace(X'*A*X);\n problem.grad = @(X) -2*Gr.egrad2rgrad(X, A*X);\n problem.hess = @(X, H) -2*Gr.ehess2rhess(X, A*X, A*H, H);\n \n % Execute some checks on the derivatives for early debugging.\n % These can be commented out.\n % checkgradient(problem);\n % pause;\n % checkhessian(problem);\n % pause;\n \n % Issue a call to a solver. A random initial guess will be chosen and\n % default options are selected except for the ones we specify here.\n options.Delta_bar = 8*sqrt(p);\n [X, costX, info, options] = trustregions(problem, [], options); %#ok\n \n fprintf('Options used:\\n');\n disp(options);\n \n % For our information, Manopt can also compute the spectrum of the\n % Riemannian Hessian on the tangent space at (any) X. Computing the\n % spectrum at the solution gives us some idea of the conditioning of\n % the problem. If we were to implement a preconditioner for the\n % Hessian, this would also inform us on its performance.\n %\n % Notice that (typically) all eigenvalues of the Hessian at the\n % solution are positive, i.e., we find an isolated minimizer. If we\n % replace the Grassmann manifold by the Stiefel manifold, hence still\n % optimizing over orthonormal matrices but ignoring the invariance\n % cost(XQ) = cost(X) for all Q orthogonal, then we see\n % dim O(p) = p(p-1)/2 zero eigenvalues in the Hessian spectrum, making\n % the optimizer not isolated anymore.\n if Gr.dim() < 512\n evs = hessianspectrum(problem, X);\n stairs(sort(evs));\n title(['Eigenvalues of the Hessian of the cost function ' ...\n 'at the solution']);\n xlabel('Eigenvalue number (sorted)');\n ylabel('Value of the eigenvalue');\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/dominant_invariant_subspace_complex.m", "content": "function [X, info] = dominant_invariant_subspace_complex(A, p)\n% Returns a unitary basis of the dominant invariant p-subspace of A.\n%\n% function X = dominant_invariant_subspace(A, p)\n%\n% Input: A complex, Hermitian matrix A of size nxn and an integer p < n.\n% Output: A complex, unitary matrix X of size nxp such that trace(X'*A*X)\n% is maximized. That is, the columns of X form a unitary basis\n% of a dominant subspace of dimension p of A.\n%\n% The optimization is performed on the complex Grassmann manifold, since\n% only the space spanned by the columns of X matters.\n%\n% See dominant_invariant_subspace for more details in the real case.\n%\n% See also: dominant_invariant_subspace grassmanncomplexfactory\n\n% This file is part of Manopt and is copyrighted. See the license file.\n%\n% Main author: Nicolas Boumal, June 30, 2015\n% Contributors:\n%\n% Change log:\n \n % Generate some random data to test the function\n if ~exist('A', 'var') || isempty(A)\n A = randn(128) + 1i*randn(128);\n A = (A+A')/2;\n end\n if ~exist('p', 'var') || isempty(p)\n p = 3;\n end\n \n % Make sure the input matrix is Hermitian\n n = size(A, 1);\n assert(size(A, 2) == n, 'A must be square.');\n assert(norm(A-A', 'fro') < n*eps, 'A must be Hermitian.');\n\tassert(p<=n, 'p must be smaller than n.');\n \n % Define the cost and its derivatives on the complex Grassmann manifold\n Gr = grassmanncomplexfactory(n, p);\n problem.M = Gr;\n problem.cost = @(X) -real(trace(X'*A*X));\n problem.egrad = @(X) -2*A*X;\n problem.ehess = @(X, H) -2*A*H;\n \n % Execute some checks on the derivatives for early debugging.\n % These can be commented out.\n % checkgradient(problem);\n % pause;\n % checkhessian(problem);\n % pause;\n \n % Issue a call to a solver. A random initial guess will be chosen and\n % default options are selected except for the ones we specify here.\n options.Delta_bar = 8*sqrt(p);\n [X, costX, info, options] = trustregions(problem, [], options); %#ok\n \n fprintf('Options used:\\n');\n disp(options);\n \n % For our information, Manopt can also compute the spectrum of the\n % Riemannian Hessian on the tangent space at (any) X. Computing the\n % spectrum at the solution gives us some idea of the conditioning of\n % the problem. If we were to implement a preconditioner for the\n % Hessian, this would also inform us on its performance.\n %\n % Notice that (typically) all eigenvalues of the Hessian at the\n % solution are positive, i.e., we find an isolated minimizer. If we\n % replace the Grassmann manifold by the Stiefel manifold, hence still\n % optimizing over orthonormal matrices but ignoring the invariance\n % cost(XQ) = cost(X) for all Q orthogonal, then we see\n % dim O(p) = p(p-1)/2 zero eigenvalues in the Hessian spectrum, making\n % the optimizer not isolated anymore.\n if Gr.dim() < 512\n evs = hessianspectrum(problem, X);\n stairs(sort(evs));\n title(['Eigenvalues of the Hessian of the cost function ' ...\n 'at the solution']);\n xlabel('Eigenvalue number (sorted)');\n ylabel('Value of the eigenvalue');\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/elliptope_SDP.m", "content": "function [Y, problem, S] = elliptope_SDP(A, p, Y0)\n% Solver for semidefinite programs (SDP's) with unit diagonal constraints.\n% \n% function [Y, problem, S] = elliptope_SDP(A)\n% function [Y, problem, S] = elliptope_SDP(A, p)\n% function [Y, problem, S] = elliptope_SDP(A, p, Y0)\n%\n% A is a real, symmetric matrix of size n.\n%\n% This function uses a local optimization method in Manopt to solve the SDP\n%\n% min_X trace(A*X) s.t. diag(X) = 1 and X is positive semidefinite.\n%\n% In practice, the symmetric matrix X of size n is parameterized\n% as X = Y*Y', where Y has size n x p. By default, p is taken large enough\n% (about sqrt(2n)) to ensure that there exists an optimal X whose rank is\n% smaller than p. This ensures that the SDP is equivalent to the new\n% problem in Y:\n%\n% min_Y trace(Y'*A*Y) s.t. diag(Y*Y') = 1.\n%\n% The constraints on Y require each row of Y to have unit norm, which is\n% why Manopt is appropriate software to solve this problem. An optional\n% initial guess can be specified via the input Y0.\n%\n% See the paper below for theory, specifically, for a proof that, for\n% almost all A, second-order critical points of the problem in Y are\n% globally optimal. In other words: there are no local traps in Y, despite\n% non-convexity.\n%\n% Outputs:\n%\n% Y: is the best point found (an nxp matrix with unit norm rows.)\n% To find X, form Y*Y' (or, more efficiently, study X through Y.)\n% \n% problem: is the Manopt problem structure used to produce Y.\n% \n% S: is a dual optimality certificate (a symmetric matrix of size n,\n% sparse if A is sparse). The optimality gap (in the cost\n% function) is at most n*min(eig(S)), for both Y and X = Y*Y'.\n% Hence, if min(eig(S)) is close to zero, Y is close to globally\n% optimal. This can be computed via eigs(S, 1, 'SR').\n% \n% Paper: https://arxiv.org/abs/1606.04970\n%\n% @inproceedings{boumal2016bmapproach,\n% author = {Boumal, N. and Voroninski, V. and Bandeira, A.S.},\n% title = {The non-convex {B}urer-{M}onteiro approach works on smooth semidefinite programs},\n% booktitle={Neural Information Processing Systems (NIPS 2016)},\n% year = {2016}\n% }\n% \n% See also: maxcut elliptope_SDP_complex\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 28, 2016\n% Contributors:\n% Change log:\n\n\n % If no inputs are provided, since this is an example file, generate\n % a random Erdos-Renyi graph. This is for illustration purposes only.\n if ~exist('A', 'var') || isempty(A)\n n = 100;\n A = triu(rand(n) <= .1, 1);\n A = (A+A.')/(2*n);\n end\n\n n = size(A, 1);\n assert(n >= 2, 'A must be at least 2x2.');\n assert(isreal(A), 'A must be real.');\n assert(size(A, 2) == n, 'A must be square.');\n \n % Force A to be symmetric\n A = (A+A.')/2;\n \n % By default, pick a sufficiently large p (number of columns of Y).\n if ~exist('p', 'var') || isempty(p)\n p = ceil(sqrt(8*n+1)/2);\n end\n \n assert(p >= 2 && p == round(p), 'p must be an integer >= 2.');\n\n % Pick the manifold of n-by-p matrices with unit norm rows.\n manifold = obliquefactory(p, n, true);\n \n problem.M = manifold;\n \n \n % These three, quick commented lines of code are sufficient to define\n % the cost function and its derivatives. This is good code to write\n % when prototyping. Below, a more advanced use of Manopt is shown,\n % where the redundant computation A*Y is avoided between the gradient\n % and the cost evaluation.\n % % problem.cost = @(Y) .5*sum(sum((A*Y).*Y));\n % % problem.egrad = @(Y) A*Y;\n % % problem.ehess = @(Y, Ydot) A*Ydot;\n \n % Products with A dominate the cost, hence we store the result.\n % This allows to share the results among cost, grad and hess.\n % This is completely optional.\n function store = prepare(Y, store)\n if ~isfield(store, 'AY')\n AY = A*Y;\n store.AY = AY;\n store.diagAYYt = sum(AY .* Y, 2);\n end\n end\n \n % Define the cost function to be /minimized/.\n problem.cost = @cost;\n function [f, store] = cost(Y, store)\n store = prepare(Y, store);\n f = .5*sum(store.diagAYYt);\n end\n\n % Define the Riemannian gradient.\n problem.grad = @grad;\n function [G, store] = grad(Y, store)\n store = prepare(Y, store);\n G = store.AY - bsxfun(@times, Y, store.diagAYYt);\n end\n\n % If you want to, you can specify the Riemannian Hessian as well.\n problem.hess = @hess;\n function [H, store] = hess(Y, Ydot, store)\n store = prepare(Y, store);\n SYdot = A*Ydot - bsxfun(@times, Ydot, store.diagAYYt);\n H = manifold.proj(Y, SYdot);\n end\n\n\n % If no initial guess is available, tell Manopt to use a random one.\n if ~exist('Y0', 'var') || isempty(Y0)\n Y0 = [];\n end\n\n % Call your favorite solver.\n opts = struct();\n opts.verbosity = 0; % Set to 0 for no output, 2 for normal output\n opts.maxinner = 500; % maximum Hessian calls per iteration\n opts.tolgradnorm = 1e-6; % tolerance on gradient norm\n Y = trustregions(problem, Y0, opts);\n \n % If required, produce an optimality certificate.\n if nargout >= 3\n S = A - spdiags(sum((A*Y).*Y, 2), 0, n, n);\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/elliptope_SDP_complex.m", "content": "function [Y, problem, S] = elliptope_SDP_complex(A, p, Y0)\n% Solver for complex semidefinite programs (SDP's) with unit diagonal.\n% \n% function [Y, problem, S] = elliptope_SDP_complex(A)\n% function [Y, problem, S] = elliptope_SDP_complex(A, p)\n% function [Y, problem, S] = elliptope_SDP_complex(A, p, Y0)\n%\n% A is a Hermitian matrix of size n.\n%\n% This function uses a local optimization method in Manopt to solve the SDP\n%\n% min_X trace(A*X) s.t. diag(X) = 1, X is complex, positive semidefinite.\n%\n% In practice, the Hermitian matrix X of size n is parameterized as\n% X = Y*Y', where Y has size n x p. By default, p is taken large enough\n% (that is, sqrt(n)) to ensure that there exists an optimal X whose rank is\n% smaller than p. This ensures that the SDP is equivalent to the new\n% problem in Y:\n%\n% min_Y trace(Y'*A*Y) s.t. diag(Y*Y') = 1, Y complex\n%\n% The constraints on Y require each row of Y to have unit norm, which is\n% why Manopt is appropriate software to solve this problem. An optional\n% initial guess can be specified via the input Y0.\n%\n% See the paper below for theory, specifically, for a proof that, for\n% almost all A, second-order critical points of the problem in Y are\n% globally optimal. In other words: there are no local traps in Y, despite\n% non-convexity.\n%\n% Outputs:\n%\n% Y: is the best point found (an nxp matrix with unit norm rows.)\n% To find X, form Y*Y' (or, more efficiently, study X through Y.)\n% \n% problem: is the Manopt problem structure used to produce Y.\n% \n% S: is a dual optimality certificate (a Hermitian matrix of size n,\n% sparse if A is sparse). The optimality gap (in the cost\n% function) is at most n*min(eig(S)), for both Y and X = Y*Y'.\n% Hence, if min(eig(S)) is close to zero, Y is close to globally\n% optimal. This can be computed via eigs(S, 1, 'SR').\n% \n% Paper: https://arxiv.org/abs/1606.04970\n%\n% @inproceedings{boumal2016bmapproach,\n% author = {Boumal, N. and Voroninski, V. and Bandeira, A.S.},\n% title = {The non-convex {B}urer-{M}onteiro approach works on smooth semidefinite programs},\n% booktitle={Neural Information Processing Systems (NIPS 2016)},\n% year = {2016}\n% }\n% \n% See also: maxcut elliptope_SDP\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Oct. 21, 2016\n% Contributors:\n% Change log:\n\n\n % If no inputs are provided, since this is an example file, generate\n % a random complex matrix. This is for illustration purposes only.\n if ~exist('A', 'var') || isempty(A)\n n = 100;\n A = randn(n) + 1i*randn(n);\n A = (A+A')/sqrt(2*n);\n end\n\n n = size(A, 1);\n assert(n >= 2, 'A must be at least 2x2.');\n assert(size(A, 2) == n, 'A must be square.');\n \n % Force A to be Hermitian\n A = (A+A')/2;\n \n % By default, pick a sufficiently large p (number of columns of Y).\n if ~exist('p', 'var') || isempty(p)\n p = floor(sqrt(n)+1);\n end\n \n assert(p >= 1 && p == round(p), 'p must be an integer >= 1.');\n\n % Pick the manifold of complex n-by-p matrices with unit norm rows.\n manifold = obliquecomplexfactory(p, n, true);\n \n problem.M = manifold;\n \n \n % These three, quick commented lines of code are sufficient to define\n % the cost function and its derivatives. This is good code to write\n % when prototyping. Below, a more advanced use of Manopt is shown,\n % where the redundant computation A*Y is avoided between the gradient\n % and the cost evaluation.\n % % problem.cost = @(Y) .5*sum(sum(real((A*Y).*conj(Y))));\n % % problem.egrad = @(Y) A*Y;\n % % problem.ehess = @(Y, Ydot) A*Ydot;\n \n % Products with A dominate the cost, hence we store the result.\n % This allows to share the results among cost, grad and hess.\n % This is completely optional.\n function store = prepare(Y, store)\n if ~isfield(store, 'AY')\n AY = A*Y;\n store.AY = AY;\n store.diagAYYt = sum(real(AY .* conj(Y)), 2);\n end\n end\n \n % Define the cost function to be /minimized/.\n problem.cost = @cost;\n function [f, store] = cost(Y, store)\n store = prepare(Y, store);\n f = .5*sum(store.diagAYYt);\n end\n\n % Define the Riemannian gradient.\n problem.grad = @grad;\n function [G, store] = grad(Y, store)\n store = prepare(Y, store);\n G = store.AY - bsxfun(@times, Y, store.diagAYYt);\n end\n\n % If you want to, you can specify the Riemannian Hessian as well.\n problem.hess = @hess;\n function [H, store] = hess(Y, Ydot, store)\n store = prepare(Y, store);\n SYdot = A*Ydot - bsxfun(@times, Ydot, store.diagAYYt);\n H = manifold.proj(Y, SYdot);\n end\n\n\n % If no initial guess is available, tell Manopt to use a random one.\n if ~exist('Y0', 'var') || isempty(Y0)\n Y0 = [];\n end\n\n % Call your favorite solver.\n opts = struct();\n opts.verbosity = 0; % Set to 0 for no output, 2 for normal output\n opts.maxinner = 500; % maximum Hessian calls per iteration\n opts.tolgradnorm = 1e-6; % tolerance on gradient norm\n Y = trustregions(problem, Y0, opts);\n \n % If required, produce an optimality certificate.\n if nargout >= 3\n S = A - spdiags(sum(real((A*Y).*conj(Y)), 2), 0, n, n);\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/essential_svd.m", "content": "function essential_svd\n% Sample solution of an optimization problem on the essential manifold.\n%\n% Solves the problem \\sum_{i=1}^N ||E_i-A_i||^2, where E_i are essential\n% matrices. Essential matrices are used in computer vision to represent the\n% epipolar constraint between projected points in two perspective views.\n%\n% Note: the essentialfactory file uses a quotient R1/R2 representation to\n% work with essential matrices. On the other hand, from a user point of \n% view, it is convenient to use the E representation (a matrix of size\n% 3-by-3) to give cost, gradient, and Hessian information. To this end, we\n% provide auxiliary files essential_costE2cost, essential_egradE2egrad, and\n% essential_ehessE2ehess that convert these ingredients to their R1/R2\n% counterparts.\n%\n% See also: essentialfactory essential_costE2cost essential_egradE2egrad\n% essential_ehessE2ehess\n \n% This file is part of Manopt: www.manopt.org.\n% Original author: Roberto Tron, Aug. 8, 2014\n% Contributors: Bamdev Mishra, May 15, 2015.\n\n\n % Make data for the test\n N = 2; % Number of matrices to process in parallel.\n A = multiprod(multiprod(randrot(3, N), essential_hat3([0; 0; 1])), randrot(3, N));\n \n % The essential manifold\n M = essentialfactory(N);\n problem.M = M;\n \n % Function handles of the essential matrix E and Euclidean gradient and Hessian\n costE = @(E) 0.5*sum(multisqnorm(E-A));\n egradE = @(E) E - A;\n ehessE = @(E, U) U;\n\n \n % Manopt descriptions\n problem.cost = @cost;\n function val = cost(X)\n val = essential_costE2cost(X, costE); % Cost\n end\n \n problem.egrad = @egrad;\n function g = egrad(X)\n g = essential_egradE2egrad(X, egradE); % Converts gradient in E to X.\n end\n \n problem.ehess = @ehess;\n function gdot = ehess(X, S)\n gdot = essential_ehessE2ehess(X, egradE, ehessE, S); % Converts Hessian in E to X.\n end\n \n \n % Numerically check the differentials.\n % checkgradient(problem); pause;\n % checkhessian(problem); pause;\n \n %Solve the problem\n Xsol = trustregions(problem);\n \n % Distance between original matrices and decompositions\n val = essential_costE2cost(Xsol, costE);\n fprintf('Distance between original matrices and decompositions is %e \\n', val);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/generalized_eigenvalue_computation.m", "content": "function [Xsol, Ssol] = generalized_eigenvalue_computation(A, B, p)\n% Returns orthonormal basis of the dominant invariant p-subspace of B^-1 A.\n%\n% function [Xsol, Ssol] = generalized_eigenvalue_computation(A, B, p)\n%\n% Input: A is a real, symmetric matrix of size nxn,\n% B is a symmetric positive definite matrix, same size as A\n% p is an integer such that p <= n.\n%\n% Output: Xsol: a real, B-orthonormal matrix X of size nxp such that\n% trace(X'*A*X) is maximized, subject to X'*B*X = identity. \n% That is, the columns of X form a B-orthonormal basis of a\n% dominant subspace of dimension p of B^(-1)*A. These are thus\n% generalized eigenvectors associated with the largest generalized\n% eigenvalues of B^(-1)*A (in no particular order). Sign is\n% important: 2 is deemed a larger eigenvalue than -5.\n% Ssol: the eigenvalues associated with the eigenvectors Xsol, in a\n% vector.\n% \n% We intend to solve the homogeneous system A*X = B*X*S,\n% where S is a diagonal matrix of dominant eigenvalues of B^-1 A.\n%\n%\n% The optimization is performed on the generalized Grassmann manifold, \n% since only the space spanned by the columns of X matters in the\n% optimization problem.\n%\n% The optimization problem that we are solving here is \n% maximize trace(X'*A*X) subject to X'*B*X = eye(p). \n% Consequently, the solutions remain invariant to transformation\n% X --> XQ, where Q is a p-by-p orthogonal matrix. The search space, in\n% essence, is set of equivalence classes\n% [X] = {XQ : X'*B*X = I and Q is orthogonal matrix}. This space is called\n% the generalized Grassmann manifold.\n% Before returning, Q is chosen such that Xsol = Xq matches the output one\n% would expect from eigs.\n%\n% See also dominant_invariant_subspace nonlinear_eigenspace\n\n\n% This file is part of Manopt and is copyrighted. See the license file.\n%\n% Main author: Bamdev Mishra, June 30, 2015.\n% Contributors:\n% Change log:\n%\n% Aug. 10, 2016 (NB): the eigenvectors Xsol are now rotated by Vsol\n% before they are returned, to ensure the output matches what you would\n% normally expect calling eigs.\n \n % Generate some random data to test the function\n if ~exist('A', 'var') || isempty(A)\n n = 128;\n A = randn(n);\n A = (A+A')/2;\n end\n if ~exist('B', 'var') || isempty(B)\n n = size(A, 1);\n e = ones(n, 1);\n B = spdiags([-e 2*e -e], -1:1, n, n); % Symmetric positive definite\n end\n \n if ~exist('p', 'var') || isempty(p)\n p = 3;\n end\n \n % Make sure the input matrix is square and symmetric\n n = size(A, 1);\n\tassert(isreal(A), 'A must be real.')\n assert(size(A, 2) == n, 'A must be square.');\n assert(norm(A-A', 'fro') < n*eps, 'A must be symmetric.');\n\tassert(p <= n, 'p must be smaller than n.');\n \n % Define the cost and its derivatives on the generalized \n % Grassmann manifold, i.e., the column space of all X such that\n % X'*B*X is identity. \n gGr = grassmanngeneralizedfactory(n, p, B);\n \n problem.M = gGr;\n problem.cost = @(X) -trace(X'*A*X);\n problem.egrad = @(X) -2*(A*X); % Only Euclidean gradient needed.\n problem.ehess = @(X, H) -2*(A*H); % Only Euclidean Hessian needed.\n \n % Execute some checks on the derivatives for early debugging.\n % These things can be commented out of course.\n % checkgradient(problem);\n % pause;\n % checkhessian(problem);\n % pause;\n \n % Issue a call to a solver. A random initial guess will be chosen and\n % default options are selected except for the ones we specify here.\n options.Delta_bar = 8*sqrt(p);\n options.tolgradnorm = 1e-7;\n options.verbosity = 2; % set to 0 to silence the solver, 2 for normal output.\n [Xsol, costXsol, info] = trustregions(problem, [], options); %#ok\n \n % To extract the eigenvalues, solve the small p-by-p symmetric \n % eigenvalue problem.\n [Vsol, Dsol] = eig(Xsol'*(A*Xsol));\n Ssol = diag(Dsol);\n \n % To extract the eigenvectors, rotate Xsol by the p-by-p orthogonal\n % matrix Vsol.\n Xsol = Xsol*Vsol;\n \n % This quantity should be small.\n % norm(A*Xsol - B*Xsol*diag(Ssol));\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/generalized_procrustes.m", "content": "function [A, R] = generalized_procrustes(A_measure)\n% Rotationally align clouds of points (generalized Procrustes problem)\n%\n% function X = generalized_procrustes(A_measure)\n%\n% The input is a 3D matrix A_measure of size nxmxN. Each of the N slices\n% A_measure(:, :, i) is a cloud of m points in R^n. These clouds are\n% assumed to be (noisy) rotated versions of a reference cloud Atrue.\n% This algorithm tries to find the optimal rotations to apply to the\n% individual clouds such that they will match each other as much as\n% possible following a least-squares cost.\n%\n% The output A is an estimate of the cloud Atrue (up to rotation). The\n% output R is a 3D matrix of size nxnxN containing the rotation matrices\n% such that R(:, :, i) * A is approximately equal to A_measure(:, :, i).\n\n% This file is part of Manopt and is copyrighted. See the license file.\n%\n% Main author: Nicolas Boumal, July 8, 2013\n% Contributors:\n%\n% Change log:\n% \n \n if ~exist('A_measure', 'var')\n % Generate random data to test the method.\n % There are N clouds of m points in R^n. Each of them is a noisy,\n % rotated version of a reference cloud A. Rotations are uniformly\n % random and noise on each rotated cloud is iid normal with\n % standard deviation sigma.\n n = 3;\n m = 10;\n N = 50;\n % The reference cloud\n Atrue = randn(n, m);\n % A 3D matrix containing the N measured clouds\n sigma = .3;\n A_measure = multiprod(randrot(n, N), Atrue) + sigma*randn(n, m, N);\n else\n [n, m, N] = size(A_measure);\n end\n \n % Construct a manifold structure representing the product of groups of\n % rotations with the Euclidean space for A. We optimize simultaneously\n % for the reference cloud and for the rotations that affect each of the\n % measured clouds. Notice that there is a group invariance because\n % there is no way of telling which orientation the reference cloud\n % should be in.\n tuple.R = rotationsfactory(n, N);\n tuple.A = euclideanfactory(n, m);\n M = productmanifold(tuple);\n\n % Define the cost function here. Points on the manifold M are\n % structures with fields X.A and X.R, containing matrices of sizes\n % respectively nxm and nxnxN. The store structure (the caching system)\n % is used to keep the residue matrix E in memory, as it is also used in\n % the computation of the gradient and of the Hessian. This way, we\n % prevent redundant computations.\n function [f, store] = cost(X, store)\n if ~isfield(store, 'E')\n R = X.R;\n A = X.A;\n store.E = multiprod(R, A) - A_measure;\n end\n E = store.E;\n f = (E(:)'*E(:))/(2*N);\n end\n\n % Riemannian gradient of the cost function.\n function [g, store] = grad(X, store)\n R = X.R;\n A = X.A;\n if ~isfield(store, 'E')\n [~, store] = cost(X, store);\n end\n E = store.E;\n % Compute the Euclidean gradient of the cost wrt the rotations R\n % and wrt the cloud A,\n egrad.R = multiprod(E, A'/N);\n egrad.A = A - mean(multiprod(multitransp(R), A_measure), 3);\n % then transform this Euclidean gradient into the Riemannian\n % gradient.\n g = M.egrad2rgrad(X, egrad);\n store.egrad = egrad;\n end\n\n % It is not necessary to define the Hessian of the cost. We do it\n % mostly to illustrate how to do it and to study the spectrum of the\n % Hessian at the solution (see further down).\n function [h, store] = hess(X, Xdot, store)\n R = X.R;\n A = X.A;\n % Careful: tangent vectors on the rotation group are represented as\n % skew symmetric matrices. To obtain the corresponding vectors in\n % the ambient space, we need a little transformation. This\n % transformation is typically not needed when we compute the\n % formulas for the gradient and the Hessian directly in Riemannian\n % form instead of resorting the egrad2rgrad and ehess2rhess. These\n % latter tools are convenient for prototyping but are not always\n % the most efficient form to execute the computations.\n Rdot = tuple.R.tangent2ambient(R, Xdot.R);\n Adot = Xdot.A;\n if ~isfield(store, 'egrad')\n [~, store] = grad(X, store);\n end\n E = store.E;\n egrad = store.egrad;\n \n ehess.R = multiprod(multiprod(Rdot, A) + multiprod(R, Adot), A') + ...\n multiprod(E, Adot');\n ehess.R = ehess.R / N;\n ehess.A = Adot-mean(multiprod(multitransp(Rdot), A_measure), 3);\n \n h = M.ehess2rhess(X, egrad, ehess, Xdot);\n end\n\n % Setup the problem structure with manifold M and cost+grad functions.\n problem.M = M;\n problem.cost = @cost;\n problem.grad = @grad;\n problem.hess = @hess;\n\n % For debugging, it's always nice to check the gradient a few times.\n % checkgradient(problem);\n % pause;\n % checkhessian(problem);\n % pause;\n \n % Call a solver on our problem. This can probably be much improved if a\n\t% clever initial guess is used instead of a random one.\n X = trustregions(problem);\n A = X.A;\n R = X.R;\n \n % To evaluate the performance of the algorithm, see how well Atrue (the\n % reference cloud) matches A (the found cloud). Since the recovery is\n % up to rotation, apply Kabsch algorithm (or standard Procrustes),\n % i.e., compute the polar factorization to best align Atrue and A.\n if exist('Atrue', 'var')\n [U, ~, V] = svd(Atrue*A');\n Ahat = (U*V')*A;\n fprintf('Registration error: %g.\\n', norm(Atrue-Ahat, 'fro'));\n end\n \n % Plot the spectrum of the Hessian at the solution found.\n % Notice that the invariance of f under a rotation yields dim SO(n),\n % that is, n*(n-1)/2 zero eigenvalues in the Hessian spectrum at the\n % solution. This indicates that critical points are not isolated and\n % can theoretically prevent quadratic convergence. One solution to\n % circumvent this would be to fix one rotation arbitrarily. Another\n % solution would be to work on a quotient manifold. Both can be\n % achieved in Manopt: they simply require a little more work on the\n % manifold description side.\n if M.dim() <= 512\n stairs(sort(hessianspectrum(problem, X)));\n title('Spectrum of the Hessian at the solution found.');\n xlabel('Eigenvalue number (sorted)');\n ylabel('Value of the eigenvalue');\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/low_rank_dist_completion.m", "content": "function [Y, infos, problem_description] = low_rank_dist_completion(problem_description)\n% Perform low-rank distance matrix completion w/ automatic rank detection.\n%\n% function Y = low_rank_dist_completion(problem_description)\n% function [Y, infos, out_problem_description] = low_rank_dist_completion(problem_description)\n%\n% It implements the ideas of Journee, Bach, Absil and Sepulchre, SIOPT, 2010,\n% applied to the problem of low-rank Euclidean distance matrix completion.\n% The details are in the paper \"Low-rank optimization for distance matrix completion\",\n% B. Mishra, G. Meyer, and R. Sepulchre, IEEE CDC, 2011.\n%\n% Paper link: http://arxiv.org/abs/1304.6663.\n%\n% Input:\n% -------\n%\n% problem_description: The problem structure with the description of the problem.\n%\n%\n% - problem_description.data_train: Data structure for known distances that are used to learn a low-rank model.\n% It contains the 3 fields that are shown\n% below. An empty \"data_train\" structure\n% will generate the 3d Helix instance.\n%\n% -- data_train.entries: A column vector consisting of known\n% distances. An empty \"data_train.entries\"\n% field will generate the 3d Helix\n% instance.\n%\n% -- data_train.rows: The row position of th corresponding\n% distances. An empty \"data_train.rows\"\n% field will generate the 3d Helix\n% instance.\n%\n% -- data_train.cols: The column position of th corresponding\n% distances. An empty \"data_train.cols\"\n% field will generate the 3d Helix\n% instance.\n%\n%\n%\n% - problem_description.data_test: Data structure to compute distances for the \"unknown\" (to the algorithm) distances.\n% It contains the 3 fields that are shown\n% below. An empty \"data_test\" structure\n% will not compute the test error.\n%\n% -- data_test.entries: A column vector consisting of \"unknown\" (to the algorithm)\n% distances. An empty \"data_test.entries\"\n% field will not compute the test error.\n% -- data_test.rows: The row position of th corresponding\n% distances. An empty \"data_test.rows\"\n% field will not compute the test error.\n% -- data_test.cols: The column position of th corresponding\n% distances. An empty \"data_test.cols\"\n% field will not compute the test error.\n%\n%\n%\n% - problem_description.n: The number of data points. An empty\n% \"n\", but complete \"data_train\" structure\n% will lead to an error, to avoid\n% potential data inconsistency.\n%\n%\n%\n%\n%\n% - problem_description.rank_initial: Starting rank. By default, it is 1.\n%\n%\n%\n% - problem_description.rank_max: Maximum rank. By default, it is equal to\n% \"problem_description.n\".\n%\n%\n%\n%\n% - problem_description.params: Structure array containing algorithm\n% parameters for stopping criteria.\n% -- params.abstolcost: Tolerance on absolute value of cost.\n% By default, it is 1e-3.\n%\n%\n% -- params.reltolcost: Tolerance on absolute value of cost.\n% By default, it is 1e-3.\n% -- params.tolgradnorm: Tolerance on the norm of the gradient.\n% By default, it is 1e-5.\n% -- params.maxiter: Maximum number of fixe-rank iterations.\n% By default, it is 100.\n% -- params.tolSmin: Tolerance on smallest eigenvalue of Sy,\n% the dual variable.\n% By default, it is 1e-5.\n% -- params.tolrankdeficiency: Tolerance on the\n% smallest singular value of Y.\n% By default, it is 1e-3.\n% -- params.solver: Fixed-rank algorithm. Options are\n% '@trustregions' for trust-regions,\n% '@conjugategradient' for conjugate gradients,\n% '@steepestdescent' for steepest descent.\n% By default, it is '@trustregions'.\n%\n%\n% Output:\n% --------\n%\n% Y: n-by-r solution matrix of rank r.\n% infos: Structure array with computed statistics.\n% problem_description: Structure array with used problem description.\n%\n%\n%\n% Please cite the Manopt paper as well as the research paper:\n% @InProceedings{mishra2011dist,\n% Title = {Low-rank optimization for distance matrix completion},\n% Author = {Mishra, B. and Meyer, G. and Sepulchre, R.},\n% Booktitle = {{50th IEEE Conference on Decision and Control}},\n% Year = {2011},\n% Organization = {{IEEE CDC}}\n% }\n\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, April 06, 2015.\n% Contributors: Nicolas Boumal.\n% Change log: \n% August 30 2016 (BM): \n% Corrected some logic flaws while plotting and storing\n% rank information. A typo was also corrected.\n\n \n % Check problem description\n if ~exist('problem_description', 'var')\n problem_description = struct();\n end\n problem_description = check_problem_description(problem_description); % Check the problem description;\n \n \n % Common quantities\n data_train = problem_description.data_train;\n data_test = problem_description.data_test;\n n = problem_description.n;\n rank_initial = problem_description.rank_initial;\n rank_max = problem_description.rank_max;\n params = problem_description.params;\n N = data_train.nentries; % Number of known distances\n EIJ = speye(n);\n EIJ = EIJ(:, data_train.rows) - EIJ(:, data_train.cols);\n rr = rank_initial; % Starting rank.\n Y = randn(n, rr); % Random starting initialization.\n \n \n % Information\n time = []; % Time for each iteration per rank\n cost = []; % Cost at each iteration per rank\n test_error = []; % Test error at each iteration per rank\n rank = []; % Rank at each iteration\n rank_change_stats = []; % Some stats relating the change of ranks\n \n \n \n % Main loop rank search\n rank_search = 0;\n while (rr <= rank_max), % When r = n a global min is attained for sure.\n rank_search = rank_search + 1;\n \n fprintf('>> Rank %d <<\\n', rr);\n \n % Follow the descent direction to compute an iterate in a higher dimension\n if (rr > rank_initial),\n if isempty(restartDir), % If no restart dir avail. do a random restart\n disp('No restart dir available, random restart is performed');\n Y = randn(n, rr);\n \n else % Perform a simple line-search based on the restart direction\n disp('>> Line-search with restart direction');\n Y(:, rr) = 0; % Append a column of zeroes\n \n Z = Y(data_train.rows, :) - Y(data_train.cols,:);\n estimDists = sum(Z.^2, 2);\n errors = (estimDists - data_train.entries);\n costBefore = 0.5*mean(errors.^2);\n fprintf('>> Cost before = %f\\n',costBefore);\n \n % Simple linesearch to maintain monotonicity\n problem.M = symfixedrankYYfactory(n, rr);\n problem.cost = @(Y) cost_evaluation(Y, data_train);\n d = zeros(size(Y));\n d(:, rr) = restartDir;\n [unused, Y] = linesearch_decrease(problem, Y, d, costBefore); %#ok\n \n Z = Y(data_train.rows, :) - Y(data_train.cols,:);\n estimDists = sum(Z.^2, 2);\n errors = (estimDists - data_train.entries);\n costAfter = 0.5*mean(errors.^2);\n \n % Check for decrease\n if costAfter >= costBefore - 1e-8\n disp('Decrease is not sufficient, random restart');\n Y = randn(n, rr);\n end\n \n end\n \n end\n \n % Fixed-rank optimization with Manopt\n [Y, infos_fixedrank] = low_rank_dist_completion_fixedrank(data_train, data_test, Y, params);\n\n % Some info logging\n thistime = [infos_fixedrank.time];\n if ~isempty(time)\n thistime = time(end) + thistime;\n end\n time = [time thistime]; %#ok\n cost = [cost [infos_fixedrank.cost]]; %#ok\n rank = [rank [infos_fixedrank.rank]]; %#ok\n rank_change_stats(rank_search).rank = rr; %#ok\n rank_change_stats(rank_search).iter = length([infos_fixedrank.cost]); %#ok\n rank_change_stats(rank_search).Y = Y; %#ok\n if isfield(infos_fixedrank, 'test_error')\n test_error = [test_error [infos_fixedrank.test_error]]; %#ok\n end\n \n \n % Evaluate gradient of the convex cost function (i.e. wrt X).\n Z = Y(data_train.rows, :) - Y(data_train.cols,:);\n estimDists = sum(Z.^2,2);\n errors = (estimDists - data_train.entries);\n \n \n % Dual variable and its minimum eigenvalue that is used to guarantee convergence.\n Sy = (0.5)*EIJ * sparse(1:N,1:N,2 * errors / N,N,N) * EIJ'; % \"0.5\" comes from 0.5 in cost evaluation \n \n \n % Compute smallest algebraic eigenvalue of Sy,\n % this gives us a descent direction for the next rank (v)\n % as well as a way to control progress toward the global\n % optimum (s_min).\n \n % Make eigs silent.\n opts.disp = 0;\n opts.issym = true;\n [v, s_min] = eigs(Sy, 1, 'SA', opts);\n \n \n % Check whether Y is rank deficient.\n vp = svd(Y);\n \n % Stopping criterion.\n fprintf('>> smin = %.3e, and min(vp) = %.3e\\n',s_min,min(vp));\n if (s_min > params.tolSmin) || (min(vp) < params.tolrankdeficiency),\n break;\n end\n \n % Update rank\n rr = rr + 1;\n \n % Compute descent direction\n if (s_min < -1e-10),\n restartDir = v;\n else\n restartDir = [];\n end\n end\n \n \n % Collect relevant statistics\n infos.time = time;\n infos.cost = cost;\n infos.rank = rank;\n infos.test_error = test_error;\n infos.rank_change_stats = rank_change_stats;\n \n % Few plots.\n show_plots(problem_description, infos);\n \nend\n\n\n\n\n%% Cost function evaluation.\nfunction val = cost_evaluation(Y, data_train)\n Z = Y(data_train.rows, :) - Y(data_train.cols,:);\n estimDists = sum(Z.^2, 2);\n errors = (estimDists - data_train.entries);\n val = 0.5*mean(errors.^2);\nend\n\n\n\n\n%% Local defaults\nfunction localdefaults = getlocaldefaults()\n localdefaults.abstolcost = 1e-3;\n localdefaults.reltolcost = 1e-3;\n localdefaults.tolSmin = -1e-3;\n localdefaults.tolrankdeficiency = 1e-3;\n localdefaults.tolgradnorm = 1e-5;\n localdefaults.maxiter = 100;\n localdefaults.solver = @trustregions; % Trust-regions\nend\n\n\n\n\n\n\n\n%% Fixed-rank optimization\nfunction [Yopt, infos] = low_rank_dist_completion_fixedrank(data_train, data_test, Y_initial, params)\n % Common quantities that are used often in the optimization process.\n [n, r] = size(Y_initial);\n EIJ = speye(n);\n EIJ = EIJ(:, data_train.rows) - EIJ(:, data_train.cols);\n \n % Create problem structure\n problem.M = symfixedrankYYfactory(n, r);\n \n \n % Cost evaluation\n problem.cost = @cost;\n function [f, store] = cost(Y, store)\n if ~isfield(store, 'xij')\n store.xij = EIJ'*Y;\n end\n xij = store.xij;\n estimDists = sum(xij.^2,2);\n f = 0.5*mean((estimDists - data_train.entries).^2);\n end\n \n % Gradient evaluation\n problem.grad = @grad;\n function [g, store] = grad(Y, store)\n N = data_train.nentries;\n if ~isfield(store, 'xij')\n store.xij = EIJ'*Y;\n end\n xij = store.xij;\n estimDists = sum(xij.^2,2);\n g = EIJ * sparse(1:N,1:N,2 * (estimDists - data_train.entries) / N, N, N) * xij;\n end\n \n \n % Hessian evaluation\n problem.hess = @hess;\n function [Hess, store] = hess(Y, eta, store)\n N = data_train.nentries;\n if ~isfield(store, 'xij')\n store.xij = EIJ'*Y;\n end\n xij = store.xij;\n zij = EIJ'*eta;\n estimDists = sum(xij.^2,2);\n crossYZ = 2*sum(xij .* zij,2);\n Hess = (EIJ*sparse(1:N,1:N,2 * (estimDists - data_train.entries) / N,N,N))*zij + (EIJ*sparse(1:N,1:N,2 * crossYZ / N,N,N))*xij;\n Hess = problem.M.proj(Y, Hess);\n end\n \n \n % % Check numerically whether gradient and Hessian are correct\n % checkgradient(problem);\n % drawnow;\n % pause;\n % checkhessian(problem);\n % drawnow;\n % pause;\n \n \n % When asked, ask Manopt to compute the test error at every iteration.\n if ~isempty(data_test)\n options.statsfun = @compute_test_error;\n EIJ_test = speye(n);\n EIJ_test = EIJ_test(:, data_test.rows) - EIJ_test(:, data_test.cols);\n end\n function stats = compute_test_error(problem, Y, stats) %#ok\n xij = EIJ_test'*Y;\n estimDists_test = sum(xij.^2,2);\n stats.test_error = 0.5*mean((estimDists_test - data_test.entries).^2);\n end\n \n \n % Stopping criteria options\n options.stopfun = @mystopfun;\n function stopnow = mystopfun(problem, Y, info, last) %#ok\n stopnow = (last >= 5 && (info(last-2).cost - info(last).cost < params.abstolcost || abs(info(last-2).cost - info(last).cost)/info(last).cost < params.reltolcost));\n end\n options.tolgradnorm = params.tolgradnorm;\n options.maxiter = params.maxiter;\n \n \n % Call appropriate algorithm\n options.solver = params.solver;\n [Yopt, ~, infos] = manoptsolve(problem, Y_initial, options);\n [infos.rank] = deal(r);\nend\n\n\n\n\n\n\n%% 3d Helix problem instance\nfunction problem_description = get_3d_Helix_instance()\n \n % Helix curve in 3d\n tvec = 0:2*pi/100:2*pi;\n tvec = tvec'; % column vector\n xvec = 4*cos(3*tvec);\n yvec = 4*sin(3*tvec);\n zvec = 2*tvec;\n Yo = [xvec, yvec, zvec];\n n = size(Yo, 1); % Number of points\n \n % Fraction of unknown distances\n fractionOfUnknown = 0.85;\n \n % True distances among points in 3d Helix\n trueDists = pdist(Yo)'.^2; % True distances\n \n \n % Add noise (set noise_level = 0 for clean measurements)\n noise_level = 0; % 0.01;\n trueDists = trueDists + noise_level * std(trueDists) * randn(size(trueDists));\n \n \n % Compute all pairs of indices\n H = tril(true(n), -1);\n [I, J] = ind2sub([n, n], find(H(:)));\n clear 'H';\n \n \n % Train data\n train = false(length(trueDists), 1);\n train(1:floor(length(trueDists)*(1- fractionOfUnknown))) = true;\n train = train(randperm(length(train)));\n \n data_train.rows = I(train);\n data_train.cols = J(train);\n data_train.entries = trueDists(train);\n data_train.nentries = length(data_train.entries);\n \n \n % Test data\n data_test.nentries = 1*data_train.nentries; % Depends how big data that we can handle.\n test = false(length(trueDists),1);\n test(1 : floor(data_test.nentries)) = true;\n test = test(randperm(length(test)));\n data_test.rows = I(test);\n data_test.cols = J(test);\n data_test.entries = trueDists(test);\n \n \n % Rank bounds\n rank_initial = 1; % Starting rank\n rank_max = n; % Maximum rank\n \n \n % Basic parameters used in optimization\n params = struct();\n params = mergeOptions(getlocaldefaults, params);\n \n \n % Problem description\n problem_description.data_train = data_train;\n problem_description.data_test = data_test;\n problem_description.n = n;\n problem_description.rank_initial = rank_initial;\n problem_description.rank_max = rank_max;\n problem_description.params = params;\n problem_description.Yo = Yo; % Store original Helix structure\nend\n\n\n\n\n\n%% Problem description check\nfunction checked_problem_description = check_problem_description(problem_description)\n checked_problem_description = problem_description;\n \n % Check train data\n if isempty(problem_description)...\n || ~all(isfield(problem_description,{'data_train'}) == 1)...\n || ~all(isfield(problem_description.data_train,{'cols', 'rows', 'entries'}) == 1)...\n || isempty(problem_description.data_train.cols)...\n || isempty(problem_description.data_train.rows)...\n || isempty(problem_description.data_train.entries)\n \n warning('low_rank_dist_completion:problem_description', ...\n 'The training set is empty or not properly defined. We work with the default 3d Helix example.\\n');\n checked_problem_description = get_3d_Helix_instance();\n checked_problem_description.helix_example = true;\n return; % No need for further check\n end\n \n \n % Check number of data points\n if ~isfield(problem_description, 'n')\n error('low_rank_dist_completion:problem_description',...\n 'Error. The scalar corresponding to field \"n\" of problem description must be given. \\n');\n end\n \n \n % Check initial rank\n if ~isfield(problem_description, 'rank_initial')...\n || isempty(problem_description.rank_initial)...\n || ~(floor(problem_description.rank_initial) == problem_description.rank_initial)\n warning('low_rank_dist_completion:problem_description', ...\n 'The field \"rank_initial\" is not properly defined. We work with the default \"1\".\\n');\n rank_initial = 1;\n else\n rank_initial = problem_description.rank_initial;\n end\n checked_problem_description.rank_initial = rank_initial;\n \n \n % Check maximum rank\n if ~isfield(problem_description, 'rank_max')...\n || isempty(problem_description.rank_max)...\n || ~(floor(problem_description.rank_max) == problem_description.rank_max)...\n || problem_description.rank_max > problem_description.n\n warning('low_rank_dist_completion:problem_description', ...\n 'The field \"rank_max\" is not properly defined. We work with the default \"n\".\\n');\n rank_max = problem_description.n;\n else\n rank_max = problem_description.rank_max;\n end\n checked_problem_description.rank_max = rank_max;\n \n \n % Check testing dataset\n if ~isfield(problem_description,{'data_test'})...\n || ~all(isfield(problem_description.data_test,{'cols', 'rows', 'entries'}) == 1)...\n || isempty(problem_description.data_test.cols)...\n || isempty(problem_description.data_test.rows)...\n || isempty(problem_description.data_test.entries)\n \n warning('low_rank_dist_completion:problem_description', ...\n 'The field \"data_test\" is not properly defined. We work with the default \"[]\".\\n');\n data_test = [];\n else\n data_test = problem_description.data_test;\n end\n checked_problem_description.data_test = data_test;\n \n \n % Check parameters\n if isfield(problem_description, 'params')\n params = problem_description.params;\n else\n params = struct();\n end\n params = mergeOptions(getlocaldefaults, params);\n checked_problem_description.params = params;\n \nend\n\n\n\n\n%% Show plots\nfunction show_plots(problem_description, infos)\n \n solver = problem_description.params.solver;\n rank_change_stats = infos.rank_change_stats;\n rank_change_stats_rank = [rank_change_stats.rank];\n rank_change_stats_iter = [rank_change_stats.iter];\n rank_change_stats_iter = cumsum(rank_change_stats_iter);\n N = problem_description.data_train.nentries;\n n = problem_description.n;\n \n \n % Plot: train error\n fs = 20;\n figure('name', 'Training on the known distances');\n \n line(1:length([infos.cost]),log10([infos.cost]),'Marker','O','LineStyle','-','Color','blue','LineWidth',1.5);\n ax1 = gca;\n \n set(ax1,'FontSize',fs);\n xlabel(ax1,'Number of iterations','FontSize',fs);\n ylabel(ax1,'Cost (log scale) on known distances','FontSize',fs);\n \n ax2 = axes('Position',get(ax1,'Position'),...\n 'XAxisLocation','top',...\n 'YAxisLocation','right',...\n 'Color','none',...\n 'XColor','k');\n \n set(ax2,'FontSize',fs);\n line(1:length([infos.cost]),log10([infos.cost]),'Marker','O','LineStyle','-','Color','blue','LineWidth',1.5,'Parent',ax2);\n set(ax2,'XTick',rank_change_stats_iter(1:max(1,end-1)),...\n 'XTickLabel',rank_change_stats_rank(1) + 1 : rank_change_stats_rank(max(1,end-1)) + 1,...\n 'YTick',[]);\n \n set(ax2,'XGrid','on');\n legend(func2str(solver));\n title('Rank');\n legend 'boxoff';\n \n \n % Plot: test error\n if isfield(infos, 'test_error') && ~isempty(infos.test_error)\n Yo = problem_description.Yo;\n \n fs = 20;\n figure('name','Test error on a set of distances different from the training set');\n \n line(1:length([infos.test_error]),log10([infos.test_error]),'Marker','O','LineStyle','-','Color','blue','LineWidth',1.5);\n ax1 = gca;\n \n set(ax1,'FontSize',fs);\n xlabel(ax1,'Number of iterations','FontSize',fs);\n ylabel(ax1,'Cost (log scale) on testing set','FontSize',fs);\n \n ax2 = axes('Position',get(ax1,'Position'),...\n 'XAxisLocation','top',...\n 'YAxisLocation','right',...\n 'Color','none',...\n 'XColor','k');\n \n set(ax2,'FontSize',fs);\n line(1:length([infos.test_error]),log10([infos.test_error]),'Marker','O','LineStyle','-','Color','blue','LineWidth',1.5,'Parent',ax2);\n set(ax2,'XTick',rank_change_stats_iter(1:max(1,end-1)),...\n 'XTickLabel',rank_change_stats_rank(1) + 1 : rank_change_stats_rank(max(1,end-1)) + 1,...\n 'YTick',[]);\n \n set(ax2,'XGrid','on');\n legend(func2str(solver));\n title('Rank');\n legend 'boxoff';\n \n \n \n end\n \n \n % Plot: visualize Helix curve\n if isfield(problem_description, 'helix_example')\n jj = ceil((length(rank_change_stats_rank) + 1)/2);\n \n \n figure('name',['3D structure with ', num2str(N/((n^2 -n)/2)),' fraction known distances'])\n fs = 20;\n ax1 = gca;\n set(ax1,'FontSize',fs);\n subplot(jj,2,1);\n plot3(Yo(:,1), Yo(:,2), Yo(:,3),'*','Color', 'b','LineWidth',1.0);\n title('Original 3D structure');\n for kk = 1 : length(rank_change_stats_rank)\n subplot(jj, 2, kk + 1);\n rank_change_stats_kk = rank_change_stats(kk);\n Ykk = rank_change_stats_kk.Y;\n if size(Ykk, 2) == 1,\n plot(Ykk(:,1), zeros(size(Ykk, 1)),'*','Color', 'r','LineWidth',1.0);\n legend(func2str(solver))\n title(['Recovery at rank ',num2str(size(Ykk, 2))]);\n \n elseif size(Ykk, 2) == 2\n plot(Ykk(:,1), Ykk(:,2),'*','Color', 'r','LineWidth',1.0);\n title(['Recovery at rank ',num2str(size(Ykk, 2))]);\n \n else % Project onto dominant 3Dsubspace\n [U1, S1, V1] = svds(Ykk, 3);\n Yhat = U1*S1*V1';\n plot3(Yhat(:,1), Yhat(:,2), Yhat(:,3),'*','Color', 'r','LineWidth',1.0);\n title(['Recovery at rank ',num2str(size(Ykk, 2))]);\n end\n \n axis equal;\n \n end\n \n % Trick to add a global title to the whole subplot collection.\n % HitTest is disabled to make it easier to select the individual\n % subplots (for example, to rotate the viewing angle).\n ha = axes('Position',[0 0 1 1],'Xlim',[0 1],'Ylim',[0 1],'Box','off','Visible','off','Units','normalized', 'clipping' , 'off' );\n set(ha, 'HitTest', 'off');\n text(0.5, 1,['Recovery of Helix from ',num2str(N/((n^2 -n)/2)),' fraction known distances'],'HorizontalAlignment','center','VerticalAlignment', 'top');\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/low_rank_matrix_completion.m", "content": "function low_rank_matrix_completion()\n% Given partial observation of a low rank matrix, attempts to complete it.\n%\n% function low_rank_matrix_completion()\n%\n% This example demonstrates how to use the geometry factory for the\n% embedded submanifold of fixed-rank matrices, fixedrankembeddedfactory.\n% This geometry is described in the paper\n% \"Low-rank matrix completion by Riemannian optimization\"\n% Bart Vandereycken - SIAM Journal on Optimization, 2013.\n%\n% This can be a starting point for many optimization problems of the form:\n%\n% minimize f(X) such that rank(X) = k, size(X) = [m, n].\n%\n% Note that the code is long because it showcases quite a few features of\n% Manopt: most of the code is optional.\n%\n% Input: None. This example file generates random data.\n% \n% Output: None.\n\n% This file is part of Manopt and is copyrighted. See the license file.\n% \n% Main author: Nicolas Boumal, July 15, 2014\n% Contributors: Bart Vandereycken\n% \n% Change log:\n% \n \n % Random data generation. First, choose the size of the problem.\n % We will complete a matrix of size mxn of rank k:\n m = 200;\n n = 500;\n k = 10;\n % Generate a random mxn matrix A of rank k\n L = randn(m, k);\n R = randn(n, k);\n A = L*R';\n % Generate a random mask for observed entries: P(i, j) = 1 if the entry\n % (i, j) of A is observed, and 0 otherwise.\n fraction = 4 * k*(m+n-k)/(m*n);\n P = sparse(rand(m, n) <= fraction);\n % Hence, we know the nonzero entries in PA:\n PA = P.*A;\n \n \n \n \n \n \n \n % Pick the manifold of matrices of size mxn of fixed rank k.\n problem.M = fixedrankembeddedfactory(m, n, k);\n\n % Define the problem cost function. The input X is a structure with\n % fields U, S, V representing a rank k matrix as U*S*V'.\n % f(X) = 1/2 * || P.*(X-A) ||^2\n problem.cost = @cost;\n function f = cost(X)\n % Note that it is very much inefficient to explicitly construct the\n % matrix X in this way. Seen as we only need to know the entries\n % of Xmat corresponding to the mask P, it would be far more\n % efficient to compute those only.\n Xmat = X.U*X.S*X.V';\n f = .5*norm( P.*Xmat - PA , 'fro')^2;\n end\n\n % Define the Euclidean gradient of the cost function, that is, the\n % gradient of f(X) seen as a standard function of X.\n % nabla f(X) = P.*(X-A)\n problem.egrad = @egrad;\n function G = egrad(X)\n % Same comment here about Xmat.\n Xmat = X.U*X.S*X.V';\n G = P.*Xmat - PA;\n end\n\n % This is optional, but it's nice if you have it.\n % Define the Euclidean Hessian of the cost at X, along H, where H is\n % represented as a tangent vector: a structure with fields Up, Vp, M.\n % This is the directional derivative of nabla f(X) at X along Xdot:\n % nabla^2 f(X)[Xdot] = P.*Xdot\n problem.ehess = @euclidean_hessian;\n function ehess = euclidean_hessian(X, H)\n % The function tangent2ambient transforms H (a tangent vector) into\n % its equivalent ambient vector representation. The output is a\n % structure with fields U, S, V such that U*S*V' is an mxn matrix\n % corresponding to the tangent vector H. Note that there are no\n % additional guarantees about U, S and V. In particular, U and V\n % are not orthonormal.\n ambient_H = problem.M.tangent2ambient(X, H);\n Xdot = ambient_H.U*ambient_H.S*ambient_H.V';\n % Same comment here about explicitly constructing the ambient\n % vector as an mxn matrix Xdot: we only need its entries\n % corresponding to the mask P, and this could be computed\n % efficiently.\n ehess = P.*Xdot;\n end\n \n\n\n\n % Check consistency of the gradient and the Hessian. Useful if you\n % adapt this example for a new cost function and you would like to make\n % sure there is no mistake.\n % warning('off', 'manopt:fixedrankembeddedfactory:exp');\n % checkgradient(problem); pause;\n % checkhessian(problem); pause;\n \n \n \n \n \n % Compute an initial guess. Points on the manifold are represented as\n % structures with three fields: U, S and V. U and V need to be\n % orthonormal, S needs to be diagonal.\n [U, S, V] = svds(PA, k);\n X0.U = U;\n X0.S = S;\n X0.V = V;\n \n % Minimize the cost function using Riemannian trust-regions, starting\n % from the initial guess X0.\n X = trustregions(problem, X0);\n \n % The reconstructed matrix is X, represented as a structure with fields\n % U, S and V.\n Xmat = X.U*X.S*X.V';\n fprintf('||X-A||_F = %g\\n', norm(Xmat - A, 'fro'));\n \n \n \n \n \n \n % Alternatively, we could decide to use a solver such as\n % steepestdescent or conjugategradient. These solvers need to solve a\n % line-search problem at each iteration. Standard line searches in\n % Manopt have generic purpose systems to do this. But for the problem\n % at hand, it so happens that we can rather accurately guess how far\n % the line-search should look, and it would be a waste to not use that.\n % Look up the paper referenced above for the mathematical explanation\n % of the code below.\n % \n % To tell Manopt about this special information, we specify the\n % linesearch hint function in the problem structure. Notice that this\n % is not the same thing as specifying a linesearch function in the\n % options structure.\n % \n % Both the SD and the CG solvers will detect that we\n % specify the hint function below, and they will use an appropriate\n % linesearch algorithm by default, as a result. Typically, they will\n % try the step t*H first, then if it does not satisfy an Armijo\n % criterion, they will decrease t geometrically until satisfaction or\n % failure.\n % \n % Just like the cost, egrad and ehess functions, the linesearch\n % function could use a store structure if you like. The present code\n % does not use the store structure, which means quite a bit of the\n % computations are made redundantly, and as a result a better method\n % could appear slower. See the Manopt tutorial about caching when you\n % are ready to switch from a proof-of-concept code to an efficient\n % code.\n %\n % The inputs are X (a point on the manifold) and H, a tangent vector at\n % X that is assumed to be a descent direction. That is, there exists a\n % positive t such that f(Retraction_X(tH)) < f(X). The function below\n % is supposed to output a \"t\" that it is a good \"guess\" at such a t.\n problem.linesearch = @linesearch_helper;\n function t = linesearch_helper(X, H)\n % Note that you would not usually need the Hessian for this.\n residual_omega = nonzeros(problem.egrad(X));\n dir_omega = nonzeros(problem.ehess(X, H));\n t = - dir_omega \\ residual_omega ;\n end\n\n % Notice that for this solver, the Hessian is not needed.\n [Xcg, xcost, info, options] = conjugategradient(problem, X0); %#ok\n \n fprintf('Take a look at the options that CG used:\\n');\n disp(options);\n fprintf('And see how many trials were made at each line search call:\\n');\n info_ls = [info.linesearch];\n disp([info_ls.costevals]);\n\n \n \n \n \n \n \n \n fprintf('Try it again without the linesearch helper.\\n');\n \n % Remove the linesearch helper from the problem structure.\n problem = rmfield(problem, 'linesearch');\n \n [Xcg, xcost, info, options] = conjugategradient(problem, X0); %#ok\n \n fprintf('Take a look at the options that CG used:\\n');\n disp(options);\n fprintf('And see how many trials were made at each line search call:\\n');\n info_ls = [info.linesearch];\n disp([info_ls.costevals]);\n \n \n \n \n \n \n\n % If the problem has a small enough dimension, we may (for analysis\n % purposes) compute the spectrum of the Hessian at a point X. This may\n % help in studying the conditioning of a problem. If you don't provide\n % the Hessian, Manopt will approximate the Hessian with finite\n % differences of the gradient and try to estimate its \"spectrum\" (it's\n % not a proper linear operator). This can give some intuition, but\n % should not be relied upon.\n if problem.M.dim() < 100\n fprintf('Computing the spectrum of the Hessian...');\n s = hessianspectrum(problem, X);\n hist(s);\n end\n \n \n \n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/low_rank_tensor_completion.m", "content": "function low_rank_tensor_completion()\n% Given partial observation of a low rank tensor, attempts to complete it.\n%\n% function low_rank_tensor_completion()\n%\n% This example demonstrates how to use the geometry factory for the\n% quotient manifold of fixed-rank tensors, \n% fixedrankfactory_tucker_preconditioned.\n%\n% This geometry is described in the technical report\n% \"Riemannian preconditioning for tensor completion\"\n% Hiroyuki Kasai and Bamdev Mishra, arXiv:1506.02159, 2015.\n%\n% This can be a starting point for many optimization problems of the form:\n%\n% minimize f(X) such that rank(X) = [r1 r2 r3], size(X) = [n1, n2, n3].\n%\n% Input: None. This example file generates random data.\n% \n% Output: None.\n%\n% Please cite the Manopt paper as well as the research paper:\n% @Techreport{kasai2015,\n% Title = {{R}iemannian preconditioning for tensor completion},\n% Author = {Kasai, H. and Mishra, B.},\n% Journal = {Arxiv preprint arXiv:1506.02159},\n% Year = {2015}\n% }\n\n% This file is part of Manopt and is copyrighted. See the license file.\n% \n% Main authors: Hiroyuki Kasai and Bamdev Mishra, June 16, 2015.\n% Contributors:\n% \n% Change log:\n% \n \n\n % Random data generation with pseudo-random numbers from a \n % uniform distribution on [0, 1].\n % First, choose the size of the problem.\n % We will complete a tensor of size n1-by-n2-by-n3 of rank (r1, r2, r3): \n n1 = 70;\n n2 = 60;\n n3 = 50;\n r1 = 3;\n r2 = 4;\n r3 = 5;\n tensor_dims = [n1 n2 n3];\n core_dims = [r1 r2 r3];\n total_entries = n1*n2*n3;\n \n % Generate a random tensor A of size n1-by-n2-by-n3 of rank (r1, r2, r3).\n [U1,R1] = qr(rand(n1, r1), 0);\n [U2,R2] = qr(rand(n2, r2), 0);\n [U3,R3] = qr(rand(n3, r3), 0);\n\n Z.U1 = R1;\n Z.U2 = R2;\n Z.U3 = R3; \n Z.G = rand( core_dims );\n Core = tucker2multiarray(Z); % Converts tucker format tensor to full tensor.\n\n Y.U1 = U1;\n Y.U2 = U2;\n Y.U3 = U3;\n Y.G = Core;\n A = tucker2multiarray(Y); \n \n % Generate a random mask P for observed entries: P(i, j, k) = 1 if the entry\n % (i, j, k) of A is observed, and 0 otherwise. \n % Observation ratio\n fraction = 0.1; % Fraction of known entries.\n nr = round(fraction * total_entries);\n ind = randperm(total_entries);\n ind = ind(1 : nr);\n P = false(tensor_dims);\n P(ind) = true; \n % Hence, we know the nonzero entries in PA:\n PA = P.*A; \n \n\n \n \n % Pick the manifold of tensors of size n1-by-n2-by-n3 of rank (r1, r2, r3).\n problem.M = fixedrankfactory_tucker_preconditioned(tensor_dims, core_dims);\n \n \n \n \n % Define the problem cost function. The input X is a structure with\n % fields U1, U2, U3, G representing a rank (r1,r2,r3) tensor.\n % f(X) = 1/2 * || P.*(X - A) ||^2\n problem.cost = @cost;\n function f = cost(X)\n Xmultiarray = tucker2multiarray(X);\n Diffmultiarray = P.*Xmultiarray - PA;\n Diffmultiarray_flat = reshape(Diffmultiarray, n1, n2*n3);\n f = .5*norm(Diffmultiarray_flat , 'fro')^2;\n end\n\n\n \n \n % Define the Euclidean gradient of the cost function, that is, the\n % gradient of f(X) seen as a standard function of X.\n % nabla f(X) = P.*(X-A)\n % We only need to give the Euclidean gradient. Manopt converts it\n % internally to the Riemannian counterpart.\n problem.egrad = @egrad;\n function [g] = egrad(X)\n Xmultiarray = tucker2multiarray(X);\n Smultiarray = P.*Xmultiarray - PA; \n\n % BM: computation of S, S1, S2, and S3\n S1multiarray = reshape(Smultiarray, [n1, n2*n3]);\n S2multiarray = reshape(permute(Smultiarray, [2 1 3]),[n2, n1*n3]);\n S3multiarray = reshape(permute(Smultiarray, [3 1 2]),[n3, n1*n2]);\n\n g.U1 = double(S1multiarray) * kron(X.U3, X.U2) * reshape(X.G, r1, r2*r3)';\n g.U2 = double(S2multiarray) * kron(X.U3, X.U1) * reshape(permute(X.G, [2 1 3]), r2, r1*r3)';\n g.U3 = double(S3multiarray) * kron(X.U2, X.U1) * reshape(permute(X.G, [3 1 2]), r3, r1*r2)';\n g.G = reshape(X.U1' * reshape(double(Smultiarray),n1,n2*n3) * kron(X.U3', X.U2')', r1, r2, r3); \n end\n \n \n \n \n \n % Define the Euclidean Hessian of the cost at X, along eta, where eta is\n % represented as a tangent vector: a structure with fields U1, U2, U3, G.\n % This is the directional derivative of nabla f(X) at X along Xdot:\n % nabla^2 f(X)[Xdot] = P.*Xdot\n % We only need to give the Euclidean Hessian. Manopt converts it\n % internally to the Riemannian counterpart.\n problem.ehess = @ehess;\n function [Hess] = ehess(X, eta)\n\n % Computing S, and its unfolding matrices, S1, S2, and S3.\n Xmultiarray = tucker2multiarray(X);\n S = P.*Xmultiarray - PA; \n S1 = reshape(S, [n1, n2*n3]);\n S2 = reshape(permute(S, [2 1 3]),[n2, n1*n3]);\n S3 = reshape(permute(S, [3 1 2]),[n3, n1*n2]); \n\n % Computing Sdot, S1dot, S2dot and S3dot.\n XG = X.G;\n etaG = eta.G;\n G1 = zeros(4*size(X.G));\n G1(1:r1, 1:r2, 1:r3) = XG;\n G1(r1 + 1 : 2*r1, r2 + 1 : 2*r2, r3 + 1 : 2*r3) = XG;\n G1(2*r1 + 1 : 3*r1, 2*r2 + 1 : 3*r2, 2*r3 + 1 : 3*r3) = XG;\n G1(3*r1 + 1 : 4*r1, 3*r2 + 1 : 4*r2, 3*r3 + 1 : 4*r3) = etaG; \n \n X1.G = G1;\n X1.U1 = [eta.U1 X.U1 X.U1 X.U1];\n X1.U2 = [X.U2 eta.U2 X.U2 X.U2];\n X1.U3 = [X.U3 X.U3 eta.U3 X.U3];\n \n X1multiarray = tucker2multiarray(X1);\n Sdot = P.*X1multiarray;\n S1dot = reshape(Sdot, [n1, n2*n3]);\n S2dot = reshape(permute(Sdot, [2 1 3]),[n2, n1*n3]);\n S3dot = reshape(permute(Sdot, [3 1 2]),[n3, n1*n2]);\n \n % Computing unfolding matrices of X.G and eta.G.\n X_G1 = reshape(double(X.G),r1, r2*r3);\n X_G2 = reshape(permute(double(X.G),[2 1 3]),r2, r1*r3);\n X_G3 = reshape(permute(double(X.G),[3 1 2]),r3, r1*r2);\n eta_G1 = reshape(double(eta.G),r1, r2*r3);\n eta_G2 = reshape(permute(double(eta.G),[2 1 3]),r2, r1*r3);\n eta_G3 = reshape(permute(double(eta.G),[3 1 2]),r3, r1*r2); \n\n % Computing Hessians for U1, U2 and U3.\n T1 = double(S1dot) * (kron(X.U3,X.U2)*X_G1') ...\n + double(S1) * (kron(eta.U3,X.U2)*X_G1' ...\n + kron(X.U3,eta.U2)*X_G1' + kron(X.U3,X.U2)*eta_G1');\n \n T2 = double(S2dot) * (kron(X.U3,X.U1)*X_G2') ...\n + double(S2) * (kron(eta.U3,X.U1)*X_G2' ...\n + kron(X.U3,eta.U1)*X_G2' + kron(X.U3,X.U1)*eta_G2');\n\n T3 = double(S3dot) * (kron(X.U2,X.U1)*X_G3') ...\n + double(S3) * (kron(eta.U2,X.U1)*X_G3' ...\n + kron(X.U2,eta.U1)*X_G3' + kron(X.U2,X.U1)*eta_G3');\n \n Hess.U1 = T1;\n Hess.U2 = T2;\n Hess.U3 = T3; \n \n % Computing Hessian for G\n N.U1 = X.U1';\n N.U2 = X.U2';\n N.U3 = X.U3';\n N.G = Sdot;\n M0array = tucker2multiarray(N);\n \n M1.U1 = eta.U1';\n M1.U2 = X.U2';\n M1.U3 = X.U3';\n M1.G = S; \n M1array = tucker2multiarray(M1);\n \n M2.U1 = X.U1';\n M2.U2 = eta.U2';\n M2.U3 = X.U3';\n M2.G = S; \n M2array = tucker2multiarray(M2); \n \n M3.U1 = X.U1';\n M3.U2 = X.U2';\n M3.U3 = eta.U3';\n M3.G = S; \n M3array = tucker2multiarray(M3); \n \n Hess.G = M0array + M1array + M2array + M3array; \n end\n \n\n \n\n % Check consistency of the gradient and the Hessian. Useful if you\n % adapt this example for a new cost function and you would like to make\n % sure there is no mistake.\n %\n % Notice that the checkhessian test fails: the slope is not right. \n % This is because the retraction is not second-order compatible with \n % the Riemannian exponential on this manifold, making \n % the checkhessian tool unusable. The Hessian is correct though. \n % % warning('off', 'manopt:fixedrankfactory_tucker_preconditioned:exp');\n % % checkgradient(problem);\n % % drawnow;\n % % pause;\n % % checkhessian(problem);\n % % drawnow;\n % % pause;\n \n\n \n % options\n options.maxiter = 200;\n options.maxinner = 30;\n options.maxtime = inf;\n options.tolgradnorm = 1e-5; \n\n\n \n \n % Minimize the cost function using Riemannian trust-regions\n Xtr = trustregions(problem, [], options);\n\n \n \n % The reconstructed tensor is X, represented as a structure with fields\n % U1, U2, U3 and G. \n Xtrmultiarray = tucker2multiarray(Xtr);\n fprintf('||X-A||_F = %g\\n', norm(reshape(Xtrmultiarray - A, [n1 n2*n3]), 'fro')); \n \n \n \n \n % Alternatively, we could decide to use a solver such as steepestdescent (SD) \n % or conjugategradient (CG). These solvers need to solve a\n % line-search problem at each iteration. Standard line searches in\n % Manopt have generic purpose systems to do this. But for the problem\n % at hand, we could exploit the least-squares structure to compute an\n % approximate stepsize for the line-search problem. The approximation\n % is obtained by linearizing the nonlinear manifold locally and further\n % approximating it with a degree 2 polynomial approximation.\n % The specific derivation is in the paper referenced above.\n \n problem.linesearch = @linesearch_helper;\n function tmin = linesearch_helper(X, eta)\n \n % term0\n Xmultiarray = tucker2multiarray(X);\n residual_mat = P.*Xmultiarray - PA; \n residual_vec = residual_mat(:);\n term0 = residual_vec;\n \n % term1\n XG = X.G;\n etaG = eta.G; \n G1 = zeros(4*size(X.G));\n G1(1:r1, 1:r2, 1:r3) = XG;\n G1(r1 + 1 : 2*r1, r2 + 1 : 2*r2, r3 + 1 : 2*r3) = XG;\n G1(2*r1 + 1 : 3*r1, 2*r2 + 1 : 3*r2, 2*r3 + 1 : 3*r3) = XG;\n G1(3*r1 + 1 : 4*r1, 3*r2 + 1 : 4*r2, 3*r3 + 1 : 4*r3) = etaG; \n\n X1.U1 = [eta.U1 X.U1 X.U1 X.U1];\n X1.U2 = [X.U2 eta.U2 X.U2 X.U2];\n X1.U3 = [X.U3 X.U3 eta.U3 X.U3];\n X1.G = G1;\n \n X1multiarray = tucker2multiarray(X1);\n term1_mat = P.*X1multiarray; \n term1 = term1_mat(:);\n \n % tmin is the solution to the problem argmin a2*t^2 + a1*t, where\n % the coefficients a1 and a2 are shown below.\n a2 = (term1'*term1);\n a1 = 2*(term1'*term0);\n tmin = - 0.5*(a1 / a2);\n \n end \n\n % Notice that for this solver, the Hessian is not needed.\n [Xcg, costcg, infocg] = conjugategradient(problem, [], options);\n \n fprintf('Take a look at the options that CG used:\\n');\n disp(options);\n fprintf('And see how many trials were made at each line search call:\\n');\n info_ls = [infocg.linesearch];\n disp([info_ls.costevals]); \n \n \n \n fprintf('Try it again without the linesearch helper.\\n');\n \n % Remove the linesearch helper from the problem structure.\n problem = rmfield(problem, 'linesearch');\n \n [Xcg, xcost, info, options] = conjugategradient(problem, []); %#ok\n \n fprintf('Take a look at the options that CG used:\\n');\n disp(options);\n fprintf('And see how many trials were made at each line search call:\\n');\n info_ls = [info.linesearch];\n disp([info_ls.costevals]);\n \n \n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/maxcut.m", "content": "function [x, cutvalue, cutvalue_upperbound, Y] = maxcut(L, r)\n% Algorithm to (try to) compute a maximum cut of a graph, via SDP approach.\n% \n% function x = maxcut(L)\n% function [x, cutvalue, cutvalue_upperbound, Y] = maxcut(L, r)\n%\n% L is the Laplacian matrix describing the graph to cut. The Laplacian of a\n% graph is L = D - A, where D is the diagonal degree matrix (D(i, i) is the\n% sum of the weights of the edges adjacent to node i) and A is the\n% symmetric adjacency matrix of the graph (A(i, j) = A(j, i) is the weight\n% of the edge joining nodes i and j). If L is sparse, this will be\n% exploited.\n%\n% If the graph has n nodes, then L is nxn and the output x is a vector of\n% length n such that x(i) is +1 or -1. This partitions the nodes of the\n% graph in two classes, in an attempt to maximize the sum of the weights of\n% the edges that go from one class to the other (MAX CUT problem).\n%\n% cutvalue is the sum of the weights of the edges 'cut' by the partition x.\n%\n% If the algorithm reached the global optimum of the underlying SDP\n% problem, then it produces an upperbound on the maximum cut value. This\n% value is returned in cutvalue_upperbound if it is found. Otherwise, that\n% output is set to NaN.\n%\n% If r is specified (by default, r = n), the algorithm will stop at rank r.\n% This may prevent the algorithm from reaching a globally optimal solution\n% for the underlying SDP problem (but can greatly help in keeping the\n% execution time under control). If a global optimum of the SDP is reached\n% before rank r, the algorithm will stop of course.\n%\n% Y is a matrix of size nxp, with p <= r, such that X = Y*Y' is the best\n% solution found for the underlying SDP problem. If cutvalue_upperbound is\n% not NaN, then Y*Y' is optimal for the SDP and cutvalue_upperbound is its\n% cut value.\n% \n% By Goemans and Williamson 1995, it is known that if the optimal value of\n% the SDP is reached, then the returned cut, in expectation, is at most at\n% a fraction 0.878 of the optimal cut. (This is not exactly valid because\n% we do not use random projection here; sign(Y*randn(size(Y, 2), 1)) will\n% give a cut that respects this statement -- it's usually worse though).\n%\n% The algorithm is essentially that of:\n% Journee, Bach, Absil and Sepulchre, SIAM 2010\n% Low-rank optimization on the cone of positive semidefinite matrices.\n%\n% It is itself based on the famous SDP relaxation of MAX CUT:\n% Goemans and Williamson, 1995\n% Improved approximation algorithms for maximum cut and satisfiability\n% problems using semidefinite programming.\n% \n% See also: elliptope_SDP\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 18, 2013\n% Contributors:\n% Change log:\n% \n% April 3, 2015 (NB):\n% L products now counted with the new shared memory system. This is\n% more reliable and more flexible than using a global variable.\n\n\n % If no inputs are provided, generate a random graph Laplacian.\n % This is for illustration purposes only.\n if ~exist('L', 'var') || isempty(L)\n n = 20;\n A = triu(randn(n) <= .4, 1);\n A = A+A';\n D = diag(sum(A, 2));\n L = D-A;\n end\n\n\n n = size(L, 1);\n assert(size(L, 2) == n, 'L must be square.');\n\n if ~exist('r', 'var') || isempty(r) || r > n\n r = n;\n end\n \n % We will let the rank increase. Each rank value will generate a cut.\n % We have to go up in the rank to eventually find a certificate of SDP\n % optimality. This in turn will provide an upperbound on the MAX CUT\n % value and ensure that we're doing well, according to Goemans and\n % Williamson's argument. In practice though, the good cuts often come\n % up for low rank values, so we better keep track of the best one.\n best_x = ones(n, 1);\n best_cutvalue = 0;\n cutvalue_upperbound = NaN;\n \n time = [];\n cost = [];\n \n for rr = 2 : r\n \n manifold = elliptopefactory(n, rr);\n \n if rr == 2\n \n % At first, for rank 2, generate a random point.\n Y0 = manifold.rand();\n \n else\n \n % To increase the rank, we could just add a column of zeros to\n % the Y matrix. Unfortunately, this lands us in a saddle point.\n % To escape from the saddle, we may compute an eigenvector of\n % Sy associated to a negative eigenvalue: that will yield a\n % (second order) descent direction Z. See Journee et al ; Sy is\n % linked to dual certificates for the SDP.\n Y0 = [Y zeros(n, 1)];\n LY0 = L*Y0;\n Dy = spdiags(sum(LY0.*Y0, 2), 0, n, n);\n Sy = (Dy - L)/4;\n % Find the smallest (the \"most negative\") eigenvalue of Sy.\n eigsopts.issym = true;\n eigsopts.isreal = true;\n [v, s] = eigs(Sy, 1, 'SA', eigsopts);\n % If there is no negative eigenvalue for Sy, than we are not at\n % a saddle point: we're actually done!\n if s >= -1e-8\n % We can stop here: we found the global optimum of the SDP,\n % and hence the reached cost is a valid upper bound on the\n % maximum cut value.\n cutvalue_upperbound = max(-[info.cost]);\n break;\n end\n \n % This is our escape direction.\n Z = manifold.proj(Y0, [zeros(n, rr-1) v]);\n \n % % These instructions can be uncommented to see what the cost\n % % function looks like at a saddle point. But will require the\n % % problem structure which is not defined here: see the helper\n % % function.\n % plotprofile(problem, Y0, Z, linspace(-1, 1, 101));\n % drawnow; pause;\n \n % Now make a step in the Z direction to escape from the saddle.\n % It is not obvious that it is ok to do a unit step ... perhaps\n % need to be cautious here with the stepsize. It's not too\n % critical though: the important point is to leave the saddle\n % point. But it's nice to guarantee monotone decrease of the\n % cost, and we can't do that with a constant step (at least,\n % not without a proper argument to back it up).\n stepsize = 1;\n Y0 = manifold.retr(Y0, Z, stepsize);\n \n end\n \n % Use the Riemannian optimization based algorithm lower in this\n % file to reach a critical point (typically a local optimizer) of\n % the max cut cost with fixed rank, starting from Y0.\n [Y, info] = maxcut_fixedrank(L, Y0);\n \n % Some info logging.\n thistime = [info.time];\n if ~isempty(time)\n thistime = time(end) + thistime;\n end\n time = [time thistime]; %#ok\n cost = [cost [info.cost]]; %#ok\n\n % Time to turn the matrix Y into a cut.\n % We can either do the random rounding as follows:\n % x = sign(Y*randn(rr, 1));\n % or extract the \"PCA direction\" of the points in Y and cut\n % orthogonally to that direction, as follows (seems faster than\n % calling svds):\n [U, ~, ~] = svd(Y, 0);\n u = U(:, 1);\n x = sign(u);\n\n cutvalue = (x'*L*x)/4;\n if cutvalue > best_cutvalue\n best_x = x;\n best_cutvalue = cutvalue;\n end\n \n end\n \n x = best_x;\n cutvalue = best_cutvalue;\n \n plot(time, -cost, '.-');\n xlabel('Time [s]');\n ylabel('Relaxed cut value');\n title('The relaxed cut value is an upper bound on the optimal cut value.');\n\nend\n\n\nfunction [Y, info] = maxcut_fixedrank(L, Y)\n% Try to solve the (fixed) rank r relaxed max cut program, based on the\n% Laplacian of the graph L and an initial guess Y. L is nxn and Y is nxr.\n\n [n, r] = size(Y);\n assert(all(size(L) == n));\n \n % The fixed rank elliptope geometry describes symmetric, positive\n % semidefinite matrices of size n with rank r and all diagonal entries\n % are 1.\n manifold = elliptopefactory(n, r);\n \n % % If you want to compare the performance of the elliptope geometry\n % % against the (conceptually simpler) oblique manifold geometry,\n % % uncomment this line.\n % manifold = obliquefactory(r, n, true);\n \n problem.M = manifold;\n \n % % For rapid prototyping, these lines suffice to describe the cost\n % % function and its gradient and Hessian (here expressed using the\n % % Euclidean gradient and Hessian).\n % problem.cost = @(Y) -trace(Y'*L*Y)/4;\n % problem.egrad = @(Y) -(L*Y)/2;\n % problem.ehess = @(Y, U) -(L*U)/2;\n \n % Instead of the prototyping version, the functions below describe the\n % cost, gradient and Hessian using the caching system (the store\n % structure). This alows to execute exactly the required number of\n % multiplications with the matrix L. These multiplications are counted\n % using the shared memory in the store structure: that memory is\n % shared , so we get access to the same data, regardless of the\n % point Y currently visited.\n\n % For every visited point Y, we will need L*Y. This function makes sure\n % the quantity L*Y is available, but only computes it if it wasn't\n % already computed.\n function store = prepare(Y, store)\n if ~isfield(store, 'LY')\n % Compute and store the product for the current point Y.\n store.LY = L*Y;\n % Create / increment the shared counter (independent of Y).\n if isfield(store.shared, 'counter')\n store.shared.counter = store.shared.counter + 1;\n else\n store.shared.counter = 1;\n end\n end\n end\n\n problem.cost = @cost;\n function [f, store] = cost(Y, store)\n store = prepare(Y, store);\n LY = store.LY;\n f = -(Y(:)'*LY(:))/4; % = -trace(Y'*LY)/4; but faster\n end\n\n problem.egrad = @egrad;\n function [g, store] = egrad(Y, store)\n store = prepare(Y, store);\n LY = store.LY;\n g = -LY/2;\n end\n\n problem.ehess = @ehess;\n function [h, store] = ehess(Y, U, store)\n store = prepare(Y, store); % this line is not strictly necessary\n LU = L*U;\n store.shared.counter = store.shared.counter + 1;\n h = -LU/2;\n end\n\n % statsfun is called exactly once after each iteration (including after\n % the evaluation of the cost at the initial guess). We then register\n % the value of the L-products counter (which counts how many products\n % were needed so far).\n % options.statsfun = @statsfun;\n % function stats = statsfun(problem, Y, stats, store) %#ok\n % stats.Lproducts = store.shared.counter;\n % end\n % Equivalent, but simpler syntax:\n options.statsfun = statsfunhelper('Lproducts', ...\n @(problem, Y, stats, store) store.shared.counter );\n \n\n % % Diagnostics tools: to make sure the gradient and Hessian are\n % % correct during the prototyping stage.\n % checkgradient(problem); pause;\n % checkhessian(problem); pause;\n \n % % To investigate the effect of the rotational invariance when using\n % % the oblique or the elliptope geometry, or to study the saddle point\n % % issue mentioned above, it is sometimes interesting to look at the\n % % spectrum of the Hessian. For large dimensions, this is slow!\n % stairs(sort(hessianspectrum(problem, Y)));\n % drawnow; pause;\n \n \n % % When facing a saddle point issue as described in the master\n % % function, and when no sure mechanism exists to find an escape\n % % direction, it may be helpful to set useRand to true and raise\n % % miniter to more than 1, when using trustregions. This will tell the\n % % solver to not stop before at least miniter iterations were\n % % accomplished (thus disregarding the zero gradient at the saddle\n % % point) and to use random search directions to kick start the inner\n % % solve (tCG) step. It is not as efficient as finding a sure escape\n % % direction, but sometimes it's the best we have.\n % options.useRand = true;\n % options.miniter = 5;\n \n options.verbosity = 2;\n [Y, Ycost, info] = trustregions(problem, Y, options); %#ok\n \n fprintf('Products with L: %d\\n', max([info.Lproducts]));\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/nonlinear_eigenspace.m", "content": "function Xsol = nonlinear_eigenspace(L, k, alpha)\n% Example of nonlinear eigenvalue problem: total energy minimization.\n%\n% function Xsol = nonlinear_eigenspace(L, k, alpha)\n%\n% L is a discrete Laplacian operator,\n% alpha is a given constant, and\n% k corresponds to the dimension of the least eigenspace sought. \n%\n% This example demonstrates how to use the Grassmann geometry factory \n% to solve the nonlinear eigenvalue problem as the optimization problem:\n%\n% minimize 0.5*trace(X'*L*X) + (alpha/4)*(rho(X)*L\\(rho(X))) \n% over X such that X'*X = Identity,\n%\n% where L is of size n-by-n,\n% X is an n-by-k matrix, and\n% rho(X) is the diagonal part of X*X'.\n%\n% This example is motivated in the paper\n% \"A Riemannian Newton Algorithm for Nonlinear Eigenvalue Problems\",\n% Zhi Zhao, Zheng-Jian Bai, and Xiao-Qing Jin,\n% SIAM Journal on Matrix Analysis and Applications, 36(2), 752-774, 2015.\n%\n\n\n% This file is part of Manopt and is copyrighted. See the license file.\n%\n% Main author: Bamdev Mishra, June 19, 2015.\n% Contributors:\n%\n% Change log:\n\n\n % If no inputs are provided, generate a discrete Laplacian operator.\n % This is for illustration purposes only.\n % The default example corresponds to Case (c) of Example 6.2 of the\n % above referenced paper.\n \n if ~exist('L', 'var') || isempty(L)\n n = 100;\n L = gallery('tridiag', n, -1, 2, -1);\n end\n \n n = size(L, 1);\n assert(size(L, 2) == n, 'L must be square.');\n \n if ~exist('k', 'var') || isempty(k) || k > n\n k = 10;\n end\n \n if ~exist('alpha', 'var') || isempty(alpha)\n alpha = 1;\n end\n \n \n % Grassmann manifold description\n Gr = grassmannfactory(n, k);\n problem.M = Gr;\n \n % Cost function evaluation\n problem.cost = @cost;\n function val = cost(X)\n rhoX = sum(X.^2, 2); % diag(X*X'); \n val = 0.5*trace(X'*(L*X)) + (alpha/4)*(rhoX'*(L\\rhoX));\n end\n \n % Euclidean gradient evaluation\n % Note: Manopt automatically converts it to the Riemannian counterpart.\n problem.egrad = @egrad;\n function g = egrad(X)\n rhoX = sum(X.^2, 2); % diag(X*X');\n g = L*X + alpha*diag(L\\rhoX)*X;\n end\n \n % Euclidean Hessian evaluation\n % Note: Manopt automatically converts it to the Riemannian counterpart.\n problem.ehess = @ehess;\n function h = ehess(X, U)\n rhoX = sum(X.^2, 2); %diag(X*X');\n rhoXdot = 2*sum(X.*U, 2); \n h = L*U + alpha*diag(L\\rhoXdot)*X + alpha*diag(L\\rhoX)*U;\n end\n \n \n % Check whether gradient and Hessian computations are correct.\n % checkgradient(problem);\n % pause;\n % checkhessian(problem);\n % pause;\n \n \n % Initialization as suggested in above referenced paper.\n X = randn(n, k);\n [U, S, V] = svd(X, 0); %#ok\n X = U*V';\n [U0, S0, V0] = eigs(L + alpha*diag(L\\(sum(X.^2, 2))), k,'sm'); %#ok\n X0 = U0;\n \n % Call manoptsolve to automatically call an appropriate solver.\n % Note: it calls the trust regions solver as we have all the required\n % ingredients, namely, gradient and Hessian, information.\n Xsol = manoptsolve(problem, X0);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/packing_on_the_sphere.m", "content": "function [X, maxdot] = packing_on_the_sphere(d, n, epsilon, X0)\n% Return a set of points spread out on the sphere.\n%\n% function [X, maxdot] = packing_on_the_sphere(d, n, epsilon, X0)\n%\n% Using optimization on the oblique manifold, that is, the product of\n% spheres, this function returns a set of n points with unit norm in R^d in\n% the form of a matrix X of size nxd, such that the points are spread out\n% on the sphere. Ideally, we would minimize the maximum inner product\n% between any two points X(i, :) and X(j, :), i~=j, but that is a nonsmooth\n% cost function. Instead, we replace the max function by a classical\n% log-sum-exp approximation and (attempt to) solve:\n%\n% min_{X in OB(d, n)} log( .5*sum_{i~=j} exp( xi'*xj/epsilon ) ),\n%\n% with xi = X(:, i) and epsilon is some \"diffusion constant\". As epsilon\n% goes to zero, the cost function is a sharper approximation of the max\n% function (under some assumptions), but the cost function becomes stiffer\n% and hence harder to optimize.\n%\n% The second output, maxdot, is the maximum inner product between any two\n% points in the returned X. This number is the one we truly are trying to\n% minimize.\n%\n% Notice that this cost function is invariant under rotation of X:\n% f(X) = f(XQ) for all orthogonal Q in O(d).\n% This calls for optimization over the set of symmetric positive\n% semidefinite matrices of size n and rank d with unit diagonal, which can\n% be thought of as the quotient of the oblique manifold OB(d, n) by O(d):\n% See elliptopefactory.\n%\n% This is known as the Thomson or, more specifically, the Tammes problem:\n% http://en.wikipedia.org/wiki/Tammes_problem\n% An interesting page by Neil Sloane collecting best known packings is\n% available here http://neilsloane.com/packings/\n\n% This file is part of Manopt and is copyrighted. See the license file.\n%\n% Main author: Nicolas Boumal, July 2, 2013\n% Contributors:\n%\n% Change log:\n% Aug. 14, 2013 (NB) : Code now compatible to experiment with both the\n% obliquefactory and the elliptopefactory.\n%\n% Jan. 7, 2014 (NB) : Added reference to Neil Sloane's page and the\n% maxdot output.\n%\n% June 24, 2014 (NB) : Now shifting exponentials to alleviate numerical\n% trouble when epsilon is too small.\n% \n \n if ~exist('d', 'var') || isempty(d)\n % Dimension of the embedding space: R^d\n d = 3;\n end\n if ~exist('n', 'var') || isempty(n)\n % Number n of points to place of the sphere in R^d.\n % For example, n=12 yields an icosahedron:\n % https://en.wikipedia.org/wiki/Icosahedron\n % Notice though that platonic solids are not always optimal.\n % Try for example n = 8: you don't get a cube.\n n = 24;\n end\n if ~exist('epsilon', 'var') || isempty(epsilon)\n % This value should be as close to 0 as affordable.\n % If it is too close to zero, optimization first becomes much\n % slower, than simply doesn't work anymore becomes of floating\n % point overflow errors (NaN's and Inf's start to appear).\n % If it is too large, then log-sum-exp is a poor approximation of\n % the max function, and the spread will be less uniform.\n % An okay value seems to be 0.01 or 0.001 for example. Note that a\n % better strategy than using a small epsilon straightaway is to\n % reduce epsilon bit by bit and to warm-start subsequent\n % optimization in that way. Trustregions will be more appropriate\n % for these fine tunings.\n epsilon = 0.0015;\n end\n \n % Pick your manifold (the elliptope factory quotients out the global\n % rotation invariance of the problem, which is more natural but\n % conceptually a bit more complicated --- for usage with the toolbox it\n % is the same though: just uncomment the appropriate line).\n manifold = obliquefactory(d, n, true);\n % manifold = elliptopefactory(n, d);\n \n % Generate a random initial guess if none was given.\n if ~exist('X0', 'var') || isempty(X0)\n X0 = manifold.rand();\n end\n\n % Define the cost function with caching system used: the store\n % structure we receive as input is tied to the input point X. Everytime\n % this cost function is called at this point X, we will receive the\n % same store structure back. We may modify the store structure inside\n % the function and return it: the changes will be remembered for next\n % time.\n function [f, store] = cost(X, store)\n if ~isfield(store, 'ready')\n XXt = X*X';\n % Shift the exponentials by the maximum value to reduce\n % numerical trouble due to possible overflows.\n s = max(max(triu(XXt, 1)));\n expXXt = exp((XXt-s)/epsilon);\n % Zero out the diagonal\n expXXt(1:(n+1):end) = 0;\n u = sum(sum(triu(expXXt, 1)));\n store.XXt = XXt;\n store.s = s;\n store.expXXt = expXXt;\n store.u = u;\n store.ready = true;\n end\n u = store.u;\n s = store.s;\n f = s + epsilon*log(u);\n end\n\n % Define the gradient of the cost. When the gradient is called at a\n % point X for which the cost was already called, the store structure we\n % receive remember everything that the cost function stored in it, so\n % we can reuse previously computed elements.\n function [g, store] = grad(X, store)\n if ~isfield(store, 'ready')\n [~, store] = cost(X, store);\n end\n % Compute the Euclidean gradient\n eg = store.expXXt*X / store.u;\n % Convert to the Riemannian gradient (by projection)\n g = manifold.egrad2rgrad(X, eg);\n end\n\n % Setup the problem structure with its manifold M and cost+grad\n % functions.\n problem.M = manifold;\n problem.cost = @cost;\n problem.grad = @grad;\n\n % For debugging, it's always nice to check the gradient a few times.\n % checkgradient(problem);\n % pause;\n \n % Call a solver on our problem with a few options defined. We did not\n % specify the Hessian but it is still okay to call trustregion: Manopt\n % will approximate the Hessian with finite differences of the gradient.\n opts.tolgradnorm = 1e-8;\n opts.maxtime = 1200;\n opts.maxiter = 1e5;\n % X = trustregions(problem, X0, opts);\n X = conjugategradient(problem, X0, opts);\n \n % Evaluate the maximum inner product between any two points of X.\n XXt = X*X';\n dots = XXt(find(triu(ones(n), 1))); %#ok\n maxdot = max(dots);\n \n % Similarly, even though we did not specify the Hessian, we may still\n % estimate its spectrum at the solution. It should reflect the\n % invariance of the cost function under a global rotatioon of the\n % sphere, which is an invariance under the group O(d) of dimension\n % d(d-1)/2 : this translates into d(d-1)/2 zero eigenvalues in the\n % spectrum of the Hessian.\n % The approximate Hessian is not a linear operator, and is it a\n % fortiori not symmetric. The result of this computation is thus not\n % reliable. It does display the zero eigenvalues as expected though.\n if manifold.dim() < 300\n evs = real(hessianspectrum(problem, X));\n figure;\n stem(1:length(evs), sort(evs), '.');\n title(['Eigenvalues of the approximate Hessian of the cost ' ...\n 'function at the solution']);\n end\n \n \n % Show how the inner products X(:, i)'*X(:, j) are distributed.\n figure;\n hist(real(acos(dots)), 20);\n title('Histogram of the geodesic distances');\n \n % This is the quantity we actually want to minimize.\n fprintf('Maximum inner product between two points: %g\\n', maxdot);\n \n \n % Give some visualization if the dimension allows\n if d == 2\n % For the circle, the optimal solution consists in spreading the\n % points with angles uniformly sampled in (0, 2pi). This\n % corresponds to the following value for the max inner product:\n fprintf('Optimal value for the max inner product: %g\\n', cos(2*pi/n));\n figure;\n t = linspace(-pi, pi, 201);\n plot(cos(t), sin(t), '-', 'LineWidth', 3, 'Color', [152,186,220]/255);\n daspect([1 1 1]);\n box off;\n axis off;\n hold on;\n plot(X(:, 1), X(:, 2), 'r.', 'MarkerSize', 25);\n hold off;\n end\n if d == 3\n figure;\n % Plot the sphere\n [sphere_x, sphere_y, sphere_z] = sphere(50);\n handle = surf(sphere_x, sphere_y, sphere_z);\n set(handle, 'FaceColor', [152,186,220]/255);\n set(handle, 'FaceAlpha', .5);\n set(handle, 'EdgeColor', [152,186,220]/255);\n set(handle, 'EdgeAlpha', .5);\n daspect([1 1 1]);\n box off;\n axis off;\n hold on;\n % Add the chosen points\n Y = 1.02*X';\n plot3(Y(1, :), Y(2, :), Y(3, :), 'r.', 'MarkerSize', 25);\n % And connect the points which are at minimal distance,\n % within some tolerance.\n min_distance = real(acos(maxdot));\n connected = real(acos(XXt)) <= 1.20*min_distance;\n [Ic, Jc] = find(triu(connected, 1));\n for k = 1 : length(Ic)\n i = Ic(k); j = Jc(k);\n plot3(Y(1, [i j]), Y(2, [i j]), Y(3, [i j]), 'k-');\n end\n hold off;\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/positive_definite_karcher_mean.m", "content": "function X = positive_definite_karcher_mean(A)\n% Computes a Karcher mean of a collection of positive definite matrices.\n%\n% function X = positive_definite_karcher_mean(A)\n%\n% Input: A 3D matrix A of size nxnxm such that each slice A(:,:,k) is a\n% positive definite matrix of size nxn.\n% \n% Output: A positive definite matrix X of size nxn which is a Karcher mean\n% of the m matrices in A, that is, X minimizes the sum of squared\n% Riemannian distances to the matrices in A:\n% f(X) = sum_k=1^m .5*dist^2(X, A(:, :, k))\n% The distance is defined by the natural metric on the set of\n% positive definite matrices: dist(X,Y) = norm(logm(X\\Y), 'fro').\n% \n% This simple example is not the best way to compute Karcher means. Its\n% purpose it to serve as base code to explore other algorithms. In\n% particular, in the presence of large noise, this algorithm seems to not\n% be able to reach points with a very small gradient norm. This may be\n% caused by insufficient accuracy in the gradient computation.\n\n% This file is part of Manopt and is copyrighted. See the license file.\n% \n% Main author: Nicolas Boumal, Sept. 3, 2013\n% Contributors:\n% \n% Change log:\n% \n \n % Generate some random data to test the function if none is given.\n if ~exist('A', 'var') || isempty(A)\n n = 5;\n m = 50;\n A = zeros(n, n, m);\n ref = diag(max(.1, 1+.1*randn(n, 1)));\n for i = 1 : m\n noise = 0.01*randn(n);\n noise = (noise + noise')/2;\n [V, D] = eig(ref + noise);\n A(:, :, i) = V*diag(max(.01, diag(D)))*V';\n end\n end\n \n % Retrieve the size of the problem:\n % There are m matrices of size nxn to average.\n n = size(A, 1);\n m = size(A, 3);\n assert(n == size(A, 2), ...\n ['The slices of A must be square, i.e., the ' ...\n\t 'first and second dimensions of A must be equal.']);\n \n % Our search space is the set of positive definite matrices of size n.\n % Notice that this is the only place we specify on which manifold we\n % wish to compute Karcher means. Replacing this factory for another\n % geometry will yield code to compute Karcher means on that other\n % manifold, provided that manifold is equipped with a dist function and\n % a logarithmic map log.\n M = sympositivedefinitefactory(n);\n \n % Define a problem structure, specifying the manifold M, the cost\n % function and its gradient.\n problem.M = M;\n problem.cost = @cost;\n problem.grad = @grad;\n \n % Explicitly pick an approximate Hessian for the trust-region method\n problem.approxhess = approxhessianFD(problem, struct('stepsize', 1e-4));\n \n % The functions below make many redundant computations. This\n % performance hit can be alleviated by using the caching system. We go\n % for a simple implementation here, as a tutorial example.\n \n % Cost function\n function f = cost(X)\n f = 0;\n for k = 1 : m\n f = f + M.dist(X, A(:, :, k))^2;\n end\n f = f/(2*m);\n end\n\n % Riemannian gradient of the cost function\n function g = grad(X)\n g = M.zerovec(X);\n for k = 1 : m\n % Update g in a linear combination of the form\n % g = g - [something]/m.\n g = M.lincomb(X, 1, g, -1/m, M.log(X, A(:, :, k)));\n end\n end\n \n % Execute some checks on the derivatives for early debugging.\n % These things can be commented out of course.\n % The slopes should agree on part of the plot at least. In this case,\n % it is sometimes necessary to inspect the plot visually to make the\n % call, but it is indeed correct.\n % checkgradient(problem);\n % pause;\n \n % Execute this if you want to force using a proper parallel vector\n % transport. This is not necessary. If you omit this, the default\n % vector transport is the identity map, which is (of course) cheaper\n % and seems to perform well in practice.\n % M.transp = M.paralleltransp;\n \n % Issue a call to a solver. Default options are selected.\n % Our initial guess is the first data point. Most solvers work well\n % with this problem. Limited-memory BFGS is one good example:\n X = rlbfgs(problem, A(:, :, 1));\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/radio_interferometric_calibration.m", "content": "function xsol = radio_interferometric_calibration(N, K)\n% Returns the gain matrices of N stations with K receivers.\n%\n% function xsol = radio_interferometric_calibration(N, K)\n%\n% N >= K is always assumed.\n%\n% The example considers calibration of an array of N stations.\n% We simulate a system with N stations, each having K receivers.\n% For radio astronomy, K = 2.\n%\n% For a detailed exposition of the problem at hand, refer to the paper:\n% \"Radio interferometric calibration using a Riemannian manifold\",\n% Sarod Yatawatta, ICASSP, 2013.\n% Available at http://dx.doi.org/10.1109/ICASSP.2013.6638382.\n%\n% The source of the signal is unpolarized (given by the matrix C).\n% The measured data is the cross correlation of the signals at each receiver.\n% So there will be N(N-1)/2 possible cross correlations.\n% Noise with given SNR is added to the signal.\n%\n% The objective is to estimate the gains of each receiver (K x K) matrix,\n% so the total size of the solutions is N x (K x K), which is written\n% as an NK x K matrix.\n%\n% Note: each station gain matrix (KxK) can have a KxK unitary ambiguity,\n% therefore we use the quotient manifold structure. The unitary ambiguity \n% is common to all stations, so the solution obtained by \n% optimization routine always has an unkown unitary matrix that makes the \n% solution different from the true solution.\n%\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Sarod Yatawatta, June 29, 2015.\n% Contributors: Bamdev Mishra.\n% Change log:\n% \n% June 28, 2016 (BM):\n% Modified the egrad and ehess operations according to \n% the modified metric in the symfixedrankYYcomplexfactory file, \n% where a factor of 2 was removed from the metric. Accordingly, \n% a factor of 2 was added to egrad and ehess operations.\n \n % Generate some random data to test the function\n \n if ~exist('N', 'var') || isempty(N)\n N = 10; \n end\n if ~exist('K', 'var') || isempty(K)\n K = 2; \n end\n \n assert(N >= K, 'N must be larger than or equal to K.');\n \n % Baselines (pairs of correlations)\n B = N*(N-1)/2;\n \n \n \n % Source coherence, at phase center\n C = eye(K);\n \n % Random J (gains) of all stations\n J = 0.2*rand(K*N,K) + 1i*rand(K*N,K);\n \n % Visibilities (cross correlations)\n V = zeros(K*B,K);\n \n ck = 1;\n for ci = 1 : N -1,\n for cj = ci + 1 : N,\n % Compute cross correlation of each receiver pair.\n V(K*(ck-1)+1:K*ck,:) = J(K*(ci-1)+1:K*ci,:)*C*J(K*(cj-1)+1:K*cj,:)';\n ck = ck + 1;\n end\n end\n \n % Generate noise\n SNR = 10000;% inf;\n nn = randn(K*B,K)+1i*randn(K*B,K);\n noise_var = norm(V)^2/(norm(nn)^2*SNR);\n nn = nn*sqrt(noise_var);\n \n % Add noise to signal\n V = V + nn;\n \n \n % Optimization part by creating the problem structure.\n % First, we use the manifold desctription.\n % Second, we define the problem cost, gradient and Hessian functions.\n \n \n % Manifold description\n % Note that the actual dimension is KN x K.\n problem.M = symfixedrankYYcomplexfactory(K*N, K);\n \n \n % Cost function\n problem.cost = @cost;\n function fval = cost(x)\n fval = 0.0;\n ck = 1;\n for p = 1 : N - 1,\n for q = p + 1 : N,\n res = V(K*(ck-1)+1:K*ck,:) - x(K*(p-1)+1:K*p,:)*C*x(K*(q-1)+1:K*q,:)'; % Residual\n fval = fval + real(res(:)'*res(:)); % Add norm of the residual.\n ck = ck + 1;\n end\n end\n end\n \n % Euclidean gradient of the cost function.\n % Manopt automatically converts it to the Riemannian couterpart.\n % The code involves for-loops for readability, but could be vectorized\n % for improved speed.\n problem.egrad = @egrad;\n function grad = egrad(x)\n grad = zeros(K*N, K);\n ck = 1;\n for p = 1 : N - 1,\n for q = p+1 : N,\n res = 2*(V(K*(ck-1)+1:K*ck,:) - x(K*(p-1)+1:K*p,:)*C*x(K*(q-1)+1:K*q,:)'); % Residual\n grad(K*(p-1)+1:K*p,:) = grad(K*(p-1)+1:K*p,:) - res*x(K*(q-1)+1:K*q,:)*C';\n grad(K*(q-1)+1:K*q,:) = grad(K*(q-1)+1:K*q,:) - res'*x(K*(p-1)+1:K*p,:)*C;\n ck = ck + 1;\n end\n end\n end\n \n % Euclidean Hessian of the cost function along a search direction eta.\n % Manopt automatically converts it to the Riemannian couterpart.\n problem.ehess = @ehess;\n function hess = ehess(x, eta)\n hess = zeros(K*N, K);\n ck = 1;\n for p = 1 : N-1,\n for q = p+1:N,\n res = 2*(V(K*(ck-1)+1:K*ck,:) -x(K*(p-1)+1:K*p,:)*C*x(K*(q-1)+1:K*q,:)'); % Residual\n resdot = 2*(-x(K*(p-1)+1:K*p,:)*C*eta(K*(q-1)+1:K*q,:)' - eta(K*(p-1)+1:K*p,:)*C*x(K*(q-1)+1:K*q,:)'); % Residual derivative\n \n hess(K*(p-1)+1:K*p,:) = hess(K*(p-1)+1:K*p,:) - (res*eta(K*(q-1)+1:K*q,:) + resdot*x(K*(q-1)+1:K*q,:))*C';\n hess(K*(q-1)+1:K*q,:) = hess(K*(q-1)+1:K*q,:) - (res'*eta(K*(p-1)+1:K*p,:) + resdot'*x(K*(p-1)+1:K*p,:))*C;\n ck = ck + 1;\n end\n end\n end\n \n \n \n % Execute some checks on the derivatives for early debugging.\n % checkgradient(problem);\n % pause;\n % checkhessian(problem);\n % pause;\n \n \n % Solve.\n [xsol, xcost, info] = trustregions(problem); \n fprintf('Final cost: %g.\\n', xcost);\n \n \n % Display some statistics.\n fs = 11;\n figure;\n semilogy([info.iter], [info.gradnorm], 'o-.','Color','blue', 'MarkerSize',6, 'LineWidth',1.1);\n ax1 = gca;\n set(ax1,'FontSize',fs);\n xlabel(ax1, 'Iteration #', 'FontSize',fs);\n ylabel(ax1, 'Gradient norm', 'FontSize',fs);\n title('Convergence of the trust-regions algorithm');\n\n % Make a plot of estimation error (only for K = 2).\n if K == 2,\n % Find unitary ambiguity first by solving min ||J - xsol U||.\n % This has a closed-form solution.\n [u, ignore, v] = svd(xsol'*J); %#ok\n\n % Error in position\n E = J - xsol*u*v'; \n\n % Normalize error\n E = E/norm(J);\n\n % Plot\n figure;\n ax1 = subplot(1,2,1);\n quiver(real(J(:,1)), imag(J(:,1)),real(E(:,1)),imag(E(:,1)));\n hold all;\n scatter(real(J(:,1)), imag(J(:,1)));\n set(ax1,'FontSize',fs);\n xlabel('Real E_1');\n ylabel('Imag E_1');\n title('Position error 1st coordinate'); \n axis equal;\n ax2 = subplot(1,2,2);\n quiver(real(J(:,2)),imag(J(:,2)),real(E(:,2)),imag(E(:,2)));\n hold all;\n scatter(real(J(:,2)),imag(J(:,2)));\n set(ax2,'FontSize',fs);\n xlabel('Real E_2');\n ylabel('Imag E_2');\n title('Position error 2nd coordinate'); \n axis equal;\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/robust_pca.m", "content": "function [U, cost] = robust_pca(X, d)\n% Computes a robust version of PCA (principal component analysis) on data.\n% \n% function [U, cost] = robustpca(X, d)\n%\n% Given a matrix X of size p by n, such that each column represents a\n% point in R^p, this computes U: an orthonormal basis of size p by d such\n% that the column space of U captures the points X as well as possible.\n% More precisely, the function attempts to compute U as the minimizer\n% over the Grassmann manifold (the set of linear subspaces) of:\n%\n% f(U) = (1/n) Sum_{i = 1:n} dist(X(:, i), the space spanned by U)\n% = (1/n) Sum_{i = 1:n} || U*U'*X(:, i) - X(:, i) ||\n%\n% The output cost represents the average distance achieved with the\n% returned U. Notice that norms are not squared, for robustness.\n%\n% In practice, because this function is nonsmooth, it is smoothed with a\n% pseudo-Huber loss function of parameter epsilon (noted e for short), and\n% the smoothing parameter is iteratively reduced (with warm starts):\n%\n% f_e(U) = (1/n) Sum_{i = 1:n} l_e(|| U*U'*X(:, i) - X(:, i) ||)\n%\n% with l_e(x) = sqrt(x^2 + e^2) - e (for e = 0, this is absolute value).\n%\n% The intermediate optimization of the smooth cost over the Grassmann\n% manifold is performed using the Manopt toolbox.\n%\n% Ideally, the non-outlier data should be centered. If not, this\n% pre-processing centers all the data, but bear in mind that outliers will\n% shift the center of mass too.\n% X = X - repmat(mean(X, 2), [1, size(X, 2)]);\n%\n% There are no guarantees that this code will return the optimal U.\n% This code is distributed to illustrate one possible way of optimizing\n% a nonsmooth cost function over a manifold, using Manopt with smoothing.\n% For practical use, the constants in the code would need to be tuned.\n\n% This file is part of Manopt and is copyrighted. See the license file.\n%\n% Main author: Nicolas Boumal and Teng Zhang, May 2, 2014\n% Contributors:\n%\n% Change log:\n%\n% March 4, 2015 (NB):\n% Uses a pseudo-Huber loss rather than a Huber loss: this has the\n% nice advantage of being smooth and simpler to code (no if's).\n%\n% April 8, 2015 (NB):\n% Built-in test data for quick tests; added comment about centering.\n\n\n\n % If no inputs, generate random data for illustration purposes.\n if nargin == 0\n % Generate some data points aligned on a subspace\n X = rand(2, 1)*(1:30) + .05*randn(2, 30).*[(1:30);(1:30)];\n % And add some random outliers to the mix\n P = randperm(size(X, 2));\n outliers = 10;\n X(:, P(1:outliers)) = 30*randn(2, outliers);\n % Center the data\n % X = X - repmat(mean(X, 2), [1, size(X, 2)]);\n d = 1;\n end\n\n\n\n\n\n % Prepare a Manopt problem structure for optimization of the given\n % cost (defined below) over the Grassmann manifold.\n [p, n] = size(X);\n manifold = grassmannfactory(p, d);\n problem.M = manifold;\n problem.cost = @robustpca_cost;\n problem.egrad = @robustpca_gradient;\n\t\n\t% Do classical PCA for the initial guess.\n\t% This is just one idea: it is not necessarily useful or ideal.\n % Using a random initial guess, and starting over for a few different\n % ones is probably much better. For this example, we keep it simple.\n [U, ~, ~] = svds(X, d);\n\n \n\t% Iteratively reduce the smoothing constant epsilon and optimize\n\t% the cost function over Grassmann.\n epsilon = 1;\n\tn_iterations = 6;\n\treduction = .5;\n\toptions.verbosity = 2; % Change this number for more or less output\n warning('off', 'manopt:getHessian:approx');\n for iter = 1 : n_iterations\n U = trustregions(problem, U, options);\n epsilon = epsilon * reduction;\n end\n warning('on', 'manopt:getHessian:approx');\n \n \n\t% Return the cost as the actual sum of distances, not smoothed.\n\tepsilon = 0;\n\tcost = robustpca_cost(U);\n \n \n \n % If working with the auto-generated input, plot the results.\n if nargin == 0\n scatter(X(1,:), X(2,:));\n hold on;\n plot(U(1)*[-1, 1]*100, U(2)*[-1 1]*100, 'r');\n hold off;\n % Compare to a standard PCA\n [Upca, ~, ~] = svds(X,1);\n hold on;\n plot(Upca(1)*[-1, 1]*100, Upca(2)*[-1 1]*100, 'k');\n hold off;\n xlim(1.1*[min(X(1,:)), max(X(1,:))]);\n ylim(1.1*[min(X(2,:)), max(X(2,:))]);\n legend('data points', 'Robust PCA fit', 'Standard PCA fit');\n end\n\n \n \n % Smoothed cost\n function value = robustpca_cost(U)\n\n vecs = U*(U'*X) - X;\n sqnrms = sum(vecs.^2, 1);\n vals = sqrt(sqnrms + epsilon^2) - epsilon;\n value = mean(vals);\n\n end\n\n % Euclidean gradient of the smoothed cost (it will be transformed into\n % the Riemannian gradient automatically by Manopt).\n function G = robustpca_gradient(U)\n\n\t\t% Note that the computation of vecs and sqnrms is redundant\n\t\t% with their computation in the cost function. To speed\n\t\t% up the code, it would be wise to use the caching capabilities\n\t\t% of Manopt (the store structure). See online documentation.\n\t\t% It is not done here to keep the code a bit simpler.\n UtX = U'*X;\n vecs = U*UtX-X;\n sqnrms = sum(vecs.^2, 1);\n % This explicit loop is a bit slow: the code below is equivalent\n % and faster to compute the gradient.\n % G = zeros(p, d);\n % for i=1:n\n % G = G + (1/sqrt(sqnrms(i) + epsilon^2)) * vecs(:,i) * UtX(:,i)';\n % end\n % G = G/n;\n G = mean(multiscale(1./sqrt(sqnrms + epsilon^2), ...\n multiprod(reshape(vecs, [p, 1, n]), ...\n multitransp(reshape(UtX, [d, 1, n])))), 3);\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/shapefit_smoothed.m", "content": "function [T_hub, T_lsq, T_cvx] = shapefit_smoothed(V, J)\n% ShapeFit formulation for sensor network localization from pair directions\n%\n% function [T_hub, T_lsq, T_cvx] = shapefit_smoothed(V, J)\n%\n% This example in based on the paper http://arxiv.org/abs/1506.01437:\n% ShapeFit: Exact location recovery from corrupted pairwise directions, 2015\n% by Paul Hand, Choongbum Lee and Vladislav Voroninski.\n%\n% The problem is the following: there are n points t_1, ..., t_n in R^d\n% which need to be estimated (localized). To this end, we are given\n% measurements of some of the pairwise directions,\n% v_ij = (t_i - t_j) / norm(t_i - t_j) + noise.\n% Assume there are m such pairwise measurements, defining a graph with m\n% edges over n nodes. J is the signed incidence matrix of this graph (see\n% in code). To build J from lists I, J in R^m of nodes, use:\n% J = sparse([I ; J], [(1:m)' ; (1:m)'], [ones(m, 1), -ones(m, 1)], n, m, 2*m);\n%\n% The measurements are arranged in the matrix V of size d x m. From V, we\n% attempt to estimate t_1, ..., t_n, arranged in T, a matrix of size d x n.\n% The estimation can only be done up to translation and scaling. The\n% returned T's are centered: the columns sum to zero.\n%\n% ShapeFit is a formulation of this estimation problem which is robust to\n% outliers. It is a nonsmooth, convex optimization problem over an affine\n% space, i.e., a linear manifold. We smooth the cost using the pseudo-Huber\n% loss cost and solve the problem using Manopt. This requires two\n% ingredients: (1) a factory to describe the affine space, see\n% shapefitfactory; (2) defining the cost and its derivative, and minimizing\n% it while progressively tightening the smooth approximation (with\n% warm-start).\n%\n% Simply run the example to see the results on random data. It compares the\n% smoothed ShapeFit formulation against a least-squares formulation, when\n% the measurements include outliers. See in code to vary the noise\n% parameters, dimension d, number of nodes n, number of measurements m, ...\n%\n% Note: since the problem is convex, this returns the global optimum.\n% This example also illustrates the use of Manopt for optimization under\n% linear constraints: admittedly a simple subcase of optimization on\n% manifolds.\n%\n%\n% See also: shapefitfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 18, 2015.\n% Contributors: \n% Change log: \n\n\n % Generic useful functions\n center_cols = @(A) bsxfun(@minus, A, mean(A, 2));\n normalize_cols = @(A) bsxfun(@times, A, 1./sqrt(sum(A.^2, 1)));\n sqnorm_cols = @(A) sum(A.^2, 1);\n\n \n % DATA GENERATION\n %\n % If no inputs are specified, generate some random data for\n % illustration purposes.\n if nargin == 0\n\n % We estimate n points in R^d\n d = 2;\n n = 500;\n\n % Those points are the columns of T : they are what we need to\n % estimate, up to scaling and translation. We center T for\n % convenience.\n T_tru = center_cols(rand(d, n));\n\n % We get a measurement of some pairs of relative directions.\n % Which pairs is encoded in this graph, with J being the (signed,\n % transposed) incidence matrix. J is n x m, sparse.\n % There are roughly edge_fraction * n * (n-1) / 2 measurements.\n edge_fraction = 0.10;\n % [ii, jj] = erdosrenyi(n, edge_fraction);\n [ii, jj] = randomgraph(n, edge_fraction*nchoosek(n, 2));\n m = length(ii);\n J = sparse([ii ; jj], [(1:m)' ; (1:m)'], ...\n [ones(m, 1), -ones(m, 1)], n, m, 2*m);\n\n % The measurements give the directions from one point to another.\n % That is: we get the position difference, normalized. Here, with\n % Gaussian noise. Least-squares will be well-suited for this.\n sigma = .0;\n V = normalize_cols(T_tru*J + sigma*randn(d, m)); % d x m\n\n % Outliers: we replace some of the direction measurements by\n % uniformly random unit-norm vectors.\n outlier_fraction = .3;\n outliers = rand(1, m) < outlier_fraction;\n V(:, outliers) = normalize_cols(randn(d, sum(outliers)));\n \n end % done generating random data\n \n \n \n \n \n [d, m] = size(V);\n n = size(J, 1);\n assert(size(J, 2) == m, 'J must be n x m, with V of size d x m.');\n\n VJt = full(V*J');\n\n % This \"manifold\" describes the Euclidean space of matrices T of size\n % d x n such that = 1 and T has centered columns: T1 = 0.\n problem.M = shapefitfactory(VJt);\n\n % This linear operator computes the orthogonal projection of each\n % difference ti - tj on the orthogonal space to v_ij.\n % If the alignment is compatible with the data, then this is zero.\n % A(T) is a d x m matrix.\n function AT = A(T)\n TJ = T*J;\n AT = TJ - bsxfun(@times, V, sum(V .* TJ, 1));\n end\n\n % Need the adjoint of A, too. Input is d x m, output is d x n.\n Astar = @(W) (W - bsxfun(@times, V, sum(V.*W, 1)))*J';\n\n \n \n % LEAST-SQUARES\n %\n % First, work with a least-squares formulation of the problem.\n % That is, we minimize a (very nice) convex cost over an affine space.\n % Since the smooth solvers in Manopt converge to critical points, this\n % means they converge to global optimizers.\n problem.cost = @(T) 0.5*norm(A(T), 'fro')^2;\n problem.egrad = @(T) Astar(A(T));\n problem.ehess = @(T, Tdot) Astar(A(Tdot));\n\n T_lsq = trustregions(problem);\n \n\n \n % PSEUDO-HUBER SMOOTHED SHAPEFIT\n %\n % Now solve the same, but with a pseudo-Huber loss instead of\n % least-squares.\n % We iteratively sharpen the Huber function, i.e., reduce delta.\n % It is important to warm start in such a fashion: trying to optimize\n % with a random initial guess and a very small delta is typically slow.\n % How fast one should decrease delta, and how accurately one should\n % optimize at each intermediate stage, is open for research.\n delta = 1;\n T_hub = []; % We could use T_lsq as initial guess, too.\n problem = rmfield(problem, 'ehess');\n warning('off', 'manopt:getHessian:approx');\n for iter = 1 : 12\n \n delta = delta / 2;\n \n h = @(x2) sqrt(x2 + delta^2) - delta; % pseudo-Huber loss\n\n problem.cost = @(T) sum(h(sqnorm_cols(A(T))));\n problem.egrad = @(T) Astar(bsxfun(@times, A(T), ...\n 1./sqrt(sqnorm_cols(A(T)) + delta^2)));\n\n % Solve, using the previous solution as initial guess.\n T_hub = trustregions(problem, T_hub);\n \n end\n \n \n \n % CVX SHAPEFIT\n %\n % Actual ShapeFit cost (nonsmooth), with CVX.\n % You can get CVX from http://cvxr.com/.\n use_cvx_if_available = false;\n if use_cvx_if_available && exist('cvx_version', 'file')\n T_cvx = shapefit_cvx(V, J);\n else\n T_cvx = NaN(d, n);\n end\n \n \n \n % VISUALIZATION\n %\n % If T_true is available, for display, we scale the estimators to match\n % the norm of the target. The scaling factor is obtained by minimizing\n % the norm of the discrepancy : norm(T_tru - scale*T_xxx, 'fro').\n % A plot is produced if d is 2 or 3.\n if exist('T_tru', 'var') && (d == 2 || d == 3)\n \n T_lsq = T_lsq * trace(T_tru'*T_lsq) / norm(T_lsq, 'fro')^2;\n T_hub = T_hub * trace(T_tru'*T_hub) / norm(T_hub, 'fro')^2;\n T_cvx = T_cvx * trace(T_tru'*T_cvx) / norm(T_cvx, 'fro')^2;\n\n \n switch d\n case 2\n plot(T_tru(1, :), T_tru(2, :), 'bo', ...\n T_lsq(1, :), T_lsq(2, :), 'rx', ...\n T_hub(1, :), T_hub(2, :), 'k.', ...\n T_cvx(1, :), T_cvx(2, :), 'g.');\n case 3\n plot3(T_tru(1, :), T_tru(2, :), T_tru(3, :), 'bo', ...\n T_lsq(1, :), T_lsq(2, :), T_lsq(3, :), 'rx', ...\n T_hub(1, :), T_hub(2, :), T_hub(3, :), 'k.', ...\n T_cvx(1, :), T_cvx(2, :), T_cvx(3, :), 'g.');\n end\n\n legend('ground truth', 'least squares', ...\n sprintf('pseudo-huber, \\\\delta = %.1e', delta), ...\n 'CVX ShapeFit');\n \n title(sprintf(['ShapeFit problem : d = %d, n = %d, edge ' ...\n 'fraction = %.2g, sigma = %.2g, outlier ' ...\n 'fraction = %.2g'], d, n, edge_fraction, sigma, ...\n outlier_fraction));\n axis equal;\n \n end\n\nend\n\n\n% If CVX is available, it can be used to solve the nonsmooth problem\n% directly, very elegantly.\nfunction T_cvx = shapefit_cvx(V, J)\n d = size(V, 1);\n n = size(J, 1); %#ok\n VJt = full(V*J');\n cvx_begin\n variable T_cvx(d, n)\n % We want to minimize this:\n % minimize sum( norms( A(T_cvx), 2, 1 ) )\n % But unfortunately, CVX doesn't handle bsxfun. Instead, we use\n % repmat, which is slower, and hence hurts the comparison in\n % disfavor of CVX.\n minimize sum( norms( T_cvx*J - V .* repmat(sum(V .* (T_cvx*J), 1), [d, 1]) , 2, 1 ) )\n sum(T_cvx, 2) == zeros(d, 1); %#ok\n VJt(:).' * T_cvx(:) == 1; %#ok\n cvx_end\nend\n\n\nfunction [I, J, A] = erdosrenyi(n, p) %#ok\n% Generate a random Erdos-Renyi graph with n nodes and edge probability p.\n%\n% [I, J, A] = erdosrenyi(n, p)\n% \n% Returns a list of edges (I(k), J(k)) for a random, undirected Erdos-Renyi\n% graph with n nodes and edge probability p. A is the adjacency matrix.\n%\n% I(k) < J(k) for all k, i.e., all(I\n\n % The optimization takes place over a Stiefel manifold whose dimension\n % is independent of n. This is especially useful when there are many\n % more variables than samples.\n St = stiefelfactory(p, m);\n problem.M = St;\n\n % In this helper function, given a point 'X' on the manifold we check\n % whether the caching structure 'store' has been populated with\n % quantities that are useful to compute at X or not. If they were not,\n % then we compute and store them now.\n function store = prepare(X, store)\n if ~isfield(store, 'ready') || ~store.ready\n store.AtX = A'*X;\n store.absAtX = abs(store.AtX);\n store.pos = max(0, store.absAtX - gamma);\n store.ready = true;\n end\n end\n\n % Define the cost function here and set it in the problem structure.\n problem.cost = @cost;\n function [f, store] = cost(X, store)\n store = prepare(X, store);\n pos = store.pos;\n f = -.5*norm(pos, 'fro')^2;\n end\n\n % Here, we chose to define the Euclidean gradient (egrad instead of\n % grad) : Manopt will take care of converting it to the Riemannian\n % gradient.\n problem.egrad = @egrad;\n function [G, store] = egrad(X, store)\n if ~isfield(store, 'G')\n store = prepare(X, store);\n pos = store.pos;\n AtX = store.AtX;\n sgAtX = sign(AtX);\n factor = pos.*sgAtX;\n store.G = -A*factor;\n end\n G = store.G;\n end\n\n % checkgradient(problem);\n % pause;\n\n % The optimization happens here. To improve the method, it may be\n % interesting to investigate better-than-random initial iterates and,\n % possibly, to fine tune the parameters of the solver.\n X = trustregions(problem);\n\n % Compute the sparsity pattern by thresholding\n P = abs(A'*X) > gamma;\n \nend\n\n\n% This post-processing algorithm produces a matrix Z of size nxm matching\n% the sparsity pattern P and representing sparse principal components for\n% A. This is to be called with the output of the main algorithm. This\n% algorithm is described in the reference paper by Journee et al.\nfunction Z = postprocess(A, P, X)\n fprintf('Post-processing... ');\n counter = 0;\n maxiter = 1000;\n tolerance = 1e-8;\n while counter < maxiter\n Z = A'*X;\n Z(~P) = 0;\n Z = Z*diag(1./sqrt(diag(Z'*Z)));\n X = ufactor(A*Z);\n counter = counter + 1;\n if counter > 1 && norm(Z0-Z, 'fro') < tolerance*norm(Z0, 'fro')\n break;\n end\n Z0 = Z;\n end\n fprintf('done, in %d iterations (max = %d).\\n', counter, maxiter);\nend\n\n% Returns the U-factor of the polar decomposition of X\nfunction U = ufactor(X)\n [W, S, V] = svd(X, 0); %#ok\n U = W*V';\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/thomson_problem.m", "content": "function X = thomson_problem(n, d)\n% Simple attempt at computing n well distributed points on a sphere in R^d.\n% \n% This is an example of how Manopt can approximate the gradient and even\n% the Hessian of a cost function based on finite differences, even if only\n% the cost function is specified without its derivatives.\n%\n% This functionality is provided only as a help for prototyping, and should\n% not be used to compare algorithms in terms of computation time or\n% accuracy, because the underlying gradient approximation scheme is slow.\n%\n% See also the derivative free solvers for an alternative:\n% pso and neldermead.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Nov. 1, 2016\n% Contributors:\n% Change log:\n\nif ~exist('n', 'var') || isempty(n)\n n = 50;\nend\nif ~exist('d', 'var') || isempty(d)\n d = 3;\nend\n\n% Define the Thomson problem with 1/r^2 potential. That is: find n points\n% x_i on a sphere in R^d such that the sum over all pairs (i, j) of the\n% potentials 1/||x_i - x_j||^2 is minimized. Since the points are on a\n% sphere, each potential is .5/(1-x_i'*x_j).\nproblem.M = obliquefactory(d, n);\nproblem.cost = @(X) sum(sum(triu(1./(1-X'*X), 1))) / n^2;\n\n% Attempt to minimize the cost. Since the gradient is not provided, Manopt\n% approximates it with finite differences. This is /slow/, since for each\n% gradient approximation, problem.M.dim()+1 calls to the cost function are\n% necessary, on top of generating an orthonormal basis of the tangent space\n% at each iterate.\n%\n% Note that it is difficult to reach high accuracy critical points with an\n% approximate gradient, hence it may be required to set a less ambitious\n% value for the gradient norm tolerance.\nopts.tolgradnorm = 1e-4;\n\n% Pick a solver. Both work fairly well on this problem.\n% X = conjugategradient(problem, [], opts);\nX = rlbfgs(problem, [], opts);\n\n% Plot the points on a translucid sphere\nif nargout == 0 && d == 3\n [x, y, z] = sphere(50);\n surf(x, y, z, 'FaceAlpha', .5);\n hold all;\n plot3(X(1, :), X(2, :), X(3, :), '.', 'MarkerSize', 20);\n axis equal;\n box off;\n axis off;\nend\n\n% For much better performance, after an early prototyping phase, the\n% gradient of the cost function should be specified, typically in\n% problem.grad or problem.egrad. See the online document at\n%\n% http://www.manopt.org\n%\n% for more information.\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/examples/truncated_svd.m", "content": "function [U, S, V, info] = truncated_svd(A, p)\n% Returns an SVD decomposition of A truncated to rank p.\n%\n% function [U, S, V, info] = truncated_svd(A, p)\n%\n% Input: A real matrix A of size mxn and an integer p <= min(m, n).\n% Output: An orthonormal matrix U of size mxp, an orthonormal matrix Y of\n% size nxp and a diagonal matrix S of size pxp with nonnegative and\n% decreasing diagonal entries such that USV.' is the best rank p\n% approximation of A according to the Frobenius norm. All real.\n% This function produces an output akin to svds.\n% \n% The decomposition is obtained by maximizing\n% f(U, V) = .5*norm(U'*A*V, 'fro')^2\n% where U, V are orthonormal. Notice that f(U*Q, V*R) = f(U, V) for all\n% Q, R orthogonal pxp matrices. Hence, only the column spaces of U and V\n% matter and we may perform the optimization over a product of two\n% Grassmannian manifolds.\n%\n% It is easy to show that maximizing f is equivalent to minimizing g with\n% g(U, V) = min_S norm(U*S*V' - A, 'fro')^2,\n% which confirms that we are going for a best low-rank approximation of A.\n% \n% The inner workings of the Grassmann manifold use the built-in svd\n% function of Matlab but only for matrices of size mxp and nxp to\n% re-orthonormalize them.\n% \n% Notice that we are actually chasing a best fixed-rank approximation of a\n% matrix, which is best obtained by working directly over a manifold of\n% fixed-rank matrices. This is simply an example script to demonstrate some\n% functionalities of the toolbox.\n% \n% The code can be modified to accept a function handle for A(x) = A*x\n% instead of a matrix A, which is often useful. This would further require\n% a function handle At for the transpose of A, such that At(x) = A.'*x.\n\n% This file is part of Manopt and is copyrighted. See the license file.\n% \n% Main author: Nicolas Boumal, July 5, 2013\n% Contributors:\n% \n% Change log:\n% \n\n \n % Generate some random data to test the function if none is given.\n if ~exist('A', 'var') || isempty(A)\n A = randn(42, 60);\n end\n if ~exist('p', 'var') || isempty(p)\n p = 5;\n end\n \n % Retrieve the size of the problem and make sure the requested\n % approximation rank is at most the maximum possible rank.\n [m, n] = size(A);\n assert(p <= min(m, n), 'p must be smaller than the smallest dimension of A.');\n \n % Define the cost and its derivatives on the Grassmann manifold\n tuple.U = grassmannfactory(m, p);\n tuple.V = grassmannfactory(n, p);\n % All of the code will work just as well if we ignore the invariance\n % property of the cost function indicated above and thus place U and V\n % on the Stiefel manifold (orthonormal matrices) instead of the\n % Grassmann manifold. Working on Stiefel is expected to be slower\n % though, partly because de search space is higher dimensional and\n % partly because the optimizers are not isolated.\n % tuple.U = stiefelfactory(m, p);\n % tuple.V = stiefelfactory(n, p);\n M = productmanifold(tuple);\n \n % Define a problem structure, specifying the manifold M, the cost\n % function and its derivatives. Here, to demonstrate the rapid\n % prototyping capabilities of Manopt, we directly define the Euclidean\n % gradient and the Euclidean Hessian egrad and ehess instead of the\n % Riemannian gradient and Hessian grad and hess. Manopt will take care\n % of the conversion. This automatic conversion is usually not\n % computationally optimal though, because much of the computations\n % involved in obtaining the gradient could be reused to obtain the\n % Hessian. After the prototyping stage, when efficiency becomes\n % important, it makes sense to define grad and hess rather than egrad\n % an ehess, and to use the caching system (the store structure).\n problem.M = M;\n problem.cost = @cost;\n problem.egrad = @egrad;\n problem.ehess = @ehess;\n \n % The functions below make many redundant computations. This\n % performance hit can be alleviated by using the caching system.\n \n % Cost function\n function f = cost(X)\n U = X.U;\n V = X.V;\n f = -.5*norm(U'*A*V, 'fro')^2;\n end\n % Euclidean gradient of the cost function\n function g = egrad(X)\n U = X.U;\n V = X.V;\n AV = A*V;\n AtU = A'*U;\n g.U = -AV*(AV'*U);\n g.V = -AtU*(AtU'*V);\n end\n % Euclidean Hessian of the cost function\n function h = ehess(X, H)\n U = X.U;\n V = X.V;\n Udot = H.U;\n Vdot = H.V;\n AV = A*V;\n AtU = A'*U;\n AVdot = A*Vdot;\n AtUdot = A'*Udot;\n h.U = -(AVdot*AV'*U + AV*AVdot'*U + AV*AV'*Udot);\n h.V = -(AtUdot*AtU'*V + AtU*AtUdot'*V + AtU*AtU'*Vdot);\n end\n \n \n % Execute some checks on the derivatives for early debugging.\n % These things can be commented out of course.\n % checkgradient(problem);\n % pause;\n % checkhessian(problem);\n % pause;\n \n % Issue a call to a solver. A random initial guess will be chosen and\n % default options are selected. Here, we specify a maximum trust\n % region radius (which in turn induces an initial trust region radius).\n % Note that this is not required: default values are used if we omit\n % this. The diameter of the manifold scales like sqrt(2*p), hence the\n % form of our (empirical) choice.\n options.Delta_bar = 4*sqrt(2*p);\n [X, Xcost, info] = trustregions(problem, [], options); %#ok\n U = X.U;\n V = X.V;\n \n % Finish the job by rotating U and V such that the middle matrix S can\n % be diagonal with nonnegative, decreasing entries. This requires a\n % small svd of size pxp.\n Spp = U'*A*V;\n [Upp, Spp, Vpp] = svd(Spp);\n U = U*Upp;\n S = Spp;\n V = V*Vpp;\n \n % For our information, Manopt can also compute the spectrum of the\n % Riemannian Hessian on the tangent space at (any) X. Computing the\n % spectrum at the solution gives us some idea of the conditioning of\n % the problem. If we were to implement a preconditioner for the\n % Hessian, this would also inform us on its performance.\n %\n % Notice that if the optimization is performed on a product of Stiefel\n % manifolds instead of a product of Grassmannians, the double\n % invariance under the orthogonal group O(p) will appear as twice\n % p*(p-1)/2, thus p*(p-1) zero eigenvalues in the spectrum of the\n % Hessian. This means that the minimizers are not isolated, which\n % typically hinders convergence of second order algorithms.\n if M.dim() < 512\n evs = hessianspectrum(problem, X);\n stairs(sort(evs));\n title(['Eigenvalues of the Hessian of the cost function ' ...\n 'at the solution']);\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/importmanopt.m", "content": "% Add Manopt to the path to make all manopt components available.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Jan. 3, 2013.\n% Contributors: \n% Change log: \n% Aug. 7, 2013 (NB): Changed to work without the import command\n% (new structure of the toolbox).\n% Aug. 8, 2013 (NB): Changed to use addpath_recursive, home brewed.\n% Aug. 22, 2013 (NB): Using genpath instead of home cooked\n% addpath_recursive.\n\naddpath(pwd);\n\n% Recursively add Manopt directories to the Matlab path.\ncd manopt;\naddpath(genpath(pwd));\ncd ..;\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/StoreDB.m", "content": "classdef StoreDB < handle_light\n% The StoreDB class is a handle class to manage caching in Manopt.\n%\n% To create an object, call: storedb = StoreDB();\n% Alternatively, call: storedb = StoreDB(storedepth); to instruct\n% the database to keep at most storedepth store's in its history.\n% (Note that clean up only happens when purge() is called).\n%\n% The storedb object is passed by reference: when it is passed to a\n% function as an input, and that function modifies it, the original\n% object is modified.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 3, 2015.\n% Contributors: \n% Change log: \n\n% TODO : protect get/setWithShared calls: limit to one, and forbid access\n% to shared memory while it has not been returned.\n% Do think of the applyStatsFun case : calls a getWithShared, does\n% not need a setWithShared. I think for statsfun there should be a\n% method \"forfeitWithShared\".\n \n properties(Access = public)\n \n % This memory is meant to be shared at all times. Users can modify\n % this at will. It is the same for all points x.\n shared = struct();\n \n % This memory is used by the toolbox for, e.g., automatic caching\n % and book keeping. Users should not overwrite this. It is the\n % same for all points x.\n internal = struct();\n \n % When calling purge(), only a certain number of stores will be\n % kept in 'history'. This parameter fixes that number. The most\n % recently modified stores are kept. Set to inf to keep all stores.\n storedepth = inf;\n \n end\n \n properties(Access = private)\n \n % This structure holds separate memories for individual points.\n % Use get and set to interact with this. The field name 'shared' is\n % reserved, for use with get/setWithShared.\n history = struct();\n \n % This internal counter is used to obtain unique key's for points.\n counter = uint32(0);\n \n % This internal counter is used to time calls to 'set', and hence\n % keep track of which stores in 'history' were last updated.\n timer = uint32(0);\n \n end\n \n \n methods(Access = public)\n \n % Constructor\n function storedb = StoreDB(storedepth)\n if nargin >= 1\n storedb.storedepth = storedepth;\n end\n end\n \n % Return the store associated to a given key.\n % If the key is unknown, returns an empty structure.\n function store = get(storedb, key)\n if isfield(storedb.history, key)\n store = storedb.history.(key);\n else\n store = struct();\n end\n end\n \n % Same as get, but adds the shared memory in store.shared.\n function store = getWithShared(storedb, key)\n store = storedb.get(key);\n store.shared = storedb.shared;\n end\n \n % Save the given store at the given key. If no key is provided, a\n % new key is generated for this store (i.e., it is assumed this\n % store pertains to a new point). The key is returned in all cases.\n % A field 'lastset__' is added/updated in the store structure,\n % keeping track of the last time that store was modified.\n function key = set(storedb, store, key)\n if nargin < 3\n key = getNewKey(storedb);\n end\n store.lastset__ = storedb.timer;\n storedb.timer = storedb.timer + 1;\n storedb.history.(key) = store;\n end\n \n % Same as set, but extracts the shared memory and saves it.\n % The stored store will still have a 'shared' field, but it will be\n % empty.\n function key = setWithShared(storedb, store, key)\n storedb.shared = store.shared;\n store.shared = [];\n key = storedb.set(store, key);\n end\n \n % Generate a unique key and return it. This should be called\n % everytime a new point is generated / stored. Keys are valid field\n % names for structures.\n function key = getNewKey(storedb)\n key = sprintf('z%d', storedb.counter);\n storedb.counter = storedb.counter + 1;\n end\n \n % Clear entries in storedb.history to limit memory usage.\n function purge(storedb)\n \n if isinf(storedb.storedepth)\n return;\n end\n \n if storedb.storedepth <= 0\n storedb.history = struct();\n return;\n end\n\n % Get list of field names (keys).\n keys = fieldnames(storedb.history);\n nkeys = length(keys);\n\n % If we need to remove some of the elements in the database,\n if nkeys > storedb.storedepth\n\n % Get the last-set counter of each element:\n % a higher number means it was modified more recently.\n lastset = zeros(nkeys, 1, 'uint32');\n for i = 1 : nkeys\n lastset(i) = storedb.history.(keys{i}).lastset__;\n end\n\n % Sort the counters and determine the threshold above which\n % the field needs to be removed.\n sortlastset = sort(lastset, 1, 'descend');\n minlastset = sortlastset(storedb.storedepth);\n\n % Remove all fields that are too old.\n storedb.history = rmfield(storedb.history, ...\n keys(lastset < minlastset));\n end\n \n end % end of purge()\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/applyStatsfun.m", "content": "function stats = applyStatsfun(problem, x, storedb, key, options, stats)\n% Apply the statsfun function to a stats structure (for solvers).\n%\n% function stats = applyStatsfun(problem, x, storedb, key, options, stats)\n%\n% Applies the options.statsfun user supplied function (if it was provided)\n% to the stats structure, and returns the (possibly) modified stats\n% structure.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% Note: if statsfun accepts a store structure as input, this structure can\n% be read but not modified (modifications will be lost) ; the store\n% structure will contain the store.shared field.\n%\n% See also: \n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 3, 2013.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n\tif isfield(options, 'statsfun')\n\t\t\n switch nargin(options.statsfun)\n case 3\n stats = options.statsfun(problem, x, stats);\n case 4\n % Obtain, pass along, and save the store for x.\n % get/setWithShared must come in pairs.\n store = storedb.getWithShared(key);\n stats = options.statsfun(problem, x, stats, store);\n storedb.setWithShared(store, key);\n otherwise\n warning('manopt:statsfun', ...\n 'statsfun unused: wrong number of inputs');\n end\n\tend\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetApproxGradient.m", "content": "function candoit = canGetApproxGradient(problem)\n% Checks whether an approximate gradient can be computed for this problem.\n%\n% function candoit = canGetApproxGradient(problem)\n%\n% Returns true if an approximate gradient of the cost function is provided\n% in the given problem description, false otherwise.\n% If a gradient is defined but no approximate gradient is defined\n% explicitly, returns false.\n%\n% Even if this returns false, calls to getApproxGradient may succeed, as\n% they will be redirected to getGradientFD. The latter simply requires\n% availability of the cost in problem.\n%\n% See also: canGetGradient getGradientFD\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Nov. 1, 2016.\n% Contributors: \n% Change log: \n\n candoit = isfield(problem, 'approxgrad');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetApproxHessian.m", "content": "function candoit = canGetApproxHessian(problem)\n% Checks whether an approximate Hessian can be computed for this problem.\n%\n% function candoit = canGetApproxHessian(problem)\n%\n% Returns true if an approximate Hessian of the cost function is provided\n% in the given problem description, false otherwise.\n% If a Hessian is defined but no approximate Hessian is defined explicitly,\n% returns false.\n%\n% Even if this returns false, calls to getApproxHessian may succeed, as\n% they will be redirected to getHessianFD. The latter simply requires\n% availability of gradients in problem, and vector transports in problem.M.\n%\n% See also: canGetHessian getHessianFD\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 8, 2015.\n% Contributors: \n% Change log: \n\n candoit = isfield(problem, 'approxhess');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetCost.m", "content": "function candoit = canGetCost(problem)\n% Checks whether the cost function can be computed for a problem structure.\n%\n% function candoit = canGetCost(problem)\n%\n% Returns true if the cost function can be computed given the problem\n% description, false otherwise.\n%\n% See also: getCost canGetDirectionalDerivative canGetGradient canGetHessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n\n\n candoit = isfield(problem, 'cost') || isfield(problem, 'costgrad');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetDirectionalDerivative.m", "content": "function candoit = canGetDirectionalDerivative(problem)\n% Checks whether dir. derivatives can be computed for a problem structure.\n% \n% function candoit = canGetDirectionalDerivative(problem)\n%\n% Returns true if the directional derivatives of the cost function can be\n% computed given the problem description, false otherwise.\n%\n% See also: canGetCost canGetGradient canGetHessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n\n candoit = isfield(problem, 'diff') || canGetGradient(problem);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetEuclideanGradient.m", "content": "function candoit = canGetEuclideanGradient(problem)\n% Checks whether the Euclidean gradient can be computed for a problem.\n%\n% function candoit = canGetEuclideanGradient(problem)\n%\n% Returns true if the Euclidean gradient can be computed given the problem\n% description, false otherwise.\n%\n% See also: canGetGradient getEuclideanGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% June 28, 2016 (NB):\n% Added support for getPartialEuclideanGradient\n\n\n candoit = isfield(problem, 'egrad') || canGetPartialEuclideanGradient(problem);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetGradient.m", "content": "function candoit = canGetGradient(problem)\n% Checks whether the gradient can be computed for a problem structure.\n% \n% function candoit = canGetGradient(problem)\n%\n% Returns true if the gradient of the cost function can be computed given\n% the problem description, false otherwise.\n%\n% See also: canGetCost canGetDirectionalDerivative canGetHessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% June 28, 2016 (NB):\n% Added support for getPartialGradient\n%\n% Nov. 1, 2016 (NB):\n% Added support for gradient from directional derivatives\n\n candoit = isfield(problem, 'grad') || isfield(problem, 'costgrad') || ...\n canGetEuclideanGradient(problem) || ...\n canGetPartialGradient(problem) || ...\n ... % Check if directional derivatives can be obtained, since\n ... % it is possible to compute the gradient from directional\n ... % derivatives (expensively). Here, it is not possible to\n ... % call canGetDirectionalDerivative, because that function\n ... % would then potentially call canGetGradient, thus \n ... % starting an infinite loop. As a result, we have some\n ... % code redundancy: the check below needs to be kept\n ... % equivalent to the check in canGetDirectionalDerivative.\n isfield(problem, 'diff');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetHessian.m", "content": "function candoit = canGetHessian(problem)\n% Checks whether the Hessian can be computed for a problem structure.\n%\n% function candoit = canGetHessian(problem)\n%\n% Returns true if the Hessian of the cost function can be computed given\n% the problem description, false otherwise.\n%\n% See also: canGetCost canGetDirectionalDerivative canGetGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n\n candoit = isfield(problem, 'hess') || ...\n (isfield(problem, 'ehess') && canGetEuclideanGradient(problem));\n \n % Display an extra warning message to the user in anticipation of\n % common mistakes.\n if ~candoit && ...\n (isfield(problem, 'ehess') && ~canGetEuclideanGradient(problem))\n warning('manopt:canGetHessian', ...\n ['If the Hessian is supplied as a Euclidean Hessian (ehess),\\n' ...\n 'then the Euclidean gradient must also be supplied (egrad).']);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetLinesearch.m", "content": "function candoit = canGetLinesearch(problem)\n% Checks whether the problem structure can give a line-search a hint.\n%\n% function candoit = canGetLinesearch(problem)\n%\n% Returns true if the the problem description includes a mechanism to give\n% line-search algorithms a hint as to \"how far to look\", false otherwise.\n%\n% See also: getLinesearch\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 17, 2014.\n% Contributors: \n% Change log: \n\n\n candoit = isfield(problem, 'linesearch');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetPartialEuclideanGradient.m", "content": "function candoit = canGetPartialEuclideanGradient(problem)\n% Checks whether the partial Euclidean gradient can be computed for a problem.\n% \n% function candoit = canGetPartialEuclideanGradient(problem)\n%\n% Returns true if the partial Euclidean gradient of the cost function can\n% be computed given the problem description, false otherwise.\n%\n% See also: getPartialEuclideanGradient canGetPartialGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 28, 2016.\n% Contributors: \n% Change log: \n\n candoit = (isfield(problem, 'partialegrad') && isfield(problem, 'ncostterms'));\n \n if isfield(problem, 'partialegrad') && ~isfield(problem, 'ncostterms')\n warning('manopt:partialegrad', ...\n ['If problem.partialegrad is specified, indicate the number n\\n' ...\n 'of terms in the cost function with problem.ncostterms = n.']);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetPartialGradient.m", "content": "function candoit = canGetPartialGradient(problem)\n% Checks whether the partial gradient can be computed for a given problem.\n% \n% function candoit = canGetPartialGradient(problem)\n%\n% Returns true if the partial gradient of the cost function can be computed\n% given the problem description, false otherwise.\n%\n% See also: getPartialGradient canGetPartialEuclideanGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 28, 2016.\n% Contributors: \n% Change log: \n\n candoit = (isfield(problem, 'partialgrad') && isfield(problem, 'ncostterms')) || ...\n canGetPartialEuclideanGradient(problem);\n \n if isfield(problem, 'partialgrad') && ~isfield(problem, 'ncostterms')\n warning('manopt:partialgrad', ...\n ['If problem.partialgrad is specified, indicate the number n\\n' ...\n 'of terms in the cost function with problem.ncostterms = n.']);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetPrecon.m", "content": "function candoit = canGetPrecon(problem)\n% Checks whether a preconditioner was specified in the problem description.\n%\n% function candoit = canGetPrecon(problem)\n%\n% Returns true if a preconditioner was specified, false otherwise. Notice\n% that even if this function returns false, it is still possible to call\n% getPrecon, as the default preconditioner is simply the identity operator.\n% This check function is mostly useful to tell whether that default\n% preconditioner will be in use or not.\n%\n% See also: getPrecon getSqrtPrecon canGetSqrtPrecon getHessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 3, 2013.\n% Contributors: \n% Change log: \n\n candoit = isfield(problem, 'precon') || canGetSqrtPrecon(problem);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetSqrtPrecon.m", "content": "function candoit = canGetSqrtPrecon(problem)\n% Checks whether a square root of preconditioner was specified in problem.\n%\n% function candoit = canGetSqrtPrecon(problem)\n%\n% Returns true if the problem structure allows for applying the square root\n% of a preconditioner to tangent vectors at a given point. The square root\n% of the preconditioner at x must be a symmetric, positive definite\n% operator Q such that applying Q twice (Q o Q) amounts to applying the\n% preconditioner once. If both a preconditioner and a square root of\n% preconditioner are provided, it is the user's responsibility to ensure\n% their compatibility.\n%\n% Similarly to getPrecon, if the present function returns false, calls to\n% getSqrtPrecon will still work: they will act as the identity. Note that\n% this may be incompatible with the preconditioner if it is given. Thus,\n% always check by calling canGetSqrtPrecon first.\n%\n% See also: canGetPrecon getSqrtPrecon getPrecon\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 3, 2015.\n% Contributors: \n% Change log: \n\n candoit = isfield(problem, 'sqrtprecon');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/canGetSubgradient.m", "content": "function candoit = canGetSubgradient(problem)\n% Checks whether a subgradient can be computed for a problem structure.\n% \n% function candoit = canGetSubgradient(problem)\n%\n% Returns true if a subgradient of the cost function can be computed given\n% the problem description, false otherwise.\n%\n% See also: canGetGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 20, 2017.\n% Contributors: \n% Change log: \n\n candoit = isfield(problem, 'subgrad') || canGetGradient(problem);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getApproxGradient.m", "content": "function approxgrad = getApproxGradient(problem, x, storedb, key)\n% Computes an approximation of the gradient of the cost function at x.\n%\n% function approxgrad = getApproxGradient(problem, x)\n% function approxgrad = getApproxGradient(problem, x, storedb)\n% function approxgrad = getApproxGradient(problem, x, storedb, key)\n%\n% Returns an approximation of the gradient at x for the cost function\n% described in the problem structure.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% If no approximate gradient was provided, this call is redirected to\n% getGradientFD.\n% \n% See also: getGradientFD canGetApproxGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Nov. 1, 2016.\n% Contributors: \n% Change log: \n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n\n if isfield(problem, 'approxgrad')\n %% Compute the approximate gradient using approxgrad.\n\t\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.approxgrad);\n case 1\n approxgrad = problem.approxgrad(x);\n case 2\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [approxgrad, store] = problem.approxgrad(x, store);\n storedb.setWithShared(store, key);\n case 3\n % Pass along the whole storedb (by reference), with key.\n approxgrad = problem.approxgrad(x, storedb, key);\n otherwise\n up = MException('manopt:getApproxGradient:badapproxgrad', ...\n 'approxgrad should accept 1, 2 or 3 inputs.');\n throw(up);\n end\n \n else\n %% Try to fall back to a standard FD approximation.\n \n approxgrad = getGradientFD(problem, x, storedb, key);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getApproxHessian.m", "content": "function approxhess = getApproxHessian(problem, x, d, storedb, key)\n% Computes an approximation of the Hessian of the cost fun. at x along d.\n%\n% function approxhess = getApproxHessian(problem, x, d)\n% function approxhess = getApproxHessian(problem, x, d, storedb)\n% function approxhess = getApproxHessian(problem, x, d, storedb, key)\n%\n% Returns an approximation of the Hessian at x along d of the cost function\n% described in the problem structure.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% If no approximate Hessian was provided, this call is redirected to\n% getHessianFD.\n% \n% See also: getHessianFD canGetApproxHessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n\n if isfield(problem, 'approxhess')\n %% Compute the approximate Hessian using approxhess.\n\t\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.approxhess);\n case 2\n approxhess = problem.approxhess(x, d);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [approxhess, store] = problem.approxhess(x, d, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n approxhess = problem.approxhess(x, d, storedb, key);\n otherwise\n up = MException('manopt:getApproxHessian:badapproxhess', ...\n 'approxhess should accept 2, 3 or 4 inputs.');\n throw(up);\n end\n \n else\n %% Try to fall back to a standard FD approximation.\n \n approxhess = getHessianFD(problem, x, d, storedb, key);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getCost.m", "content": "function cost = getCost(problem, x, storedb, key)\n% Computes the cost function at x.\n%\n% function cost = getCost(problem, x)\n% function cost = getCost(problem, x, storedb)\n% function cost = getCost(problem, x, storedb, key)\n%\n% Returns the value at x of the cost function described in the problem\n% structure.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: canGetCost\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n\n if isfield(problem, 'cost')\n %% Compute the cost function using cost.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.cost)\n case 1\n cost = problem.cost(x);\n case 2\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [cost, store] = problem.cost(x, store);\n storedb.setWithShared(store, key);\n case 3\n % Pass along the whole storedb (by reference), with key.\n cost = problem.cost(x, storedb, key);\n otherwise\n up = MException('manopt:getCost:badcost', ...\n 'cost should accept 1, 2 or 3 inputs.');\n throw(up);\n end\n \n elseif isfield(problem, 'costgrad')\n %% Compute the cost function using costgrad.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.costgrad)\n case 1\n cost = problem.costgrad(x);\n case 2\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [cost, grad, store] = problem.costgrad(x, store); %#ok\n storedb.setWithShared(store, key);\n case 3\n % Pass along the whole storedb (by reference), with key.\n cost = problem.costgrad(x, storedb, key);\n otherwise\n up = MException('manopt:getCost:badcostgrad', ...\n 'costgrad should accept 1, 2 or 3 inputs.');\n throw(up);\n end\n\n else\n %% Abandon computing the cost function.\n\n up = MException('manopt:getCost:fail', ...\n ['The problem description is not explicit enough to ' ...\n 'compute the cost.']);\n throw(up);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getCostGrad.m", "content": "function [cost, grad] = getCostGrad(problem, x, storedb, key)\n% Computes the cost function and the gradient at x in one call if possible.\n%\n% function [cost, grad] = getCostGrad(problem, x)\n% function [cost, grad] = getCostGrad(problem, x, storedb)\n% function [cost, grad] = getCostGrad(problem, x, storedb, key)\n%\n% Returns the value at x of the cost function described in the problem\n% structure, as well as the gradient at x.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: canGetCost canGetGradient getCost getGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n\n if isfield(problem, 'costgrad')\n %% Compute the cost/grad pair using costgrad.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.costgrad)\n case 1\n [cost, grad] = problem.costgrad(x);\n case 2\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [cost, grad, store] = problem.costgrad(x, store);\n storedb.setWithShared(store, key);\n case 3\n % Pass along the whole storedb (by reference), with key.\n [cost, grad] = problem.costgrad(x, storedb, key);\n otherwise\n up = MException('manopt:getCostGrad:badcostgrad', ...\n 'costgrad should accept 1, 2 or 3 inputs.');\n throw(up);\n end\n\n else\n %% Revert to calling getCost and getGradient separately\n \n cost = getCost(problem, x, storedb, key);\n grad = getGradient(problem, x, storedb, key);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getDirectionalDerivative.m", "content": "function diff = getDirectionalDerivative(problem, x, d, storedb, key)\n% Computes the directional derivative of the cost function at x along d.\n%\n% function diff = getDirectionalDerivative(problem, x, d)\n% function diff = getDirectionalDerivative(problem, x, d, storedb)\n% function diff = getDirectionalDerivative(problem, x, d, storedb, key)\n%\n% Returns the derivative at x along d of the cost function described in the\n% problem structure.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: getGradient canGetDirectionalDerivative\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n \n if isfield(problem, 'diff')\n %% Compute the directional derivative using diff.\n\t\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.diff)\n case 2\n diff = problem.diff(x, d);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [diff, store] = problem.diff(x, d, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n diff = problem.diff(x, d, storedb, key);\n otherwise\n up = MException('manopt:getDirectionalDerivative:baddiff', ...\n 'diff should accept 2, 3 or 4 inputs.');\n throw(up);\n end\n \n elseif canGetGradient(problem)\n %% Compute the directional derivative using the gradient.\n \n % Compute the gradient at x, then compute its inner product with d.\n grad = getGradient(problem, x, storedb, key);\n diff = problem.M.inner(x, grad, d);\n \n else\n %% Abandon computing the directional derivative.\n \n up = MException('manopt:getDirectionalDerivative:fail', ...\n ['The problem description is not explicit enough to ' ...\n 'compute the directional derivatives of f.']);\n throw(up);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getEuclideanGradient.m", "content": "function egrad = getEuclideanGradient(problem, x, storedb, key)\n% Computes the Euclidean gradient of the cost function at x.\n%\n% function egrad = getEuclideanGradient(problem, x)\n% function egrad = getEuclideanGradient(problem, x, storedb)\n% function egrad = getEuclideanGradient(problem, x, storedb, key)\n%\n% Returns the Euclidean gradient at x of the cost function described in the\n% problem structure.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% Because computing the Hessian based on the Euclidean Hessian will require\n% the Euclidean gradient every time, to avoid overly redundant\n% computations, if the egrad function does not use the store caching\n% capabilites, this implements an automatic caching functionality. Writing\n% egrad to accept the optional store or storedb parameter will disable\n% automatic caching, but allow user controlled caching.\n%\n% See also: getGradient canGetGradient canGetEuclideanGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 9, 2013.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% June 28, 2016 (NB):\n% Added support for getPartialEuclideanGradient\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n \n if isfield(problem, 'egrad')\n %% Compute the Euclidean gradient using egrad.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.egrad)\n case 1\n % If it does not want to deal with the store structure,\n % then we do some caching of our own. There is a small\n % performance hit for this is some cases, but we expect\n % that this is most often the preferred choice.\n store = storedb.get(key);\n if ~isfield(store, 'egrad__')\n store.egrad__ = problem.egrad(x);\n storedb.set(store, key);\n end\n egrad = store.egrad__;\n case 2\n % Obtain, pass along, and save the store for x.\n % If the user deals with the store structure, then we don't\n % do any automatic caching: the user is in control.\n store = storedb.getWithShared(key);\n [egrad, store] = problem.egrad(x, store);\n storedb.setWithShared(store, key);\n case 3\n % Pass along the whole storedb (by reference), with key.\n % Same here: no automatic caching.\n egrad = problem.egrad(x, storedb, key);\n otherwise\n up = MException('manopt:getEuclideanGradient:badegrad', ...\n 'egrad should accept 1, 2 or 3 inputs.');\n throw(up);\n end\n \n elseif canGetPartialEuclideanGradient(problem)\n %% Compute the Euclidean gradient using a full partial Euclidean gradient.\n \n d = problem.ncostterms;\n egrad = getPartialEuclideanGradient(problem, x, 1:d, storedb, key);\n\n else\n %% Abandon computing the Euclidean gradient\n \n up = MException('manopt:getEuclideanGradient:fail', ...\n ['The problem description is not explicit enough to ' ...\n 'compute the Euclidean gradient of the cost.']);\n throw(up);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getGlobalDefaults.m", "content": "function opts = getGlobalDefaults()\n% Returns a structure with default option values for Manopt.\n%\n% function opts = getGlobalDefaults()\n%\n% Returns a structure opts containing the global default options such as\n% verbosity level etc. Typically, global defaults are overwritten by solver\n% defaults, which are in turn overwritten by user-specified options.\n% See the online Manopt documentation for details on options.\n%\n% See also: mergeOptions\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n\n\n % There should be no reason to modify this file.\n % For better compatibility with future Manopt versions,\n % use the options structure of solvers.\n %\n % Really: don't modify it.\n \n\n % Verbosity level: 0 is no output at all. The higher the verbosity, the\n % more info is printed / displayed during solver execution.\n opts.verbosity = 3;\n \n % If debug is set to true, additional computations may be performed and\n % debugging information is outputed during solver execution.\n opts.debug = false;\n \n % Maximum number of store structures to store. If set to 0, caching\n % capabilities are not disabled, but the cache will be emptied at each\n % iteration of iterative solvers (more specifically: every time the\n % solver calls to purge the storedb).\n opts.storedepth = 20;\n \n % Maximum amount of time a solver may execute, in seconds.\n opts.maxtime = inf;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getGradient.m", "content": "function grad = getGradient(problem, x, storedb, key)\n% Computes the gradient of the cost function at x.\n%\n% function grad = getGradient(problem, x)\n% function grad = getGradient(problem, x, storedb)\n% function grad = getGradient(problem, x, storedb, key)\n%\n% Returns the gradient at x of the cost function described in the problem\n% structure.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: getDirectionalDerivative canGetGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% June 28, 2016 (NB):\n% Works with getPartialGradient.\n%\n% Nov. 1, 2016 (NB):\n% Added support for gradient from directional derivatives.\n% Last resort is call to getApproxGradient instead of an exception.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n \n if isfield(problem, 'grad')\n %% Compute the gradient using grad.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.grad)\n case 1\n grad = problem.grad(x);\n case 2\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [grad, store] = problem.grad(x, store);\n storedb.setWithShared(store, key);\n case 3\n % Pass along the whole storedb (by reference), with key.\n grad = problem.grad(x, storedb, key);\n otherwise\n up = MException('manopt:getGradient:badgrad', ...\n 'grad should accept 1, 2 or 3 inputs.');\n throw(up);\n end\n \n elseif isfield(problem, 'costgrad')\n %% Compute the gradient using costgrad.\n\t\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.costgrad)\n case 1\n [unused, grad] = problem.costgrad(x); %#ok\n case 2\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [unused, grad, store] = problem.costgrad(x, store); %#ok\n storedb.setWithShared(store, key);\n case 3\n % Pass along the whole storedb (by reference), with key.\n [unused, grad] = problem.costgrad(x, storedb, key); %#ok\n otherwise\n up = MException('manopt:getGradient:badcostgrad', ...\n 'costgrad should accept 1, 2 or 3 inputs.');\n throw(up);\n end\n \n elseif canGetEuclideanGradient(problem)\n %% Compute the gradient using the Euclidean gradient.\n \n egrad = getEuclideanGradient(problem, x, storedb, key);\n grad = problem.M.egrad2rgrad(x, egrad);\n \n elseif canGetPartialGradient(problem)\n %% Compute the gradient using a full partial gradient.\n \n d = problem.ncostterms;\n grad = getPartialGradient(problem, x, 1:d, storedb, key);\n \n elseif canGetDirectionalDerivative(problem)\n %% Compute gradient based on directional derivatives; expensive!\n \n B = tangentorthobasis(problem.M, x);\n df = zeros(size(B));\n for k = 1 : numel(B)\n df(k) = getDirectionalDerivative(problem, x, B{k}, storedb, key);\n end\n grad = lincomb(problem.M, x, B, df);\n\n else\n %% Attempt the computation of an approximation of the gradient.\n \n grad = getApproxGradient(problem, x, storedb, key);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getGradientFD.m", "content": "function gradfd = getGradientFD(problem, x, storedb, key)\n% Computes an approx. of the gradient w/ finite differences of the cost.\n%\n% function gradfd = getGradientFD(problem, x)\n% function gradfd = getGradientFD(problem, x, storedb)\n% function gradfd = getGradientFD(problem, x, storedb, key)\n%\n% Returns a finite difference approximation of the gradient at x for\n% the cost function described in the problem structure. The finite\n% difference is based on M.dim()+1 computations of the cost.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% If the cost cannot be computed, an exception is thrown.\n%\n% See also: approxgradientFD\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Nov. 1, 2016.\n% Contributors: \n% Change log: \n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n % This gradient approximation is based on the cost:\n % check availability.\n if ~canGetCost(problem)\n up = MException('manopt:getGradientFD:nocost', ...\n 'getGradientFD requires the cost to be computable.');\n throw(up);\n end\n \n \n % Default parameters. See approxgradientFD for explicit user access to\n % these parameters.\n stepsize = 2^-23;\n subspacedim = [];\n \n \n % Evaluate the cost at the root point\n fx = getCost(problem, x, storedb, key);\n\n % Pick an orthonormal basis for the tangent space at x, or a subspace\n % thereof. The default is a full subspace. If a strict subspace is\n % picked, the returned vector approximates the orthogonal projection of\n % the gradient to that subspace.\n B = tangentorthobasis(problem.M, x, subspacedim);\n \n % Use finite differences to approximate the directional derivative\n % along each direction in the basis B.\n df = zeros(size(B));\n for k = 1 : numel(B)\n % Move in the B{k} direction\n xk = problem.M.retr(x, B{k}, stepsize);\n % Evaluate the cost there\n fxk = getCost(problem, xk, storedb);\n % Finite difference\n df(k) = (fxk - fx)/stepsize;\n end\n \n % Build the gradient approximation.\n gradfd = lincomb(problem.M, x, B, df);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getHessian.m", "content": "function hess = getHessian(problem, x, d, storedb, key)\n% Computes the Hessian of the cost function at x along d.\n%\n% function hess = getHessian(problem, x, d)\n% function hess = getHessian(problem, x, d, storedb)\n% function hess = getHessian(problem, x, d, storedb, key)\n%\n% Returns the Hessian at x along d of the cost function described in the\n% problem structure.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% If an exact Hessian is not provided, an approximate Hessian is returned\n% if possible, without warning. If not possible, an exception will be\n% thrown. To check whether an exact Hessian is available or not (typically\n% to issue a warning if not), use canGetHessian.\n%\n% See also: getPrecon getApproxHessian canGetHessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n \n \n if isfield(problem, 'hess')\n %% Compute the Hessian using hess.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.hess)\n case 2\n hess = problem.hess(x, d);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [hess, store] = problem.hess(x, d, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n hess = problem.hess(x, d, storedb, key);\n otherwise\n up = MException('manopt:getHessian:badhess', ...\n 'hess should accept 2, 3 or 4 inputs.');\n throw(up);\n end\n \n elseif isfield(problem, 'ehess') && canGetEuclideanGradient(problem)\n %% Compute the Hessian using ehess.\n \n % We will need the Euclidean gradient for the conversion from the\n % Euclidean Hessian to the Riemannian Hessian.\n egrad = getEuclideanGradient(problem, x, storedb, key);\n\t\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.ehess)\n case 2\n ehess = problem.ehess(x, d);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [ehess, store] = problem.ehess(x, d, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n ehess = problem.ehess(x, d, storedb, key);\n otherwise\n up = MException('manopt:getHessian:badehess', ...\n 'ehess should accept 2, 3 or 4 inputs.');\n throw(up);\n end\n \n % Convert to the Riemannian Hessian\n hess = problem.M.ehess2rhess(x, egrad, ehess, d);\n \n else\n %% Attempt the computation of an approximation of the Hessian.\n \n hess = getApproxHessian(problem, x, d, storedb, key);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getHessianFD.m", "content": "function hessfd = getHessianFD(problem, x, d, storedb, key)\n% Computes an approx. of the Hessian w/ finite differences of the gradient.\n%\n% function hessfd = getHessianFD(problem, x, d)\n% function hessfd = getHessianFD(problem, x, d, storedb)\n% function hessfd = getHessianFD(problem, x, d, storedb, key)\n%\n% Returns a finite difference approximation of the Hessian at x along d of\n% the cost function described in the problem structure. The finite\n% difference is based on computations of the gradient.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% If the gradient cannot be computed, an exception is thrown.\n%\n% See also: approxhessianFD\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% Feb. 19, 2015 (NB):\n% It is sufficient to ensure positive radial linearity to guarantee\n% (together with other assumptions) that this approximation of the\n% Hessian will confer global convergence to the trust-regions method.\n% Formerly, in-code comments referred to the necessity of having\n% complete radial linearity, and that this was harder to achieve.\n% This appears not to be necessary after all, which simplifies the\n% code.\n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% Nov. 1, 2016 (NB):\n% Removed exception in case of unavailable gradient, as getGradient\n% now knows to fall back to an approximate gradient if need be.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n\t% Step size\n norm_d = problem.M.norm(x, d);\n \n % First, check whether the step d is not too small\n if norm_d < eps\n hessfd = problem.M.zerovec(x);\n return;\n end\n \n % Parameter: how far do we look?\n % If you need to change this parameter, use approxhessianFD explicitly.\n % A power of 2 is chosen so that scaling by epsilon does not incur any\n % round-off error in IEEE arithmetic.\n epsilon = 2^-14;\n \n c = epsilon/norm_d;\n \n % Compute the gradient at the current point.\n grad = getGradient(problem, x, storedb, key);\n \n % Compute a point a little further along d and the gradient there.\n % Since this is a new point, we need a new key for it, for the storedb.\n x1 = problem.M.retr(x, d, c);\n key1 = storedb.getNewKey();\n grad1 = getGradient(problem, x1, storedb, key1);\n \n % Transport grad1 back from x1 to x.\n grad1 = problem.M.transp(x1, x, grad1);\n \n % Return the finite difference of them.\n hessfd = problem.M.lincomb(x, 1/c, grad1, -1/c, grad);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getLinesearch.m", "content": "function t = getLinesearch(problem, x, d, storedb, key)\n% Returns a hint for line-search algorithms.\n%\n% function t = getLinesearch(problem, x, d)\n% function t = getLinesearch(problem, x, d, storedb)\n% function t = getLinesearch(problem, x, d, storedb, key)\n%\n% For a line-search problem at x along the tangent direction d, computes\n% and returns t such that retracting t*d at x yields a good point around\n% where to look for a line-search solution. That is: t is a hint as to\n% \"how far to look\" along the line.\n% \n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: canGetLinesearch\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 17, 2014.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n\n if isfield(problem, 'linesearch')\n %% Compute the line-search hint function using linesearch.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.linesearch)\n case 2\n t = problem.linesearch(x, d);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [t, store] = problem.linesearch(x, d, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n t = problem.linesearch(x, d, storedb, key);\n otherwise\n up = MException('manopt:getLinesearch:badfun', ...\n 'linesearch should accept 2, 3 or 4 inputs.');\n throw(up);\n end\n\n else\n %% Abandon computing the line-search function.\n\n up = MException('manopt:getLinesearch:fail', ...\n ['The problem description is not explicit enough to ' ...\n 'compute a line-search hint.']);\n throw(up);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getPartialEuclideanGradient.m", "content": "function egrad = getPartialEuclideanGradient(problem, x, I, storedb, key)\n% Computes the Euclidean gradient of a subset of terms in cost function.\n%\n% function egrad = getPartialEuclideanGradient(problem, x, I)\n% function egrad = getPartialEuclideanGradient(problem, x, I, storedb)\n% function egrad = getPartialEuclideanGradient(problem, x, I, storedb, key)\n%\n% Assume the cost function described in the problem structure is a sum of\n% many terms, as\n%\n% f(x) = sum_i f_i(x) for i = 1:d,\n\n% where d is specified as d = problem.ncostterms.\n% \n% For a subset I of 1:d, getPartialEuclideanGradient obtains the Euclidean\n% gradient of the partial cost function\n% \n% f_I(x) = sum_i f_i(x) for i = I.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: getGradient canGetPartialEuclidean Gradient getPartialGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 28, 2016\n% Contributors: \n% Change log: \n\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n \n % Make sure I is a row vector, so that it is natural to loop over it\n % with \" for i = I \".\n I = (I(:)).';\n \n \n if isfield(problem, 'partialegrad')\n %% Compute the partial Euclidean gradient using partialegrad.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.partialegrad)\n case 2\n egrad = problem.partialegrad(x, I);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [egrad, store] = problem.partialegrad(x, I, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n egrad = problem.partialegrad(x, I, storedb, key);\n otherwise\n up = MException('manopt:getPartialEuclideanGradient:badpartialegrad', ...\n 'partialegrad should accept 2, 3 or 4 inputs.');\n throw(up);\n end\n \n else\n %% Abandon computing the partial Euclidean gradient.\n \n up = MException('manopt:getPartialEuclideanGradient:fail', ...\n ['The problem description is not explicit enough to ' ...\n 'compute the partial Euclidean gradient of the cost.']);\n throw(up);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getPartialGradient.m", "content": "function grad = getPartialGradient(problem, x, I, storedb, key)\n% Computes the gradient of a subset of terms in the cost function at x.\n%\n% function grad = getPartialGradient(problem, x, I)\n% function grad = getPartialGradient(problem, x, I, storedb)\n% function grad = getPartialGradient(problem, x, I, storedb, key)\n%\n% Assume the cost function described in the problem structure is a sum of\n% many terms, as\n%\n% f(x) = sum_i f_i(x) for i = 1:d,\n\n% where d is specified as d = problem.ncostterms.\n% \n% For a subset I of 1:d, getPartialGradient obtains the gradient of the\n% partial cost function\n% \n% f_I(x) = sum_i f_i(x) for i = I.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: getGradient canGetPartialGradient getPartialEuclideanGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 28, 2016\n% Contributors: \n% Change log: \n\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n \n \n % Make sure I is a row vector, so that it is natural to loop over it\n % with \" for i = I \".\n I = (I(:)).';\n\n \n if isfield(problem, 'partialgrad')\n %% Compute the partial gradient using partialgrad.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.partialgrad)\n case 2\n grad = problem.partialgrad(x, I);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [grad, store] = problem.partialgrad(x, I, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n grad = problem.partialgrad(x, I, storedb, key);\n otherwise\n up = MException('manopt:getPartialGradient:badpartialgrad', ...\n 'partialgrad should accept 2, 3 or 4 inputs.');\n throw(up);\n end\n \n elseif canGetPartialEuclideanGradient(problem)\n %% Compute the partial gradient using the Euclidean partial gradient.\n \n egrad = getPartialEuclideanGradient(problem, x, I, storedb, key);\n grad = problem.M.egrad2rgrad(x, egrad);\n\n else\n %% Abandon computing the partial gradient.\n \n up = MException('manopt:getPartialGradient:fail', ...\n ['The problem description is not explicit enough to ' ...\n 'compute the partial gradient of the cost.']);\n throw(up);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getPrecon.m", "content": "function Pd = getPrecon(problem, x, d, storedb, key)\n% Applies the preconditioner for the Hessian of the cost at x along d.\n%\n% function Pd = getPrecon(problem, x, d)\n% function Pd = getPrecon(problem, x, d, storedb)\n% function Pd = getPrecon(problem, x, d, storedb, key)\n%\n% Returns as Pd the result of applying the Hessian preconditioner to the\n% tangent vector d at point x. The preconditioner is supposed to be a\n% symmetric, positive definite approximation of the inverse of the Hessian.\n% \n% If no preconditioner is available, Pd = d (identity).\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: getHessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n \n if isfield(problem, 'precon')\n %% Precondition using precon.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.precon)\n case 2\n Pd = problem.precon(x, d);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [Pd, store] = problem.precon(x, d, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n Pd = problem.precon(x, d, storedb, key);\n otherwise\n up = MException('manopt:getPrecon:badprecon', ...\n 'precon should accept 2, 3 or 4 inputs.');\n throw(up);\n end \n\n elseif canGetSqrtPrecon(problem)\n %% Precondition by applying the square root of the preconditioner twice.\n \n sqrtPd = getSqrtPrecon(problem, x, d, storedb, key);\n Pd = getSqrtPrecon(problem, x, sqrtPd, storedb, key);\n \n else\n %% No preconditioner provided, so just use the identity.\n \n Pd = d;\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getSqrtPrecon.m", "content": "function sqrtPd = getSqrtPrecon(problem, x, d, storedb, key)\n% Applies the square root of the Hessian preconditioner at x along d.\n%\n% function sqrtPd = getSqrtPrecon(problem, x, d)\n% function sqrtPd = getSqrtPrecon(problem, x, d, storedb)\n% function sqrtPd = getSqrtPrecon(problem, x, d, storedb, key)\n%\n% Returns as sqrtPd the result of applying the square root of the Hessian\n% preconditioner to the tangent vector d at point x. The preconditioner is\n% supposed to be a symmetric, positive definite approximation of the\n% inverse of the Hessian. Its square root must thus be symmetric and\n% positive definite itself.\n% \n% If no square root of preconditioner is available, sqrtPd = d (identity).\n% Note that this may be incompatible with the preconditioner, if that one\n% is supplied in the problem description. Always check with canGetPrecon\n% and canGetSqrtPrecon.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: getPrecon canGetPrecon canGetSqrtPrecon getHessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 3, 2015.\n% Contributors: \n% Change log: \n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n \n if isfield(problem, 'sqrtprecon')\n %% Apply sqrtprecon for the square root of the preconditioner\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.sqrtprecon)\n case 2\n sqrtPd = problem.sqrtprecon(x, d);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [sqrtPd, store] = problem.sqrtprecon(x, d, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n sqrtPd = problem.sqrtprecon(x, d, storedb, key);\n otherwise\n up = MException('manopt:getSqrtPrecon:badsqrtprecon', ...\n 'sqrtprecon should accept 2, 3 or 4 inputs.');\n throw(up);\n end\n \n else\n %% No preconditioner square root provided, so just use the identity.\n \n sqrtPd = d;\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getStore.m", "content": "function store = getStore(problem, x, storedb) %#ok\n\n error('This file was removed from Manopt. Please use the StoreDB class.');\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/getSubgradient.m", "content": "function subgrad = getSubgradient(problem, x, tol, storedb, key)\n% Computes a subgradient of the cost function at x, up to a tolerance\n%\n% function subgrad = getSubgradient(problem, x)\n% function subgrad = getSubgradient(problem, x, tol)\n% function subgrad = getSubgradient(problem, x, tol, storedb)\n% function subgrad = getSubgradient(problem, x, tol, storedb, key)\n%\n% Returns a subgradient at x of the cost function described in the problem\n% structure. A tolerance tol ( >= 0 ) can also be specified. By default,\n% tol = 0.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% See also: getDirectionalDerivative canGetGradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 20, 2017.\n% Contributors: \n% Change log: \n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n \n % Default tolerance is 0\n if ~exist('tol', 'var') || isempty(tol)\n tol = 0;\n end\n\n \n if isfield(problem, 'subgrad')\n %% Compute a subgradient using subgrad.\n\t\n % Check whether this function wants to deal with storedb or not.\n switch nargin(problem.subgrad)\n case 1\n warning('manopt:subgradient', ...\n ['problem.subgrad normally admits a second\\n' ...\n 'parameter, tol >= 0, as a tolerance.\\n']);\n subgrad = problem.subgrad(x); % tol is not passed here\n case 2\n subgrad = problem.subgrad(x, tol);\n case 3\n % Obtain, pass along, and save the store for x.\n store = storedb.getWithShared(key);\n [subgrad, store] = problem.subgrad(x, tol, store);\n storedb.setWithShared(store, key);\n case 4\n % Pass along the whole storedb (by reference), with key.\n subgrad = problem.subgrad(x, tol, storedb, key);\n otherwise\n up = MException('manopt:getSubgradient:badsubgrad', ...\n 'subgrad should accept 1, 2, 3 or 4 inputs.');\n throw(up);\n end\n \n elseif canGetGradient(problem)\n %% The gradient is a subgradient.\n \n subgrad = getGradient(problem, x, storedb, key);\n \n else\n %% Abandon\n \n up = MException('manopt:getSubgradient:fail', ...\n ['The problem description is not explicit enough to ' ...\n 'compute a subgradient.']);\n throw(up);\n \n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/handle_light.m", "content": "classdef handle_light < handle\n% Trick class to hide methods inherited from the handle class\n% when calling methods(myclass).\n%\n% Source:\n% http://stackoverflow.com/questions/6621850/is-it-possible-to-hide-the-methods-inherited-from-the-handle-class-in-matlab\n% Posted by sclarke81 on StackOverflow on Oct. 24, 2012.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: sclarke81, added April 3, 2013.\n% Contributors: \n% Change log: \n\n methods(Hidden)\n function lh = addlistener(varargin)\n lh = addlistener@handle(varargin{:});\n end\n function notify(varargin)\n notify@handle(varargin{:});\n end\n function delete(varargin)\n delete@handle(varargin{:});\n end\n function Hmatch = findobj(varargin)\n Hmatch = findobj@handle(varargin{:});\n end\n function p = findprop(varargin)\n p = findprop@handle(varargin{:});\n end\n function TF = eq(varargin)\n TF = eq@handle(varargin{:});\n end\n function TF = ne(varargin)\n TF = ne@handle(varargin{:});\n end\n function TF = lt(varargin)\n TF = lt@handle(varargin{:});\n end\n function TF = le(varargin)\n TF = le@handle(varargin{:});\n end\n function TF = gt(varargin)\n TF = gt@handle(varargin{:});\n end\n function TF = ge(varargin)\n TF = ge@handle(varargin{:});\n end\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/mergeOptions.m", "content": "function opts = mergeOptions(opts1, opts2)\n% Merges two options structures with one having precedence over the other.\n%\n% function opts = mergeOptions(opts1, opts2)\n%\n% input: opts1 and opts2 are two structures.\n% output: opts is a structure containing all fields of opts1 and opts2.\n% Whenever a field is present in both opts1 and opts2, it is the value in\n% opts2 that is kept.\n%\n% The typical usage is to have opts1 contain default options and opts2\n% contain user-specified options that overwrite the defaults.\n%\n% See also: getGlobalDefaults\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n\n\n if isempty(opts1)\n opts1 = struct();\n end\n if isempty(opts2)\n opts2 = struct();\n end\n\n opts = opts1;\n fields = fieldnames(opts2);\n for i = 1 : length(fields)\n opts.(fields{i}) = opts2.(fields{i});\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/purgeStoredb.m", "content": "function storedb = purgeStoredb(storedb, storedepth) %#ok\n\n error('This file was removed from Manopt. Please use the StoreDB class.');\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/setStore.m", "content": "function storedb = setStore(problem, x, storedb, store) %#ok\n\n error('This file was removed from Manopt. Please use the StoreDB class.');\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/core/stoppingcriterion.m", "content": "function [stop, reason] = stoppingcriterion(problem, x, options, info, last)\n% Checks for standard stopping criteria, as a helper to solvers.\n%\n% function [stop, reason] = stoppingcriterion(problem, x, options, info, last)\n%\n% Executes standard stopping criterion checks, based on what is defined in\n% the info(last) stats structure and in the options structure.\n%\n% The returned number 'stop' is 0 if none of the stopping criteria\n% triggered, and a (strictly) positive integer otherwise. The integer\n% identifies which criterion triggered:\n% 0 : Nothing triggered;\n% 1 : Cost tolerance reached;\n% 2 : Gradient norm tolerance reached;\n% 3 : Max time exceeded;\n% 4 : Max iteration count reached;\n% 5 : Maximum number of cost evaluations reached;\n% 6 : User defined stopfun criterion triggered.\n%\n% The output 'reason' is a string describing the triggered event.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 2, 2015 (NB):\n% 'reason' now contains the option (name and value) that triggered.\n\n\n stop = 0;\n reason = '';\n \n stats = info(last);\n\n % Target cost attained\n if isfield(stats, 'cost') && isfield(options, 'tolcost') && ...\n stats.cost <= options.tolcost\n reason = sprintf('Cost tolerance reached; options.tolcost = %g.', options.tolcost);\n stop = 1;\n return;\n end\n\n % Target gradient norm attained\n if isfield(stats, 'gradnorm') && isfield(options, 'tolgradnorm') && ...\n stats.gradnorm < options.tolgradnorm\n reason = sprintf('Gradient norm tolerance reached; options.tolgradnorm = %g.', options.tolgradnorm);\n stop = 2;\n return;\n end\n\n % Allotted time exceeded\n if isfield(stats, 'time') && isfield(options, 'maxtime') && ...\n stats.time >= options.maxtime\n reason = sprintf('Max time exceeded; options.maxtime = %g.', options.maxtime);\n stop = 3;\n return;\n end\n\n % Allotted iteration count exceeded\n if isfield(stats, 'iter') && isfield(options, 'maxiter') && ...\n stats.iter >= options.maxiter\n reason = sprintf('Max iteration count reached; options.maxiter = %g.', options.maxiter);\n stop = 4;\n return;\n end\n \n % Allotted function evaluation count exceeded\n if isfield(stats, 'costevals') && isfield(options, 'maxcostevals') && ...\n stats.costevals >= options.maxcostevals\n reason = sprintf('Maximum number of cost evaluations reached; options.maxcostevals = %g.', options.maxcostevals);\n stop = 5;\n end\n\n % Check whether the possibly user defined stopping criterion\n % triggers or not.\n if isfield(options, 'stopfun')\n userstop = options.stopfun(problem, x, info, last);\n if userstop\n reason = 'User defined stopfun criterion triggered; see options.stopfun.';\n stop = 6;\n return;\n end\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/complexcircle/complexcirclefactory.m", "content": "function M = complexcirclefactory(n)\n% Returns a manifold struct to optimize over unit-modulus complex numbers.\n%\n% function M = complexcirclefactory()\n% function M = complexcirclefactory(n)\n%\n% Description of vectors z in C^n (complex) such that each component z(i)\n% has unit modulus. The manifold structure is the Riemannian submanifold\n% structure from the embedding space R^2 x ... x R^2, i.e., the complex\n% circle is identified with the unit circle in the real plane.\n%\n% By default, n = 1.\n%\n% See also spherecomplexfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% July 7, 2014 (NB): Added ehess2rhess function.\n%\n% Sep. 3, 2014 (NB): Correction to the dist function (extract real part).\n%\n% April 13, 2015 (NB): Fixed logarithm.\n%\n% Oct. 8, 2016 (NB)\n% Code for exponential was simplified to only treat the zero vector\n% as a particular case.\n%\n% July 20, 2017 (NB)\n% The distance function is now even more accurate. Improved logarithm\n% accordingly.\n \n if ~exist('n', 'var')\n n = 1;\n end\n\n M.name = @() sprintf('Complex circle (S^1)^%d', n);\n \n M.dim = @() n;\n \n M.inner = @(z, v, w) real(v'*w);\n \n M.norm = @(x, v) norm(v);\n \n M.dist = @(x, y) norm(real(2*asin(.5*abs(x - y))));\n \n M.typicaldist = @() pi*sqrt(n);\n \n M.proj = @(z, u) u - real( conj(u) .* z ) .* z;\t\n \n M.tangent = M.proj;\n \n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(z, egrad, ehess, zdot)\n rhess = M.proj(z, ehess - real(z.*conj(egrad)).*zdot);\n end\n \n M.exp = @exponential;\n function y = exponential(z, v, t)\n \n if nargin == 2\n % t = 1;\n tv = v;\n else\n tv = t*v;\n end\n\n y = zeros(n, 1);\n\n nrm_tv = abs(tv);\n \n % We need to be careful for zero steps.\n mask = (nrm_tv > 0);\n y(mask) = z(mask).*cos(nrm_tv(mask)) + ...\n tv(mask).*(sin(nrm_tv(mask))./nrm_tv(mask));\n y(~mask) = z(~mask);\n \n end\n \n M.retr = @retraction;\n function y = retraction(z, v, t)\n if nargin == 2\n % t = 1;\n tv = v;\n else\n tv = t*v;\n end\n y = sign(z+tv);\n end\n\n M.log = @logarithm;\n function v = logarithm(x1, x2)\n v = M.proj(x1, x2 - x1);\n di = real(2*asin(.5*abs(x1 - x2)));\n nv = abs(v);\n factors = di ./ nv;\n factors(di <= 1e-10) = 1;\n\t\tv = v .* factors;\n end\n \n M.hash = @(z) ['z' hashmd5( [real(z(:)) ; imag(z(:))] ) ];\n \n M.rand = @random;\n function z = random()\n z = sign(randn(n, 1) + 1i*randn(n, 1));\n end\n \n M.randvec = @randomvec;\n function v = randomvec(z)\n % i*z(k) is a basis vector of the tangent vector to the k-th circle\n v = randn(n, 1) .* (1i*z);\n v = v / norm(v);\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, 1);\n \n M.transp = @(x1, x2, d) M.proj(x2, d);\n \n M.pairmean = @pairmean;\n function z = pairmean(z1, z2)\n z = sign(z1+z2);\n end\n\n M.vec = @(x, u_mat) [real(u_mat) ; imag(u_mat)];\n M.mat = @(x, u_vec) u_vec(1:n) + 1i*u_vec((n+1):end);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/complexcircle/realphasefactory.m", "content": "function M = realphasefactory(n, z0, zmax)\n% Returns a manifold struct to optimize over phases of fft's of real signals\n%\n% function M = realphasefactory(n)\n% function M = realphasefactory(n, z0)\n% function M = realphasefactory(n, z0, zmax)\n%\n% If x is a real vector of length n, then y = fft(x) is a complex vector\n% which obeys certain symmetries. Specifically, for any integer k,\n%\n% y(1+mod(k, n)) = conj(y(1+mod(n-k, n)))\n%\n% The same holds for the phases of the Fourier transform z = sign(y).\n%\n% This factory returns a Manopt manifold structure which represents the set\n% of complex vectors z of length n which could be the phases of the Fourier\n% transform of a real signal of length n:\n%\n% abs(z) = 1 and z(1+mod(k, n)) = conj(z(1+mod(n-k, n))) for each k.\n%\n% For k = 1, this readily implies that z(1) is +1 or -1, so that the set of\n% possible z's is disconnected. To choose which connected component to work\n% with, set the second input z0 to +1 or -1 (this is the sign of the mean\n% of x). By default, z0 = 1.\n%\n% Furthermore, if n is even, then k = n/2 implies z(1+n/2) is +1 or -1 as\n% well, thus further disconnecting the set of acceptable z's. To choose\n% which component to work with, set the third input zmax to +1 or -1. By\n% default, it is +1.\n%\n% The Riemannian manifold structure is the Riemannian submanifold\n% structure from the embedding space R^2 x ... x R^2, i.e., the complex\n% circles are identified with the unit circle in the real plane.\n% Concretely, this means the inner product is _z = real(u'*v).\n% Tangent vectors at z are complex vectors of length n which notably\n% satisfy z(1+0) = 0 and, if n is even, z(1+n/2) = 0.\n%\n% n must be integer and n >= 3 (for n = 1:2 the manifold has dimension 0).\n%\n% Extra functions available in M include M.up, M.down and M.downup. They\n% allow to capture the symmetries concisely, as:\n%\n% M.up(z) == conj(M.down(z)).\n%\n% See in code for more details.\n%\n% See also complexcirclefactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Feb. 2, 2017.\n% Contributors: joint work with Tamir Bendory, Zhizhen Zhao and Amit Singer\n% Change log: \n%\n% July 20, 2017 (NB)\n% The distance function is now more accurate. Improved logarithm\n% accordingly.\n\n assert(n == round(n) && n >= 3, 'n must be an integer >= 3.');\n \n even_n = (round(n/2) == n/2);\n \n if ~exist('z0', 'var') || isempty(z0)\n z0 = 1;\n end\n if ~exist('zmax', 'var') || isempty(zmax)\n zmax = 1;\n end\n \n assert(z0 == 1 || z0 == -1, 'z0 must be +1 or -1.');\n assert(zmax == 1 || zmax == -1, 'zmax must be +1 or -1.');\n\n if even_n\n M.name = @() sprintf('Phases of fft''s of real signals of length %d (z0 = %d, zmax = %d)', n, z0, zmax);\n else\n M.name = @() sprintf('Phases of fft''s of real signals of length %d (z0 = %d)', n, z0);\n end\n \n M.dim = @() floor((n-1)/2);\n \n M.inner = @(z, v, w) real(v'*w);\n \n M.norm = @(z, u) norm(u);\n \n M.dist = @(z1, z2) norm(real(2*asin(.5*abs(z1 - z2))));\n \n M.typicaldist = @() pi*sqrt(n/2);\n \n % Special functions to ease working with the symmetries.\n down = @(u) u;\n up = @(u) u([1 ; (n:-1:2)']);\n downup = @(u) (down(u) + conj(up(u)))/2;\n M.down = down;\n M.up = up;\n M.downup = downup;\n \n M.proj = @proj;\n function pu = proj(z, u)\n duu = downup(u);\n pu = duu - real(duu .* conj(z)).*z;\n % Note that there is no need to enforce pu(1) = 0 or (if n is even)\n % pu(1+n/2) = 0 manually, since the IEEE standard ensures that the\n % above operation will be exact for those entries provided z(1)\n % (and possibly z(1+n/2) is +1 or -1, as should be the case.\n end\n \n M.tangent = M.proj;\n \n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(z, egrad, ehess, zdot)\n rhess = M.proj(z, ehess - real(downup(egrad) .* conj(z)).*zdot);\n end\n \n M.exp = @exponential;\n function y = exponential(z, v, t)\n \n if nargin == 2\n % t = 1;\n tv = v;\n else\n tv = t*v;\n end\n\n y = zeros(n, 1);\n\n nrm_tv = abs(tv);\n \n % We need to be careful for zero steps.\n mask = (nrm_tv > 0);\n y(mask) = z(mask).*cos(nrm_tv(mask)) + ...\n tv(mask).*(sin(nrm_tv(mask))./nrm_tv(mask));\n y(~mask) = z(~mask);\n \n end\n \n M.retr = @retraction;\n function y = retraction(z, v, t)\n if nargin == 2\n % t = 1;\n tv = v;\n else\n tv = t*v;\n end\n y = sign(z+tv);\n end\n\n M.log = @logarithm;\n function v = logarithm(x1, x2)\n v = M.proj(x1, x2 - x1);\n di = real(2*asin(.5*abs(x1 - x2)));\n nv = abs(v);\n factors = di ./ nv;\n factors(di <= 1e-6) = 1;\n\t\tv = v .* factors;\n end\n \n M.hash = @(z) ['z' hashmd5( [real(z(:)) ; imag(z(:))] ) ];\n \n M.rand = @random;\n function z = random()\n z = sign(downup(randn(n, 1) + 1i*randn(n, 1)));\n z(1) = z0;\n if even_n\n z(1 + n/2) = zmax;\n end\n end\n \n M.randvec = @randomvec;\n function v = randomvec(z)\n v = M.proj(z, randn(n, 1) + 1i*randn(n, 1));\n v = v / norm(v);\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(z) zeros(n, 1);\n \n M.transp = @(z1, z2, u) M.proj(z2, u);\n \n M.pairmean = @pairmean;\n function z = pairmean(z1, z2)\n z = sign(z1+z2);\n end\n\n % This vec/mat pair is an isometry which allows to switch between the\n % classical representation of tangent vectors---as complex vectors of\n % length n---to real vectors of length M.dim() whose entries are the\n % coordinates of the tangent vector in the basis 1i*z, for the first\n % half. A scaling of sqrt(2) is applied to ensure isometry, since\n % tangent vectors are represented with only half of their entries.\n I = 2 : floor((n+1)/2);\n if even_n\n middle = 0;\n else\n middle = [];\n end\n M.vec = @(z, u_mat) sqrt(2)*real(u_mat(I) .* conj(1i*z(I)));\n M.mat = @(z, u_vec) [0 ; u_vec.*(1i*z(I)) ; middle ; ...\n flipud(conj(u_vec.*(1i*z(I))))]/sqrt(2);\n M.vecmatareisometries = @() true;\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/README_Essential.txt", "content": "# A Riemannian quotient representation for the essential manifold\nContributed by Roberto Tron.\n\nThe essential matrix is a 3x3 matrix that encodes the epipolar\nconstraint between the homogeneous coordinates of the projection of a\ncommon 3-D point in two cameras. Not all the 3x3 matrices are essential matrices, as\nthese need to encode the relative pose of the two cameras (up to a\nglobal scaling). The space of valid essential matrices can be endowed\nwith a Riemannian structure by following the derivations presented in:\n\n R. Tron, K. Daniilidis,\n \"The Space of Essential Matrices as a Riemannian Quotient Manifold -\n Geometric Interpretation and Optimization Algorithms\" \n International Journal of Computer Vision, (submitted).\n\nThis work shows that the essential manifold can be seen as a quotient\nmanifold of $SO(3) \\times SO(3)$, where $SO(3)$ is the manifold of 3-D\nrotations. In Matlab, we represents k points on the essential\nmanifold as array of dimension $[3 \\times 6 \\times k]$, where each $[3 \\times 3]$\nsub matrix is a 3-D rotation.\n\nThe implementation provides both the \"signed\" and \"unsigned\" version\nof the manifold presented in the paper. The only difference between\nthe two is in how the logarithm, and hence the distance, are\ncomputed. In the signed version, the points related by the twisted pair\nambiguity are considered as distinct; in practice, this is the case when the\ncheirality constraint is used to remove the ambiguity. In the unsigned\nversion, points related by the twisted pair ambiguity belong to the\nsame class; in practice, it means that they all produce equivalent\nepipolar constraints. See the paper for details.\n\nFactory call:\nM=essentialfactory(k,signature).\nBy default, k equals 1. The string signature should be set to \"signed\"\n(resp. \"unsigned\") to use the signed (resp. unsigned) version of the\nmanifold. By default, signature equals \"signed\".\n\nSee the paper for the definition of the set and tangent spaces.\n\nNote: following the representation of tangent vectors for SO(3) in\nMANOPT, tangent vectors for the essential manifold are represented as\n$[3 \\times 6 \\times k]$ matrices, where each $[3 \\times 3]$\nsub matrix is skew-symmetric. The real tangent vector in the ambient\nspace is obtained by multiplying on the left each $[3 \\times 3]$\nskew-symmetric matrix with the corresponding rotation from the base\npoint.\n\n## Toolset\nThe following list contains some of the nontrivial available functions\nin the structure M.\n\n- Dimension\nM.dim()\n$\\dim M=5k$\n\n- Metric\nM.inner(X,S,T)\n$\\langle U, V \\rangle = \\sum_{i=1}^k trace(S_i^TT_i)$, where S and T\nare representation of two tangent vectors at X.\n\n- Norm\nM.norm(X,S)\n$\\norm{U}=\\sqrt{\\langle U, U \\rangle}$\n\n- Distance\nM.dist(X,Y)\n$\\dist(X,Y)=\\sqrt(\\sum_{i=1}^k \\norm{\\log(X_i,Y_i)}$, see M.log(X,Y)\nbelow\n\n- Typical distance\nM.typicaldist()\n\\pi\\sqrt{k}\n\n- Vertical tangent space projector\nM.vertproj(X,H)\nProjects a point in the ambient space onto the vertical space at\nX. See the paper for details. Note that this operation returns an\narray containing skew-symmetric matrices.\n\n- Tangent space projector\nM.proj(X,H)\nProjects a point in the ambient space onto the horizontal space at\nX. See the paper for details. Note that this operation returns an\narray containing skew-symmetric matrices.\n\n- Tangent space to ambient space\nM.tangent2ambient(X,S)\nComputes a matrix H where H(1:3,:,i)=X(1:3,:,i)*S(1:3,:,i) and\nH(4:6,:,i)=X(4:6,:,i)*S(4:6,:,i). This function is necessary because\nthe proj operator takes as input an ambient vector and returns a\ntangent vector. To apply the proj again to the result (which should\nchange nothing), it is necessary to first represent the tangent vector\nobtained as an ambient vector. This function is here because of formal\npeculiarities and is likely to disappear at some point. \n\n- Essential matrix\nM.E(X)\nReturns the 3\\times 3 essential matrix corresponding to the point on\nthe manifold X.\n\n- Tangent of the essential matrix\nM.dE(X,S)\nReturns the matrix $\\dot{E}$ obtained from a point X moving on a curve\nwith tangent S. Mathematically, this is the push-forward of S through\nthe mapping M.E(X)\n\n- Double tangent of the essential matrix\nM.ddE(X,S)\nReturns the matrix $\\ddot{E}$ obtained from a point X moving on a\n*geodesic* curve (i.e., with zero acceleration) with tangent S. \nMathematically, this is the push-forward of S through the mapping M.dE(X,S)\n\n- Euclidean to Riemannian function\nM.ef2rf(X,ef)\nReturns the value of ef evaluated at M.E(X). ef must be a function handle\n\n- Euclidean gradient of a function of E to Euclidean gradient of a function of X\nM.egradE2egrad(X,egradE)\nReturns the Euclidean gradient (matrix of partial derivatives) in the\nentries of X (taken as a $3 \\times 6$ matrix) given the Euclidean gradient\nof a function of E (which is a $3 \\times 3$ matrix). egrad must be\na function handle for which egrad(E) returns the $3 \\times 3$\nEuclidean gradient of a function evaluated at the essential matrix E=M.E(X)\nNote: this function uses a different convention than egrad2rgrad for\nother manifolds. In this case egradE is a function handle, while in the\nother cases egrad is a matrix.\n\n- Euclidean to Riemannian gradient\nM.egrad2rgrad(X,egrad)\nReturns the Riemannian gradient (a tangent vector at X) corresponding\nto the Euclidean gradient of a function of X taken as a matrix. egrad must be\na function handle for which egrad(X) returns the $3 \\times 6$\nEuclidean gradient of a function evaluated at the point X.\nNote: this function uses a different convention than egrad2rgrad for\nother manifolds. In this case egrad is a function handle, while in the\nother cases egrad is a matrix.\n\n- Euclidean gradient of a function of E to Riemannian gradient\nM.egradE2rgrad(X,egradE)\nThis function is the combination of M.egradE2egrad and\nM.egrad2rgrad. See the respective comments for more information.\n\n- Euclidean Hessian of a function of E to to Euclidean Hessian of a function of X \nM.ehessE2ehess(X,egradE, ehessE, V)\nReturns the Euclidean Hessian (operator given by second order partial\nderivatives) in the entries of X (taken as a $3 \\times 6$ matrix)\nevaluated in the direction V (which represents a direction in the\nambient space) given the Euclidean Hessian operator of a function of E\n(which is a $3 \\times 3$ matrix). ehessE must be a function handle for which\negrad(E,dE) returns the $3 \\times 3$ Euclidean Hessian of a function\nevaluated at the essential matrix E for the tangent vector dE. See\nalso M.egradE2egrad.\n\n- Euclidean to Riemannian Hessian \nM.ehessE2rhess(X,egrad, ehess, V)\nThis function is the combination of M.ehessE2ehess and\nM.ehess2rhess. See the respective comments for more information.\n\n- Exponential map\nM.exp(X,S,t)\nReturns the point obtained by following the normal geodesic starting from X\nwith tangent S for a length t. This function does not check that S is\nhorizontal: it simply applies the exponential map on each copy of\nSO(3)\n\n- Logarithm map\nM.log(X,Y)\nThe inverse of the exponential map. It is guaranteed to correspond to\nthe horizontal vector pointing in the direction of the shortest\ngeodesic from X to Y.\n\n- Transport\nM.transp(X1,X2,S1)\nTransport a vector from the tangent space of X1 to the tangent space\nof X2, using left translations in SO(3)^2. This transport preserves\nthe length of the vectors.\n\n- Distance\nM.dist(X,Y)\n$\\dist(X,Y)=\\|\\log(X,Y)\\|$\nCompute the shortest geodesic distance between X and Y. \n\n- Pair mean\nM.pairmean(X,Y)\nMid-point of the shortest geodesic between X and Y.\n\n\n## Example\n\nThe file essential_svd.m contains an example of the use of the\nessential manifold in MANOPT. It first builds random essential\nmatrices A_i, i=1,..,k. It then tries to find matrices E_i, i=1,...,k\nwhich minimize\n\n\\sum_{i=1}^k \\frac{1}{2}\\|E_i-A_i\\|^2.\n\nThe i-th component of the Euclidean gradient is simply E_i-A_i and the\nHessian operator is the identity.\n\nThis problem is trivial, as the cost function is separable in each i\nand the solution is simply E_i=A_i. However, this example shows\nhow to define the gradient and hessian of the cost function with k>1\nand shows that indeed the optimization procedure converges\nto the expected minimizer.\n\n## Files\nWith respect to a vanilla installation of MANOPT, the implementation\nof the essential manifold adds the following files and directories\n\nmanopt/manifolds/essential\nexamples/essential_svd.m\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/essential_costE2cost.m", "content": "function val = essential_costE2cost(X, costE)\n% Cost evaluation at X given function handle in the Essential matrix E.\n%\n% function val = essential_costE2cost(X, costE)\n%\n% costE is the function handle for the cost function in E.\n%\n% See also: essential_egradE2egrad essential_ehessE2ehess\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Roberto Tron, Aug. 8, 2014\n% Contributors: Bamdev Mishra, May 22, 2015.\n\n e3hat = [0 -1 0; 1 0 0; 0 0 0];\n \n RA = X(:,1:3,:); \n RB = X(:,4:6,:); \n E = multiprod(multiprod(multitransp(RA), e3hat), RB); \n \n val = costE(E);\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/essential_egradE2egrad.m", "content": "function egrad = essential_egradE2egrad(X, egradE)\n% Converts the gradient in essential matrix E to the gradient in X.\n%\n% function egrad = essential_egradE2egrad(X, egradE)\n%\n% egradE is the function handle for the gradient in E.\n% \n% The output is a matrix in the space of X.\n%\n% See also: essential_costE2cost essential_ehessE2ehess\n\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Roberto Tron, Aug. 8, 2014\n% Contributors: Bamdev Mishra, May 22, 2015.\n\n e3hat = [0 -1 0; 1 0 0; 0 0 0];\n RA = X(:,1:3,:); \n RB = X(:,4:6,:);\n E = multiprod(multiprod(multitransp(RA), e3hat), RB); \n G = egradE(E); \n \n %The following is the vectorized version of egrad = e3hat*[RB*G' -RA*G];\n egrad = multiprod(e3hat, cat(2,...\n multiprod(RB, multitransp(G)),...\n -multiprod(RA, G)));\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/essential_ehessE2ehess.m", "content": "function ehess = essential_ehessE2ehess(X, egradE, ehessE, S)\n% Converts the Hessian in essential matrix E to the Hessian in X.\n%\n% function ehess = essential_ehessE2ehess(X, egradE, ehessE, S)\n%\n% egradE is the function handle for the gradient in E.\n% ehessE is the function handle for the Hessian in E.\n% S is the search direction in the space of X.\n%\n% The output is a matrix in the space of X.\n%\n% See also: essential_costE2cost essential_egradE2egrad\n\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Roberto Tron, Aug. 8, 2014\n% Contributors: Bamdev Mishra, May 22, 2015.\n \n e3hat = [0 -1 0; 1 0 0; 0 0 0];\n \n RA = X(:,1:3,:); \n RB = X(:,4:6,:);\n E = multiprod(multiprod(multitransp(RA), e3hat), RB); % M.E(X);\n G = egradE(E); \n \n V = essential_sharp(multiprod(essential_flat(X), essential_flat(S)));\n VA = V(:,1:3,:);\n VB = V(:,4:6,:);\n \n dE = multiprod(multiprod(multitransp(RA), e3hat), VB)...\n + multiprod(multiprod(multitransp(VA), e3hat), RB);\n dG = ehessE(E, dE);\n \n %The following is the vectorized version of ehess = e3hat*[(VB*G'+RB*H') -(VA*G+RA*H)]\n ehess = multiprod(e3hat,cat(2,...\n multiprod(VB, multitransp(G)) + multiprod(RB, multitransp(dG)),...\n -multiprod(VA, G) - multiprod(RA, dG)));\n \nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/essential_flat.m", "content": "function Hp = essential_flat(H)\n %Reshape a [3x6xk] matrix to a [3x3x2k] matrix\n Hp = reshape(H,3,3,[]);\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/essential_hat3.m", "content": "%Compute the matrix representation of the cross product\n%function [V,vShift] = essential_hat3(v)\n%V is a [3x3xN] array of skew-symmetric matrices where each [3x3] block is\n%the matrix representation of the cross product of one of the columns of v\n%vShift is equal to permute(v,[1 3 2]).\nfunction [V, vShift] = essential_hat3(v)\n N = size(v,2);\n V = zeros(3,3,N);\n vShift = permute(v,[1 3 2]);\n V(1,2,:) = -vShift(3,:,:);\n V(2,1,:) = vShift(3,:,:);\n V(1,3,:) = vShift(2,:,:);\n V(3,1,:) = -vShift(2,:,:);\n V(2,3,:) = -vShift(1,:,:);\n V(3,2,:) = vShift(1,:,:);\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/essential_sharp.m", "content": "function H = essential_sharp(Hp)\n %Reshape a [3x3x2k] matrix to a [3x6xk] matrix\n H = reshape(Hp,3,6,[]);\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/essentialfactory.m", "content": "function M = essentialfactory(k, strSigned)\n% Manifold structure to optimize over the space of essential matrices.\n%\n% function M = essentialfactory(k)\n% function M = essentialfactory(k, 'signed')\n% function M = essentialfactory(k, 'unsigned')\n%\n%\n% Quotient representation of the essential manifold: deals with the\n% representation of the space of essential matrices M_rE. These are used in\n% computer vision to represent the epipolar constraint between projected\n% points in two perspective views.\n%\n% The space is represented as the quotient (SO(3)^2/SO(2)).\n% See the following references for details:\n%\n% R. Tron, K. Daniilidis,\n% \"On the quotient representation of the essential manifold\"\n% IEEE Conference on Computer Vision and Pattern Recognition, 2014\n%\n% For computational purposes, each essential matrix is represented as a\n% [3x6] matrix where each [3x3] block is a rotation.\n%\n% The metric used is the one induced by the submersion of M_rE in SO(3)^2.\n%\n% Tangent vectors are represented in the Lie algebra of SO(3)^2, i.e., as\n% [3x6] matrices where each [3x3] block is a skew-symmetric matrix.\n% Use the function tangent2ambient(X, H) to switch from the Lie algebra\n% representation to the embedding space representation in R^(3x6).\n%\n% By default, k = 1, and the geometry is 'signed'.\n%\n% Optional arguments:\n% \"signed\" selects the signed version of the manifold\n% \"unsigned\" selects the unsigned version of the manifold\n%\n% See also rotationsfactory\n\n% Please cite the Manopt paper as well as the research paper:\n% @InProceedings{tron2014essential,\n% Title = {On the quotient representation of the essential manifold},\n% Author = {Tron, R. and Daniilidis, K.},\n% Booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},\n% Year = {2014},\n% Organization = {{IEEE CVPR}}\n% }\n\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Roberto Tron, Aug. 8, 2014\n% Contributors: Bamdev Mishra, May 15, 2015.\n%\n%\n% RT: General implementation note: to streamline component-wise\n% computations, in tangentProjection and exponential,\n% we flatten out the arguments into [3 x 3 x 2K] arrays, compute the\n% components all together, and then sharp the result again into [3 x 6 x K]\n% arrays.\n\n\n % Optional parameters to switch between the signed and unsigned\n % versions of the manifold.\n if ~exist('strSigned', 'var') || isempty(strSigned)\n strSigned = 'signed';\n end\n switch(strSigned)\n case 'signed'\n flagSigned = true;\n case 'unsigned'\n flagSigned = false;\n otherwise\n error('Second argument can be either empty, ''signed'', or ''unsigned''.');\n end\n\n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n \n if k == 1\n M.name = @() sprintf('Quotient representation of the essential manifold, %s', strSigned);\n elseif k > 1 && k == round(k)\n M.name = @() sprintf('Product of %d quotient representations of the essential manifold, %s', k, strSigned);\n else\n error('k must be an integer no less than 1.');\n end\n \n M.dim = @() k*5;\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:));\n \n M.typicaldist = @() pi*sqrt(2*k);\n \n M.proj = @tangentProjection;\n function HProjHoriz=tangentProjection(X,H)\n % Project H on the tangent space of SO(3)^2\n HProj = essential_sharp(multiskew(multiprod(multitransp(essential_flat(X)), essential_flat(H))));\n \n % Compute projection on vertical component\n p = vertproj(X, HProj);\n \n HProjHoriz = HProj - multiprod(p/2,[essential_hat3(permute(X(3,1:3,:),[2 3 1])) essential_hat3(permute(X(3,4:6,:),[2 3 1]))]);% BM: okay\n end\n \n \n M.tangent = @(X, H) essential_sharp(multiskew(essential_flat(H)));\n \n M.egrad2rgrad=@egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n rgrad = M.proj(X, egrad);\n end\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, S)\n % Reminder: S contains skew-symmeric matrices. The actual\n % direction that the point X is moved along is X*S.\n RA = p1(X);\n RB = p2(X);\n SA = p1(S);\n SB = p2(S);\n \n G = egrad; \n GA = p1(G);\n GB = p2(G);\n \n H = ehess; \n \n % RT: We now compute the connection, i.e. the part of the derivative\n % given by the curvature of the space (as opposed to a simple\n % Euclidean derivative).\n \n % The following is the vectorized version of connection=-[multisym(GA'*RA)*SA multisym(GB'*RB)*SB];\n connection = tangent2ambient(X,-cat(2,...\n multiprod(multisym(multiprod(multitransp(GA), RA)), SA),...\n multiprod(multisym(multiprod(multitransp(GB), RB)), SB)));\n rhess = M.proj(X,H + connection);\n end\n \n \n \n M.exp = @exponential;\n function Y = exponential(X, U, t)\n if nargin == 3\n U = t*U;\n end\n \n UFlat = essential_flat(U);\n exptUFlat = rot3_exp(UFlat);\n Y = essential_sharp(multiprod(essential_flat(X), exptUFlat));\n end\n \n M.retr = @exponential;\n \n M.log = @logarithm; \n function U = logarithm(X, Y)\n \n QX = [X(:,1:3,:);X(:,4:6,:)];\n QY = [Y(:,1:3,:);Y(:,4:6,:)];\n QYr = essential_closestRepresentative(QX,QY,'flagSigned',flagSigned);\n Yr = [QYr(1:3,:,:) QYr(4:6,:,:)];\n U = zeros(size(X));\n U(:,1:3,:) = rot3_log(multiprod(multitransp(X(:,1:3,:)),Yr(:,1:3,:)));\n U(:,4:6,:) = rot3_log(multiprod(multitransp(X(:,4:6,:)),Yr(:,4:6,:)));\n end\n \n M.hash = @(X) ['z' hashmd5(X(:))];\n \n M.rand = @() randessential(k);\n function Q = randessential(N)\n % Generates random essential matrices.\n %\n % function Q = randessential(N)\n %\n % Q is a [3x6] matrix where each [3x3] block is a uniformly distributed\n % matrix.\n \n % This file is part of Manopt: www.manopt.org.\n % Original author: Roberto Tron, Aug. 8, 2014\n % Contributors:\n % Change log:\n \n if nargin < 1\n N = 1;\n end\n \n Q = [randrot(3,N) randrot(3,N)];\n end\n \n M.randvec = @randomvec;\n function U = randomvec(X)\n U = tangentProjection(X, essential_sharp(randskew(3, 2*k)));\n U = U / sqrt(M.inner([],U,U));\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(3, 6, k);\n \n M.transp = @transport;\n function S2 = transport(X1, X2, S1)\n % Transport a vector from the tangent space at X1 to the tangent\n % space at X2. This transport uses the left translation of the\n % ambient group and preserves the norm of S1. The left translation\n % aligns the vertical spaces at the two elements.\n \n % Group operation in the ambient group, X12=X2'*X1\n X12 = essential_sharp(multiprod(multitransp(essential_flat(X2)),essential_flat(X1)));\n X12Flat = essential_flat(X12);\n \n % Left translation, S2=X12*S*X12'\n S2 = essential_sharp(multiprod(X12Flat,multiprod(essential_flat(S1),multitransp(X12Flat))));\n end\n \n M.pairmean = @pairmean;\n function Y = pairmean(X1, X2)\n V = M.log(X1, X2);\n Y = M.exp(X1, .5*V);\n end\n \n M.dist = @(x, y) M.norm(x, M.log(x, y)); \n \n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, [3, 6, k]);\n M.vecmatareisometries = @() true;\n \n \n \n p1 = @(X) X(:,1:3,:);\n p2 = @(X) X(:,4:6,:);\n \n \n vertproj = @(X,H) multiprod(X(3,1:3,:),permute(vee3(H(:,1:3,:)),[1 3 2]))+multiprod(X(3,4:6,:),permute(vee3(H(:,4:6,:)),[1 3 2]));\n \n tangent2ambient = @(X, H) essential_sharp(multiprod(essential_flat(X), essential_flat(H)));\n \n \nend\n\n\n%% Some functions used by the essential factory\n\nfunction v = vee3(V)\n v = squeeze([V(3,2,:)-V(2,3,:); V(1,3,:)-V(3,1,:); V(2,1,:)-V(1,2,:)])/2;\nend\n\n\n% Compute the exponential map in SO(3) using Rodrigues' formula\n% function R = rot3_exp(V)\n% V must be a [3x3xN] array of [3x3] skew-symmetric matrices.\nfunction R = rot3_exp(V)\n v = vee3(V);\n nv = cnorm(v);\n idxZero = nv < 1e-15;\n nvMod = nv;\n nvMod(idxZero) = 1;\n \n vNorm = v./([1;1;1]*nvMod);\n \n % Matrix exponential using Rodrigues' formula\n nv = shiftdim(nv,-1);\n c = cos(nv);\n s = sin(nv);\n [VNorm,vNormShift] = essential_hat3(vNorm);\n vNormvNormT = multiprod(vNormShift,multitransp(vNormShift));\n R=multiprod(eye(3),c)+multiprod(VNorm,s)+multiprod(vNormvNormT,1-c);\nend\n\n\n\n% Compute the logarithm map in SO(3)\n% function V = rot3_log(R)\n% V is a [3x3xN] array of [3x3] skew-symmetric matrices\nfunction V = rot3_log(R)\n skewR = multiskew(R);\n ctheta = (multitrace(R)'-1)/2;\n stheta = cnorm(vee3(skewR));\n theta = atan2(stheta,ctheta);\n \n V=skewR;\n for ik=1:size(R,3)\n V(:,:,ik)=V(:,:,ik)/sincN(theta(ik));\n end\nend\n\n\nfunction sx = sincN(x)\n sx = sin(x)./x;\n sx(x==0) = 1;\nend\n\nfunction nv = cnorm(v)\n nv = sqrt(sum(v.^2));\nend\n\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/privateessential/essential_closestRepresentative.m", "content": "function Q2r=essential_closestRepresentative(Q1,Q2,varargin)\n[tMin,~,Q2]=essential_distMinAngle(Q1,Q2,varargin{:});\nNQ1=size(Q1,3);\nNQ2=size(Q2,3);\n\nif NQ1>1 && NQ2==1\n Q2=repmat(Q2,[1 1 NQ1]);\nend\nNQ=max(NQ1,NQ2);\n\nQ2r=zeros(size(Q2));\nfor iQ=1:NQ\n t=tMin(iQ);\n Rz=[cos(t) -sin(t) 0; sin(t) cos(t) 0; 0 0 1];\n Q2r(1:3,1:3,iQ)=Rz*Q2(1:3,1:3,iQ);\n Q2r(4:6,1:3,iQ)=Rz*Q2(4:6,1:3,iQ);\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/privateessential/essential_distMinAngle.m", "content": "function [tMin,fMin,Q2Flip,output]=essential_distMinAngle(Q1,Q2,varargin)\nNQ1=size(Q1,3);\nNQ2=size(Q2,3);\n\nif NQ1==1 && NQ2>1\n Q1=repmat(Q1,[1 1 NQ2]);\n NQ1=NQ2;\nend\nif NQ1>1 && NQ2==1\n Q2=repmat(Q2,[1 1 NQ1]);\nend\n\nif NQ1>1\n tMin=zeros(NQ1,1);\n fMin=zeros(NQ1,1);\n Q2Flip=zeros(6,3,NQ1);\n if nargout>3\n output=repmat(struct('tMin',[],'fMin',[],'tBreak1',[],'tBreak2',[]),NQ1,1);\n end\n for iQ=1:NQ1\n if nargout>3\n [tMin(iQ),fMin(iQ),Q2Flip(:,:,iQ),output(iQ)]=...\n essential_distMinAngle(Q1(:,:,iQ),Q2(:,:,iQ),varargin{:});\n else\n [tMin(iQ),fMin(iQ),Q2Flip(:,:,iQ)]=...\n essential_distMinAngle(Q1(:,:,iQ),Q2(:,:,iQ),varargin{:});\n end\n end\nelse\n flagModTMin=false;\n flagSigned=false;\n\n %optional parameters\n ivarargin=1;\n while(ivarargin<=length(varargin))\n switch(lower(varargin{ivarargin}))\n case 'flagmodtmin'\n ivarargin=ivarargin+1;\n flagModTMin=varargin{ivarargin};\n case 'signed'\n flagSigned=true;\n case 'flagsigned'\n ivarargin=ivarargin+1;\n flagSigned=varargin{ivarargin};\n otherwise\n error(['Argument ' varargin{ivarargin} ' not valid!'])\n end\n ivarargin=ivarargin+1;\n end\n\n tMin=zeros(4,1);\n fMin=zeros(4,1);\n tBreak1=zeros(4,1);\n tBreak2=zeros(4,1);\n Q2Flip=zeros(6,3,4);\n if ~flagSigned\n for k=1:4\n [tMin(k),fMin(k),tBreak1(k),tBreak2(k),Q2Flip(:,:,k)]=...\n essential_distMinAnglePair(Q1,Q2,k);\n end\n else\n [tMin,fMin,tBreak1,tBreak2,Q2Flip]=...\n essential_distMinAnglePair(Q1,Q2,1);\n end \n\n if flagModTMin\n tMin=modAngle(tMin);\n end\n\n if nargout>3\n output.tMin=tMin;\n output.fMin=fMin;\n output.tBreak1=tBreak1;\n output.tBreak2=tBreak2;\n end\n\n if ~flagSigned\n [fMin,idxMin]=min(fMin);\n fMin=max(fMin,0);\n tMin=tMin(idxMin);\n Q2Flip=Q2Flip(:,:,idxMin);\n if nargout>3\n output.idxMin=idxMin;\n end\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/privateessential/essential_distMinAnglePair.m", "content": "function [tMin,fMin,tBreak1,tBreak2,Q2,tMinAll]=essential_distMinAnglePair(Q1,Q2,kFlip)\n\nswitch kFlip\n case 1\n %nothing to do\n case 2\n Q2([2 3 4 6],:)=-Q2([2 3 4 6],:);\n case 3\n Q2([4 5],:)=-Q2([4 5],:);\n case 4\n Q2([2 3 5 6],:)=-Q2([2 3 5 6],:);\n otherwise\n error('Value of kFlip invalid')\nend\n\nQ11=Q1(1:3,:);\nQ12=Q1(4:6,:);\nQ21=Q2(1:3,:);\nQ22=Q2(4:6,:);\n\nQ211=Q21*Q11';\nQ212=Q22*Q12';\n[tMin,fMin,tBreak1,tBreak2,tMinAll]=essential_distMinAnglePair_base(Q211,Q212);\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/essential/privateessential/essential_distMinAnglePair_base.m", "content": "function [tMin,fMin,tBreak1,tBreak2,tMinAll]=essential_distMinAnglePair_base(Q211,Q212)\nflagCheckFirstDer=true;\nflagUseNewton=true; %Note: requires flagCheckFirstDer=true\ntolMZero=1e-15;\ntMinAll=[];\n\n[tBreak1,~,~,c1,m1,p1]=essential_distMinAnglePair_discontinuityDistance(Q211);\n[tBreak2,~,~,c2,m2,p2]=essential_distMinAnglePair_discontinuityDistance(Q212);\n\n%check for the degenerate case where the cost is constant\nif abs(m1)tSearch2\n tSearch1=tSearch1-2*pi;\n end\n\n if flagCheckFirstDer\n %compute derivatives of each term at discontinuity points\n df1Break1=essential_distMinAnglePair_computeDfBreak(tBreak1,Q211);\n df2Break2=essential_distMinAnglePair_computeDfBreak(tBreak2,Q212);\n% disp('[df1Break1 df2Break2]')\n% disp([df1Break1 df2Break2])\n %compute derivative of each term at other's discontinuity\n %(unroll two calls to dfi)\n theta1Break2=acos(clip((m1*sin(tBreak2+p1)+c1-1)/2));\n df1Break2=-theta1Break2*(m1*cos(tBreak2+p1))/(2*sin(theta1Break2));\n theta2Break1=acos(clip((m2*sin(tBreak1+p2)+c2-1)/2));\n df2Break1=-theta2Break1*(m2*cos(tBreak1+p2))/(2*sin(theta2Break1));\n\n %compute left and right derivatives of sum of the two terms\n dfBreak1n=+df1Break1+df2Break1;\n dfBreak1p=-df1Break1+df2Break1;\n dfBreak2n=+df2Break2+df1Break2;\n dfBreak2p=-df2Break2+df1Break2;\n\n flagSearch1=false;\n % plot([tBreak1 tBreak2],[dfBreak1p dfBreak2p],'cx','MarkerSize',10)\n % plot([tBreak1 tBreak2],[dfBreak1n dfBreak2n],'mx','MarkerSize',10)\n if sign(dfBreak1p)~=sign(dfBreak2n)\n if flagUseNewton\n %parabolic prediction of min\n tMin0=tSearch1-dfBreak1p*(tSearch2-tSearch1)/(dfBreak2n-dfBreak1p);\n %tMin0=(tSearch1+tSearch2)/2;\n [tMin,fMin]=essential_distMinAnglePair_dfNewton(m1,p1,c1,m2,p2,c2,tMin0,tSearch1,tSearch2);\n %fMin=essential_distMinAnglePair_ft(m1,p1,c1,m2,p2,c2,tMin);\n else\n [tMin,fMin]=fminbnd(essential_distMinAnglePair_ft,tSearch1,tSearch2);\n end\n tMinAll=[tMinAll tMin];\n flagSearch1=true;\n end\n tSearch1=tSearch1+2*pi;\n if sign(dfBreak2p)~=sign(dfBreak1n)\n if flagUseNewton\n %parabolic prediction of min\n tMin0=tSearch2-dfBreak2p*(tSearch1-tSearch2)/(dfBreak1n-dfBreak2p);\n %tMin0=(tSearch1+tSearch2)/2;\n [tMin2,fMin2]=essential_distMinAnglePair_dfNewton(m1,p1,c1,m2,p2,c2,tMin0,tSearch2,tSearch1);\n %fMin2=essential_distMinAnglePair_ft(m1,p1,c1,m2,p2,c2,tMin2);\n else\n [tMin2,fMin2]=fminbnd(essential_distMinAnglePair_ft,tSearch2,tSearch1);\n end\n if ~flagSearch1 || (flagSearch1 && fMin2\n u = randn(dimensions_vec) + 1i*randn(dimensions_vec);\n u = u / norm(u(:), 'fro');\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(dimensions_vec);\n \n M.transp = @(x1, x2, d) d;\n \n M.pairmean = @(x1, x2) .5*(x1+x2);\n \n sz = prod(dimensions_vec);\n M.vec = @(x, u_mat) [real(u_mat(:)) ; imag(u_mat(:))];\n M.mat = @(x, u_vec) reshape(u_vec(1:sz), dimensions_vec) ...\n + 1i*reshape(u_vec((sz+1):end), dimensions_vec);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/euclidean/euclideanfactory.m", "content": "function M = euclideanfactory(m, n)\n% Returns a manifold struct to optimize over real matrices.\n%\n% function M = euclideanfactory(m)\n% function M = euclideanfactory(m, n)\n% function M = euclideanfactory([n1, n2, ...])\n%\n% Returns M, a structure describing the Euclidean space of real matrices,\n% equipped with the standard Frobenius distance and associated trace inner\n% product, as a manifold for Manopt.\n%\n% m and n in general can be vectors to handle multidimensional arrays.\n% If either of m or n is a vector, they are concatenated as [m, n].\n%\n% Using this simple linear manifold, Manopt can be used to solve standard\n% unconstrained optimization problems, for example in replacement of\n% Matlab's fminunc.\n%\n% See also: euclideancomplexfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: Bamdev Mishra, May 4, 2015.\n% Change log: \n%\n% July 5, 2013 (NB):\n% Added egred2rgrad, ehess2rhess, mat, vec, tangent.\n% May 4, 2015 (BM):\n% Added functionality to handle multidimensional arrays.\n\n\n % The size can be defined using both m and n, or simply with m.\n % If m is a scalar, then n is implicitly 1.\n % This mimics the use of built-in Matlab functions such as zeros(...).\n if ~exist('n', 'var') || isempty(n)\n if numel(m) == 1\n n = 1;\n else\n n = [];\n end\n end\n \n dimensions_vec = [m(:)', n(:)']; % We have a row vector.\n \n M.size = @() dimensions_vec;\n \n M.name = @() sprintf('Euclidean space R^(%s)', num2str(dimensions_vec));\n \n M.dim = @() prod(dimensions_vec);\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:), 'fro');\n \n M.dist = @(x, y) norm(x(:) - y(:), 'fro');\n \n M.typicaldist = @() sqrt(prod(dimensions_vec));\n \n M.proj = @(x, d) d;\n \n M.egrad2rgrad = @(x, g) g;\n \n M.ehess2rhess = @(x, eg, eh, d) eh;\n \n M.tangent = M.proj;\n \n M.exp = @exp;\n function y = exp(x, d, t)\n if nargin == 3\n y = x + t*d;\n else\n y = x + d;\n end\n end\n \n M.retr = M.exp;\n\t\n\tM.log = @(x, y) y-x;\n\n M.hash = @(x) ['z' hashmd5(x(:))];\n \n M.rand = @() randn(dimensions_vec);\n \n M.randvec = @randvec;\n function u = randvec(x) %#ok\n u = randn(dimensions_vec);\n u = u / norm(u(:), 'fro');\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(dimensions_vec);\n \n M.transp = @(x1, x2, d) d;\n \n M.pairmean = @(x1, x2) .5*(x1+x2);\n \n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, dimensions_vec);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/euclidean/shapefitfactory.m", "content": "function M = shapefitfactory(VJt)\n% Linear manifold structure for optimization over the ShapeFit search space\n%\n% function M = shapefitfactory(VJt)\n%\n% Input: VJt is a matrix of size dxn, such that VJt * ones(n, 1) = 0.\n%\n% Returns M, a structure describing the Euclidean space of d-by-n matrices\n% equipped with the standard Frobenius distance and associated trace inner\n% product, as a manifold for Manopt. Matrices on M, denoted by T, have size\n% dxn and obey T*ones(n, 1) = 0 (centered columns) and = 1, where\n% = Trace(A' * B).\n%\n% See this paper: http://arxiv.org/abs/1506.01437\n% ShapeFit: Exact location recovery from corrupted pairwise directions, 2015\n% Paul Hand, Choongbum Lee, Vladislav Voroninski\n%\n% See also: shapefit_smoothed\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 18, 2015.\n% Contributors: \n% Change log: \n%\n% Jan. 25, 2017 (NB):\n% M.tangent = M.proj now, instead of being identity. This is notably\n% necessary so that checkgradient will pick up on gradients that do\n% not lie in the appropriate tangent space.\n \n [d, n] = size(VJt);\n\n M.name = @() sprintf('ShapeFit space of size %d x %d', d, n);\n \n M.dim = @() d*n - d - 1;\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d, 'fro');\n \n M.dist = @(x, y) norm(x-y, 'fro');\n \n M.typicaldist = @() sqrt(d*n);\n \n M.proj = @(T, U) projection(U);\n VJt_normed = VJt / norm(VJt, 'fro');\n function PU = projection(U)\n % Center the columns\n PU = bsxfun(@minus, U, mean(U, 2));\n % Remove component along VJt\n % Note: these two actions can be executed separately, without\n % interference, owing to VJt having centered columns itself.\n PU = PU - (VJt_normed(:)'*U(:))*VJt_normed;\n end\n \n M.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @(x, eg, eh, d) projection(eh);\n \n M.tangent = M.proj;\n \n M.exp = @exp;\n function y = exp(x, d, t)\n if nargin == 3\n y = x + t*d;\n else\n y = x + d;\n end\n end\n \n M.retr = M.exp;\n\t\n\tM.log = @(x, y) y-x;\n\n M.hash = @(x) ['z' hashmd5(x(:))];\n \n M.randvec = @(x) randvec();\n function u = randvec()\n u = projection(randn(d, n));\n u = u / norm(u, 'fro');\n end\n \n % We exploit the fact that VJt_normed belongs to the manifold\n M.rand = @() VJt_normed + randn(1) * randvec();\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(d, n);\n \n M.transp = @(x1, x2, d) d;\n \n M.pairmean = @(x1, x2) .5*(x1+x2);\n \n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, [d, n]);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/euclidean/skewsymmetricfactory.m", "content": "function M = skewsymmetricfactory(n, k)\n% Returns a manifold struct to optimize over k skew-symmetric matrices of size n\n%\n% function M = skewsymmetricfactory(n)\n% function M = skewsymmetricfactory(n, k)\n%\n% Returns M, a structure describing the Euclidean space of n-by-n\n% skew-symmetric matrices equipped with the standard Frobenius distance and\n% associated trace inner product, as a manifold for Manopt.\n%\n% By default, k = 1. If k > 1, points and vectors are stored in 3D matrices\n% X of size nxnxk such that each slice X(:, :, i), for i = 1:k, is\n% skew-symmetric.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 28, 2016.\n% Contributors: \n% Change log: \n%\n% Jan. 25, 2017 (NB):\n% M.tangent = M.proj now, instead of being identity. This is notably\n% necessary so that checkgradient will pick up on gradients that do\n% not lie in the appropriate tangent space.\n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n\n M.name = @() sprintf('(Skew-symmetric matrices of size %d)^%d', n, k);\n \n M.dim = @() k*n*(n-1)/2;\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:), 'fro');\n \n M.dist = @(x, y) norm(x(:)-y(:), 'fro');\n \n M.typicaldist = @() sqrt(k)*n;\n \n M.proj = @(x, d) multiskew(d);\n \n M.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @(x, eg, eh, d) M.proj(x, eh);\n \n M.tangent = M.proj;\n \n M.exp = @exp;\n function y = exp(x, d, t)\n if nargin == 3\n y = x + t*d;\n else\n y = x + d;\n end\n end\n \n M.retr = M.exp;\n\t\n\tM.log = @(x, y) y-x;\n\n M.hash = @(x) ['z' hashmd5(x(:))];\n \n M.rand = @() multiskew(randn(n, n, k));\n \n M.randvec = @randvec;\n function u = randvec(x) %#ok\n u = multiskew(randn(n, n, k));\n u = u / norm(u(:), 'fro');\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, n, k);\n \n M.transp = @(x1, x2, d) d;\n \n M.pairmean = @(x1, x2) .5*(x1+x2);\n \n \n % Elaborate list of indices of strictly upper-triangular entries.\n single_upper_triangle = find(triu(ones(n), 1));\n all_upper_triangle = bsxfun(@plus, single_upper_triangle, n^2*(0:(k-1)));\n all_upper_triangle = all_upper_triangle(:);\n \n % To vectorize a matrix, we extract all upper-triangular entries and\n % scale by sqrt(2) to ensure isometry, that is: given two tangent\n % vectors U and V at a point X, M.inner(X, U, V) is equal to u'*v,\n % where u = M.vec(X, U) and likewise for v. This construction has the\n % advantage of providing a vectorized representation of matrices that\n % has the same length as the intrinsic dimension of the space they live\n % in.\n M.vec = @(x, u_mat) sqrt(2)*u_mat(all_upper_triangle);\n M.mat = @matricize;\n function u_mat = matricize(X, u_vec) %#ok\n u_mat = zeros(n, n, k);\n u_mat(all_upper_triangle) = u_vec((k*n+1):end) / sqrt(2);\n u_mat = u_mat - multitransp(u_mat);\n end\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/euclidean/symmetricfactory.m", "content": "function M = symmetricfactory(n, k)\n% Returns a manifold struct to optimize over k symmetric matrices of size n\n%\n% function M = symmetricfactory(n)\n% function M = symmetricfactory(n, k)\n%\n% Returns M, a structure describing the Euclidean space of n-by-n symmetric\n% matrices equipped with the standard Frobenius distance and associated\n% trace inner product, as a manifold for Manopt.\n% \n% By default, k = 1. If k > 1, points and vectors are stored in 3D matrices\n% X of size nxnxk such that each slice X(:, :, i), for i = 1:k, is\n% symmetric.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Jan. 22, 2014.\n% Contributors: \n% Change log: \n%\n% Jan. 25, 2017 (NB):\n% M.tangent = M.proj now, instead of being identity. This is notably\n% necessary so that checkgradient will pick up on gradients that do\n% not lie in the appropriate tangent space.\n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n\n M.name = @() sprintf('(Symmetric matrices of size %d)^%d', n, k);\n \n M.dim = @() k*n*(n+1)/2;\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:), 'fro');\n \n M.dist = @(x, y) norm(x(:)-y(:), 'fro');\n \n M.typicaldist = @() sqrt(k)*n;\n \n M.proj = @(x, d) multisym(d);\n \n M.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @(x, eg, eh, d) M.proj(x, eh);\n \n M.tangent = M.proj;\n \n M.exp = @exp;\n function y = exp(x, d, t)\n if nargin == 3\n y = x + t*d;\n else\n y = x + d;\n end\n end\n \n M.retr = M.exp;\n\t\n\tM.log = @(x, y) y-x;\n\n M.hash = @(x) ['z' hashmd5(x(:))];\n \n M.rand = @() multisym(randn(n, n, k));\n \n M.randvec = @randvec;\n function u = randvec(x) %#ok\n u = multisym(randn(n, n, k));\n u = u / norm(u(:), 'fro');\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, n, k);\n \n M.transp = @(x1, x2, d) d;\n \n M.pairmean = @(x1, x2) .5*(x1+x2);\n \n \n % Elaborate list of indices of diagonal entries of an nxnxk matrix.\n single_diag_entries = (1:(n+1):n^2)';\n all_diag_entries = bsxfun(@plus, single_diag_entries, n^2*(0:(k-1)));\n all_diag_entries = all_diag_entries(:);\n \n % Likewise, elaborate list of indices of upper-triangular entries.\n single_upper_triangle = find(triu(ones(n), 1));\n all_upper_triangle = bsxfun(@plus, single_upper_triangle, n^2*(0:(k-1)));\n all_upper_triangle = all_upper_triangle(:);\n \n % To vectorize a matrix, we extract all diagonal entries, then all\n % upper-triangular entries, the latter being scaled by sqrt(2) to\n % ensure isometry, that is: given two tangent vectors U and V at a\n % point X, M.inner(X, U, V) is equal to u'*v, where u = M.vec(X, U) and\n % likewise for v. This construction has the advantage of providing a\n % vectorized representation of matrices that has the same length as the\n % intrinsic dimension of the space they live in.\n M.vec = @(x, u_mat) [u_mat(all_diag_entries) ; ...\n sqrt(2)*u_mat(all_upper_triangle)];\n M.mat = @matricize;\n function u_mat = matricize(X, u_vec) %#ok\n u_mat = zeros(n, n, k);\n u_mat(all_upper_triangle) = u_vec((k*n+1):end) / sqrt(2);\n u_mat = u_mat + multitransp(u_mat);\n u_mat(all_diag_entries) = u_vec(1:(k*n));\n end\n M.vecmatareisometries = @() true;\n\nend\n\n% Former, easier versions for vec / mat. They had the disadvantage of\n% giving vector representations of length k*n^2, instead of k*n*(n+1).\n% M.vec = @(x, u_mat) u_mat(:);\n% M.mat = @(x, u_vec) reshape(u_vec, [m, n]);\n% M.vecmatareisometries = @() true;\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedrank/fixedrankMNquotientfactory.m", "content": "function M = fixedrankMNquotientfactory(m, n, k)\n% Manifold of m-by-n matrices of rank k with two factor quotient geometry.\n%\n% function M = fixedrankMNquotientfactory(m, n, k)\n%\n% This follows the quotient geometry described in the following paper:\n% P.-A. Absil, L. Amodei and G. Meyer,\n% \"Two Newton methods on the manifold of fixed-rank matrices endowed\n% with Riemannian quotient geometries\", arXiv, 2012.\n%\n% Paper link: http://arxiv.org/abs/1209.0068\n%\n% A point X on the manifold is represented as a structure with two\n% fields: M and N. The matrix M (mxk) is orthonormal, while the matrix N\n% (nxk) is full-rank such that X = M*N';\n%\n% Tangent vectors are represented as a structure with two fields (M, N).\n%\n% Please cite the Manopt paper as well as the research paper:\n% @Article{absil2014fixedrank,\n% Title = {Two Newton methods on the manifold of fixed-rank matrices endowed with Riemannian quotient geometries},\n% Author = {Absil, P.-A. and Amodei, L. and Meyer, G.},\n% Journal = {Computational Statistics},\n% Year = {2014},\n% Number = {3-4},\n% Pages = {569--590},\n% Volume = {29},\n% Doi = {10.1007/s00180-013-0441-6}\n% }\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors:\n% Change log:\n \n \n M.name = @() sprintf('MN'' quotient manifold of %dx%d matrices of rank %d', m, n, k);\n \n M.dim = @() (m+n-k)*k;\n \n % Choice of the metric is motivated by the symmetry present in the\n % space.\n M.inner = @(X, eta, zeta) eta.M(:).'*zeta.M(:) + eta.N(:).'*zeta.N(:);\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(x, y) error('fixedrankMNquotientfactory.dist not implemented yet.');\n \n M.typicaldist = @() 10*k;\n \n symm = @(X) .5*(X+X');\n stiefel_proj = @(M, H) H - M*symm(M'*H);\n \n M.egrad2rgrad = @egrad2rgrad;\n function eta = egrad2rgrad(X, eta)\n eta.M = stiefel_proj(X.M, eta.M);\n end\n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(X, egrad, ehess, eta)\n \n % Directional derivative of the Riemannian gradient.\n Hess.M = ehess.M - eta.M*symm(X.M'*egrad.M);\n Hess.M = stiefel_proj(X.M, Hess.M);\n \n Hess.N = ehess.N;\n \n % Projection onto the horizontal space.\n Hess = M.proj(X, Hess);\n end\n \n \n M.proj = @projection;\n function etaproj = projection(X, eta)\n \n % Start by projecting the vector from Rmp x Rnp to the tangent\n % space to the total space, that is, eta.M should be in the\n % tangent space to Stiefel at X.M and eta.N is arbitrary.\n eta.M = stiefel_proj(X.M, eta.M);\n \n % Now project from the tangent space to the horizontal space, that\n % is, take care of the quotient.\n \n % First solve a Sylvester equation (A symm., B skew-symm.)\n A = X.N'*X.N + eye(k);\n B = eta.M'*X.M + eta.N'*X.N;\n B = B-B';\n omega = lyap(A, -B);\n \n % And project along the vertical space to the horizontal space.\n etaproj.M = eta.M + X.M*omega;\n etaproj.N = eta.N + X.N*omega;\n \n end\n \n M.exp = @exponential;\n function Y = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n A = t*X.M'*eta.M;\n S = t^2*eta.M'*eta.M;\n Y.M = [X.M t*eta.M]*expm([A -S ; eye(k) A])*eye(2*k, k)*expm(-A);\n \n % re-orthonormalize (seems necessary from time to time).\n [Q R] = qr(Y.M, 0);\n Y.M = Q * diag(sign(diag(R)));\n \n Y.N = X.N + t*eta.N;\n \n end\n \n % Factor M lives on the Stiefel manifold, hence we will reuse its\n % random generator.\n stiefelm = stiefelfactory(m, k);\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n Y.M = uf(X.M + t*eta.M); % This is a valid retraction\n Y.N = X.N + t*eta.N;\n end\n \n M.hash = @(X) ['z' hashmd5([X.M(:) ; X.N(:)])];\n \n M.rand = @random;\n function X = random()\n X.M = stiefelm.rand();\n X.N = randn(n, k);\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(X)\n eta.M = randn(m, k);\n eta.N = randn(n, k);\n eta = projection(X, eta);\n nrm = M.norm(X, eta);\n eta.M = eta.M / nrm;\n eta.N = eta.N / nrm;\n end\n \n M.lincomb = @lincomb;\n \n M.zerovec = @(X) struct('M', zeros(m, k), 'N', zeros(n, k));\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \nend\n\n\n% Linear combination of tangent vectors\nfunction d = lincomb(x, a1, d1, a2, d2) %#ok\n \n if nargin == 3\n d.M = a1*d1.M;\n d.N = a1*d1.N;\n elseif nargin == 5\n d.M = a1*d1.M + a2*d2.M;\n d.N = a1*d1.N + a2*d2.N;\n else\n error('Bad use of fixedrankMNquotientfactory.lincomb.');\n end\n \nend\n\n\nfunction A = uf(A)\n [L, unused, R] = svd(A, 0);\n A = L*R';\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedrank/fixedrankembeddedfactory.m", "content": "function M = fixedrankembeddedfactory(m, n, k)\n% Manifold struct to optimize fixed-rank matrices w/ an embedded geometry.\n%\n% function M = fixedrankembeddedfactory(m, n, k)\n%\n% Manifold of m-by-n real matrices of fixed rank k. This follows the\n% embedded geometry described in Bart Vandereycken's 2013 paper:\n% \"Low-rank matrix completion by Riemannian optimization\".\n% \n% Paper link: http://arxiv.org/pdf/1209.3834.pdf\n%\n% A point X on the manifold is represented as a structure with three\n% fields: U, S and V. The matrices U (mxk) and V (nxk) are orthonormal,\n% while the matrix S (kxk) is any /diagonal/, full rank matrix.\n% Following the SVD formalism, X = U*S*V'. Note that the diagonal entries\n% of S are not constrained to be nonnegative.\n%\n% Tangent vectors are represented as a structure with three fields: Up, M\n% and Vp. The matrices Up (mxk) and Vp (mxk) obey Up'*U = 0 and Vp'*V = 0.\n% The matrix M (kxk) is arbitrary. Such a structure corresponds to the\n% following tangent vector in the ambient space of mxn matrices:\n% Z = U*M*V' + Up*V' + U*Vp'\n% where (U, S, V) is the current point and (Up, M, Vp) is the tangent\n% vector at that point.\n%\n% Vectors in the ambient space are best represented as mxn matrices. If\n% these are low-rank, they may also be represented as structures with\n% U, S, V fields, such that Z = U*S*V'. There are no resitrictions on what\n% U, S and V are, as long as their product as indicated yields a real, mxn\n% matrix.\n%\n% The chosen geometry yields a Riemannian submanifold of the embedding\n% space R^(mxn) equipped with the usual trace (Frobenius) inner product.\n%\n%\n% Please cite the Manopt paper as well as the research paper:\n% @Article{vandereycken2013lowrank,\n% Title = {Low-rank matrix completion by {Riemannian} optimization},\n% Author = {Vandereycken, B.},\n% Journal = {SIAM Journal on Optimization},\n% Year = {2013},\n% Number = {2},\n% Pages = {1214--1236},\n% Volume = {23},\n% Doi = {10.1137/110845768}\n% }\n%\n% See also: fixedrankfactory_2factors fixedrankfactory_3factors\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n%\tFeb. 20, 2014 (NB):\n% Added function tangent to work with checkgradient.\n%\n% June 24, 2014 (NB):\n% A couple modifications following\n% Bart Vandereycken's feedback:\n% - The checksum (hash) was replaced for a faster alternative: it's a\n% bit less \"safe\" in that collisions could arise with higher\n% probability, but they're still very unlikely.\n% - The vector transport was changed.\n% The typical distance was also modified, hopefully giving the\n% trustregions method a better initial guess for the trust region\n% radius, but that should be tested for different cost functions too.\n%\n% July 11, 2014 (NB):\n% Added ehess2rhess and tangent2ambient, supplied by Bart.\n%\n% July 14, 2014 (NB):\n% Added vec, mat and vecmatareisometries so that hessianspectrum now\n% works with this geometry. Implemented the tangent function.\n% Made it clearer in the code and in the documentation in what format\n% ambient vectors may be supplied, and generalized some functions so\n% that they should now work with both accepted formats.\n% It is now clearly stated that for a point X represented as a\n% triplet (U, S, V), the matrix S needs to be diagonal.\n\n M.name = @() sprintf('Manifold of %dx%d matrices of rank %d', m, n, k);\n \n M.dim = @() (m+n-k)*k;\n \n M.inner = @(x, d1, d2) d1.M(:).'*d2.M(:) + d1.Up(:).'*d2.Up(:) ...\n + d1.Vp(:).'*d2.Vp(:);\n \n M.norm = @(x, d) sqrt(M.inner(x, d, d));\n \n M.dist = @(x, y) error('fixedrankembeddedfactory.dist not implemented yet.');\n \n M.typicaldist = @() M.dim();\n \n % Given Z in tangent vector format, projects the components Up and Vp\n % such that they satisfy the tangent space constraints up to numerical\n % errors. If Z was indeed a tangent vector at X, this should barely\n % affect Z (it would not at all if we had infinite numerical accuracy).\n M.tangent = @tangent;\n function Z = tangent(X, Z)\n Z.Up = Z.Up - X.U*(X.U'*Z.Up);\n Z.Vp = Z.Vp - X.V*(X.V'*Z.Vp);\n end\n\n % For a given ambient vector Z, applies it to a matrix W. If Z is given\n % as a matrix, this is straightfoward. If Z is given as a structure\n % with fields U, S, V such that Z = U*S*V', the product is executed\n % efficiently.\n function ZW = apply_ambient(Z, W)\n if ~isstruct(Z)\n ZW = Z*W;\n else\n ZW = Z.U*(Z.S*(Z.V'*W));\n end\n end\n\n % Same as apply_ambient, but applies Z' to W.\n function ZtW = apply_ambient_transpose(Z, W)\n if ~isstruct(Z)\n ZtW = Z'*W;\n else\n ZtW = Z.V*(Z.S'*(Z.U'*W));\n end\n end\n \n % Orthogonal projection of an ambient vector Z represented as an mxn\n % matrix or as a structure with fields U, S, V to the tangent space at\n % X, in a tangent vector structure format.\n M.proj = @projection;\n function Zproj = projection(X, Z)\n \n ZV = apply_ambient(Z, X.V);\n UtZV = X.U'*ZV;\n ZtU = apply_ambient_transpose(Z, X.U);\n\n Zproj.M = UtZV;\n Zproj.Up = ZV - X.U*UtZV;\n Zproj.Vp = ZtU - X.V*UtZV';\n\n end\n\n M.egrad2rgrad = @projection;\n \n % Code supplied by Bart.\n % Given the Euclidean gradient at X and the Euclidean Hessian at X\n % along H, where egrad and ehess are vectors in the ambient space and H\n % is a tangent vector at X, returns the Riemannian Hessian at X along\n % H, which is a tangent vector.\n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, H)\n \n % Euclidean part\n rhess = projection(X, ehess);\n \n % Curvature part\n T = apply_ambient(egrad, H.Vp)/X.S;\n rhess.Up = rhess.Up + (T - X.U*(X.U'*T));\n T = apply_ambient_transpose(egrad, H.Up)/X.S;\n rhess.Vp = rhess.Vp + (T - X.V*(X.V'*T));\n \n end\n\n % Transforms a tangent vector Z represented as a structure (Up, M, Vp)\n % into a structure with fields (U, S, V) that represents that same\n % tangent vector in the ambient space of mxn matrices, as U*S*V'.\n % This matrix is equal to X.U*Z.M*X.V' + Z.Up*X.V' + X.U*Z.Vp'. The\n % latter is an mxn matrix, which could be too large to build\n % explicitly, and this is why we return a low-rank representation\n % instead. Note that there are no guarantees on U, S and V other than\n % that USV' is the desired matrix. In particular, U and V are not (in\n % general) orthonormal and S is not (in general) diagonal.\n % (In this implementation, S is identity, but this might change.)\n M.tangent2ambient = @tangent2ambient;\n function Zambient = tangent2ambient(X, Z)\n Zambient.U = [X.U*Z.M + Z.Up, X.U];\n Zambient.S = eye(2*k);\n Zambient.V = [X.V, Z.Vp];\n end\n \n % This retraction is second order, following general results from\n % Absil, Malick, \"Projection-like retractions on matrix manifolds\",\n % SIAM J. Optim., 22 (2012), pp. 135-158.\n M.retr = @retraction;\n function Y = retraction(X, Z, t)\n if nargin < 3\n t = 1.0;\n end\n\n % See personal notes June 28, 2012 (NB)\n [Qu, Ru] = qr(Z.Up, 0);\n [Qv, Rv] = qr(Z.Vp, 0);\n \n % Calling svds or svd should yield the same result, but BV\n % advocated svd is more robust, and it doesn't change the\n % asymptotic complexity to call svd then trim rather than call\n % svds. Also, apparently Matlab calls ARPACK in a suboptimal way\n % for svds in this scenario.\n % [Ut St Vt] = svds([X.S+t*Z.M , t*Rv' ; t*Ru , zeros(k)], k);\n [Ut, St, Vt] = svd([X.S+t*Z.M , t*Rv' ; t*Ru , zeros(k)]);\n \n Y.U = [X.U Qu]*Ut(:, 1:k);\n Y.V = [X.V Qv]*Vt(:, 1:k);\n Y.S = St(1:k, 1:k) + eps*eye(k);\n \n % equivalent but very slow code\n % [U S V] = svds(X.U*X.S*X.V' + t*(X.U*Z.M*X.V' + Z.Up*X.V' + X.U*Z.Vp'), k);\n % Y.U = U; Y.V = V; Y.S = S;\n \n end\n\n\n % Orthographic retraction provided by Teng Zhang. One interst of the\n % orthographic retraction is that if matrices are represented in full\n % size, it can be computed without any SVDs. If for an application it\n % makes sense to represent the matrices in full size, this may be a\n % good idea, but it won't shine in the present implementation of the\n % manifold.\n M.retr_ortho = @retraction_orthographic;\n function Y = retraction_orthographic(X, Z, t)\n if nargin < 3\n t = 1.0;\n end\n \n % First, write Y (the output) as U1*S0*V1', where U1 and V1 are\n % orthogonal matrices and S0 is of size r by r.\n [U1, ~] = qr(t*(X.U*Z.M + Z.Up) + X.U*X.S, 0);\n [V1, ~] = qr(t*(X.V*Z.M' + Z.Vp) + X.V*X.S, 0);\n S0 = (U1'*X.U)*(X.S + t*Z.M)*(X.V'*V1) + ...\n t*((U1'*Z.Up)*(X.V'*V1) + (U1'*X.U)*(Z.Vp'*V1));\n \n % Then, obtain the singular value decomposition of Y.\n [U2, S2, V2] = svd(S0);\n Y.U = U1*U2;\n Y.S = S2;\n Y.V = V1*V2;\n \n end\n\n \n M.exp = @exponential;\n function Y = exponential(X, Z, t)\n if nargin < 3\n t = 1.0;\n end\n Y = retraction(X, Z, t);\n warning('manopt:fixedrankembeddedfactory:exp', ...\n ['Exponential for fixed rank ' ...\n 'manifold not implemented yet. Used retraction instead.']);\n end\n\n % Less safe but much faster checksum, June 24, 2014.\n % Older version right below.\n M.hash = @(X) ['z' hashmd5([sum(X.U(:)) ; sum(X.S(:)); sum(X.V(:)) ])];\n %M.hash = @(X) ['z' hashmd5([X.U(:) ; X.S(:) ; X.V(:)])];\n \n M.rand = @random;\n % Factors U and V live on Stiefel manifolds, hence we will reuse\n % their random generator.\n stiefelm = stiefelfactory(m, k);\n stiefeln = stiefelfactory(n, k);\n function X = random()\n X.U = stiefelm.rand();\n X.V = stiefeln.rand();\n X.S = diag(sort(rand(k, 1), 1, 'descend'));\n end\n \n % Generate a random tangent vector at X.\n % TODO: consider a possible imbalance between the three components Up,\n % Vp and M, when m, n and k are widely different (which is typical).\n M.randvec = @randomvec;\n function Z = randomvec(X)\n Z.Up = randn(m, k);\n Z.Vp = randn(n, k);\n Z.M = randn(k);\n Z = tangent(X, Z);\n nrm = M.norm(X, Z);\n Z.Up = Z.Up / nrm;\n Z.Vp = Z.Vp / nrm;\n Z.M = Z.M / nrm;\n end\n \n M.lincomb = @lincomb;\n \n M.zerovec = @(X) struct('Up', zeros(m, k), 'M', zeros(k, k), ...\n 'Vp', zeros(n, k));\n \n % New vector transport on June 24, 2014 (as indicated by Bart)\n % Reference: Absil, Mahony, Sepulchre 2008 section 8.1.3:\n % For Riemannian submanifolds of a Euclidean space, it is acceptable to\n % transport simply by orthogonal projection of the tangent vector\n % translated in the ambient space.\n M.transp = @project_tangent;\n function Z2 = project_tangent(X1, X2, Z1)\n Z2 = projection(X2, tangent2ambient(X1, Z1));\n end\n\n\n M.vec = @vec;\n function Zvec = vec(X, Z)\n Zamb = tangent2ambient(X, Z);\n Zamb_mat = Zamb.U*Zamb.S*Zamb.V';\n Zvec = Zamb_mat(:);\n end\n M.mat = @(X, Zvec) projection(X, reshape(Zvec, [m, n]));\n M.vecmatareisometries = @() true;\n\nend\n\n% Linear combination of tangent vectors\nfunction d = lincomb(x, a1, d1, a2, d2) %#ok\n\n if nargin == 3\n d.Up = a1*d1.Up;\n d.Vp = a1*d1.Vp;\n d.M = a1*d1.M;\n elseif nargin == 5\n d.Up = a1*d1.Up + a2*d2.Up;\n d.Vp = a1*d1.Vp + a2*d2.Vp;\n d.M = a1*d1.M + a2*d2.M;\n else\n error('fixedrank.lincomb takes either 3 or 5 inputs.');\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedrank/fixedrankfactory_2factors.m", "content": "function M = fixedrankfactory_2factors(m, n, k)\n% Manifold of m-by-n matrices of rank k with balanced quotient geometry.\n%\n% function M = fixedrankfactory_2factors(m, n, k)\n%\n% The first-order geometry follows the balanced quotient geometry described \n% in the paper, \n% \"Linear regression under fixed-rank constraints: a Riemannian approach\",\n% G. Meyer, S. Bonnabel and R. Sepulchre, ICML 2011.\n%\n% Paper link: http://www.icml-2011.org/papers/350_icmlpaper.pdf.\n%\n% The second-order geometry follows from the paper\n% \"Fixed-rank matrix factorizations and Riemannian low-rank optimization\",\n% B. Mishra, R. Meyer, S. Bonnabel and R. Sepulchre,\n% Computational Statistics, 29(3 - 4), pp. 591 - 621, 2014.\n%\n% A point X on the manifold is represented as a structure with two\n% fields: L and R. The matrices L (mxk) and R (nxk) are full column-rank\n% matrices such that X = L*R'.\n%\n% Tangent vectors are represented as a structure with two fields: L, R.\n% \n% For first-order geometry, please cite the Manopt paper as well as the research paper:\n% @InProceedings{meyer2011linear,\n% Title = {Linear regression under fixed-rank constraints: a {R}iemannian approach},\n% Author = {Meyer, G. and Bonnabel, S. and Sepulchre, R.},\n% Booktitle = {{28th International Conference on Machine Learning}},\n% Year = {2011},\n% Organization = {{ICML}}\n% }\n%\n% For second-order geometry, please cite the Manopt paper as well as the research paper:\n% @Article{mishra2014fixedrank,\n% Title = {Fixed-rank matrix factorizations and {Riemannian} low-rank optimization},\n% Author = {Mishra, B. and Meyer, G. and Bonnabel, S. and Sepulchre, R.},\n% Journal = {Computational Statistics},\n% Year = {2014},\n% Number = {3-4},\n% Pages = {591--621},\n% Volume = {29},\n% Doi = {10.1007/s00180-013-0464-z}\n% }\n%\n%\n% See also fixedrankembeddedfactory fixedrankfactory_3factors fixedrankfactory_2factors_preconditioned\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors:\n% Change log:\n%\n% July 10, 2013 (NB):\n% Added vec, mat, tangent, tangent2ambient.\n%\n%\tJuly 03, 2015 (BM):\n% Cosmetic changes including avoiding storing the inverse of a\n% k-by-k matrix.\n \n \n M.name = @() sprintf('LR'' quotient manifold of %dx%d matrices of rank %d', m, n, k);\n \n M.dim = @() (m+n-k)*k;\n \n % Some precomputations at the point X to be used in the inner product \n % (and pretty much everywhere else).\n function X = prepare(X)\n if ~all(isfield(X,{'LtL','RtR'}))\n L = X.L;\n R = X.R;\n X.LtL = L'*L;\n X.RtR = R'*R;\n end\n end\n \n % Choice of the metric is motivated by the symmetry present in the\n % space. The metric is the natural Grassmannian metric on L and R.\n M.inner = @iproduct;\n function ip = iproduct(X, eta, zeta)\n X = prepare(X);\n ip = trace(X.LtL\\(eta.L'*zeta.L)) + trace( X.RtR\\(eta.R'*zeta.R));\n end\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(x, y) error('fixedrankfactory_2factors.dist not implemented yet.');\n \n M.typicaldist = @() 10*k;\n \n symm = @(M) .5*(M+M');\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n X = prepare(X);\n rgrad.L = egrad.L*X.LtL;\n rgrad.R = egrad.R*X.RtR;\n end\n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(X, egrad, ehess, eta)\n X = prepare(X);\n \n % Riemannian gradient computation.\n rgrad = egrad2rgrad(X, egrad);\n \n % Directional derivative of the Riemannian gradient.\n Hess.L = ehess.L*X.LtL + 2*egrad.L*symm(eta.L'*X.L);\n Hess.R = ehess.R*X.RtR + 2*egrad.R*symm(eta.R'*X.R);\n \n % We need a correction term for the non-constant metric.\n Hess.L = Hess.L - rgrad.L*(X.LtL\\(symm(X.L'*eta.L))) - eta.L*(X.LtL\\(symm(X.L'*rgrad.L))) + X.L*(X.LtL\\(symm(eta.L'*rgrad.L)));\n Hess.R = Hess.R - rgrad.R*(X.RtR\\(symm(X.R'*eta.R))) - eta.R*(X.RtR\\(symm(X.R'*rgrad.R))) + X.R*(X.RtR\\(symm(eta.R'*rgrad.R)));\n \n % Projection onto the horizontal space.\n Hess = M.proj(X, Hess);\n end\n \n M.proj = @projection;\n % Projection of the vector eta in the ambient space onto the horizontal space.\n function etaproj = projection(X, eta)\n X = prepare(X);\n \n SS = (X.LtL)*(X.RtR);\n AS = (X.LtL)*(X.R'*eta.R) - (eta.L'*X.L)*(X.RtR);\n Omega = lyap(SS, SS,-AS);\n etaproj.L = eta.L + X.L*Omega';\n etaproj.R = eta.R - X.R*Omega;\n end\n \n M.tangent = M.proj;\n M.tangent2ambient = @(X, eta) eta;\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n Y.L = X.L + t*eta.L;\n Y.R = X.R + t*eta.R;\n \n % Numerical conditioning step: A simpler version.\n % We need to ensure that L and R do not have very relative\n % skewed norms.\n \n scaling = norm(X.L, 'fro')/norm(X.R, 'fro');\n scaling = sqrt(scaling);\n Y.L = Y.L / scaling;\n Y.R = Y.R * scaling;\n \n % These are reused in the computation of the gradient and Hessian.\n Y = prepare(Y);\n end\n \n M.exp = @exponential;\n function Y = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n Y = retraction(X, eta, t);\n warning('manopt:fixedrankfactory_2factors:exp', ...\n ['Exponential for fixed rank ' ...\n 'manifold not implemented yet. Used retraction instead.']);\n end\n \n M.hash = @(X) ['z' hashmd5([X.L(:) ; X.R(:)])];\n \n M.rand = @random;\n function X = random()\n % A random point on the total space.\n X.L = randn(m, k);\n X.R = randn(n, k);\n X = prepare(X);\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(X)\n % A random vector in the horizontal space.\n eta.L = randn(m, k);\n eta.R = randn(n, k);\n eta = projection(X, eta);\n nrm = M.norm(X, eta);\n eta.L = eta.L / nrm;\n eta.R = eta.R / nrm;\n end\n \n M.lincomb = @lincomb;\n \n M.zerovec = @(X) struct('L', zeros(m, k),'R', zeros(n, k));\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \n % vec and mat are not isometries, because of the unusual inner metric.\n M.vec = @(X, U) [U.L(:) ; U.R(:)];\n M.mat = @(X, u) struct('L', reshape(u(1:(m*k)), m, k), ...\n 'R', reshape(u((m*k+1):end), n, k));\n M.vecmatareisometries = @() false;\n \nend\n\n% Linear combination of tangent vectors.\nfunction d = lincomb(x, a1, d1, a2, d2) %#ok\n \n if nargin == 3\n d.L = a1*d1.L;\n d.R = a1*d1.R;\n elseif nargin == 5\n d.L = a1*d1.L + a2*d2.L;\n d.R = a1*d1.R + a2*d2.R;\n else\n error('Bad use of fixedrankfactory_2factors.lincomb.');\n end\n \nend\n\n\n\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedrank/fixedrankfactory_2factors_preconditioned.m", "content": "function M = fixedrankfactory_2factors_preconditioned(m, n, k)\n% Manifold of m-by-n matrices of rank k with two factor quotient geometry.\n%\n% function M = fixedrankfactory_2factors_preconditioned(m, n, k)\n%\n% This geometry is tuned to least-squares problems such as low-rank matrix\n% completion with ell-2 loss.\n%\n% A point X on the manifold is represented as a structure with two\n% fields: L and R. The matrices L (m-by-k) and R (n-by-k) are \n% full column-rank matrices such that X = L*R'.\n%\n% Tangent vectors are represented as a structure with two fields: L, R.\n% \n% Please cite the Manopt paper as well as the research paper:\n% @Techreport{mishra2012optimized,\n% Title = {A {R}iemannian geometry for low-rank matrix completion},\n% Author = {Mishra, B. and Adithya Apuroop, K. and Sepulchre, R.},\n% Journal = {Arxiv preprint arXiv:1211.1550},\n% Year = {2012}\n% }\n%\n%\n% See also: fixedrankembeddedfactory fixedrankfactory_2factors fixedrankfactory_3factors_preconditioned\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors:\n% Change log:\n%\n%\tApril 04, 2015 (BM):\n% Cosmetic changes including avoiding storing the inverse of a\n% k-by-k matrix.\n \n \n M.name = @() sprintf('LR''(tuned to least square problems) quotient manifold of %dx%d matrices of rank %d', m, n, k);\n \n M.dim = @() (m+n-k)*k;\n \n \n % Some precomputations at the point X to be used in the inner product \n % (and pretty much everywhere else).\n function X = prepare(X)\n if ~all(isfield(X,{'LtL','RtR'}))\n L = X.L;\n R = X.R;\n X.LtL = L'*L;\n X.RtR = R'*R;\n end\n end\n \n \n % The choice of metric is motivated by symmetry and \n % tuned to least-squares cost function.\n M.inner = @iproduct;\n function ip = iproduct(X, eta, zeta)\n X = prepare(X);\n ip = trace(X.RtR*(eta.L'*zeta.L)) + trace(X.LtL*(eta.R'*zeta.R)); % Scaled metric\n end\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(x, y) error('fixedrankfactory_2factors_preconditioned.dist not implemented yet.');\n \n M.typicaldist = @() 10*k;\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n X = prepare(X);\n \n % Riemannian gradient\n rgrad.L = egrad.L/X.RtR;\n rgrad.R = egrad.R/X.LtL;\n end\n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(X, egrad, ehess, eta)\n X = prepare(X);\n \n % Riemannian gradient.\n rgrad = egrad2rgrad(X, egrad);\n \n % Directional derivative of the Riemannian gradient.\n Hess.L = ehess.L/X.RtR - 2*egrad.L*(X.RtR \\ (symm(eta.R'*X.R) / X.RtR));\n Hess.R = ehess.R/X.LtL - 2*egrad.R*(X.LtL \\ (symm(eta.L'*X.L) / X.LtL));\n \n % We still need a correction factor for the non-constant metric.\n Hess.L = Hess.L + rgrad.L*(symm(eta.R'*X.R)/X.RtR) + eta.L*(symm(rgrad.R'*X.R)/X.RtR) - X.L*(symm(eta.R'*rgrad.R)/X.RtR);\n Hess.R = Hess.R + rgrad.R*(symm(eta.L'*X.L)/X.LtL) + eta.R*(symm(rgrad.L'*X.L)/X.LtL) - X.R*(symm(eta.L'*rgrad.L)/X.LtL);\n \n % Project on the horizontal space.\n Hess = M.proj(X, Hess);\n end\n \n M.proj = @projection;\n function etaproj = projection(X, eta)\n X = prepare(X);\n \n % Projection onto the horizontal space.\n Lambda = 0.5*((eta.R'*X.R)/X.RtR - X.LtL\\(X.L'*eta.L));\n etaproj.L = eta.L + X.L*Lambda;\n etaproj.R = eta.R - X.R*Lambda';\n end\n \n M.tangent = M.proj;\n \n M.tangent2ambient = @(X, eta) eta;\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Y.L = X.L + t*eta.L;\n Y.R = X.R + t*eta.R;\n \n % Numerical conditioning step: a simpler version.\n % We need to ensure that L and R are do not have very relative\n % skewed norms.\n \n scaling = norm(X.L, 'fro')/norm(X.R, 'fro');\n scaling = sqrt(scaling);\n Y.L = Y.L / scaling;\n Y.R = Y.R * scaling;\n \n % These are reused in the computations of gradient and Hessian.\n Y = prepare(Y);\n end\n \n \n M.exp = @exponential;\n function Y = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n Y = retraction(X, eta, t);\n warning('manopt:fixedrankfactory_2factors_preconditioned:exp', ...\n ['Exponential for fixed rank ' ...\n 'manifold not implemented yet. Used retraction instead.']);\n end\n \n M.hash = @(X) ['z' hashmd5([X.L(:) ; X.R(:)])];\n \n M.rand = @random;\n \n function X = random()\n X.L = randn(m, k);\n X.R = randn(n, k);\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(X)\n eta.L = randn(m, k);\n eta.R = randn(n, k);\n eta = projection(X, eta);\n nrm = M.norm(X, eta);\n eta.L = eta.L / nrm;\n eta.R = eta.R / nrm;\n end\n \n M.lincomb = @lincomb;\n \n M.zerovec = @(X) struct('L', zeros(m, k),'R', zeros(n, k));\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \n % vec and mat are not isometries, because of the scaled inner metric.\n M.vec = @(X, U) [U.L(:) ; U.R(:)];\n \n M.mat = @(X, u) struct('L', reshape(u(1:(m*k)), m, k), ...\n 'R', reshape(u((m*k+1):end), n, k));\n \n M.vecmatareisometries = @() false;\n \n % Auxiliary functions\n symm = @(M) .5*(M+M');\nend\n\n% Linear combination of tangent vectors.\nfunction d = lincomb(x, a1, d1, a2, d2) %#ok\n \n if nargin == 3\n d.L = a1*d1.L;\n d.R = a1*d1.R;\n elseif nargin == 5\n d.L = a1*d1.L + a2*d2.L;\n d.R = a1*d1.R + a2*d2.R;\n else\n error('Bad use of fixedrankfactory_2factors_preconditioned.lincomb.');\n end\n \nend\n\n\n\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedrank/fixedrankfactory_2factors_subspace_projection.m", "content": "function M = fixedrankfactory_2factors_subspace_projection(m, n, k)\n% Manifold of m-by-n matrices of rank k with two factor quotient geometry.\n%\n% function M = fixedrankfactory_2factors_subspace_projection(m, n, k)\n%\n% A point X on the manifold is represented as a structure with two\n% fields: L and R. The matrix L (mxk) is orthonormal,\n% while the matrix R (nxk) is a full column-rank\n% matrix such that X = L*R'.\n%\n% Tangent vectors are represented as a structure with two fields: L, R.\n%\n% Note: L is orthonormal, i.e., columns are orthogonal to each other.\n% Such a geometry might be of interest where the left factor has a\n% subspace interpretation. A motivation is in Sections 3.3 and 6.4 of the\n% paper below.\n%\n% Please cite the Manopt paper as well as the research paper:\n% @Article{mishra2014fixedrank,\n% Title = {Fixed-rank matrix factorizations and {Riemannian} low-rank optimization},\n% Author = {Mishra, B. and Meyer, G. and Bonnabel, S. and Sepulchre, R.},\n% Journal = {Computational Statistics},\n% Year = {2014},\n% Number = {3-4},\n% Pages = {591--621},\n% Volume = {29},\n% Doi = {10.1007/s00180-013-0464-z}\n% }\n%\n% See also: fixedrankfactory_2factors fixedrankembeddedfactory fixedrankfactory_2factors_preconditioned\n\n\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors:\n% Change log:\n \n M.name = @() sprintf('LR'' quotient manifold of %dx%d matrices of rank %d', m, n, k);\n \n M.dim = @() (m+n-k)*k;\n \n % Some precomputations at the point X to be used in the inner product (and\n % pretty much everywhere else).\n function X = prepare(X)\n if ~all(isfield(X,{'RtR'}) == 1)\n X.RtR = X.R'*X.R;\n end\n end\n \n % The choice of the metric is motivated by symmetry and scale\n % invariance in the total space.\n M.inner = @iproduct;\n function ip = iproduct(X, eta, zeta)\n X = prepare(X);\n \n ip = eta.L(:).'*zeta.L(:) + trace(X.RtR\\(eta.R'*zeta.R));\n end\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(x, y) error('fixedrankfactory_2factors_subspace_projection.dist not implemented yet.');\n \n M.typicaldist = @() 10*k;\n \n skew = @(X) .5*(X-X');\n symm = @(X) .5*(X+X');\n stiefel_proj = @(L, H) H - L*symm(L'*H);\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n X = prepare(X);\n \n rgrad.L = stiefel_proj(X.L, egrad.L);\n rgrad.R = egrad.R*X.RtR;\n end\n \n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(X, egrad, ehess, eta)\n X = prepare(X);\n \n % Riemannian gradient.\n rgrad = egrad2rgrad(X, egrad);\n \n % Directional derivative of the Riemannian gradient.\n Hess.L = ehess.L - eta.L*symm(X.L'*egrad.L);\n Hess.L = stiefel_proj(X.L, Hess.L);\n \n Hess.R = ehess.R*X.RtR + 2*egrad.R*symm(eta.R'*X.R);\n \n % Correction factor for the non-constant metric on the factor R.\n Hess.R = Hess.R - rgrad.R*(X.RtR\\(symm(X.R'*eta.R))) - eta.R*(X.RtR\\(symm(X.R'*rgrad.R))) + X.R*(X.RtR\\(symm(eta.R'*rgrad.R)));\n \n % Projection onto the horizontal space.\n Hess = M.proj(X, Hess);\n end\n \n \n M.proj = @projection;\n function etaproj = projection(X, eta)\n X = prepare(X);\n \n eta.L = stiefel_proj(X.L, eta.L); % On the tangent space.\n SS = X.RtR;\n AS1 = 2*X.RtR*skew(X.L'*eta.L)*X.RtR;\n AS2 = 2*skew(X.RtR*(X.R'*eta.R));\n AS = skew(AS1 + AS2);\n \n Omega = nested_sylvester(SS,AS);\n etaproj.L = eta.L - X.L*Omega;\n etaproj.R = eta.R - X.R*Omega;\n end\n \n M.tangent = M.proj;\n M.tangent2ambient = @(X, eta) eta;\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Y.L = uf(X.L + t*eta.L);\n Y.R = X.R + t*eta.R;\n \n % These are reused in the computation of the gradient and Hessian.\n Y = prepare(Y);\n end\n \n M.exp = @exponential;\n function R = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n R = retraction(X, eta, t);\n warning('manopt:fixedrankfactory_2factors_subspace_projection:exp', ...\n ['Exponential for fixed rank ' ...\n 'manifold not implemented yet. Lsed retraction instead.']);\n end\n \n M.hash = @(X) ['z' hashmd5([X.L(:) ; X.R(:)])];\n \n M.rand = @random;\n % Factors L lives on Stiefel manifold, hence we will reuse\n % its random generator.\n stiefelm = stiefelfactory(m, k);\n function X = random()\n X.L = stiefelm.rand();\n X.R = randn(n, k);\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(X)\n eta.L = randn(m, k);\n eta.R = randn(n, k);\n eta = projection(X, eta);\n nrm = M.norm(X, eta);\n eta.L = eta.L / nrm;\n eta.R = eta.R / nrm;\n end\n \n M.lincomb = @lincomb;\n \n M.zerovec = @(X) struct('L', zeros(m, k),...\n 'R', zeros(n, k));\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \n % vec and mat are not isometries, because of the scaled inner metric.\n M.vec = @(X, U) [U.L(:) ; U.R(:)];\n M.mat = @(X, u) struct('L', reshape(u(1:(m*k)), m, k), ...\n 'R', reshape(u((m*k+1):end), n, k));\n M.vecmatareisometries = @() false;\n \n \nend\n\n% Linear combination of tangent vectors.\nfunction d = lincomb(x, a1, d1, a2, d2) %#ok\n \n if nargin == 3\n d.L = a1*d1.L;\n d.R = a1*d1.R;\n elseif nargin == 5\n d.L = a1*d1.L + a2*d2.L;\n d.R = a1*d1.R + a2*d2.R;\n else\n error('Bad use of fixedrankfactory_2factors_subspace_projection.lincomb.');\n end\n \nend\n\nfunction A = uf(A)\n [L, unused, R] = svd(A, 0); %#ok\n A = L*R';\nend\n\nfunction omega = nested_sylvester(sym_mat, asym_mat)\n % omega=nested_sylvester(sym_mat,asym_mat)\n % This function solves the system of nested Sylvester equations:\n %\n % X*sym_mat + sym_mat*X = asym_mat\n % Omega*sym_mat+sym_mat*Omega = X\n % Mishra, Meyer, Bonnabel and Sepulchre, 'Fixed-rank matrix factorizations and Riemannian low-rank optimization'\n \n % Uses built-in lyap function, but does not exploit the fact that it's\n % twice the same sym_mat matrix that comes into play.\n \n X = lyap(sym_mat, -asym_mat);\n omega = lyap(sym_mat, -X);\n \nend\n\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedrank/fixedrankfactory_3factors.m", "content": "function M = fixedrankfactory_3factors(m, n, k)\n% Manifold of m-by-n matrices of rank k with polar quotient geometry.\n%\n% function M = fixedrankfactory_3factors(m, n, k)\n%\n% The first-order geometry follows the balanced quotient geometry described \n% in the paper, \n% \"Linear regression under fixed-rank constraints: a Riemannian approach\",\n% G. Meyer, S. Bonnabel and R. Sepulchre, ICML 2011.\n%\n% Paper link: http://www.icml-2011.org/papers/350_icmlpaper.pdf.\n%\n% The second-order geometry follows from the paper\n% \"Fixed-rank matrix factorizations and Riemannian low-rank optimization\",\n% B. Mishra, R. Meyer, S. Bonnabel and R. Sepulchre,\n% Computational Statistics, 29(3 - 4), pp. 591 - 621, 2014.\n%\n% A point X on the manifold is represented as a structure with three\n% fields: L, S and R. The matrices L (mxk) and R (nxk) are orthonormal,\n% while the matrix S (kxk) is a symmetric positive definite full rank\n% matrix.\n%\n% Tangent vectors are represented as a structure with three fields: L, S\n% and R.\n%\n% \n% For first-order geometry, please cite the Manopt paper as well as the research paper:\n% @InProceedings{meyer2011linear,\n% Title = {Linear regression under fixed-rank constraints: a {R}iemannian approach},\n% Author = {Meyer, G. and Bonnabel, S. and Sepulchre, R.},\n% Booktitle = {{28th International Conference on Machine Learning}},\n% Year = {2011},\n% Organization = {{ICML}}\n% }\n% For second-order geometry, please cite the Manopt paper as well as the research paper:\n% @Article{mishra2014fixedrank,\n% Title = {Fixed-rank matrix factorizations and {Riemannian} low-rank optimization},\n% Author = {Mishra, B. and Meyer, G. and Bonnabel, S. and Sepulchre, R.},\n% Journal = {Computational Statistics},\n% Year = {2014},\n% Number = {3-4},\n% Pages = {591--621},\n% Volume = {29},\n% Doi = {10.1007/s00180-013-0464-z}\n% }\n%\n%\n% See also fixedrankembeddedfactory fixedrankfactory_2factors fixedrankfactory_3factors_preconditioned\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors:\n% Change log:\n \n M.name = @() sprintf('LSR'' quotient manifold of %dx%d matrices of rank %d', m, n, k);\n \n M.dim = @() (m+n-k)*k;\n \n % Choice of the metric on the orthnormal space is motivated by the symmetry present in the\n % space. The metric on the positive definite space is its natural metric.\n M.inner = @(X, eta, zeta) eta.L(:).'*zeta.L(:) + eta.R(:).'*zeta.R(:) ...\n + trace( (X.S\\eta.S) * (X.S\\zeta.S) );\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(x, y) error('fixedrankfactory_3factors.dist not implemented yet.');\n \n M.typicaldist = @() 10*k;\n \n skew = @(X) .5*(X-X');\n symm = @(X) .5*(X+X');\n stiefel_proj = @(L, H) H - L*symm(L'*H);\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n rgrad.L = stiefel_proj(X.L, egrad.L);\n rgrad.S = X.S*symm(egrad.S)*X.S;\n rgrad.R = stiefel_proj(X.R, egrad.R);\n end\n \n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(X, egrad, ehess, eta)\n \n % Riemannian gradient for the factor S.\n rgrad.S = X.S*symm(egrad.S)*X.S;\n \n % Directional derivatives of the Riemannian gradient.\n Hess.L = ehess.L - eta.L*symm(X.L'*egrad.L);\n Hess.L = stiefel_proj(X.L, Hess.L);\n \n Hess.R = ehess.R - eta.R*symm(X.R'*egrad.R);\n Hess.R = stiefel_proj(X.R, Hess.R);\n \n Hess.S = X.S*symm(ehess.S)*X.S + 2*symm(eta.S*symm(egrad.S)*X.S);\n \n % Correction factor for the non-constant metric on the factor S.\n Hess.S = Hess.S - symm(eta.S*(X.S\\rgrad.S));\n \n % Projection onto the horizontal space.\n Hess = M.proj(X, Hess);\n end\n \n \n M.proj = @projection;\n function etaproj = projection(X, eta)\n % First, projection onto the tangent space of the total space.\n eta.L = stiefel_proj(X.L, eta.L);\n eta.R = stiefel_proj(X.R, eta.R);\n eta.S = symm(eta.S);\n \n % Then, projection onto the horizontal space.\n SS = X.S*X.S;\n AS = X.S*(skew(X.L'*eta.L) + skew(X.R'*eta.R) - 2*skew(X.S\\eta.S))*X.S;\n omega = lyap(SS, -AS);\n \n etaproj.L = eta.L - X.L*omega;\n etaproj.S = eta.S - (X.S*omega - omega*X.S);\n etaproj.R = eta.R - X.R*omega;\n end\n \n M.tangent = M.proj;\n M.tangent2ambient = @(X, eta) eta;\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n L = chol(X.S);\n Y.S = L'*expm(L'\\(t*eta.S)/L)*L;\n Y.L = uf(X.L + t*eta.L);\n Y.R = uf(X.R + t*eta.R);\n end\n \n M.exp = @exponential;\n function Y = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Y = retraction(X, eta, t);\n warning('manopt:fixedrankfactory_3factors:exp', ...\n ['Exponential for fixed rank ' ...\n 'manifold not implemented yet. Lsed retraction instead.']);\n end\n \n M.hash = @(X) ['z' hashmd5([X.L(:) ; X.S(:) ; X.R(:)])];\n \n M.rand = @random;\n % Factors L and R are on Stiefel manifolds, hence we reuse\n % their random generators.\n stiefelm = stiefelfactory(m, k);\n stiefeln = stiefelfactory(n, k);\n function X = random()\n X.L = stiefelm.rand();\n X.R = stiefeln.rand();\n X.S = diag(1+rand(k, 1));\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(X)\n % A random vector on the horizontal space.\n eta.L = randn(m, k);\n eta.R = randn(n, k);\n eta.S = randn(k, k);\n eta = projection(X, eta);\n nrm = M.norm(X, eta);\n eta.L = eta.L / nrm;\n eta.R = eta.R / nrm;\n eta.S = eta.S / nrm;\n end\n \n M.lincomb = @lincomb;\n \n M.zerovec = @(X) struct('L', zeros(m, k), 'S', zeros(k, k), ...\n 'R', zeros(n, k));\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \n % vec and mat are not isometries, because of the scaled inner metric.\n M.vec = @(X, U) [U.L(:) ; U.S(:); U.R(:)];\n M.mat = @(X, u) struct('L', reshape(u(1:(m*k)), m, k), ...\n 'S', reshape(u((m*k+1): m*k + k*k), k, k), ...\n 'R', reshape(u((m*k+ k*k + 1):end), n, k));\n M.vecmatareisometries = @() false;\n \nend\n\n% Linear combination of tangent vectors.\nfunction d = lincomb(x, a1, d1, a2, d2) %#ok\n \n if nargin == 3\n d.L = a1*d1.L;\n d.R = a1*d1.R;\n d.S = a1*d1.S;\n elseif nargin == 5\n d.L = a1*d1.L + a2*d2.L;\n d.R = a1*d1.R + a2*d2.R;\n d.S = a1*d1.S + a2*d2.S;\n else\n error('Bad use of fixedrankfactory_3factors.lincomb.');\n end\n \nend\n\nfunction A = uf(A)\n [L, unused, R] = svd(A, 0); %#ok\n A = L*R';\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedrank/fixedrankfactory_3factors_preconditioned.m", "content": "function M = fixedrankfactory_3factors_preconditioned(m, n, k)\n% Manifold of m-by-n matrices of rank k with three factor quotient geometry.\n%\n% function M = fixedrankfactory_3factors_preconditioned(m, n, k)\n%\n% This geometry is tuned to least squares problems such as low-rank matrix\n% completion with ell-2 loss.\n%\n% A point X on the manifold is represented as a structure with three\n% fields: L, S and R. The matrices L (mxk) and R (nxk) are orthonormal,\n% while the matrix S (kxk) is a full rank matrix such that X = L*S*R'.\n%\n% Tangent vectors are represented as a structure with three fields: L, S\n% and R.\n%\n% Please cite the Manopt paper as well as the research paper:\n% @InProceedings{mishra2014r3mc,\n% Title = {{R3MC}: A {R}iemannian three-factor algorithm for low-rank matrix completion},\n% Author = {Mishra, B. and Sepulchre, R.},\n% Booktitle = {{53rd IEEE Conference on Decision and Control}},\n% Year = {2014},\n% Organization = {{IEEE CDC}}\n% }\n%\n%\n% See also: fixedrankfactory_3factors fixedrankfactory_2factors_preconditioned\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors:\n% Change log:\n%\n%\tApril 04, 2015 (BM):\n% Cosmetic changes including avoiding storing the inverse of a kxk matrix.\n\n \n M.name = @() sprintf('LSR'' (tuned for least square problems) quotient manifold of %dx%d matrices of rank %d', m, n, k);\n \n M.dim = @() (m+n-k)*k;\n \n % Some precomputations at the point X that are to be used in the inner product (and\n % pretty much everywhere else).\n function X = prepare(X)\n if ~all(isfield(X,{'StS','SSt'}) == 1)\n X.SSt = X.S*X.S';\n X.StS = X.S'*X.S;\n end\n end\n \n % The choice of metric is motivated by symmetry and tuned to least square\n % objective function.\n M.inner = @iproduct;\n function ip = iproduct(X, eta, zeta)\n X = prepare(X);\n \n ip = trace(X.SSt*(eta.L'*zeta.L)) + trace(X.StS*(eta.R'*zeta.R)) ...\n + trace(eta.S'*zeta.S);\n end\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(x, y) error('fixedrankfactory_3factors_preconditioned.dist not implemented yet.');\n \n M.typicaldist = @() 10*k;\n \n skew = @(X) .5*(X-X');\n symm = @(X) .5*(X+X');\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n X = prepare(X);\n \n SSL = X.SSt;\n ASL = 2*symm(SSL*(egrad.S*X.S'));\n \n SSR = X.StS;\n ASR = 2*symm(SSR*(egrad.S'*X.S));\n \n [BL, BR] = tangent_space_lyap(X.S, ASL, ASR); % It computes the solution without calling Matlab's Lyap.\n \n rgrad.L = (egrad.L - X.L*BL)/X.SSt;\n rgrad.R = (egrad.R - X.R*BR)/X.StS;\n rgrad.S = egrad.S;\n \n % Debug\n % BL1 = lyap(SSL, -ASL); % Alternate way\n % BR1 = lyap(SSR, -ASR);\n % norm(skew(X.SSt*(rgrad.L'*X.L) + rgrad.S*X.S'), 'fro')\n % norm(skew(X.StS*(rgrad.R'*X.R) - X.S'*rgrad.S), 'fro')\n \n end\n \n \n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(X, egrad, ehess, eta)\n X = prepare(X);\n \n % Riemannian gradient.\n SSL = X.SSt;\n ASL = 2*symm(SSL*(egrad.S*X.S'));\n SSR = X.StS;\n ASR = 2*symm(SSR*(egrad.S'*X.S));\n [BL, BR] = tangent_space_lyap(X.S, ASL, ASR);\n \n rgrad.L = (egrad.L - X.L*BL)/X.SSt;\n rgrad.R = (egrad.R - X.R*BR)/X.StS;\n rgrad.S = egrad.S;\n \n % Directional derivative of the Riemannian gradient.\n ASLdot = 2*symm((2*symm(X.S*eta.S')*(egrad.S*X.S')) + X.SSt*(ehess.S*X.S' + egrad.S*eta.S')) - 4*symm(symm(eta.S*X.S')*BL);\n ASRdot = 2*symm((2*symm(X.S'*eta.S)*(egrad.S'*X.S)) + X.StS*(ehess.S'*X.S + egrad.S'*eta.S)) - 4*symm(symm(eta.S'*X.S)*BR);\n \n % SSLdot = X.SSt;\n % SSRdot = X.StS;\n % BLdot = lyap(SSLdot, -ASLdot);\n % BRdot = lyap(SSRdot, -ASRdot);\n \n [BLdot, BRdot] = tangent_space_lyap(X.S, ASLdot, ASRdot);\n \n Hess.L = (ehess.L - eta.L*BL - X.L*BLdot - 2*rgrad.L*symm(eta.S*X.S'))/X.SSt;\n Hess.R = (ehess.R - eta.R*BR - X.R*BRdot - 2*rgrad.R*symm(eta.S'*X.S))/X.StS;\n Hess.S = ehess.S;\n \n \n \n % BM: Till this, everything seems correct.\n % We still need a correction factor for the non-constant metric\n % that is imposed.\n % The computation of the correction factor owes itself to the Koszul formula.\n % This corresponds to the Riemannian connection in the Euclidean space with the\n % scaled metric.\n Hess.L = Hess.L + (eta.L*symm(rgrad.S*X.S') + rgrad.L*symm(eta.S*X.S'))/X.SSt;\n Hess.R = Hess.R + (eta.R*symm(rgrad.S'*X.S) + rgrad.R*symm(eta.S'*X.S))/X.StS;\n Hess.S = Hess.S - symm(rgrad.L'*eta.L)*X.S - X.S*symm(rgrad.R'*eta.R);\n \n % The Riemannian connection on the quotient space is the\n % projection of the Riemmian connection in the ambient space onto the tangent space of the total space and\n % then onto the horizontal space. \n % This is accomplished by the following operation.\n Hess = M.proj(X, Hess);\n \n % Debug\n % norm(skew(X.SSt*(Hess.L'*X.L) + Hess.S*X.S'))\n % norm(skew(X.StS*(Hess.R'*X.R) - X.S'*Hess.S))\n \n end\n \n \n \n \n M.proj = @projection;\n function etaproj = projection(X, eta)\n X = prepare(X);\n \n % First, projection onto the tangent space of the total space.\n SSL = X.SSt;\n ASL = 2*symm(X.SSt*(X.L'*eta.L)*X.SSt);\n BL = lyap(SSL, -ASL);\n eta.L = eta.L - X.L*(BL/X.SSt);\n \n SSR = X.StS;\n ASR = 2*symm(X.StS*(X.R'*eta.R)*X.StS);\n BR = lyap(SSR, -ASR);\n eta.R = eta.R - X.R*(BR/X.StS);\n \n % Project onto the horizontal space\n PU = skew((X.L'*eta.L)*X.SSt) + skew(X.S*eta.S');\n PV = skew((X.R'*eta.R)*X.StS) + skew(X.S'*eta.S);\n [Omega1, Omega2] = coupled_lyap(X.S, PU, PV);\n % norm(2*skew(Omega1*X.SSt) - PU -(X.S*Omega2*X.S'),'fro' )\n % norm(2*skew(Omega2*X.StS) - PV -(X.S'*Omega1*X.S),'fro' )\n %\n \n etaproj.L = eta.L - (X.L*Omega1);\n etaproj.S = eta.S - (X.S*Omega2 - Omega1*X.S) ;\n etaproj.R = eta.R - (X.R*Omega2);\n \n \n % Debug\n % norm(skew(X.SSt*(etaproj.L'*X.L) + etaproj.S*X.S'))\n % norm(skew(X.StS*(etaproj.R'*X.R) - X.S'*etaproj.S))\n %\n % norm(skew(X.SSt*(etaproj.L'*X.L) - X.S*etaproj.S'))\n % norm(skew(X.StS*(etaproj.R'*X.R) + etaproj.S'*X.S))\n \n end\n \n \n M.tangent = M.proj;\n M.tangent2ambient = @(X, eta) eta;\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n Y.S = (X.S + t*eta.S);\n Y.L = uf((X.L + t*eta.L));\n Y.R = uf((X.R + t*eta.R));\n \n Y = prepare(Y);\n end\n \n M.exp = @exponential;\n function Y = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Y = retraction(X, eta, t);\n warning('manopt:fixedrankfactory_3factors_preconditioned:exp', ...\n ['Exponential for fixed rank ' ...\n 'manifold not implemented yet. Used retraction instead.']);\n end\n \n M.hash = @(X) ['z' hashmd5([X.L(:) ; X.S(:) ; X.R(:)])];\n \n M.rand = @random;\n % Factors L and R live on Stiefel manifolds, hence we will reuse\n % their random generator.\n stiefelm = stiefelfactory(m, k);\n stiefeln = stiefelfactory(n, k);\n function X = random()\n X.L = stiefelm.rand();\n X.R = stiefeln.rand();\n X.S = diag(1+rand(k, 1));\n \n X = prepare(X);\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(X)\n % A random vector on the horizontal space\n eta.L = randn(m, k);\n eta.R = randn(n, k);\n eta.S = randn(k, k);\n eta = projection(X, eta);\n nrm = M.norm(X, eta);\n eta.L = eta.L / nrm;\n eta.R = eta.R / nrm;\n eta.S = eta.S / nrm;\n end\n \n M.lincomb = @lincomb;\n \n M.zerovec = @(X) struct('L', zeros(m, k), 'S', zeros(k, k), ...\n 'R', zeros(n, k));\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \n % vec and mat are not isometries, because of the unusual inner metric.\n M.vec = @(X, U) [U.L(:) ; U.S(:); U.R(:)];\n M.mat = @(X, u) struct('L', reshape(u(1:(m*k)), m, k), ...\n 'S', reshape(u((m*k+1): m*k + k*k), k, k), ...\n 'R', reshape(u((m*k+ k*k + 1):end), n, k));\n M.vecmatareisometries = @() false;\n \nend\n\n% Linear combination of tangent vectors\nfunction d = lincomb(x, a1, d1, a2, d2) %#ok\n \n if nargin == 3\n d.L = a1*d1.L;\n d.R = a1*d1.R;\n d.S = a1*d1.S;\n elseif nargin == 5\n d.L = a1*d1.L + a2*d2.L;\n d.R = a1*d1.R + a2*d2.R;\n d.S = a1*d1.S + a2*d2.S;\n else\n error('Bad use of fixedrankfactory_3factors_preconditioned.lincomb.');\n end\n \nend\n\nfunction A = uf(A)\n [L, unused, R] = svd(A, 0); %#ok\n A = L*R';\nend\n\nfunction[BU, BV] = tangent_space_lyap(R, E, F)\n % We intent to solve a linear system RR^T BU + BU RR^T = E\n % R^T R BV + BV R^T R = F\n % for BU and BV.\n %\n % This can be solved using two calls to the Matlab's lyap.\n % However, we can still have a more efficient implementation\n % that does not require the full functionaliyt of Matlab's lyap.\n \n [U, Sigma, V] = svd(R);\n E_mod = U'*E*U;\n F_mod = V'*F*V;\n b1 = E_mod(:);\n b2 = F_mod(:);\n \n r = size(Sigma, 1);\n sig = diag(Sigma); % all the singular values in a vector\n sig1 = sig*ones(1, r); % columns repeat\n sig1t = sig1'; % rows repeat\n s1 = sig1(:);\n s2 = sig1t(:);\n \n % The block elements\n a = s1.^2 + s2.^2; % a column vector\n \n % Solve the linear system of equations\n cu = b1./a; %a.\\b1;\n cv = b2./a; %a.\\b2;\n \n % Matricize\n CU = reshape(cu, r, r);\n CV = reshape(cv, r, r);\n \n % Do the similarity transforms\n BU = U*CU*U';\n BV = V*CV*V';\n \n % %% Debug\n %\n % norm(R*R'*BU + BU*R*R' - E, 'fro');\n % norm((Sigma.^2)*CU + CU*(Sigma.^2) - E_mod, 'fro');\n % norm(a.*cu - b1, 'fro');\n %\n % norm(R'*R*BV + BV*R'*R - F, 'fro');\n %\n % BU1 = lyap(R*R', - E);\n % norm(R*R'*BU1 + BU1*R*R' - E, 'fro');\n %\n % BV1 = lyap(R'*R, - F);\n % norm(R'*R*BV1 + BV1*R'*R - F, 'fro');\n %\n % % as accurate as the lyap\n % norm(BU - BU1, 'fro')\n % norm(BV - BV1, 'fro')\nend\n\n\n\nfunction[Omega1, Omega2] = coupled_lyap(R, E, F)\n % We intent to solve the coupled system of Lyapunov equations\n %\n % RR^T Omega1 + Omega1 RR^T - R Omega2 R^T = E\n % R^T R Omega2 + Omega1 R^T R - R^T Omega2 R = F,\n %\n % for Omega1 and Omega2, both are skew symmetric matrices.\n %\n % Below is an efficient implementation\n \n [U, Sigma, V] = svd(R);\n E_mod = U'*E*U;\n F_mod = V'*F*V;\n b1 = E_mod(:);\n b2 = F_mod(:);\n \n r = size(Sigma, 1);\n sig = diag(Sigma); % All the singular values in a vector\n sig1 = sig*ones(1, r); % Columns repeat\n sig1t = sig1'; % Rows repeat\n s1 = sig1(:);\n s2 = sig1t(:);\n \n % The block elements\n a = s1.^2 + s2.^2; % A column vector\n c = s1.*s2;\n \n % Solve directly using the formula\n % A = diag(a);\n % C = diag(c);\n % Y1_sol = (A*(C\\A) - C) \\ (b2 + A*(C\\b1));\n % Y2_sol = A\\(b2 + C*Y1_sol);\n \n Y1_sol = (b2 + (a./c).*b1) ./ ((a.^2)./c - c);\n Y2_sol = (b2 + c.*Y1_sol)./a;\n \n % Matricize\n Omega1 = reshape(Y1_sol, r, r);\n Omega2 = reshape(Y2_sol, r, r);\n \n % Do the similarity transforms\n Omega1 = U*Omega1*U';\n Omega2 = V*Omega2*V';\n \n % %% Debug: whether we have the right solution.\n % norm(R*R'*Omega1 + Omega1*R*R' - R*Omega2*R' - E, 'fro')\n % norm(R'*R*Omega2 + Omega2*R'*R - R'*Omega1*R - F, 'fro')\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedranktensors/fixedrankfactory_tucker_preconditioned.m", "content": "function M = fixedrankfactory_tucker_preconditioned(tensor_size, tensor_rank)\n% Manifold of fixed multilinear rank tensors in Tucker format.\n%\n% function M = fixedrankfactory_tucker_preconditioned(tensor_size, tensor_rank)\n%\n% n1 = tensor_size(1);\n% n2 = tensor_size(2);\n% n3 = tensor_size(3);\n% r1 = tensor_rank(1);\n% r2 = tensor_rank(2);\n% r3 = tensor_rank(3);\n%\n% A point X on the manifold is represented as a structure with four\n% fields: U1, U2, U3 and G. The matrices U1 (n1-by-r1), U2 (n2-by-r2),\n% and U3 (n3-by-r3) are orthogonal matrices. G (r1-by-r2-by-r3) is a \n% multidimensional array.\n%\n% Tangent vectors are represented as a structure with four fields: \n% U1, U2, U3, and G.\n%\n% We exploit the quotient nature of Tucker decompositions to impose a\n% scaled inner product on the manifold. This suits least-squares problems.\n% For details, refer to the technical report:\n% \"{R}iemannian preconditioning for tensor completion\",\n% H. Kasai and B. Mishra, Arxiv preprint arXiv:1506.02159, 2015.\n%\n% Paper link: http://arxiv.org/abs/1506.02159.\n%\n% Please cite the Manopt paper as well as the research paper:\n% @TechReport{kasai2015precon,\n% Title = {{R}iemannian preconditioning for tensor completion},\n% Author = {Kasai, H. and Mishra, B.},\n% Journal = {Arxiv preprint arXiv:1506.02159},\n% Year = {2015}\n% }\n\n% Original authors: Hiroyuki Kasai and Bamdev Mishra, June 5, 2015.\n% Contributors: \n% Change log:\n\n if length(tensor_rank) > 3\n error('Bad usage of fixedrankfactory_tucker_preconditioned. Currently, only handles 3-order tensors.');\n end\n \n % Tensor size\n n1 = tensor_size(1);\n n2 = tensor_size(2);\n n3 = tensor_size(3);\n \n % Core size or multilinear rank\n r1 = tensor_rank(1);\n r2 = tensor_rank(2);\n r3 = tensor_rank(3);\n \n \n speyer1 = speye(r1); % Sparse version of identity that is used in M.proj\n speyer2 = speye(r2);\n speyer3 = speye(r3);\n \n\n M.name = @() sprintf('G x U1 x U2 x U3 quotient Tucker manifold of %d-by-%d-by-%d tensor of rank %d-by-%d-by-%d.', n1, n2, n3, r1, r2, r3);\n \n M.dim = @() n1*r1-r1^2 + n2*r2-r2^2 + n3*r3-r3^2 + r1*r2*r3;\n \n % Some precomputations at point X to be used in the inner product (and\n % pretty much everywhere else)\n function X = prepare(X)\n if ~all(isfield(X,{'G1G1t','G1',...\n 'G2G2t','G2', ...\n 'G3G3t','G3'}) == 1)\n \n X.G1 = reshape(X.G, r1, r2*r3);\n X.G1G1t = X.G1*X.G1'; % Positive definite \n \n \n X.G2 = reshape(permute(X.G, [2 1 3]), r2, r1*r3); \n X.G2G2t = X.G2*X.G2'; % Positive definite \n \n \n X.G3 = reshape(permute(X.G, [3 1 2]), r3, r1*r2); \n X.G3G3t = X.G3*X.G3'; % Positive definite \n end\n \n \n end\n \n % Choice of metric is motivated by symmetry and tuned to least-squares\n % cost function\n M.inner = @iproduct;\n function ip = iproduct(X, eta, zeta)\n X = prepare(X);\n ip = trace(X.G1G1t*(eta.U1'*zeta.U1)) ...\n + trace(X.G2G2t*(eta.U2'*zeta.U2)) ...\n + trace(X.G3G3t*(eta.U3'*zeta.U3)) ...\n + (eta.G(:)'*zeta.G(:));\n end\n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(x, y) error('fixedrankfactory_tucker_preconditioned.dist not implemented yet.');\n \n M.typicaldist = @() 10*n1*r1; % BM: To do \n \n skew = @(X) .5*(X-X');\n symm = @(X) .5*(X+X');\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n X = prepare(X); % Reuse already computed terms\n \n SSU1 = X.G1G1t;\n ASU1 = 2*symm(SSU1*(X.U1' * egrad.U1));\n \n SSU2 = X.G2G2t;\n ASU2 = 2*symm(SSU2*(X.U2' * egrad.U2));\n \n SSU3 = X.G3G3t;\n ASU3 = 2*symm(SSU3*(X.U3' * egrad.U3));\n \n \n BU1 = lyap(SSU1, -ASU1);\n BU2 = lyap(SSU2, -ASU2);\n BU3 = lyap(SSU3, -ASU3);\n \n % The lyap solutions ensure that the Riemannian gradient rgrad \n % is now on the tangent space. From the Riemannian submersion \n % theory, it also belongs to the horizontal space. Therefore,\n % no need to further project it on the horizontal space.\n \n rgrad.U1 = (egrad.U1 - X.U1*BU1)/X.G1G1t;\n rgrad.U2 = (egrad.U2 - X.U2*BU2)/X.G2G2t;\n rgrad.U3 = (egrad.U3 - X.U3*BU3)/X.G3G3t;\n rgrad.G = egrad.G;\n\n \n end\n \n \n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(X, egrad, ehess, eta) \n X = prepare(X); % Reuse already computed terms\n \n % Riemannian gradient\n SSU1 = X.G1G1t;\n ASU1 = 2*symm(SSU1*(X.U1' * egrad.U1));\n SSU2 = X.G2G2t;\n ASU2 = 2*symm(SSU2*(X.U2' * egrad.U2));\n SSU3 = X.G3G3t;\n ASU3 = 2*symm(SSU3*(X.U3' * egrad.U3));\n \n BU1 = lyap(SSU1, -ASU1);\n BU2 = lyap(SSU2, -ASU2);\n BU3 = lyap(SSU3, -ASU3);\n \n rgrad.U1 = (egrad.U1 - X.U1*BU1)/X.G1G1t;\n rgrad.U2 = (egrad.U2 - X.U2*BU2)/X.G2G2t;\n rgrad.U3 = (egrad.U3 - X.U3*BU3)/X.G3G3t;\n rgrad.G = egrad.G;\n \n % Directional derivative of Riemannian gradient\n \n eta_G1 = reshape(eta.G, r1, r2*r3); % double(tenmat(eta.G,1));\n eta_G2 = reshape(permute(eta.G, [2 1 3]), r2, r1*r3); % double(tenmat(eta.G,2));\n eta_G3 = reshape(permute(eta.G, [3 1 2]), r3, r1*r2); % double(tenmat(eta.G,3));\n egrad_G1 = reshape(egrad.G, r1, r2*r3); % double(tenmat(egrad.G,1));\n egrad_G2 = reshape(permute(egrad.G, [2 1 3]), r2, r1*r3); % double(tenmat(egrad.G,2));\n egrad_G3 = reshape(permute(egrad.G, [3 1 2]), r3, r1*r2); % double(tenmat(egrad.G,3));\n ehess_G1 = reshape(ehess.G, r1, r2*r3); % double(tenmat(ehess.G,1));\n ehess_G2 = reshape(permute(ehess.G, [2 1 3]), r2, r1*r3); % double(tenmat(ehess.G,2));\n ehess_G3 = reshape(permute(ehess.G, [3 1 2]), r3, r1*r2); % double(tenmat(ehess.G,3));\n rgrad_G1 = reshape(rgrad.G, r1, r2*r3); % double(tenmat(rgrad.G,1));\n rgrad_G2 = reshape(permute(rgrad.G, [2 1 3]), r2, r1*r3); % double(tenmat(rgrad.G,2));\n rgrad_G3 = reshape(permute(rgrad.G, [3 1 2]), r3, r1*r2); % double(tenmat(rgrad.G,3));\n \n ASU1dot = 2*symm((2*symm(X.G1*eta_G1')*(egrad_G1*X.G1')) + X.G1G1t*(ehess_G1*X.G1' + egrad_G1*eta_G1')) - 4*symm(symm(eta_G1*X.G1')*BU1);\n ASU2dot = 2*symm((2*symm(X.G2*eta_G2')*(egrad_G2*X.G2')) + X.G2G2t*(ehess_G2*X.G2' + egrad_G2*eta_G2')) - 4*symm(symm(eta_G2*X.G2')*BU2);\n ASU3dot = 2*symm((2*symm(X.G3*eta_G3')*(egrad_G3*X.G3')) + X.G3G3t*(ehess_G3*X.G3' + egrad_G3*eta_G3')) - 4*symm(symm(eta_G3*X.G3')*BU3);\n \n \n SSU1dot = X.G1G1t;\n SSU2dot = X.G2G2t;\n SSU3dot = X.G3G3t;\n BU1dot = lyap(SSU1dot, -ASU1dot);\n BU2dot = lyap(SSU2dot, -ASU2dot);\n BU3dot = lyap(SSU3dot, -ASU3dot);\n \n \n Hess.U1 = (ehess.U1 - eta.U1*BU1 - X.U1*BU1dot - 2*rgrad.U1*symm(eta_G1*X.G1'))/X.G1G1t;\n Hess.U2 = (ehess.U2 - eta.U2*BU2 - X.U2*BU2dot - 2*rgrad.U2*symm(eta_G2*X.G2'))/X.G2G2t;\n Hess.U3 = (ehess.U3 - eta.U3*BU3 - X.U3*BU3dot - 2*rgrad.U3*symm(eta_G3*X.G3'))/X.G3G3t;\n Hess.G = ehess.G;\n \n \n \n % BM: we need a correction factor for the non-constant metric\n % The correction factor owes itself to the Koszul formula.\n % This is the Riemannian connection in the Euclidean space with the\n % scaled metric.\n \n \n Hess.U1 = Hess.U1 + (eta.U1*symm(rgrad_G1*X.G1') + rgrad.U1*symm(eta_G1*X.G1'))/X.G1G1t;\n Hess.U2 = Hess.U2 + (eta.U2*symm(rgrad_G2*X.G2') + rgrad.U2*symm(eta_G2*X.G2'))/X.G2G2t;\n Hess.U3 = Hess.U3 + (eta.U3*symm(rgrad_G3*X.G3') + rgrad.U3*symm(eta_G3*X.G3'))/X.G3G3t;\n Hess.G = Hess.G - permute(reshape(symm(rgrad.U1'*eta.U1)*X.G1,r1,r2,r3), [1 2 3]) ...\n - permute(reshape(symm(rgrad.U2'*eta.U2)*X.G2,r2,r1,r3), [2 1 3]) ...\n - permute(reshape(symm(rgrad.U3'*eta.U3)*X.G3,r3,r1,r2), [2 3 1]);\n \n % The Riemannian connection on the quotient space is the\n % projection on the tangent space of the total space and then onto the horizontal\n % space. This is accomplished with the following operation.\n \n Hess = M.proj(X, Hess);\n \n \n end\n \n \n \n \n M.proj = @projection;\n function etaproj = projection(X, eta)\n X = prepare(X); % Reuse already computed terms\n \n % First, projection onto tangent space of total space\n SSU1 = X.G1G1t;\n ASU1 = 2*symm(X.G1G1t*(X.U1'*eta.U1)*X.G1G1t);\n BU1 = lyap(SSU1, -ASU1);\n eta.U1 = eta.U1 - X.U1*(BU1/X.G1G1t);\n \n SSU2 = X.G2G2t;\n ASU2 = 2*symm(X.G2G2t*(X.U2'*eta.U2)*X.G2G2t);\n BU2 = lyap(SSU2, -ASU2);\n eta.U2 = eta.U2 - X.U2*(BU2/X.G2G2t);\n \n SSU3 = X.G3G3t;\n ASU3 = 2*symm(X.G3G3t*(X.U3'*eta.U3)*X.G3G3t);\n BU3 = lyap(SSU3, -ASU3);\n eta.U3 = eta.U3 - X.U3*(BU3/X.G3G3t);\n \n\n eta_G1 = reshape(eta.G, r1, r2*r3); \n eta_G2 = reshape(permute(eta.G, [2 1 3]), r2, r1*r3); \n eta_G3 = reshape(permute(eta.G, [3 1 2]), r3, r1*r2);\n \n \n % Project onto the horizontal space.\n PU1 = skew((X.U1'*eta.U1)*X.G1G1t) + skew(X.G1*eta_G1');\n PU2 = skew((X.U2'*eta.U2)*X.G2G2t) + skew(X.G2*eta_G2');\n PU3 = skew((X.U3'*eta.U3)*X.G3G3t) + skew(X.G3*eta_G3');\n \n % Calculate Omega1, Omega2, Omega3 that are required in finding the\n % horizontal component. \n % We use the Matlab's pcg function to solve the system efficiently.\n % We exploit the structure by designing a good preconditioner as well.\n % The preconditioner takes the block positive definite part of the\n % linear system.\n \n % Options for PCG\n tol_omegax_pcg = 1e-6; % BM: standard tolerance as suggested in PCG.\n max_iterations_pcg = 15;% BM: fix this to 15 for simulations. In practice, it requires 7 to 10 iteraions.\n \n % Preconditioner for PCG\n M1 = kron(speyer1,SSU1) + kron(SSU1, speyer1);\n M2 = kron(speyer2,SSU2) + kron(SSU2, speyer2);\n M3 = kron(speyer3,SSU3) + kron(SSU3, speyer3);\n \n Mprecon_pcg = sparse(zeros(r1^2 + r2^2 + r3^2));\n Mprecon_pcg(1 : r1^2, 1 : r1^2 ) = M1;\n Mprecon_pcg(1 + r1^2 : r1^2 + r2^2, 1 + r1^2 : r1^2 + r2^2) = M2;\n Mprecon_pcg(1 + r1^2 + r2^2 : end, 1 + r1^2 + r2^2 : end) = M3;\n \n % Call PCG\n [Omegaxsol, unused] = pcg(@compute_residual, [PU1(:); PU2(:); PU3(:)], tol_omegax_pcg, max_iterations_pcg, Mprecon_pcg);\n \n Omega1 = reshape(Omegaxsol(1:r1^2), r1, r1);\n Omega2 = reshape(Omegaxsol(1 + r1^2 : r1^2 + r2^2), r2, r2);\n Omega3 = reshape(Omegaxsol(1 + r1^2 + r2^2 : end), r3, r3);\n \n function AOmegax = compute_residual(Omegax)\n Omegax1 = reshape(Omegax(1:r1^2), r1, r1);\n Omegax2 = reshape(Omegax(1 + r1^2 : r1^2 + r2^2), r2, r2);\n Omegax3 = reshape(Omegax(1 + r1^2 + r2^2 : end), r3, r3);\n \n OffsetU1 = X.G1*((kron(speyer3,Omegax2) + kron(Omegax3, speyer2))*X.G1');\n OffsetU2 = X.G2*((kron(speyer3,Omegax1) + kron(Omegax3, speyer1))*X.G2');\n OffsetU3 = X.G3*((kron(speyer2,Omegax1) + kron(Omegax2, speyer1))*X.G3');\n \n residual1 = Omegax1*SSU1 + SSU1*Omegax1 - OffsetU1;\n residual2 = Omegax2*SSU2 + SSU2*Omegax2 - OffsetU2;\n residual3 = Omegax3*SSU3 + SSU3*Omegax3 - OffsetU3;\n \n AOmegax = [residual1(:); residual2(:); residual3(:)];\n end\n \n \n % Calculate projection along U1, U2, and U3\n etaproj.U1 = eta.U1 - (X.U1*Omega1);\n etaproj.U2 = eta.U2 - (X.U2*Omega2);\n etaproj.U3 = eta.U3 - (X.U3*Omega3);\n \n % Calculate projection algong G \n GOmega1 = reshape(Omega1*X.G1, r1, r2, r3);\n GOmega2 = permute(reshape(Omega2*X.G2, r2, r1, r3), [2 1 3]);\n GOmega3 = permute(reshape(Omega3*X.G3, r3, r1, r2), [2 3 1]); \n etaproj.G = eta.G -(-(GOmega1+GOmega2+GOmega3));\n \n end\n \n \n \n M.tangent = M.proj;\n M.tangent2ambient = @(X, eta) eta;\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n Y.G = (X.G + t*eta.G);\n Y.U1 = uf((X.U1 + t*eta.U1)); % U factor of Polar factorization\n Y.U2 = uf((X.U2 + t*eta.U2));\n Y.U3 = uf((X.U3 + t*eta.U3));\n \n Y = prepare(Y);\n end\n \n M.exp = @exponential;\n function Y = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Y = retraction(X, eta, t);\n warning('manopt:fixedrankfactory_tucker_preconditioned:exp', ...\n ['Exponential for fixed rank ' ...\n 'Tucker manifold not implemented yet. Used retraction instead.']);\n end\n \n M.hash = @(X) ['z' hashmd5([sum(X.U1(:)) ; sum(X.U2(:)); sum(X.U3(:)); sum(X.G(:)) ])]; % Efficient, suggested by Bart Vandereycken.\n % M.hash = @(X) ['z' hashmd5([X.U1(:); X.U2(:); X.U3(:); X.G(:)])];\n \n M.rand = @random;\n function X = random()\n % % Random generator on the total space\n % % Factors U1, U2, and U3 are on Stiefel manifolds, hence we reuse\n % % their random generator.\n % stiefell = stiefelfactory(n1, r1);\n % stiefelm = stiefelfactory(n2, r2);\n % stiefeln = stiefelfactory(n3, r3);\n %\n % X.U1 = stiefell.rand();\n % X.U2 = stiefelm.rand();\n % X.U3 = stiefeln.rand();\n %\n % % Random initialization: generalization of randn(r1, r1 = r2) in the\n % % matrix case.\n % X.G = randn(r1,r2,r3);\n \n \n % Random generator on the fixed-rank space from a uniform distribution on [0, 1].\n [U1, R1] = qr(rand(n1, r1), 0);\n [U2, R2] = qr(rand(n2, r2), 0);\n [U3, R3] = qr(rand(n3, r3), 0);\n C = rand(r1, r2, r3);\n \n C1 = reshape(C, r1, r2*r3);\n CR1 = reshape(R1*C1, r1, r2, r3); % Multplication by R1\n \n C2 = reshape(permute(CR1, [2 1 3]), r2, r1*r3);\n CR1R2 = permute(reshape(R2*C2, r2, r1, r3), [2 1 3]); % Multplication by R2\n \n C3 = reshape(permute(CR1R2, [3 1 2]), r3, r1*r2);\n CR1R2R3 = permute(reshape(R3*C3, r3, r1, r2), [2 3 1]); % Multplication by R3\n \n X.U1 = U1;\n X.U2 = U2;\n X.U3 = U3;\n X.G = CR1R2R3;\n \n \n % Compute some terms that are used subsequently.\n X = prepare(X);\n \n end\n \n M.randvec = @randomvec;\n function eta = randomvec(X)\n % A random vector on the horizontal space\n eta.U1 = randn(n1, r1);\n eta.U2 = randn(n2, r2);\n eta.U3 = randn(n3, r3);\n eta.G = randn(r1, r2, r3);\n eta = projection(X, eta);\n nrm = M.norm(X, eta);\n eta.U1 = eta.U1 / nrm;\n eta.U2 = eta.U2 / nrm;\n eta.U3 = eta.U3 / nrm;\n eta.G = eta.G / nrm;\n end\n \n M.lincomb = @lincomb;\n \n M.zerovec = @(X) struct('U1', zeros(n1, r1), 'U2', zeros(n2, r2), ...\n 'U3', zeros(n3, r3), 'G', zeros(r1, r2, r3));\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \n % vec and mat are not isometries, because of the scaled metric.\n M.vec = @(X, U1) [U1.U1(:); U1.U2(:); U1.U3(:); U1.G(:)];\n M.mat = @(X, u) struct ...\n ('U1', reshape(u(1 : n1*r1), n1, r1), ...\n 'U2', reshape(u(n1*r1 + 1 : n1*r1 + n2*r2), n2, r2), ...\n 'U3', reshape(u(n1*r1 + n2*r2 + 1 : n1*r1 + n2*r2 + n3*r3), n3, r3), ...\n 'G', reshape(u(n1*r1 + n2*r2 + n3*r3 + 1 : end), r1, r2, r3));\n M.vecmatareisometries = @() false;\n \nend\n\n% Linear combination of tangent vectors\nfunction d = lincomb(X, a1, d1, a2, d2) %#ok\n \n if nargin == 3\n d.U1 = a1*d1.U1;\n d.U2 = a1*d1.U2;\n d.U3 = a1*d1.U3;\n d.G = a1*d1.G;\n elseif nargin == 5\n d.U1 = a1*d1.U1 + a2*d2.U1;\n d.U2 = a1*d1.U2 + a2*d2.U2;\n d.U3 = a1*d1.U3 + a2*d2.U3;\n d.G = a1*d1.G + a2*d2.G;\n else\n error('Bad use of fixedrankfactory_tucker_preconditioned.lincomb.');\n end\n \nend\n\nfunction U = uf(A) % U factor of Polar factorization of a tall matrix A.\n [L, unused, R] = svd(A, 0); %#ok\n U = L*R';\nend\n\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/fixedranktensors/tucker2multiarray.m", "content": "function Xtensor = tucker2multiarray(X)\n% Converts a 3d Tucker form tensor to a multiarray.\n%\n% function Xtensor = tucker2multiarray(X)\n%\n% X has fields U1, U2, U3, and G.\n%\n% The matrices U1 (n1-by-r1), U2 (n2-by-r2) and U3 (n3-by-r3) are\n% orthogonal matrices.\n% G (r1-by-r2-by-r3) is a multidimensional array.\n%\n% See also: fixedrankfactory_tucker_preconditioned\n\n% This file is part of Manopt: www.manopt.org.\n% Original authors: Hiroyuki Kasai and Bamdev Mishra, June 05, 2015.\n% Contributors:\n% Change log:\n \n U1 = X.U1;\n U2 = X.U2;\n U3 = X.U3;\n G = X.G;\n \n % Tensor size\n n1 = size(U1, 1);\n n2 = size(U2, 1);\n n3 = size(U3, 1);\n \n % Core size\n [r1, r2, r3] = size(G);\n \n % Multplication by U1\n G1 = reshape(G, r1, r2*r3);\n GU1 = reshape(U1*G1, n1, r2, r3);\n \n % Further multplication by U2\n G2 = reshape(permute(GU1, [2 1 3]), r2, n1*r3);\n GU1U2 = permute(reshape(U2*G2, n2, n1, r3), [2 1 3]);\n \n % Further multplication by U3\n G3 = reshape(permute(GU1U2, [3 1 2]), r3, n1*n2); \n GU1U2U3 = permute(reshape(U3*G3, n3, n1, n2), [2 3 1]);\n \n Xtensor = GU1U2U3;% Full tensor\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/grassmann/grassmanncomplexfactory.m", "content": "function M = grassmanncomplexfactory(n, p, k)\n% Returns a manifold struct to optimize over the set of subspaces in C^n.\n%\n% function M = grassmanncomplexfactory(n, p)\n% function M = grassmanncomplexfactory(n, p, k)\n%\n% Complex Grassmann manifold: each point on this manifold is a collection\n% of k vector subspaces of dimension p embedded in C^n.\n%\n% The metric is obtained by making the Grassmannian a Riemannian quotient\n% manifold of the complex Stiefel manifold, i.e., the manifold of\n% orthonormal matrices, itself endowed with a metric by making it a\n% Riemannian submanifold of the Euclidean space, endowed with the usual\n% real-trace inner product, that is, it is the usual metric for the complex\n% plane identified with R^2.\n% \n% This structure deals with complex matrices X of size n x p x k\n% (or n x p if k = 1, which is the default) such that each n x p matrix is\n% orthonormal, i.e., X'*X = eye(p) if k = 1, or X(:, :, i)' * X(:, :, i) =\n% eye(p) for i = 1 : k if k > 1. Each n x p matrix is a numerical\n% representation of the vector subspace its columns span.\n%\n% By default, k = 1.\n%\n% See also: grassmannfactory, stiefelcomplexfactory, grassmanngeneralizedfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Hiroyuki Sato, May 21, 2015.\n% Contributors: \n% Change log: \n\n assert(n >= p, ...\n ['The dimension n of the ambient space must be larger ' ...\n\t 'than the dimension p of the subspaces.']);\n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n \n if k == 1\n M.name = @() sprintf('Complex Grassmann manifold Gr(%d, %d)', n, p);\n elseif k > 1\n M.name = @() sprintf(['Multi complex Grassmann manifold ' ...\n 'Gr(%d, %d)^%d'], n, p, k);\n else\n error('k must be an integer no less than 1.');\n end\n \n M.dim = @() 2*k*p*(n-p); %! k*p*(n-p) -> 2*k*p*(n-p)\n \n M.inner = @(x, d1, d2) real(d1(:)'*d2(:)); %! trace -> real-trace\n \n M.norm = @(x, d) norm(d(:));\n \n M.dist = @distance;\n function d = distance(x, y)\n principal_angles = zeros(p, k);\n XHY = multiprod(multihconj(x), y); %! XtY -> XHY, multitransp -> multihconj\n for i = 1 : k\n cos_princ_angle = svd(XHY(:, :, i));\n principal_angles(:, i) = acos(cos_princ_angle);\n end\n d = norm(real(principal_angles), 'fro');\n end\n \n M.typicaldist = @() sqrt(p*k);\n \n % Orthogonal projection of an ambient vector U to the horizontal space\n % at X.\n M.proj = @projection;\n function Up = projection(X, U)\n \n XHU = multiprod(multihconj(X), U); %! XtU -> XHU, multitransp -> multihconj\n Up = U - multiprod(X, XHU); %! XtU -> XHU\n\n end\n \n M.tangent = M.proj;\n \n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, H)\n PXehess = projection(X, ehess);\n XHG = multiprod(multihconj(X), egrad); %! XtG -> XHG, multitransp -> multihconj\n HXHG = multiprod(H, XHG); %! HXtG -> HXHG, XtG -> XHG\n rhess = PXehess - HXHG; %! HXtG -> HXHG\n end\n \n M.retr = @retraction;\n function Y = retraction(X, U, t)\n if nargin < 3\n t = 1.0;\n end\n Y = X + t*U;\n for i = 1 : k \n\t\t\n % Compute the polar factorization of Y = X+tU\n [u, s, v] = svd(Y(:, :, i), 'econ'); %#ok\n Y(:, :, i) = u*v';\n\t\t\t\n % Another popular retraction uses QR instead of SVD.\n % As compared with the Stiefel factory, we do not need to\n\t\t\t% worry about flipping signs of columns here, since only\n\t\t\t% the column space is important, not the actual columns.\n % [Q, unused] = qr(Y(:, :, i), 0); %#ok\n % Y(:, :, i) = Q;\n\t\t\t\n end\n end\n \n M.exp = @exponential;\n function Y = exponential(X, U, t)\n if nargin == 3\n tU = t*U;\n else\n tU = U;\n end\n Y = zeros(size(X));\n for i = 1 : k\n [u, s, v] = svd(tU(:, :, i), 0);\n cos_s = diag(cos(diag(s)));\n sin_s = diag(sin(diag(s)));\n Y(:, :, i) = X(:, :, i)*v*cos_s*v' + u*sin_s*v';\n % From numerical experiments, it seems necessary to\n % re-orthonormalize. This is overall quite expensive.\n [q, unused] = qr(Y(:, :, i), 0); %#ok\n Y(:, :, i) = q;\n end\n end\n\n % Test code for the logarithm:\n % Gr = grassmanncomplexfactory(5, 2, 3);\n % x = Gr.rand()\n % y = Gr.rand()\n % u = Gr.log(x, y)\n % Gr.dist(x, y) % These two numbers should\n % Gr.norm(x, u) % be the same.\n % z = Gr.exp(x, u) % z needs not be the same matrix as y, but it should\n % v = Gr.log(x, z) % be the same point as y on Grassmann: dist almost 0.\n M.log = @logarithm;\n function U = logarithm(X, Y)\n U = zeros(n, p, k);\n for i = 1 : k\n x = X(:, :, i);\n y = Y(:, :, i);\n yHx = y'*x; %! ytx -> yHx, y.' -> y'\n AH = y'-yHx*x'; %! At -> AH, x.' -> x', y.' -> y'\n BH = yHx\\AH; %! Bt -> BH, ytx -> yHx, At -> AH\n [u, s, v] = svd(BH', 'econ'); %! Bt.' -> BH'\n\n u = u(:, 1:p);\n s = diag(s);\n s = s(1:p);\n v = v(:, 1:p);\n\n U(:, :, i) = u*diag(atan(s))*v'; %! v.' -> v'\n end\n end\n\n M.hash = @(X) ['z' hashmd5([real(X(:)); imag(X(:))])]; %! X(:) -> [real(X(:)); imag(X(:))]\n \n M.rand = @random;\n function X = random()\n X = zeros(n, p, k);\n for j = 1 : k\n [Q, unused] = qr(randn(n, p) + 1i*randn(n, p), 0); %#ok %! Complex version\n X(:, :, j) = Q;\n end\n end\n \n M.randvec = @randomvec;\n function U = randomvec(X)\n U = projection(X, randn(n, p, k) + 1i*randn(n, p, k)); %! Complex version\n U = U / norm(U(:));\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, p, k);\n \n % This transport is compatible with the polar retraction.\n M.transp = @(x1, x2, d) projection(x2, d);\n \n M.vec = @(x, u_mat) [real(u_mat(:)) ; imag(u_mat(:))];\n M.mat = @(x, u_vec) reshape(u_vec(1:(n*p*k)) + 1i*u_vec((n*p*k+1):end), [n, p, k]);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/grassmann/grassmannfactory.m", "content": "function M = grassmannfactory(n, p, k)\n% Returns a manifold struct to optimize over the space of vector subspaces.\n%\n% function M = grassmannfactory(n, p)\n% function M = grassmannfactory(n, p, k)\n%\n% Grassmann manifold: each point on this manifold is a collection of k\n% vector subspaces of dimension p embedded in R^n.\n%\n% The metric is obtained by making the Grassmannian a Riemannian quotient\n% manifold of the Stiefel manifold, i.e., the manifold of orthonormal\n% matrices, itself endowed with a metric by making it a Riemannian\n% submanifold of the Euclidean space, endowed with the usual inner product.\n% In short: it is the usual metric used in most cases.\n% \n% This structure deals with matrices X of size n x p x k (or n x p if\n% k = 1, which is the default) such that each n x p matrix is orthonormal,\n% i.e., X'*X = eye(p) if k = 1, or X(:, :, i)' * X(:, :, i) = eye(p) for\n% i = 1 : k if k > 1. Each n x p matrix is a numerical representation of\n% the vector subspace its columns span.\n%\n% By default, k = 1.\n%\n% See also: stiefelfactory grassmanncomplexfactory grassmanngeneralizedfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n% March 22, 2013 (NB) :\n% Implemented geodesic distance.\n% \n% April 17, 2013 (NB) :\n% Retraction changed to the polar decomposition, so that the vector\n% transport is now correct, in the sense that it is compatible with\n% the retraction, i.e., transporting a tangent vector G from U to V\n% where V = Retr(U, H) will give Z, and transporting GQ from UQ to VQ\n% will give ZQ: there is no dependence on the representation, which\n% is as it should be. Notice that the polar factorization requires an\n% SVD whereas the qfactor retraction requires a QR decomposition,\n% which is cheaper. Hence, if the retraction happens to be a\n% bottleneck in your application and you are not using vector\n% transports, you may want to replace the retraction with a qfactor.\n% \n% July 4, 2013 (NB) :\n% Added support for the logarithmic map 'log'.\n%\n% July 5, 2013 (NB) :\n% Added support for ehess2rhess.\n%\n% June 24, 2014 (NB) :\n% Small bug fix in the retraction, and added final\n% re-orthonormalization at the end of the exponential map. This\n% follows discussions on the forum where it appeared there is a\n% significant loss in orthonormality without that extra step. Also\n% changed the randvec function so that it now returns a globally\n% normalized vector, not a vector where each component is normalized\n% (this only matters if k>1).\n\n assert(n >= p, ...\n ['The dimension n of the ambient space must be larger ' ...\n\t 'than the dimension p of the subspaces.']);\n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n \n if k == 1\n M.name = @() sprintf('Grassmann manifold Gr(%d, %d)', n, p);\n elseif k > 1\n M.name = @() sprintf('Multi Grassmann manifold Gr(%d, %d)^%d', ...\n n, p, k);\n else\n error('k must be an integer no less than 1.');\n end\n \n M.dim = @() k*p*(n-p);\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:));\n \n M.dist = @distance;\n function d = distance(x, y)\n square_d = 0;\n XtY = multiprod(multitransp(x), y);\n for i = 1 : k\n cos_princ_angle = svd(XtY(:, :, i));\n square_d = square_d + sum(real(acos(cos_princ_angle)).^2);\n end\n d = sqrt(square_d);\n end\n \n M.typicaldist = @() sqrt(p*k);\n \n % Orthogonal projection of an ambient vector U to the horizontal space\n % at X.\n M.proj = @projection;\n function Up = projection(X, U)\n \n XtU = multiprod(multitransp(X), U);\n Up = U - multiprod(X, XtU);\n\n end\n \n M.tangent = M.proj;\n \n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, H)\n PXehess = projection(X, ehess);\n XtG = multiprod(multitransp(X), egrad);\n HXtG = multiprod(H, XtG);\n rhess = PXehess - HXtG;\n end\n \n M.retr = @retraction;\n function Y = retraction(X, U, t)\n if nargin < 3\n t = 1.0;\n end\n Y = X + t*U;\n for i = 1 : k\n\t\t\n % Compute the polar factorization of Y = X+tU\n [u, s, v] = svd(Y(:, :, i), 'econ'); %#ok\n Y(:, :, i) = u*v';\n\t\t\t\n % Another popular retraction uses QR instead of SVD.\n % As compared with the Stiefel factory, we do not need to\n\t\t\t% worry about flipping signs of columns here, since only\n\t\t\t% the column space is important, not the actual columns.\n % [Q, unused] = qr(Y(:, :, i), 0); %#ok\n % Y(:, :, i) = Q;\n\t\t\t\n end\n end\n \n M.exp = @exponential;\n function Y = exponential(X, U, t)\n if nargin == 3\n tU = t*U;\n else\n tU = U;\n end\n Y = zeros(size(X));\n for i = 1 : k\n [u, s, v] = svd(tU(:, :, i), 0);\n cos_s = diag(cos(diag(s)));\n sin_s = diag(sin(diag(s)));\n Y(:, :, i) = X(:, :, i)*v*cos_s*v' + u*sin_s*v';\n % From numerical experiments, it seems necessary to\n % re-orthonormalize. This is overall quite expensive.\n [q, unused] = qr(Y(:, :, i), 0); %#ok\n Y(:, :, i) = q;\n end\n end\n\n % Test code for the logarithm:\n % Gr = grassmannfactory(5, 2, 3);\n % x = Gr.rand()\n % y = Gr.rand()\n % u = Gr.log(x, y)\n % Gr.dist(x, y) % These two numbers should\n % Gr.norm(x, u) % be the same.\n % z = Gr.exp(x, u) % z needs not be the same matrix as y, but it should\n % v = Gr.log(x, z) % be the same point as y on Grassmann: dist almost 0.\n M.log = @logarithm;\n function U = logarithm(X, Y)\n U = zeros(n, p, k);\n for i = 1 : k\n x = X(:, :, i);\n y = Y(:, :, i);\n ytx = y.'*x;\n At = y.'-ytx*x.';\n Bt = ytx\\At;\n [u, s, v] = svd(Bt.', 'econ');\n\n u = u(:, 1:p);\n s = diag(s);\n s = s(1:p);\n v = v(:, 1:p);\n\n U(:, :, i) = u*diag(atan(s))*v.';\n end\n end\n\n M.hash = @(X) ['z' hashmd5(X(:))];\n \n M.rand = @random;\n function X = random()\n X = zeros(n, p, k);\n for i = 1 : k\n [Q, unused] = qr(randn(n, p), 0); %#ok\n X(:, :, i) = Q;\n end\n end\n \n M.randvec = @randomvec;\n function U = randomvec(X)\n U = projection(X, randn(n, p, k));\n U = U / norm(U(:));\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, p, k);\n \n % This transport is compatible with the polar retraction.\n M.transp = @(x1, x2, d) projection(x2, d);\n \n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, [n, p, k]);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/grassmann/grassmanngeneralizedfactory.m", "content": "function M = grassmanngeneralizedfactory(n, p, B)\n% Returns a manifold struct of \"scaled\" vector subspaces.\n%\n% function M = grassmanngeneralizedfactory(n, p)\n% function M = grassmanngeneralizedfactory(n, p, B)\n%\n% Generalized Grassmann manifold: each point on this manifold is a\n% collection of \"scaled\" vector subspaces of dimension p embedded in R^n.\n% The scaling is due to the symmetric positive definite matrix B.\n%\n% When B is identity, the manifold is the standard Grassmann manifold.\n%\n% The metric is obtained by viewing the generalized Grassmannian\n% a Riemannian quotient manifold of the generalized Stiefel manifold, \n% which is the manifold of \"scaled\" orthonormal matrices. Specifically, \n% the scaled Stiefel manifold is the set {X : X'*B*X = I}. \n% The generalized Grassmann manifold is the Grassmannian of the \n% generalized Stiefel manifold.\n%\n% The generalized Stiefel manifold is endowed with a scaled metric\n% by viewing it as a Riemannian submanifold of the Euclidean space, which\n% is again endowed with the scaled inner product.\n%\n% Some notions (not all) are from Section 4.5 of the paper\n% \"The geometry of algorithms with orthogonality constraints\",\n% A. Edelman, T. A. Arias, S. T. Smith, SIMAX, 1998.\n%\n% Paper link: http://arxiv.org/abs/physics/9806030.\n%\n% \n% Note: some computations such as restricted_svd, distance, logarithm, and \n% exponential are new and we believe them to be correct.\n% Also, we hope that the computations are numerically stable.\n% In case some things do not work out as expected or there is some trouble,\n% please contact us at http://www.manopt.org.\n%\n% Note: egrad2rgrad and ehess2rhess involve solving linear systems in B. If\n% this is a bottleneck for a specific application, then a way forward is to\n% create a modified version of this file which preprocesses B to speed this\n% up (typically, by computing a Cholesky factorization of it, then calling\n% an appropriate solver).\n%\n% See also: stiefelgeneralizedfactory stiefelfactory grassmannfactory\n\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, June 30, 2015.\n% Contributors:\n%\n% Change log:\n% \n \n assert(n >= p, ...\n ['The dimension n of the ambient space must be larger ' ...\n 'than the dimension p of the subspaces.']);\n \n if ~exist('B', 'var') || isempty(B)\n B = speye(n); % Standard Grassmann manifold.\n end\n \n M.name = @() sprintf('Generalized Grassmann manifold Gr(%d, %d)', n, p);\n \n M.dim = @() p*(n - p); \n \n M.inner = @(X, eta, zeta) trace(eta'*(B*zeta)); % Scaled metric, but horizontally invariant.\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @distance; \n function d = distance(X, Y)\n XtBY = X'*(B*Y); % XtY ---> XtBY\n cos_princ_angle = svd(XtBY); % svd(XtY) ---> svd(XtBY)\n % Two next instructions not necessary: the imaginary parts that\n % would appear if the cosines are not between -1 and 1, when\n % passed to the acos function, would be very small, and would\n % thus vanish when the norm is taken.\n % cos_princ_angle = min(cos_princ_angle, 1);\n % cos_princ_angle = max(cos_princ_angle, -1);\n square_d = norm(acos(cos_princ_angle))^2;\n \n d = sqrt(square_d);\n end\n \n M.typicaldist = @() sqrt(p);\n \n \n % Orthogonal projection of an ambient vector U onto the \n % horizontal space at X.\n M.proj = @projection;\n function Up = projection(X, U)\n BX = B*X;\n \n % Projection onto the tangent space\n % U = U - X*symm(BX'*U);\n % Projection onto the horizontal space\n % Up = U - X*skew(BX'*U);\n \n Up = U - X*(BX'*U);\n end\n \n M.tangent = M.proj;\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n \n % First, scale egrad according to the scaled metric in the\n % Euclidean space.\n egrad_scaled = B\\egrad;\n \n % Second, project onto the tangent space.\n % No need to project onto the horizontal space as\n % by the Riemannian submersion theory, this quantity automatically\n % belongs to the horizontal space.\n %\n %\n % rgrad = egrad_scaled - X*symm((B*X)'*egrad_scaled);\n %\n % Verify that symm(BX'*egrad_scaled) = symm(X'*egrad).\n \n rgrad = egrad_scaled - X*symm(X'*egrad);\n end\n \n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, H)\n egraddot = ehess;\n Xdot = H;\n \n % Directional derivative of the Riemannian gradient.\n egrad_scaleddot = B\\egraddot;\n rgraddot = egrad_scaleddot - Xdot*symm(X'*egrad)...\n - X*symm(Xdot'*egrad)...\n - X*symm(X'*egraddot);\n \n % Project onto the horizontal space.\n rhess = M.proj(X, rgraddot);\n end\n \n \n M.retr = @retraction;\n function Y = retraction(X, U, t)\n if nargin < 3\n t = 1.0;\n end\n Y = guf(X + t*U); % Ensure that Y'*B*Y is identity.\n end\n \n \n M.exp = @exponential;\n function Y = exponential(X, U, t)\n if nargin == 3\n tU = t*U;\n else\n tU = U;\n end\n \n % restricted_svd is defined later in the file.\n [u, s, v] = restricted_svd(tU);% svd(tU, 0) ---> restricted_svd(tU).\n cos_s = diag(cos(diag(s)));\n sin_s = diag(sin(diag(s)));\n Y = X*v*cos_s*v' + u*sin_s*v';% Verify that Y'*B*Y is identity\n \n % From numerical experiments, it seems necessary to\n % re-orthonormalize.\n Y = guf(Y);% Ensure that Y'*B*Y is identity.\n end\n \n \n \n % Test code for the logarithm:\n % gGr = grassmanngeneralizedfactory(5, 2, diag(rand(5,1)));\n % x = gGr.rand()\n % y = gGr.rand()\n % u = gGr.log(x, y)\n % gGr.dist(x, y) % These two numbers should\n % gGr.norm(x, u) % be the same.\n % z = gGr.exp(x, u) % z needs not be the same matrix as y, but it should\n % v = gGr.log(x, z) % be the same point as y on Grassmann: dist almost 0.\n % gGr.dist(z, y)\n M.log = @logarithm;\n function U = logarithm(X, Y)\n YtBX = Y'*(B*X); % YtX ---> YtBX.\n At = (Y' - YtBX*X');\n Bt = YtBX\\At;\n [u, s, v] = restricted_svd(Bt');% svd(Bt', 'econ') ---> restricted_svd(Bt').\n \n u = u(:, 1:p);\n s = diag(s);\n s = s(1:p);\n v = v(:, 1:p);\n U = u*diag(atan(s))*v'; % A horizontal vector, i.e., U'*(B*X) is zero.\n end\n \n \n M.hash = @(X) ['z' hashmd5(X(:))];\n \n M.rand = @random;\n function X = random()\n X = guf(randn(n, p)); % Ensure that X'*B*X is identity;\n end\n \n M.randvec = @randomvec;\n function U = randomvec(X)\n U = projection(X, randn(n, p));\n U = U / norm(U(:));\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(X) zeros(n, p);\n \n % This transport is compatible with the generalized polar retraction.\n M.transp = @(X1, X2, d) projection(X2, d);\n \n M.vec = @(X, u_mat) u_mat(:);\n M.mat = @(X, u_vec) reshape(u_vec, [n, p]);\n M.vecmatareisometries = @() false;\n \n % Some auxiliary functions\n symm = @(D) (D + D')/2;\n \n function X = guf(Y)\n % Generalized polar decomposition of an n-by-p matrix Y.\n % X'*B*X is identity.\n \n % Method 1\n [u, ~, v] = svd(Y, 0);\n \n % Instead of the following three steps, an equivalent, but an \n % expensive, way is to do X = u*(sqrtm(u'*(B*u))\\(v')).\n [q, ssquare] = eig(u'*(B*u));\n qsinv = q/sparse(diag(sqrt(diag(ssquare))));\n X = u*((qsinv*q')*v'); % X'*B*X is identity.\n \n \n % Another computation using restricted_svd\n % [u, ~, v] = restricted_svd(Y);\n % X = u*v'; % X'*B*X is identity.\n \n end\n \n function [u, s, v] = restricted_svd(Y)\n % We compute a thin svd-like decomposition of an n-by-p matrix Y \n % into matrices u, s, and v such that u is an n-by-p matrix\n % with u'*B*u being identity, s is a p-by-p diagonal matrix \n % with positive entries, and v is a p-by-p orthogonal matrix.\n % Y = u*s*v'.\n \n [v, ssquare] = eig(symm(Y'*(B*Y))); % Y*B*Y is positive definite\n ssquarevec = diag(ssquare);\n \n s = sparse(diag(abs(sqrt(ssquarevec))));\n u = Y*(v/s); % u'*B*u is identity.\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/multinomial/multinomialfactory.m", "content": "function M = multinomialfactory(n, m)\n% Manifold of n-by-m column-stochastic matrices with positive entries.\n%\n% function M = multinomialfactory(n)\n% function M = multinomialfactory(n, m)\n%\n% The returned structure M is a Manopt manifold structure to optimize over\n% the set of n-by-m matrices with (strictly) positive entries and such that\n% the entries of each column sum to one. By default, m = 1.\n%\n% The metric imposed on the manifold is the Fisher metric such that \n% the set of n-by-m column-stochastic matrices (aka the multinomial manifold)\n% is a Riemannian submanifold of the space of n-by-m matrices. Also it\n% should be noted that the retraction operation that we define \n% is first order and as such the checkhessian tool cannot verify \n% the slope correctly.\n% \n% The file is based on developments in the research paper\n% Y. Sun, J. Gao, X. Hong, B. Mishra, and B. Yin,\n% \"Heterogeneous tensor decomposition for clustering via manifold\n% optimization\", arXiv:1504.01777, 2015.\n%\n% Link to the paper: http://arxiv.org/abs/1504.01777.\n%\n% Please cite the Manopt paper as well as the research paper:\n% @Article{sun2015multinomial,\n% author = {Y. Sun and J. Gao and X. Hong and B. Mishra and B. Yin},\n% title = {Heterogeneous Tensor Decomposition for Clustering via Manifold Optimization},\n% journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},\n% year = {2016},\n% volume = {38},\n% number = {3},\n% pages = {476--489},\n% doi = {10.1109/TPAMI.2015.2465901}\n% }\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, April 06, 2015.\n% Contributors:\n% Change log:\n \n if ~exist('m', 'var') || isempty(m)\n m = 1;\n end\n\n M.name = @() sprintf('%dx%d column-stochastic matrices with positive entries', n, m);\n \n M.dim = @() (n-1)*m;\n \n % We impose the Fisher metric.\n M.inner = @iproduct;\n function ip = iproduct(X, eta, zeta)\n ip = sum((eta(:).*zeta(:))./X(:));\n end\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(X, Y) error('multinomialfactory.dist not implemented yet.');\n \n M.typicaldist = @() m*pi/2; % This is an approximation.\n \n % Column vector of ones of length n. \n e = ones(n, 1);\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n lambda = -sum(X.*egrad, 1); % Row vector of length m.\n rgrad = X.*egrad + (e*lambda).*X; % This is in the tangent space.\n end\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, eta)\n \n % Riemannian gradient computation.\n % lambda is a row vector of length m.\n lambda = - sum(X.*egrad, 1);\n rgrad = X.*egrad + (e*lambda).*X;\n \n % Directional derivative of the Riemannian gradient.\n % lambdadot is a row vector of length m.\n lambdadot = -sum(eta.*egrad, 1) - sum(X.*ehess, 1); \n rgraddot = eta.*egrad + X.*ehess + (e*lambdadot).*X + (e*lambda).*eta;\n \n % Correction term because of the non-constant metric that we\n % impose. The computation of the correction term follows the use of\n % Koszul formula.\n correction_term = - 0.5*(eta.*rgrad)./X;\n rhess = rgraddot + correction_term;\n \n % Finally, projection onto the tangent space.\n rhess = M.proj(X, rhess);\n end\n \n % Projection of the vector eta in the ambeint space onto the tangent\n % space.\n M.proj = @projection;\n function etaproj = projection(X, eta)\n alpha = sum(eta, 1); % Row vector of length m.\n etaproj = eta - (e*alpha).*X;\n end\n \n M.tangent = M.proj;\n M.tangent2ambient = @(X, eta) eta;\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n % A first-order retraction.\n Y = X.*exp(t*(eta./X)); % Based on mapping for positive scalars.\n Y = Y./(e*(sum(Y, 1))); % Projection onto the constraint set.\n % For numerical reasons, so that we avoid entries going to zero:\n Y = max(Y, eps);\n end\n \n M.exp = @exponential;\n function Y = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Y = retraction(X, eta, t);\n warning('manopt:multinomialfactory:exp', ...\n ['Exponential for the Multinomial manifold' ...\n 'manifold not implemented yet. Used retraction instead.']);\n end\n \n M.hash = @(X) ['z' hashmd5(X(:))];\n \n M.rand = @random;\n function X = random()\n % A random point in the ambient space.\n X = rand(n, m); %\n X = X./(e*(sum(X, 1)));\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(X)\n % A random vector in the tangent space\n eta = randn(n, m);\n eta = M.proj(X, eta); % Projection onto the tangent space.\n nrm = M.norm(X, eta);\n eta = eta / nrm;\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(X) zeros(n, m);\n \n M.transp = @(X1, X2, d) projection(X2, d);\n \n % vec and mat are not isometries, because of the scaled metric.\n M.vec = @(X, U) U(:);\n M.mat = @(X, u) reshape(u, n, m);\n M.vecmatareisometries = @() false;\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/oblique/obliquecomplexfactory.m", "content": "function M = obliquecomplexfactory(n, m, transposed)\n% Returns a manifold struct defining complex matrices w/ unit-norm columns.\n%\n% function M = obliquecomplexfactory(n, m)\n% function M = obliquecomplexfactory(n, m, transposed)\n%\n% Oblique manifold: deals with complex matrices of size n x m such that\n% each column has unit 2-norm, i.e., is a point on the unit sphere in C^n.\n% The geometry is a product geometry of m unit spheres in C^n. For the\n% metric, C^n is treated as R^(2n), so that the real part and imaginary\n% parts are treated separately as 2n real coordinates. As such, the complex\n% oblique manifold is a Riemannian submanifold of (R^2)^(n x m), with the\n% usual metric = real(u'*v).\n% \n% If transposed is set to true (it is false by default), then the matrices\n% are transposed: a point Y on the manifold is a matrix of size m x n and\n% each row has unit 2-norm. It is the same geometry, just a different\n% representation.\n%\n% In transposed form, a point Y is such that Y*Y' is a Hermitian, positive\n% semidefinite matrix of size m and of rank at most n, such that all the\n% diagonal entries are equal to 1.\n%\n% Note: obliquecomplexfactory(1, n, true) is equivalent to (but potentially\n% slower than) complexcirclefactory(n).\n%\n% See also: spherecomplexfactory complexcirclefactory obliquefactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Sep. 3, 2014.\n% Contributors: \n% Change log: \n%\n% Oct. 21, 2016 (NB)\n% Formatted for inclusion in Manopt release.\n%\n% July 20, 2017 (NB)\n% Distance function is now accurate for close-by points. See notes\n% inside the spherefactory file for details. Also improvies distances\n% computation as part of the log function.\n\n \n if ~exist('transposed', 'var') || isempty(transposed)\n transposed = false;\n end\n \n if transposed\n trnsp = @(X) X.';\n else\n trnsp = @(X) X;\n end\n\n M.name = @() sprintf('Complex oblique manifold COB(%d, %d)', n, m);\n \n M.dim = @() (2*n-1)*m;\n \n M.inner = @(x, d1, d2) real(d1(:)'*d2(:));\n \n M.norm = @(x, d) norm(d(:));\n \n M.dist = @(x, y) norm(real(2*asin(.5*sqrt(sum(trnsp(abs(x - y).^2), 1)))));\n \n M.typicaldist = @() pi*sqrt(m);\n \n M.proj = @(X, U) trnsp(projection(trnsp(X), trnsp(U)));\n \n M.tangent = M.proj;\n \n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, U)\n X = trnsp(X);\n egrad = trnsp(egrad);\n ehess = trnsp(ehess);\n U = trnsp(U);\n \n PXehess = projection(X, ehess);\n inners = sum(real(conj(X).*egrad), 1);\n rhess = PXehess - bsxfun(@times, U, inners);\n \n rhess = trnsp(rhess);\n end\n \n M.exp = @exponential;\n % Exponential on the complex oblique manifold\n function y = exponential(x, d, t)\n x = trnsp(x);\n d = trnsp(d);\n \n if nargin == 2\n % t = 1;\n td = d;\n else\n td = t*d;\n end\n\n nrm_td = sqrt(sum(real(td).^2 + imag(td).^2, 1));\n\n y = bsxfun(@times, x, cos(nrm_td)) + ...\n bsxfun(@times, td, sin(nrm_td) ./ nrm_td);\n \n % For those columns where the step is 0, replace y by x\n exclude = (nrm_td == 0);\n y(:, exclude) = x(:, exclude);\n\n y = trnsp(y);\n end\n\n M.log = @logarithm;\n function v = logarithm(x1, x2)\n x1 = trnsp(x1);\n x2 = trnsp(x2);\n \n v = projection(x1, x2 - x1);\n dists = real(2*asin(.5*sqrt(sum(trnsp(abs(x - y).^2), 1))));\n norms = sqrt(sum(real(v).^2 + imag(v).^2, 1));\n\t\tfactors = dists./norms;\n % For very close points, dists is almost equal to norms, but\n % because they are both almost zero, the division above can return\n % NaN's. To avoid that, we force those ratios to 1.\n\t\tfactors(dists <= 1e-10) = 1;\n\t\tv = bsxfun(@times, v, factors);\n \n v = trnsp(v);\n end\n\n M.retr = @retraction;\n % Retraction on the oblique manifold\n function y = retraction(x, d, t)\n x = trnsp(x);\n d = trnsp(d);\n \n if nargin < 3\n td = d;\n else\n td = t*d;\n end\n\n y = normalize_columns(x + td);\n \n y = trnsp(y);\n end\n\n M.hash = @(x) ['z' hashmd5([real(x(:)) ; imag(x(:))])];\n \n M.rand = @() trnsp(random(n, m));\n \n M.randvec = @(x) trnsp(randomvec(n, m, trnsp(x)));\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) trnsp(zeros(n, m));\n \n M.transp = @(x1, x2, d) M.proj(x2, d);\n \n M.pairmean = @pairmean;\n function y = pairmean(x1, x2)\n y = trnsp(x1+x2);\n y = normalize_columns(y);\n y = trnsp(y);\n end\n\n % vec returns a vector representation of an input tangent vector which\n % is represented as a matrix. mat returns the original matrix\n % representation of the input vector representation of a tangent\n % vector. vec and mat are thus inverse of each other. They are\n % furthermore isometries between a subspace of R^2nm and the tangent\n % space at x.\n vect = @(X) X(:);\n M.vec = @(x, u_mat) [vect(real(trnsp(u_mat))) ; ...\n vect(imag(trnsp(u_mat)))];\n M.mat = @(x, u_vec) trnsp(reshape(u_vec(1:(n*m)), [n, m])) + ...\n 1i*trnsp(reshape(u_vec((n*m+1):end), [n, m]));\n M.vecmatareisometries = @() true;\n\nend\n\n% Given a matrix X, returns the same matrix but with each column scaled so\n% that they have unit 2-norm.\nfunction X = normalize_columns(X)\n\tnorms = sqrt(sum(real(X).^2 + imag(X).^2, 1));\n\tX = bsxfun(@times, X, 1./norms);\nend\n\n% Orthogonal projection of the ambient vector H onto the tangent space at X\nfunction PXH = projection(X, H)\n\n % Compute the inner product between each vector H(:, i) with its root\n % point X(:, i), that is, real(X(:, i)' * H(:, i)).\n % Returns a row vector.\n inners = real(sum(conj(X).*H, 1));\n \n % Subtract from H the components of the H(:, i)'s that are parallel to\n % the root points X(:, i).\n PXH = H - bsxfun(@times, X, inners);\n\nend\n\n% Uniform random sampling on the sphere.\nfunction x = random(n, m)\n\n x = normalize_columns(randn(n, m) + 1i*randn(n, m));\n\nend\n\n% Random normalized tangent vector at x.\nfunction d = randomvec(n, m, x)\n\n d = randn(n, m) + 1i*randn(n, m);\n d = projection(x, d);\n d = d / norm(d(:));\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/oblique/obliquefactory.m", "content": "function M = obliquefactory(n, m, transposed)\n% Returns a manifold struct to optimize over matrices w/ unit-norm columns.\n%\n% function M = obliquefactory(n, m)\n% function M = obliquefactory(n, m, transposed)\n%\n% Oblique manifold: deals with matrices of size n x m such that each column\n% has unit 2-norm, i.e., is a point on the unit sphere in R^n. The metric\n% is such that the oblique manifold is a Riemannian submanifold of the\n% space of nxm matrices with the usual trace inner product, i.e., the usual\n% metric.\n%\n% If transposed is set to true (it is false by default), then the matrices\n% are transposed: a point Y on the manifold is a matrix of size m x n and\n% each row has unit 2-norm. It is the same geometry, just a different\n% representation.\n%\n% See also: spherefactory obliquecomplexfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n%\tJuly 16, 2013 (NB) :\n% Added 'transposed' option, mainly for ease of comparison with the\n% elliptope geometry.\n%\n%\tNov. 29, 2013 (NB) :\n% Added normalize_columns function to make it easier to exploit the\n% bsxfun formulation of column normalization, which avoids using for\n% loops and provides performance gains. The exponential still uses a\n% for loop.\n%\n%\tApril 4, 2015 (NB) :\n% Log function modified to avoid NaN's appearing for close by points.\n%\n%\tApril 13, 2015 (NB) :\n% Exponential now without for-loops.\n%\n% Oct. 8, 2016 (NB)\n% Code for exponential was simplified to only treat the zero vector\n% as a particular case.\n%\n% Oct. 21, 2016 (NB)\n% Bug caught in M.log: the function called v = M.proj(x1, x2 - x1),\n% which internally applies transp to inputs and outputs. But since\n% M.log had already taken care of transposing things, this introduced\n% a bug (which only triggered if using M.log in transposed mode.)\n% The code now calls \"v = projection(x1, x2 - x1);\" since projection\n% assumes the inputs and outputs do not need to be transposed.\n%\n% July 20, 2017 (NB)\n% Distance function is now accurate for close-by points. See notes\n% inside the spherefactory file for details. Also improvies distances\n% computation as part of the log function.\n\n \n if ~exist('transposed', 'var') || isempty(transposed)\n transposed = false;\n end\n \n if transposed\n trnsp = @(X) X.';\n else\n trnsp = @(X) X;\n end\n\n M.name = @() sprintf('Oblique manifold OB(%d, %d)', n, m);\n \n M.dim = @() (n-1)*m;\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:));\n \n M.dist = @(x, y) norm(real(2*asin(.5*sqrt(sum(trnsp(x - y).^2, 1)))));\n \n M.typicaldist = @() pi*sqrt(m);\n \n M.proj = @(X, U) trnsp(projection(trnsp(X), trnsp(U)));\n \n M.tangent = M.proj;\n \n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, U)\n X = trnsp(X);\n egrad = trnsp(egrad);\n ehess = trnsp(ehess);\n U = trnsp(U);\n \n PXehess = projection(X, ehess);\n inners = sum(X.*egrad, 1);\n rhess = PXehess - bsxfun(@times, U, inners);\n \n rhess = trnsp(rhess);\n end\n \n M.exp = @exponential;\n % Exponential on the oblique manifold\n function y = exponential(x, d, t)\n x = trnsp(x);\n d = trnsp(d);\n \n if nargin < 3\n % t = 1;\n td = d;\n else\n td = t*d;\n end\n\n nrm_td = sqrt(sum(td.^2, 1));\n\n y = bsxfun(@times, x, cos(nrm_td)) + ...\n bsxfun(@times, td, sin(nrm_td) ./ nrm_td);\n \n % For those columns where the step is 0, replace y by x\n exclude = (nrm_td == 0);\n y(:, exclude) = x(:, exclude);\n\n y = trnsp(y);\n end\n\n M.log = @logarithm;\n function v = logarithm(x1, x2)\n x1 = trnsp(x1);\n x2 = trnsp(x2);\n \n v = projection(x1, x2 - x1);\n dists = real(2*asin(.5*sqrt(sum((x1 - x2).^2, 1))));\n norms = real(sqrt(sum(v.^2, 1)));\n\t\tfactors = dists./norms;\n % For very close points, dists is almost equal to norms, but\n % because they are both almost zero, the division above can return\n % NaN's. To avoid that, we force those ratios to 1.\n\t\tfactors(dists <= 1e-10) = 1;\n\t\tv = bsxfun(@times, v, factors);\n \n v = trnsp(v);\n end\n\n M.retr = @retraction;\n % Retraction on the oblique manifold\n function y = retraction(x, d, t)\n x = trnsp(x);\n d = trnsp(d);\n \n if nargin < 3\n % t = 1;\n td = d;\n else\n td = t*d;\n end\n \n y = normalize_columns(x + td);\n \n y = trnsp(y);\n end\n\n M.hash = @(x) ['z' hashmd5(x(:))];\n \n M.rand = @() trnsp(random(n, m));\n \n M.randvec = @(x) trnsp(randomvec(n, m, trnsp(x)));\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) trnsp(zeros(n, m));\n \n M.transp = @(x1, x2, d) M.proj(x2, d);\n \n M.pairmean = @pairmean;\n function y = pairmean(x1, x2)\n y = trnsp(x1+x2);\n y = normalize_columns(y);\n y = trnsp(y);\n end\n\n % vec returns a vector representation of an input tangent vector which\n % is represented as a matrix. mat returns the original matrix\n % representation of the input vector representation of a tangent\n % vector. vec and mat are thus inverse of each other. They are\n % furthermore isometries between a subspace of R^nm and the tangent\n % space at x.\n vect = @(X) X(:);\n M.vec = @(x, u_mat) vect(trnsp(u_mat));\n M.mat = @(x, u_vec) trnsp(reshape(u_vec, [n, m]));\n M.vecmatareisometries = @() true;\n\nend\n\n% Given a matrix X, returns the same matrix but with each column scaled so\n% that they have unit 2-norm.\nfunction X = normalize_columns(X)\n\t% This is faster than norms(X, 2, 1) for small X, and as fast for large X.\n\tnrms = sqrt(sum(X.^2, 1));\n\tX = bsxfun(@times, X, 1./nrms);\nend\n\n% Orthogonal projection of the ambient vector H onto the tangent space at X\nfunction PXH = projection(X, H)\n\n % Compute the inner product between each vector H(:, i) with its root\n % point X(:, i), that is, X(:, i).' * H(:, i). Returns a row vector.\n inners = sum(X.*H, 1);\n \n % Subtract from H the components of the H(:, i)'s that are parallel to\n % the root points X(:, i).\n PXH = H - bsxfun(@times, X, inners);\n\n % % Equivalent but slow code:\n % m = size(X, 2);\n % PXH = zeros(size(H));\n % for i = 1 : m\n % PXH(:, i) = H(:, i) - X(:, i) * (X(:, i)'*H(:, i));\n % end\n\nend\n\n% Uniform random sampling on the sphere.\nfunction x = random(n, m)\n\n x = normalize_columns(randn(n, m));\n\nend\n\n% Random normalized tangent vector at x.\nfunction d = randomvec(n, m, x)\n\n d = randn(n, m);\n d = projection(x, d);\n d = d / norm(d(:));\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/rotations/randrot.m", "content": "function R = randrot(n, N)\n% Generates uniformly random rotation matrices.\n%\n% function R = randrot(n, N)\n%\n% R is a n-by-n-by-N matrix such that each slice R(:, :, i) is an\n% orthogonal matrix of size n of determinant +1 (i.e., a matrix in SO(n)).\n% By default, N = 1.\n% Complexity: N times O(n^3).\n% Theory in Diaconis and Shahshahani 1987 for the uniformity on O(n);\n% With details in Mezzadri 2007,\n% \"How to generate random matrices from the classical compact groups.\"\n% To ensure matrices in SO(n), we permute the two first columns when\n% the determinant is -1.\n%\n% See also: randskew\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Sept. 25, 2012.\n% Contributors: \n% Change log: \n\n if nargin < 2\n N = 1;\n end\n \n if n == 1\n R = ones(1, 1, N);\n return;\n end\n \n R = zeros(n, n, N);\n \n for i = 1 : N\n \n % Generated as such, Q is uniformly distributed over O(n), the set\n % of orthogonal matrices.\n A = randn(n);\n [Q, RR] = qr(A);\n Q = Q * diag(sign(diag(RR))); %% Mezzadri 2007\n \n % If Q is in O(n) but not in SO(n), we permute the two first\n % columns of Q such that det(new Q) = -det(Q), hence the new Q will\n % be in SO(n), uniformly distributed.\n if det(Q) < 0\n Q(:, [1 2]) = Q(:, [2 1]);\n end\n \n R(:, :, i) = Q;\n \n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/rotations/randskew.m", "content": "function S = randskew(n, N)\n% Generates random skew symmetric matrices with normal entries.\n% \n% function S = randskew(n, N)\n%\n% S is an n-by-n-by-N matrix where each slice S(:, :, i) for i = 1..N is a\n% random skew-symmetric matrix with upper triangular entries distributed\n% independently following a normal distribution (Gaussian, zero mean, unit\n% variance).\n%\n% See also: randrot\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Sept. 25, 2012.\n% Contributors: \n% Change log: \n\n\n if nargin < 2\n N = 1;\n end\n\n % Subindices of the (strictly) upper triangular entries of an n-by-n\n % matrix\n [I J] = find(triu(ones(n), 1));\n \n K = repmat(1:N, n*(n-1)/2, 1);\n \n % Indices of the strictly upper triangular entries of all N slices of\n % an n-by-n-by-N matrix\n L = sub2ind([n n N], repmat(I, N, 1), repmat(J, N, 1), K(:));\n \n % Allocate memory for N random skew matrices of size n-by-n and\n % populate each upper triangular entry with a random number following a\n % normal distribution and copy them with opposite sign on the\n % corresponding lower triangular side.\n S = zeros(n, n, N);\n S(L) = randn(size(L));\n S = S-multitransp(S);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/rotations/rotationsfactory.m", "content": "function M = rotationsfactory(n, k)\n% Returns a manifold structure to optimize over rotation matrices.\n% \n% function M = rotationsfactory(n)\n% function M = rotationsfactory(n, k)\n%\n% Special orthogonal group (the manifold of rotations): deals with matrices\n% R of size n x n x k (or n x n if k = 1, which is the default) such that\n% each n x n matrix is orthogonal, with determinant 1, i.e., X'*X = eye(n)\n% if k = 1, or X(:, :, i)' * X(:, :, i) = eye(n) for i = 1 : k if k > 1.\n%\n% This is a description of SO(n)^k with the induced metric from the\n% embedding space (R^nxn)^k, i.e., this manifold is a Riemannian\n% submanifold of (R^nxn)^k endowed with the usual trace inner product.\n%\n% Tangent vectors are represented in the Lie algebra, i.e., as skew\n% symmetric matrices. Use the function M.tangent2ambient(X, H) to switch\n% from the Lie algebra representation to the embedding space\n% representation. This is often necessary when defining\n% problem.ehess(X, H).\n%\n% By default, the retraction is only a first-order approximation of the\n% exponential. To force the use of a second-order approximation, call\n% M.retr = M.retr2 after creating M. This switches from a QR-based\n% computation to an SVD-based computation.\n%\n% By default, k = 1.\n%\n% See also: stiefelfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log:\n% Jan. 31, 2013 (NB)\n% Added egrad2rgrad and ehess2rhess\n% Oct. 21, 2016 (NB)\n% Added M.retr2: a second-order retraction based on SVD.\n\n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n \n if k == 1\n M.name = @() sprintf('Rotations manifold SO(%d)', n);\n elseif k > 1\n M.name = @() sprintf('Product rotations manifold SO(%d)^%d', n, k);\n else\n error('k must be an integer no less than 1.');\n end\n \n M.dim = @() k*nchoosek(n, 2);\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:));\n \n M.typicaldist = @() pi*sqrt(n*k);\n \n M.proj = @(X, H) multiskew(multiprod(multitransp(X), H));\n \n M.tangent = @(X, H) multiskew(H);\n \n M.tangent2ambient = @(X, U) multiprod(X, U);\n\t\n\tM.egrad2rgrad = M.proj;\n\t\n\tM.ehess2rhess = @ehess2rhess;\n\tfunction Rhess = ehess2rhess(X, Egrad, Ehess, H)\n % Reminder : H contains skew-symmeric matrices. The actual\n % direction that the point X is moved along is X*H.\n\t\tXt = multitransp(X);\n\t\tXtEgrad = multiprod(Xt, Egrad);\n symXtEgrad = multisym(XtEgrad);\n\t\tXtEhess = multiprod(Xt, Ehess);\n\t\tRhess = multiskew( XtEhess - multiprod(H, symXtEgrad) );\n\tend\n \n M.retr = @retraction;\n function Y = retraction(X, U, t)\n if nargin == 3\n tU = t*U;\n else\n tU = U;\n end\n Y = X + multiprod(X, tU);\n for i = 1 : k\n % This QR-based retraction is only a first-order approximation\n % of the exponential map, not a second-order one.\n [Q, R] = qr(Y(:, :, i));\n % The instruction with R ensures we are not flipping signs\n % of some columns, which should never happen in modern Matlab\n % versions but may be an issue with older versions.\n Y(:, :, i) = Q * diag(sign(sign(diag(R))+.5));\n % This is guaranteed to always yield orthogonal matrices with\n % determinant +1. Simply look at the eigenvalues of a skew\n % symmetric matrix, than at those of identity plus that matrix,\n % and compute their product for the determinant: it's stricly\n % positive in all cases.\n end\n end\n \n % A second order retraction is implemented here. To force its use,\n % after creating the factory M, execute M.retr = M.retr2.\n M.retr2 = @retraction2;\n function Y = retraction2(X, U, t)\n if nargin == 3\n tU = t*U;\n else\n tU = U;\n end\n Y = X + multiprod(X, tU);\n for i = 1 : k\n [Uk, ~, Vk] = svd(Y(:, :, k));\n Y(:, :, k) = Uk*Vk';\n end\n end\n \n M.exp = @exponential;\n function Y = exponential(X, U, t)\n if nargin == 3\n exptU = t*U;\n else\n exptU = U;\n end\n for i = 1 : k\n exptU(:, :, i) = expm(exptU(:, :, i));\n end\n Y = multiprod(X, exptU);\n end\n \n M.log = @logarithm;\n function U = logarithm(X, Y)\n\t\tU = multiprod(multitransp(X), Y);\n for i = 1 : k\n % The result of logm should be real in theory, but it is\n % numerically useful to force it.\n U(:, :, i) = real(logm(U(:, :, i)));\n end\n % Ensure the tangent vector is in the Lie algebra.\n U = multiskew(U);\n end\n\n M.hash = @(X) ['z' hashmd5(X(:))];\n \n M.rand = @() randrot(n, k);\n \n M.randvec = @randomvec;\n function U = randomvec(X) %#ok\n U = randskew(n, k);\n nrmU = sqrt(U(:).'*U(:));\n U = U / nrmU;\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, n, k);\n \n M.transp = @(x1, x2, d) d;\n \n M.pairmean = @pairmean;\n function Y = pairmean(X1, X2)\n V = M.log(X1, X2);\n Y = M.exp(X1, .5*V);\n end\n \n M.dist = @(x, y) M.norm(x, M.log(x, y));\n \n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, [n, n, k]);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/specialeuclidean/specialeuclideanfactory.m", "content": "function M = specialeuclideanfactory(n, k)\n% Returns a manifold structure to optimize over the special Euclidean group\n% \n% function M = specialeuclideanfactory(n)\n% function M = specialeuclideanfactory(n, k)\n%\n% The special Euclidean group (the manifold of rigid transformations):\n% This is a product manifold of the rotations group SO(n) and the\n% translation group R^n, copied k times. (See note below.)\n%\n% Points on the manifold are represented as structures X with two fields.\n% X.R is a 3D array of size nxnxk such that each slice X.R(:, :, i)\n% corresponds to a rotation matrix (orthogonal with determinant 1).\n% X.t is a matrix of size nxk such that each column X.t(:, i) corresponds\n% to a translation vector.\n%\n% Tangent vectors are represented as structures with the same fields. Note\n% that rotational components of the tangent vectors are represented in the\n% Lie algebra, i.e., each slice Xdot.R(:, :, i) is a skew-symmetric matrix.\n% Use M.tangent2ambient(X, Xdot) to obtain a representation in the ambient\n% space. This is often necessary when defining problem.ehess(X, Xdot).\n%\n% This is a description of SE(n)^k with the induced metric from the\n% embedding space (R^nxn)^k x (R^n)^k, i.e., this manifold is a Riemannian\n% submanifold of the embedding Euclidean space with the usual inner\n% product.\n%\n% By default, k = 1.\n%\n% Note: this is a product geometry: it may not be the \"appropriate\"\n% geometry to give to SE(n) for your application. In particular, this is\n% not the Lie geometry of SE(n), because SE(n) is not a direct product of\n% SO(n) and R^n: it is only a semidirect product. Following a comment by\n% Martijn Zeestraten on the Manopt forum, see this file for more\n% information about the Lie geometry:\n% http://ethaneade.com/lie.pdf\n%\n% See rotationsfactory and euclideanfactory for details.\n%\n% See also: rotationsfactory euclideanfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Sep. 23, 2014.\n% Contributors: \n% Change log:\n\n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n \n elements = struct();\n elements.R = rotationsfactory(n, k);\n elements.t = euclideanfactory(n, k);\n \n M = productmanifold(elements);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/sphere/spherecomplexfactory.m", "content": "function M = spherecomplexfactory(n, m)\n% Returns a manifold struct to optimize over unit-norm complex matrices.\n%\n% function M = spherecomplexfactory(n)\n% function M = spherecomplexfactory(n, m)\n%\n% Manifold of n-by-m complex matrices of unit Frobenius norm.\n% By default, m = 1, which corresponds to the unit sphere in C^n. The\n% metric is such that the sphere is a Riemannian submanifold of the space\n% of 2nx2m real matrices with the usual trace inner product, i.e., the\n% usual metric.\n% \n% See also: spherefactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% Sep. 4, 2014 (NB):\n% Added ehess2rhess.\n%\n% April 7, 2015 (NB):\n% Added vec/mat pair (for use with hessianspectrum, for example).\n%\n% April 13, 2015 (NB):\n% Added logarithm\n%\n% Oct. 8, 2016 (NB)\n% Code for exponential was simplified to only treat the zero vector\n% as a particular case.\n%\n% Oct. 22, 2016 (NB)\n% Distance function dist now significantly more accurate for points\n% within 1e-7 and less from each other.\n\n \n if ~exist('m', 'var')\n m = 1;\n end\n\n if m == 1\n M.name = @() sprintf('Complex sphere S^%d', n-1);\n else\n M.name = @() sprintf('Unit F-norm %dx%d complex matrices', n, m);\n end\n \n M.dim = @() 2*(n*m)-1;\n \n M.inner = @(x, d1, d2) real(d1(:)'*d2(:));\n \n M.norm = @(x, d) norm(d, 'fro');\n \n M.dist = @(x, y) real(2*asin(.5*norm(x - y, 'fro')));\n \n M.typicaldist = @() pi;\n \n M.proj = @(x, d) reshape(d(:) - x(:)*(real(x(:)'*d(:))), n, m);\n \n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = M.proj;\n \n\tM.ehess2rhess = @ehess2rhess;\n\tfunction rhess = ehess2rhess(x, egrad, ehess, u)\n rhess = M.proj(x, ehess) - real((x(:)'*egrad(:)))*u;\n\tend\n \n\tM.tangent = M.proj;\n \n M.exp = @exponential;\n \n M.retr = @retraction;\n\n M.log = @logarithm;\n function v = logarithm(x1, x2)\n v = M.proj(x1, x2 - x1);\n di = M.dist(x1, x2);\n % If the two points are \"far apart\", correct the norm.\n if di > 1e-6\n nv = norm(v, 'fro');\n v = v * (di / nv);\n end\n end\n \n M.hash = @(x) ['z' hashmd5([real(x(:)) ; imag(x(:))])];\n \n M.rand = @() random(n, m);\n \n M.randvec = @(x) randomvec(n, m, x);\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, m);\n \n M.transp = @(x1, x2, d) M.proj(x2, d);\n \n M.pairmean = @pairmean;\n function y = pairmean(x1, x2)\n y = x1+x2;\n y = y / norm(y, 'fro');\n end\n\n mn = m*n;\n M.vec = @(x, u_mat) [real(u_mat(:)) ; imag(u_mat(:))];\n M.mat = @(x, u_vec) reshape(u_vec(1:mn), m, n) + 1i*reshape(u_vec((mn+1):end), m, n);\n M.vecmatareisometries = @() true;\n\nend\n\n% Exponential on the sphere\nfunction y = exponential(x, d, t)\n\n if nargin == 2\n % t = 1;\n td = d;\n else\n td = t*d;\n end\n \n nrm_td = norm(td, 'fro');\n \n if nrm_td > 0\n y = x*cos(nrm_td) + td*(sin(nrm_td)/nrm_td);\n else\n y = x;\n end\n\nend\n\n% Retraction on the sphere\nfunction y = retraction(x, d, t)\n\n if nargin == 2\n t = 1;\n end\n \n y = x+t*d;\n y = y/norm(y, 'fro');\n\nend\n\n% Uniform random sampling on the sphere.\nfunction x = random(n, m)\n\n x = randn(n, m) + 1i*randn(n, m);\n x = x/norm(x, 'fro');\n\nend\n\n% Random normalized tangent vector at x.\nfunction d = randomvec(n, m, x)\n\n d = randn(n, m) + 1i*randn(n, m);\n d = reshape(d(:) - x(:)*(real(x(:)'*d(:))), n, m);\n d = d / norm(d, 'fro');\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/sphere/spherefactory.m", "content": "function M = spherefactory(n, m)\n% Returns a manifold struct to optimize over unit-norm vectors or matrices.\n%\n% function M = spherefactory(n)\n% function M = spherefactory(n, m)\n%\n% Manifold of n-by-m real matrices of unit Frobenius norm.\n% By default, m = 1, which corresponds to the unit sphere in R^n. The\n% metric is such that the sphere is a Riemannian submanifold of the space\n% of nxm matrices with the usual trace inner product, i.e., the usual\n% metric.\n% \n% See also: obliquefactory spherecomplexfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% Oct. 8, 2016 (NB)\n% Code for exponential was simplified to only treat the zero vector\n% as a particular case.\n%\n% Oct. 22, 2016 (NB)\n% Distance function dist now significantly more accurate for points\n% within 1e-7 and less from each other.\n%\n% July 20, 2017 (NB)\n% Following conversations with Bruno Iannazzo and P.-A. Absil,\n% the distance function is now even more accurate.\n%\n% Sep. 7, 2017 (NB)\n% New isometric vector transport available in M.isotransp,\n% contributed by Changshuo Liu.\n\n \n if ~exist('m', 'var')\n m = 1;\n end\n\n if m == 1\n M.name = @() sprintf('Sphere S^%d', n-1);\n else\n M.name = @() sprintf('Unit F-norm %dx%d matrices', n, m);\n end\n \n M.dim = @() n*m-1;\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d, 'fro');\n \n M.dist = @dist;\n function d = dist(x, y)\n \n % The following code is mathematically equivalent to the\n % computation d = acos(x(:)'*y(:)) but is much more accurate when\n % x and y are close.\n \n chordal_distance = norm(x - y, 'fro');\n d = real(2*asin(.5*chordal_distance));\n \n % Note: for x and y almost antipodal, the accuracy is good but not\n % as good as possible. One way to improve it is by using the\n % following branching:\n % % if chordal_distance > 1.9\n % % d = pi - dist(x, -y);\n % % end\n % It is rarely necessary to compute distance between\n % almost-antipodal points with full accuracy in Manopt, hence we\n % favor a simpler code.\n \n end\n \n M.typicaldist = @() pi;\n \n M.proj = @(x, d) d - x*(x(:).'*d(:));\n \n M.tangent = M.proj;\n\t\n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = M.proj;\n\t\n\tM.ehess2rhess = @ehess2rhess;\n\tfunction rhess = ehess2rhess(x, egrad, ehess, u)\n rhess = M.proj(x, ehess) - (x(:)'*egrad(:))*u;\n\tend\n \n M.exp = @exponential;\n \n M.retr = @retraction;\n\n M.log = @logarithm;\n function v = logarithm(x1, x2)\n v = M.proj(x1, x2 - x1);\n di = M.dist(x1, x2);\n % If the two points are \"far apart\", correct the norm.\n if di > 1e-6\n nv = norm(v, 'fro');\n v = v * (di / nv);\n end\n end\n \n M.hash = @(x) ['z' hashmd5(x(:))];\n \n M.rand = @() random(n, m);\n \n M.randvec = @(x) randomvec(n, m, x);\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, m);\n \n M.transp = @(x1, x2, d) M.proj(x2, d);\n \n % Isometric vector transport of d from the tangent space at x1 to x2.\n % This is actually a parallel vector transport, see 5 in\n % http://epubs.siam.org/doi/pdf/10.1137/16M1069298\n % \"A Riemannian Gradient Sampling Algorithm for Nonsmooth Optimization\n % on Manifolds\", by Hosseini and Uschmajew, SIOPT 2017\n M.isotransp = @(x1, x2, d) isometricTransp(x1, x2, d);\n function Td = isometricTransp(x1, x2, d)\n v = logarithm(x1, x2);\n dist_x1x2 = norm(v, 'fro');\n if dist_x1x2 > 0\n u = v / dist_x1x2;\n utd = u(:)'*d(:);\n Td = d + (cos(dist_x1x2)-1)*utd*u ...\n - sin(dist_x1x2) *utd*x1;\n else\n % x1 == x2, so the transport is identity\n Td = d;\n end\n end\n \n M.pairmean = @pairmean;\n function y = pairmean(x1, x2)\n y = x1+x2;\n y = y / norm(y, 'fro');\n end\n\n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, [n, m]);\n M.vecmatareisometries = @() true;\n\nend\n\n% Exponential on the sphere\nfunction y = exponential(x, d, t)\n\n if nargin == 2\n % t = 1\n td = d;\n else\n td = t*d;\n end\n \n nrm_td = norm(td, 'fro');\n \n % Former versions of Manopt avoided the computation of sin(a)/a for\n % small a, but further investigations suggest this computation is\n % well-behaved numerically.\n if nrm_td > 0\n y = x*cos(nrm_td) + td*(sin(nrm_td)/nrm_td);\n else\n y = x;\n end\n\nend\n\n% Retraction on the sphere\nfunction y = retraction(x, d, t)\n\n if nargin == 2\n % t = 1;\n td = d;\n else\n td = t*d;\n end\n \n y = x + td;\n y = y / norm(y, 'fro');\n\nend\n\n% Uniform random sampling on the sphere.\nfunction x = random(n, m)\n\n x = randn(n, m);\n x = x / norm(x, 'fro');\n\nend\n\n% Random normalized tangent vector at x.\nfunction d = randomvec(n, m, x)\n\n d = randn(n, m);\n d = d - x*(x(:).'*d(:));\n d = d / norm(d, 'fro');\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/sphere/spheresymmetricfactory.m", "content": "function M = spheresymmetricfactory(n)\n% Returns a manifold struct to optimize over unit-norm symmetric matrices.\n%\n% function M = spheresymmetricfactory(n)\n%\n% Manifold of n-by-n real symmetric matrices of unit Frobenius norm.\n% The metric is such that the sphere is a Riemannian submanifold of the\n% space of nxn symmetric matrices with the usual trace inner product, i.e.,\n% the usual metric = trace(A'*B).\n% \n% See also: spherefactory obliquefactory spherecomplexfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 17, 2015.\n% Contributors: \n% Change log: \n%\n% Oct. 8, 2016 (NB)\n% Code for exponential was simplified to only treat the zero vector\n% as a particular case.\n%\n% Oct. 22, 2016 (NB)\n% Distance function dist now significantly more accurate for points\n% within 1e-7 and less from each other.\n%\n% July 20, 2017 (NB)\n% The distance function is now even more accurate.\n\n\n M.name = @() sprintf('Sphere of symmetric matrices of size %d', n);\n \n M.dim = @() n*(n+1)/2 - 1;\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d, 'fro');\n \n M.dist = @(x, y) real(2*asin(.5*norm(x - y, 'fro')));\n \n M.typicaldist = @() pi;\n \n M.proj = @proj;\n function xdot = proj(x, d)\n d = (d+d.')/2;\n xdot = d - x*(x(:).'*d(:));\n end\n \n M.tangent = @proj;\n\t\n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = @proj;\n\t\n\tM.ehess2rhess = @ehess2rhess;\n\tfunction rhess = ehess2rhess(x, egrad, ehess, u)\n % these are not explicitly required, given the use.\n % egrad = (egrad + egrad.')/2;\n % ehess = (ehess + ehess.')/2;\n rhess = proj(x, ehess) - (x(:)'*egrad(:))*u;\n\tend\n \n M.exp = @exponential;\n \n M.retr = @retraction;\n\n M.log = @logarithm;\n function v = logarithm(x1, x2)\n v = proj(x1, x2 - x1);\n di = M.dist(x1, x2);\n % If the two points are \"far apart\", correct the norm.\n if di > 1e-6\n nv = norm(v, 'fro');\n v = v * (di / nv);\n end\n end\n \n M.hash = @(x) ['z' hashmd5(x(:))];\n \n M.rand = @() random(n);\n \n M.randvec = @(x) randomvec(n, x);\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n);\n \n M.transp = @(x1, x2, d) proj(x2, d);\n \n M.pairmean = @pairmean;\n function y = pairmean(x1, x2)\n y = x1+x2;\n y = y / norm(y, 'fro');\n end\n\n % TODO : check isometry and fix.\n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, [n, m]);\n M.vecmatareisometries = @() false;\n\nend\n\n% Exponential on the sphere\nfunction y = exponential(x, d, t)\n\n if nargin == 2\n % t = 1;\n td = d;\n else\n td = t*d;\n end\n \n nrm_td = norm(td, 'fro');\n \n if nrm_td > 0\n y = x*cos(nrm_td) + td*(sin(nrm_td)/nrm_td);\n else\n y = x;\n end\n\nend\n\n% Retraction on the sphere\nfunction y = retraction(x, d, t)\n\n if nargin == 2\n t = 1;\n end\n \n y = x + t*d;\n y = y / norm(y, 'fro');\n\nend\n\n% Uniform random sampling on the sphere.\nfunction x = random(n)\n\n x = randn(n);\n x = (x + x.')/2;\n x = x/norm(x, 'fro');\n\nend\n\n% Random normalized tangent vector at x.\nfunction d = randomvec(n, x)\n\n d = randn(n);\n d = (d + d.')/2;\n d = d - x*(x(:).'*d(:));\n d = d / norm(d, 'fro');\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/stiefel/stiefelcomplexfactory.m", "content": "function M = stiefelcomplexfactory(n, p, k)\n% Returns a manifold struct. to optimize over complex orthonormal matrices.\n%\n% function M = stiefelcomplexfactory(n, p)\n% function M = stiefelcomplexfactory(n, p, k)\n%\n% The complex Stiefel manifold is the set of complex orthonormal nxp\n% matrices. If k is larger than 1, this is the Cartesian product of the\n% complex Stiefel manifold taken k times. The metric is such that the\n% manifold is a Riemannian submanifold of C^nxp equipped with the usual\n% real-trace inner product, that is, it is the usual metric for the complex\n% plane identified with R^2.\n%\n% Points are represented as matrices X of size n x p x k (or n x p if k=1,\n% which is the default) such that each complex n x p matrix is orthonormal,\n% i.e., X'*X = eye(p) if k = 1, or X(:, :, i)' * X(:, :, i) = eye(p) for\n% i = 1 : k if k > 1. Tangent vectors are represented as matrices the same\n% size as points.\n%\n% By default, k = 1.\n%\n%\n% Please cite the Manopt paper as well as either of these research papers\n% pertaining to this specific geometry:\n% @InProceedings{sato2013complex,\n% Title = {A complex singular value decomposition algorithm based on the {R}iemannian {N}ewton method},\n% Author = {Sato, H. and Iwai, T.},\n% Booktitle = {Decision and Control ({CDC}), 2013 {IEEE} 52nd Annual Conference on},\n% Year = {2013},\n% Organization = {IEEE},\n% Pages = {2972--2978}\n% }\n% @InProceedings{sato2014Riemannian,\n% Title = {{R}iemannian conjugate gradient method for complex singular value decomposition problem},\n% Author = {Sato, H.},\n% Booktitle = {Decision and Control ({CDC}), 2014 {IEEE} 53rd Annual Conference on},\n% Year = {2014},\n% Organization = {IEEE},\n% Pages = {5849--5854}\n% }\n%\n%\n% See also: stiefelfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Hiroyuki Sato, April 27, 2015.\n% Contributors: \n% Change log: \n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n \n if k == 1\n M.name = @() sprintf('Complex Stiefel manifold St(%d, %d)', n, p);\n elseif k > 1\n M.name = @() sprintf('Product complex Stiefel manifold St(%d, %d)^%d', n, p, k);\n else\n error('k must be an integer no less than 1.');\n end\n \n M.dim = @() k*(2*n*p - p^2); %! k*(n*p - .5*p*(p+1)) -> k*(2*n*p - p^2)\n \n M.inner = @(x, d1, d2) real(d1(:)'*d2(:)); %! trace -> real-trace\n \n M.norm = @(x, d) norm(d(:));\n \n M.dist = @(x, y) error('stiefel.dist not implemented yet.');\n \n M.typicaldist = @() sqrt(p*k);\n \n M.proj = @projection;\n function Up = projection(X, U)\n \n XHU = multiprod(multihconj(X), U); %! XtU -> XHU, multitransp -> multihconj\n herXHU = multiherm(XHU); %! symXtU -> herXHU, multisym -> multiherm\n Up = U - multiprod(X, herXHU); %! symXtU -> herXHU\n \n end\n \n M.tangent = M.proj;\n \n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, H)\n XHG = multiprod(multihconj(X), egrad); %! XtG -> XHG, multitransp -> multihconj\n herXHG = multiherm(XHG); %! symXtG -> herXHG, multisym(XtG) -> multiherm(XHG)\n HherXHG = multiprod(H, herXHG); %! HsymXtG -> HherXHG, symXtG -> herXHG\n rhess = projection(X, ehess - HherXHG); %! HsymXtG -> HherXHG\n end\n \n M.retr = @retraction;\n function Y = retraction(X, U, t)\n if nargin < 3\n t = 1.0;\n end\n Y = X + t*U;\n for i = 1 : k\n [Q, R] = qr(Y(:, :, i), 0);\n % The instruction with R assures we are not flipping signs\n % of some columns, which should never happen in modern Matlab\n % versions but may be an issue with older versions.\n Y(:, :, i) = Q * diag(sign(sign(diag(R))+.5));\n end\n end\n \n M.exp = @exponential;\n function Y = exponential(X, U, t)\n if nargin == 2\n t = 1;\n end\n tU = t*U;\n Y = zeros(size(X));\n for i = 1 : k\n % From a formula by Ross Lippert, Example 5.4.2 in AMS08.\n Xi = X(:, :, i);\n Ui = tU(:, :, i);\n Y(:, :, i) = [Xi Ui] * ...\n expm([Xi'*Ui , -Ui'*Ui ; eye(p) , Xi'*Ui]) * ...\n [ expm(-Xi'*Ui) ; zeros(p) ];\n end\n \n end\n\n M.hash = @(X) ['z' hashmd5([real(X(:)) ; imag(X(:))])]; %! X(:) -> [real(X(:)) ; imag(X(:))]\n \n M.rand = @random;\n function X = random()\n X = zeros(n, p, k);\n for i = 1 : k\n [Q, unused] = qr(randn(n, p) + 1i*randn(n,p), 0); %#ok %! Complex version\n X(:, :, i) = Q;\n end\n end\n \n M.randvec = @randomvec;\n function U = randomvec(X)\n U = projection(X, randn(n, p, k) + 1i*randn(n, p, k)); %! Complex version\n U = U / norm(U(:));\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, p, k);\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \n M.vec = @(x, u_mat) [real(u_mat(:)) ; imag(u_mat(:))];\n M.mat = @(x, u_vec) reshape(u_vec(1:(n*p*k)) + 1i*u_vec((n*p*k+1):end), [n, p, k]);\n M.vecmatareisometries = @() true; % TODO : to check.\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/stiefel/stiefelfactory.m", "content": "function M = stiefelfactory(n, p, k)\n% Returns a manifold structure to optimize over orthonormal matrices.\n%\n% function M = stiefelfactory(n, p)\n% function M = stiefelfactory(n, p, k)\n%\n% The Stiefel manifold is the set of orthonormal nxp matrices. If k\n% is larger than 1, this is the Cartesian product of the Stiefel manifold\n% taken k times. The metric is such that the manifold is a Riemannian\n% submanifold of R^nxp equipped with the usual trace inner product, that\n% is, it is the usual metric.\n%\n% Points are represented as matrices X of size n x p x k (or n x p if k=1,\n% which is the default) such that each n x p matrix is orthonormal,\n% i.e., X'*X = eye(p) if k = 1, or X(:, :, i)' * X(:, :, i) = eye(p) for\n% i = 1 : k if k > 1. Tangent vectors are represented as matrices the same\n% size as points.\n%\n% By default, k = 1.\n%\n% See also: grassmannfactory rotationsfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n% July 5, 2013 (NB) : Added ehess2rhess.\n% Jan. 27, 2014 (BM) : Bug in ehess2rhess corrected.\n% June 24, 2014 (NB) : Added true exponential map and changed the randvec\n% function so that it now returns a globally\n% normalized vector, not a vector where each\n% component is normalized (this only matters if k>1).\n\n \n if ~exist('k', 'var') || isempty(k)\n k = 1;\n end\n \n if k == 1\n M.name = @() sprintf('Stiefel manifold St(%d, %d)', n, p);\n elseif k > 1\n M.name = @() sprintf('Product Stiefel manifold St(%d, %d)^%d', n, p, k);\n else\n error('k must be an integer no less than 1.');\n end\n \n M.dim = @() k*(n*p - .5*p*(p+1));\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:));\n \n M.dist = @(x, y) error('stiefel.dist not implemented yet.');\n \n M.typicaldist = @() sqrt(p*k);\n \n M.proj = @projection;\n function Up = projection(X, U)\n \n XtU = multiprod(multitransp(X), U);\n symXtU = multisym(XtU);\n Up = U - multiprod(X, symXtU);\n \n% The code above is equivalent to, but much faster than, the code below.\n% \n% Up = zeros(size(U));\n% function A = sym(A), A = .5*(A+A'); end\n% for i = 1 : k\n% Xi = X(:, :, i);\n% Ui = U(:, :, i);\n% Up(:, :, i) = Ui - Xi*sym(Xi'*Ui);\n% end\n\n end\n \n M.tangent = M.proj;\n \n % For Riemannian submanifolds, converting a Euclidean gradient into a\n % Riemannian gradient amounts to an orthogonal projection.\n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, H)\n XtG = multiprod(multitransp(X), egrad);\n symXtG = multisym(XtG);\n HsymXtG = multiprod(H, symXtG);\n rhess = projection(X, ehess - HsymXtG);\n end\n \n M.retr = @retraction;\n function Y = retraction(X, U, t)\n if nargin < 3\n t = 1.0;\n end\n Y = X + t*U;\n for i = 1 : k\n [Q, R] = qr(Y(:, :, i), 0);\n % The instruction with R assures we are not flipping signs\n % of some columns, which should never happen in modern Matlab\n % versions but may be an issue with older versions.\n Y(:, :, i) = Q * diag(sign(sign(diag(R))+.5));\n end\n end\n \n M.exp = @exponential;\n function Y = exponential(X, U, t)\n if nargin == 2\n t = 1;\n end\n tU = t*U;\n Y = zeros(size(X));\n for i = 1 : k\n % From a formula by Ross Lippert, Example 5.4.2 in AMS08.\n Xi = X(:, :, i);\n Ui = tU(:, :, i);\n Y(:, :, i) = [Xi Ui] * ...\n expm([Xi'*Ui , -Ui'*Ui ; eye(p) , Xi'*Ui]) * ...\n [ expm(-Xi'*Ui) ; zeros(p) ];\n end\n \n end\n\n M.hash = @(X) ['z' hashmd5(X(:))];\n \n M.rand = @random;\n function X = random()\n X = zeros(n, p, k);\n for i = 1 : k\n [Q, unused] = qr(randn(n, p), 0); %#ok\n X(:, :, i) = Q;\n end\n end\n \n M.randvec = @randomvec;\n function U = randomvec(X)\n U = projection(X, randn(n, p, k));\n U = U / norm(U(:));\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, p, k);\n \n M.transp = @(x1, x2, d) projection(x2, d);\n \n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, [n, p, k]);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/stiefel/stiefelgeneralizedfactory.m", "content": "function M = stiefelgeneralizedfactory(n, p, B)\n% Returns a manifold structure of \"scaled\" orthonormal matrices.\n%\n% function M = stiefelgeneralizedfactory(n, p)\n% function M = stiefelgeneralizedfactory(n, p, B)\n%\n% The generalized Stiefel manifold is the set of \"scaled\" orthonormal \n% nxp matrices X such that X'*B*X is identity. B must be positive definite.\n% If B is identity, then this is the standard Stiefel manifold.\n%\n% The generalized Stiefel manifold is endowed with a scaled metric\n% by making it a Riemannian submanifold of the Euclidean space,\n% again endowed with the scaled inner product.\n%\n% Some notions (not all) are from Section 4.5 of the paper\n% \"The geometry of algorithms with orthogonality constraints\",\n% A. Edelman, T. A. Arias, S. T. Smith, SIMAX, 1998.\n%\n% Paper link: http://arxiv.org/abs/physics/9806030.\n%\n% Note: egrad2rgrad and ehess2rhess involve solving linear systems in B. If\n% this is a bottleneck for a specific application, then a way forward is to\n% create a modified version of this file which preprocesses B to speed this\n% up (typically, by computing a Cholesky factorization of it, then calling\n% an appropriate solver).\n%\n% See also: stiefelfactory grassmannfactory grassmanngeneralizedfactory \n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, June 30, 2015.\n% Contributors:\n%\n% Change log:\n% \n\n \n if ~exist('B', 'var') || isempty(B)\n B = speye(n); % Standard Stiefel manifold.\n end\n \n M.name = @() sprintf('Generalized Stiefel manifold St(%d, %d)', n, p);\n \n M.dim = @() (n*p - .5*p*(p+1));\n \n M.inner = @(X, eta, zeta) trace(eta'*(B*zeta)); % Scaled metric.\n \n M.norm = @(X, eta) sqrt(M.inner(X, eta, eta));\n \n M.dist = @(X, Y) error('stiefelgeneralizedfactory.dist not implemented yet.');\n \n M.typicaldist = @() sqrt(p);\n \n % Orthogonal projection of an ambient vector U to the tangent space\n % at X.\n M.proj = @projection;\n function Up = projection(X, U)\n BX = B*X;\n \n % Projection onto the tangent space\n Up = U - X*symm(BX'*U); \n end\n \n M.tangent = M.proj;\n \n M.egrad2rgrad = @egrad2rgrad;\n function rgrad = egrad2rgrad(X, egrad)\n \n % First, scale egrad according the to the scaled metric in the\n % Euclidean space.\n egrad_scaled = B\\egrad;\n \n % Second, project onto the tangent space.\n % rgrad = egrad_scaled - X*symm((B*X)'*egrad_scaled);\n %\n % Verify that symm(BX'*egrad_scaled) = symm(X'*egrad).\n \n rgrad = egrad_scaled - X*symm(X'*egrad);\n end\n \n \n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(X, egrad, ehess, H)\n egraddot = ehess;\n Xdot = H;\n \n % Directional derivative of the Riemannian gradient.\n egrad_scaleddot = B\\egraddot;\n rgraddot = egrad_scaleddot - Xdot*symm(X'*egrad)...\n - X*symm(Xdot'*egrad)...\n - X*symm(X'*egraddot);\n \n % Project onto the tangent space.\n rhess = M.proj(X, rgraddot);\n end\n \n \n M.retr = @retraction;\n function Y = retraction(X, U, t)\n if nargin < 3\n t = 1.0;\n end\n Y = guf(X + t*U); % Ensure that Y'*B*Y is identity.\n end\n \n \n M.exp = @exponential;\n function Y = exponential(X, Z, t)\n if nargin < 3\n t = 1.0;\n end\n Y = retraction(X, Z, t);\n warning('manopt:stiefelgeneralizedfactory:exp', ...\n ['Exponential for generalized Stiefel manifold ' ...\n 'manifold not implemented yet. Used retraction instead.']);\n end\n\n\n M.hash = @(X) ['z' hashmd5(X(:))];\n \n M.rand = @random;\n function X = random()\n X = guf(randn(n, p)); % Ensure that X'*B*X is identity;\n end\n \n M.randvec = @randomvec;\n function U = randomvec(X)\n U = projection(X, randn(n, p));\n U = U / norm(U(:));\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(X) zeros(n, p);\n \n % This transport is compatible with the generalized polar retraction.\n M.transp = @(X1, X2, d) projection(X2, d);\n \n M.vec = @(X, u_mat) u_mat(:);\n M.mat = @(X, u_vec) reshape(u_vec, [n, p]);\n M.vecmatareisometries = @() false;\n \n % Some auxiliary functions\n symm = @(D) (D + D')/2;\n \n function X = guf(Y)\n % Generalized polar decomposition of an n-by-p matrix Y.\n % X'*B*X is identity.\n \n % Method 1\n [u, ~, v] = svd(Y, 0);\n \n % Instead of the following three steps, an equivalent, but an \n % expensive way is to do X = u*(sqrtm(u'*(B*u))\\(v')).\n [q, ssquare] = eig(u'*(B*u));\n qsinv = q/sparse(diag(sqrt(diag(ssquare))));\n X = u*((qsinv*q')*v'); % X'*B*X is identity.\n \n \n % Another computation using restricted_svd\n % [u, ~, v] = restricted_svd(Y);\n % X = u*v'; % X'*B*X is identity.\n \n end\n \n function [u, s, v] = restricted_svd(Y)\n % We compute a thin svd-like decomposition of an n-by-p matrix Y \n % into matrices u, s, and v such that u is an n-by-p matrix\n % with u'*B*u being identity, s is a p-by-p diagonal matrix \n % with positive entries, and v is a p-by-p orthogonal matrix.\n % Y = u*s*v'.\n [v, ssquare] = eig(symm(Y'*(B*Y))); % Y*B*Y is positive definite\n ssquarevec = diag(ssquare);\n \n s = sparse(diag(abs(sqrt(ssquarevec))));\n u = Y*(v/s); % u'*B*u is identity.\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/stiefel/stiefelstackedfactory.m", "content": "function M = stiefelstackedfactory(m, d, k)\n% Stiefel(k, d)^m, represented as matrices of size m*d-by-k.\n%\n% function M = stiefelstackedfactory(m, d, k)\n%\n% Points on this manifold are matrices Y of size n x k, with n = m*d.\n% Y is thought of as m matrices of size d x k each, stacked on top of each\n% other. Call them Y1, ..., Ym. Each Yi is an orthonormal matrix, that is,\n% its d rows are unit norm and are orthogonal to each other. Thus, this\n% geometry is a product of Stiefel manifolds.\n% \n% To easily transform matrices Y to 3D arrays Y3 of size d x k x m such\n% that each slice Y3(:, :, i) corresponds to one of the matrices Yi, use\n% the functions\n% \n% Y3 = M.to3D(Y) and Y = M.to2D(Y3).\n%\n% The ambient space R^(nxk) is endowed with the usual inner product\n% = trace(A'*B). This inner product is restricted to the tangent\n% spaces of the present manifold, thus making it a Riemannian submanifold\n% of the Euclidean space R^(nxk). Tangent vectors are represented as\n% matrices of the same size as Y, and can likewise be converted to 3D\n% arrays and back using to3D() and to2D().\n%\n% In dealing with this geometry, especially when dealing with the 3D array\n% representations of points and tangent vectors, the tools multiprod,\n% multitransp, multitrace, multiscale etc. available in Manopt are often\n% useful.\n%\n% See also: stiefelfactory obliquefactory multiprod multitransp\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, May 4, 2015.\n% Contributors: \n% Change log: \n\n assert(k >= d, 'k must be at least as large as d.');\n\n n = m*d;\n \n M.name = @() sprintf('Manifold of %d orthonormal matrices of size %dx%d, stacked', m, d, k);\n \n M.dim = @() m*(k*d - .5*d*(d+1));\n \n M.size = @() [m, d, k];\n \n M.inner = @(x, d1, d2) d1(:).'*d2(:);\n \n M.norm = @(x, d) norm(d(:));\n \n M.dist = @(x, y) error('stiefelstackedfactory.dist not implemented yet.');\n \n M.typicaldist = @() sqrt(M.dim());\n\n % Convert a dxkxm matrix to an nxk matrix\n M.to2D = @to2D;\n function A2 = to2D(A3)\n A2 = reshape(multitransp(A3), [k, m*d])';\n end\n\n % Convert an nxk matrix to a dxkxm matrix\n M.to3D = @to3D;\n function A3 = to3D(A2)\n A3 = multitransp(reshape(A2', [k, d, m]));\n end\n\n % Given 2 3D matrices A and B of size dxkxm, returns a 3D matrix C of\n % size dxdxm such that each slice C(:, :, i) is the symmetric part of\n % the product A(:, :, i) * B(:, :, i)'. The name is short for\n % \"symmetric-block-diagonal\", because if A and B were transformed to\n % their 2D equivalents via to2D, then the output would contain the\n % symmetric parts of the diagonal blocks of A*B'.\n M.symbdiag = @symbdiag;\n function C = symbdiag(A, B)\n C = multisym(multiprod(A, multitransp(B)));\n end\n \n % Orthogonal projection from the ambient space R^(nxk) to the tangent\n % space at X.\n M.proj = @projection;\n function Zt = projection(Y, Z)\n Y3 = to3D(Y);\n Z3 = to3D(Z);\n Lambda = symbdiag(Y3, Z3);\n Zt3 = Z3 - multiprod(Lambda, Y3);\n Zt = to2D(Zt3);\n end \n \n M.tangent = M.proj;\n \n\tM.egrad2rgrad = M.proj;\n \n M.ehess2rhess = @ehess2rhess;\n function rhess = ehess2rhess(Y, egrad, ehess, Ydot)\n Y3 = to3D(Y);\n Ydot3 = to3D(Ydot);\n egrad3 = to3D(egrad);\n C = symbdiag(Y3, egrad3);\n CYdot = to2D(multiprod(C, Ydot3));\n rhess = projection(Y, ehess - CYdot);\n end\n \n M.retr = @retraction;\n function Y = retraction(Y, U, t)\n if nargin < 3\n t = 1.0;\n end\n Y = Y + t*U;\n Y3 = to3D(Y);\n for i = 1 : m\n % Orthonormalize the rows of Y3(:, :, i):\n [u, s, v] = svd(Y3(:, :, i), 'econ'); %#ok\n Y3(:, :, i) = u*v';\n % Alternative code if one desires to use QR instead of SVD.\n % The instruction with the signs of R assures we are not\n % flipping signs of some columns.\n % [Q, R] = qr(Y3(:, :, i)', 0);\n % Y3(:, :, i) = (Q * diag(sign(sign(diag(R))+.5)))';\n end\n Y = to2D(Y3);\n end\n \n M.exp = @exponential;\n function Y = exponential(Y, U, t)\n if nargin == 2\n t = 1;\n end\n tU3 = multitransp(to3D(t*U));\n Y3 = multitransp(to3D(Y));\n % From a formula by Ross Lippert, Example 5.4.2 in AMS08.\n for i = 1 : m\n X = Y3(:, :, i);\n Z = tU3(:, :, i);\n Y3(:, :, i) = [X, Z] * ...\n expm([ X'*Z , -Z'*Z ; eye(d) , X'*Z]) * ...\n [ expm(-X'*Z) ; zeros(d) ];\n % We may loose orthonormality here. Just to be sure:\n [u, s, v] = svd(Y3(:, :, i), 'econ'); %#ok\n Y3(:, :, i) = u*v';\n end\n Y = to2D(multitransp(Y3));\n end\n\n M.hash = @(Y) ['z' hashmd5(Y(:))];\n \n M.rand = @random;\n function Y = random()\n Y3 = zeros(d, k, m);\n for i = 1 : m\n [Q, unused] = qr(randn(k, d), 0); %#ok\n Y3(:, :, i) = Q';\n end\n Y = to2D(Y3);\n end\n \n M.randvec = @randomvec;\n function U = randomvec(Y)\n U = projection(Y, randn(n, k));\n U = U / M.norm(Y, U);\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(x) zeros(n, k);\n \n M.transp = @(x1, x2, u) projection(x2, u);\n \n M.vec = @(x, u_mat) u_mat(:);\n M.mat = @(x, u_vec) reshape(u_vec, [n, k]);\n M.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/symfixedrank/elliptopefactory.m", "content": "function M = elliptopefactory(n, k)\n% Manifold of n-by-n psd matrices of rank k with unit diagonal elements.\n%\n% function M = elliptopefactory(n, k)\n%\n% A point X on the manifold is parameterized as YY^T where Y is a matrix of\n% size nxk. As such, X is symmetric, positive semidefinite. We restrict to\n% full-rank Y's, such that X has rank exactly k. The point X is numerically\n% represented by Y (this is more efficient than working with X, which may\n% be big). Tangent vectors are represented as matrices of the same size as\n% Y, call them Ydot, so that Xdot = Y Ydot' + Ydot Y and diag(Xdot) == 0.\n% The metric is the canonical Euclidean metric on Y.\n% \n% The diagonal constraints on X (X(i, i) == 1 for all i) translate to\n% unit-norm constraints on the rows of Y: norm(Y(i, :)) == 1 for all i.\n% The set of such Y's forms the oblique manifold. But because for any\n% orthogonal Q of size k, it holds that (YQ)(YQ)' = YY', we \"group\" all\n% matrices of the form YQ in an equivalence class. The set of equivalence\n% classes is a Riemannian quotient manifold, implemented here.\n%\n% Note that this geometry formally breaks down at rank-deficient Y's.\n% This does not appear to be a major issue in practice when optimization\n% algorithms converge to rank-deficient Y's, but convergence theorems no\n% longer hold. As an alternative, you may use the oblique manifold (it has\n% larger dimension, but does not break down at rank drop.)\n%\n% The geometry is taken from the 2010 paper:\n% M. Journee, P.-A. Absil, F. Bach and R. Sepulchre,\n% \"Low-Rank Optimization on the Cone of Positive Semidefinite Matrices\".\n% Paper link: http://www.di.ens.fr/~fbach/journee2010_sdp.pdf\n% \n% \n% Please cite the Manopt paper as well as the research paper:\n% @Article{journee2010low,\n% Title = {Low-rank optimization on the cone of positive semidefinite matrices},\n% Author = {Journ{\\'e}e, M. and Bach, F. and Absil, P.-A. and Sepulchre, R.},\n% Journal = {SIAM Journal on Optimization},\n% Year = {2010},\n% Number = {5},\n% Pages = {2327--2351},\n% Volume = {20},\n% Doi = {10.1137/080731359}\n% }\n% \n%\n% See also: obliquefactory symfixedrankYYfactory spectrahedronfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, July 12, 2013.\n% Contributors:\n% Change log:\n% July 18, 2013 (NB):\n% Fixed projection operator for rank-deficient Y'Y.\n% \n% Aug. 8, 2013 (NB):\n% No longer using nested functions, to aim at Octave compatibility.\n% Sign error in right hand side of the call to minres corrected.\n% \n% June 24, 2014 (NB):\n% Used code snippets from obliquefactory to speed up projection,\n% retraction, egrad2rgrad and rand: the code now uses bsxfun for this.\n% \n% April 3, 2015 (NB):\n% Replaced trace(A'*B) by A(:)'*B(:) : equivalent but faster.\n\n% TODO: modify normalize_rows and project_rows to work without transposes.\n% TODO: enhance ehess2rhess to also use bsxfun.\n \n\t\n\tif ~exist('lyap', 'file')\n\t\twarning('manopt:elliptopefactory:slowlyap', ...\n\t\t ['The function lyap to solve Lyapunov equations seems not to ' ...\n\t\t\t\t'be available. This may slow down optimization over this ' ...\n\t\t\t\t'manifold significantly. lyap is part of the control system ' ...\n\t\t\t\t'toolbox.']);\n end\n \n if k < 2\n warning('manopt:elliptopefactory:lowk', ...\n 'k should be an integer >= 2. At k = 1, the set is discrete.');\n end\n \n \n M.name = @() sprintf('YY'' quotient manifold of %dx%d psd matrices of rank %d with diagonal elements being 1', n, k);\n \n M.dim = @() n*(k-1) - k*(k-1)/2; % Extra -1 is because of the diagonal constraint that\n \n % Euclidean metric on the total space\n M.inner = @(Y, eta, zeta) eta(:)'*zeta(:);\n \n M.norm = @(Y, eta) sqrt(M.inner(Y, eta, eta));\n \n M.dist = @(Y, Z) error('elliptopefactory.dist not implemented yet.');\n \n M.typicaldist = @() 10*k;\n \n M.proj = @projection;\n \n M.tangent = M.proj;\n M.tangent2ambient = @(Y, eta) eta;\n \n M.retr = @retraction;\n \n M.egrad2rgrad = @egrad2rgrad;\n \n M.ehess2rhess = @ehess2rhess;\n \n M.exp = @exponential;\n \n % Notice that the hash of two equivalent points will be different...\n M.hash = @(Y) ['z' hashmd5(Y(:))];\n \n M.rand = @() random(n, k);\n \n M.randvec = @randomvec;\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(Y) zeros(n, k);\n \n M.transp = @(Y1, Y2, d) projection(Y2, d);\n \n M.vec = @(Y, u_mat) u_mat(:);\n M.mat = @(Y, u_vec) reshape(u_vec, [n, k]);\n M.vecmatareisometries = @() true;\n \nend\n\n% Given a matrix X, returns the same matrix but with each column scaled so\n% that they have unit 2-norm.\n% See obliquefactory.\nfunction X = normalize_rows(X)\n X = X';\n\tnorms = sqrt(sum(X.^2, 1));\n\tX = bsxfun(@times, X, 1./norms);\n X = X';\nend\n\n% Orthogonal projection of each row of H to the tangent space at the\n% corresponding row of X, seen as a point on a sphere.\n% See obliquefactory.\nfunction PXH = project_rows(X, H)\n X = X';\n H = H';\n % Compute the inner product between each vector H(:, i) with its root\n % point X(:, i), that is, X(:, i).' * H(:, i). Returns a row vector.\n inners = sum(X.*H, 1);\n % Subtract from H the components of the H(:, i)'s that are parallel to\n % the root points X(:, i).\n PXH = H - bsxfun(@times, X, inners);\n PXH = PXH';\nend\n\n\n% Projection onto the tangent space, i.e., on the tangent space of\n% ||Y(i, :)|| = 1\nfunction etaproj = projection(Y, eta)\n [unused, k] = size(Y); %#ok\n eta = project_rows(Y, eta);\n\n % Projection onto the horizontal space\n YtY = Y'*Y;\n SS = YtY;\n AS = Y'*eta - eta'*Y;\n try\n % This is supposed to work and indeed return a skew-symmetric\n % solution Omega.\n Omega = lyap(SS, -AS);\n catch up %#ok\n % It can happen though that SS will be rank deficient. The\n % Lyapunov equation we solve still has a unique skew-symmetric\n % solution, but solutions with a symmetric part now also exist,\n % and the lyap function doesn't like that. So we want to\n % extract the minimum norm solution. This is also useful if lyap is\n\t\t% not available (it is part of the control system toolbox).\n mat = @(x) reshape(x, [k k]);\n vec = @(X) X(:);\n is_octave = exist('OCTAVE_VERSION', 'builtin');\n if ~is_octave\n [vecomega, unused] = minres(@(x) vec(SS*mat(x) + mat(x)*SS), vec(AS)); %#ok\n else\n [vecomega, unused] = gmres(@(x) vec(SS*mat(x) + mat(x)*SS), vec(AS)); %#ok\n end\n Omega = mat(vecomega);\n end\n % % Make sure the result is skew-symmetric (does not seem necessary).\n % Omega = (Omega-Omega')/2;\n etaproj = eta - Y*Omega;\nend\n\n% Retraction\nfunction Ynew = retraction(Y, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Ynew = Y + t*eta;\n Ynew = normalize_rows(Ynew);\nend\n\n% Exponential map\nfunction Ynew = exponential(Y, eta, t)\n if nargin < 3\n t = 1.0;\n end\n\n Ynew = retraction(Y, eta, t);\n warning('manopt:elliptopefactory:exp', ...\n ['Exponential for fixed rank spectrahedron ' ...\n 'manifold not implemented yet. Used retraction instead.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:elliptopefactory:exp'')']);\nend\n\n% Euclidean gradient to Riemannian gradient conversion.\n% We only need the ambient space projection: the remainder of the\n% projection function is not necessary because the Euclidean gradient must\n% already be orthogonal to the vertical space.\nfunction rgrad = egrad2rgrad(Y, egrad)\n rgrad = project_rows(Y, egrad);\nend\n\n% Euclidean Hessian to Riemannian Hessian conversion.\n% TODO: speed this function up using bsxfun.\nfunction Hess = ehess2rhess(Y, egrad, ehess, eta)\n k = size(Y, 2);\n\n % Directional derivative of the Riemannian gradient\n scaling_grad = sum((egrad.*Y), 2); % column vector of size n\n scaling_grad_repeat = scaling_grad*ones(1, k);\n\n Hess = ehess - scaling_grad_repeat.*eta;\n\n scaling_hess = sum((eta.*egrad) + (Y.*ehess), 2);\n scaling_hess_repeat = scaling_hess*ones(1, k);\n % directional derivative of scaling_grad_repeat\n Hess = Hess - scaling_hess_repeat.*Y;\n\n % Project on the horizontal space\n Hess = projection(Y, Hess);\nend\n\n% Random point generation on the manifold\nfunction Y = random(n, k)\n Y = randn(n, k);\n Y = normalize_rows(Y);\nend\n\n% Random vector generation at Y\nfunction eta = randomvec(Y)\n eta = randn(size(Y));\n eta = projection(Y, eta);\n nrm = norm(eta, 'fro');\n eta = eta / nrm;\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/symfixedrank/spectrahedronfactory.m", "content": "function M = spectrahedronfactory(n, k)\n% Manifold of n-by-n symmetric positive semidefinite matrices of rank k\n% with trace (sum of diagonal elements) equal to 1.\n%\n% function M = spectrahedronfactory(n, k)\n%\n% A point X on the manifold is parameterized as YY^T where Y is a matrix of\n% size nxk. As such, X is symmetric, positive semidefinite. We restrict to\n% full-rank Y's, such that X has rank exactly k. The point X is numerically\n% represented by Y (this is more efficient than working with X, which may\n% be big). Tangent vectors are represented as matrices of the same size as\n% Y, call them Ydot, so that Xdot = Y Ydot' + Ydot Y and trace(Xdot) == 0.\n% The metric is the canonical Euclidean metric on Y.\n% \n% The trace constraint on X (trace(X) == 1) translates to a unit Frobenius\n% norm constraint on Y: trace(X) = norm(Y, 'fro')^2 == 1. The set of such\n% Y's forms the unit sphere in R^(nxk): see spherefactory. But because for\n% any orthogonal Q of size k, it holds that (YQ)(YQ)' = YY', we \"group\" all\n% matrices of the form YQ in an equivalence class. The set of equivalence\n% classes is a Riemannian quotient manifold, implemented here.\n%\n%\n% Note that this geometry formally breaks down at rank-deficient Y's.\n% As an alternative, you may use the sphere manifold (it has larger\n% dimension (by 1), but does not break down at rank drop.)\n%\n% The geometry is taken from the 2010 paper:\n% M. Journee, P.-A. Absil, F. Bach and R. Sepulchre,\n% \"Low-Rank Optimization on the Cone of Positive Semidefinite Matrices\".\n% Paper link: http://www.di.ens.fr/~fbach/journee2010_sdp.pdf\n% \n% \n% Please cite the Manopt paper as well as the research paper:\n% @Article{journee2010low,\n% Title = {Low-rank optimization on the cone of positive semidefinite matrices},\n% Author = {Journ{\\'e}e, M. and Bach, F. and Absil, P.-A. and Sepulchre, R.},\n% Journal = {SIAM Journal on Optimization},\n% Year = {2010},\n% Number = {5},\n% Pages = {2327--2351},\n% Volume = {20},\n% Doi = {10.1137/080731359}\n% }\n% \n%\n% See also: spherefactory elliptopefactory symfixedrankYYfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, July 11, 2013.\n% Contributors: Nicolas Boumal\n% Change log:\n%\n% April 2, 2015 (NB):\n% Replaced trace(A'*B) by A(:)'*B(:) (equivalent but faster).\n% Updated documentation.\n \n \n \n M.name = @() sprintf('YY'' quotient manifold of %dx%d psd matrices of rank %d with trace 1', n, k);\n \n M.dim = @() n*k - 1 - k*(k-1)/2;\n \n % Euclidean metric on the total space\n M.inner = @(Y, eta, zeta) eta(:)'*zeta(:);\n \n M.norm = @(Y, eta) sqrt(M.inner(Y, eta, eta));\n \n M.dist = @(Y, Z) error('spectrahedronfactory.dist not implemented yet.');\n \n M.typicaldist = @() 10*k;\n \n M.proj = @projection;\n function etaproj = projection(Y, eta)\n % Projection onto the tangent space, i.e., on the tangent space of\n % ||Y|| = 1\n \n eta = eta - (eta(:)'*Y(:))*Y;\n \n % Projection onto the horizontal space\n YtY = Y'*Y;\n SS = YtY;\n AS = Y'*eta - eta'*Y;\n Omega = lyap(SS, -AS);\n etaproj = eta - Y*Omega;\n end\n \n M.tangent = M.proj;\n M.tangent2ambient = @(Y, eta) eta;\n \n M.retr = @retraction;\n function Ynew = retraction(Y, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Ynew = Y + t*eta;\n Ynew = Ynew/norm(Ynew, 'fro');\n end\n \n \n M.egrad2rgrad = @(Y, eta) eta - (eta(:)'*Y(:))*Y;\n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(Y, egrad, ehess, eta)\n \n % Directional derivative of the Riemannian gradient\n Hess = ehess - (egrad(:)'*Y(:))*eta - ( (ehess(:)'*Y(:)) + (eta(:)'*egrad(:)) )*Y;\n Hess = Hess - (Hess(:)'*Y(:))*Y;\n \n % Project on the horizontal space\n Hess = M.proj(Y, Hess);\n \n end\n \n M.exp = @exponential;\n function Ynew = exponential(Y, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n Ynew = retraction(Y, eta, t);\n warning('manopt:spectrahedronfactory:exp', ...\n ['Exponential for fixed rank spectrahedron ' ...\n 'manifold not implenented yet. Used retraction instead.']);\n end\n \n % Notice that the hash of two equivalent points will be different...\n M.hash = @(Y) ['z' hashmd5(Y(:))];\n \n M.rand = @random;\n \n function Y = random()\n Y = randn(n, k);\n Y = Y/norm(Y,'fro');\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(Y)\n eta = randn(n, k);\n eta = projection(Y, eta);\n nrm = M.norm(Y, eta);\n eta = eta / nrm;\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(Y) zeros(n, k);\n \n M.transp = @(Y1, Y2, d) projection(Y2, d);\n \n M.vec = @(Y, u_mat) u_mat(:);\n M.mat = @(Y, u_vec) reshape(u_vec, [n, k]);\n M.vecmatareisometries = @() true;\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/symfixedrank/symfixedrankYYcomplexfactory.m", "content": "function M = symfixedrankYYcomplexfactory(n, k)\n% Manifold of n x n complex Hermitian pos. semidefinite matrices of rank k.\n%\n% function M = symfixedrankYYcomplexfactory(n, k)\n%\n% Manifold of n-by-n complex Hermitian positive semidefinite matrices of \n% fixed rank k. This follows the quotient geometry described \n% in Sarod Yatawatta's 2013 paper:\n% \"Radio interferometric calibration using a Riemannian manifold\", ICASSP.\n%\n% Paper link: http://dx.doi.org/10.1109/ICASSP.2013.6638382.\n%\n% A point X on the manifold M is parameterized as YY^*, where \n% Y is a complex matrix of size nxk. For any point Y on the manifold M, \n% given any kxk complex unitary matrix U, we say Y*U is equivalent to Y, \n% i.e., YY^* does not change. Therefore, M is the set of equivalence \n% classes and is a Riemannian quotient manifold C^{nk}/SU(k). \n% The metric is the usual real-trace inner product, that is, \n% it is the usual metric for the complex plane identified with R^2.\n%\n% Notice that this manifold is not complete: if optimization leads Y to be\n% rank-deficient, the geometry will break down. Hence, this geometry should\n% only be used if it is expected that the points of interest will have rank\n% exactly k. Reduce k if that is not the case.\n%\n% The geometry is based on the following papers (and references therein).\n% Please cite the Manopt paper as well as the research papers:\n%\n% @INPROCEEDINGS{Yatawatta2013A,\n% author={Yatawatta, S.},\n% booktitle={Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on},\n% title={Radio interferometric calibration using a {R}iemannian manifold},\n% year={2013},\n% month={May},\n% pages={3866--3870},\n% doi={10.1109/ICASSP.2013.6638382},\n% ISSN={1520-6149},\n% }\n%\n% @article{Yatawatta2013B,\n% author = {Yatawatta, S.}, \n% title = {On the interpolation of calibration solutions obtained in radio interferometry},\n% volume = {428}, \n% number = {1}, \n% pages = {828--833}, \n% year = {2013}, \n% doi = {10.1093/mnras/sts069}, \n% journal = {Monthly Notices of the Royal Astronomical Society} \n% }\n%\n% See also: symfixedrankYYfactory sympositivedefinitefactory\n\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Sarod Yatawatta, June 29, 2015.\n% Contributors: Bamdev Mishra.\n% Change log:\n%\n% June 28, 2016 (NB):\n% Metric scaled down by factor 2 to match the metric used in\n% euclideancomplexfactory.\n \n M.name = @() sprintf('YY'' quotient manifold of Hermitian %dx%d complex matrices of rank %d.', n, n, k);\n \n M.dim = @() 2*k*n - k*k; % SY: dim of ambient space (2*k*n) - dim of kxk unitary matrix (k^2).\n \n % Euclidean metric on the total space.\n % BM: equivalent to real(trace(eta'*zeta)), but more efficient.\n M.inner = @(Y, eta, zeta) real(eta(:)'*zeta(:));\n \n M.norm = @(Y, eta) sqrt(M.inner(Y, eta, eta));\n \n % Find unitary U to minimize ||Y - Z*U||,\n % i.e., the Procrustes problem, with svd(Y'*Z).\n M.dist = @(Y, Z) distance;\n function distval = distance(Y, Z)\n [u, ignore, v] = svd(Z'*Y); %#ok\n E = Y - Z*u*v'; % SY: checked.\n distval = real(E(:)'*E(:));\n end\n \n M.typicaldist = @() 10*k; % BM: To do.\n \n M.proj = @projection;\n function etaproj = projection(Y, eta)\n % Projection onto the horizontal space\n xx = Y'*Y;\n rr = Y'*eta - eta'*Y;\n Omega = lyap(xx, -rr);\n etaproj = eta - Y*Omega;\n end\n \n M.tangent = M.proj;\n M.tangent2ambient = @(Y, eta) eta;\n \n M.retr = @retraction;\n function Ynew = retraction(Y, eta, t)\n if nargin < 3\n t = 1.0;\n end\n Ynew = Y + t*eta;\n end\n \n \n M.egrad2rgrad = @(Y, eta) eta;\n M.ehess2rhess = @(Y, egrad, ehess, U) M.proj(Y, ehess);\n \n \n M.exp = @exponential;\n function Ynew = exponential(Y, eta, t)\n if nargin < 3\n t = 1.0;\n end\n \n Ynew = retraction(Y, eta, t);\n warning('manopt:symfixedrankYYcomplexfactory:exp', ...\n ['Exponential for symmetric fixed-rank complex ' ...\n 'manifold not implemented yet. Used retraction instead.']);\n end\n \n % Notice that the hash of two equivalent points will be different...\n M.hash = @(Y) ['z' hashmd5([real(Y(:)); imag(Y(:))])];\n \n M.rand = @random;\n function Y = random()\n Y = randn(n, k) + 1i*randn(n,k);\n end\n \n M.randvec = @randomvec;\n function eta = randomvec(Y)\n eta = randn(n, k) + 1i*randn(n,k);\n eta = projection(Y, eta);\n nrm = M.norm(Y, eta);\n eta = eta / nrm;\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(Y) zeros(n, k);\n \n M.transp = @(Y1, Y2, d) projection(Y2, d);\n \n M.vec = @(Y, u_mat) [real(u_mat(:)); imag(u_mat(:))];\n M.mat = @(Y, u_vec) reshape(u_vec(1 : n*k), [n, k]) + 1i*reshape(u_vec(n*k + 1: end), [n, k]);\n M.vecmatareisometries = @() true; \n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/symfixedrank/symfixedrankYYfactory.m", "content": "function M = symfixedrankYYfactory(n, k)\n% Manifold of n-by-n symmetric positive semidefinite matrices of rank k.\n%\n% function M = symfixedrankYYfactory(n, k)\n%\n% A point X on the manifold is parameterized as YY^T where Y is a matrix of\n% size nxk. As such, X is symmetric, positive semidefinite. We restrict to\n% full-rank Y's, such that X has rank exactly k. The point X is numerically\n% represented by Y (this is more efficient than working with X, which may\n% be big). Tangent vectors are represented as matrices of the same size as\n% Y, call them Ydot, so that Xdot = Y Ydot' + Ydot Y. The metric is the\n% canonical Euclidean metric on Y.\n% \n% Since for any orthogonal Q of size k, it holds that (YQ)(YQ)' = YY',\n% we \"group\" all matrices of the form YQ in an equivalence class. The set\n% of equivalence classes is a Riemannian quotient manifold, implemented\n% here.\n%\n% Notice that this manifold is not complete: if optimization leads Y to be\n% rank-deficient, the geometry will break down. Hence, this geometry should\n% only be used if it is expected that the points of interest will have rank\n% exactly k. Reduce k if that is not the case.\n% \n% An alternative, complete, geometry for positive semidefinite matrices of\n% rank k is described in Bonnabel and Sepulchre 2009, \"Riemannian Metric\n% and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank\",\n% SIAM Journal on Matrix Analysis and Applications.\n%\n%\n% The geometry here implemented is the simplest case of the 2010 paper:\n% M. Journee, P.-A. Absil, F. Bach and R. Sepulchre,\n% \"Low-Rank Optimization on the Cone of Positive Semidefinite Matrices\".\n% Paper link: http://www.di.ens.fr/~fbach/journee2010_sdp.pdf\n% \n% \n% Please cite the Manopt paper as well as the research paper:\n% @Article{journee2010low,\n% Title = {Low-rank optimization on the cone of positive semidefinite matrices},\n% Author = {Journ{\\'e}e, M. and Bach, F. and Absil, P.-A. and Sepulchre, R.},\n% Journal = {SIAM Journal on Optimization},\n% Year = {2010},\n% Number = {5},\n% Pages = {2327--2351},\n% Volume = {20},\n% Doi = {10.1137/080731359}\n% }\n%\n% See also: elliptopefactory spectrahedronfactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors:\n% Change log:\n%\n% July 10, 2013 (NB):\n% Added vec, mat, tangent, tangent2ambient ;\n% Correction for the dimension of the manifold.\n%\n% April 2, 2015 (NB):\n% Replaced trace(A'*B) by A(:)'*B(:) (equivalent but faster).\n\n\n\tM.name = @() sprintf('YY'' quotient manifold of %dx%d psd matrices of rank %d', n, k);\n\n\tM.dim = @() k*n - k*(k-1)/2;\n\n\t% Euclidean metric on the total space\n\tM.inner = @(Y, eta, zeta) eta(:)'*zeta(:);\n\n\tM.norm = @(Y, eta) sqrt(M.inner(Y, eta, eta));\n\n\tM.dist = @(Y, Z) error('symfixedrankYYfactory.dist not implemented yet.');\n\n\tM.typicaldist = @() 10*k;\n\n\tM.proj = @projection;\n\tfunction etaproj = projection(Y, eta)\n\t\t% Projection onto the horizontal space\n\t\tYtY = Y'*Y;\n\t\tSS = YtY;\n\t\tAS = Y'*eta - eta'*Y;\n\t\tOmega = lyap(SS, -AS);\n\t\tetaproj = eta - Y*Omega;\n\tend\n\n\tM.tangent = M.proj;\n\tM.tangent2ambient = @(Y, eta) eta;\n\n\tM.retr = @retraction;\n\tfunction Ynew = retraction(Y, eta, t)\n\t\tif nargin < 3\n\t\t\tt = 1.0;\n\t\tend\n\t\tYnew = Y + t*eta;\n\tend\n\n\n\tM.egrad2rgrad = @(Y, eta) eta;\n\tM.ehess2rhess = @(Y, egrad, ehess, U) M.proj(Y, ehess);\n\n\tM.exp = @exponential;\n\tfunction Ynew = exponential(Y, eta, t)\n\t\tif nargin < 3\n\t\t\tt = 1.0;\n\t\tend\n\t\t\n\t\tYnew = retraction(Y, eta, t);\n\t\twarning('manopt:symfixedrankYYfactory:exp', ...\n\t\t\t['Exponential for symmetric, fixed-rank ' ...\n\t\t\t'manifold not implemented yet. Used retraction instead.']);\n\tend\n\n\t% Notice that the hash of two equivalent points will be different...\n\tM.hash = @(Y) ['z' hashmd5(Y(:))];\n\n\tM.rand = @random;\n\tfunction Y = random()\n\t\tY = randn(n, k);\n\tend\n\n\tM.randvec = @randomvec;\n\tfunction eta = randomvec(Y)\n\t\teta = randn(n, k);\n\t\teta = projection(Y, eta);\n\t\tnrm = M.norm(Y, eta);\n\t\teta = eta / nrm;\n\tend\n\n\tM.lincomb = @matrixlincomb;\n\n\tM.zerovec = @(Y) zeros(n, k);\n\n\tM.transp = @(Y1, Y2, d) projection(Y2, d);\n\t\t\n\tM.vec = @(Y, u_mat) u_mat(:);\n\tM.mat = @(Y, u_vec) reshape(u_vec, [n, k]);\n\tM.vecmatareisometries = @() true;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/manifolds/symfixedrank/sympositivedefinitefactory.m", "content": "function M = sympositivedefinitefactory(n)\n% Manifold of n-by-n symmetric positive definite matrices with\n% the bi-invariant geometry.\n%\n% function M = sympositivedefinitefactory(n)\n%\n% A point X on the manifold is represented as a symmetric positive definite\n% matrix X (nxn). Tangent vectors are symmetric matrices of the same size\n% (but not necessarily definite).\n%\n% The Riemannian metric is the bi-invariant metric, described notably in\n% Chapter 6 of the 2007 book \"Positive definite matrices\"\n% by Rajendra Bhatia, Princeton University Press.\n%\n%\n% The retraction / exponential map involves expm (the matrix exponential).\n% If too large a vector is retracted / exponentiated (e.g., a solver tries\n% to make too big a step), this may result in NaN's in the returned point,\n% which most likely would lead to NaN's in the cost / gradient / ... and\n% will result in failure of the optimization. For trustregions, this can be\n% controlled by setting options.Delta0 and options.Delta_bar, to prevent\n% too large steps.\n%\n%\n% Note also that many of the functions involve solving linear systems in X\n% (a point on the manifold), taking matrix exponentals and logarithms, etc.\n% It could therefore be beneficial to do some precomputation on X (an\n% eigenvalue decomposition for example) and store both X and the\n% preprocessing in a structure. This would require modifying the present\n% factory to work with such structures to represent both points and tangent\n% vectors. We omit this in favor of simplicity, but it may be good to keep\n% this in mind if efficiency becomes an issue in your application.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, August 29, 2013.\n% Contributors: Nicolas Boumal\n% Change log:\n%\n% March 5, 2014 (NB)\n% There were a number of mistakes in the code owing to the tacit\n% assumption that if X and eta are symmetric, then X\\eta is\n% symmetric too, which is not the case. See discussion on the Manopt\n% forum started on Jan. 19, 2014. Functions norm, dist, exp and log\n% were modified accordingly. Furthermore, they only require matrix\n% inversion (as well as matrix log or matrix exp), not matrix square\n% roots or their inverse.\n% \n% July 28, 2014 (NB)\n% The dim() function returned n*(n-1)/2 instead of n*(n+1)/2.\n% Implemented proper parallel transport from Sra and Hosseini (not\n% used by default).\n% Also added symmetrization in exp and log (to be sure).\n% \n% April 3, 2015 (NB):\n% Replaced trace(A*B) by a faster equivalent that does not compute\n% the whole product A*B, for inner product, norm and distance.\n%\n% May 23, 2017 (NB):\n% As seen in a talk of Wen Huang at the SIAM Optimization Conference\n% today, replaced the retraction of this factory (which was simply\n% equal to the exponential map) with a simpler, second-order\n% retraction. That this retraction is second order can be verified\n% numerically with checkretraction(sympositivedefinitefactory(5));\n% Notice that, for this retraction, it would be cheap to evaluate for\n% many values of t, that is, it is cheap to retract many points along\n% the same tangent direction. This could in principle be exploited to\n% speed up line-searches.\n \n symm = @(X) .5*(X+X');\n \n M.name = @() sprintf('Symmetric positive definite geometry of %dx%d matrices', n, n);\n \n M.dim = @() n*(n+1)/2;\n \n\t% Helpers to avoid computing full matrices simply to extract their trace\n\tvec = @(A) A(:);\n\ttrinner = @(A, B) vec(A')'*vec(B); % = trace(A*B)\n\ttrnorm = @(A) sqrt(trinner(A, A)); % = sqrt(trace(A^2))\n\t\n % Choice of the metric on the orthonormal space is motivated by the\n % symmetry present in the space. The metric on the positive definite\n % cone is its natural bi-invariant metric.\n\t% The result is equal to: trace( (X\\eta) * (X\\zeta) )\n M.inner = @(X, eta, zeta) trinner(X\\eta, X\\zeta);\n \n % Notice that X\\eta is *not* symmetric in general.\n\t% The result is equal to: sqrt(trace((X\\eta)^2))\n % There should be no need to take the real part, but rounding errors\n % may cause a small imaginary part to appear, so we discard it.\n M.norm = @(X, eta) real(trnorm(X\\eta));\n \n % Same here: X\\Y is not symmetric in general.\n % Same remark about taking the real part.\n M.dist = @(X, Y) real(trnorm(real(logm(X\\Y))));\n \n \n M.typicaldist = @() sqrt(n*(n+1)/2);\n \n \n M.egrad2rgrad = @egrad2rgrad;\n function eta = egrad2rgrad(X, eta)\n eta = X*symm(eta)*X;\n end\n \n \n M.ehess2rhess = @ehess2rhess;\n function Hess = ehess2rhess(X, egrad, ehess, eta)\n % Directional derivatives of the Riemannian gradient\n Hess = X*symm(ehess)*X + 2*symm(eta*symm(egrad)*X);\n \n % Correction factor for the non-constant metric\n Hess = Hess - symm(eta*symm(egrad)*X);\n end\n \n \n M.proj = @(X, eta) symm(eta);\n \n M.tangent = M.proj;\n M.tangent2ambient = @(X, eta) eta;\n \n M.retr = @retraction;\n function Y = retraction(X, eta, t)\n if nargin < 3\n teta = eta;\n else\n teta = t*eta;\n end\n % The symm() call is mathematically unnecessary but numerically\n % necessary.\n Y = symm(X + teta + .5*teta*(X\\teta));\n end\n \n M.exp = @exponential;\n function Y = exponential(X, eta, t)\n if nargin < 3\n t = 1.0;\n end\n % The symm() and real() calls are mathematically not necessary but\n % are numerically necessary.\n Y = symm(X*real(expm(X\\(t*eta))));\n end\n \n M.log = @logarithm;\n function H = logarithm(X, Y)\n % Same remark regarding the calls to symm() and real().\n H = symm(X*real(logm(X\\Y)));\n end\n \n M.hash = @(X) ['z' hashmd5(X(:))];\n \n % Generate a random symmetric positive definite matrix following a\n % certain distribution. The particular choice of a distribution is of\n % course arbitrary, and specific applications might require different\n % ones.\n M.rand = @random;\n function X = random()\n D = diag(1+rand(n, 1));\n [Q, R] = qr(randn(n)); %#ok\n X = Q*D*Q';\n end\n \n % Generate a uniformly random unit-norm tangent vector at X.\n M.randvec = @randomvec;\n function eta = randomvec(X)\n eta = symm(randn(n));\n nrm = M.norm(X, eta);\n eta = eta / nrm;\n end\n \n M.lincomb = @matrixlincomb;\n \n M.zerovec = @(X) zeros(n);\n \n % Poor man's vector transport: exploit the fact that all tangent spaces\n % are the set of symmetric matrices, so that the identity is a sort of\n % vector transport. It may perform poorly if the origin and target (X1\n % and X2) are far apart though. This should not be the case for typical\n % optimization algorithms, which perform small steps.\n M.transp = @(X1, X2, eta) eta;\n \n % For reference, a proper vector transport is given here, following\n % work by Sra and Hosseini: \"Conic geometric optimisation on the\n % manifold of positive definite matrices\", to appear in SIAM J. Optim.\n % in 2015; also available here: http://arxiv.org/abs/1312.1039\n % This will not be used by default. To force the use of this transport,\n % execute \"M.transp = M.paralleltransp;\" on your M returned by the\n % present factory.\n M.paralleltransp = @parallel_transport;\n function zeta = parallel_transport(X, Y, eta)\n E = sqrtm((Y/X));\n zeta = E*eta*E';\n end\n \n % vec and mat are not isometries, because of the unusual inner metric.\n M.vec = @(X, U) U(:);\n M.mat = @(X, u) reshape(u, n, n);\n M.vecmatareisometries = @() false;\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/readme", "content": "test\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/barzilaiborwein/barzilaiborwein.m", "content": "function [x, cost, info, options] = barzilaiborwein(problem, x, options)\n% Riemannian Barzilai-Borwein solver with non-monotone line-search.\n%\n% function [x, cost, info, options] = barzilaiborwein(problem)\n% function [x, cost, info, options] = barzilaiborwein(problem, x0)\n% function [x, cost, info, options] = barzilaiborwein(problem, x0, options)\n% function [x, cost, info, options] = barzilaiborwein(problem, [], options)\n%\n% Apply the Barzilai-Borwein minimization algorithm to the problem defined\n% in the problem structure, starting at x0 if it is provided (otherwise, at\n% a random point on the manifold). To specify options whilst not specifying\n% an initial guess, give x0 as [] (the empty matrix).\n%\n% The algorithm uses its own special non-monotone line-search strategy.\n% Therefore, no lin-search algorithm should be specified in the problem\n% structure or in the options structure.\n%\n% In most of the examples bundled with the toolbox (see link below), the\n% solver can be replaced by the present one if need be.\n%\n% The outputs x and cost are the last reached point on the manifold and its\n% cost. This is not necessarily the best point generated since the method\n% is not monotone. The struct-array info contains information about the\n% iterations:\n% iter : the iteration number (0 for the initial guess)\n% cost : cost value\n% time : elapsed time in seconds\n% gradnorm : Riemannian norm of the gradient\n% stepsize : norm of the last tangent vector retracted\n% linesearch : information logged by the line-search algorithm\n% And possibly additional information logged by options.statsfun.\n% For example, type [info.gradnorm] to obtain a vector of the successive\n% gradient norms reached.\n%\n% The options structure is used to overwrite the default values. All\n% options have a default value and are hence optional. To force an option\n% value, pass an options structure with a field options.optionname, where\n% optionname is one of the following and the default value is indicated\n% between parentheses:\n%\n% tolgradnorm (1e-6)\n% The algorithm terminates if the norm of the gradient drops below this.\n% maxiter (1000)\n% The algorithm terminates if maxiter iterations have been executed.\n% maxtime (Inf)\n% The algorithm terminates if maxtime seconds elapsed.\n% minstepsize (1e-10)\n% The algorithm terminates if the linesearch returns a displacement\n% vector (to be retracted) smaller in norm than this value.\n% linesearch (@linesearch_hint)\n% This option should not be changed, as the present solver has its\n% own dedicated line-search strategy.\n% strategy ('direct')\n% The strategy used for the Barzilai-Borwein stepsize\n% 'direct', compute the direct step /\n% 'inverse', compute the inverse step /\n% 'alternate', alternates between direct and inverse step\n% lambdamax (1e3)\n% The maximum stepsize allowed by the Barzilai-Borwein method\n% lambdamin (1e-3)\n% The minimum stepsize allowed by the Barzilai-Borwein method\n% lambda0 (1/10)\n% The initial stepsize of the Barzilai-Borwein method\n% ls_nmsteps (10)\n% The non-monotone line-search checks a sufficient decrease with respect\n% to the previous ls_nmsteps objective function values.\n% statsfun (none)\n% Function handle to a function that will be called after each\n% iteration to provide the opportunity to log additional statistics.\n% They will be returned in the info struct. See the generic Manopt\n% documentation about solvers for further information.\n% stopfun (none)\n% Function handle to a function that will be called at each iteration\n% to provide the opportunity to specify additional stopping criteria.\n% See the generic Manopt documentation about solvers for further\n% information.\n% verbosity (3)\n% Integer number used to tune the amount of output the algorithm\n% generates during execution (mostly as text in the command window).\n% The higher, the more output. 0 means silent.\n% storedepth (2)\n% Maximum number of different points x of the manifold for which a\n% store structure will be kept in memory in the storedb. If the\n% caching features of Manopt are not used, this is irrelevant. For\n% this algorithm, a store depth of 2 should always be sufficient.\n% \n%\n% The implementation of the Barzilai-Borwein method is based on the paper:\n%\n% B. Iannazzo, M. Porcelli, \"The Riemannian Barzilai-Borwein method with \n% nonmonotone line-search and the matrix geometric mean computation\",\n% IMA Journal of Numerical Analysis, to appear, https://doi.org/10.1093/imanum/drx015.\n%\n% See also: steepestdescent conjugategradient trustregions\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Margherita Porcelli, May 31, 2017\n% Contributors: Nicolas Boumal, Bruno Iannazzo\n% Change log: \n\n \n % Verify that the problem description is sufficient for the solver.\n if ~canGetCost(problem)\n warning('manopt:getCost', ...\n 'No cost provided. The algorithm will likely abort.'); \n end\n if ~canGetGradient(problem) && ~canGetApproxGradient(problem)\n % Note: we do not give a warning if an approximate gradient is\n % explicitly given in the problem description, as in that case the\n % user seems to be aware of the issue.\n warning('manopt:getGradient:approx', ...\n ['No gradient provided. Using an FD approximation instead (slow).\\n' ...\n 'It may be necessary to increase options.tolgradnorm.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:getGradient:approx'')']);\n problem.approxgrad = approxgradientFD(problem);\n end\n\n % Ensure options exists as a structure\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n \n % Set local defaults here\n localdefaults.minstepsize = 1e-10;\n localdefaults.maxiter = 1000;\n localdefaults.tolgradnorm = 1e-6;\n\n % Upper and lower bound for the Barzilai-Borwein stepsize\n localdefaults.lambdamax = 1e3;\n localdefaults.lambdamin = 1e-3;\n % Initial Barzilai-Borwein stepsize\n localdefaults.lambda0 = 1/10;\n\n % Barzilai-Borwein strategy (direct, inverse or alternate)\n localdefaults.strategy = 'direct';\n\n % Line-search parameters\n % 1) Make sure the user didn't try to define a line search\n if canGetLinesearch(problem) || isfield(options, 'linesearch')\n error('manopt:BB:ls', ...\n ['The problem structure may not specify a line-search ' ...\n 'hint for the BB solver,\\nand the options structure ' ...\n 'may not specify a line-search algorithm for BB.']);\n end\n % 2) Define the line-search parameters\n problem.linesearch = @(x, d, storedb, key) 1;\n options.linesearch = @linesearch_hint;\n % The Armijo sufficient decrease parameter\n localdefaults.ls_suff_decr = 1e-4;\n % The previous steps checked in the non-monotone line-search strategy\n localdefaults.ls_nmsteps = 10;\n \n \n % Merge global and local defaults, then merge w/ user options, if any.\n localdefaults = mergeOptions(getGlobalDefaults(), localdefaults);\n options = mergeOptions(localdefaults, options); \n\n \n % Shorthands for some parameters\n strategy = options.strategy;\n lambdamax = options.lambdamax;\n lambdamin = options.lambdamin;\n lambda0 = options.lambda0;\n \n timetic = tic();\n \n \n % If no initial point x is given by the user, generate one at random.\n if ~exist('x', 'var') || isempty(x)\n x = problem.M.rand();\n end\n\n % Create a store database and get a key for the current x\n storedb = StoreDB(options.storedepth);\n key = storedb.getNewKey();\n \n % Compute objective-related quantities for x\n [cost, grad] = getCostGrad(problem, x, storedb, key);\n gradnorm = problem.M.norm(x, grad);\n\n % Some variables below need to store information about iterations. We\n % preallocate for a reasonable amount of intended iterations to avoid\n % memory re-allocations.\n mem_init_size = min(10000, options.maxiter+1);\n \n % Store the cost value\n f = zeros(mem_init_size, 1);\n f(1) = cost;\n \n % Iteration counter (at any point, iter is the number of fully executed\n % iterations so far)\n iter = 0;\n \n % Save stats in a struct array info, and preallocate.\n stats = savestats();\n info(1) = stats;\n info(mem_init_size).iter = [];\n \n if options.verbosity >= 2\n fprintf(' iter\\t cost val\\t grad. norm\\n');\n end\n\n % Set the initial Barzilai-Borwein stepsize\n lambda = lambda0;\n\n % Start iterating until stopping criterion triggers\n while true\n\n % Display iteration information\n if options.verbosity >= 2\n fprintf('%5d\\t%+.16e\\t%.8e\\n', iter, cost, gradnorm);\n end\n \n % Start timing this iteration\n timetic = tic();\n \n % Run standard stopping criterion checks\n [stop, reason] = stoppingcriterion(problem, x, options, ...\n info, iter+1);\n \n % If none triggered, run specific stopping criterion check\n if ~stop && stats.stepsize < options.minstepsize\n stop = true;\n reason = sprintf(['Last stepsize smaller than minimum ' ...\n 'allowed; options.minstepsize = %g.'], ...\n options.minstepsize);\n end\n \n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n\n % Pick the descent direction as minus the gradient (scaled)\n desc_dir = problem.M.lincomb(x, -lambda, grad);\n\n % Execute the nonmonotone line search\n k = iter + 1; \n start = max(1, k - options.ls_nmsteps + 1);\n \n [stepsize, newx, newkey, lsstats] = ...\n options.linesearch(problem, x, desc_dir, max(f(start:k)), ...\n -lambda * gradnorm^2, options, storedb, key);\n\n % Updates the value of lambda\n lambda = lambda * lsstats.alpha;\n\n % Compute the new cost-related quantities for newx\n [newcost, newgrad] = getCostGrad(problem, newx, storedb, newkey);\n newgradnorm = problem.M.norm(newx, newgrad);\n\n %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n % BARZILAI-BORWEIN STRATEGY\n\n % Store the cost value\n f(iter+2) = newcost;\n \n % Transport the old gradient to newx\n grad_transp = problem.M.transp(x, newx, grad);\n\n % Compute the difference between grandients \n Y = problem.M.lincomb(newx, 1, newgrad, -1, grad_transp);\n\n % Compute the transported step\n Stransp = problem.M.lincomb(x, -lambda, grad_transp); \n\n % Compute the new Barzilai-Borwein step following the strategy\n % direct strategy\n if strcmp(strategy, 'direct')\n num = problem.M.norm(newx, Stransp)^2; \n den = problem.M.inner(newx, Stransp, Y);\n if den > 0\n lambda = min( lambdamax, max(lambdamin, num/den) );\n else\n lambda = lambdamax;\n end\n end\n\n % inverse strategy\n if strcmp(strategy, 'inverse')\n num = problem.M.inner(newx, Stransp, Y);\n den = problem.M.norm(newx, Y)^2;\n\n if num > 0 \n lambda = min( lambdamax, max(lambdamin, num/den) );\n else\n lambda = lambdamax;\n end\n end\n\n % alternate strategy\n if strcmp(strategy, 'alternate')\n num = problem.M.norm(newx, Stransp)^2; \n den = problem.M.inner(newx, Stransp, Y);\n den2 = problem.M.norm(newx, Y)^2;\n if (den > 0) \n if mod(iter,2)==0\n \tlambda = min( lambdamax, max(lambdamin, num/den) );\n\t else\n lambda = min( lambdamax, max(lambdamin, den/den2) );\n end\n else\n lambda = lambdamax;\n end\n end\n \n %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n % Make sure we don't use too much memory for the store database\n storedb.purge();\n \n % Update iterate info\n x = newx;\n key = newkey;\n cost = newcost;\n grad = newgrad;\n gradnorm = newgradnorm;\n\n % iter is the number of iterations we have accomplished.\n iter = iter + 1;\n \n % Log statistics for freshly executed iteration\n stats = savestats();\n info(iter+1) = stats;\n \n end\n \n info = info(1:iter+1);\n\n if options.verbosity >= 1\n fprintf('Total time is %f [s] (excludes statsfun)\\n', ...\n info(end).time);\n end\n \n \n \n % Routine in charge of collecting the current iteration stats\n function stats = savestats()\n stats.iter = iter;\n stats.cost = cost;\n stats.gradnorm = gradnorm;\n if iter == 0\n stats.stepsize = NaN;\n stats.time = toc(timetic);\n stats.linesearch = [];\n else\n stats.stepsize = stepsize;\n stats.time = info(iter).time + toc(timetic);\n stats.linesearch = lsstats;\n end\n stats = applyStatsfun(problem, x, storedb, key, options, stats);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/bfgs/rlbfgs.m", "content": "function [x, cost, info, options] = rlbfgs(problem, x0, options)\n% Riemannian limited memory BFGS solver for smooth objective functions.\n%\n% function [x, cost, info, options] = rlbfgs(problem)\n% function [x, cost, info, options] = rlbfgs(problem, x0)\n% function [x, cost, info, options] = rlbfgs(problem, x0, options)\n% function [x, cost, info, options] = rlbfgs(problem, [], options)\n%\n%\n% This is a Riemannian limited memory BFGS solver (quasi-Newton method), \n% which aims to minimize the cost function in the given problem structure.\n% It requires access to the gradient of the cost function.\n%\n% Parameter options.memory can be used to specify the number of iterations\n% the algorithm remembers and uses to approximate the inverse Hessian of\n% the cost. Default value is 30.\n% For unlimited memory, set options.memory = Inf.\n%\n%\n% For a description of the algorithm and theorems offering convergence\n% guarantees, see the references below.\n%\n% The initial iterate is x0 if it is provided. Otherwise, a random point on\n% the manifold is picked. To specify options whilst not specifying an\n% initial iterate, give x0 as [] (the empty matrix).\n%\n% The two outputs 'x' and 'cost' are the last reached point on the manifold\n% and its cost. \n% \n% The output 'info' is a struct-array which contains information about the\n% iterations:\n% iter (integer)\n% The iteration number. The initial guess is 0.\n%\tcost (double)\n% The corresponding cost value.\n%\tgradnorm (double)\n% The (Riemannian) norm of the gradient.\n%\ttime (double)\n% The total elapsed time in seconds to reach the corresponding cost.\n%\tstepsize (double)\n% The size of the step from the previous to the new iterate.\n% accepted (Boolean)\n% true if step is accepted in the cautious update. 0 otherwise.\n% And possibly additional information logged by options.statsfun.\n% For example, type [info.gradnorm] to obtain a vector of the successive\n% gradient norms reached at each iteration.\n%\n% The options structure is used to overwrite the default values. All\n% options have a default value and are hence optional. To force an option\n% value, pass an options structure with a field options.optionname, where\n% optionname is one of the following and the default value is indicated\n% between parentheses:\n%\n% tolgradnorm (1e-6)\n% The algorithm terminates if the norm of the gradient drops below\n% this. For well-scaled problems, a rule of thumb is that you can\n% expect to reduce the gradient norm by 8 orders of magnitude\n% (sqrt(eps)) compared to the gradient norm at a \"typical\" point (a\n% rough initial iterate for example). Further decrease is sometimes\n% possible, but inexact floating point arithmetic will eventually\n% limit the final accuracy. If tolgradnorm is set too low, the\n% algorithm may end up iterating forever (or at least until another\n% stopping criterion triggers).\n% maxiter (1000)\n% The algorithm terminates if maxiter iterations were executed.\n% maxtime (Inf)\n% The algorithm terminates if maxtime seconds elapsed.\n% minstepsize (1e-10)\n% The minimum norm of the tangent vector that points from the current\n% point to the next point. If the norm is less than minstepsize, the \n% program will terminate.\n% memory (30)\n% The number of previous iterations the program remembers. This is used \n% to approximate the inverse Hessian at the current point. Because of\n% difficulty of maintaining a representation of operators in terms of\n% coordinates, a recursive method is used. The number of steps in the\n% recursion is at most options.memory. This parameter can take any\n% integer value >= 0, or Inf, which is taken to be options.maxiter. If\n% options.maxiter has value Inf, then it will take value 10000 and a\n% warning will be displayed.\n% linesearch (@linesearch_hint)\n% Function handle to a line search function. The options structure is\n% passed to the line search too, so you can pass it parameters. See\n% each line search's documentation for info.\n% By default, the intial multiplier tried is alpha = 1. This can be\n% changed with options.linesearch: see help of linesearch_hint.\n% strict_inc_func (@(t) t)\n% The Cautious step needs a real function that has value 0 at t = 0,\n% and is strictly increasing. See details in Wen Huang's paper\n% \"A Riemannian BFGS Method without Differentiated Retraction for \n% Nonconvex Optimization Problems\"\n% statsfun (none)\n% Function handle to a function that will be called after each\n% iteration to provide the opportunity to log additional statistics.\n% They will be returned in the info struct. See the generic Manopt\n% documentation about solvers for further information. statsfun is\n% called with the point x that was reached last.\n% stopfun (none)\n% Function handle to a function that will be called at each iteration\n% to provide the opportunity to specify additional stopping criteria.\n% See the generic Manopt documentation about solvers for further\n% information.\n% verbosity (2)\n% Integer number used to tune the amount of output the algorithm\n% generates during execution (mostly as text in the command window).\n% The higher, the more output. 0 means silent. 3 and above includes a\n% display of the options structure at the beginning of the execution.\n% debug (false)\n% Set to true to allow the algorithm to perform additional\n% computations for debugging purposes. If a debugging test fails, you\n% will be informed of it, usually via the command window. Be aware\n% that these additional computations appear in the algorithm timings\n% too, and may interfere with operations such as counting the number\n% of cost evaluations, etc. (the debug calls get storedb too).\n% storedepth (30)\n% Maximum number of different points x of the manifold for which a\n% store structure will be kept in memory in the storedb. If the\n% caching features of Manopt are not used, this is irrelevant. If\n% memory usage is an issue, you may try to lower this number.\n% Profiling may then help to investigate if a performance hit was\n% incurred as a result.\n%\n%\n% Please cite the Manopt paper as well as the research paper:\n% @InBook{Huang2016,\n% title = {A {R}iemannian {BFGS} Method for Nonconvex Optimization Problems},\n% author = {Huang, W. and Absil, P.-A. and Gallivan, K.A.},\n% year = {2016},\n% publisher = {Springer International Publishing},\n% editor = {Karas{\\\"o}zen, B{\\\"u}lent and Manguo{\\u{g}}lu, Murat and Tezer-Sezgin, M{\\\"u}nevver and G{\\\"o}ktepe, Serdar and U{\\u{g}}ur, {\\\"O}m{\\\"u}r},\n% address = {Cham},\n% booktitle = {Numerical Mathematics and Advanced Applications ENUMATH 2015},\n% pages = {627--634},\n% doi = {10.1007/978-3-319-39929-4_60}\n% }\n%\n\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Changshuo Liu, July 19, 2017.\n% Contributors: Nicolas Boumal\n% Change log: \n\n\n % Verify that the problem description is sufficient for the solver.\n if ~canGetCost(problem)\n warning('manopt:getCost', ...\n 'No cost provided. The algorithm will likely abort.');\n end\n if ~canGetGradient(problem) && ~canGetApproxGradient(problem)\n % Note: we do not give a warning if an approximate gradient is\n % explicitly given in the problem description, as in that case the user\n % seems to be aware of the issue.\n warning('manopt:getGradient:approx', ...\n ['No gradient provided. Using an FD approximation instead (slow).\\n' ...\n 'It may be necessary to increase options.tolgradnorm.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:getGradient:approx'')']);\n problem.approxgrad = approxgradientFD(problem);\n end\n \n % Local defaults for the program\n localdefaults.minstepsize = 1e-10;\n localdefaults.maxiter = 1000;\n localdefaults.tolgradnorm = 1e-6;\n localdefaults.memory = 30;\n localdefaults.strict_inc_func = @(t) t;\n localdefaults.ls_max_steps = 25;\n localdefaults.storedepth = 30;\n localdefaults.linesearch = @linesearch_hint;\n \n % Merge global and local defaults, then merge w/ user options, if any.\n localdefaults = mergeOptions(getGlobalDefaults(), localdefaults);\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n % To make sure memory in range [0, Inf)\n options.memory = max(options.memory, 0);\n if options.memory == Inf\n if isinf(options.maxiter)\n options.memory = 10000;\n warning('rlbfgs:memory', ['options.memory and options.maxiter' ...\n ' are both Inf; options.memory has been changed to 10000.']);\n else\n options.memory = options.maxiter;\n end\n end\n \n M = problem.M;\n \n % Create a random starting point if no starting point is provided.\n if ~exist('x0', 'var')|| isempty(x0)\n xCur = M.rand(); \n else\n xCur = x0;\n end\n \n timetic = tic();\n \n % Create a store database and get a key for the current x\n storedb = StoreDB(options.storedepth);\n key = storedb.getNewKey();\n \n % __________Initialization of variables______________\n % Number of iterations since the last restart\n k = 0; \n % Total number of BFGS iterations\n iter = 0; \n \n % This cell stores step vectors which point from x_{t} to x_{t+1} for t\n % indexing the last iterations, capped at options.memory.\n % That is, it stores up to options.memory of the most recent step\n % vectors. However, the implementation below does not need step vectors \n % in their respective tangent spaces at x_{t}'s. Rather, it requires\n % them transported to the current point's tangent space by vector\n % tranport. For details regarding the requirements on the the vector\n % tranport, see the reference paper by Huang et al.\n % In this implementation, those step vectors are iteratively \n % transported to the current point's tangent space after every\n % iteration. Thus, at every iteration, vectors in sHistory are in the\n % current point's tangent space.\n sHistory = cell(1, options.memory);\n \n % This cell stores the differences for latest t's of the gradient at\n % x_{t+1} and the gradient at x_{t}, transported to x_{t+1}'s tangent\n % space. The memory is also capped at options.memory.\n yHistory = cell(1, options.memory);\n \n % rhoHistory{t} stores the reciprocal of the inner product between\n % sHistory{t} and yHistory{t}.\n rhoHistory = cell(1, options.memory);\n \n % Scaling of direction given by getDirection for acceptable step\n alpha = 1; \n \n % Scaling of initial matrix, Barzilai-Borwein.\n scaleFactor = 1;\n \n % Norm of the step\n stepsize = 1;\n \n % Stores whether the step is accepted by the cautious update check.\n accepted = true;\n \n % Query the cost function and its gradient\n [xCurCost, xCurGradient] = getCostGrad(problem, xCur, storedb, key);\n \n xCurGradNorm = M.norm(xCur, xCurGradient);\n \n % Line-search statistics for recording in info.\n lsstats = [];\n \n % Flag to control restarting scheme to avoid infinite loops (see below)\n ultimatum = false;\n \n % Save stats in a struct array info, and preallocate.\n stats = savestats();\n info(1) = stats;\n info(min(10000, options.maxiter+1)).iter = [];\n \n if options.verbosity >= 2\n fprintf(' iter cost val grad. norm alpha\\n');\n end\n \n % Main iteration\n while true\n\n % Display iteration information\n if options.verbosity >= 2\n fprintf('%5d %+.16e %.8e %.4e\\n', ...\n iter, xCurCost, xCurGradNorm, alpha);\n end\n \n % Start timing this iteration\n timetic = tic();\n \n % Run standard stopping criterion checks\n [stop, reason] = stoppingcriterion(problem, xCur, options, ...\n info, iter+1);\n \n % If none triggered, run specific stopping criterion check\n if ~stop \n if stats.stepsize < options.minstepsize\n % To avoid infinite loop and to push the search further\n % in case BFGS approximation of Hessian is off towards\n % the end, we erase the memory by setting k = 0;\n % In this way, it starts off like a steepest descent.\n % If even steepest descent does not work, then it is \n % hopeless and we will terminate.\n if ~ultimatum\n if options.verbosity >= 2\n fprintf(['stepsize is too small, restarting ' ...\n 'the bfgs procedure at the current point.\\n']);\n end\n k = 0;\n ultimatum = true;\n else\n stop = true;\n reason = sprintf(['Last stepsize smaller than ' ...\n 'minimum allowed; options.minstepsize = %g.'], ...\n options.minstepsize);\n end\n else\n % We are not in trouble: lift the ultimatum if it was on.\n ultimatum = false;\n end\n end \n \n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n\n \n % Compute BFGS direction\n p = getDirection(M, xCur, xCurGradient, sHistory,...\n yHistory, rhoHistory, scaleFactor, min(k, options.memory));\n\n % Execute line-search\n [stepsize, xNext, newkey, lsstats] = ...\n linesearch_hint(problem, xCur, p, xCurCost, ...\n M.inner(xCur, xCurGradient, p), ...\n options, storedb, key);\n \n % Record the BFGS step-multiplier alpha which as effectively\n % selected. Toward convergence, we hope to see alpha = 1.\n alpha = stepsize/M.norm(xCur, p);\n step = M.lincomb(xCur, alpha, p);\n \n \n % Query cost and gradient at the candidate new point.\n [xNextCost, xNextGrad] = getCostGrad(problem, xNext, storedb, newkey);\n \n % Compute sk and yk\n sk = M.transp(xCur, xNext, step);\n yk = M.lincomb(xNext, 1, xNextGrad, ...\n -1, M.transp(xCur, xNext, xCurGradient));\n\n % Computation of the BFGS step is invariant under scaling of sk and\n % yk by a common factor. For numerical reasons, we scale sk and yk\n % so that sk is a unit norm vector.\n norm_sk = M.norm(xNext, sk);\n sk = M.lincomb(xNext, 1/norm_sk, sk);\n yk = M.lincomb(xNext, 1/norm_sk, yk);\n \n inner_sk_yk = M.inner(xNext, sk, yk);\n inner_sk_sk = M.norm(xNext, sk)^2; % ensures nonnegativity\n \n \n % If the cautious step is accepted (which is the intended\n % behavior), we record sk, yk and rhok and need to do some\n % housekeeping. If the cautious step is rejected, these are not\n % recorded. In all cases, xNext is the next iterate: the notion of\n % accept/reject here is limited to whether or not we keep track of\n % sk, yk, rhok to update the BFGS operator.\n cap = options.strict_inc_func(xCurGradNorm);\n if inner_sk_sk ~= 0 && (inner_sk_yk / inner_sk_sk) >= cap\n \n accepted = true;\n \n rhok = 1/inner_sk_yk;\n \n scaleFactor = inner_sk_yk / M.norm(xNext, yk)^2;\n \n % Time to store the vectors sk, yk and the scalar rhok.\n % Remember: we need to transport all vectors to the most\n % current tangent space.\n \n % If we are out of memory\n if k >= options.memory\n \n % sk and yk are saved from 1 to the end with the most \n % current recorded to the rightmost hand side of the cells\n % that are occupied. When memory is full, do a shift so\n % that the rightmost is earliest and replace it with the\n % most recent sk, yk.\n for i = 2 : options.memory\n sHistory{i} = M.transp(xCur, xNext, sHistory{i});\n yHistory{i} = M.transp(xCur, xNext, yHistory{i});\n end\n if options.memory > 1\n sHistory = sHistory([2:end, 1]);\n yHistory = yHistory([2:end, 1]);\n rhoHistory = rhoHistory([2:end 1]);\n end\n if options.memory > 0\n sHistory{options.memory} = sk;\n yHistory{options.memory} = yk;\n rhoHistory{options.memory} = rhok;\n end\n \n % If we are not out of memory\n else\n \n for i = 1:k\n sHistory{i} = M.transp(xCur, xNext, sHistory{i});\n yHistory{i} = M.transp(xCur, xNext, yHistory{i});\n end\n sHistory{k+1} = sk;\n yHistory{k+1} = yk;\n rhoHistory{k+1} = rhok;\n \n end\n \n k = k + 1;\n \n % The cautious step is rejected: we do not store sk, yk, rhok but\n % we still need to transport stored vectors to the new tangent\n % space.\n else\n \n accepted = false;\n \n for i = 1 : min(k, options.memory)\n sHistory{i} = M.transp(xCur, xNext, sHistory{i});\n yHistory{i} = M.transp(xCur, xNext, yHistory{i});\n end\n \n end\n \n % Update variables to new iterate\n iter = iter + 1;\n xCur = xNext;\n key = newkey;\n xCurGradient = xNextGrad;\n xCurGradNorm = M.norm(xNext, xNextGrad);\n xCurCost = xNextCost;\n \n \n % Make sure we don't use too much memory for the store database\n % (this is independent from the BFGS memory.)\n storedb.purge();\n \n \n % Log statistics for freshly executed iteration\n stats = savestats();\n info(iter+1) = stats; \n \n end\n\n \n % Housekeeping before we return\n info = info(1:iter+1);\n x = xCur;\n cost = xCurCost;\n\n if options.verbosity >= 1\n fprintf('Total time is %f [s] (excludes statsfun)\\n', ...\n info(end).time);\n end\n\n \n % Routine in charge of collecting the current iteration stats\n function stats = savestats()\n stats.iter = iter;\n stats.cost = xCurCost;\n stats.gradnorm = xCurGradNorm;\n if iter == 0\n stats.stepsize = NaN;\n stats.time = toc(timetic);\n stats.accepted = NaN;\n else\n stats.stepsize = stepsize;\n stats.time = info(iter).time + toc(timetic);\n stats.accepted = accepted;\n end\n stats.linesearch = lsstats;\n stats = applyStatsfun(problem, xCur, storedb, key, options, stats);\n end\n\nend\n\n\n\n\n% BFGS step, see Wen's paper for details. This functon takes in a tangent\n% vector g, and applies an approximate inverse Hessian P to it to get Pg.\n% Then, -Pg is returned.\n%\n% Theory requires the vector transport to be isometric and to satisfy the\n% locking condition (see paper), but these properties do not seem to be\n% crucial in practice. If your manifold provides M.isotransp, it may be\n% good to do M.transp = M.isotransp; after loading M with a factory.\n%\n% This implementation operates in the tangent space of the most recent\n% point since all vectors in sHistory and yHistory have been transported\n% there.\nfunction dir = getDirection(M, xCur, xCurGradient, sHistory, yHistory, ...\n rhoHistory, scaleFactor, k)\n \n q = xCurGradient;\n \n inner_s_q = zeros(1, k);\n \n for i = k : -1 : 1\n inner_s_q(1, i) = rhoHistory{i} * M.inner(xCur, sHistory{i}, q);\n q = M.lincomb(xCur, 1, q, -inner_s_q(1, i), yHistory{i});\n end\n \n r = M.lincomb(xCur, scaleFactor, q);\n \n for i = 1 : k\n omega = rhoHistory{i} * M.inner(xCur, yHistory{i}, r);\n r = M.lincomb(xCur, 1, r, inner_s_q(1, i)-omega, sHistory{i});\n end\n \n dir = M.lincomb(xCur, -1, r);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/conjugategradient/conjugategradient.m", "content": "function [x, cost, info, options] = conjugategradient(problem, x, options)\n% Conjugate gradient minimization algorithm for Manopt.\n%\n% function [x, cost, info, options] = conjugategradient(problem)\n% function [x, cost, info, options] = conjugategradient(problem, x0)\n% function [x, cost, info, options] = conjugategradient(problem, x0, options)\n% function [x, cost, info, options] = conjugategradient(problem, [], options)\n%\n% Apply the conjugate gradient minimization algorithm to the problem\n% defined in the problem structure, starting at x0 if it is provided\n% (otherwise, at a random point on the manifold). To specify options whilst\n% not specifying an initial guess, give x0 as [] (the empty matrix).\n%\n% The outputs x and cost are the best reached point on the manifold and its\n% cost. The struct-array info contains information about the iterations:\n% iter : the iteration number (0 for the initial guess)\n% cost : cost value\n% time : elapsed time in seconds\n% gradnorm : Riemannian norm of the gradient\n% stepsize : norm of the last tangent vector retracted\n% beta : value of the beta parameter (see options.beta_type)\n% linesearch : information logged by options.linesearch\n% And possibly additional information logged by options.statsfun.\n% For example, type [info.gradnorm] to obtain a vector of the successive\n% gradient norms reached.\n%\n% The options structure is used to overwrite the default values. All\n% options have a default value and are hence optional. To force an option\n% value, pass an options structure with a field options.optionname, where\n% optionname is one of the following and the default value is indicated\n% between parentheses:\n%\n% tolgradnorm (1e-6)\n% The algorithm terminates if the norm of the gradient drops below this.\n% maxiter (1000)\n% The algorithm terminates if maxiter iterations have been executed.\n% maxtime (Inf)\n% The algorithm terminates if maxtime seconds elapsed.\n% minstepsize (1e-10)\n% The algorithm terminates if the linesearch returns a displacement\n% vector (to be retracted) smaller in norm than this value.\n% beta_type ('H-S')\n% Conjugate gradient beta rule used to construct the new search\n% direction, based on a linear combination of the previous search\n% direction and the new (preconditioned) gradient. Possible values\n% for this parameter are:\n% 'S-D', 'steep' for beta = 0 (preconditioned steepest descent)\n% 'F-R' for Fletcher-Reeves's rule\n% 'P-R' for Polak-Ribiere's modified rule\n% 'H-S' for Hestenes-Stiefel's modified rule\n% 'H-Z' for Hager-Zhang's modified rule\n% See Hager and Zhang 2006, \"A survey of nonlinear conjugate gradient\n% methods\" for a description of these rules in the Euclidean case and\n% for an explanation of how to adapt them to the preconditioned case.\n% The adaption to the Riemannian case is straightforward: see in code\n% for details. Modified rules take the max between 0 and the computed\n% beta value, which provides automatic restart, except for H-Z which\n% uses a different modification.\n% orth_value (Inf)\n% Following Powell's restart strategy (Math. prog. 1977), restart CG\n% (that is, make a -preconditioned- gradient step) if two successive\n% -preconditioned- gradients are \"too\" parallel. See for example\n% Hager and Zhang 2006, \"A survey of nonlinear conjugate gradient\n% methods\", page 12. An infinite value disables this strategy. See in\n% code formula for the specific criterion used.\n% linesearch (@linesearch_adaptive or @linesearch_hint)\n% Function handle to a line search function. The options structure is\n% passed to the line search too, so you can pass it parameters. See\n% each line search's documentation for info. Another available line\n% search in manopt is @linesearch, in /manopt/linesearch/linesearch.m\n% If the problem structure includes a line search hint, then the\n% default line search used is @linesearch_hint.\n% statsfun (none)\n% Function handle to a function that will be called after each\n% iteration to provide the opportunity to log additional statistics.\n% They will be returned in the info struct. See the generic Manopt\n% documentation about solvers for further information.\n% stopfun (none)\n% Function handle to a function that will be called at each iteration\n% to provide the opportunity to specify additional stopping criteria.\n% See the generic Manopt documentation about solvers for further\n% information.\n% verbosity (3)\n% Integer number used to tune the amount of output the algorithm\n% generates during execution (mostly as text in the command window).\n% The higher, the more output. 0 means silent.\n% storedepth (2)\n% Maximum number of different points x of the manifold for which a\n% store structure will be kept in memory in the storedb. If the\n% caching features of Manopt are not used, this is irrelevant. For\n% the CG algorithm, a store depth of 2 should always be sufficient.\n%\n%\n% In most of the examples bundled with the toolbox (see link below), the\n% solver can be replaced by the present one if need be.\n%\n% See also: steepestdescent trustregions manopt/solvers/linesearch manopt/examples\n\n% An explicit, general listing of this algorithm, with preconditioning,\n% can be found in the following paper:\n% @Article{boumal2015lowrank,\n% Title = {Low-rank matrix completion via preconditioned optimization on the {G}rassmann manifold},\n% Author = {Boumal, N. and Absil, P.-A.},\n% Journal = {Linear Algebra and its Applications},\n% Year = {2015},\n% Pages = {200--239},\n% Volume = {475},\n% Doi = {10.1016/j.laa.2015.02.027},\n% }\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors: Nicolas Boumal\n% Change log: \n%\n% March 14, 2013, NB:\n% Added preconditioner support : see Section 8 in\n% https://www.math.lsu.edu/~hozhang/papers/cgsurvey.pdf\n% \n% Sept. 13, 2013, NB:\n% Now logging beta parameter too.\n% \n%\tNov. 7, 2013, NB:\n% The search direction is no longer normalized before it is passed\n% to the linesearch. This way, it is up to the designers of the\n% linesearch to decide whether they want to use the norm of the\n% search direction in their algorithm or not. There are reasons\n% against it, but practical evidence that it may help too, so we\n% allow it. The default linesearch_adaptive used does exploit the\n% norm information. The base linesearch does not. You may select it\n% by setting options.linesearch = @linesearch;\n%\n%\tNov. 29, 2013, NB:\n% Documentation improved: options are now explicitly described.\n% Removed the Daniel rule for beta: it was not appropriate for\n% preconditioned CG and I could not find a proper reference for it.\n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n% Verify that the problem description is sufficient for the solver.\nif ~canGetCost(problem)\n warning('manopt:getCost', ...\n 'No cost provided. The algorithm will likely abort.');\nend\nif ~canGetGradient(problem) && ~canGetApproxGradient(problem)\n warning('manopt:getGradient:approx', ...\n ['No gradient provided. Using an FD approximation instead (slow).\\n' ...\n 'It may be necessary to increase options.tolgradnorm.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:getGradient:approx'')']);\n problem.approxgrad = approxgradientFD(problem);\nend\n\n% Set local defaults here\nlocaldefaults.minstepsize = 1e-10;\nlocaldefaults.maxiter = 1000;\nlocaldefaults.tolgradnorm = 1e-6;\nlocaldefaults.storedepth = 20;\n% Changed by NB : H-S has the \"auto restart\" property.\n% See Hager-Zhang 2005/2006 survey about CG methods.\n% The auto restart comes from the 'max(0, ...)', not so much from the\n% reason stated in Hager-Zhang I think. P-R also has auto restart.\nlocaldefaults.beta_type = 'H-S';\nlocaldefaults.orth_value = Inf; % by BM as suggested in Nocedal and Wright\n\n \n% Depending on whether the problem structure specifies a hint for\n% line-search algorithms, choose a default line-search that works on\n% its own (typical) or that uses the hint.\nif ~canGetLinesearch(problem)\n localdefaults.linesearch = @linesearch_adaptive;\nelse\n localdefaults.linesearch = @linesearch_hint;\nend\n\n% Merge global and local defaults, then merge w/ user options, if any.\nlocaldefaults = mergeOptions(getGlobalDefaults(), localdefaults);\nif ~exist('options', 'var') || isempty(options)\n options = struct();\nend\noptions = mergeOptions(localdefaults, options);\n\n% For convenience\ninner = problem.M.inner;\nlincomb = problem.M.lincomb;\n\ntimetic = tic();\n\n% If no initial point x is given by the user, generate one at random.\nif ~exist('x', 'var') || isempty(x)\n x = problem.M.rand();\nend\n\n% Create a store database and generate a key for the current x\nstoredb = StoreDB(options.storedepth);\nkey = storedb.getNewKey();\n\n% Compute cost-related quantities for x\n[cost, grad] = getCostGrad(problem, x, storedb, key);\ngradnorm = problem.M.norm(x, grad);\nPgrad = getPrecon(problem, x, grad, storedb, key);\ngradPgrad = inner(x, grad, Pgrad);\n\n% Iteration counter (at any point, iter is the number of fully executed\n% iterations so far)\niter = 0;\n\n% Save stats in a struct array info and preallocate.\nstats = savestats();\ninfo(1) = stats;\ninfo(min(10000, options.maxiter+1)).iter = [];\n\n\nif options.verbosity >= 2\n fprintf(' iter\\t cost val\\t grad. norm\\n');\nend\n\n% Compute a first descent direction (not normalized)\ndesc_dir = lincomb(x, -1, Pgrad);\n\n\n% Start iterating until stopping criterion triggers\nwhile true\n \n % Display iteration information\n if options.verbosity >= 2\n fprintf('%5d\\t%+.16e\\t%.8e\\n', iter, cost, gradnorm);\n end\n \n % Start timing this iteration\n timetic = tic();\n \n % Run standard stopping criterion checks\n [stop, reason] = stoppingcriterion(problem, x, options, info, iter+1);\n \n % Run specific stopping criterion check\n if ~stop && abs(stats.stepsize) < options.minstepsize\n stop = true;\n reason = sprintf(['Last stepsize smaller than minimum ' ...\n 'allowed; options.minstepsize = %g.'], ...\n options.minstepsize);\n end\n \n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n \n \n % The line search algorithms require the directional derivative of the\n % cost at the current point x along the search direction.\n df0 = inner(x, grad, desc_dir);\n \n % If we didn't get a descent direction: restart, i.e., switch to the\n % negative gradient. Equivalent to resetting the CG direction to a\n % steepest descent step, which discards the past information.\n if df0 >= 0\n \n % Or we switch to the negative gradient direction.\n if options.verbosity >= 3\n fprintf(['Conjugate gradient info: got an ascent direction '...\n '(df0 = %2e), reset to the (preconditioned) '...\n 'steepest descent direction.\\n'], df0);\n end\n % Reset to negative gradient: this discards the CG memory.\n desc_dir = lincomb(x, -1, Pgrad);\n df0 = -gradPgrad;\n \n end\n \n \n % Execute line search\n [stepsize, newx, newkey, lsstats] = options.linesearch( ...\n problem, x, desc_dir, cost, df0, options, storedb, key);\n \n \n % Compute the new cost-related quantities for newx\n [newcost, newgrad] = getCostGrad(problem, newx, storedb, newkey);\n newgradnorm = problem.M.norm(newx, newgrad);\n Pnewgrad = getPrecon(problem, newx, newgrad, storedb, newkey);\n newgradPnewgrad = inner(newx, newgrad, Pnewgrad);\n \n \n % Apply the CG scheme to compute the next search direction.\n %\n % This paper https://www.math.lsu.edu/~hozhang/papers/cgsurvey.pdf\n\t% by Hager and Zhang lists many known beta rules. The rules defined\n % here can be found in that paper (or are provided with additional\n % references), adapted to the Riemannian setting.\n\t% \n if strcmpi(options.beta_type, 'steep') || ...\n strcmpi(options.beta_type, 'S-D') % Gradient Descent\n \n beta = 0;\n desc_dir = lincomb(x, -1, Pnewgrad);\n \n else\n \n oldgrad = problem.M.transp(x, newx, grad);\n orth_grads = inner(newx, oldgrad, Pnewgrad) / newgradPnewgrad;\n \n % Powell's restart strategy (see page 12 of Hager and Zhang's\n % survey on conjugate gradient methods, for example)\n if abs(orth_grads) >= options.orth_value,\n beta = 0;\n desc_dir = lincomb(x, -1, Pnewgrad);\n \n else % Compute the CG modification\n \n desc_dir = problem.M.transp(x, newx, desc_dir);\n \n switch upper(options.beta_type)\n \n case 'F-R' % Fletcher-Reeves\n beta = newgradPnewgrad / gradPgrad;\n \n case 'P-R' % Polak-Ribiere+\n % vector grad(new) - transported grad(current)\n diff = lincomb(newx, 1, newgrad, -1, oldgrad);\n ip_diff = inner(newx, Pnewgrad, diff);\n beta = ip_diff / gradPgrad;\n beta = max(0, beta);\n \n case 'H-S' % Hestenes-Stiefel+\n diff = lincomb(newx, 1, newgrad, -1, oldgrad);\n ip_diff = inner(newx, Pnewgrad, diff);\n beta = ip_diff / inner(newx, diff, desc_dir);\n beta = max(0, beta);\n\n case 'H-Z' % Hager-Zhang+\n diff = lincomb(newx, 1, newgrad, -1, oldgrad);\n Poldgrad = problem.M.transp(x, newx, Pgrad);\n Pdiff = lincomb(newx, 1, Pnewgrad, -1, Poldgrad);\n deno = inner(newx, diff, desc_dir);\n numo = inner(newx, diff, Pnewgrad);\n numo = numo - 2*inner(newx, diff, Pdiff)*...\n inner(newx, desc_dir, newgrad) / deno;\n beta = numo / deno;\n\n % Robustness (see Hager-Zhang paper mentioned above)\n desc_dir_norm = problem.M.norm(newx, desc_dir);\n eta_HZ = -1 / ( desc_dir_norm * min(0.01, gradnorm) );\n beta = max(beta, eta_HZ);\n\n otherwise\n error(['Unknown options.beta_type. ' ...\n 'Should be steep, S-D, F-R, P-R, H-S or H-Z.']);\n end\n \n desc_dir = lincomb(newx, -1, Pnewgrad, beta, desc_dir);\n \n end\n \n end\n \n % Make sure we don't use too much memory for the store database\n storedb.purge();\n \n % Transfer iterate info\n x = newx;\n key = newkey;\n cost = newcost;\n grad = newgrad;\n Pgrad = Pnewgrad;\n gradnorm = newgradnorm;\n gradPgrad = newgradPnewgrad;\n \n % iter is the number of iterations we have accomplished.\n iter = iter + 1;\n \n % Log statistics for freshly executed iteration\n stats = savestats();\n info(iter+1) = stats; %#ok\n \nend\n\n\ninfo = info(1:iter+1);\n\nif options.verbosity >= 1\n fprintf('Total time is %f [s] (excludes statsfun)\\n', info(end).time);\nend\n\n\n% Routine in charge of collecting the current iteration stats\nfunction stats = savestats()\n stats.iter = iter;\n stats.cost = cost;\n stats.gradnorm = gradnorm;\n if iter == 0\n stats.stepsize = nan;\n stats.time = toc(timetic);\n stats.linesearch = [];\n stats.beta = 0;\n else\n stats.stepsize = stepsize;\n stats.time = info(iter).time + toc(timetic);\n stats.linesearch = lsstats;\n stats.beta = beta;\n end\n stats = applyStatsfun(problem, x, storedb, key, options, stats);\nend\n\nend\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/conjugategradient/linear_conjugategradient.m", "content": "function [x, cost, info, options] = linear_conjugategradient(problem, x, options)\n% Conjugate gradient minimization algorithm for Manopt.\n%\n% function [x, cost, info, options] = linear_conjugategradient(problem)\n% function [x, cost, info, options] = linear_conjugategradient(problem, x0)\n% function [x, cost, info, options] = linear_conjugategradient(problem, x0, options)\n% function [x, cost, info, options] = linear_conjugategradient(problem, [], options)\n%\n% Apply the conjugate gradient minimization algorithm to the problem\n% defined in the problem structure, starting at x0 if it is provided\n% (otherwise, at a random point on the manifold). To specify options whilst\n% not specifying an initial guess, give x0 as [] (the empty matrix).\n%\n% The outputs x and cost are the best reached point on the manifold and its\n% cost. The struct-array info contains information about the iterations:\n% iter : the iteration number (0 for the initial guess)\n% cost : cost value\n% time : elapsed time in seconds\n% gradnorm : Riemannian norm of the gradient\n% stepsize : norm of the last tangent vector retracted\n% beta : value of the beta parameter (see options.beta_type)\n% linesearch : information logged by options.linesearch\n% And possibly additional information logged by options.statsfun.\n% For example, type [info.gradnorm] to obtain a vector of the successive\n% gradient norms reached.\n%\n% The options structure is used to overwrite the default values. All\n% options have a default value and are hence optional. To force an option\n% value, pass an options structure with a field options.optionname, where\n% optionname is one of the following and the default value is indicated\n% between parentheses:\n%\n% tolgradnorm (1e-6)\n% The algorithm terminates if the norm of the gradient drops below this.\n% maxiter (1000)\n% The algorithm terminates if maxiter iterations have been executed.\n% maxtime (Inf)\n% The algorithm terminates if maxtime seconds elapsed.\n% minstepsize (1e-10)\n% The algorithm terminates if the linesearch returns a displacement\n% vector (to be retracted) smaller in norm than this value.\n% beta_type ('H-S')\n% Conjugate gradient beta rule used to construct the new search\n% direction, based on a linear combination of the previous search\n% direction and the new (preconditioned) gradient. Possible values\n% for this parameter are:\n% 'S-D', 'steep' for beta = 0 (preconditioned steepest descent)\n% 'F-R' for Fletcher-Reeves's rule\n% 'P-R' for Polak-Ribiere's modified rule\n% 'H-S' for Hestenes-Stiefel's modified rule\n% 'H-Z' for Hager-Zhang's modified rule\n% See Hager and Zhang 2006, \"A survey of nonlinear conjugate gradient\n% methods\" for a description of these rules in the Euclidean case and\n% for an explanation of how to adapt them to the preconditioned case.\n% The adaption to the Riemannian case is straightforward: see in code\n% for details. Modified rules take the max between 0 and the computed\n% beta value, which provides automatic restart, except for H-Z which\n% uses a different modification.\n% orth_value (Inf)\n% Following Powell's restart strategy (Math. prog. 1977), restart CG\n% (that is, make a -preconditioned- gradient step) if two successive\n% -preconditioned- gradients are \"too\" parallel. See for example\n% Hager and Zhang 2006, \"A survey of nonlinear conjugate gradient\n% methods\", page 12. An infinite value disables this strategy. See in\n% code formula for the specific criterion used.\n% linesearch (@linesearch_adaptive or @linesearch_hint)\n% Function handle to a line search function. The options structure is\n% passed to the line search too, so you can pass it parameters. See\n% each line search's documentation for info. Another available line\n% search in manopt is @linesearch, in /manopt/linesearch/linesearch.m\n% If the problem structure includes a line search hint, then the\n% default line search used is @linesearch_hint.\n% statsfun (none)\n% Function handle to a function that will be called after each\n% iteration to provide the opportunity to log additional statistics.\n% They will be returned in the info struct. See the generic Manopt\n% documentation about solvers for further information.\n% stopfun (none)\n% Function handle to a function that will be called at each iteration\n% to provide the opportunity to specify additional stopping criteria.\n% See the generic Manopt documentation about solvers for further\n% information.\n% verbosity (3)\n% Integer number used to tune the amount of output the algorithm\n% generates during execution (mostly as text in the command window).\n% The higher, the more output. 0 means silent.\n% storedepth (2)\n% Maximum number of different points x of the manifold for which a\n% store structure will be kept in memory in the storedb. If the\n% caching features of Manopt are not used, this is irrelevant. For\n% the CG algorithm, a store depth of 2 should always be sufficient.\n%\n%\n% In most of the examples bundled with the toolbox (see link below), the\n% solver can be replaced by the present one if need be.\n%\n% See also: steepestdescent trustregions manopt/solvers/linesearch manopt/examples\n\n% An explicit, general listing of this algorithm, with preconditioning,\n% can be found in the following paper:\n% @Article{boumal2015lowrank,\n% Title = {Low-rank matrix completion via preconditioned optimization on the {G}rassmann manifold},\n% Author = {Boumal, N. and Absil, P.-A.},\n% Journal = {Linear Algebra and its Applications},\n% Year = {2015},\n% Pages = {200--239},\n% Volume = {475},\n% Doi = {10.1016/j.laa.2015.02.027},\n% }\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors: Nicolas Boumal\n% Change log: \n%\n% March 14, 2013, NB:\n% Added preconditioner support : see Section 8 in\n% https://www.math.lsu.edu/~hozhang/papers/cgsurvey.pdf\n% \n% Sept. 13, 2013, NB:\n% Now logging beta parameter too.\n% \n%\tNov. 7, 2013, NB:\n% The search direction is no longer normalized before it is passed\n% to the linesearch. This way, it is up to the designers of the\n% linesearch to decide whether they want to use the norm of the\n% search direction in their algorithm or not. There are reasons\n% against it, but practical evidence that it may help too, so we\n% allow it. The default linesearch_adaptive used does exploit the\n% norm information. The base linesearch does not. You may select it\n% by setting options.linesearch = @linesearch;\n%\n%\tNov. 29, 2013, NB:\n% Documentation improved: options are now explicitly described.\n% Removed the Daniel rule for beta: it was not appropriate for\n% preconditioned CG and I could not find a proper reference for it.\n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n% Verify that the problem description is sufficient for the solver.\nif ~canGetCost(problem)\n warning('manopt:getCost', ...\n 'No cost provided. The algorithm will likely abort.');\nend\nif ~canGetGradient(problem) && ~canGetApproxGradient(problem)\n warning('manopt:getGradient:approx', ...\n ['No gradient provided. Using an FD approximation instead (slow).\\n' ...\n 'It may be necessary to increase options.tolgradnorm.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:getGradient:approx'')']);\n problem.approxgrad = approxgradientFD(problem);\nend\n\n% Set local defaults here\nlocaldefaults.minstepsize = 1e-10;\nlocaldefaults.maxiter = 1000;\nlocaldefaults.tolgradnorm = 1e-6;\nlocaldefaults.storedepth = 20;\n% Changed by NB : H-S has the \"auto restart\" property.\n% See Hager-Zhang 2005/2006 survey about CG methods.\n% The auto restart comes from the 'max(0, ...)', not so much from the\n% reason stated in Hager-Zhang I think. P-R also has auto restart.\nlocaldefaults.beta_type = 'H-S';\nlocaldefaults.orth_value = Inf; % by BM as suggested in Nocedal and Wright\n\n \n% Depending on whether the problem structure specifies a hint for\n% line-search algorithms, choose a default line-search that works on\n% its own (typical) or that uses the hint.\nif ~canGetLinesearch(problem)\n localdefaults.linesearch = @linesearch_adaptive;\nelse\n localdefaults.linesearch = @linesearch_hint;\nend\n\n% Merge global and local defaults, then merge w/ user options, if any.\nlocaldefaults = mergeOptions(getGlobalDefaults(), localdefaults);\nif ~exist('options', 'var') || isempty(options)\n options = struct();\nend\noptions = mergeOptions(localdefaults, options);\n\n% For convenience\ninner = problem.M.inner;\nlincomb = problem.M.lincomb;\n\ntimetic = tic();\n\n% If no initial point x is given by the user, generate one at random.\nif ~exist('x', 'var') || isempty(x)\n x = problem.M.rand();\nend\n\n% Create a store database and generate a key for the current x\nstoredb = StoreDB(options.storedepth);\nkey = storedb.getNewKey();\n\n% Compute cost-related quantities for x\n[cost, grad] = getCostGrad(problem, x, storedb, key);\ngradnorm = problem.M.norm(x, grad);\nPgrad = getPrecon(problem, x, grad, storedb, key);\ngradPgrad = inner(x, grad, Pgrad);\n\n% Iteration counter (at any point, iter is the number of fully executed\n% iterations so far)\niter = 0;\n\n% Save stats in a struct array info and preallocate.\nstats = savestats();\ninfo(1) = stats;\ninfo(min(10000, options.maxiter+1)).iter = [];\n\n\nif options.verbosity >= 2\n fprintf(' iter\\t cost val\\t grad. norm\\n');\nend\n\n% Compute a first descent direction (not normalized)\ndesc_dir = lincomb(x, -1, Pgrad);\n\n\n% Start iterating until stopping criterion triggers\nwhile true\n \n % Display iteration information\n if options.verbosity >= 2\n fprintf('%5d\\t%+.16e\\t%.8e\\n', iter, cost, gradnorm);\n end\n \n % Start timing this iteration\n timetic = tic();\n \n % Run standard stopping criterion checks\n [stop, reason] = stoppingcriterion(problem, x, options, info, iter+1);\n \n % Run specific stopping criterion check\n if ~stop && abs(stats.stepsize) < options.minstepsize\n stop = true;\n reason = sprintf(['Last stepsize smaller than minimum ' ...\n 'allowed; options.minstepsize = %g.'], ...\n options.minstepsize);\n end\n \n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n \n \n % The line search algorithms require the directional derivative of the\n % cost at the current point x along the search direction.\n df0 = inner(x, grad, desc_dir);\n \n % If we didn't get a descent direction: restart, i.e., switch to the\n % negative gradient. Equivalent to resetting the CG direction to a\n % steepest descent step, which discards the past information.\n if df0 >= 0\n \n % Or we switch to the negative gradient direction.\n if options.verbosity >= 3\n fprintf(['Conjugate gradient info: got an ascent direction '...\n '(df0 = %2e), reset to the (preconditioned) '...\n 'steepest descent direction.\\n'], df0);\n end\n % Reset to negative gradient: this discards the CG memory.\n desc_dir = lincomb(x, -1, Pgrad);\n df0 = -gradPgrad;\n \n end\n \n \n % Execute line search\n [stepsize, newx, newkey, lsstats] = options.linesearch( ...\n problem, x, desc_dir, cost, df0, options, storedb, key);\n \n \n % Compute the new cost-related quantities for newx\n [newcost, newgrad] = getCostGrad(problem, newx, storedb, newkey);\n newgradnorm = problem.M.norm(newx, newgrad);\n Pnewgrad = getPrecon(problem, newx, newgrad, storedb, newkey);\n newgradPnewgrad = inner(newx, newgrad, Pnewgrad);\n \n \n % Apply the CG scheme to compute the next search direction.\n %\n % This paper https://www.math.lsu.edu/~hozhang/papers/cgsurvey.pdf\n\t% by Hager and Zhang lists many known beta rules. The rules defined\n % here can be found in that paper (or are provided with additional\n % references), adapted to the Riemannian setting.\n\t% \n if strcmpi(options.beta_type, 'steep') || ...\n strcmpi(options.beta_type, 'S-D') % Gradient Descent\n \n beta = 0;\n desc_dir = lincomb(x, -1, Pnewgrad);\n \n else\n \n oldgrad = problem.M.transp(x, newx, grad);\n orth_grads = inner(newx, oldgrad, Pnewgrad) / newgradPnewgrad;\n \n % Powell's restart strategy (see page 12 of Hager and Zhang's\n % survey on conjugate gradient methods, for example)\n if abs(orth_grads) >= options.orth_value,\n beta = 0;\n desc_dir = lincomb(x, -1, Pnewgrad);\n \n else % Compute the CG modification\n \n desc_dir = problem.M.transp(x, newx, desc_dir);\n \n switch upper(options.beta_type)\n \n case 'F-R' % Fletcher-Reeves\n beta = newgradPnewgrad / gradPgrad;\n \n case 'P-R' % Polak-Ribiere+\n % vector grad(new) - transported grad(current)\n diff = lincomb(newx, 1, newgrad, -1, oldgrad);\n ip_diff = inner(newx, Pnewgrad, diff);\n beta = ip_diff / gradPgrad;\n beta = max(0, beta);\n \n case 'H-S' % Hestenes-Stiefel+\n diff = lincomb(newx, 1, newgrad, -1, oldgrad);\n ip_diff = inner(newx, Pnewgrad, diff);\n beta = ip_diff / inner(newx, diff, desc_dir);\n beta = max(0, beta);\n\n case 'H-Z' % Hager-Zhang+\n diff = lincomb(newx, 1, newgrad, -1, oldgrad);\n Poldgrad = problem.M.transp(x, newx, Pgrad);\n Pdiff = lincomb(newx, 1, Pnewgrad, -1, Poldgrad);\n deno = inner(newx, diff, desc_dir);\n numo = inner(newx, diff, Pnewgrad);\n numo = numo - 2*inner(newx, diff, Pdiff)*...\n inner(newx, desc_dir, newgrad) / deno;\n beta = numo / deno;\n\n % Robustness (see Hager-Zhang paper mentioned above)\n desc_dir_norm = problem.M.norm(newx, desc_dir);\n eta_HZ = -1 / ( desc_dir_norm * min(0.01, gradnorm) );\n beta = max(beta, eta_HZ);\n\n otherwise\n error(['Unknown options.beta_type. ' ...\n 'Should be steep, S-D, F-R, P-R, H-S or H-Z.']);\n end\n \n desc_dir = lincomb(newx, -1, Pnewgrad, beta, desc_dir);\n \n end\n \n end\n \n % Make sure we don't use too much memory for the store database\n storedb.purge();\n \n % Transfer iterate info\n x = newx;\n key = newkey;\n cost = newcost;\n grad = newgrad;\n Pgrad = Pnewgrad;\n gradnorm = newgradnorm;\n gradPgrad = newgradPnewgrad;\n \n % iter is the number of iterations we have accomplished.\n iter = iter + 1;\n \n % Log statistics for freshly executed iteration\n stats = savestats();\n info(iter+1) = stats; %#ok\n \nend\n\n\ninfo = info(1:iter+1);\n\nif options.verbosity >= 1\n fprintf('Total time is %f [s] (excludes statsfun)\\n', info(end).time);\nend\n\n\n% Routine in charge of collecting the current iteration stats\nfunction stats = savestats()\n stats.iter = iter;\n stats.cost = cost;\n stats.gradnorm = gradnorm;\n if iter == 0\n stats.stepsize = nan;\n stats.time = toc(timetic);\n stats.linesearch = [];\n stats.beta = 0;\n else\n stats.stepsize = stepsize;\n stats.time = info(iter).time + toc(timetic);\n stats.linesearch = lsstats;\n stats.beta = beta;\n end\n stats = applyStatsfun(problem, x, storedb, key, options, stats);\nend\n\nend\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/gradientapproximations/approxgradientFD.m", "content": "function gradfun = approxgradientFD(problem, options)\n% Gradient approx. fnctn handle based on finite differences of the cost.\n%\n% function gradfun = approxgradientFD(problem)\n% function gradfun = approxgradientFD(problem, options)\n%\n% Input:\n%\n% A Manopt problem structure (already containing the manifold and enough\n% information to compute the cost) and an options structure (optional),\n% containing one option:\n% options.stepsize (positive double; default: 2^-23).\n% options.subspacedim (positive integer; default: [], for M.dim()).\n%\n% If the cost cannot be computed on 'problem', a warning is issued.\n%\n% Output:\n% \n% Returns a function handle, encapsulating a generic finite difference\n% approximation of the gradient of the problem cost. The finite difference\n% is based on M.dim()+1 computations of the cost.\n% \n% The returned gradfun has this calling pattern:\n% \n% function gradfd = gradfun(x)\n% function gradfd = gradfun(x, storedb)\n% function gradfd = gradfun(x, storedb, key)\n% \n% x is a point on the manifold problem.M, storedb is a StoreDB object,\n% and key is the StoreDB key to point x.\n%\n% Usage:\n%\n% Typically, the user will set problem.M and other fields to define the\n% cost (typically, problem.cost). Then, to use this generic purpose\n% gradient approximation:\n%\n% problem.approxgrad = approxgradientFD(problem, options);\n%\n% See also: steepestdescent conjugategradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Nov. 1, 2016.\n% Contributors: \n% Change log: \n\n % This gradient approximation is based on the cost:\n % check availability.\n if ~canGetCost(problem)\n warning('manopt:approxgradFD:nocost', ...\n 'approxgradFD requires the cost to be computable.');\n end\n\n % Set local defaults here, and merge with user options, if any.\n localdefaults.stepsize = 2^-23;\n localdefaults.subspacedim = [];\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n % % Finite-difference parameters\n % How far do we look?\n stepsize = options.stepsize;\n % Approximate the projection of the gradient on a random subspace of\n % what dimension? If [], uses full tangent space.\n subspacedim = options.subspacedim;\n \n % Build and return the function handle here. This extra construct via\n % funhandle makes it possible to make storedb and key optional.\n gradfun = @funhandle;\n function gradfd = funhandle(x, storedb, key)\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n gradfd = gradientFD(stepsize, subspacedim, problem, x, storedb, key);\n end\n \nend\n\n\nfunction gradfd = gradientFD(stepsize, subspacedim, problem, x, storedb, key)\n% This function does the actual work.\n%\n% Original code: Nov. 1, 2016 (NB).\n\t\n % Evaluate the cost at the root point\n fx = getCost(problem, x, storedb, key);\n\n % Pick an orthonormal basis for the tangent space at x, or a subspace\n % thereof. The default is a full subspace. If a strict subspace is\n % picked, the returned vector approximates the orthogonal projection of\n % the gradient to that subspace.\n B = tangentorthobasis(problem.M, x, subspacedim);\n \n % Use finite differences to approximate the directional derivative\n % along each direction in the basis B.\n df = zeros(size(B));\n for k = 1 : numel(B)\n % Move in the B{k} direction\n xk = problem.M.retr(x, B{k}, stepsize);\n % Evaluate the cost there\n fxk = getCost(problem, xk, storedb);\n % Finite difference\n df(k) = (fxk - fx)/stepsize;\n end\n \n % Build the gradient approximation.\n gradfd = lincomb(problem.M, x, B, df);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/hessianapproximations/approxhessianFD.m", "content": "function hessfun = approxhessianFD(problem, options)\n% Hessian approx. fnctn handle based on finite differences of the gradient.\n%\n% function hessfun = approxhessianFD(problem)\n% function hessfun = approxhessianFD(problem, options)\n%\n% Input:\n%\n% A Manopt problem structure (already containing the manifold and enough\n% information to compute the cost gradient) and an options structure\n% (optional), containing one option:\n% options.stepsize (positive double; default: 2^-14).\n%\n% If the gradient cannot be computed or approximated on 'problem',\n% a warning is issued.\n%\n% Output:\n% \n% Returns a function handle, encapsulating a generic finite difference\n% approximation of the Hessian of the problem cost. The finite difference\n% is based on computations of the gradient.\n% \n% The returned hessfun has this calling pattern:\n% \n% function hessfd = hessfun(x, xdot)\n% function hessfd = hessfun(x, xdot, storedb)\n% function hessfd = hessfun(x, xdot, storedb, key)\n% \n% x is a point on the manifold problem.M, xdot is a tangent vector to that\n% manifold at x, storedb is a StoreDB object, and key is the StoreDB key to\n% point x.\n%\n% Usage:\n%\n% Typically, the user will set problem.M and other fields to define the\n% cost and the gradient (typically, problem.cost and problem.grad or\n% problem.egrad). Then, to use this generic purpose Hessian approximation:\n%\n% problem.approxhess = approxhessianFD(problem, options);\n%\n% See also: trustregions\n\n% The Riemannian Trust-Region method, used in combination with the present\n% Hessian approximation, is called RTR-FD. Some convergence theory for it\n% is available in this paper:\n%\n% @incollection{boumal2015rtrfd\n% \tauthor={Boumal, N.},\n% \ttitle={Riemannian trust regions with finite-difference Hessian approximations are globally convergent},\n% \tyear={2015},\n% \tbooktitle={Geometric Science of Information}\n% }\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 8, 2015.\n% Contributors: \n% Change log: \n%\n% Feb. 19, 2015 (NB):\n% It is sufficient to ensure positive radial linearity to guarantee\n% (together with other assumptions) that this approximation of the\n% Hessian will confer global convergence to the trust-regions method.\n% Formerly, in-code comments referred to the necessity of having\n% complete radial linearity, and that this was harder to achieve.\n% This appears not to be necessary after all, which simplifies the\n% code.\n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% April 8, 2015 (NB):\n% Changed to approxhessianFD, which now returns a function handle\n% that encapsulates the getHessianFD functionality. Will be better\n% aligned with the other Hessian approximations to come (which may\n% want to use storedb.internal), and now allows specifying the step\n% size.\n\n % This Hessian approximation is based on the gradient:\n % check availability.\n if ~canGetGradient(problem) && ~canGetApproxGradient(problem)\n warning('manopt:approxhessianFD:nogradient', ...\n 'approxhessianFD requires the gradient to be computable.');\n end\n\n % Set local defaults here, and merge with user options, if any.\n localdefaults.stepsize = 2^-14;\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n % Finite-difference parameter: how far do we look?\n stepsize = options.stepsize;\n \n % Build and return the function handle here. This extra construct via\n % funhandle makes it possible to make storedb and key optional.\n hessfun = @funhandle;\n function hessfd = funhandle(x, xdot, storedb, key)\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end \n hessfd = hessianFD(stepsize, problem, x, xdot, storedb, key);\n end\n \nend\n\n\nfunction hessfd = hessianFD(stepsize, problem, x, xdot, storedb, key)\n% This function does the actual work.\n%\n% Original code: Dec. 30, 2012 (NB).\n\t\n\t% Extract the input vector norm.\n norm_xdot = problem.M.norm(x, xdot);\n \n % First, check whether the step xdot is not too small.\n if norm_xdot < eps\n hessfd = problem.M.zerovec(x);\n return;\n end\n \n % Determine how far to retract xdot, so that the point reached does not\n % depend on the norm of xdot. This is what ensures radial linearity of\n % this present Hessian approximation.\n c = stepsize / norm_xdot;\n \n % Compute the gradient at the current point.\n grad = getGradient(problem, x, storedb, key);\n \n % Compute a point a little further along xdot, and the gradient there.\n % Since this is a new point, we need a new key for it, for storedb.\n x1 = problem.M.retr(x, xdot, c);\n key1 = storedb.getNewKey();\n grad1 = getGradient(problem, x1, storedb, key1);\n \n % Transport grad1 back from x1 to x.\n grad1 = problem.M.transp(x1, x, grad1);\n \n % Return the finite difference of them: (grad1 - grad)/c.\n hessfd = problem.M.lincomb(x, 1/c, grad1, -1/c, grad);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/linesearch/linesearch.m", "content": "function [stepsize, newx, newkey, lsstats] = ...\n linesearch(problem, x, d, f0, df0, options, storedb, key)\n% Standard line-search algorithm (step size selection) for descent methods.\n%\n% function [stepsize, newx, newkey, lsstats] = \n% linesearch(problem, x, d, f0, df0, options, storedb, key)\n%\n% Base line-search algorithm for descent methods, based on a simple\n% backtracking method. The search direction provided has to be a descent\n% direction, as indicated by a negative df0 = directional derivative of f\n% at x along d.\n%\n% The algorithm is invariant under positive scaling of the cost function\n% and under offsetting, that is: if the cost function f is replaced by\n% 8*f+3 for example, the returned step size will be the same. Furthermore,\n% the returned step size is independent of the norm of the search direction\n% vector d: only the direction of d is important.\n% \n% Below, the step is constructed as alpha*d, and the step size is the norm\n% of that vector, thus: stepsize = alpha*norm_d. The step is executed by\n% retracting the vector alpha*d from the current point x, giving newx.\n%\n% This line-search may create and maintain a structure called lsmem inside\n% storedb.internal. This gives the linesearch the opportunity to remember\n% what happened in the previous calls. This is typically used to make a\n% first guess at the step size, based on previous events.\n%\n% Inputs\n%\n% problem : structure holding the description of the optimization problem\n% x : current point on the manifold problem.M\n% d : tangent vector at x (descent direction) -- its norm is irrelevant\n% f0 : cost value at x\n% df0 : directional derivative at x along d\n% options : options structure (see in code for usage)\n% storedb : StoreDB object (handle class: passed by reference) for caching\n% key : key associated to point x in storedb\n%\n% options, storedb and key are optional.\n%\n% Outputs\n%\n% stepsize : norm of the vector retracted to reach newx from x.\n% newx : next iterate suggested by the line-search algorithm, such that\n% the retraction at x of the vector alpha*d reaches newx.\n% newkey : key associated to newx in storedb\n% lsstats : statistics about the line-search procedure\n% (stepsize, number of trials etc).\n%\n% See also: steepestdescent conjugategradients linesearch_adaptive\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n%\tSept. 13, 2013 (NB):\n% User control over the parameters of the linesearch via the options\n% ls_contraction_factor, ls_optimism, ls_suff_decr and ls_max_steps.\n% See in code for the effect of those.\n% \n% Sept. 13, 2013 (NB):\n% The automatic direction reversal feature was removed (it triggered\n% when df0 > 0). Direction reversal is a decision that needs to be\n% made by the solver, so it can know about it.\n% \n%\tSept. 13, 2013 (NB):\n% The linesearch is now invariant under rescaling of the cost\n% function f. That is, if f is replaced by 8*f (and hence the\n% directional derivatives of f are scaled accordingly), the\n% stepsizes computed will not change.\n% \n% Nov. 7, 2013 (NB):\n% The linesearch is now invariant under rescaling of the search\n% direction d. The meaning of stepsize is also more clear in the\n% comments. Added a parameter ls_initial_stepsize to give users\n% control over the first step size trial.\n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% April 8, 2015 (NB):\n% Got rid of lsmem input/output: now maintained in storedb.internal.\n%\n% Oct. 7, 2016 (NB):\n% Thanks to Wen Huang, a bug was corrected in the logic around\n% lsmem handling. Specifically, lsmem = storedb.internal.lsmem;\n% was erroneously coded as lsmem = storedb.internal;\n\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n % Backtracking default parameters. These can be overwritten in the\n % options structure which is passed to the solver.\n default_options.ls_contraction_factor = .5;\n default_options.ls_optimism = 1/.5;\n default_options.ls_suff_decr = 1e-4;\n default_options.ls_max_steps = 25;\n default_options.ls_initial_stepsize = 1;\n \n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(default_options, options);\n \n contraction_factor = options.ls_contraction_factor;\n optimism = options.ls_optimism;\n suff_decr = options.ls_suff_decr;\n max_ls_steps = options.ls_max_steps;\n initial_stepsize = options.ls_initial_stepsize;\n \n % Compute the norm of the search direction.\n % This is useful to make the linesearch algorithm invariant under the\n % scaling of d. The rationale is that the important information is the\n % search direction, not the size of that vector. The question of how\n % far we should go is precisely what the linesearch algorithm is\n % supposed to answer: the calling algorithm should not need to care.\n norm_d = problem.M.norm(x, d);\n \n % At first, we have no idea of what the step size should be.\n alpha = NaN;\n \n % If we know about what happened at the previous step, we can leverage\n % that to compute an initial guess for the step size, as inspired from\n % Nocedal&Wright, p59.\n if isfield(storedb.internal, 'lsmem')\n lsmem = storedb.internal.lsmem;\n if isfield(lsmem, 'f0')\n % Pick initial step size based on where we were last time,\n alpha = 2*(f0 - lsmem.f0) / df0;\n % and go look a little further (or less far), just in case.\n alpha = optimism*alpha;\n end\n end\n \n % If we have no information about the previous iteration (maybe this is\n % the first one?) or if the above formula gave a too small step size\n % (perhaps it is even negative), then fall back to a user supplied\n % suggestion for the first step size (the \"a priori\").\n % At any rate, the choice should be invariant under rescaling of the\n % cost function f and of the search direction d, and it should be\n % bounded away from zero for convergence guarantees. We must allow it\n % to be close to zero though, for fine convergence.\n if isnan(alpha) || alpha*norm_d <= eps\n alpha = initial_stepsize/norm_d;\n end\n \n\n % Make the chosen step and compute the cost there.\n newx = problem.M.retr(x, d, alpha);\n newkey = storedb.getNewKey();\n newf = getCost(problem, newx, storedb, newkey);\n cost_evaluations = 1;\n \n % Backtrack while the Armijo criterion is not satisfied\n while newf > f0 + suff_decr*alpha*df0\n \n % Reduce the step size,\n alpha = contraction_factor * alpha;\n \n % and look closer down the line\n newx = problem.M.retr(x, d, alpha);\n newkey = storedb.getNewKey();\n newf = getCost(problem, newx, storedb, newkey);\n cost_evaluations = cost_evaluations + 1;\n \n % Make sure we don't run out of budget\n if cost_evaluations >= max_ls_steps\n break;\n end\n \n end\n \n % If we got here without obtaining a decrease, we reject the step.\n if newf > f0\n alpha = 0;\n newx = x;\n newkey = key;\n newf = f0; %#ok\n end\n \n % As seen outside this function, stepsize is the size of the vector we\n % retract to make the step from x to newx. Since the step is alpha*d:\n stepsize = alpha * norm_d;\n\n % Save the situtation faced now so that, at the next iteration,\n % we will know something about the previous decision.\n storedb.internal.lsmem.f0 = f0;\n storedb.internal.lsmem.df0 = df0;\n storedb.internal.lsmem.stepsize = stepsize;\n \n % Return some statistics also, for possible analysis.\n lsstats.costevals = cost_evaluations;\n lsstats.stepsize = stepsize;\n lsstats.alpha = alpha;\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/linesearch/linesearch_adaptive.m", "content": "function [stepsize, newx, newkey, lsstats] = ...\n linesearch_adaptive(problem, x, d, f0, df0, options, storedb, key)\n% Adaptive line search algorithm (step size selection) for descent methods.\n%\n% function [stepsize, newx, newkey, lsstats] = \n% linesearch_adaptive(problem, x, d, f0, df0, options, storedb, key)\n%\n% Adaptive linesearch algorithm for descent methods, based on a simple\n% backtracking method. Contrary to linesearch.m, this function is not\n% invariant under rescaling of the search direction d. These two line\n% search methods vary mainly in their strategy to pick the initial step\n% size.\n% \n% Below, the step is constructed as alpha*d, and the step size is the norm\n% of that vector, thus: stepsize = alpha*norm_d. The step is executed by\n% retracting the vector alpha*d from the current point x, giving newx.\n%\n% This line-search may create and maintain a structure called lsmem inside\n% storedb.internal. This gives the linesearch the opportunity to remember\n% what happened in the previous calls. This is typically used to make a\n% first guess at the step size, based on previous events.\n%\n% Inputs/Outputs : see help for linesearch\n%\n% See also: steepestdescent conjugategradients linesearch\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Bamdev Mishra, Dec. 30, 2012.\n% Contributors: Nicolas Boumal\n% Change log:\n%\n% Sept. 13, 2013 (NB) :\n% The automatic direction reversal feature was removed (it triggered\n% when df0 > 0). Direction reversal is a decision that needs to be\n% made by the solver, so it can know about it.\n%\n%\tNov. 7, 2013 (NB) :\n% The whole function has been recoded to mimick more closely the new\n% version of linesearch.m. The parameters are available through the\n% options structure passed to the solver and have the same names and\n% same meaning as for the base linesearch. The information is logged\n% more reliably.\n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% April 8, 2015 (NB):\n% Got rid of lsmem input/output: now maintained in storedb.internal.\n\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n % Backtracking default parameters. These can be overwritten in the\n % options structure which is passed to the solver.\n default_options.ls_contraction_factor = .5;\n default_options.ls_suff_decr = .5;\n default_options.ls_max_steps = 10;\n default_options.ls_initial_stepsize = 1;\n \n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(default_options, options);\n \n contraction_factor = options.ls_contraction_factor;\n suff_decr = options.ls_suff_decr;\n max_ls_steps = options.ls_max_steps;\n initial_stepsize = options.ls_initial_stepsize;\n \n % Compute the norm of the search direction.\n norm_d = problem.M.norm(x, d);\n \n % If this is not the first iteration, then lsmem should have been\n % filled with a suggestion for the initial step.\n if isfield(storedb.internal, 'lsmem')\n lsmem = storedb.internal.lsmem;\n if isfield(lsmem, 'init_alpha')\n % Pick initial step size based on where we were last time,\n alpha = lsmem.init_alpha;\n end\n % Otherwise, fall back to a user supplied suggestion.\n else\n alpha = initial_stepsize / norm_d;\n end\n\n % Make the chosen step and compute the cost there.\n newx = problem.M.retr(x, d, alpha);\n newkey = storedb.getNewKey();\n newf = getCost(problem, newx, storedb, newkey);\n cost_evaluations = 1;\n \n % Backtrack while the Armijo criterion is not satisfied\n while newf > f0 + suff_decr*alpha*df0\n \n % Reduce the step size,\n alpha = contraction_factor * alpha;\n \n % and look closer down the line\n newx = problem.M.retr(x, d, alpha);\n newkey = storedb.getNewKey();\n newf = getCost(problem, newx, storedb, newkey);\n cost_evaluations = cost_evaluations + 1;\n \n % Make sure we don't run out of budget\n if cost_evaluations >= max_ls_steps\n break;\n end\n \n end\n \n % If we got here without obtaining a decrease, we reject the step.\n if newf > f0\n alpha = 0;\n newx = x;\n newkey = key;\n newf = f0; %#ok\n end\n \n % As seen outside this function, stepsize is the size of the vector we\n % retract to make the step from x to newx. Since the step is alpha*d:\n stepsize = alpha * norm_d;\n\n % Fill lsmem with a suggestion for what the next initial step size\n % trial should be. On average we intend to do only one extra cost\n % evaluation. Notice how the suggestion is not about stepsize but about\n % alpha. This is the reason why this line search is not invariant under\n % rescaling of the search direction d.\n switch cost_evaluations\n case 1\n % If things go very well, push your luck.\n init_alpha = 2 * alpha;\n case 2\n % If things go reasonably well, try to keep pace.\n init_alpha = alpha;\n otherwise\n % If we backtracked a lot, the new stepsize is probably quite\n % small: try to recover.\n init_alpha = 2 * alpha;\n end\n storedb.internal.lsmem.init_alpha = init_alpha;\n \n % Return some statistics also, for possible analysis.\n lsstats.costevals = cost_evaluations;\n lsstats.stepsize = stepsize;\n lsstats.alpha = alpha;\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/linesearch/linesearch_decrease.m", "content": "function [stepsize, newx, newkey, lsstats] = ...\n linesearch_decrease(problem, x, d, f0, ~, options, storedb, key)\n% Backtracking line-search aiming merely for a decrease in cost value.\n%\n% function [stepsize, newx, newkey, lsstats] = \n% linesearch_decrease(problem, x, d, f0, df0, options, storedb, key)\n%\n% Line-search algorithm based on a simple backtracking method. The search\n% direction provided has to be a descent direction, but needs not be a\n% first-order descent, i.e.: this line-search can be used even if x is a\n% critical point, as long as the cost function is strictly decreasing\n% along the direction d.\n%\n% The line-search merely guarantees a decrease in the cost (unless a\n% stopping criterion triggers first, such as exceeding a maximal number of\n% iterations). This is typically useful to escape saddle points (critical\n% points admitting descent directions at the second order). Escape\n% directions can be computed using hessianextreme, for example.\n% \n% Below, the step is constructed as alpha*d, and the step size is the norm\n% of that vector, thus: stepsize = alpha*norm_d. The step is executed by\n% retracting the vector alpha*d from the current point x, giving newx.\n% An initial stepsize of norm_d thus means the first candidate x is\n% obtained by retracting d at x, as is.\n%\n% Options:\n% options.ls_max_steps (25): maximum number of cost evaluations.\n% options.ls_initial_stepsize (norm_d): first stepsize trial.\n% options.ls_contraction_factor (0.5): stepsize reduction per iteration.\n%\n%\n% Inputs/Outputs : see help for linesearch.\n% f0 is the cost at x.\n% df0 is unused.\n% options, storedb and key are optional.\n% Thus, a simplified calling pattern is (with all outputs still\n% available): linesearch_decrease(problem, x, d, f0)\n%\n% See also: steepestdescent linesearch hessianextreme\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 8, 2015.\n% Contributors: \n% Change log: \n\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n \n norm_d = problem.M.norm(x, d);\n\n % Backtracking default parameters. These can be overwritten in the\n % options structure which is passed to the solver.\n default_options.ls_contraction_factor = .5;\n default_options.ls_initial_stepsize = norm_d;\n default_options.ls_max_steps = 25;\n \n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(default_options, options);\n \n contraction_factor = options.ls_contraction_factor;\n initial_stepsize = options.ls_initial_stepsize;\n max_ls_steps = options.ls_max_steps;\n \n % Initial step size as a mutliplier of d.\n alpha = initial_stepsize / norm_d;\n \n % Make the chosen step and compute the cost there.\n newx = problem.M.retr(x, d, alpha);\n newkey = storedb.getNewKey();\n newf = getCost(problem, newx, storedb, newkey);\n cost_evaluations = 1;\n \n % Backtrack while no cost decrease is obtained.\n while newf >= f0\n \n % Reduce the step size,\n alpha = contraction_factor * alpha;\n \n % and look closer down the line\n newx = problem.M.retr(x, d, alpha);\n newkey = storedb.getNewKey();\n newf = getCost(problem, newx, storedb, newkey);\n cost_evaluations = cost_evaluations + 1;\n \n % Make sure we don't run out of budget\n if cost_evaluations >= max_ls_steps\n break;\n end\n \n end\n \n % If we got here without obtaining a decrease, we reject the step.\n % Equal cost is accepted, since if x is critical, it is important to\n % move away from x more than it is important to decrease the cost.\n if newf > f0\n alpha = 0;\n newx = x;\n newkey = key;\n newf = f0; %#ok\n end\n \n % As seen outside this function, stepsize is the size of the vector we\n % retract to make the step from x to newx. Since the step is alpha*d:\n stepsize = alpha * norm_d;\n \n % Return some statistics also, for possible analysis.\n lsstats.costevals = cost_evaluations;\n lsstats.stepsize = stepsize;\n lsstats.alpha = alpha;\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/linesearch/linesearch_hint.m", "content": "function [stepsize, newx, newkey, lsstats] = ...\n linesearch_hint(problem, x, d, f0, df0, options, storedb, key)\n% Armijo line-search based on the line-search hint in the problem structure.\n%\n% function [stepsize, newx, newkey, lsstats] = \n% linesearch_hint(problem, x, d, f0, df0, options, storedb, key)\n%\n% Base line-search algorithm for descent methods, based on a simple\n% backtracking method. The search direction provided has to be a descent\n% direction, as indicated by a negative df0 = directional derivative of f\n% at x along d.\n%\n% The algorithm obtains an initial step size candidate from the problem\n% structure, typically through the problem.linesearch function. If that\n% step does not fulfill the Armijo sufficient decrease criterion, that step\n% size is reduced geometrically until a satisfactory step size is obtained\n% or until a failure criterion triggers. If the problem structure does not\n% provide an initial alpha, then alpha = 1 is tried first.\n% \n% Below, the step is constructed as alpha*d, and the step size is the norm\n% of that vector, thus: stepsize = alpha*norm_d. The step is executed by\n% retracting the vector alpha*d from the current point x, giving newx.\n%\n% Inputs/Outputs : see help for linesearch\n%\n% See also: steepestdescent conjugategradients linesearch\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 17, 2014.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% April 8, 2015 (NB):\n% Got rid of lsmem input/output.\n%\n% July 20, 2017 (NB):\n% Now using alpha = 1 by default.\n%\n% Aug. 28, 2017 (NB):\n% Adding two options: ls_backtrack and ls_force_decrease, both true\n% by default. Setting them to false can disable parts of the line\n% search that, respectively, execute an Armijo backtracking and\n% reject a cost increasing step.\n\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n % Backtracking default parameters. These can be overwritten in the\n % options structure which is passed to the solver.\n default_options.ls_contraction_factor = .5;\n default_options.ls_suff_decr = 1e-4;\n default_options.ls_max_steps = 25;\n default_options.ls_backtrack = true;\n default_options.ls_force_decrease = true;\n \n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(default_options, options);\n \n contraction_factor = options.ls_contraction_factor;\n suff_decr = options.ls_suff_decr;\n max_ls_steps = options.ls_max_steps;\n \n % Obtain an initial guess at alpha from the problem structure. It is\n % assumed that the present line-search is only called when the problem\n % structure provides enough information for the call here to work.\n if canGetLinesearch(problem)\n alpha = getLinesearch(problem, x, d, storedb, key);\n else\n alpha = 1;\n end\n \n % Make the chosen step and compute the cost there.\n newx = problem.M.retr(x, d, alpha);\n newkey = storedb.getNewKey();\n newf = getCost(problem, newx, storedb, newkey);\n cost_evaluations = 1;\n \n % Backtrack while the Armijo criterion is not satisfied\n while options.ls_backtrack && newf > f0 + suff_decr*alpha*df0\n \n % Reduce the step size,\n alpha = contraction_factor * alpha;\n \n % and look closer down the line\n newx = problem.M.retr(x, d, alpha);\n newkey = storedb.getNewKey();\n newf = getCost(problem, newx, storedb, newkey);\n cost_evaluations = cost_evaluations + 1;\n \n % Make sure we don't run out of budget\n if cost_evaluations >= max_ls_steps\n break;\n end\n \n end\n \n % If we got here without obtaining a decrease, we reject the step.\n if options.ls_force_decrease && newf > f0\n alpha = 0;\n newx = x;\n newkey = key;\n newf = f0; %#ok\n end\n \n % As seen outside this function, stepsize is the size of the vector we\n % retract to make the step from x to newx. Since the step is alpha*d:\n norm_d = problem.M.norm(x, d);\n stepsize = alpha * norm_d;\n \n % Return some statistics also, for possible analysis.\n lsstats.costevals = cost_evaluations;\n lsstats.stepsize = stepsize;\n lsstats.alpha = alpha;\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/neldermead/centroid.m", "content": "function y = centroid(M, x)\n% Attempts the computation of a centroid of a set of points on a manifold.\n% \n% function y = centroid(M, x)\n%\n% M is a structure representing a manifold.\n% x is a cell of points on that manifold.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n\n\n % For now, just apply a few steps of gradient descent for Karcher means\n \n n = numel(x);\n \n problem.M = M;\n \n problem.cost = @cost;\n function val = cost(y)\n val = 0;\n for i = 1 : n\n val = val + M.dist(y, x{i})^2;\n end\n val = val/2;\n end\n\n problem.grad = @grad;\n function g = grad(y)\n g = M.zerovec(y);\n for i = 1 : n\n g = M.lincomb(y, 1, g, -1, M.log(y, x{i}));\n end\n end\n\n % This line can be uncommented to check that the gradient is indeed\n % correct. This should always be the case if the dist and the log\n % functions in the manifold are correct.\n % checkgradient(problem); pause;\n \n query = warning('query', 'manopt:getHessian:approx');\n warning('off', 'manopt:getHessian:approx');\n options.verbosity = 0;\n options.maxiter = 15;\n y = trustregions(problem, x{randi(n)}, options);\n warning(query.state, 'manopt:getHessian:approx');\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/neldermead/neldermead.m", "content": "function [x, cost, info, options] = neldermead(problem, x, options)\n% Nelder Mead optimization algorithm for derivative-free minimization.\n%\n% function [x, cost, info, options] = neldermead(problem)\n% function [x, cost, info, options] = neldermead(problem, x0)\n% function [x, cost, info, options] = neldermead(problem, x0, options)\n% function [x, cost, info, options] = neldermead(problem, [], options)\n%\n% Apply a Nelder-Mead minimization algorithm to the problem defined in\n% the problem structure, starting with the population x0 if it is provided\n% (otherwise, a random population on the manifold is generated). A\n% population is a cell containing points on the manifold. The number of\n% elements in the cell must be dim+1, where dim is the dimension of the\n% manifold: problem.M.dim().\n%\n% To specify options whilst not specifying an initial guess, give x0 as []\n% (the empty matrix).\n%\n% This algorithm is a plain adaptation of the Euclidean Nelder-Mead method\n% to the Riemannian setting. It comes with no convergence guarantees and\n% there is room for improvement. In particular, we compute centroids as\n% Karcher means, which seems overly expensive: cheaper forms of\n% average-like quantities might work better.\n% This solver is useful nonetheless for problems for which no derivatives\n% are available, and it may constitute a starting point for the development\n% of other Riemannian derivative-free methods.\n%\n% None of the options are mandatory. See in code for details.\n%\n% Requires problem.M.pairmean(x, y) to be defined (computes the average\n% between two points, x and y).\n%\n% If options.statsfun is defined, it will receive a cell of points x (the\n% current simplex being considered at that iteration), and, if required,\n% one store structure corresponding to the best point, x{1}. The points are\n% ordered by increasing cost: f(x{1}) <= f(x{2}) <= ... <= f(x{dim+1}),\n% where dim = problem.M.dim().\n%\n% Based on http://www.optimization-online.org/DB_FILE/2007/08/1742.pdf.\n%\n% See also: manopt/solvers/pso/pso\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 4, 2015 (NB):\n% Working with the new StoreDB class system.\n% Clarified interactions with statsfun and store.\n%\n% Nov. 11, 2016 (NB):\n% If options.verbosity is < 2, prints minimal output.\n\n \n % Verify that the problem description is sufficient for the solver.\n if ~canGetCost(problem)\n warning('manopt:getCost', ...\n 'No cost provided. The algorithm will likely abort.'); \n end\n \n % Dimension of the manifold\n dim = problem.M.dim();\n\n % Set local defaults here\n localdefaults.storedepth = 0; % no need for caching\n localdefaults.maxcostevals = max(1000, 2*dim);\n localdefaults.maxiter = max(2000, 4*dim);\n \n localdefaults.reflection = 1;\n localdefaults.expansion = 2;\n localdefaults.contraction = .5;\n % forced to .5 to enable using pairmean functions in manifolds.\n % localdefaults.shrinkage = .5;\n \n % Merge global and local defaults, then merge w/ user options, if any.\n localdefaults = mergeOptions(getGlobalDefaults(), localdefaults);\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n % Start timing for initialization.\n timetic = tic();\n \n % If no initial simplex x is given by the user, generate one at random.\n if ~exist('x', 'var') || isempty(x)\n x = cell(dim+1, 1);\n for i = 1 : dim+1\n x{i} = problem.M.rand();\n end\n end\n \n % Create a store database and a key for each point.\n storedb = StoreDB(options.storedepth);\n key = cell(size(x));\n for i = 1 : dim+1;\n key{i} = storedb.getNewKey();\n end\n \n % Compute objective-related quantities for x, and setup a\n % function evaluations counter.\n costs = zeros(dim+1, 1);\n for i = 1 : dim+1\n costs(i) = getCost(problem, x{i}, storedb, key{i});\n end\n costevals = dim+1;\n \n % Sort simplex points by cost.\n [costs, order] = sort(costs);\n x = x(order);\n key = key(order);\n \n % Iteration counter.\n % At any point, iter is the number of fully executed iterations so far.\n iter = 0;\n \n % Save stats in a struct array info, and preallocate.\n % savestats will be called twice for the initial iterate (number 0),\n % which is unfortunate, but not problematic.\n stats = savestats();\n info(1) = stats;\n info(min(10000, options.maxiter+1)).iter = [];\n \n % Start iterating until stopping criterion triggers.\n while true\n \n % Make sure we don't use to much memory for the store database.\n storedb.purge();\n \n stats = savestats();\n info(iter+1) = stats; %#ok\n iter = iter + 1;\n \n % Start timing this iteration.\n timetic = tic();\n \n % Sort simplex points by cost.\n [costs, order] = sort(costs);\n x = x(order);\n key = key(order);\n\n % Log / display iteration information here.\n if options.verbosity >= 2\n fprintf('Cost evals: %7d\\tBest cost: %+.4e\\t', ...\n costevals, costs(1));\n end\n \n % Run standard stopping criterion checks.\n [stop, reason] = stoppingcriterion(problem, x, options, info, iter);\n \n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n \n % Compute a centroid for the dim best points.\n xbar = centroid(problem.M, x(1:end-1));\n \n % Compute the direction for moving along the axis xbar - worst x.\n vec = problem.M.log(xbar, x{end});\n \n % Reflection step\n xr = problem.M.exp(xbar, vec, -options.reflection);\n keyr = storedb.getNewKey();\n costr = getCost(problem, xr, storedb, keyr);\n costevals = costevals + 1;\n \n % If the reflected point is honorable, drop the worst point,\n % replace it by the reflected point and start new iteration.\n if costr >= costs(1) && costr < costs(end-1)\n if options.verbosity >= 2\n fprintf('Reflection\\n');\n end\n costs(end) = costr;\n x{end} = xr;\n key{end} = keyr;\n continue;\n end\n \n % If the reflected point is better than the best point, expand.\n if costr < costs(1)\n xe = problem.M.exp(xbar, vec, -options.expansion);\n keye = storedb.getNewKey();\n coste = getCost(problem, xe, storedb, keye);\n costevals = costevals + 1;\n if coste < costr\n if options.verbosity >= 2\n fprintf('Expansion\\n');\n end\n costs(end) = coste;\n x{end} = xe;\n key{end} = keye;\n continue;\n else\n if options.verbosity >= 2\n fprintf('Reflection (failed expansion)\\n');\n end\n costs(end) = costr;\n x{end} = xr;\n key{end} = keyr;\n continue;\n end\n end\n \n % If the reflected point is worse than the second to worst point,\n\t\t% contract.\n if costr >= costs(end-1)\n if costr < costs(end)\n % do an outside contraction\n xoc = problem.M.exp(xbar, vec, -options.contraction);\n keyoc = storedb.getNewKey();\n costoc = getCost(problem, xoc, storedb, keyoc);\n costevals = costevals + 1;\n if costoc <= costr\n if options.verbosity >= 2\n fprintf('Outside contraction\\n');\n end\n costs(end) = costoc;\n x{end} = xoc;\n key{end} = keyoc;\n continue;\n end\n else\n % do an inside contraction\n xic = problem.M.exp(xbar, vec, options.contraction);\n keyic = storedb.getNewKey();\n costic = getCost(problem, xic, storedb, keyic);\n costevals = costevals + 1;\n if costic <= costs(end)\n if options.verbosity >= 2\n fprintf('Inside contraction\\n');\n end\n costs(end) = costic;\n x{end} = xic;\n key{end} = keyic;\n continue;\n end\n end\n end\n \n % If we get here, shrink the simplex around x{1}.\n if options.verbosity >= 2\n fprintf('Shrinkage\\n');\n end\n for i = 2 : dim+1\n x{i} = problem.M.pairmean(x{1}, x{i});\n key{i} = storedb.getNewKey();\n costs(i) = getCost(problem, x{i}, storedb, key{i});\n end\n costevals = costevals + dim;\n \n end\n \n \n info = info(1:iter);\n \n % Iteration done: return only the best point found.\n cost = costs(1);\n x = x{1};\n key = key{1};\n \n \n \n % Routine in charge of collecting the current iteration stats.\n function stats = savestats()\n stats.iter = iter;\n stats.cost = costs(1);\n stats.costevals = costevals;\n if iter == 0\n stats.time = toc(timetic);\n else\n stats.time = info(iter).time + toc(timetic);\n end\n % The statsfun can only possibly receive one store structure. We\n % pass the key to the best point, so that the best point's store\n % will be passed. But the whole cell x of points is passed through.\n stats = applyStatsfun(problem, x, storedb, key{1}, options, stats);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/preconditioners/preconhessiansolve.m", "content": "function preconfun = preconhessiansolve(problem, options)\n% Preconditioner based on the inverse Hessian, by solving linear systems.\n%\n% function preconfun = preconhessiansolve(problem)\n% function preconfun = preconhessiansolve(problem, options)\n%\n% Input:\n%\n% A Manopt problem structure (already containing the manifold and enough\n% information to compute the Hessian of the cost) and an options structure\n% (optional, currently ignored). Notice that if the Hessian is not positive\n% definite, then its inverse is not positive definite either and this\n% preconditioner is not suitable.\n%\n% If the Hessian cannot be computed on 'problem', a warning is issued. An\n% approximation of the Hessian will be used instead, and the present\n% preconditioner will attempt to invert that (although it may not be a\n% linear operator). If no approximate Hessian is provided either, a generic\n% approximation is used. Behavior is unspecified.\n%\n% Output:\n% \n% Returns a function handle, encapsulating a generic preconditioner of the\n% Hessian based on solving linear systems of the form:\n% Hessian(x)[preconfun(x, xdot)] = xdot,\n% where x is the point on the manifold, xdot is the input to the\n% preconditioner (a tangent vector) and preconfun(x, xdot) is returned\n% (also a tangent vector). The solve may be approximate.\n% \n% The returned preconfun has this calling pattern:\n% \n% function precxdot = preconfun(x, xdot)\n% function precxdot = preconfun(x, xdot, storedb)\n% function precxdot = preconfun(x, xdot, storedb, key)\n% \n% x is a point on the manifold problem.M, xdot is a tangent vector to that\n% manifold at x, storedb is a StoreDB object, and key is the StoreDB key to\n% point x.\n%\n% Usage:\n%\n% Typically, the user will set problem.M and other fields to define the\n% cost, the gradient and the Hessian (typically, problem.cost, problem.grad\n% and problem.hess, or problem.egrad and problem.ehess). Then, to use this\n% generic purpose Hessian preconditioner:\n%\n% problem.precon = preconhessiansolve(problem, options);\n%\n% Passing that problem structure to the conjugategradients solver\n% (which uses preconditioning) configured in steepest descent mode results\n% in a type of Riemannian Newton method.\n%\n% See also: conjugategradients\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 9, 2015.\n% Contributors: \n% Change log: \n\n % Check availability of the Hessian, or at least of an approximation.\n if ~canGetHessian(problem) && ~canGetApproxHessian(problem)\n % Note: we do not give a warning if an approximate Hessian is\n % explicitly given in the problem description, as in that case the\n % user seems to be aware of the issue.\n warning('manopt:getHessian:approx', ...\n ['No Hessian provided. Using an FD approximation instead.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:getHessian:approx'')']);\n problem.approxhess = approxhessianFD(problem);\n end\n\n % Set local defaults here, and merge with user options, if any.\n localdefaults = struct();\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n\n % Build and return the function handle here. This extra construct via\n % funhandle makes it possible to make storedb and key optional.\n preconfun = @funhandle;\n function precxdot = funhandle(x, xdot, storedb, key)\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end \n precxdot = hessiansolvehelper(options, problem, x, xdot, ...\n storedb, key);\n end\n \nend\n\n\nfunction precxdot = hessiansolvehelper(options, problem, x, xdot, storedb, key)\n% This function does the actual work.\n \n % Exclude the case where xdot is zero\n norm_xdot = problem.M.norm(x, xdot);\n if norm_xdot < eps\n precxdot = problem.M.zerovec(x);\n return;\n end\n \n % Get a shorthand for the Hessian of the cost on M at x.\n hessian = @(u) getHessian(problem, x, u, storedb, key);\n \n % Setup an optimization problem on the tangent space to problem.M at x.\n M = problem.M;\n tgtspace = tangentspacefactory(M, x);\n prblm.M = tgtspace;\n prblm.cost = @cost;\n prblm.grad = @grad;\n prblm.hess = @(u, udot) 2*hessian(hessian(udot))/norm_xdot;\n \n function [f, store] = cost(u, store)\n if ~isfield(store, 'residue')\n Hu = hessian(u);\n store.residue = M.lincomb(x, 1, Hu, -1, xdot);\n end\n f = M.norm(x, store.residue).^2 / norm_xdot;\n end\n function [g, store] = grad(u, store)\n if ~isfield(store, 'residue')\n Hu = hessian(u);\n store.residue = M.lincomb(x, 1, Hu, -1, xdot);\n end\n g = 2 * hessian(store.residue) / norm_xdot;\n end\n \n % checkgradient(prblm); pause;\n % checkhessian(prblm); pause;\n \n localdefaults.solver = @trustregions;\n localdefaults.verbosity = 0;\n % Merge local defaults with user options, if any.\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n % Solve the linear system by solving the optimization problem.\n precxdot = manoptsolve(prblm, M.zerovec(), options);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/pso/pso.m", "content": "function [xbest, fbest, info, options] = pso(problem, x, options)\n% Particle swarm optimization (PSO) for derivative-free minimization.\n%\n% function [x, cost, info, options] = pso(problem)\n% function [x, cost, info, options] = pso(problem, x0)\n% function [x, cost, info, options] = pso(problem, x0, options)\n% function [x, cost, info, options] = pso(problem, [], options)\n%\n% Apply the Particle Swarm Optimization minimization algorithm to\n% the problem defined in the problem structure, starting with the\n% population x0 if it is provided (otherwise, a random population on the\n% manifold is generated). A population is a cell containing points on the\n% manifold. The number of elements in the cell must match the parameter\n% options.populationsize.\n%\n% To specify options whilst not specifying an initial guess, give x0 as []\n% (the empty matrix).\n%\n% None of the options are mandatory. See in code for details.\n%\n% Based on the original PSO description in\n% http://particleswarm.info/nn951942.ps.\n%\n% See also: manopt/solvers/neldermead/neldermead\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Pierre Borckmans, Dec. 30, 2012.\n% Contributors: Bamdev Mishra, June 18, 2014.\n% Change log:\n%\n% June 18, 2014 (BM) :\n% Modified for handling product manifolds. Still need overall cleanup\n% to avoid potential issues, in particular wrt logarithms.\n%\n% June 23, 2014 (NB) :\n% Added some logic for handling of the populationsize option.\n%\n% April 5, 2015 (NB):\n% Working with the new StoreDB class system. The code keeps track of\n% storedb keys for all points, even though it is not strictly\n% necessary. This extra bookkeeping should help maintaining the code.\n \n \n % Verify that the problem description is sufficient for the solver.\n if ~canGetCost(problem)\n warning('manopt:getCost', ...\n 'No cost provided. The algorithm will likely abort.');\n end\n \n % Dimension of the manifold\n dim = problem.M.dim();\n \n % Set local defaults here\n localdefaults.storedepth = 0; % no need for caching\n localdefaults.maxcostevals = max(5000, 2*dim);\n localdefaults.maxiter = max(500, 4*dim);\n \n localdefaults.populationsize = min(40, 10*dim);\n localdefaults.nostalgia = 1.4;\n localdefaults.social = 1.4;\n \n % Merge global and local defaults, then merge w/ user options, if any.\n localdefaults = mergeOptions(getGlobalDefaults(), localdefaults);\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n \n if ~isfield(problem.M, 'log') % BM\n error(['The manifold problem.M must provide a logarithmic map, ' ...\n 'M.log(x, y). An approximate logarithm will do too.']);\n end\n \n % Start timing for initialization\n timetic = tic();\n \n % If no initial population x is given by the user,\n % generate one at random.\n if ~exist('x', 'var') || isempty(x)\n x = cell(options.populationsize, 1);\n for i = 1 : options.populationsize\n x{i} = problem.M.rand();\n end\n else\n if ~iscell(x)\n error('The initial guess x0 must be a cell (a population).');\n end\n if length(x) ~= options.populationsize\n options.populationsize = length(x);\n warning('manopt:pso:size', ...\n ['The option populationsize was forced to the size' ...\n ' of the given initial population x0.']);\n end\n end\n \n \n % Create a store database and a key for each point x{i}\n storedb = StoreDB(options.storedepth);\n xkey = cell(size(x));\n for i = 1 : numel(x)\n xkey{i} = storedb.getNewKey();\n end\n \n % Initialize personal best positions to the initial population\n y = x;\n ykey = xkey;\n \n % Save a copy of the swarm at the previous iteration\n xprev = x;\n xprevkey = xkey; %#ok\n \n % Initialize velocities for each particle\n v = cell(size(x));\n for i = 1 : numel(x)\n % random velocity to improve initial exploration\n v{i} = problem.M.randvec(x{i});\n % or null velocity\n % v{i} = problem.M.zerovec();\n end\n \n % Compute cost for each particle xi,\n % initialize personal best costs,\n % and setup a function evaluations counter.\n costs = zeros(size(x));\n for i = 1 : numel(x)\n costs(i) = getCost(problem, x{i}, storedb, xkey{i});\n end\n fy = costs;\n costevals = options.populationsize;\n \n % Identify the best particle and store its cost/position\n [fbest, imin] = min(costs);\n xbest = x{imin};\n xbestkey = xkey{imin}; %#ok\n \n % Iteration counter (at any point, iter is the number of fully executed\n % iterations so far)\n iter = 0;\n \n % Save stats in a struct array info, and preallocate.\n % savestats will be called twice for the initial iterate (number 0),\n % which is unfortunate, but not problematic.\n stats = savestats();\n info(1) = stats;\n info(min(10000, options.maxiter+1)).iter = [];\n \n % Start iterating until stopping criterion triggers\n while true\n \n stats = savestats();\n info(iter+1) = stats; %#ok\n iter = iter + 1;\n \n % Make sure we don't use too much memory for the store database\n storedb.purge();\n \n % Log / display iteration information here.\n if options.verbosity >= 2\n fprintf('Cost evals: %7d\\tBest cost: %+.8e\\n', costevals, fbest);\n end\n \n % Start timing this iteration\n timetic = tic();\n \n % BM: Run standard stopping criterion checks.\n % BM: Stop if any particle triggers a stopping criterion.\n for i = numel(x)\n [stop, reason] = stoppingcriterion(problem, x{i}, options, info, iter);\n if stop\n break;\n end\n end\n \n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n \n \n % Compute the inertia factor\n % (linearly decreasing from .9 to .4, from iter=0 to maxiter)\n w = 0.4 + 0.5*(1-iter/options.maxiter);\n \n % Compute velocities\n for i = 1 : numel(x)\n \n % Get the position and past best position of particle i\n xi = x{i};\n yi = y{i};\n \n % Get the previous position and velocity of particle i\n xiprev = xprev{i};\n vi = v{i};\n \n % Compute new velocity of particle i,\n % composed of 3 contributions\n inertia = problem.M.lincomb(xi, w , problem.M.transp(xiprev, xi, vi));\n nostalgia = problem.M.lincomb(xi, rand(1)*options.nostalgia, problem.M.log(xi, yi) );\n social = problem.M.lincomb(xi, rand(1) * options.social, problem.M.log(xi, xbest));\n \n v{i} = problem.M.lincomb(xi, 1, inertia, 1, problem.M.lincomb(xi, 1, nostalgia, 1, social));\n \n end\n \n % Backup the current swarm positions\n xprev = x;\n xprevkey = xkey; %#ok\n \n % Update positions, personal bests and global best\n for i = 1 : numel(x)\n % compute new position of particle i\n x{i} = problem.M.retr(x{i}, v{i});\n xkey{i} = storedb.getNewKey();\n % compute new cost of particle i\n fxi = getCost(problem, x{i}, storedb, xkey{i});\n costevals = costevals + 1;\n \n % update costs of the swarm\n costs(i) = fxi;\n % update self-best if necessary\n if fxi < fy(i)\n % update self-best cost and position\n fy(i) = fxi;\n y{i} = x{i};\n ykey{i} = xkey{i};\n % update global-best if necessary\n if fy(i) < fbest\n fbest = fy(i);\n xbest = y{i};\n xbestkey = ykey{i}; %#ok\n end\n end\n end\n end\n \n \n info = info(1:iter);\n \n % Routine in charge of collecting the current iteration stats\n function stats = savestats()\n stats.iter = iter;\n stats.cost = fbest;\n stats.costevals = costevals;\n stats.x = x;\n stats.v = v;\n stats.xbest = xbest;\n if iter == 0\n stats.time = toc(timetic);\n else\n stats.time = info(iter).time + toc(timetic);\n end\n \n % BM: Begin storing user defined stats for the entire population\n num_old_fields = size(fieldnames(stats), 1);\n trialstats = applyStatsfun(problem, x{1}, storedb, xkey{1}, options, stats);% BM\n new_fields = fieldnames(trialstats);\n num_new_fields = size(fieldnames(trialstats), 1);\n num_additional_fields = num_new_fields - num_old_fields; % User has defined new fields\n for jj = 1 : num_additional_fields % New fields added\n tempfield = new_fields(num_old_fields + jj);\n stats.(char(tempfield)) = cell(options.populationsize, 1);\n end\n for ii = 1 : options.populationsize % Adding information for each element of the population\n tempstats = applyStatsfun(problem, x{ii}, storedb, xkey{ii}, options, stats);\n for jj = 1 : num_additional_fields\n tempfield = new_fields(num_old_fields + jj);\n tempfield_value = tempstats.(char(tempfield));\n stats.(char(tempfield)){ii} = tempfield_value;\n end\n end\n % BM: End storing\n \n end\n \n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/steepestdescent/steepestdescent.m", "content": "function [x, cost, info, options] = steepestdescent(problem, x, options)\n% Steepest descent (gradient descent) minimization algorithm for Manopt.\n%\n% function [x, cost, info, options] = steepestdescent(problem)\n% function [x, cost, info, options] = steepestdescent(problem, x0)\n% function [x, cost, info, options] = steepestdescent(problem, x0, options)\n% function [x, cost, info, options] = steepestdescent(problem, [], options)\n%\n% Apply the steepest descent minimization algorithm to the problem defined\n% in the problem structure, starting at x0 if it is provided (otherwise, at\n% a random point on the manifold). To specify options whilst not specifying\n% an initial guess, give x0 as [] (the empty matrix).\n%\n% In most of the examples bundled with the toolbox (see link below), the\n% solver can be replaced by the present one if need be.\n%\n% The outputs x and cost are the best reached point on the manifold and its\n% cost. The struct-array info contains information about the iterations:\n% iter : the iteration number (0 for the initial guess)\n% cost : cost value\n% time : elapsed time in seconds\n% gradnorm : Riemannian norm of the gradient\n% stepsize : norm of the last tangent vector retracted\n% linesearch : information logged by options.linesearch\n% And possibly additional information logged by options.statsfun.\n% For example, type [info.gradnorm] to obtain a vector of the successive\n% gradient norms reached.\n%\n% The options structure is used to overwrite the default values. All\n% options have a default value and are hence optional. To force an option\n% value, pass an options structure with a field options.optionname, where\n% optionname is one of the following and the default value is indicated\n% between parentheses:\n%\n% tolgradnorm (1e-6)\n% The algorithm terminates if the norm of the gradient drops below this.\n% maxiter (1000)\n% The algorithm terminates if maxiter iterations have been executed.\n% maxtime (Inf)\n% The algorithm terminates if maxtime seconds elapsed.\n% minstepsize (1e-10)\n% The algorithm terminates if the linesearch returns a displacement\n% vector (to be retracted) smaller in norm than this value.\n% linesearch (@linesearch or @linesearch_hint)\n% Function handle to a line search function. The options structure is\n% passed to the line search too, so you can pass it parameters. See\n% each line search's documentation for info. Another available line\n% search in manopt is @linesearch_adaptive, in\n% /manopt/linesearch/linesearch_adaptive.m\n% If the problem structure includes a line search hint, then the\n% default line search used is @linesearch_hint.\n% statsfun (none)\n% Function handle to a function that will be called after each\n% iteration to provide the opportunity to log additional statistics.\n% They will be returned in the info struct. See the generic Manopt\n% documentation about solvers for further information.\n% stopfun (none)\n% Function handle to a function that will be called at each iteration\n% to provide the opportunity to specify additional stopping criteria.\n% See the generic Manopt documentation about solvers for further\n% information.\n% verbosity (3)\n% Integer number used to tune the amount of output the algorithm\n% generates during execution (mostly as text in the command window).\n% The higher, the more output. 0 means silent.\n% storedepth (2)\n% Maximum number of different points x of the manifold for which a\n% store structure will be kept in memory in the storedb. If the\n% caching features of Manopt are not used, this is irrelevant. For\n% the SD algorithm, a store depth of 2 should always be sufficient.\n%\n%\n% See also: conjugategradient trustregions manopt/solvers/linesearch manopt/examples\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n \n % Verify that the problem description is sufficient for the solver.\n if ~canGetCost(problem)\n warning('manopt:getCost', ...\n 'No cost provided. The algorithm will likely abort.');\n end\n if ~canGetGradient(problem) && ~canGetApproxGradient(problem)\n % Note: we do not give a warning if an approximate gradient is\n % explicitly given in the problem description, as in that case the\n % user seems to be aware of the issue.\n warning('manopt:getGradient:approx', ...\n ['No gradient provided. Using an FD approximation instead (slow).\\n' ...\n 'It may be necessary to increase options.tolgradnorm.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:getGradient:approx'')']);\n problem.approxgrad = approxgradientFD(problem);\n end\n \n % Set local defaults here\n localdefaults.minstepsize = 1e-10;\n localdefaults.maxiter = 1000;\n localdefaults.tolgradnorm = 1e-6;\n \n % Depending on whether the problem structure specifies a hint for\n % line-search algorithms, choose a default line-search that works on\n % its own (typical) or that uses the hint.\n if ~canGetLinesearch(problem)\n localdefaults.linesearch = @linesearch;\n else\n localdefaults.linesearch = @linesearch_hint;\n end\n \n % Merge global and local defaults, then merge w/ user options, if any.\n localdefaults = mergeOptions(getGlobalDefaults(), localdefaults);\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n timetic = tic();\n \n % If no initial point x is given by the user, generate one at random.\n if ~exist('x', 'var') || isempty(x)\n x = problem.M.rand();\n end\n \n % Create a store database and get a key for the current x\n storedb = StoreDB(options.storedepth);\n key = storedb.getNewKey();\n \n % Compute objective-related quantities for x\n [cost, grad] = getCostGrad(problem, x, storedb, key);\n gradnorm = problem.M.norm(x, grad);\n \n % Iteration counter.\n % At any point, iter is the number of fully executed iterations so far.\n iter = 0;\n \n % Save stats in a struct array info, and preallocate.\n stats = savestats();\n info(1) = stats;\n info(min(10000, options.maxiter+1)).iter = [];\n \n if options.verbosity >= 2\n fprintf(' iter\\t cost val\\t grad. norm\\n');\n end\n \n % Start iterating until stopping criterion triggers\n while true\n\n % Display iteration information\n if options.verbosity >= 2\n fprintf('%5d\\t%+.16e\\t%.8e\\n', iter, cost, gradnorm);\n end\n \n % Start timing this iteration\n timetic = tic();\n \n % Run standard stopping criterion checks\n [stop, reason] = stoppingcriterion(problem, x, options, ...\n info, iter+1);\n \n % If none triggered, run specific stopping criterion check\n if ~stop && stats.stepsize < options.minstepsize\n stop = true;\n reason = sprintf(['Last stepsize smaller than minimum ' ...\n 'allowed; options.minstepsize = %g.'], ...\n options.minstepsize);\n end\n \n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n\n % Pick the descent direction as minus the gradient\n desc_dir = problem.M.lincomb(x, -1, grad);\n \n % Execute the line search\n [stepsize, newx, newkey, lsstats] = options.linesearch( ...\n problem, x, desc_dir, cost, -gradnorm^2, ...\n options, storedb, key);\n \n % Compute the new cost-related quantities for x\n [newcost, newgrad] = getCostGrad(problem, newx, storedb, newkey);\n newgradnorm = problem.M.norm(newx, newgrad);\n \n % Make sure we don't use too much memory for the store database\n storedb.purge();\n \n % Transfer iterate info\n x = newx;\n key = newkey;\n cost = newcost;\n grad = newgrad;\n gradnorm = newgradnorm;\n \n % iter is the number of iterations we have accomplished.\n iter = iter + 1;\n \n % Log statistics for freshly executed iteration\n stats = savestats();\n info(iter+1) = stats;\n \n end\n \n \n info = info(1:iter+1);\n\n if options.verbosity >= 1\n fprintf('Total time is %f [s] (excludes statsfun)\\n', ...\n info(end).time);\n end\n \n \n \n % Routine in charge of collecting the current iteration stats\n function stats = savestats()\n stats.iter = iter;\n stats.cost = cost;\n stats.gradnorm = gradnorm;\n if iter == 0\n stats.stepsize = NaN;\n stats.time = toc(timetic);\n stats.linesearch = [];\n else\n stats.stepsize = stepsize;\n stats.time = info(iter).time + toc(timetic);\n stats.linesearch = lsstats;\n end\n stats = applyStatsfun(problem, x, storedb, key, options, stats);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/stochasticgradient/stepsize_sg.m", "content": "function [stepsize, newx, newkey, ssstats] = ...\n stepsize_sg(problem, x, d, iter, options, storedb, key) %#ok\n% Standard step size selection algorithm for the stochastic gradient method\n%\n% Given a problem structure, a point x on the manifold problem.d and a\n% tangent vector d at x, produces a stepsize (a positive real number) and a\n% new point newx obtained by retraction -stepsize*d at x. Additional inputs\n% include iter (the iteration number of x, where 0 marks the initial\n% guess), an options structure, a storedb database and the key of point x\n% in that database. Additional outputs include the key of newx in the\n% database, newkey, as well as a structure ssstats collecting statistics\n% about the work done during the call to this function.\n%\n% See in code for the role of available options:\n% options.stepsize_type\n% options.stepsize_init\n% options.stepsize_lambda\n% options.stepsize_decaysteps\n%\n% This function may create and maintain a structure called sssgmem inside\n% storedb.internal. This gives the function the opportunity to remember\n% what happened in previous calls.\n%\n% See also: stochasticgradient\n\n% This file is part of Manopt: www.manopt.org.\n% Original authors: Bamdev Mishra and Nicolas Boumal, March 30, 2017.\n% Contributors: Hiroyuki Kasai and Hiroyuki Sato.\n% Change log: \n\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey(); %#ok\n end\n \n\n % Initial stepsize guess.\n default_options.stepsize_init = 0.1;\n % Stepsize evolution type. Options are 'decay', 'fix' and 'hybrid'.\n default_options.stepsize_type = 'decay';\n % If stepsize_type = 'decay' or 'hybrid', lambda is a weighting factor.\n default_options.stepsize_lambda = 0.1;\n % If stepsize_type = 'hybrid', decaysteps states for how many\n % iterations the step size decays before becoming constant.\n default_options.stepsize_decaysteps = 100;\n \n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(default_options, options);\n \n\n type = options.stepsize_type;\n init = options.stepsize_init;\n lambda = options.stepsize_lambda;\n decaysteps = options.stepsize_decaysteps;\n\n \n switch lower(type)\n \n % Step size decays as O(1/iter).\n case 'decay'\n stepsize = init / (1 + init*lambda*iter);\n\n % Step size is fixed.\n case {'fix', 'fixed'}\n stepsize = init;\n\n % Step size decays only for the few initial iterations.\n case 'hybrid'\n if iter < decaysteps\n stepsize = init / (1 + init*lambda*iter);\n else\n stepsize = init / (1 + init*lambda*decaysteps);\n end\n\n otherwise\n error(['Unknown options.stepsize_type. ' ...\n 'Should be ''fix'', ''decay'' or ''hybrid''.']);\n \n end\n\n % Store some information.\n ssstats = struct();\n ssstats.stepsize = stepsize;\n\n % Compute the new point and give it a key.\n newx = problem.M.retr(x, d, -stepsize);\n newkey = storedb.getNewKey();\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/stochasticgradient/stochasticgradient.m", "content": "function [x, info, options] = stochasticgradient(problem, x, options)\n% Stochastic gradient (SG) minimization algorithm for Manopt.\n%\n% function [x, info, options] = stochasticgradient(problem)\n% function [x, info, options] = stochasticgradient(problem, x0)\n% function [x, info, options] = stochasticgradient(problem, x0, options)\n% function [x, info, options] = stochasticgradient(problem, [], options)\n%\n% Apply the Riemannian stochastic gradient algorithm to the problem defined\n% in the problem structure, starting at x0 if it is provided (otherwise, at\n% a random point on the manifold). To specify options whilst not specifying\n% an initial guess, give x0 as [] (the empty matrix).\n%\n% The problem structure must contain the following fields:\n%\n% problem.M:\n% Defines the manifold to optimize over, given by a factory.\n%\n% problem.partialgrad or problem.partialegrad (or equivalent)\n% Describes the partial gradients of the cost function. If the cost\n% function is of the form f(x) = sum_{k=1}^N f_k(x),\n% then partialegrad(x, K) = sum_{k \\in K} grad f_k(x).\n% As usual, partialgrad must define the Riemannian gradient, whereas\n% partialegrad defines a Euclidean (classical) gradient which will be\n% converted automatically to a Riemannian gradient. Use the tool\n% checkgradient(problem) to check it.\n%\n% problem.ncostterms\n% An integer specifying how many terms are in the cost function (in\n% the example above, that would be N.)\n%\n% Importantly, the cost function itself needs not be specified.\n%\n% Some of the options of the solver are specific to this file. Please have\n% a look inside the code.\n%\n% To record the value of the cost function or the norm of the gradient for\n% example (which are statistics the algorithm does not require and hence\n% does not compute by default), one can set the following options:\n%\n% metrics.cost = @(problem, x) getCost(problem, x);\n% metrics.gradnorm = @(problem, x) problem.M.norm(x, getGradient(problem, x));\n% options.statsfun = statsfunhelper(metrics);\n%\n% Important caveat: stochastic algorithms usually return an average of the\n% last few iterates. Computing averages on manifolds can be expensive.\n% Currently, this solver does not compute averages and simply returns the\n% last iterate. Using options.statsfun, it is possible for the user to\n% compute averages manually. If you have ideas on how to do this\n% generically, we welcome feedback. In particular, approximate means could\n% be computed with M.pairmean which is available in many geometries.\n%\n% See also: steepestdescent\n\n% This file is part of Manopt: www.manopt.org.\n% Original authors: Bamdev Mishra ,\n% Hiroyuki Kasai , and\n% Hiroyuki Sato , 22 April 2016.\n% Contributors: Nicolas Boumal\n% Change log: \n \n\n % Verify that the problem description is sufficient for the solver.\n if ~canGetPartialGradient(problem)\n warning('manopt:getPartialGradient', ...\n 'No partial gradient provided. The algorithm will likely abort.');\n end\n \n \n % Set local default\n localdefaults.maxiter = 1000; % Maximum number of iterations\n localdefaults.batchsize = 1; % Batchsize (# cost terms per iter)\n localdefaults.verbosity = 2; % Output verbosity (0, 1 or 2)\n localdefaults.storedepth = 20; % Limit amount of caching\n \n % Check stopping criteria and save stats every checkperiod iterations.\n localdefaults.checkperiod = 100;\n \n % stepsizefun is a function implementing a step size selection\n % algorithm. See that function for help with options, which can be\n % specified in the options structure passed to the solver directly.\n localdefaults.stepsizefun = @stepsize_sg;\n \n % Merge global and local defaults, then merge w/ user options, if any.\n localdefaults = mergeOptions(getGlobalDefaults(), localdefaults);\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n \n assert(options.checkperiod >= 1, ...\n 'options.checkperiod must be a positive integer (>= 1).');\n \n \n % If no initial point x is given by the user, generate one at random.\n if ~exist('x', 'var') || isempty(x)\n x = problem.M.rand();\n end\n \n % Create a store database and get a key for the current x\n storedb = StoreDB(options.storedepth);\n key = storedb.getNewKey();\n \n \n % Elapsed time for the current set of iterations, where a set of\n % iterations comprises options.checkperiod iterations. We do not\n % count time spent for such things as logging statistics, as these are\n % not relevant to the actual optimization process.\n elapsed_time = 0;\n \n % Total number of completed steps\n iter = 0;\n \n \n % Total number of saved stats at this point.\n savedstats = 0;\n \n % Collect and save stats in a struct array info, and preallocate.\n stats = savestats();\n info(1) = stats;\n savedstats = savedstats + 1;\n if isinf(options.maxiter)\n % We trust that if the user set maxiter = inf, then they defined\n % another stopping criterion.\n preallocate = 1e5;\n else\n preallocate = ceil(options.maxiter / options.checkperiod) + 1;\n end\n info(preallocate).iter = [];\n \n \n % Display information header for the user.\n if options.verbosity >= 2\n fprintf(' iter time [s] step size\\n');\n end\n \n \n % Main loop.\n stop = false;\n while iter < options.maxiter\n \n % Record start time.\n start_time = tic();\n \n % Draw the samples with replacement.\n idx_batch = randi(problem.ncostterms, options.batchsize, 1);\n \n % Compute partial gradient on this batch.\n pgrad = getPartialGradient(problem, x, idx_batch, storedb, key);\n \n % Compute a step size and the corresponding new point x.\n [stepsize, newx, newkey, ssstats] = ...\n options.stepsizefun(problem, x, pgrad, iter, ...\n options, storedb, key);\n \n % Make the step.\n x = newx;\n key = newkey;\n \n % Total number of completed steps.\n iter = iter + 1;\n \n % Make sure we do not use too much memory for the store database.\n storedb.purge();\n \n % Elapsed time doing actual optimization work so far in this\n % set of options.checkperiod iterations.\n elapsed_time = elapsed_time + toc(start_time);\n \n \n % Check stopping criteria and save stats every checkperiod iters.\n if mod(iter, options.checkperiod) == 0\n \n % Log statistics for freshly executed iteration.\n stats = savestats();\n info(savedstats+1) = stats;\n savedstats = savedstats + 1;\n \n % Reset timer.\n elapsed_time = 0;\n \n % Print output.\n if options.verbosity >= 2\n fprintf('%8d %10.2f %.3e\\n', ...\n iter, stats.time, stepsize);\n end\n \n % Run standard stopping criterion checks.\n [stop, reason] = stoppingcriterion(problem, x, ...\n options, info, savedstats);\n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n \n end\n\n end\n \n \n % Keep only the relevant portion of the info struct-array.\n info = info(1:savedstats);\n \n \n % Display a final information message.\n if options.verbosity >= 1\n if ~stop\n % We stopped not because of stoppingcriterion but because the\n % loop came to an end, which means maxiter triggered.\n msg = 'Max iteration count reached; options.maxiter = %g.\\n';\n fprintf(msg, options.maxiter);\n end\n fprintf('Total time is %f [s] (excludes statsfun)\\n', ...\n info(end).time + elapsed_time);\n end\n \n \n % Helper function to collect statistics to be saved at\n % index checkperiodcount+1 in info.\n function stats = savestats()\n stats.iter = iter;\n if savedstats == 0\n stats.time = 0;\n stats.stepsize = NaN;\n stats.stepsize_stats = [];\n else\n stats.time = info(savedstats).time + elapsed_time;\n stats.stepsize = stepsize;\n stats.stepsize_stats = ssstats;\n end\n stats = applyStatsfun(problem, x, storedb, key, options, stats);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/trustregions/license for original GenRTR code.txt", "content": "Copyright (c) 2007,2012 Christopher G. Baker, Pierre-Antoine Absil, Kyle A. Gallivan\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are met:\n * Redistributions of source code must retain the above copyright\n notice, this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above copyright\n notice, this list of conditions and the following disclaimer in the\n documentation and/or other materials provided with the distribution.\n * Neither the names of the contributors nor of their affiliated \n institutions may be used to endorse or promote products\n derived from this software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND\nANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\nWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY\nDIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\nLOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND\nON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\nSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\nFor questions, please contact Chris Baker (chris@cgbaker.net)\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/trustregions/tCG.m", "content": "function [eta, Heta, inner_it, stop_tCG] ...\n = tCG(problem, x, grad, eta, Delta, options, storedb, key)\n% tCG - Truncated (Steihaug-Toint) Conjugate-Gradient method\n% minimize + .5*\n% subject to _[inverse precon] <= Delta^2\n%\n% See also: trustregions\n\n% This file is part of Manopt: www.manopt.org.\n% This code is an adaptation to Manopt of the original GenRTR code:\n% RTR - Riemannian Trust-Region\n% (c) 2004-2007, P.-A. Absil, C. G. Baker, K. A. Gallivan\n% Florida State University\n% School of Computational Science\n% (http://www.math.fsu.edu/~cbaker/GenRTR/?page=download)\n% See accompanying license file.\n% The adaptation was executed by Nicolas Boumal.\n%\n% Change log:\n%\n% NB Feb. 12, 2013:\n% We do not project r back to the tangent space anymore: it was not\n% necessary, and as of Manopt 1.0.1, the proj operator does not\n% coincide with this notion anymore.\n%\n% NB April 3, 2013:\n% tCG now also returns Heta, the Hessian at x along eta. Additional\n% esthetic modifications.\n%\n% NB Dec. 2, 2013:\n% If options.useRand is activated, we now make sure the preconditio-\n% ner is not used, as was originally intended in GenRTR. In time, we\n% may want to investigate whether useRand can be modifed to work well\n% with preconditioning too.\n%\n% NB Jan. 9, 2014:\n% Now checking explicitly for model decrease at each iteration. The\n% first iteration is a Cauchy point, which necessarily realizes a\n% decrease of the model cost. If a model increase is witnessed\n% (which is theoretically impossible if a linear operator is used for\n% the Hessian approximation), then we return the previous eta. This\n% ensures we always achieve at least the Cauchy decrease, which\n% should be sufficient for convergence.\n%\n% NB Feb. 17, 2015:\n% The previous update was in effect verifying that the current eta\n% performed at least as well as the first eta (the Cauchy step) with\n% respect to the model cost. While this is an acceptable strategy,\n% the documentation (and the original intent) was to ensure a\n% monotonic decrease of the model cost at each new eta. This is now\n% the case, with the added line: \"model_value = new_model_value;\".\n%\n% NB April 3, 2015:\n% Works with the new StoreDB class system.\n\n\n% All terms involving the trust-region radius will use an inner product\n% w.r.t. the preconditioner; this is because the iterates grow in\n% length w.r.t. the preconditioner, guaranteeing that we will not\n% re-enter the trust-region.\n%\n% The following recurrences for Prec-based norms and inner\n% products come from [CGT2000], pg. 205, first edition.\n% Below, P is the preconditioner.\n%\n% = \n% beta_k-1 * ( + alpha_k-1 |delta_k-1|^2_P )\n% |delta_k|^2_P = + beta_k-1^2 |delta_k-1|^2_P\n%\n% therefore, we need to keep track of\n% 1) |delta_k|^2_P\n% 2) = _P\n% 3) |eta_k |^2_P\n%\n% initial values are given by:\n% |delta_0|_P = \n% |eta_0|_P = 0\n% _P = 0\n% because we take eta_0 = 0 (if useRand = false).\n%\n% [CGT2000] Conn, Gould and Toint: Trust-region methods, 2000.\n\ninner = problem.M.inner;\nlincomb = problem.M.lincomb;\n\ntheta = options.theta;\nkappa = options.kappa;\n\nif ~options.useRand % and therefore, eta == 0\n Heta = problem.M.zerovec(x);\n r = grad;\n e_Pe = 0;\nelse % and therefore, no preconditioner\n % eta (presumably) ~= 0 was provided by the caller.\n Heta = getHessian(problem, x, eta, storedb, key);\n r = lincomb(x, 1, grad, 1, Heta);\n e_Pe = inner(x, eta, eta);\nend\nr_r = inner(x, r, r);\nnorm_r = sqrt(r_r);\nnorm_r0 = norm_r;\n\n% Precondition the residual.\nif ~options.useRand\n z = getPrecon(problem, x, r, storedb, key);\nelse\n z = r;\nend\n\n% Compute z'*r.\nz_r = inner(x, z, r);\nd_Pd = z_r;\n\n% Initial search direction.\ndelta = lincomb(x, -1, z);\nif ~options.useRand % and therefore, eta == 0\n e_Pd = 0;\nelse % and therefore, no preconditioner\n e_Pd = inner(x, eta, delta);\nend\n\n% If the Hessian or a linear Hessian approximation is in use, it is\n% theoretically guaranteed that the model value decreases strictly\n% with each iteration of tCG. Hence, there is no need to monitor the model\n% value. But, when a nonlinear Hessian approximation is used (such as the\n% built-in finite-difference approximation for example), the model may\n% increase. It is then important to terminate the tCG iterations and return\n% the previous (the best-so-far) iterate. The variable below will hold the\n% model value.\nmodel_fun = @(eta, Heta) inner(x, eta, grad) + .5*inner(x, eta, Heta);\nif ~options.useRand\n model_value = 0;\nelse\n model_value = model_fun(eta, Heta);\nend\n\n% Pre-assume termination because j == end.\nstop_tCG = 5;\n\n% Begin inner/tCG loop.\nj = 0;\nfor j = 1 : options.maxinner\n \n % This call is the computationally expensive step.\n Hdelta = getHessian(problem, x, delta, storedb, key);\n \n % Compute curvature (often called kappa).\n d_Hd = inner(x, delta, Hdelta);\n \n \n % Note that if d_Hd == 0, we will exit at the next \"if\" anyway.\n alpha = z_r/d_Hd;\n % _P =\n % _P + 2*alpha*_P + alpha*alpha*_P\n e_Pe_new = e_Pe + 2.0*alpha*e_Pd + alpha*alpha*d_Pd;\n \n if options.debug > 2,\n fprintf('DBG: (r,r) : %e\\n', r_r);\n fprintf('DBG: (d,Hd) : %e\\n', d_Hd);\n fprintf('DBG: alpha : %e\\n', alpha);\n end\n \n % Check against negative curvature and trust-region radius violation.\n % If either condition triggers, we bail out.\n if d_Hd <= 0 || e_Pe_new >= Delta^2,\n % want\n % ee = _prec,x\n % ed = _prec,x\n % dd = _prec,x\n tau = (-e_Pd + sqrt(e_Pd*e_Pd + d_Pd*(Delta^2-e_Pe))) / d_Pd;\n if options.debug > 2,\n fprintf('DBG: tau : %e\\n', tau);\n end\n eta = lincomb(x, 1, eta, tau, delta);\n \n % If only a nonlinear Hessian approximation is available, this is\n % only approximately correct, but saves an additional Hessian call.\n Heta = lincomb(x, 1, Heta, tau, Hdelta);\n \n % Technically, we may want to verify that this new eta is indeed\n % better than the previous eta before returning it (this is always\n % the case if the Hessian approximation is linear, but I am unsure\n % whether it is the case or not for nonlinear approximations.)\n % At any rate, the impact should be limited, so in the interest of\n % code conciseness (if we can still hope for that), we omit this.\n \n if d_Hd <= 0,\n stop_tCG = 1; % negative curvature\n else\n stop_tCG = 2; % exceeded trust region\n end\n break;\n end\n \n % No negative curvature and eta_prop inside TR: accept it.\n e_Pe = e_Pe_new;\n new_eta = lincomb(x, 1, eta, alpha, delta);\n \n % If only a nonlinear Hessian approximation is available, this is\n % only approximately correct, but saves an additional Hessian call.\n new_Heta = lincomb(x, 1, Heta, alpha, Hdelta);\n \n % Verify that the model cost decreased in going from eta to new_eta. If\n % it did not (which can only occur if the Hessian approximation is\n % nonlinear or because of numerical errors), then we return the\n % previous eta (which necessarily is the best reached so far, according\n % to the model cost). Otherwise, we accept the new eta and go on.\n new_model_value = model_fun(new_eta, new_Heta);\n if new_model_value >= model_value\n stop_tCG = 6;\n break;\n end\n \n eta = new_eta;\n Heta = new_Heta;\n model_value = new_model_value; %% added Feb. 17, 2015\n \n % Update the residual.\n r = lincomb(x, 1, r, alpha, Hdelta);\n \n % Compute new norm of r.\n r_r = inner(x, r, r);\n norm_r = sqrt(r_r);\n \n % Check kappa/theta stopping criterion.\n % Note that it is somewhat arbitrary whether to check this stopping\n % criterion on the r's (the gradients) or on the z's (the\n % preconditioned gradients). [CGT2000], page 206, mentions both as\n % acceptable criteria.\n if j >= options.mininner && norm_r <= norm_r0*min(norm_r0^theta, kappa)\n % Residual is small enough to quit\n if kappa < norm_r0^theta,\n stop_tCG = 3; % linear convergence\n else\n stop_tCG = 4; % superlinear convergence\n end\n break;\n end\n \n % Precondition the residual.\n if ~options.useRand\n z = getPrecon(problem, x, r, storedb, key);\n else\n z = r;\n end\n \n % Save the old z'*r.\n zold_rold = z_r;\n % Compute new z'*r.\n z_r = inner(x, z, r);\n \n % Compute new search direction.\n beta = z_r/zold_rold;\n delta = lincomb(x, -1, z, beta, delta);\n \n % Update new P-norms and P-dots [CGT2000, eq. 7.5.6 & 7.5.7].\n e_Pd = beta*(e_Pd + alpha*d_Pd);\n d_Pd = z_r + beta*beta*d_Pd;\n \nend % of tCG loop\ninner_it = j;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/solvers/trustregions/trustregions.m", "content": "function [x, cost, info, options] = trustregions(problem, x, options)\n% Riemannian trust-regions solver for optimization on manifolds.\n%\n% function [x, cost, info, options] = trustregions(problem)\n% function [x, cost, info, options] = trustregions(problem, x0)\n% function [x, cost, info, options] = trustregions(problem, x0, options)\n% function [x, cost, info, options] = trustregions(problem, [], options)\n%\n% This is the Riemannian Trust-Region solver (with tCG inner solve), named\n% RTR. This solver will attempt to minimize the cost function described in\n% the problem structure. It requires the availability of the cost function\n% and of its gradient. It will issue calls for the Hessian. If no Hessian\n% nor approximate Hessian is provided, a standard approximation of the\n% Hessian based on the gradient will be computed. If a preconditioner for\n% the Hessian is provided, it will be used.\n%\n% If no gradient is provided, an approximation of the gradient is computed,\n% but this can be slow for manifolds of high dimension.\n%\n% For a description of the algorithm and theorems offering convergence\n% guarantees, see the references below. Documentation for this solver is\n% available online at:\n%\n% http://www.manopt.org/solver_documentation_trustregions.html\n%\n%\n% The initial iterate is x0 if it is provided. Otherwise, a random point on\n% the manifold is picked. To specify options whilst not specifying an\n% initial iterate, give x0 as [] (the empty matrix).\n%\n% The two outputs 'x' and 'cost' are the last reached point on the manifold\n% and its cost. Notice that x is not necessarily the best reached point,\n% because this solver is not forced to be a descent method. In particular,\n% very close to convergence, it is sometimes preferable to accept very\n% slight increases in the cost value (on the order of the machine epsilon)\n% in the process of reaching fine convergence.\n% \n% The output 'info' is a struct-array which contains information about the\n% iterations:\n% iter (integer)\n% The (outer) iteration number, or number of steps considered\n% (whether accepted or rejected). The initial guess is 0.\n%\tcost (double)\n% The corresponding cost value.\n%\tgradnorm (double)\n% The (Riemannian) norm of the gradient.\n%\tnuminner (integer)\n% The number of inner iterations executed to compute this iterate.\n% Inner iterations are truncated-CG steps. Each one requires a\n% Hessian (or approximate Hessian) evaluation.\n%\ttime (double)\n% The total elapsed time in seconds to reach the corresponding cost.\n%\trho (double)\n% The performance ratio for the iterate.\n%\trhonum, rhoden (double)\n% Regularized numerator and denominator of the performance ratio:\n% rho = rhonum/rhoden. See options.rho_regularization.\n%\taccepted (boolean)\n% Whether the proposed iterate was accepted or not.\n%\tstepsize (double)\n% The (Riemannian) norm of the vector returned by the inner solver\n% tCG and which is retracted to obtain the proposed next iterate. If\n% accepted = true for the corresponding iterate, this is the size of\n% the step from the previous to the new iterate. If accepted is\n% false, the step was not executed and this is the size of the\n% rejected step.\n%\tDelta (double)\n% The trust-region radius at the outer iteration.\n%\tcauchy (boolean)\n% Whether the Cauchy point was used or not (if useRand is true).\n% And possibly additional information logged by options.statsfun.\n% For example, type [info.gradnorm] to obtain a vector of the successive\n% gradient norms reached at each (outer) iteration.\n%\n% The options structure is used to overwrite the default values. All\n% options have a default value and are hence optional. To force an option\n% value, pass an options structure with a field options.optionname, where\n% optionname is one of the following and the default value is indicated\n% between parentheses:\n%\n% tolgradnorm (1e-6)\n% The algorithm terminates if the norm of the gradient drops below\n% this. For well-scaled problems, a rule of thumb is that you can\n% expect to reduce the gradient norm by 8 orders of magnitude\n% (sqrt(eps)) compared to the gradient norm at a \"typical\" point (a\n% rough initial iterate for example). Further decrease is sometimes\n% possible, but inexact floating point arithmetic will eventually\n% limit the final accuracy. If tolgradnorm is set too low, the\n% algorithm may end up iterating forever (or at least until another\n% stopping criterion triggers).\n% maxiter (1000)\n% The algorithm terminates if maxiter (outer) iterations were executed.\n% maxtime (Inf)\n% The algorithm terminates if maxtime seconds elapsed.\n%\tminiter (3)\n% Minimum number of outer iterations (used only if useRand is true).\n%\tmininner (1)\n% Minimum number of inner iterations (for tCG).\n%\tmaxinner (problem.M.dim() : the manifold's dimension)\n% Maximum number of inner iterations (for tCG).\n%\tDelta_bar (problem.M.typicaldist() or sqrt(problem.M.dim()))\n% Maximum trust-region radius. If you specify this parameter but not\n% Delta0, then Delta0 will be set to 1/8 times this parameter.\n% Delta0 (Delta_bar/8)\n% Initial trust-region radius. If you observe a long plateau at the\n% beginning of the convergence plot (gradient norm VS iteration), it\n% may pay off to try to tune this parameter to shorten the plateau.\n% You should not set this parameter without setting Delta_bar too (at\n% a larger value).\n%\tuseRand (false)\n% Set to true if the trust-region solve is to be initiated with a\n% random tangent vector. If set to true, no preconditioner will be\n% used. This option is set to true in some scenarios to escape saddle\n% points, but is otherwise seldom activated.\n%\tkappa (0.1)\n% tCG inner kappa convergence tolerance.\n% kappa > 0 is the linear convergence target rate: tCG will terminate\n% early if the residual was reduced by a factor of kappa.\n%\ttheta (1.0)\n% tCG inner theta convergence tolerance.\n% 1+theta (theta between 0 and 1) is the superlinear convergence\n% target rate. tCG will terminate early if the residual was reduced\n% by a power of 1+theta.\n%\trho_prime (0.1)\n% Accept/reject threshold : if rho is at least rho_prime, the outer\n% iteration is accepted. Otherwise, it is rejected. In case it is\n% rejected, the trust-region radius will have been decreased.\n% To ensure this, rho_prime >= 0 must be strictly smaller than 1/4.\n% If rho_prime is negative, the algorithm is not guaranteed to\n% produce monotonically decreasing cost values. It is strongly\n% recommended to set rho_prime > 0, to aid convergence.\n% rho_regularization (1e3)\n% Close to convergence, evaluating the performance ratio rho is\n% numerically challenging. Meanwhile, close to convergence, the\n% quadratic model should be a good fit and the steps should be\n% accepted. Regularization lets rho go to 1 as the model decrease and\n% the actual decrease go to zero. Set this option to zero to disable\n% regularization (not recommended). See in-code for the specifics.\n% When this is not zero, it may happen that the iterates produced are\n% not monotonically improving the cost when very close to\n% convergence. This is because the corrected cost improvement could\n% change sign if it is negative but very small.\n% statsfun (none)\n% Function handle to a function that will be called after each\n% iteration to provide the opportunity to log additional statistics.\n% They will be returned in the info struct. See the generic Manopt\n% documentation about solvers for further information. statsfun is\n% called with the point x that was reached last, after the\n% accept/reject decision. See comment below.\n% stopfun (none)\n% Function handle to a function that will be called at each iteration\n% to provide the opportunity to specify additional stopping criteria.\n% See the generic Manopt documentation about solvers for further\n% information.\n% verbosity (2)\n% Integer number used to tune the amount of output the algorithm\n% generates during execution (mostly as text in the command window).\n% The higher, the more output. 0 means silent. 3 and above includes a\n% display of the options structure at the beginning of the execution.\n% debug (false)\n% Set to true to allow the algorithm to perform additional\n% computations for debugging purposes. If a debugging test fails, you\n% will be informed of it, usually via the command window. Be aware\n% that these additional computations appear in the algorithm timings\n% too, and may interfere with operations such as counting the number\n% of cost evaluations, etc. (the debug calls get storedb too).\n% storedepth (20)\n% Maximum number of different points x of the manifold for which a\n% store structure will be kept in memory in the storedb. If the\n% caching features of Manopt are not used, this is irrelevant. If\n% memory usage is an issue, you may try to lower this number.\n% Profiling may then help to investigate if a performance hit was\n% incurred as a result.\n%\n% Notice that statsfun is called with the point x that was reached last,\n% after the accept/reject decision. Hence: if the step was accepted, we get\n% that new x, with a store which only saw the call for the cost and for the\n% gradient. If the step was rejected, we get the same x as previously, with\n% the store structure containing everything that was computed at that point\n% (possibly including previous rejects at that same point). Hence, statsfun\n% should not be used in conjunction with the store to count operations for\n% example. Instead, you should use storedb's shared memory for such\n% purposes (either via storedb.shared, or via store.shared, see\n% online documentation). It is however possible to use statsfun with the\n% store to compute, for example, other merit functions on the point x\n% (other than the actual cost function, that is).\n%\n%\n% Please cite the Manopt paper as well as the research paper:\n% @Article{genrtr,\n% Title = {Trust-region methods on {Riemannian} manifolds},\n% Author = {Absil, P.-A. and Baker, C. G. and Gallivan, K. A.},\n% Journal = {Foundations of Computational Mathematics},\n% Year = {2007},\n% Number = {3},\n% Pages = {303--330},\n% Volume = {7},\n% Doi = {10.1007/s10208-005-0179-9}\n% }\n%\n% See also: steepestdescent conjugategradient manopt/examples\n\n% An explicit, general listing of this algorithm, with preconditioning,\n% can be found in the following paper:\n% @Article{boumal2015lowrank,\n% Title = {Low-rank matrix completion via preconditioned optimization on the {G}rassmann manifold},\n% Author = {Boumal, N. and Absil, P.-A.},\n% Journal = {Linear Algebra and its Applications},\n% Year = {2015},\n% Pages = {200--239},\n% Volume = {475},\n% Doi = {10.1016/j.laa.2015.02.027},\n% }\n\n% When the Hessian is not specified, it is approximated with\n% finite-differences of the gradient. The resulting method is called\n% RTR-FD. Some convergence theory for it is available in this paper:\n% @incollection{boumal2015rtrfd\n% \tauthor={Boumal, N.},\n% \ttitle={Riemannian trust regions with finite-difference Hessian approximations are globally convergent},\n% \tyear={2015},\n% \tbooktitle={Geometric Science of Information}\n% }\n\n\n% This file is part of Manopt: www.manopt.org.\n% This code is an adaptation to Manopt of the original GenRTR code:\n% RTR - Riemannian Trust-Region\n% (c) 2004-2007, P.-A. Absil, C. G. Baker, K. A. Gallivan\n% Florida State University\n% School of Computational Science\n% (http://www.math.fsu.edu/~cbaker/GenRTR/?page=download)\n% See accompanying license file.\n% The adaptation was executed by Nicolas Boumal.\n%\n%\n% Change log: \n%\n% NB April 3, 2013:\n% tCG now returns the Hessian along the returned direction eta, so\n% that we do not compute that Hessian redundantly: some savings at\n% each iteration. Similarly, if the useRand flag is on, we spare an\n% extra Hessian computation at each outer iteration too, owing to\n% some modifications in the Cauchy point section of the code specific\n% to useRand = true.\n%\n% NB Aug. 22, 2013:\n% This function is now Octave compatible. The transition called for\n% two changes which would otherwise not be advisable. (1) tic/toc is\n% now used as is, as opposed to the safer way:\n% t = tic(); elapsed = toc(t);\n% And (2), the (formerly inner) function savestats was moved outside\n% the main function to not be nested anymore. This is arguably less\n% elegant, but Octave does not (and likely will not) support nested\n% functions.\n%\n% NB Dec. 2, 2013:\n% The in-code documentation was largely revised and expanded.\n%\n% NB Dec. 2, 2013:\n% The former heuristic which triggered when rhonum was very small and\n% forced rho = 1 has been replaced by a smoother heuristic which\n% consists in regularizing rhonum and rhoden before computing their\n% ratio. It is tunable via options.rho_regularization. Furthermore,\n% the solver now detects if tCG did not obtain a model decrease\n% (which is theoretically impossible but may happen because of\n% numerical errors and/or because of a nonlinear/nonsymmetric Hessian\n% operator, which is the case for finite difference approximations).\n% When such an anomaly is detected, the step is rejected and the\n% trust region radius is decreased.\n% Feb. 18, 2015 note: this is less useful now, as tCG now guarantees\n% model decrease even for the finite difference approximation of the\n% Hessian. It is still useful in case of numerical errors, but this\n% is less stringent.\n%\n% NB Dec. 3, 2013:\n% The stepsize is now registered at each iteration, at a small\n% additional cost. The defaults for Delta_bar and Delta0 are better\n% defined. Setting Delta_bar in the options will automatically set\n% Delta0 accordingly. In Manopt 1.0.4, the defaults for these options\n% were not treated appropriately because of an incorrect use of the\n% isfield() built-in function.\n%\n% NB Feb. 18, 2015:\n% Added some comments. Also, Octave now supports safe tic/toc usage,\n% so we reverted the changes to use that again (see Aug. 22, 2013 log\n% entry).\n%\n% NB April 3, 2015:\n% Works with the new StoreDB class system.\n%\n% NB April 8, 2015:\n% No Hessian warning if approximate Hessian explicitly available.\n%\n% NB Nov. 1, 2016:\n% Now uses approximate gradient via finite differences if need be.\n\n\n% Verify that the problem description is sufficient for the solver.\nif ~canGetCost(problem)\n warning('manopt:getCost', ...\n 'No cost provided. The algorithm will likely abort.'); \nend\nif ~canGetGradient(problem) && ~canGetApproxGradient(problem)\n % Note: we do not give a warning if an approximate gradient is\n % explicitly given in the problem description, as in that case the user\n % seems to be aware of the issue.\n warning('manopt:getGradient:approx', ...\n ['No gradient provided. Using an FD approximation instead (slow).\\n' ...\n 'It may be necessary to increase options.tolgradnorm.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:getGradient:approx'')']);\n problem.approxgrad = approxgradientFD(problem);\nend\nif ~canGetHessian(problem) && ~canGetApproxHessian(problem)\n % Note: we do not give a warning if an approximate Hessian is\n % explicitly given in the problem description, as in that case the user\n % seems to be aware of the issue.\n warning('manopt:getHessian:approx', ...\n ['No Hessian provided. Using an FD approximation instead.\\n' ...\n 'To disable this warning: warning(''off'', ''manopt:getHessian:approx'')']);\n problem.approxhess = approxhessianFD(problem);\nend\n\n% Define some strings for display\ntcg_stop_reason = {'negative curvature',...\n 'exceeded trust region',...\n 'reached target residual-kappa (linear)',...\n 'reached target residual-theta (superlinear)',...\n 'maximum inner iterations',...\n 'model increased'};\n\n% Set local defaults here\nlocaldefaults.verbosity = 2;\nlocaldefaults.maxtime = inf;\nlocaldefaults.miniter = 3;\nlocaldefaults.maxiter = 1000;\nlocaldefaults.mininner = 1;\nlocaldefaults.maxinner = problem.M.dim();\nlocaldefaults.tolgradnorm = 1e-6;\nlocaldefaults.kappa = 0.1;\nlocaldefaults.theta = 1.0;\nlocaldefaults.rho_prime = 0.1;\nlocaldefaults.useRand = false;\nlocaldefaults.rho_regularization = 1e3;\n\n% Merge global and local defaults, then merge w/ user options, if any.\nlocaldefaults = mergeOptions(getGlobalDefaults(), localdefaults);\nif ~exist('options', 'var') || isempty(options)\n options = struct();\nend\noptions = mergeOptions(localdefaults, options);\n\n% Set default Delta_bar and Delta0 separately to deal with additional\n% logic: if Delta_bar is provided but not Delta0, let Delta0 automatically\n% be some fraction of the provided Delta_bar.\nif ~isfield(options, 'Delta_bar')\n if isfield(problem.M, 'typicaldist')\n options.Delta_bar = problem.M.typicaldist();\n else\n options.Delta_bar = sqrt(problem.M.dim());\n end \nend\nif ~isfield(options,'Delta0')\n options.Delta0 = options.Delta_bar / 8;\nend\n\n% Check some option values\nassert(options.rho_prime < 1/4, ...\n 'options.rho_prime must be strictly smaller than 1/4.');\nassert(options.Delta_bar > 0, ...\n 'options.Delta_bar must be positive.');\nassert(options.Delta0 > 0 && options.Delta0 < options.Delta_bar, ...\n 'options.Delta0 must be positive and smaller than Delta_bar.');\n\n% It is sometimes useful to check what the actual option values are.\nif options.verbosity >= 3\n disp(options);\nend\n\nticstart = tic();\n\n% If no initial point x is given by the user, generate one at random.\nif ~exist('x', 'var') || isempty(x)\n x = problem.M.rand();\nend\n\n% Create a store database and get a key for the current x\nstoredb = StoreDB(options.storedepth);\nkey = storedb.getNewKey();\n\n%% Initializations\n\n% k counts the outer (TR) iterations. The semantic is that k counts the\n% number of iterations fully executed so far.\nk = 0;\n\n% Initialize solution and companion measures: f(x), fgrad(x)\n[fx, fgradx] = getCostGrad(problem, x, storedb, key);\nnorm_grad = problem.M.norm(x, fgradx);\n\n% Initialize trust-region radius\nDelta = options.Delta0;\n\n% Save stats in a struct array info, and preallocate.\nif ~exist('used_cauchy', 'var')\n used_cauchy = [];\nend\nstats = savestats(problem, x, storedb, key, options, k, fx, norm_grad, Delta, ticstart);\ninfo(1) = stats;\ninfo(min(10000, options.maxiter+1)).iter = [];\n\n% ** Display:\nif options.verbosity == 2\n fprintf(['%3s %3s %5s %5s ',...\n 'f: %+e |grad|: %e\\n'],...\n ' ',' ',' ',' ', fx, norm_grad);\nelseif options.verbosity > 2\n fprintf('************************************************************************\\n');\n fprintf('%3s %3s k: %5s num_inner: %5s %s\\n',...\n '','','______','______','');\n fprintf(' f(x) : %+e |grad| : %e\\n', fx, norm_grad);\n fprintf(' Delta : %f\\n', Delta);\nend\n\n% To keep track of consecutive radius changes, so that we can warn the\n% user if it appears necessary.\nconsecutive_TRplus = 0;\nconsecutive_TRminus = 0;\n\n\n% **********************\n% ** Start of TR loop **\n% **********************\nwhile true\n \n\t% Start clock for this outer iteration\n ticstart = tic();\n\n % Run standard stopping criterion checks\n [stop, reason] = stoppingcriterion(problem, x, options, info, k+1);\n \n % If the stopping criterion that triggered is the tolerance on the\n % gradient norm but we are using randomization, make sure we make at\n % least miniter iterations to give randomization a chance at escaping\n % saddle points.\n if stop == 2 && options.useRand && k < options.miniter\n stop = 0;\n end\n \n if stop\n if options.verbosity >= 1\n fprintf([reason '\\n']);\n end\n break;\n end\n\n if options.verbosity > 2 || options.debug > 0\n fprintf('************************************************************************\\n');\n end\n\n % *************************\n % ** Begin TR Subproblem **\n % *************************\n \n % Determine eta0\n if ~options.useRand\n % Pick the zero vector\n eta = problem.M.zerovec(x);\n else\n % Random vector in T_x M (this has to be very small)\n eta = problem.M.lincomb(x, 1e-6, problem.M.randvec(x));\n % Must be inside trust-region\n while problem.M.norm(x, eta) > Delta\n eta = problem.M.lincomb(x, sqrt(sqrt(eps)), eta);\n end\n end\n\n % Solve TR subproblem approximately\n [eta, Heta, numit, stop_inner] = ...\n tCG(problem, x, fgradx, eta, Delta, options, storedb, key);\n srstr = tcg_stop_reason{stop_inner};\n\n % If using randomized approach, compare result with the Cauchy point.\n % Convergence proofs assume that we achieve at least (a fraction of)\n % the reduction of the Cauchy point. After this if-block, either all\n % eta-related quantities have been changed consistently, or none of\n % them have changed.\n if options.useRand\n used_cauchy = false;\n % Check the curvature,\n Hg = getHessian(problem, x, fgradx, storedb, key);\n g_Hg = problem.M.inner(x, fgradx, Hg);\n if g_Hg <= 0\n tau_c = 1;\n else\n tau_c = min( norm_grad^3/(Delta*g_Hg) , 1);\n end\n % and generate the Cauchy point.\n eta_c = problem.M.lincomb(x, -tau_c * Delta / norm_grad, fgradx);\n Heta_c = problem.M.lincomb(x, -tau_c * Delta / norm_grad, Hg);\n\n % Now that we have computed the Cauchy point in addition to the\n % returned eta, we might as well keep the best of them.\n mdle = fx + problem.M.inner(x, fgradx, eta) ...\n + .5*problem.M.inner(x, Heta, eta);\n mdlec = fx + problem.M.inner(x, fgradx, eta_c) ...\n + .5*problem.M.inner(x, Heta_c, eta_c);\n if mdlec < mdle\n eta = eta_c;\n Heta = Heta_c; % added April 11, 2012\n used_cauchy = true;\n end\n end\n \n \n % This is only computed for logging purposes, because it may be useful\n % for some user-defined stopping criteria. If this is not cheap for\n % specific applications (compared to evaluating the cost), we should\n % reconsider this.\n norm_eta = problem.M.norm(x, eta);\n \n if options.debug > 0\n testangle = problem.M.inner(x, eta, fgradx) / (norm_eta*norm_grad);\n end\n \n\n\t% Compute the tentative next iterate (the proposal)\n\tx_prop = problem.M.retr(x, eta);\n key_prop = storedb.getNewKey();\n\n\t% Compute the function value of the proposal\n\tfx_prop = getCost(problem, x_prop, storedb, key_prop);\n\n\t% Will we accept the proposal or not?\n % Check the performance of the quadratic model against the actual cost.\n rhonum = fx - fx_prop;\n rhoden = -problem.M.inner(x, fgradx, eta) ...\n -.5*problem.M.inner(x, eta, Heta);\n % rhonum could be anything.\n % rhoden should be nonnegative, as guaranteed by tCG, baring numerical\n % errors.\n \n % Heuristic -- added Dec. 2, 2013 (NB) to replace the former heuristic.\n % This heuristic is documented in the book by Conn Gould and Toint on\n % trust-region methods, section 17.4.2.\n % rhonum measures the difference between two numbers. Close to\n % convergence, these two numbers are very close to each other, so\n % that computing their difference is numerically challenging: there may\n % be a significant loss in accuracy. Since the acceptance or rejection\n % of the step is conditioned on the ratio between rhonum and rhoden,\n % large errors in rhonum result in a very large error in rho, hence in\n % erratic acceptance / rejection. Meanwhile, close to convergence,\n % steps are usually trustworthy and we should transition to a Newton-\n % like method, with rho=1 consistently. The heuristic thus shifts both\n % rhonum and rhoden by a small amount such that far from convergence,\n % the shift is irrelevant and close to convergence, the ratio rho goes\n % to 1, effectively promoting acceptance of the step.\n % The rationale is that close to convergence, both rhonum and rhoden\n % are quadratic in the distance between x and x_prop. Thus, when this\n % distance is on the order of sqrt(eps), the value of rhonum and rhoden\n % is on the order of eps, which is indistinguishable from the numerical\n % error, resulting in badly estimated rho's.\n % For abs(fx) < 1, this heuristic is invariant under offsets of f but\n % not under scaling of f. For abs(fx) > 1, the opposite holds. This\n % should not alarm us, as this heuristic only triggers at the very last\n % iterations if very fine convergence is demanded.\n rho_reg = max(1, abs(fx)) * eps * options.rho_regularization;\n rhonum = rhonum + rho_reg;\n rhoden = rhoden + rho_reg;\n \n if options.debug > 0\n fprintf('DBG: rhonum : %e\\n', rhonum);\n fprintf('DBG: rhoden : %e\\n', rhoden);\n end\n \n % This is always true if a linear, symmetric operator is used for the\n % Hessian (approximation) and if we had infinite numerical precision.\n % In practice, nonlinear approximations of the Hessian such as the\n % built-in finite difference approximation and finite numerical\n % accuracy can cause the model to increase. In such scenarios, we\n % decide to force a rejection of the step and a reduction of the\n % trust-region radius. We test the sign of the regularized rhoden since\n % the regularization is supposed to capture the accuracy to which\n % rhoden is computed: if rhoden were negative before regularization but\n % not after, that should not be (and is not) detected as a failure.\n % \n % Note (Feb. 17, 2015, NB): the most recent version of tCG already\n % includes a mechanism to ensure model decrease if the Cauchy step\n % attained a decrease (which is theoretically the case under very lax\n % assumptions). This being said, it is always possible that numerical\n % errors will prevent this, so that it is good to keep a safeguard.\n %\n % The current strategy is that, if this should happen, then we reject\n % the step and reduce the trust region radius. This also ensures that\n % the actual cost values are monotonically decreasing.\n model_decreased = (rhoden >= 0);\n \n if ~model_decreased \n srstr = [srstr ', model did not decrease']; %#ok\n end\n \n rho = rhonum / rhoden;\n \n % Added June 30, 2015 following observation by BM.\n % With this modification, it is guaranteed that a step rejection is\n % always accompanied by a TR reduction. This prevents stagnation in\n % this \"corner case\" (NaN's really aren't supposed to occur, but it's\n % nice if we can handle them nonetheless).\n if isnan(rho)\n fprintf('rho is NaN! Forcing a radius decrease. This should not happen.\\n');\n if isnan(fx_prop)\n fprintf('The cost function returned NaN (perhaps the retraction returned a bad point?)\\n');\n else\n fprintf('The cost function did not return a NaN value.');\n end\n end\n \n if options.debug > 0\n m = @(x, eta) ...\n getCost(problem, x, storedb, key) + ...\n getDirectionalDerivative(problem, x, eta, storedb, key) + ...\n .5*problem.M.inner(x, getHessian(problem, x, eta, storedb, key), eta);\n zerovec = problem.M.zerovec(x);\n actrho = (fx - fx_prop) / (m(x, zerovec) - m(x, eta));\n fprintf('DBG: new f(x) : %+e\\n', fx_prop);\n fprintf('DBG: actual rho : %e\\n', actrho);\n fprintf('DBG: used rho : %e\\n', rho);\n end\n\n % Choose the new TR radius based on the model performance\n trstr = ' ';\n % If the actual decrease is smaller than 1/4 of the predicted decrease,\n % then reduce the TR radius.\n if rho < 1/4 || ~model_decreased || isnan(rho)\n trstr = 'TR-';\n Delta = Delta/4;\n consecutive_TRplus = 0;\n consecutive_TRminus = consecutive_TRminus + 1;\n if consecutive_TRminus >= 5 && options.verbosity >= 2\n consecutive_TRminus = -inf;\n fprintf(' +++ Detected many consecutive TR- (radius decreases).\\n');\n fprintf(' +++ Consider decreasing options.Delta_bar by an order of magnitude.\\n');\n fprintf(' +++ Current values: options.Delta_bar = %g and options.Delta0 = %g.\\n', options.Delta_bar, options.Delta0);\n end\n % If the actual decrease is at least 3/4 of the precicted decrease and\n % the tCG (inner solve) hit the TR boundary, increase the TR radius.\n % We also keep track of the number of consecutive trust-region radius\n % increases. If there are many, this may indicate the need to adapt the\n % initial and maximum radii.\n elseif rho > 3/4 && (stop_inner == 1 || stop_inner == 2)\n trstr = 'TR+';\n Delta = min(2*Delta, options.Delta_bar);\n consecutive_TRminus = 0;\n consecutive_TRplus = consecutive_TRplus + 1;\n if consecutive_TRplus >= 5 && options.verbosity >= 1\n consecutive_TRplus = -inf;\n fprintf(' +++ Detected many consecutive TR+ (radius increases).\\n');\n fprintf(' +++ Consider increasing options.Delta_bar by an order of magnitude.\\n');\n fprintf(' +++ Current values: options.Delta_bar = %g and options.Delta0 = %g.\\n', options.Delta_bar, options.Delta0);\n end\n else\n % Otherwise, keep the TR radius constant.\n consecutive_TRplus = 0;\n consecutive_TRminus = 0;\n end\n\n % Choose to accept or reject the proposed step based on the model\n % performance. Note the strict inequality.\n if model_decreased && rho > options.rho_prime\n accept = true;\n accstr = 'acc';\n x = x_prop;\n key = key_prop;\n fx = fx_prop;\n fgradx = getGradient(problem, x, storedb, key);\n norm_grad = problem.M.norm(x, fgradx);\n else\n accept = false;\n accstr = 'REJ';\n end\n \n \n % Make sure we don't use too much memory for the store database\n storedb.purge();\n \n % k is the number of iterations we have accomplished.\n k = k + 1;\n\n % Log statistics for freshly executed iteration.\n % Everything after this in the loop is not accounted for in the timing.\n stats = savestats(problem, x, storedb, key, options, k, fx, ...\n norm_grad, Delta, ticstart, info, rho, rhonum, ...\n rhoden, accept, numit, norm_eta, used_cauchy);\n info(k+1) = stats; %#ok\n\n \n % ** Display:\n if options.verbosity == 2,\n fprintf(['%3s %3s k: %5d num_inner: %5d ', ...\n 'f: %+e |grad|: %e %s\\n'], ...\n accstr,trstr,k,numit,fx,norm_grad,srstr);\n elseif options.verbosity > 2,\n if options.useRand && used_cauchy,\n fprintf('USED CAUCHY POINT\\n');\n end\n\t\tfprintf('%3s %3s k: %5d num_inner: %5d %s\\n', ...\n\t\t\t\taccstr, trstr, k, numit, srstr);\n\t\tfprintf(' f(x) : %+e |grad| : %e\\n',fx,norm_grad);\n\t\tif options.debug > 0\n\t\t\tfprintf(' Delta : %f |eta| : %e\\n',Delta,norm_eta);\n\t\tend\n\t\tfprintf(' rho : %e\\n',rho);\n end\n if options.debug > 0,\n fprintf('DBG: cos ang(eta,gradf): %d\\n',testangle);\n if rho == 0\n fprintf('DBG: rho = 0, this will likely hinder further convergence.\\n');\n end\n end\n\nend % of TR loop (counter: k)\n\n% Restrict info struct-array to useful part\ninfo = info(1:k+1);\n\n\nif (options.verbosity > 2) || (options.debug > 0),\n fprintf('************************************************************************\\n');\nend\nif (options.verbosity > 0) || (options.debug > 0)\n fprintf('Total time is %f [s] (excludes statsfun)\\n', info(end).time);\nend\n\n% Return the best cost reached\ncost = fx;\n\nend\n\n\n\n \n\n% Routine in charge of collecting the current iteration stats\nfunction stats = savestats(problem, x, storedb, key, options, k, fx, ...\n norm_grad, Delta, ticstart, info, rho, rhonum, ...\n rhoden, accept, numit, norm_eta, used_cauchy)\n stats.iter = k;\n stats.cost = fx;\n stats.gradnorm = norm_grad;\n stats.Delta = Delta;\n if k == 0\n stats.time = toc(ticstart);\n stats.rho = inf;\n stats.rhonum = NaN;\n stats.rhoden = NaN;\n stats.accepted = true;\n stats.numinner = NaN;\n stats.stepsize = NaN;\n if options.useRand\n stats.cauchy = false;\n end\n else\n stats.time = info(k).time + toc(ticstart);\n stats.rho = rho;\n stats.rhonum = rhonum;\n stats.rhoden = rhoden;\n stats.accepted = accept;\n stats.numinner = numit;\n stats.stepsize = norm_eta;\n if options.useRand,\n stats.cauchy = used_cauchy;\n end\n end\n \n % See comment about statsfun above: the x and store passed to statsfun\n % are that of the most recently accepted point after the iteration\n % fully executed.\n stats = applyStatsfun(problem, x, storedb, key, options, stats);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/checkdiff.m", "content": "function checkdiff(problem, x, d, force_gradient)\n% Checks the consistency of the cost function and directional derivatives.\n%\n% function checkdiff(problem)\n% function checkdiff(problem, x)\n% function checkdiff(problem, x, d)\n%\n% checkdiff performs a numerical test to check that the directional\n% derivatives defined in the problem structure agree up to first order with\n% the cost function at some point x, along some direction d. The test is\n% based on a truncated Taylor series (see online Manopt documentation).\n%\n% Both x and d are optional and will be sampled at random if omitted.\n%\n% See also: checkgradient checkhessian\n\n% If force_gradient = true (hidden parameter), then the function will call\n% getGradient and infer the directional derivative, rather than call\n% getDirectionalDerivative directly. This is used by checkgradient.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% March 26, 2017 (JB):\n% Detects if the approximated linear model is exact\n% and provides the user with the corresponding feedback.\n% \n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n\n if ~exist('force_gradient', 'var')\n force_gradient = false;\n end\n \n % Verify that the problem description is sufficient.\n if ~canGetCost(problem)\n error('It seems no cost was provided.');\n end\n if ~force_gradient && ~canGetDirectionalDerivative(problem)\n error('It seems no directional derivatives were provided.');\n end\n if force_gradient && ~canGetGradient(problem)\n % Would normally issue a warning, but this function should only be\n % called with force_gradient on by checkgradient, which will\n % already have issued a warning.\n end\n \n x_isprovided = exist('x', 'var') && ~isempty(x);\n d_isprovided = exist('d', 'var') && ~isempty(d);\n \n if ~x_isprovided && d_isprovided\n error('If d is provided, x must be too, since d is tangent at x.');\n end\n \n % If x and / or d are not specified, pick them at random.\n if ~x_isprovided\n x = problem.M.rand();\n end\n if ~d_isprovided\n d = problem.M.randvec(x);\n end\n\n % Compute the value f0 at f and directional derivative at x along d.\n storedb = StoreDB();\n xkey = storedb.getNewKey();\n f0 = getCost(problem, x, storedb, xkey);\n \n if ~force_gradient\n df0 = getDirectionalDerivative(problem, x, d, storedb, xkey);\n else\n grad = getGradient(problem, x, storedb, xkey);\n df0 = problem.M.inner(x, grad, d);\n end\n \n % Compute the value of f at points on the geodesic (or approximation\n % of it) originating from x, along direction d, for stepsizes in a\n % large range given by h.\n h = logspace(-8, 0, 51);\n value = zeros(size(h));\n for i = 1 : length(h)\n y = problem.M.exp(x, d, h(i));\n ykey = storedb.getNewKey();\n value(i) = getCost(problem, y, storedb, ykey);\n end\n \n % Compute the linear approximation of the cost function using f0 and\n % df0 at the same points.\n model = polyval([df0 f0], h);\n \n % Compute the approximation error\n err = abs(model - value);\n \n % And plot it.\n loglog(h, err);\n title(sprintf(['Directional derivative check.\\nThe slope of the '...\n 'continuous line should match that of the dashed\\n'...\n '(reference) line over at least a few orders of '...\n 'magnitude for h.']));\n xlabel('h');\n ylabel('Approximation error');\n \n line('xdata', [1e-8 1e0], 'ydata', [1e-8 1e8], ...\n 'color', 'k', 'LineStyle', '--', ...\n 'YLimInclude', 'off', 'XLimInclude', 'off');\n \n \n if ~all( err < 1e-12 )\n % In a numerically reasonable neighborhood, the error should\n % decrease as the square of the stepsize, i.e., in loglog scale,\n % the error should have a slope of 2.\n isModelExact = false;\n window_len = 10;\n [range, poly] = identify_linear_piece(log10(h), log10(err), window_len);\n else\n % The 1st order model is exact: all errors are (numerically) zero\n % Fit line from all points, use log scale only in h.\n isModelExact = true;\n range = 1:numel(h);\n poly = polyfit(log10(h), err, 1);\n % Set mean error in log scale for plot.\n poly(end) = log10(poly(end));\n % Change title to something more descriptive for this special case.\n title(sprintf(...\n ['Directional derivative check.\\n'...\n 'It seems the linear model is exact:\\n'...\n 'Model error is numerically zero for all h.']));\n end\n hold all;\n loglog(h(range), 10.^polyval(poly, log10(h(range))), 'LineWidth', 3);\n hold off;\n \n if ~isModelExact\n fprintf('The slope should be 2. It appears to be: %g.\\n', poly(1));\n fprintf(['If it is far from 2, then directional derivatives ' ...\n 'might be erroneous.\\n']);\n else\n fprintf(['The linear model appears to be exact ' ...\n '(within numerical precision),\\n'...\n 'hence the slope computation is irrelevant.\\n']);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/checkgradient.m", "content": "function checkgradient(problem, x, d)\n% Checks the consistency of the cost function and the gradient.\n%\n% function checkgradient(problem)\n% function checkgradient(problem, x)\n% function checkgradient(problem, x, d)\n%\n% checkgradient performs a numerical test to check that the gradient\n% defined in the problem structure agrees up to first order with the cost\n% function at some point x, along some direction d. The test is based on a\n% truncated Taylor series (see online Manopt documentation).\n%\n% It is also tested that the gradient is indeed a tangent vector.\n% \n% Both x and d are optional and will be sampled at random if omitted.\n%\n% See also: checkdiff checkhessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% Nov. 1, 2016 (NB):\n% Now calls checkdiff with force_gradient = true, instead of doing an\n% rmfield of problem.diff. This became necessary after getGradient\n% was updated to know how to compute the gradient from directional\n% derivatives.\n\n \n % Verify that the problem description is sufficient.\n if ~canGetCost(problem)\n % The call to canGetPartialGradient will readily issue a warning if\n % problem.ncostterms is not defined even though it is expected.\n if ~canGetPartialGradient(problem)\n error('getCost:checkgradient', 'It seems no cost was provided.');\n else\n error('getCost:stochastic', ['It seems no cost was provided.\\n' ...\n 'If you intend to use a stochastic solver, you still\\n' ...\n 'need to define problem.cost to use checkgradient.']);\n end\n end\n if ~canGetGradient(problem)\n warning('manopt:checkgradient:nograd', ...\n 'It seems no gradient was provided.');\n end\n \n x_isprovided = exist('x', 'var') && ~isempty(x);\n d_isprovided = exist('d', 'var') && ~isempty(d);\n \n if ~x_isprovided && d_isprovided\n error('If d is provided, x must be too, since d is tangent at x.');\n end\n \n % If x and / or d are not specified, pick them at random.\n if ~x_isprovided\n x = problem.M.rand();\n end\n if ~d_isprovided\n d = problem.M.randvec(x);\n end\n\n %% Check that the gradient yields a first order model of the cost.\n \n % Call checkdiff with force_gradient set to true, to force that\n % function to make a gradient call.\n checkdiff(problem, x, d, true);\n title(sprintf(['Gradient check.\\nThe slope of the continuous line ' ...\n 'should match that of the dashed\\n(reference) line ' ...\n 'over at least a few orders of magnitude for h.']));\n xlabel('h');\n ylabel('Approximation error');\n \n %% Try to check that the gradient is a tangent vector.\n if isfield(problem.M, 'tangent')\n storedb = StoreDB();\n key = storedb.getNewKey();\n grad = getGradient(problem, x, storedb, key);\n pgrad = problem.M.tangent(x, grad);\n residual = problem.M.lincomb(x, 1, grad, -1, pgrad);\n err = problem.M.norm(x, residual);\n fprintf('The residual should be 0, or very close. Residual: %g.\\n', err);\n fprintf('If it is far from 0, then the gradient is not in the tangent space.\\n');\n else\n fprintf(['Unfortunately, Manopt was unable to verify that the '...\n 'gradient is indeed a tangent vector.\\nPlease verify ' ...\n 'this manually or implement the ''tangent'' function ' ...\n 'in your manifold structure.']);\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/checkhessian.m", "content": "function checkhessian(problem, x, d)\n% Checks the consistency of the cost function and the Hessian.\n%\n% function checkhessian(problem)\n% function checkhessian(problem, x)\n% function checkhessian(problem, x, d)\n%\n% checkhessian performs a numerical test to check that the directional\n% derivatives and Hessian defined in the problem structure agree up to\n% second order with the cost function at some point x, along some direction\n% d. The test is based on a truncated Taylor series (see online Manopt\n% documentation).\n% \n% It is also tested that the result of applying the Hessian along that\n% direction is indeed a tangent vector, and that the Hessian operator is\n% symmetric w.r.t. the Riemannian metric.\n% \n% Both x and d are optional and will be sampled at random if omitted.\n%\n% See also: checkdiff checkgradient checkretraction\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n%\n% March 26, 2017 (JB):\n% Detects if the approximated quadratic model is exact\n% and provides the user with the corresponding feedback.\n% \n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% Nov. 1, 2016 (NB):\n% Issues a call to getGradient rather than getDirectionalDerivative.\n\n \n % Verify that the problem description is sufficient.\n if ~canGetCost(problem)\n error('It seems no cost was provided.');\n end\n if ~canGetGradient(problem)\n warning('manopt:checkhessian:nograd', ...\n 'It seems no gradient was provided.');\n end\n if ~canGetHessian(problem)\n warning('manopt:checkhessian:nohess', ...\n 'It seems no Hessian was provided.');\n end\n \n x_isprovided = exist('x', 'var') && ~isempty(x);\n d_isprovided = exist('d', 'var') && ~isempty(d);\n \n if ~x_isprovided && d_isprovided\n error('If d is provided, x must be too, since d is tangent at x.');\n end\n \n % If x and / or d are not specified, pick them at random.\n if ~x_isprovided\n x = problem.M.rand();\n end\n if ~d_isprovided\n d = problem.M.randvec(x);\n end\n \n %% Check that the directional derivative and the Hessian at x along d\n %% yield a second order model of the cost function.\n \n % Compute the value f0 at f, directional derivative df0 at x along d,\n % and Hessian along [d, d].\n storedb = StoreDB();\n xkey = storedb.getNewKey();\n f0 = getCost(problem, x, storedb, xkey);\n df0 = problem.M.inner(x, d, getGradient(problem, x, storedb, xkey));\n d2f0 = problem.M.inner(x, d, getHessian(problem, x, d, storedb, xkey));\n \n % Compute the value of f at points on the geodesic (or approximation\n % of it) originating from x, along direction d, for stepsizes in a\n % large range given by h.\n h = logspace(-8, 0, 51);\n value = zeros(size(h));\n for i = 1 : length(h)\n y = problem.M.exp(x, d, h(i));\n ykey = storedb.getNewKey();\n value(i) = getCost(problem, y, storedb, ykey);\n end\n \n % Compute the quadratic approximation of the cost function using f0,\n % df0 and d2f0 at the same points.\n model = polyval([.5*d2f0 df0 f0], h);\n \n % Compute the approximation error\n err = abs(model - value);\n \n % And plot it.\n loglog(h, err);\n title(sprintf(['Hessian check.\\nThe slope of the continuous line ' ...\n 'should match that of the dashed\\n(reference) line ' ...\n 'over at least a few orders of magnitude for h.']));\n xlabel('h');\n ylabel('Approximation error');\n \n line('xdata', [1e-8 1e0], 'ydata', [1e-16 1e8], ...\n 'color', 'k', 'LineStyle', '--', ...\n 'YLimInclude', 'off', 'XLimInclude', 'off');\n \n \n if ~all( err < 1e-12 )\n % In a numerically reasonable neighborhood, the error should\n % decrease as the cube of the stepsize, i.e., in loglog scale, the\n % error should have a slope of 3.\n isModelExact = false;\n window_len = 10;\n [range, poly] = identify_linear_piece(log10(h), log10(err), window_len);\n else\n % The 2nd order model is exact: all errors are (numerically) zero\n % Fit line from all points, use log scale only in h.\n isModelExact = true;\n range = 1:numel(h);\n poly = polyfit(log10(h), err, 1);\n % Set mean error in log scale for plot\n poly(end) = log10(poly(end));\n % Change title to something more descriptive for this special case.\n title(sprintf(...\n ['Hessian check.\\n'...\n 'It seems the quadratic model is exact:\\n'...\n 'Model error is numerically zero for all h.']));\n end\n hold all;\n loglog(h(range), 10.^polyval(poly, log10(h(range))), 'LineWidth', 3);\n hold off;\n \n if ~isModelExact\n fprintf('The slope should be 3. It appears to be: %g.\\n', poly(1));\n fprintf(['If it is far from 3, then directional derivatives or ' ...\n 'the Hessian might be erroneous.\\n']);\n fprintf(['Note: if the exponential map is only approximate, and it '...\n 'is not a second-order approximation,\\nthen it is normal ' ...\n 'for the slope test to reach 2 instead of 3. Check the ' ...\n 'factory for this.\\n' ...\n 'If tested at a critical point, then even for a first-order '...\n 'retraction the slope test should yield 3.\\n']);\n else\n fprintf(['The quadratic model appears to be exact ' ...\n '(within numerical precision),\\n'...\n 'hence the slope computation is irrelevant.\\n']);\n end\n\n \n %% Check that the Hessian at x along direction d is a tangent vector.\n if isfield(problem.M, 'tangent')\n hess = getHessian(problem, x, d, storedb, xkey);\n phess = problem.M.tangent(x, hess);\n residual = problem.M.lincomb(x, 1, hess, -1, phess);\n err = problem.M.norm(x, residual);\n fprintf('The residual should be zero, or very close. ');\n fprintf('Residual: %g.\\n', err);\n fprintf(['If it is far from 0, then the Hessian is not in the ' ...\n 'tangent plane.\\n']);\n else\n fprintf(['Unfortunately, Manopt was unable to verify that the '...\n 'Hessian is indeed a tangent vector.\\nPlease verify ' ...\n 'this manually.']);\n end \n \n %% Check that the Hessian at x is symmetric.\n d1 = problem.M.randvec(x);\n d2 = problem.M.randvec(x);\n h1 = getHessian(problem, x, d1, storedb, xkey);\n h2 = getHessian(problem, x, d2, storedb, xkey);\n v1 = problem.M.inner(x, d1, h2);\n v2 = problem.M.inner(x, h1, d2);\n value = v1-v2;\n fprintf([' - should be zero, or very close.' ...\n '\\n\\tValue: %g - %g = %g.\\n'], v1, v2, value);\n fprintf('If it is far from 0, then the Hessian is not symmetric.\\n');\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/checkretraction.m", "content": "function checkretraction(M, x, v)\n% Check the order of agreement of a retraction with an exponential.\n% \n% function checkretraction(M)\n% function checkretraction(M, x)\n% function checkretraction(M, x, v)\n%\n% checkretraction performs a numerical test to check the order of agreement\n% between the retraction and the exponential map in a given Manopt\n% manifold structure M. The test is performed at the point x if it is\n% provided (otherwise, the point is picked at random) and along the tangent\n% vector v at x if one is provided (otherwise, a tangent vector at x is\n% picked at random.)\n%\n% See also: checkdiff checkgradient checkhessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Oct. 21, 2016.\n% Contributors: \n% Change log: \n\n if ~exist('x', 'var') || isempty(x)\n x = M.rand();\n v = M.randvec(x);\n end\n \n if ~exist('v', 'var') || isempty(v)\n v = M.randvec(x);\n end\n \n % Compare the retraction and the exponential over steps of varying\n % length, on a wide log-scale.\n tt = logspace(-12, 0, 251);\n ee = zeros(size(tt));\n for k = 1 : numel(tt)\n t = tt(k);\n ee(k) = M.dist(M.exp(x, v, t), M.retr(x, v, t));\n end\n \n % Plot the difference between the exponential and the retration over\n % that span of steps, in log-log scale.\n loglog(tt, ee);\n \n % We hope to see a slope of 3, to confirm a second-order retraction. If\n % the slope is only 2, we have a first-order retration. If the slope is\n % less than 2, this is not a retraction.\n % Slope 3\n line('xdata', [1e-12 1e0], 'ydata', [1e-30 1e6], ...\n 'color', 'k', 'LineStyle', '--', ...\n 'YLimInclude', 'off', 'XLimInclude', 'off');\n % Slope 2\n line('xdata', [1e-14 1e0], 'ydata', [1e-20 1e8], ...\n 'color', 'k', 'LineStyle', ':', ...\n 'YLimInclude', 'off', 'XLimInclude', 'off');\n \n\n % Figure out the slope of the error in log-log, by identifying a piece\n % of the error curve which is mostly linear.\n window_len = 10;\n [range, poly] = identify_linear_piece(log10(tt), log10(ee), window_len);\n hold all;\n loglog(tt(range), 10.^polyval(poly, log10(tt(range))), 'LineWidth', 3);\n hold off;\n \n xlabel('Step size multiplier t');\n ylabel('Distance between Exp(x, v, t) and Retr(x, v, t)');\n title(sprintf('Retraction check.\\nA slope of 2 is required, 3 is desired.'));\n \n fprintf('Check agreement between M.exp and M.retr. Please check the\\n');\n fprintf('factory file of M to ensure M.exp is a proper exponential.\\n');\n fprintf('The slope must be at least 2 to have a proper retraction.\\n');\n fprintf('For the retraction to be second order, the slope should be 3.\\n');\n fprintf('It appears the slope is: %g.\\n', poly(1));\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/criticalpointfinder.m", "content": "function problem_critpt = criticalpointfinder(problem)\n% Creates a Manopt problem whose optima are the critical points of another.\n%\n% problem_critpt = criticalpointfinder(problem)\n%\n% Given a Manopt problem structure 'problem', this tool returns a new\n% problem structure, 'problem_critpt', such that the global optima of the\n% new problem coincide with the critical points of the original problem.\n% This can be useful notably in empirical studies of the properties of\n% saddle points of a problem.\n%\n% Concretely, if f is the cost function of the given problem, grad f\n% denotes its (Riemannian) gradient and Hess f denotes its (Riemannian)\n% Hessian, then the new problem has a cost function g defined by:\n%\n% g(x) = (1/2)*norm(grad f(x))^2,\n%\n% where x is a point on the manifold problem.M (the new problem lives on\n% the same manifold), and norm(.) = problem.M.norm(x, .) is the Riemannian\n% norm on the tangent space at x. The Riemannian gradient of g is elegantly\n% obtained from knowledge of f:\n%\n% grad g(x) = Hess f(x)[grad f(x)]\n%\n% If the Hessian of f is not available in the given problem, Manopt will\n% approximate it automatically to compute an approximate gradient of g.\n% If the Hessian of f is available, then an approximate Hessian of g is\n% defined in the returned problem as\n%\n% approxhess g(x)[u] = Hess f(x)[ Hess f(x)[u] ].\n%\n% This approximation is exact if x is a critical point of f, which is\n% enough to ensure superlinear local convergence to critical points of f\n% using the trustregions algorithm, for example.\n%\n% Once problem_critpt is obtained, it can be passed to any of the solvers\n% of Manopt to compute critical points of the original problem. Supplying\n% an initial point to the solver allows to aim for a critical point in a\n% specific neighborhood of the search space.\n%\n%\n% Usage example:\n% \n% The code below creates a problem whose optima are dominant eigenvectors\n% of a matrix A and whose critical points are any eigenvectors of A, then\n% compute critical points using the present tool:\n%\n% n = 100; A = randn(n); A = .5*(A+A');\n% problem.M = spherefactory(n);\n% problem.cost = @(x) -x'*(A*x);\n% problem.egrad = @(x) -2*A*x;\n% problem.ehess = @(x, xdot) -2*A*xdot;\n% problem_critpt = criticalpointfinder(problem);\n% opts.tolcost = .5*(1e-5)^2; % aim for a gradient smaller than 1e-5\n% [x, fx] = trustregions(problem_critpt, [], opts); % random initial guess\n% fprintf('Norm of the gradient at x: %g\\n', sqrt(2*fx));\n% fprintf('This is small if x is close to being an eigenvector: %g\\n',...\n% norm((x'*A*x)*x - A*x));\n% % The two displayed numbers are equal up to a factor 2.\n%\n%\n% See also: trustregions\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Jan. 25, 2017.\n% Contributors: \n% Change log: \n\n% TODO: Determine a safe way of using the caching functionalities of Manopt\n% with this tool. The issue in passing along storedb and key in the\n% costgrad and approxhess functions is that the storedb will be\n% associated to problem_critpt, not to problem. This may cause bugs\n% that would be very difficult to catch. To be on the safe side,\n% caching is not used at all here, but this may cause running times\n% to be longer than necessary. To create a local storedb associated\n% to problem and to only use the key seems to also not be a viable\n% solution, since there is no clear way of resetting it to zero\n% everytime a solver is called on problem_critpt.\n% -- Jan. 26, 2017 (NB)\n\n problem_critpt.M = problem.M;\n problem_critpt.costgrad = @costgrad;\n \n % If the Hessian is available for the problem, we build an approximate\n % Hessian based on it. Otherwise, there is no reason to believe that\n % this approximate Hessian would be better than the standard\n % approximate Hessian created by Manopt.\n if canGetHessian(problem)\n problem_critpt.approxhess = @approxhess;\n end\n \n function [g, gradg] = costgrad(x)\n \n gradf = getGradient(problem, x);\n Hessf_gradf = getHessian(problem, x, gradf);\n \n g = .5*problem.M.norm(x, gradf)^2;\n gradg = Hessf_gradf;\n \n end\n \n % This is not quite the Hessian because there should be a third-order\n % derivative term (which is inaccessible), but: at critical points\n % (where grad f(x) = 0 for the f of problem.cost) this Hessian is\n % exact, so it will allow for superlinear local convergence in\n % algorithms such as trustregions.\n function HHu = approxhess(x, u)\n \n Hu = getHessian(problem, x, u);\n HHu = getHessian(problem, x, Hu);\n \n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/dexpm.m", "content": "function D = dexpm(X, H)\n% Frchet derivative of the matrix exponential.\n%\n% function D = dexpm(X, H)\n%\n% Computes the directional derivative (the Frchet derivative) of expm at X\n% along H (square matrices).\n%\n% Thus, D = lim_(t -> 0) (expm(X + tH) - expm(X)) / t.\n%\n% Note: the adjoint of dexpm(X, .) is dexpm(X', .), which is a fact often\n% useful to derive gradients of matrix functions involving expm(X).\n% (This is wrt the inner product inner = @(A, B) real(trace(A'*B))).\n% \n% See also: dfunm dlogm dsqrtm\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 3, 2015.\n% Contributors:\n% Change log:\n \n D = dfunm(@expm, X, H);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/dfunm.m", "content": "function D = dfunm(funm, X, H)\n% Frchet derivative of matrix functions.\n%\n% function D = dfunm(funm, X, H)\n%\n% Computes the directional derivative (the Frchet derivative) of a matrix\n% function (such as @logm, @expm, ...) at X along H (square matrices),\n% according to a very nice trick which appears in this paper:\n% \n% \"Computing the Frchet derivative of the matrix exponential, with an\n% application to condition number estimation\",\n% Awad H. Al-Mohy and Nicholas J. Higham, 2009.\n% http://eprints.ma.man.ac.uk/1218/01/covered/MIMS_ep2008_26.pdf\n%\n% Thus, D = lim_(t -> 0) (funm(X + tH) - funm(X)) / t.\n%\n% This code is simple, but may not be the most efficient. In particular, it\n% requires computing the matrix function on matrices which are four times\n% as big, and which may have lost important structure (such as symmetry).\n% \n% See also: dlogm dexpm dsqrtm\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 3, 2015.\n% Contributors:\n% Change log:\n \n n = size(X, 1);\n \n assert(length(size(X)) == 2, 'X and H must be square matrices.');\n assert(length(size(H)) == 2, 'X and H must be square matrices.');\n assert(size(X, 1) == size(X, 2), 'X and H must be square matrices.');\n assert(all(size(X) == size(H)), 'X and H must have the same size.');\n \n Z = zeros(n);\n A = funm([X, H ; Z, X]);\n D = A(1:n, (n+1):end);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/diagsum.m", "content": "function [tracedtensor] = diagsum(tensor1, d1, d2)\n% C = DIAGSUM(A, d1, d2) Performs the trace\n% C(i[1],...,i[d1-1],i[d1+1],...,i[d2-1],i[d2+1],...i[n]) =\n% A(i[1],...,i[d1-1],k,i[d1+1],...,i[d2-1],k,i[d2+1],...,i[n])\n% (Sum on k).\n%\n% C = DIAGSUM(A, d1, d2) traces A along the diagonal formed by dimensions d1\n% and d2. If the lengths of these dimensions are not equal, DIAGSUM traces\n% until the end of the shortest of dimensions d1 and d2 is reached. This is\n% an analogue of the built in TRACE function.\n%\n% Wynton Moore, January 2006\n\n\ndim1=size(tensor1);\nnumdims=length(dim1);\n\n\n%check inputs\nif d1==d2\n tracedtensor=squeeze(sum(tensor1,d1));\nelseif numdims==2\n tracedtensor=trace(tensor1);\nelseif dim1(d1)==1 && dim1(d2)==1\n tracedtensor=squeeze(tensor1);\nelse\n\n\n %determine correct permutation\n swapd1=d1;swapd2=d2;\n \n if d1~=numdims-1 && d1~=numdims && d2~=numdims-1\n swapd1=numdims-1;\n elseif d1~=numdims-1 && d1~=numdims && d2~=numdims\n swapd1=numdims;\n end\n if d2~=numdims-1 && d2~=numdims && swapd1~=numdims-1\n swapd2=numdims-1;\n elseif d2~=numdims-1 && d2~=numdims && swapd1~=numdims\n swapd2=numdims;\n end\n \n \n %prepare for construction of selector tensor\n temp1=eye(numdims);\n permmatrix=temp1;\n permmatrix(:,d1)=temp1(:,swapd1);\n permmatrix(:,swapd1)=temp1(:,d1);\n permmatrix(:,d2)=temp1(:,swapd2);\n permmatrix(:,swapd2)=temp1(:,d2);\n\n selectordim=dim1*permmatrix;\n permvector=(1:numdims)*permmatrix;\n\n\n %construct selector tensor\n if numdims>3\n selector = ipermute(outer(ones(selectordim(1:numdims-2)), ...\n eye(selectordim(numdims-1), ...\n selectordim(numdims)), ...\n 0), ...\n permvector);\n else\n %when numdims=3, the above line gives ndims(selector)=4. This\n %routine avoids that error. When used with GMDMP, numdims will be\n %at least 4, so this routine will be unnecessary.\n selector2=eye(selectordim(numdims-1), selectordim(numdims));\n selector=zeros(selectordim);\n for j=1:selectordim(1)\n selector(j, :, :)=selector2;\n end\n selector=ipermute(selector, permvector);\n end\n \n \n %perform trace, discard resulting singleton dimensions\n tracedtensor=sum(sum(tensor1.*selector, d1), d2);\n tracedtensor=squeeze(tracedtensor);\n\t\n \nend\n\n\n%correction for abberation in squeeze function:\n%size(squeeze(rand(1,1,2)))=[2 1]\nnontracedimensions=dim1;\nnontracedimensions(d1)=[];\nif d2>d1\n nontracedimensions(d2-1)=[];\nelse\n nontracedimensions(d2)=[];\nend\ntracedsize=size(tracedtensor);\n% Next line modified, Nicolas Boumal, April 30, 2012, such that\n% diagsum(A, 1, 2) would compute the trace of A, a 2D matrix.\nif length(tracedsize)==2 && tracedsize(2)==1 && ...\n (isempty(nontracedimensions) || tracedsize(1)~=nontracedimensions(1))\n\n tracedtensor=tracedtensor.';\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/dlogm.m", "content": "function D = dlogm(X, H)\n% Frchet derivative of the matrix logarithm.\n%\n% function D = dlogm(X, H)\n%\n% Computes the directional derivative (the Frchet derivative) of logm at X\n% along H (square matrices).\n%\n% Thus, D = lim_(t -> 0) (logm(X + tH) - logm(X)) / t.\n% \n% See also: dfunm dexpm dsqrtm\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 3, 2015.\n% Contributors:\n% Change log:\n \n D = dfunm(@logm, X, H);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/dsqrtm.m", "content": "function D = dsqrtm(X, H)\n% Frchet derivative of the matrix square root.\n%\n% function D = dsqrtm(X, H)\n%\n% Computes the directional derivative (the Frchet derivative) of sqrtm at\n% X along H (square matrices).\n%\n% Thus, D = lim_(t -> 0) (sqrtm(X + tH) - sqrtm(X)) / t.\n% \n% See also: dfunm dlogm dexpm\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 3, 2015.\n% Contributors:\n% Change log:\n \n D = dfunm(@sqrtm, X, H);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/grammatrix.m", "content": "function G = grammatrix(M, x, vectors)\n% Computes the Gram matrix of tangent vectors in the Manopt framework.\n%\n% function G = grammatrix(M, x, vectors)\n%\n% M is a Manopt manifold structure obtained from a factory.\n% x is a point on the manifold M.\n% vectors is a cell containing n tangent vectors at x.\n%\n% G is an n-by-n symmetric positive semidefinite matrix such that G(i, j)\n% is the inner product between vectors{i} and vectors{j}, with respect to\n% the metric on the tangent space to M at x.\n%\n% See also: orthogonalize tangentorthobasis\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 28, 2016.\n% Contributors: \n% Change log: \n\n\n n = numel(vectors);\n \n G = zeros(n);\n \n for i = 1 : n\n \n vi = vectors{i};\n \n G(i, i) = M.inner(x, vi, vi);\n \n for j = (i+1) : n\n \n vj = vectors{j};\n G(i, j) = M.inner(x, vi, vj);\n\t\t\t\n\t\t\t% Manopt is designed to work with real inner products,\n\t\t\t% but it does not hurt to allow for complex inner products\n\t\t\t% here by taking the conjugate.\n G(j, i) = G(i, j)';\n \n end\n \n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/hashmd5.m", "content": "function h = hashmd5(inp)\n% Computes the MD5 hash of input data.\n%\n% function h = hashmd5(inp)\n% \n% Returns a string containing the MD5 hash of the input variable. The input\n% variable may be of any class that can be typecast to uint8 format, which\n% is fairly non-restrictive.\n\n% This file is part of Manopt: www.manopt.org.\n% This code is a stripped version of more general hashing code by\n% Michael Kleder, Nov 2005.\n% Change log: \n% \n% Aug. 8, 2013 (NB):\n% Made x a static (persistent) variable, in the hope it will speed\n% it up. Furthermore, the function is now Octave compatible.\n\n is_octave = exist('OCTAVE_VERSION', 'builtin');\n \n persistent x;\n if isempty(x) && ~is_octave\n x = java.security.MessageDigest.getInstance('MD5');\n end\n\n inp=inp(:);\n % Convert strings and logicals into uint8 format\n if ischar(inp) || islogical(inp)\n inp=uint8(inp);\n else % Convert everything else into uint8 format without loss of data\n inp=typecast(inp,'uint8');\n end\n \n % Create hash\n if ~is_octave\n x.update(inp);\n h = typecast(x.digest, 'uint8');\n h = dec2hex(h)';\n % Remote possibility: all hash bytes < 128, so pad:\n if(size(h,1))==1\n h = [repmat('0',[1 size(h,2)]);h];\n end\n h = lower(h(:)');\n else\n h = md5sum(char(inp'), true);\n end\n\t\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/hessianextreme.m", "content": "function [y, lambda, info] = hessianextreme(problem, x, side, y0, options, storedb, key)\n% Compute an extreme eigenvector / eigenvalue of the Hessian of a problem.\n%\n% [u, lambda, info] = hessianextreme(problem, x)\n% [u, lambda, info] = hessianextreme(problem, x, side)\n% [u, lambda, info] = hessianextreme(problem, x, side, u0)\n% [u, lambda, info] = hessianextreme(problem, x, side, u0, options)\n% [u, lambda, info] = hessianextreme(problem, x, side, u0, options, storedb)\n% [u, lambda, info] = hessianextreme(problem, x, side, u0, options, storedb, key)\n% \n% (For side, u0 and options, pass [] to omit any.)\n%\n% Given a Manopt problem structure and a point x on the manifold problem.M,\n% this function computes a tangent vector u at x of unit norm such that the\n% Hessian quadratic form is minimized or maximized:\n%\n% minimize or maximize such that = 1,\n%\n% where <.,.> is the Riemannian metric on the tangent space at x. Choose\n% between minimizing and maximizing by setting side = 'min' or 'max', with\n% 'min' being the default. The value attained is returned as lambda, and\n% is the minimal or maximal eigenvalue of the Hessian (actually, the last\n% value attained when the solver stopped). This is a real number since the\n% Hessian is a symmetric operator.\n%\n% If u0 is specified, it should be a unit-norm tangent vector at x. It is\n% then used as initial guess to solve the above problem. Pass [] to omit.\n%\n% The options structure, if provided, will be passed along to manoptsolve.\n% As such, you may choose which solver to use to solve the above\n% optimization problem by setting options.solver. See manoptsolve's help.\n% The other options will be passed along to the chosen solver too.\n% Pass [] to omit.\n%\n% Often times, it is only necessary to compute a vector u such that the\n% quadratic form is negative, if that is at all possible. To do so, set the\n% following stopping criterion: options.tolcost = -1e-10; (for example)\n% and side = 'min'. The solver will return as soon as the quadratic cost\n% defined above drops below the set value (or sooner if another stopping\n% criterion triggers first.)\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% info is the info struct-array returned by the solver.\n%\n% See also: hessianspectrum manoptsolve tangentspherefactory\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Aug. 13, 2014.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% May 7, 2015 (NB):\n% Default solver options: verbosity = 0 and defaults to trustregions.\n%\n% Nov 27, 2015 (NB):\n% The function now also returns the info struct-array.\n\n \n % By default, minimize\n if ~exist('side', 'var') || isempty(side)\n side = 'min';\n end\n \n % If no initial guess was specified, prepare the empty one.\n if ~exist('y0', 'var')\n y0 = [];\n end\n\n % Merge default solver options with potential user-specified options.\n % Set local defaults here\n localdefaults.verbosity = 0;\n localdefaults.solver = @trustregions;\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n \n % Convert the side into a sign.\n % Since Manopt minimizes, 'min' asks for no sign change.\n switch lower(side)\n case 'min'\n sign = +1;\n case 'max'\n sign = -1;\n otherwise\n error('The side should be either ''min'' or ''max''.');\n end\n\n % We define a manifold that is actually the unit sphere on the tangent\n % space to problem.M at x. A generalization would be to consider\n % Stiefel or Grassmann on the tangent space, but this would require\n % manipulating collections of tangent vectors, which in full generality\n % may be more complex (from a programming point of view).\n % Points are represented as tangent vectors of unit norm.\n % Tangent vectors are represented as tangent vectors orthogonal to the\n % root point, with respect to the Riemannian metric on the tangent\n % space.\n \n % M is the original manifold. x is a point on M.\n M = problem.M;\n \n % N is the manifold we build. y will be a point on N, thus also a\n % tangent vector to M at x. This is a typical Riemannian submanifold of\n % a Euclidean space, hence it is easy to describe in terms of the tools\n % available for M.\n N = tangentspherefactory(M, x);\n \n % It is usually a good idea to force a gradient computation to make\n % sure precomputable things are precomputed.\n if canGetGradient(problem)\n [unused1, unused2] = getCostGrad(problem, x, storedb, key); %#ok\n end\n \n % This is the star operator of this party.\n hessian = @(y) getHessian(problem, x, y, storedb, key);\n \n % Start a Manopt problem structure for the quadratic optimization\n % problem on the sphere N.\n new_problem.M = N;\n \n % Define the cost function, its gradient and its Hessian.\n\n new_problem.cost = @cost;\n function [f, store] = cost(y, store)\n store = prepare(y, store);\n f = sign*store.f;\n end\n\n new_problem.grad = @grad;\n function [g, store] = grad(y, store)\n store = prepare(y, store);\n g = N.lincomb(y, sign*2, store.Hy, sign*(-2)*store.f, y);\n end\n\n new_problem.hess = @hess;\n function [h, store] = hess(y, ydot, store)\n store = prepare(y, store);\n Hydot = hessian(ydot);\n h = N.lincomb(y, sign*2, Hydot, sign*(-2)*store.f, ydot);\n h = N.proj(y, h);\n end\n\n % This helper makes sure we do not duplicate Hessian computations.\n function store = prepare(y, store)\n if ~isfield(store, 'ready')\n Hy = hessian(y);\n store.f = M.inner(x, y, Hy);\n store.Hy = Hy;\n store.ready = true;\n end\n end\n \n % Call a Manopt solver to solve the quadratic optimization problem on\n % the abstract sphere N.\n [y, lambda, info] = manoptsolve(new_problem, y0, options);\n lambda = sign*lambda;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/hessianmatrix.m", "content": "function [H, basis] = hessianmatrix(problem, x, basis)\n% Computes a matrix which represents the Hessian in some tangent basis.\n%\n% [H, basis] = hessianmatrix(problem, x)\n% [H, basis] = hessianmatrix(problem, x, basis)\n%\n% problem is a Manopt problem structure with a manifold and cost function.\n% x is a point on the manifold problem.M.\n% basis (optional) is an orthonormal basis for the tangent space to the\n% manifold at x. If no basis is supplied, one will be generated at random.\n% If the basis spans only a subspace of the tangent space at x,\n% then the returned matrix represents the Hessian restricted to that subspace.\n%\n% H is an n-by-n symmetric matrix (with n the number of vectors in the basis)\n% such that H(i, j) is the inner product between basis{i}\n% and Hess(basis{j}), with respect to the metric on the tangent space to\n% problem.M at x, where Hess(basis{j}) is the vector obtained after\n% applying the Hessian at x to basis{j}.\n%\n% For optimization, it is usually not useful to compute the Hessian matrix,\n% as this quickly becomes expensive. This tool is provided mostly for\n% exploration and debugging rather than to be used algorithmically in\n% solvers. To access the spectrum of the Hessian, it may be more practical\n% to call hessianextreme or hessianspectrum. This should coincide with eig(H).\n%\n%\n% Example of equivalence:\n%\n% Hu = getHessian(problem, x, u)\n%\n% is equivalent to (but much faster than):\n%\n% B = tangentorthobasis(M, x);\n% H = hessianmatrix(problem, x, B);\n% u_vec = tangent2vec(M, x, B, u);\n% Hu_vec = H*u_vec;\n% Hu = lincomb(M, x, B, Hu_vec);\n%\n% Note that there will be some error due to numerical round-off.\n% \n%\n% See also: hessianspectrum hessianextreme tangentorthobasis orthogonalize tangent2vec\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 14, 2016.\n% Contributors: \n% Change log: \n\n\n % No warning if an approximate Hessian is available, as then the user\n % is presumably aware of what they are doing.\n if ~canGetHessian(problem) && ~canGetApproxHessian(problem)\n warning('manopt:hessianmatrix:nohessian', ...\n ['The Hessian appears to be unavailable.\\n' ...\n 'Will try to use an approximate Hessian instead.\\n'...\n 'Since this approximation may not be linear or '...\n 'symmetric,\\nthe computation might fail and the '...\n 'results (if any)\\nmight make no sense.']);\n end\n \n\n % Unless an orthonormal basis for the tangent space at x is provided,\n % pick a random one.\n if ~exist('basis', 'var') || isempty(basis)\n\t n = problem.M.dim();\n basis = tangentorthobasis(problem.M, x, n);\n\telse\n\t n = numel(basis);\n end\n \n % Create a store database and get a key for x\n storedb = StoreDB(1);\n key = storedb.getNewKey();\n \n % Apply the Hessian at x to each basis vector\n Hbasis = cell(n, 1);\n for k = 1 : numel(Hbasis)\n Hbasis{k} = getHessian(problem, x, basis{k}, storedb, key);\n end\n \n % H is the matrix which contains the inner products of\n % the ((basis vectors)) with the ((Hessian applied to basis vectors)).\n H = zeros(n);\n for i = 1 : n\n H(i, i) = problem.M.inner(x, basis{i}, Hbasis{i});\n for j = (i+1) : n\n H(i, j) = problem.M.inner(x, basis{i}, Hbasis{j});\n H(j, i) = H(i, j);\n end\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/hessianspectrum.m", "content": "function lambdas = hessianspectrum(problem, x, usepreconstr, storedb, key)\n% Returns the eigenvalues of the (preconditioned) Hessian at x.\n% \n% function lambdas = hessianspectrum(problem, x)\n% function lambdas = hessianspectrum(problem, x, useprecon)\n% function lambdas = hessianspectrum(problem, x, useprecon, storedb)\n% function lambdas = hessianspectrum(problem, x, useprecon, storedb, key)\n%\n% If useprecon is not set, or if it is set to 'noprecon' (default), this\n% computes and returns the eigenvalues of the Hessian operator (which needs\n% to be symmetric but not necessarily definite) on the tangent space at x.\n% There are problem.M.dim() eigenvalues. Matlab's eigs is used internally.\n%\n% If useprecon is set to 'precon', the eigenvalues of the composition of\n% the Hessian with the preconditioner at x are computed: Precon o Hessian.\n% The preconditioner must have been defined in the problem structure and\n% has to be symmetric, positive definite. It is supposed to approximate the\n% inverse of the (Riemannian) Hessian. Ideally, the preconditioned Hessian\n% is better conditioned (smaller ratio of largest to smallest eigenvalue in\n% magnitude) than the non-preconditioned spectrum. The present tool can\n% help assess that.\n%\n% The typical ways to define a preconditioner are via problem.precon or\n% problem.sqrtprecon (see comment below). These should be function handles\n% with the same input/output system as problem.hess for the Hessian.\n%\n% If the Hessian is not available from the problem structure, an\n% approximate Hessian will be used. There are no guarantees of\n% interpretability, but this may nevertheless be useful at times.\n%\n% Even though the Hessian and the preconditioner are both symmetric, their\n% composition is not symmetric. This can slow down the call to 'eigs'\n% substantially. If possible, you may specify the square root of the\n% preconditioner in the problem structure, as sqrtprecon. This operator on\n% the tangent space at x must also be symmetric, positive definite, and\n% such that SqrtPrecon o SqrtPrecon = Precon. Then, the spectrum of the\n% symmetric operator SqrtPrecon o Hessian o SqrtPrecon is computed: it is\n% the same as the spectrum of Precon o Hessian, but is usually faster to\n% compute. If both Precon and SqrtPrecon are provided, only SqrtPrecon will\n% be used.\n%\n% The input and the output of the Hessian and of the preconditioner are\n% projected on the tangent space to avoid undesired contributions of the\n% ambient space.\n%\n% storedb is a StoreDB object, key is the StoreDB key to point x.\n%\n% Requires the manifold description in problem.M to have these functions:\n% \n% u_vec = vec(x, u_mat) :\n% Returns a column vector representation of the normal (usually\n% matrix) representation of the tangent vector u_mat. vec must be an\n% isometry between the tangent space (with its Riemannian metric) and\n% a subspace of R^n where n = length(u_vec), with the 2-norm on R^n.\n% In other words: it is an orthogonal projector.\n%\n% u_mat = mat(x, u_vec) :\n% The inverse of vec (its adjoint).\n%\n% u_mat_clean = tangent(x, u_mat) :\n% Subtracts from the tangent vector u_mat any component that would\n% make it \"not really tangent\", by projection.\n%\n% answer = vecmatareisometries() :\n% Returns true if the linear maps encoded by vec and mat are\n% isometries, false otherwise. It is better if the answer is yes.\n%\n% See also: hessianextreme canGetPrecon canGetSqrtPrecon\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 3, 2013.\n% Contributors: \n% Change log:\n%\n% Dec. 18, 2014 (NB):\n% The lambdas are now sorted when they are returned.\n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n% Does no longer accept sqrtprecon as an input: the square root of\n% the preconditioner may now be specified directly in the problem\n% structure, following the same syntax as the preconditioner precon.\n%\n% April 4, 2015 (NB):\n% By default, the spectrum is computed without the preconditioner's\n% effect, even if it is available. A new input option allows to\n% switch this behavior without the need to change the problem\n% structure.\n\n % Allow omission of the key, and even of storedb.\n if ~exist('key', 'var')\n if ~exist('storedb', 'var')\n storedb = StoreDB();\n end\n key = storedb.getNewKey();\n end\n\n % Manage the option to use or not use a preconditioner.\n % The input is a string. It is here transformed into a Boolean.\n if ~exist('usepreconstr', 'var') || isempty(usepreconstr)\n usepreconstr = 'noprecon';\n end\n switch lower(usepreconstr)\n case 'noprecon'\n useprecon = false;\n case 'precon'\n useprecon = true;\n otherwise\n % A bit of legacy code heads up.\n if isa(usepreconstr, 'function_handle')\n warning('manopt:hessianspectrum:oldsyntax', ...\n ['This function no longer expects sqrtprecon ' ...\n 'as input. Place it in the problem structure.']);\n end\n error('Input useprecon must be either ''precon'' or ''noprecon''.');\n end\n\n % No warning if an approximate Hessian is available, as then the user\n % is presumably aware of what they are doing.\n if ~canGetHessian(problem) && ~canGetApproxHessian(problem)\n warning('manopt:hessianspectrum:nohessian', ...\n ['The Hessian appears to be unavailable.\\n' ...\n 'Will try to use an approximate Hessian instead.\\n'...\n 'Since this approximation may not be linear or '...\n 'symmetric,\\nthe computation might fail and the '...\n 'results (if any)\\nmight make no sense.']);\n end\n\n vec = @(u_mat) problem.M.vec(x, u_mat);\n mat = @(u_vec) problem.M.mat(x, u_vec);\n tgt = @(u_mat) problem.M.tangent(x, u_mat);\n \n % n: size of a vectorized tangent vector\n % dim: dimension of the tangent space\n % necessarily, n >= dim.\n % The vectorized operators we build below will have at least n - dim\n % zero eigenvalues.\n n = length(vec(problem.M.zerovec(x)));\n dim = problem.M.dim();\n \n % It is usually a good idea to force a gradient computation to make\n % sure precomputable things are precomputed.\n if canGetGradient(problem)\n [unused1, unused2] = getCostGrad(problem, x, storedb, key); %#ok\n end\n \n hess = @(u_mat) tgt(getHessian(problem, x, tgt(u_mat), storedb, key));\n hess_vec = @(u_vec) vec(hess(mat(u_vec)));\n \n % Regardless of preconditioning, we can only have a symmetric\n % eigenvalue problem if the vec/mat pair of the manifold is an\n % isometry:\n vec_mat_are_isometries = problem.M.vecmatareisometries();\n \n \n if ~useprecon\n\n % No preconditioner to use: simply use the Hessian as is.\n\n eigs_opts.issym = vec_mat_are_isometries;\n eigs_opts.isreal = true;\n lambdas = eigs(hess_vec, n, dim, 'LM', eigs_opts);\n \n elseif canGetSqrtPrecon(problem)\n\n % There is a preconditioner, and we have its square root: deal with\n % the symmetric composition SqrtPrecon o Hessian o SqrtPrecon.\n\n sqrtprec = @(u_mat) tgt(getSqrtPrecon(problem, x, tgt(u_mat), storedb, key));\n sqrtprec_vec = @(u_vec) vec(sqrtprec(mat(u_vec)));\n\n eigs_opts.issym = vec_mat_are_isometries;\n eigs_opts.isreal = true;\n lambdas = eigs(@(u_vec) ...\n sqrtprec_vec(hess_vec(sqrtprec_vec(u_vec))), ...\n n, dim, 'LM', eigs_opts);\n \n elseif canGetPrecon(problem)\n \n % There is a preconditioner, but we don't have its square root:\n % deal with the non-symmetric composition Precon o Hessian.\n\n prec = @(u_mat) tgt(getPrecon(problem, x, tgt(u_mat), storedb, key));\n prec_vec = @(u_vec) vec(prec(mat(u_vec)));\n % prec_inv_vec = @(u_vec) pcg(prec_vec, u_vec);\n\n eigs_opts.issym = false;\n eigs_opts.isreal = true;\n lambdas = eigs(@(u_vec) prec_vec(hess_vec(u_vec)), ...\n n, dim, 'LM', eigs_opts);\n \n else\n \n error('No preconditioner is available in the problem structure.');\n \n end\n \n lambdas = sort(lambdas);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/identify_linear_piece.m", "content": "function [range, poly] = identify_linear_piece(x, y, window_length)\n% Identify a segment of the curve (x, y) that appears to be linear.\n%\n% function [range poly] = identify_linear_piece(x, y, window_length)\n%\n% This function attempts to identify a contiguous segment of the curve\n% defined by the vectors x and y that appears to be linear. A line is fit\n% through the data over all windows of length window_length and the best\n% fit is retained. The output specifies the range of indices such that\n% x(range) is the portion over which (x, y) is the most linear and the\n% output poly specifies a first order polynomial that best fits (x, y) over\n% that range, following the usual matlab convention for polynomials\n% (highest degree coefficients first).\n%\n% See also: checkdiff checkgradient checkhessian\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 8, 2013.\n% Contributors: \n% Change log: \n\n residues = zeros(length(x)-window_length, 1);\n polys = zeros(2, length(residues));\n for i = 1 : length(residues)\n range = i:(i+window_length);\n [poly, meta] = polyfit(x(range), y(range), 1);\n residues(i) = meta.normr;\n polys(:, i) = poly';\n end\n [unused, best] = min(residues); %#ok\n range = best:(best+window_length);\n poly = polys(:, best)';\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/lincomb.m", "content": "function vec = lincomb(M, x, vecs, coeffs)\n% Computes a linear combination of tangent vectors in the Manopt framework.\n%\n% vec = lincomb(M, x, vecs, coeffs)\n%\n% M is a Manopt manifold structure obtained from a factory.\n% x is a point on the manifold M.\n% vecs is a cell containing n tangent vectors at x.\n% coeffs is a vector of length n\n%\n% vec is a tangent vector at x obtained as the linear combination\n%\n% vec = coeffs(1)*vecs{1} + ... + coeffs(n)*vecs{n}\n%\n% If vecs is an orthonormal basis, then tangent2vec is the inverse of\n% lincomb.\n%\n% See also: grammatrix orthogonalize tangentorthobasis tangent2vec\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 28, 2016.\n% Contributors: \n% Change log: \n\n\n n = numel(vecs);\n assert(numel(coeffs) == n);\n \n switch n\n \n case 0\n \n vec = M.zerovec(x);\n \n case 1\n \n vec = M.lincomb(x, coeffs(1), vecs{1});\n \n otherwise\n \n vec = M.lincomb(x, coeffs(1), vecs{1}, coeffs(2), vecs{2});\n \n for k = 3 : n\n \n vec = M.lincomb(x, 1, vec, coeffs(k), vecs{k});\n \n end\n \n end\n \n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/manoptsolve.m", "content": "function [x, cost, info, options] = manoptsolve(problem, x0, options)\n% Gateway helper function to call a Manopt solver, chosen in the options.\n%\n% function [x, cost, info, options] = manoptsolve(problem)\n% function [x, cost, info, options] = manoptsolve(problem, x0)\n% function [x, cost, info, options] = manoptsolve(problem, x0, options)\n% function [x, cost, info, options] = manoptsolve(problem, [], options)\n%\n% Depending on what is available in the Manopt problem structure, one of\n% the Manopt solvers will be called and the outputs passed along. It is\n% also possible to force the choice of a solver by specifying it in the\n% options structure. For example:\n%\n% options.solver = @trustregions;\n%\n% Simply specify a function handle to a Manopt solver.\n%\n% See also: trustregions conjugategradient steepestdescent\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Aug. 13, 2014.\n% Contributors: \n% Change log: \n\n % At the very least, we need a cost function.\n if ~canGetCost(problem)\n error('The problem structure must specify a cost function.');\n end\n \n % Depending on the number of differentials available, pick a different\n % default solver.\n if ~canGetGradient(problem)\n localdefaults.solver = @neldermead;\n elseif ~canGetHessian(problem)\n localdefaults.solver = @conjugategradient;\n else\n localdefaults.solver = @trustregions;\n end\n \n % Merge local defaults with user options, if any.\n if ~exist('options', 'var') || isempty(options)\n options = struct();\n end\n options = mergeOptions(localdefaults, options);\n \n % If no initial guess was specified, prepare the empty one.\n if ~exist('x0', 'var')\n x0 = [];\n end\n \n % Issue the actual call.\n [x, cost, info, options] = options.solver(problem, x0, options);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/matrixlincomb.m", "content": "function v = matrixlincomb(x, a1, d1, a2, d2) %#ok\n% Linear combination function for tangent vectors represented as matrices.\n%\n% function v = lincomb(x, a1, d1)\n% function v = lincomb(x, a1, d1, a2, d2)\n%\n% Given a point x, two tangent vectors d1 and d2 at x, and two real\n% coefficients a1 and a2, returns a tangent vector at x representing\n% a1*d1 + a2*d2, if d1 and d2 are represented as matrices (or more\n% generally as arrays in Matlab).\n%\n% If a2 and d2 are omitted, the returned tangent vector is a1*d1.\n%\n% The input x is actually unused.\n%\n% This function is a helper to define manifolds in Manopt.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, July 2, 2015.\n% Contributors: \n% Change log: \n\n if nargin == 3\n v = a1*d1;\n elseif nargin == 5\n v = a1*d1 + a2*d2;\n else\n error('matrixlincomb takes either 3 or 5 inputs.');\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multihconj.m", "content": "function b = multihconj(a, dim)\n%MULTIHCONJ Hermitian conjugating arrays of matrices.\n% B = MULTIHCONJ(A) is equivalent to B = MULTIHCONJ(A, DIM), where\n% DIM = 1.\n%\n% B = MULTIHCONJ(A, DIM) is equivalent to\n% B = PERMUTE(A, [1:DIM-1, DIM+1, DIM, DIM+2:NDIMS(A)]), where A is an\n% array containing N P-by-Q matrices along its dimensions DIM and DIM+1,\n% and B is an array containing the Q-by-P Hermitian conjugate (') of\n% those N matrices along the same dimensions. N = NUMEL(A) / (P*Q), i.e.\n% N is equal to the number of elements in A divided by the number of\n% elements in each matrix.\n%\n%\n% Example:\n% A 5-by-9-by-3-by-2 array may be considered to be a block array\n% containing ten 9-by-3 matrices along dimensions 2 and 3. In this\n% case, its size is so indicated: 5-by-(9-by-3)-by-2 or 5x(9x3)x2.\n% If A is ................ a 5x(9x3)x2 array of 9x3 matrices,\n% C = MULTIHCONJ(A, 2) is a 5x(3x9)x2 array of 3x9 matrices.\n%\n% See also MULTITRANSP MULTIHERM.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Hiroyuki Sato, April 27, 2015.\n% Contributors: \n% Change log: \n\n % Setting DIM if not supplied.\n if nargin == 1, dim = 1; end\n\n % Transposing\n b = multitransp(a, dim);\n\n %Conjugating\n b = conj(b);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multiherm.m", "content": "function Y = multiherm(X)\n% Returns the Hermitian parts of the matrices in the 3D matrix X\n%\n% function Y = multiherm(X)\n%\n% Y is a 3D matrix the same size as X. Each slice Y(:, :, i) is the\n% Hermitian part of the slice X(:, :, i).\n%\n% See also: multiprod multitransp multihconj multiscale multiskew\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Hiroyuki Sato, April 27, 2015.\n% Contributors: \n% Change log: \n\n Y = .5*(X + multihconj(X));\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multiprod.m", "content": "function c = multiprod(a, b, idA, idB)\n% Multiplying 1-D or 2-D subarrays contained in two N-D arrays.\n% \n% C = MULTIPROD(A,B) is equivalent to C = MULTIPROD(A,B,[1 2],[1 2])\n% C = MULTIPROD(A,B,[D1 D2]) is eq. to C = MULTIPROD(A,B,[D1 D2],[D1 D2])\n% C = MULTIPROD(A,B,D1) is equival. to C = MULTIPROD(A,B,D1,D1)\n%\n% MULTIPROD performs multiple matrix products, with array expansion (AX)\n% enabled. Its first two arguments A and B are \"block arrays\" of any\n% size, containing one or more 1-D or 2-D subarrays, called \"blocks\" (*).\n% For instance, a 5x6x3 array may be viewed as an array containing five\n% 6x3 blocks. In this case, its size is denoted by 5x(6x3). The 1 or 2\n% adjacent dimensions along which the blocks are contained are called the\n% \"internal dimensions\" (IDs) of the array ().\n%\n% 1) 2-D by 2-D BLOCK(S) (*)\n% C = MULTIPROD(A, B, [DA1 DA2], [DB1 DB2]) contains the products\n% of the PxQ matrices in A by the RxS matrices in B. [DA1 DA2] are\n% the IDs of A; [DB1 DB2] are the IDs of B.\n%\n% 2) 2-D by 1-D BLOCK(S) (*)\n% C = MULTIPROD(A, B, [DA1 DA2], DB1) contains the products of the\n% PxQ matrices in A by the R-element vectors in B. The latter are\n% considered to be Rx1 matrices. [DA1 DA2] are the IDs of A; DB1 is\n% the ID of B.\n%\n% 3) 1-D by 2-D BLOCK(S) (*)\n% C = MULTIPROD(A, B, DA1, [DB1 DB2]) contains the products of the \n% Q-element vectors in A by the RxS matrices in B. The vectors in A\n% are considered to be 1xQ matrices. DA1 is the ID of A; [DB1 DB2]\n% are the IDs of B.\n%\n% 4) 1-D BY 1-D BLOCK(S) (*)\n% (a) If either SIZE(A, DA1) == 1 or SIZE(B, DB1) == 1, or both,\n% C = MULTIPROD(A, B, DA1, DB1) returns products of scalars by \n% vectors, or vectors by scalars or scalars by scalars.\n% (b) If SIZE(A, DA1) == SIZE(B, DB1), \n% C = MULTIPROD(A, B, [0 DA1], [DB1 0]) or \n% C = MULTIPROD(A, B, DA1, DB1) virtually turns the vectors\n% contained in A and B into 1xP and Px1 matrices, respectively,\n% then returns their products, similar to scalar products.\n% Namely, C = DOT2(A, B, DA1, DB1) is equivalent to \n% C = MULTIPROD(CONJ(A), B, [0 DA1], [DB1 0]).\n% (c) Without limitations on the length of the vectors in A and B,\n% C = MULTIPROD(A, B, [DA1 0], [0 DB1]) turns the vectors\n% contained in A and B into Px1 and 1xQ matrices, respectively,\n% then returns their products, similar to outer products.\n% Namely, C = OUTER(A, B, DA1, DB1) is equivalent to\n% C = MULTIPROD(CONJ(A), B, [DA1 0], [0 DB1]).\n%\n% Common constraints for all syntaxes:\n% The external dimensions of A and B must either be identical or \n% compatible with AX rules. The internal dimensions of each block\n% array must be adjacent (DA2 == DA1 + 1 and DB2 == DB1 + 1 are\n% required). DA1 and DB1 are allowed to be larger than NDIMS(A) and\n% NDIMS(B). In syntaxes 1, 2, and 3, Q == R is required, unless the\n% blocks in A or B are scalars. \n%\n% Array expansion (AX):\n% AX is a powerful generalization to N-D of the concept of scalar\n% expansion. Indeed, A and B may be scalars, vectors, matrices or\n% multi-dimensional arrays. Scalar expansion is the virtual\n% replication or annihilation of a scalar which allows you to combine\n% it, element by element, with an array X of any size (e.g. X+10,\n% X*10, or []-10). Similarly, in MULTIPROD, the purpose of AX is to\n% automatically match the size of the external dimensions (EDs) of A\n% and B, so that block-by-block products can be performed. ED matching\n% is achieved by means of a dimension shift followed by a singleton\n% expansion:\n% 1) Dimension shift (see SHIFTDIM).\n% Whenever DA1 ~= DB1, a shift is applied to impose DA1 == DB1.\n% If DA1 > DB1, B is shifted to the right by DA1 - DB1 steps.\n% If DB1 > DA1, A is shifted to the right by DB1 - DA1 steps.\n% 2) Singleton expansion (SX).\n% Whenever an ED of either A or B is singleton and the\n% corresponding ED of the other array is not, the mismatch is\n% fixed by virtually replicating the array (or diminishing it to\n% length 0) along that dimension.\n% \n% MULTIPROD is a generalization for N-D arrays of the matrix\n% multiplication function MTIMES, with AX enabled. Vector inner, outer,\n% and cross products generalized for N-D arrays and with AX enabled are\n% performed by DOT2, OUTER, and CROSS2 (MATLAB Central, file #8782).\n% Elementwise multiplications (see TIMES) and other elementwise binary\n% operations with AX enabled are performed by BAXFUN (MATLAB Central,\n% file #23084). Together, these functions make up the \"ARRAYLAB toolbox\".\n%\n% Input and output format:\n% The size of the EDs of C is determined by AX. Block size is\n% determined as follows, for each of the above-listed syntaxes:\n% 1) C contains PxS matrices along IDs MAX([DA1 DA2], [DB1 DB2]).\n% 2) Array Block size ID(s)\n% ----------------------------------------------------\n% A PxQ (2-D) [DA1 DA2]\n% B R (1-D) DB1\n% C (a) P (1-D) MAX(DA1, DB1)\n% C (b) PxQ (2-D) MAX([DA1 DA2], [DB1 DB1+1])\n% ----------------------------------------------------\n% (a) The 1-D blocks in B are not scalars (R > 1).\n% (b) The 1-D blocks in B are scalars (R = 1).\n% 3) Array Block size ID(s)\n% ----------------------------------------------------\n% A Q (1-D) DA1\n% B RxS (2-D) [DB1 DB2]\n% C (a) S (1-D) MAX(DA1, DB1)\n% C (b) RxS (2-D) MAX([DA1 DA1+1], [DB1 DB2])\n% ----------------------------------------------------\n% (a) The 1-D blocks in A are not scalars (Q > 1).\n% (b) The 1-D blocks in A are scalars (Q = 1).\n% 4) Array Block size ID(s)\n% --------------------------------------------------------------\n% (a) A P (1-D) DA1\n% B Q (1-D) DB1\n% C MAX(P,Q) (1-D) MAX(DA1, DB1)\n% --------------------------------------------------------------\n% (b) A P (1-D) DA1\n% B P (1-D) DB1\n% C 1 (1-D) MAX(DA1, DB1)\n% --------------------------------------------------------------\n% (c) A P (1-D) DA1\n% B Q (1-D) DB1\n% C PxQ (2-D) MAX([DA1 DA1+1], [DB1 DB1+1])\n% --------------------------------------------------------------\n%\n% Terminological notes:\n% (*) 1-D and 2-D blocks are generically referred to as \"vectors\" and \n% \"matrices\", respectively. However, both may be also called\n% \"scalars\" if they have a single element. Moreover, matrices with a\n% single row or column (e.g. 1x3 or 3x1) may be also called \"row\n% vectors\" or \"column vectors\".\n% () Not to be confused with the \"inner dimensions\" of the two matrices\n% involved in a product X * Y, defined as the 2nd dimension of X and\n% the 1st of Y (DA2 and DB1 in syntaxes 1, 2, 3).\n%\n% Examples:\n% 1) If A is .................... a 5x(6x3)x2 array,\n% and B is .................... a 5x(3x4)x2 array,\n% C = MULTIPROD(A, B, [2 3]) is a 5x(6x4)x2 array.\n%\n% A single matrix A pre-multiplies each matrix in B\n% If A is ........................... a (1x3) single matrix,\n% and B is ........................... a 10x(3x4) 3-D array,\n% C = MULTIPROD(A, B, [1 2], [3 4]) is a 10x(1x4) 3-D array.\n%\n% Each matrix in A pre-multiplies each matrix in B (all possible\n% combinations)\n% If A is .................... a (6x3)x5 array,\n% and B is .................... a (3x4)x1x2 array,\n% C = MULTIPROD(A, B, [1 2]) is a (6x4)x5x2 array.\n%\n% 2a) If A is ........................... a 5x(6x3)x2 4-D array,\n% and B is ........................... a 5x(3)x2 3-D array,\n% C = MULTIPROD(A, B, [2 3], [2]) is a 5x(6)x2 3-D array.\n%\n% 2b) If A is ........................... a 5x(6x3)x2 4-D array,\n% and B is ........................... a 5x(1)x2 3-D array,\n% C = MULTIPROD(A, B, [2 3], [2]) is a 5x(6x3)x2 4-D array.\n%\n% 4a) If both A and B are .................. 5x(6)x2 3-D arrays,\n% C = MULTIPROD(A, B, 2) is .......... a 5x(1)x2 3-D array, while\n% 4b) C = MULTIPROD(A, B, [2 0], [0 2]) is a 5x(6x6)x2 4-D array\n%\n% See also DOT2, OUTER, CROSS2, BAXFUN, MULTITRANSP, MULTITRACE, MULTISCALE.\n\n% $ Version: 2.1 $\n% CODE by: Paolo de Leva\n% (Univ. of Rome, Foro Italico, IT) 2009 Jan 24\n% optimized by: Paolo de Leva\n% Jinhui Bai (Georgetown Univ., D.C.) 2009 Jan 24\n% COMMENTS by: Paolo de Leva 2009 Feb 24\n% OUTPUT tested by: Paolo de Leva 2009 Feb 24\n% -------------------------------------------------------------------------\n\nassert(nargin >= 2 && nargin <= 4, 'Takes from 2 to 4 inputs.');\n\nswitch nargin % Setting IDA and/or IDB\n case 2, idA = [1 2]; idB = [1 2];\n case 3, idB = idA;\nend\n\n% ESC 1 - Special simple case (both A and B are 2D), solved using C = A * B\n\n if ndims(a)==2 && ndims(b)==2 && ...\n isequal(idA,[1 2]) && isequal(idB,[1 2])\n c = a * b; return\n end\n\n% MAIN 0 - Checking and evaluating array size, block size, and IDs\n\n sizeA0 = size(a);\n sizeB0 = size(b);\n [sizeA, sizeB, shiftC, delC, sizeisnew, idA, idB, ...\n squashOK, sxtimesOK, timesOK, mtimesOK, sumOK] = ...\n sizeval(idA,idB, sizeA0,sizeB0);\n\n% MAIN 1 - Applying dimension shift (first step of AX) and \n% turning both A and B into arrays of either 1-D or 2-D blocks\n\n if sizeisnew(1), a = reshape(a, sizeA); end \n if sizeisnew(2), b = reshape(b, sizeB); end\n\n% MAIN 2 - Performing products with or without SX (second step of AX)\n\n if squashOK % SQUASH + MTIMES (fastest engine)\n c = squash2D_mtimes(a,b, idA,idB, sizeA,sizeB, squashOK); \n elseif timesOK % TIMES (preferred w.r. to SX + TIMES)\n if sumOK, c = sum(a .* b, sumOK);\n else c = a .* b; end\n elseif sxtimesOK % SX + TIMES\n if sumOK, c = sum(bsxfun(@times, a, b), sumOK);\n else c = bsxfun(@times, a, b); end\n elseif mtimesOK % MTIMES (rarely used)\n c = a * b;\n end\n\n% MAIN 3 - Reshaping C (by inserting or removing singleton dimensions)\n\n [sizeC sizeCisnew] = adjustsize(size(c), shiftC, false, delC, false);\n if sizeCisnew, c = reshape(c, sizeC); end\n\n\nfunction c = squash2D_mtimes(a, b, idA, idB, sizeA, sizeB, squashOK)\n% SQUASH2D_MTIMES Multiproduct with single-block expansion (SBX).\n% Actually, no expansion is performed. The multi-block array is\n% rearranged from N-D to 2-D, then MTIMES is applied, and eventually the\n% result is rearranged back to N-D. No additional memory is required.\n% One and only one of the two arrays must be single-block, and its IDs\n% must be [1 2] (MAIN 1 removes leading singletons). Both arrays\n% must contain 2-D blocks (MAIN 1 expands 1-D blocks to 2-D).\n\n if squashOK == 1 % A is multi-block, B is single-block (squashing A)\n\n % STEP 1 - Moving IDA(2) to last dimension\n nd = length(sizeA);\n d2 = idA(2); \n order = [1:(d2-1) (d2+1):nd d2]; % Partial shifting\n a = permute(a, order); % ...xQ\n\n % STEP 2 - Squashing A from N-D to 2-D \n q = sizeB(1);\n s = sizeB(2);\n lengthorder = length(order);\n collapsedsize = sizeA(order(1:lengthorder-1)); \n n = prod(collapsedsize);\n a = reshape(a, [n, q]); % NxQ \n fullsize = [collapsedsize s]; % Size to reshape C back to N-D\n\n else % B is multi-block, A is single-block (squashing B)\n\n % STEP 1 - Moving IDB(1) to first dimension\n nd = length(sizeB);\n d1 = idB(1); \n order = [d1 1:(d1-1) (d1+1):nd]; % Partial shifting\n b = permute(b, order); % Qx...\n\n % STEP 2 - Squashing B from N-D to 2-D \n p = sizeA(1);\n q = sizeA(2);\n lengthorder = length(order);\n collapsedsize = sizeB(order(2:lengthorder)); \n n = prod(collapsedsize);\n b = reshape(b, [q, n]); % QxN\n fullsize = [p collapsedsize]; % Size to reshape C back to N-D\n\n end\n\n % FINAL STEPS - Multiplication, reshape to N-D, inverse permutation\n invorder(order) = 1 : lengthorder;\n c = permute (reshape(a*b, fullsize), invorder);\n\n\nfunction [sizeA, sizeB, shiftC, delC, sizeisnew, idA, idB, ...\n squashOK, sxtimesOK, timesOK, mtimesOK, sumOK] = ...\n sizeval(idA0,idB0, sizeA0,sizeB0)\n%SIZEVAL Evaluation of array size, block size, and IDs\n% Possible values for IDA and IDB:\n% [DA1 DA2], [DB1 DB2]\n% [DA1 DA2], [DB1]\n% [DA1], [DB1 DB2]\n% [DA1], [DB1]\n% [DA1 0], [0 DB1]\n% [0 DA1], [DB1 0]\n%\n% sizeA/B Equal to sizeA0/B0 if RESHAPE is not needed in MAIN 1\n% shiftC, delC Variables controlling MAIN 3.\n% sizeisnew 1x2 logical array; activates reshaping of A and B.\n% idA/B May change only if squashOK ~= 0\n% squashOK If only A or B is a multi-block array (M-B) and the other\n% is single-block (1-B), it will be rearranged from N-D to\n% 2-D. If both A and B are 1-B or M-B arrays, squashOK = 0.\n% If only A (or B) is a M-B array, squashOK = 1 (or 2).\n% sxtimesOK, timesOK, mtimesOK Flags controlling MAIN 2 (TRUE/FALSE).\n% sumOK Dimension along which SUM is performed. If SUM is not\n% needed, sumOK = 0.\n\n% Initializing output arguments\n\n idA = idA0;\n idB = idB0;\n squashOK = 0;\n sxtimesOK = false;\n timesOK = false;\n mtimesOK = false;\n sumOK = 0;\n shiftC = 0;\n delC = 0;\n\n% Checking for gross input errors\n\n NidA = numel(idA);\n NidB = numel(idB);\n idA1 = idA(1);\n idB1 = idB(1);\n if NidA>2 || NidB>2 || NidA==0 || NidB==0 || ...\n ~isreal(idA1) || ~isreal(idB1) || ...\n ~isnumeric(idA1) || ~isnumeric(idB1) || ...\n 0>idA1 || 0>idB1 || ... % negative \n idA1~=fix(idA1) || idB1~=fix(idB1) || ... % non-integer\n ~isfinite(idA1) || ~isfinite(idB1) % Inf or NaN \n error('MULTIPROD:InvalidDimensionArgument', ...\n ['Internal-dimension arguments (e.g., [IDA1 IDA2]) must\\n', ...\n 'contain only one or two non-negative finite integers']);\n end\n\n% Checking Syntaxes containing zeros (4b/c)\n\n declared_outer = false;\n idA2 = idA(NidA); % It may be IDA1 = IDA2 (1-D block)\n idB2 = idB(NidB);\n\n if any(idA==0) || any(idB==0)\n \n % \"Inner products\": C = MULTIPROD(A, B, [0 DA1], [DB1 0])\n if idA1==0 && idA2>0 && idB1>0 && idB2==0\n idA1 = idA2;\n idB2 = idB1;\n % \"Outer products\": C = MULTIPROD(A, B, [DA1 0], [0 DB1]) \n elseif idA1>0 && idA2==0 && idB1==0 && idB2>0\n declared_outer = true;\n idA2 = idA1;\n idB1 = idB2;\n else\n error('MULTIPROD:InvalidDimensionArgument', ...\n ['Misused zeros in the internal-dimension arguments\\n', ...\n '(see help heads 4b and 4c)']);\n end\n NidA = 1; \n NidB = 1;\n idA = idA1;\n idB = idB1;\n\n elseif (NidA==2 && idA2~=idA1+1) || ... % Non-adjacent IDs\n (NidB==2 && idB2~=idB1+1)\n error('MULTIPROD:InvalidDimensionArgument', ...\n ['If an array contains 2-D blocks, its two internal dimensions', ... \n 'must be adjacent (e.g. IDA2 == IDA1+1)']);\n end\n\n% ESC - Case for which no reshaping is needed (both A and B are scalars)\n\n scalarA = isequal(sizeA0, [1 1]);\n scalarB = isequal(sizeB0, [1 1]);\n if scalarA && scalarB\n sizeA = sizeA0;\n sizeB = sizeB0;\n sizeisnew = [false false];\n timesOK = true; return\n end\n\n% Computing and checking adjusted sizes\n% The lengths of ADJSIZEA and ADJSIZEB must be >= IDA(END) and IDB(END)\n\n NsA = idA2 - length(sizeA0); % Number of added trailing singletons\n NsB = idB2 - length(sizeB0);\n adjsizeA = [sizeA0 ones(1,NsA)];\n adjsizeB = [sizeB0 ones(1,NsB)];\n extsizeA = adjsizeA([1:idA1-1, idA2+1:end]); % Size of EDs\n extsizeB = adjsizeB([1:idB1-1, idB2+1:end]);\n p = adjsizeA(idA1);\n q = adjsizeA(idA2);\n r = adjsizeB(idB1);\n s = adjsizeB(idB2); \n scalarsinA = (p==1 && q==1);\n scalarsinB = (r==1 && s==1);\n singleA = all(extsizeA==1);\n singleB = all(extsizeB==1);\n if q~=r && ~scalarsinA && ~scalarsinB && ~declared_outer\n error('MULTIPROD:InnerDimensionsMismatch', ...\n 'Inner matrix dimensions must agree.');\n end\n\n% STEP 1/3 - DIMENSION SHIFTING (FIRST STEP OF AX)\n% Pipeline 1 (using TIMES) never needs left, and may need right shifting.\n% Pipeline 2 (using MTIMES) may need left shifting of A and right of B.\n\n shiftA = 0;\n shiftB = 0;\n diffBA = idB1 - idA1; \n if scalarA % Do nothing\n elseif singleA && ~scalarsinB, shiftA = -idA1 + 1; % Left shifting A\n elseif idB1 > idA1, shiftA = diffBA; % Right shifting A \n end \n if scalarB % Do nothing\n elseif singleB && ~scalarsinA, shiftB = -idB1 + 1; % Left shifting B\n elseif idA1 > idB1, shiftB = -diffBA; % Right shifting B\n end\n\n% STEP 2/3 - SELECTION OF PROPER ENGINE AND BLOCK SIZE ADJUSTMENTS\n\n addA = 0; addB = 0;\n delA = 0; delB = 0;\n swapA = 0; swapB = 0;\n idC1 = max(idA1, idB1);\n idC2 = idC1 + 1;\n checktimes = false;\n\n if (singleA||singleB) &&~scalarsinA &&~scalarsinB % Engine using MTIMES\n\n if singleA && singleB \n mtimesOK = true;\n shiftC=idC1-1; % Right shifting C\n idC1=1; idC2=2;\n elseif singleA\n squashOK = 2;\n idB = [idB1, idB1+1] + shiftB;\n else % singleB\n squashOK = 1;\n idA = [idA1, idA1+1] + shiftA;\n end\n\n if NidA==2 && NidB==2 % 1) 2-D BLOCKS BY 2-D BLOCKS\n % OK \n elseif NidA==2 % 2) 2-D BLOCKS BY 1-D BLOCKS\n addB=idB1+1; delC=idC2;\n elseif NidB==2 % 3) 1-D BLOCKS BY 2-D BLOCKS\n addA=idA1; delC=idC1;\n else % 4) 1-D BLOCKS BY 1-D BLOCKS\n if declared_outer\n addA=idA1+1; addB=idB1;\n else\n addA=idA1; addB=idB1+1; delC=idC2;\n end\n end \n\n else % Engine using TIMES (also used if SCALARA || SCALARB)\n \n sxtimesOK = true;\n\n if NidA==2 && NidB==2 % 1) 2-D BLOCKS BY 2-D BLOCKS\n\n if scalarA || scalarB\n timesOK=true; \n elseif scalarsinA && scalarsinB % scal-by-scal\n checktimes=true;\n elseif scalarsinA || scalarsinB || ... % scal-by-mat\n (q==1 && r==1) % vec-by-vec (\"outer\")\n elseif p==1 && s==1 % vec-by-vec (\"inner\")\n swapA=idA1; sumOK=idC1; checktimes=true;\n elseif s==1 % mat-by-vec\n swapB=idB1; sumOK=idC2;\n elseif p==1 % vec-by-mat\n swapA=idA1; sumOK=idC1;\n else % mat-by-mat\n addA=idA2+1; addB=idB1; sumOK=idC2; delC=idC2;\n end\n\n elseif NidA==2 % 2) 2-D BLOCKS BY 1-D BLOCKS\n\n if scalarA || scalarB\n timesOK=true; \n elseif scalarsinA && scalarsinB % scal-by-scal\n addB=idB1; checktimes=true;\n elseif scalarsinA % scal-by-vec\n delA=idA1;\n elseif scalarsinB % mat-by-scal\n addB=idB1;\n elseif p==1 % vec-by-vec (\"inner\")\n delA=idA1; sumOK=idC1; checktimes=true;\n else % mat-by-vec\n addB=idB1; sumOK=idC2; delC=idC2;\n end\n\n elseif NidB==2 % 3) 1-D BLOCKS BY 2-D BLOCKS\n\n if scalarA || scalarB\n timesOK=true; \n elseif scalarsinA && scalarsinB % scal-by-scal\n addA=idA1+1; checktimes=true;\n elseif scalarsinB % vec-by-scal\n delB=idB2;\n elseif scalarsinA % scal-by-mat\n addA=idA1+1;\n elseif s==1 % vec-by-vec (\"inner\")\n delB=idB2; sumOK=idC1; checktimes=true;\n else % vec-by-mat\n addA=idA1+1; sumOK=idC1; delC=idC1;\n end\n\n else % 4) 1-D BLOCKS BY 1-D BLOCKS\n\n if scalarA || scalarB\n timesOK=true; \n elseif declared_outer % vec-by-vec (\"outer\")\n addA=idA1+1; addB=idB1;\n elseif scalarsinA && scalarsinB % scal-by-scal\n checktimes=true;\n elseif scalarsinA || scalarsinB % vec-by-scal\n else % vec-by-vec\n sumOK=idC1; checktimes=true;\n end\n end\n end\n\n% STEP 3/3 - Adjusting the size of A and B. The size of C is adjusted\n% later, because it is not known yet.\n\n [sizeA, sizeisnew(1)] = adjustsize(sizeA0, shiftA, addA, delA, swapA);\n [sizeB, sizeisnew(2)] = adjustsize(sizeB0, shiftB, addB, delB, swapB);\n\n if checktimes % Faster than calling BBXFUN\n diff = length(sizeB) - length(sizeA);\n if isequal([sizeA ones(1,diff)], [sizeB ones(1,-diff)])\n timesOK = true;\n end\n end\n\n\nfunction [sizeA, sizeisnew] = adjustsize(sizeA0, shiftA, addA, delA, swapA)\n% ADJUSTSIZE Adjusting size of a block array.\n\n % Dimension shifting (by adding or deleting trailing singleton dim.)\n if shiftA>0, [sizeA,newA1] = addsing(sizeA0, 1, shiftA);\n elseif shiftA<0, [sizeA,newA1] = delsing(sizeA0, 1,-shiftA); \n else sizeA = sizeA0; newA1 = false;\n end\n % Modifying block size (by adding, deleting, or moving singleton dim.)\n if addA, [sizeA,newA2] = addsing(sizeA, addA+shiftA, 1); % 1D-->2D \n elseif delA, [sizeA,newA2] = delsing(sizeA, delA+shiftA, 1); % 2D-->1D\n elseif swapA, [sizeA,newA2] = swapdim(sizeA,swapA+shiftA); % ID Swapping\n else newA2 = false;\n end\n sizeisnew = newA1 || newA2;\n\n\nfunction [newsize, flag] = addsing(size0, dim, ns)\n%ADDSING Adding NS singleton dimensions to the size of an array.\n% Warning: NS is assumed to be a positive integer.\n% Example: If the size of A is ..... SIZE0 = [5 9 3]\n% NEWSIZE = ADDSING(SIZE0, 3, 2) is [5 9 1 1 3]\n\n if dim > length(size0)\n newsize = size0;\n flag = false;\n else \n newsize = [size0(1:dim-1), ones(1,ns), size0(dim:end)];\n flag = true;\n end\n\n\nfunction [newsize, flag] = delsing(size0, dim, ns)\n%DELSING Removing NS singleton dimensions from the size of an array.\n% Warning: Trailing singletons are not removed\n% Example: If the size of A is SIZE0 = [1 1 1 5 9 3]\n% NEWSIZE = DELSING(SIZE, 1, 3) is [5 9 3]\n\n if dim > length(size0)-ns % Trailing singletons are not removed\n newsize = size0;\n flag = false;\n else % Trailing singl. added, so NEWSIZE is guaranteed to be 2D or more\n newsize = size0([1:dim-1, dim+ns:end, dim]);\n flag = true;\n end\n\n\nfunction [newsize, flag] = swapdim(size0, dim)\n%SWAPDIM Swapping two adjacent dimensions of an array (DIM and DIM+1).\n% Used only when both A and B are multi-block arrays with 2-D blocks.\n% Example: If the size of A is .......... 5x(6x3)\n% NEWSIZE = SWAPIDS(SIZE0, 2) is 5x(3x6)\n\n newsize = [size0 1]; % Guarantees that dimension DIM+1 exists.\n newsize = newsize([1:dim-1, dim+1, dim, dim+2:end]);\n flag = true;\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multiprodmultitransp_license.txt", "content": "Copyright (c) 2009, Paolo de Leva\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without \nmodification, are permitted provided that the following conditions are \nmet:\n\n * Redistributions of source code must retain the above copyright \n notice, this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above copyright \n notice, this list of conditions and the following disclaimer in \n the documentation and/or other materials provided with the distribution\n \nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" \nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE \nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE \nARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE \nLIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR \nCONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF \nSUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS \nINTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN \nCONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) \nARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE \nPOSSIBILITY OF SUCH DAMAGE.\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multiscale.m", "content": "function A = multiscale(scale, A)\n% Multiplies the 2D slices in a 3D matrix by individual scalars.\n%\n% function A = multiscale(scale, A)\n%\n% Given a vector scale of length N and a 3-dimensional matrix A of size\n% n-by-m-by-N, returns a matrix A of same size such that\n% A(:, :, k) := scale(k) * A(:, :, k);\n%\n% See also: multiprod multitransp multitrace\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n\n\n\tassert(ndims(A) <= 3, ...\n ['multiscale is only well defined for matrix arrays of 3 ' ...\n 'or less dimensions.']);\n\t[n, m, N] = size(A);\n\tassert(numel(scale) == N, ...\n ['scale must be a vector whose length equals the third ' ...\n 'dimension of A, that is, the number of 2D matrix slices ' ...\n 'in the 3D matrix A.']);\n\n scale = scale(:);\n A = reshape(bsxfun(@times, reshape(A, n*m, N), scale'), n, m, N);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multiskew.m", "content": "function Y = multiskew(X)\n% Returns the skew-symmetric parts of the matrices in the 3D matrix X.\n%\n% function Y = multiskew(X)\n%\n% Y is a 3D matrix the same size as X. Each slice Y(:, :, i) is the\n% skew-symmetric part of the slice X(:, :, i).\n%\n% See also: multiprod multitransp multiscale multisym\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Jan. 31, 2013.\n% Contributors: \n% Change log: \n\n Y = .5*(X - multitransp(X));\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multisqnorm.m", "content": "function sqnorm = multisqnorm(A)\n% Returns the squared Frobenius norms of the slices of a 3D matrix.\n%\n% function sqnorm = multisqnorm(A)\n%\n% Given a 3-dimensional matrix A of size n-by-m-by-N, returns a column\n% vector of length N such that sqnorm(i) = norm(A(:, :, i), 'fro')^2.\n%\n% See also: multiprod multitransp multitrace norms\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 17, 2015.\n% Contributors: \n% Change log: \n\n\n\tassert(ndims(A) <= 3, ...\n ['multisqnorm is only well defined for matrix arrays of 3 ' ...\n 'or less dimensions.']);\n\t[n, m, N] = size(A);\n \n % This is equivalent to squeeze(sum(norms(A, 2, 1).^2)), but faster.\n sqnorm = sum(reshape(A, n*m, N).^2, 1)';\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multisym.m", "content": "function Y = multisym(X)\n% Returns the symmetric parts of the matrices in the 3D matrix X\n%\n% function Y = multisym(X)\n%\n% Y is a 3D matrix the same size as X. Each slice Y(:, :, i) is the\n% symmetric part of the slice X(:, :, i).\n%\n% See also: multiprod multitransp multiscale multiskew\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Jan. 31, 2013.\n% Contributors: \n% Change log: \n\n Y = .5*(X + multitransp(X));\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multitrace.m", "content": "function tr = multitrace(A)\n% Computes the traces of the 2D slices in a 3D matrix.\n% \n% function tr = multitrace(A)\n%\n% For a 3-dimensional matrix A of size n-by-n-by-N, returns a column vector\n% tr of length N such that tr(k) = trace(A(:, :, k));\n%\n% See also: multiprod multitransp multiscale\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n\n \n assert(ndims(A) <= 3, ...\n ['multitrace is only well defined for matrix arrays of 3 ' ...\n 'or less dimensions.']);\n\n\ttr = diagsum(A, 1, 2);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/multitransp.m", "content": "function b = multitransp(a, dim)\n% Transposing arrays of matrices.\n% \n% B = MULTITRANSP(A) is equivalent to B = MULTITRANSP(A, DIM), where\n% DIM = 1.\n%\n% B = MULTITRANSP(A, DIM) is equivalent to\n% B = PERMUTE(A, [1:DIM-1, DIM+1, DIM, DIM+2:NDIMS(A)]), where A is an\n% array containing N P-by-Q matrices along its dimensions DIM and DIM+1,\n% and B is an array containing the Q-by-P transpose (.') of those N\n% matrices along the same dimensions. N = NUMEL(A) / (P*Q), i.e. N is\n% equal to the number of elements in A divided by the number of elements\n% in each matrix.\n%\n% MULTITRANSP, PERMUTE and IPERMUTE are a generalization of TRANSPOSE\n% (.') for N-D arrays.\n%\n% Example:\n% A 5-by-9-by-3-by-2 array may be considered to be a block array\n% containing ten 9-by-3 matrices along dimensions 2 and 3. In this\n% case, its size is so indicated: 5-by-(9-by-3)-by-2 or 5x(9x3)x2.\n% If A is ................ a 5x(9x3)x2 array of 9x3 matrices,\n% C = MULTITRANSP(A, 2) is a 5x(3x9)x2 array of 3x9 matrices.\n%\n% See also PERMUTE, IPERMUTE, MULTIPROD, MULTITRACE, MULTISCALE.\n\n% $ Version: 1.0 $\n% CODE by: Paolo de Leva (IUSM, Rome, IT) 2005 Sep 9\n% COMMENTS by: Code author 2006 Nov 21\n% OUTPUT tested by: Code author 2005 Sep 13\n% -------------------------------------------------------------------------\n\n% Setting DIM if not supplied.\nif nargin == 1, dim = 1; end\n\n% Transposing\norder = [1:dim-1, dim+1, dim, dim+2:ndims(a)];\nb = permute(a, order);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/orthogonalize.m", "content": "function [orthobasis, L] = orthogonalize(M, x, basis)\n% Orthonormalizes a basis of tangent vectors in the Manopt framework.\n%\n% function [orthobasis, L] = orthogonalize(M, x, basis)\n%\n% M is a Manopt manifold structure obtained from a factory.\n% x is a point on the manifold M.\n% basis is a cell containing n linearly independent tangent vectors at x.\n%\n% orthobasis is a cell of same size as basis which contains an orthonormal\n% basis for the same subspace as that spanned by basis. Orthonormality is\n% assessed with respect to the metric on the tangent space to M at x.\n% L is upper triangular of size n x n if basis has n vectors, such that,\n% basis{k} = sum_j=1^k orthobasis{j} * L(j, k) (akin to R in a QR\n% factorization.)\n%\n% See also: grammatrix tangentorthobasis\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 28, 2016.\n% Contributors: \n% Change log: \n\n\n n = numel(basis);\n orthobasis = cell(size(basis));\n \n % Build the Gram matrix of the basis vectors.\n G = grammatrix(M, x, basis);\n \n % If the vectors in 'basis' were the columns of V, and the inner\n % product were the classical dot product, then G = V'*V. We are looking\n % for R, an invertible matrix such that V*R is orthogonal. Thus, R\n % satisfies R'*V'*V*R = eye(n); equivalently:\n % G = inv(R)'*inv(R).\n % Computing a Cholesky factorization of G yields L such that G = L'*L.\n % Thus, R = inv(L). Each column of R states exactly which linear\n % combinations of the vectors in 'basis' must be computed to produce\n % the orthonormal basis.\n %\n % Of course, in that formalism, we could directly take a qr of V, but\n % in the actual setting V is not available; the only simple object\n % available is G.\n\t%\n\t% If this simple code turns out not to be satisfactory (most likely\n\t% because of numerical instability), it may be good to consider\n\t% implementing a modified Gram-Schmidt algorithm instead, and even to\n\t% provide a helper function which calls it twice.\n L = chol(G);\n R = inv(L);\n \n % Note that R is upper triangular.\n % We now compute the n linear combinations.\n \n for k = 1 : n\n \n orthobasis{k} = lincomb(M, x, basis(1:k), R(1:k, k));\n \n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/plotprofile.m", "content": "function cost = plotprofile(problem, x, d, t)\n% Plot the cost function along a geodesic or a retraction path.\n%\n% function plotprofile(problem)\n% function plotprofile(problem, x)\n% function plotprofile(problem, x, d)\n% function plotprofile(problem, x, d, t)\n% function plotprofile(problem, x, [], t)\n% function plotprofile(problem, [], [], t)\n%\n% function costs = plotprofile(problem, x, d, t)\n%\n% Plot profile evaluates the cost function along a geodesic gamma(t) such\n% that gamma(0) = x and the derivative of gamma at 0 is the direction d.\n% The input t is a vector specifying for which values of t we must evaluate\n% f(gamma(t)) (it may include negative values).\n%\n% If the function is called with an output, the plot is not drawn and the\n% values of the cost are returned for the instants t.\n%\n% If x is omitted, a random point is picked. If d is omitted, a random\n% tangent vector at x is picked. If t is omitted, it is generated as a\n% linspace over [-1, 1].\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Jan. 9, 2013.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% Nov. 12, 2016 (NB):\n% Making more inputs optional.\n\n % Verify that the problem description is sufficient.\n if ~canGetCost(problem)\n error('It seems no cost was provided.'); \n end\n \n if ~exist('x', 'var') || isempty(x)\n x = problem.M.rand();\n if exist('d', 'var') && ~isempty(d)\n error('If x is omitted, d should not be specified.');\n end\n end\n if ~exist('d', 'var') || isempty(d)\n d = problem.M.randvec(x);\n end\n if ~exist('t', 'var') || isempty(t)\n t = linspace(-1, 1, 101);\n end\n \n if isfield(problem.M, 'exp')\n expo = problem.M.exp;\n str = 'Exp';\n else\n expo = problem.M.retr;\n str = 'Retr';\n end\n \n storedb = StoreDB();\n linesearch_fun = @(t) getCost(problem, expo(x, d, t), storedb);\n \n cost = zeros(size(t));\n for i = 1 : numel(t)\n cost(i) = linesearch_fun(t(i));\n end\n \n if nargout == 0\n plot(t, cost);\n xlabel('t');\n ylabel(['f(' str '_x(t*d))']);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/powermanifold.m", "content": "function Mn = powermanifold(M, n)\n% Returns a structure describing a power manifold M^n = M x M x ... x M.\n%\n% function Mn = powermanifold(M, n)\n%\n% Input: a manifold structure M and an integer n >= 1.\n% \n% Output: a manifold structure Mn representing M x ... x M (n copies of M)\n% with the metric of M extended element-wise. Points and vectors are stored\n% as cells of size nx1.\n%\n% This code is for prototyping uses. The structures returned are often\n% inefficient representations of power manifolds owing to their use of\n% for-loops, but they should allow to rapidly try out an idea.\n%\n% Example (an inefficient representation of the oblique manifold (3, 10)):\n% Mn = powermanifold(spherefactory(3), 10)\n% disp(Mn.name());\n% x = Mn.rand()\n%\n% See also: productmanifold\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log:\n% NB, July 4, 2013: Added support for vec, mat, tangent.\n% Added support for egrad2rgrad and ehess2rhess.\n\n \n assert(n >= 1, 'n must be an integer larger than or equal to 1.');\n \n Mn.name = @() sprintf('[%s]^%d', M.name(), n);\n \n Mn.dim = @() n*M.dim();\n \n Mn.inner = @inner;\n function val = inner(x, u, v)\n val = 0;\n for i = 1 : n\n val = val + M.inner(x{i}, u{i}, v{i});\n end\n end\n\n Mn.norm = @(x, d) sqrt(Mn.inner(x, d, d));\n\n Mn.dist = @dist;\n function d = dist(x, y)\n sqd = 0;\n for i = 1 : n\n sqd = sqd + M.dist(x{i}, y{i})^2;\n end\n d = sqrt(sqd);\n end\n\n Mn.typicaldist = @typicaldist;\n function d = typicaldist()\n sqd = 0;\n for i = 1 : n\n sqd = sqd + M.typicaldist()^2;\n end\n d = sqrt(sqd);\n end\n \n Mn.proj = @proj;\n function u = proj(x, u)\n for i = 1 : n\n u{i} = M.proj(x{i}, u{i});\n end\n end\n \n Mn.tangent = @tangent;\n function u = tangent(x, u)\n for i = 1 : n\n u{i} = M.tangent(x{i}, u{i});\n end\n end\n \n if isfield(M, 'tangent2ambient')\n Mn.tangent2ambient = @tangent2ambient;\n else\n Mn.tangent2ambient = @(x, u) u;\n end\n function u = tangent2ambient(x, u)\n for i = 1 : n\n u{i} = M.tangent2ambient(x{i}, u{i});\n end\n end\n \n Mn.egrad2rgrad = @egrad2rgrad;\n function g = egrad2rgrad(x, g)\n for i = 1 : n\n g{i} = M.egrad2rgrad(x{i}, g{i});\n end\n end\n \n Mn.ehess2rhess = @ehess2rhess;\n function h = ehess2rhess(x, eg, eh, h)\n for i = 1 : n\n h{i} = M.ehess2rhess(x{i}, eg{i}, eh{i}, h{i});\n end\n end\n \n Mn.exp = @expo;\n function x = expo(x, u, t)\n if nargin < 3\n t = 1.0;\n end\n for i = 1 : n\n x{i} = M.exp(x{i}, u{i}, t);\n end\n end\n \n Mn.retr = @retr;\n function x = retr(x, u, t)\n if nargin < 3\n t = 1.0;\n end\n for i = 1 : n\n x{i} = M.retr(x{i}, u{i}, t);\n end\n end\n \n if isfield(M, 'log')\n Mn.log = @loga;\n end\n function u = loga(x, y)\n u = cell(n, 1);\n for i = 1 : n\n u{i} = M.log(x{i}, y{i});\n end\n end\n \n Mn.hash = @hash;\n function str = hash(x)\n str = '';\n for i = 1 : n\n str = [str M.hash(x{i})]; %#ok\n end\n str = ['z' hashmd5(str)];\n end\n\n Mn.lincomb = @lincomb;\n function x = lincomb(x, a1, u1, a2, u2)\n if nargin == 3\n for i = 1 : n\n x{i} = M.lincomb(x{i}, a1, u1{i});\n end\n elseif nargin == 5\n for i = 1 : n\n x{i} = M.lincomb(x{i}, a1, u1{i}, a2, u2{i});\n end\n else\n error('Bad usage of powermanifold.lincomb');\n end\n end\n\n Mn.rand = @rand;\n function x = rand()\n x = cell(n, 1);\n for i = 1 : n\n x{i} = M.rand();\n end\n end\n\n Mn.randvec = @randvec;\n function u = randvec(x)\n u = cell(n, 1);\n for i = 1 : n\n u{i} = M.randvec(x{i});\n end\n u = Mn.lincomb(x, 1/sqrt(n), u);\n end\n\n Mn.zerovec = @zerovec;\n function u = zerovec(x)\n u = cell(n, 1);\n for i = 1 : n\n u{i} = M.zerovec(x{i});\n end\n end\n\n if isfield(M, 'transp')\n Mn.transp = @transp;\n end\n function u = transp(x1, x2, u)\n for i = 1 : n\n u{i} = M.transp(x1{i}, x2{i}, u{i});\n end\n end\n\n if isfield(M, 'pairmean')\n Mn.pairmean = @pairmean;\n end\n function y = pairmean(x1, x2)\n y = cell(n, 1);\n for i = 1 : n\n y{i} = M.pairmean(x1{i}, x2{i});\n end\n end\n\n % Compute the length of a vectorized tangent vector of M at x, assuming\n % this length is independent of the point x (that should be fine).\n if isfield(M, 'vec')\n rand_x = M.rand();\n zero_u = M.zerovec(rand_x);\n len_vec = length(M.vec(rand_x, zero_u));\n\n Mn.vec = @vec;\n \n if isfield(M, 'mat')\n Mn.mat = @mat;\n end\n \n end\n \n function u_vec = vec(x, u_mat)\n u_vec = zeros(len_vec, n);\n for i = 1 : n\n u_vec(:, i) = M.vec(x{i}, u_mat{i});\n end\n u_vec = u_vec(:);\n end\n\n function u_mat = mat(x, u_vec)\n u_mat = cell(n, 1);\n u_vec = reshape(u_vec, len_vec, n);\n for i = 1 : n\n u_mat{i} = M.mat(x{i}, u_vec(:, i));\n end\n end\n\n if isfield(M, 'vecmatareisometries')\n Mn.vecmatareisometries = M.vecmatareisometries;\n else\n Mn.vecmatareisometries = @() false;\n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/productmanifold.m", "content": "function M = productmanifold(elements)\n% Returns a structure describing a product manifold M = M1 x M2 x ... x Mn.\n%\n% function M = productmanifold(elements)\n%\n% Input: an elements structure such that each field contains a manifold\n% structure.\n% \n% Output: a manifold structure M representing the manifold obtained by\n% taking the Cartesian product of the manifolds described in the elements\n% structure, with the metric obtainded by element-wise extension. Points\n% and vectors are stored as structures with the same fieldnames as in\n% elements.\n%\n% Example:\n% M = productmanifold(struct('X', spherefactory(3), 'Y', spherefactory(4)))\n% disp(M.name());\n% x = M.rand()\n%\n% Points of M = S^2 x S^3 are represented as structures with two fields, X\n% and Y. The values associated to X are points of S^2, and likewise points\n% of S^3 for the field Y. Tangent vectors are also represented as\n% structures with two corresponding fields X and Y.\n% \n% See also: powermanifold\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 30, 2012.\n% Contributors: \n% Change log: \n% NB, July 4, 2013: Added support for vec, mat, tangent.\n% Added support for egrad2rgrad and ehess2rhess.\n% Modified hash function to make hash strings shorter.\n\n\n elems = fieldnames(elements);\n nelems = numel(elems);\n \n assert(nelems >= 1, ...\n 'elements must be a structure with at least one field.');\n \n M.name = @name;\n function str = name()\n str = 'Product manifold: ';\n str = [str sprintf('[%s: %s]', ...\n elems{1}, elements.(elems{1}).name())];\n for i = 2 : nelems\n str = [str sprintf(' x [%s: %s]', ...\n elems{i}, elements.(elems{i}).name())]; %#ok\n end\n end\n \n M.dim = @dim;\n function d = dim()\n d = 0;\n for i = 1 : nelems\n d = d + elements.(elems{i}).dim();\n end\n end\n \n M.inner = @inner;\n function val = inner(x, u, v)\n val = 0;\n for i = 1 : nelems\n val = val + elements.(elems{i}).inner(x.(elems{i}), ...\n u.(elems{i}), v.(elems{i}));\n end\n end\n\n M.norm = @(x, d) sqrt(M.inner(x, d, d));\n\n M.dist = @dist;\n function d = dist(x, y)\n sqd = 0;\n for i = 1 : nelems\n sqd = sqd + elements.(elems{i}).dist(x.(elems{i}), ...\n y.(elems{i}))^2;\n end\n d = sqrt(sqd);\n end\n \n M.typicaldist = @typicaldist;\n function d = typicaldist\n sqd = 0;\n for i = 1 : nelems\n sqd = sqd + elements.(elems{i}).typicaldist()^2;\n end\n d = sqrt(sqd);\n end\n\n M.proj = @proj;\n function v = proj(x, u)\n for i = 1 : nelems\n v.(elems{i}) = elements.(elems{i}).proj(x.(elems{i}), ...\n u.(elems{i}));\n end\n end\n\n M.tangent = @tangent;\n function v = tangent(x, u)\n for i = 1 : nelems\n v.(elems{i}) = elements.(elems{i}).tangent(x.(elems{i}), ...\n u.(elems{i}));\n end\n end\n\n M.tangent2ambient = @tangent2ambient;\n function v = tangent2ambient(x, u)\n for i = 1 : nelems\n if isfield(elements.(elems{i}), 'tangent2ambient')\n v.(elems{i}) = ...\n elements.(elems{i}).tangent2ambient( ...\n x.(elems{i}), u.(elems{i}));\n else\n v.(elems{i}) = u.(elems{i});\n end\n end\n end\n\n M.egrad2rgrad = @egrad2rgrad;\n function g = egrad2rgrad(x, g)\n for i = 1 : nelems\n g.(elems{i}) = elements.(elems{i}).egrad2rgrad(...\n x.(elems{i}), g.(elems{i}));\n end\n end\n\n M.ehess2rhess = @ehess2rhess;\n function h = ehess2rhess(x, eg, eh, h)\n for i = 1 : nelems\n h.(elems{i}) = elements.(elems{i}).ehess2rhess(...\n x.(elems{i}), eg.(elems{i}), eh.(elems{i}), h.(elems{i}));\n end\n end\n \n M.exp = @exp;\n function y = exp(x, u, t)\n if nargin < 3\n t = 1.0;\n end\n for i = 1 : nelems\n y.(elems{i}) = elements.(elems{i}).exp(x.(elems{i}), ...\n u.(elems{i}), t);\n end\n end\n \n M.retr = @retr;\n function y = retr(x, u, t)\n if nargin < 3\n t = 1.0;\n end\n for i = 1 : nelems\n y.(elems{i}) = elements.(elems{i}).retr(x.(elems{i}), ...\n u.(elems{i}), t);\n end\n end\n \n M.log = @log;\n function u = log(x1, x2)\n for i = 1 : nelems\n u.(elems{i}) = elements.(elems{i}).log(x1.(elems{i}), ...\n x2.(elems{i}));\n end\n end\n\n M.hash = @hash;\n function str = hash(x)\n str = '';\n for i = 1 : nelems\n str = [str elements.(elems{i}).hash(x.(elems{i}))]; %#ok\n end\n str = ['z' hashmd5(str)];\n end\n\n M.lincomb = @lincomb;\n function v = lincomb(x, a1, u1, a2, u2)\n if nargin == 3\n for i = 1 : nelems\n v.(elems{i}) = elements.(elems{i}).lincomb(x.(elems{i}), ...\n a1, u1.(elems{i}));\n end\n elseif nargin == 5\n for i = 1 : nelems\n v.(elems{i}) = elements.(elems{i}).lincomb(x.(elems{i}), ...\n a1, u1.(elems{i}), a2, u2.(elems{i}));\n end\n else\n error('Bad usage of productmanifold.lincomb');\n end\n end\n\n M.rand = @rand;\n function x = rand()\n for i = 1 : nelems\n x.(elems{i}) = elements.(elems{i}).rand();\n end\n end\n\n M.randvec = @randvec;\n function u = randvec(x)\n for i = 1 : nelems\n u.(elems{i}) = elements.(elems{i}).randvec(x.(elems{i}));\n end\n u = M.lincomb(x, 1/sqrt(nelems), u);\n end\n\n M.zerovec = @zerovec;\n function u = zerovec(x)\n for i = 1 : nelems\n u.(elems{i}) = elements.(elems{i}).zerovec(x.(elems{i}));\n end\n end\n\n M.transp = @transp;\n function v = transp(x1, x2, u)\n for i = 1 : nelems\n v.(elems{i}) = elements.(elems{i}).transp(x1.(elems{i}), ...\n x2.(elems{i}), u.(elems{i}));\n end\n end\n\n M.pairmean = @pairmean;\n function y = pairmean(x1, x2)\n for i = 1 : nelems\n y.(elems{i}) = elements.(elems{i}).pairmean(x1.(elems{i}), ...\n x2.(elems{i}));\n end\n end\n\n\n % Gather the length of the column vector representations of tangent\n % vectors for each of the manifolds. Raise a flag if any of the base\n % manifolds has no vec function available.\n vec_available = true;\n vec_lens = zeros(nelems, 1);\n for ii = 1 : nelems\n Mi = elements.(elems{ii});\n if isfield(Mi, 'vec')\n rand_x = Mi.rand();\n zero_u = Mi.zerovec(rand_x);\n vec_lens(ii) = length(Mi.vec(rand_x, zero_u));\n else\n vec_available = false;\n break;\n end\n end\n vec_pos = cumsum([1 ; vec_lens]);\n \n if vec_available\n M.vec = @vec;\n M.mat = @mat;\n end\n \n function u_vec = vec(x, u_mat)\n u_vec = zeros(vec_pos(end)-1, 1);\n for i = 1 : nelems\n range = vec_pos(i) : (vec_pos(i+1)-1);\n u_vec(range) = elements.(elems{i}).vec(x.(elems{i}), ...\n u_mat.(elems{i}));\n end\n end\n\n function u_mat = mat(x, u_vec)\n u_mat = struct();\n for i = 1 : nelems\n range = vec_pos(i) : (vec_pos(i+1)-1);\n u_mat.(elems{i}) = elements.(elems{i}).mat(x.(elems{i}), ...\n u_vec(range));\n end\n end\n\n vecmatareisometries = true;\n for ii = 1 : nelems\n if ~isfield(elements.(elems{ii}), 'vecmatareisometries') || ...\n ~elements.(elems{ii}).vecmatareisometries()\n vecmatareisometries = false;\n break;\n end\n end\n M.vecmatareisometries = @() vecmatareisometries; \n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/smallestinconvexhull.m", "content": "function [u_norm, coeffs, u] = smallestinconvexhull(M, x, U, tol)\n% Computes a minimal norm convex combination of given tangent vectors in Manopt.\n%\n% function [u_norm, coeffs, u] = smallestinconvexhull(M, x, U)\n% function [u_norm, coeffs, u] = smallestinconvexhull(M, x, U, tol)\n%\n% M is a manifold as returned by a Manopt factory.\n% x is a point on this manifold.\n% U is a cell containing N tangent vectors U{1} to U{N} at x.\n% tol (default: 1e-8): tolerance for solving the quadratic program.\n% \n% This function computes u, a tangent vector at x contained in the convex\n% hull spanned by the N vectors U{i}, with minimal norm (according to the\n% Riemannian metric on M). This is obtained by solving a convex quadratic\n% program involving the Gram matrix of the given tangent vectors.\n% The quadratic program is solved using Matlab's built-in quadprog,\n% which requires the optimization toolbox. If this toolbox is not\n% available, consider replacing with CVX for example.\n%\n%\n% u_norm is the norm of the smallest vector u.\n% coeffs is a vector of length N with entries in [0, 1] summing to 1.\n% u is the sought vector: u = coeffs(1)*U{1} + ... + coeffs(N)*U{N}.\n%\n% Nicolas Boumal, Feb. 19, 2013\n% Modified April 6, 2016 to work with Manopt.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, June 28, 2016.\n% Contributors: \n% Change log: \n%\n% June 28, 2016 (NB):\n% Adapted for Manopt from original code by same author (Feb. 19, 2013)\n\n% Example code: pick a manifold, a point, and a collection of tangent\n% vectors at that point, then get the smallest vector in the convex hull\n% of those:\n% \n% M = spherefactory(5);\n% x = M.rand();\n% N = 3;\n% U = cell(N,1);\n% for k = 1 : N, U{k} = M.randvec(x); end\n% [u_norm, coeffs, u] = smallestinconvexhull(M, x, U)\n\n % We simply need to solve the following quadratic program:\n % minimize ||u||^2 such that u = sum_i s_i U_i, 0 <= s_i <= 1\n % and sum_i s_i = 1\n %\n % This is equivalent to solving:\n % min s'*G*s s.t. 0 <= s <= 1, s'*ones = 1, with G(i, j) = (Gram matrix)\n % Then our solution is s_1 U_1 + ... + s_N U_N.\n \n \n % Compute the Gram matrix of the given tangent vectors\n N = numel(U);\n G = grammatrix(M, x, U);\n \n % Solve the quadratic program.\n % If the optimization toolbox is not available, consider replacing with\n % CVX.\n \n if ~exist('tol', 'var') || isempty(tol)\n tol = 1e-8;\n end\n \n opts = optimset('Display', 'off', 'TolFun', tol);\n [s_opt, cost_opt] ...\n = quadprog(G, zeros(N, 1), ... % objective (squared norm)\n [], [], ... % inequalities (none)\n ones(1, N), 1, ... % equality (sum to 1)\n zeros(N, 1), ... % lower bounds (s_i >= 0)\n ones(N, 1), ... % upper bounds (s_i <= 1)\n [], ... % we do not specify an initial guess\n opts);\n\n % Norm of the smallest tangent vector in the convex hull:\n u_norm = real(sqrt(2*cost_opt));\n\n % Keep track of optimal coefficients\n coeffs = s_opt;\n \n % If required, construct the vector explicitly.\n if nargout >= 3\n u = lincomb(M, x, U, coeffs);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/statsfunhelper.m", "content": "function statsfun = statsfunhelper(inp1, inp2)\n% Helper tool to create a statsfun for the options structure of solvers.\n%\n% function statsfun = statsfunhelper(name, fun)\n% function statsfun = statsfunhelper(S)\n%\n% Usage with (name, fun):\n%\n% Input 1: name is a string which is a valid field name (no spaces, starts\n% with a letter or an underscore, only alphanumeric characters and\n% underscores).\n% \n% Input2: fun is a function handle with one output and 1 to 4 inputs, as\n% follows (your choice):\n% \n% fun(x) or fun(problem, x) or \n% fun(problem, x, stats) or fun(problem, x, stats, store)\n% \n% where the inputs are the ones that would be given to options.statsfun, as\n% described in the help of the solver used. Typically, x is the point on\n% the manifold at the current iterate, problem is the Manopt problem\n% structure, stats is all the current statistics recorded for that iterate\n% and store is the cache structure at the current iterate.\n%\n% When calling a Manopt solver with the options structure, such as for\n% example with:\n%\n% [x, xcost, info] = steepestdescent(problem, [], options);\n%\n% you may set a field of the options structure as follows:\n%\n% options.statsfun = statsfunhelper('nameofthefield', fun);\n%\n% As a result, at each iteration, the stats structure will contain a field\n% stats.nameofthefield with the value returned by the call to fun at that\n% iterate. The stats structures are stored in the struct-array info.\n% As an example, if the value returned by fun is a scalar, then\n% [info.nameofthefield] is a vector containing all returned values.\n%\n%\n% Usage with S:\n%\n% The input S is a structure. For each field of S, say S.field, the stats\n% structure will be augmented with stats.field = fun(..), where fun is the\n% function handle stored in S.field, and with the same conventions as\n% above. This version allows to record more than one bit of information at\n% each iteration. Example:\n% \n% metrics.nameofthefield = fun;\n% metrics.othername = otherfun;\n% options.statsfun = statsfunhelper(metrics);\n%\n% The different function handles (here, fun and otherfun) can take 1 to 4\n% inputs too, and they do not have to take the same number of inputs.\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Dec. 17, 2014.\n% Contributors: \n% Change log: \n\n if (nargin == 1) && isstruct(inp1)\n S = inp1;\n elseif (nargin == 2)\n S = struct(inp1, inp2);\n else\n error('statsfunhelper takes 1 or 2 inputs. If 1 input, it must be a structure.');\n end\n\n\n function stats = thestatsfun(problem, x, stats, store)\n names = fieldnames(S);\n for it = 1 : length(names)\n name = names{it};\n fun = S.(name);\n switch nargin(fun)\n case 1\n stats.(name) = fun(x);\n case 2\n stats.(name) = fun(problem, x);\n case 3\n stats.(name) = fun(problem, x, stats);\n case 4\n stats.(name) = fun(problem, x, stats, store);\n otherwise\n error('The functions passed to statsfunhelper must take 1 to 4 inputs.');\n end\n end\n end\n\n statsfun = @thestatsfun;\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/surfprofile.m", "content": "function costs = surfprofile(problem, x, d1, d2, t1, t2)\n% Plot the cost function as a surface over a 2-dimensional subspace.\n%\n% function surfprofile(problem, x, d1, d2, t1, t2)\n% function costs = surfprofile(problem, x, d1, d2, t1, t2)\n%\n% Evaluates the cost function at points\n%\n% gamma(t1, t2) = exponential_x(t1*d1 + t2*d2)\n% \n% where the exponential map at x is specified by problem.M.exp (retr is\n% used instead if needed). d1 and d2 are two tangent vectors to problem.M\n% at the point x. The values assigned to t1 and t2 are as specified in the\n% two input vectors t1 and t2.\n% \n% If the function is called with an output, the plot is not drawn and the\n% values of the cost are returned in a matrix of size\n% length(t1)*length(t2). To plot a surf, call surf(t1, t2, costs.') (notice\n% the transpose).\n%\n% If x is omitted, a point is generated at random. If d1 is omitted, a\n% random tangent vector at x is generated. If d2 is omitted, a random\n% tangent vector at x is generated, orthogonally to d1. If t1, t2 are\n% omitted, they are generated with linspace's in [-1, 1].\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Sep. 1, 2014.\n% Contributors: \n% Change log: \n%\n% April 3, 2015 (NB):\n% Works with the new StoreDB class system.\n%\n% Nov. 12, 2016 (NB):\n% Most inputs are now optional.\n\n % Verify that the problem description is sufficient.\n if ~canGetCost(problem)\n error('It seems no cost was provided.'); \n end\n \n\n if ~exist('x', 'var') || isempty(x)\n x = problem.M.rand();\n if (exist('d1', 'var') && ~isempty(d1)) || ...\n (exist('d2', 'var') && ~isempty(d2))\n error('If x is omitted, d1, d2 should not be specified.');\n end\n end\n if ~exist('d1', 'var') || isempty(d1)\n d1 = problem.M.randvec(x);\n end\n if ~exist('d2', 'var') || isempty(d2)\n d2 = problem.M.randvec(x);\n % Make it orthogonal to d1\n coeff = problem.M.inner(x, d1, d2) / problem.M.inner(x, d1, d1);\n d2 = problem.M.lincomb(x, 1, d2, -coeff, d1);\n end\n if ~exist('t1', 'var') || isempty(t1)\n t1 = linspace(-1, 1, 51);\n end\n if ~exist('t2', 'var') || isempty(t2)\n t2 = linspace(-1, 1, 51);\n end\n \n \n if isfield(problem.M, 'exp')\n expo = problem.M.exp;\n str = 'Exp';\n else\n expo = problem.M.retr;\n str = 'Retr';\n end\n \n storedb = StoreDB();\n linesearch_fun = @(ta, tb) getCost(problem, ...\n expo(x, problem.M.lincomb(x, ta, d1, tb, d2)), ...\n storedb);\n \n costs = zeros(length(t1), length(t2));\n for i = 1 : length(t1)\n for j = 1 : length(t2)\n costs(i, j) = linesearch_fun(t1(i), t2(j));\n end\n end\n \n if nargout == 0\n surf(t1, t2, costs.');\n xlabel('t1');\n ylabel('t2');\n zlabel(['f(' str '_x(t1*d1+t2*d2))']);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/tangent2vec.m", "content": "function vec = tangent2vec(M, x, basis, u)\n% Expands a tangent vector into an orthogonal basis in the Manopt framework\n%\n% vec = tangent2vec(M, x, basis, u)\n%\n% The inverse operation is lincomb (see below).\n%\n% M is a Manopt manifold structure obtained from a factory.\n% x is a point on the manifold M.\n% basis is a cell containing n orthonormal tangent vectors at x, forming an\n% orthonormal basis of the tangent space at x.\n% u is a tangent vector at x\n%\n% vec is a column vector of length n which contains the coefficients of the\n% expansion of u into the basis. Thus:\n%\n% vec(k) = _x <- vec = tangent2vec(M, x, basis, u)\n%\n% u = sum_{k=1}^n vec(k)*basis{k} <- u = lincomb(M, x, basis, vec)\n%\n% Note that tangent2vec is an isometry, that is, up to numerical round-off\n% errors, with u and v two tangent vectors at x:\n%\n% M.inner(x, u, v) == uu'*vv,\n%\n% where uu = tangent2vec(M, x, basis, u), vv = tangent2vec(M, x, basis, v).\n%\n% See also: lincomb tangentorthobasis orthogonalize grammatrix hessianmatrix\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, Feb. 3, 2017.\n% Contributors: \n% Change log: \n\n\n n = numel(basis);\n \n vec = zeros(n, 1);\n \n for k = 1 : n\n \n vec(k) = M.inner(x, basis{k}, u);\n \n end\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/tangentorthobasis.m", "content": "function orthobasis = tangentorthobasis(M, x, n)\n% Returns an orthonormal basis of tangent vectors in the Manopt framework.\n%\n% function orthobasis = tangentorthobasis(M, x)\n% function orthobasis = tangentorthobasis(M, x, n)\n%\n% M is a Manopt manifold structure obtained from a factory.\n% x is a point on the manifold M.\n% n (optional) is the dimension of the random subspace to span; by default,\n% n = M.dim() so that the returned basis spans the whole tangent space.\n%\n% orthobasis is a cell of n tangent vectors at x.\n% With high probability, they form an orthonormal basis of the tangent\n% space at x. If necessary, this can be checked by calling\n% G = grammatrix(M, x, orthobasis)\n% and verifying that norm(G - eye(size(G))) is close to zero.\n%\n% Note: if extra accuracy is required, it may help to re-orthogonalize the\n% basis returned by this function once, as follows:\n% B = tangentorthobasis(M, x, n);\n% B = orthogonalize(M, x, B);\n%\n% See also: grammatrix orthogonalize lincomb plotprofile\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 28, 2016.\n% Contributors: \n% Change log: \n\n dim = M.dim();\n if ~exist('n', 'var') || isempty(n)\n n = dim;\n end\n assert(n >= 0 && n <= dim && n == round(n), ...\n 'n must be an integer between 0 and M.dim().');\n \n basis = cell(n, 1);\n \n % With high probability, n vectors taken at random in the tangent space\n % are linearly independent.\n for k = 1 : n\n basis{k} = M.randvec(x);\n end\n \n % The Gram-Schmidt process transforms any n linearly independent\n % vectors into n orthonormal vectors spanning the same subspace.\n orthobasis = orthogonalize(M, x, basis);\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/tangentspacefactory.m", "content": "function N = tangentspacefactory(M, x)\n% Returns a manifold structure representing the tangent space to M at x.\n%\n% N = tangentspacefactory(M, x)\n%\n% N defines a (linear) manifold that is the tangent space to M at x. Points\n% are represented as tangent vectors to M at x. Tangent vectors are also\n% represented as tangent vectors to M at x.\n%\n% This is chiefly useful to solve optimization problems involving tangent\n% vectors to M at x, which notably comes up when solving linear systems\n% involving, for example, the Hessian of the cost on M at x (think of the\n% Newton equations.) The Riemannian (actually, Euclidean) structure on N is\n% that of the tangent space to M, that is, the inner product is inherited.\n%\n% See also: preconhessiansolve\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, April 9, 2015.\n% Contributors: \n% Change log: \n%\n% Jan. 25, 2017 (NB):\n% Following a comment by Jesus Briales on the Manopt forum, the\n% functions N.egrad2rgrad, N.ehess2rhess and N.tangent now include a\n% projection (they were formerly identities.)\n%\n% Feb. 2, 2017 (NB):\n% Following a comment by Jesus Briales on the Manopt forum, the\n% function N.proj now calls M.proj(x, .) instead of M.proj(y, .).\n% Furthermore, N.ehess2rhess was corrected in the same way.\n\n % N is the manifold we build. y will be a point on N, thus also a\n % tangent vector to M at x. This is a typical Euclidean space, hence it\n % will be easy to describe in terms of the tools available for M.\n N = struct();\n \n % u, u1 and u2 will be tangent vectors to N at y. The tangent space to\n % N at y is the tangent space to M at x, thus u, u1 and u2 are also\n % tangent vectors to M at x.\n \n N.dim = @() M.dim();\n N.inner = @(y, u1, u2) M.inner(x, u1, u2);\n N.norm = @(y, u) M.norm(x, u);\n N.proj = @(y, u) M.proj(x, u);\n N.typicaldist = @() sqrt(N.dim());\n N.tangent = N.proj;\n N.egrad2rgrad = N.proj;\n N.ehess2rhess = @(y, eg, eh, d) M.proj(x, eh);\n N.exp = @exponential;\n N.retr = @exponential;\n N.log = @(y1, y2) M.lincomb(x, 1, y2, -1, y1);\n N.pairmean = @(y1, y2) M.lincomb(x, 0.5, y1, 0.5, y2);\n N.rand = @() M.randvec(x);\n N.randvec = @(y) M.randvec(x);\n N.zerovec = M.zerovec;\n N.lincomb = M.lincomb;\n N.transp = @(y1, y2, u) u;\n N.hash = @(y) ['z' hashmd5(M.vec(x, y))];\n \n % In a Euclidean space, the exponential is merely the sum: y + tu.\n function yy = exponential(y, u, t)\n if nargin == 2\n t = 1;\n end\n yy = M.lincomb(x, 1, y, t, u);\n end\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt/tools/tangentspherefactory.m", "content": "function N = tangentspherefactory(M, x)\n% Returns a manifold struct. for the sphere on the tangent space to M at x.\n%\n% N = tangentspherefactory(M, x)\n%\n% N defines a manifold that is the unit sphere on the tangent space to M\n% at x. Points are represented as tangent vectors of unit norm. Tangent\n% vectors are represented as tangent vectors orthogonal to the root point,\n% with respect to the Riemannian metric on the tangent space.\n%\n% This is chiefly useful to solve optimization problems involving unit norm\n% tangent vectors to M at x, which notably comes up when looking for\n% extreme eigenvectors of the Hessian of a cost function on M at x, for\n% example. The Riemannian structure on this sphere is that of a Riemannian\n% submanifold of the (Euclidean) tangent space, equipped with the\n% Riemannian metric of M at that point.\n%\n% See also: hessianextreme\n\n% This file is part of Manopt: www.manopt.org.\n% Original author: Nicolas Boumal, March 16, 2015.\n% Contributors: \n% Change log: \n%\n% Nov 27, 2015 (NB):\n% Extra projection added in the retraction, to prevent numerical\n% drift.\n\n % N is the manifold we build. y will be a point on N, thus also a\n % tangent vector to M at x. This is a typical Riemannian submanifold of\n % a Euclidean space, hence it will be easy to describe in terms of the\n % tools available for M.\n N = struct();\n \n % u, u1 and u2 will be tangent vectors to N at y. The tangent space to\n % N at y is a subspace of the tangent space to M at x, thus u, u1 and\n % u2 are also tangent vectors to M at x.\n \n N.dim = @() M.dim() - 1;\n N.inner = @(y, u1, u2) M.inner(x, u1, u2);\n N.norm = @(y, u) M.norm(x, u);\n N.proj = @(y, v) M.lincomb(x, 1, v, -M.inner(x, v, y), y);\n N.typicaldist = @() 1;\n N.tangent = N.proj;\n N.egrad2rgrad = N.proj;\n N.retr = @retraction;\n N.exp = N.retr;\n function yy = retraction(y, u, t)\n if nargin == 2\n t = 1;\n end\n y_plus_tu = M.lincomb(x, 1, y, t, u);\n % This extra projection is not required mathematically,\n % but appears to be necessary numerically, sometimes.\n % The reason is that, as many retractions are operated,\n % there is a risk that the points generated would leave\n % the tangent space. If this proves to be a huge slow down,\n % one could consider adding a type of counter that only\n % executes this extra projection every so often, instead\n % of at every call.\n y_plus_tu = M.proj(x, y_plus_tu);\n nrm = M.norm(x, y_plus_tu);\n yy = M.lincomb(x, 1/nrm, y_plus_tu);\n end\n N.rand = @random;\n function y = random()\n y = M.randvec(x);\n nrm = M.norm(x, y);\n y = M.lincomb(x, 1/nrm, y);\n end\n N.randvec = @randvec;\n function u = randvec(y)\n u = N.proj(y, N.rand());\n nrm = N.norm(y, u);\n u = M.lincomb(x, 1/nrm, u);\n end\n N.zerovec = M.zerovec;\n N.lincomb = M.lincomb;\n N.transp = @(y1, y2, u) N.proj(y2, u);\n N.hash = @(y) ['z' hashmd5(M.vec(x, y))];\n \nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/manopt_version.m", "content": "function [version, released] = manopt_version()\n% Returns the version of the Manopt package you are running, as a vector.\n%\n% function [version, released] = manopt_version()\n%\n% version(1) is the primary version number.\n% released is the date this version was released, in the same format as the\n% date() function in Matlab.\n\n version = [4, 0, 0];\n released = '09-Sep-2017';\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/manopt/readme", "content": "test\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR/readme", "content": "test\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/RR_Assessment.tex", "content": "\\begin{tabular}{|l|c|c|c|}\n\\hline\n&\\textbf{Q2n}&\\textbf{SAM}&\\textbf{ERGAS}\\\\\\hline\n\\textbf{GT}&1.0000&0.0000&0.0000\\\\\\hline\n\\textbf{EXP}&0.6513&7.2118&8.1106\\\\\\hline\n\\textbf{BT-H}&0.9241&6.4530&3.9714\\\\\\hline\n\\textbf{BDSD-PC}&0.9327&6.8388&3.8905\\\\\\hline\n\\textbf{C-GSA}&0.9213&6.6967&4.0504\\\\\\hline\n\\textbf{SR-D}&0.9113&6.6269&4.3472\\\\\\hline\n\\textbf{MTF-GLP-HPM-R}&0.9228&7.0038&4.0692\\\\\\hline\n\\textbf{MTF-GLP-FS}&0.9228&6.7650&4.0434\\\\\\hline\n\\textbf{TV}&0.9277&6.6213&4.0630\\\\\\hline\n\\textbf{PanNet}&0.9238&6.9050&4.2365\\\\\\hline\n\\textbf{DRPNN}&0.9205&7.3887&4.2504\\\\\\hline\n\\textbf{MSDCNN}&0.9087&7.5139&4.4214\\\\\\hline\n\\textbf{BDPN}&0.9180&7.7148&4.4522\\\\\\hline\n\\textbf{DiCNN}&0.8567&8.0256&5.5124\\\\\\hline\n\\textbf{PNN}&0.8849&12.6019&6.7233\\\\\\hline\n\\textbf{APNN}&0.9132&7.6201&4.4536\\\\\\hline\n\\textbf{FusionNet}&0.8499&8.3823&6.0458\\\\\\hline\n\\end{tabular}\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/SR-D/CS.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description:\n% CSDetails is the Compressive Sensing (CS) approach for Pansharpening proposed in [Vicinanza15].\n%\n% Interface:\n% I_Fus_CS = CSDetails(I_MS, I_PAN, I_MS_LR, resize_fact, sensor, TS, ol, n_atoms)\n%\n% Inputs:\n% I_MS: Multispectral (MS) original image upsampled to the PAN scale;\n% I_PAN: Panchromatic (PAN) image;\n% I_MS_LR: MS original image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% sensor: String for type of sensor (e.g. 'WV2', 'IKONOS');\n% TS: Tiling (dimensions of the patches are TS x TS, e.g. 7 x 7);\n% ol: Overlap in pixels between contiguous tile;\n% n_atoms: max number of representation atoms (default value = 10).\n%\n% Output:\n% I_Fus_CS: Fusion image using the CS approach in [Vicinanza15].\n%\n% References:\n% [Vicinanza15] M.R. Vicinanza, R. Restaino, G. Vivone, M. Dalla Mura, and J. Chanussot, \"A pansharpening method based on the sparse representation of injected details\",\n% IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 1, pp. 180-184, 2015.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n% % % % % % % % % % % % %\n%\n% Version: 1\n%\n% % % % % % % % % % % % %\n%\n% Copyright (C) 2019\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_CS = CS(I_MS, I_PAN, I_MS_LR, ratio, sensor, TS, ol, n_atoms)\n\nif nargin < 9\n n_atoms = 10;\nend\n\nimageLR = double(I_MS);\nimageHR = double(I_PAN);\nimageLR_LR = double(I_MS_LR);\n\n%%% Equalization\nimageHR = repmat (imageHR, [1 1 size(I_MS,3)]);\nfor ii = 1 : size(imageLR_LR,3)\n % imageHR(:,:,ii) = equalize_image (imageHR(:,:,ii), imageLR(:,:,ii));\n imageHR(:,:,ii) = (imageHR(:,:,ii) - mean2(imageHR(:,:,ii))) / std2(imageHR(:,:,ii))...\n * std2(imageLR(:,:,ii)) + mean2(imageLR(:,:,ii));\nend\n\n%%% Extract details using MTF-based filters\nimageLR_LP = MTF(imageLR, sensor, ratio);\nimageLR_D = imageLR - imageLR_LP;\nimageHR_LP = MTF(imageHR, sensor, ratio);\nfor ii = 1:size(imageHR,3)\n imageHR_LP(:,:,ii) = imresize(imresize(imageHR_LP(:,:,ii), 1/ratio, 'nearest'), ratio);\nend\nimageHR_D = imageHR - imageHR_LP;\n\n%%% Decimation MS\nfor ii = 1 : size(imageLR,3)\n imageLR_LR(:,:,ii) = double(imresize(imageLR_D(:,:,ii),1/ratio, 'nearest'));\nend\n\n%%% Degradation PAN\nimageHR_LR = resize_images(imageHR_D, 1, ratio, sensor);\n\n%%% Dictionary learning\n[Dh, Dl, ytilde_k] = Dict_Learn(imageHR_D, imageHR_LR, imageLR_LR, ratio, TS, ol);\n\n%%% Sparse coefficient estimation and HR signal reconstruction\nI_Fus_CS = OMP_Rec_Detile(Dl, Dh, ytilde_k, size(imageHR,1), size(imageHR,2), size(imageLR_LR, 3), ratio, ol , TS, n_atoms);\n\n\nI_Fus_CS = imageLR + I_Fus_CS;\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/SR-D/Dict_Learn.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Dict_Learn is the dictionary learning method for the \n% compressive sensing approach for Pansharpening proposed in [Vicinanza15].\n% \n% INPUTS\n% I_PAN_D: Details of the panchromatic image;\n% I_PAN_LR_D: Details of the low resolution panchromatic image;\n% I_MS_LR_D: Details of the MS original image or the MS original image (depending on the flag \"do_detail\" in CSDetails);\n% resize_fact: Resize factor (ratio between PAN and MS images);\n% TS: Tiling (dimensions of the patches are TS x TS, e.g. 7 x 7);\n% ol: Overlap in pixels between contiguous tiles.\n% \n% OUTPUTS\n% Dh: High spatial resolution dictionary (PAN details) built as in [Vicinanza15]; \n% Dl: Low spatial resolution dictionary (Low resolution PAN details) built as in [Vicinanza15];\n% ytilde_k: Patches in column form of the details of the MS original image or the MS original image (depending on the flag \"do_detail\" in CSDetails).\n% \n% REFERENCE\n% [Vicinanza15] M.R. Vicinanza, et al. \"A pansharpening method based on the sparse representation of injected details.\" \n% IEEE Geoscience and Remote Sensing Letters 12.1 (2015): 180-184.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [Dh, Dl, ytilde_k] = Dict_Learn(I_PAN_D, I_PAN_LR_D, I_MS_LR_D, resize_fact, TS, ol)\n\nnr = ceil ((size(I_PAN_D,1)/resize_fact - ol) / (TS - ol));\nnc = ceil ((size(I_PAN_D,2)/resize_fact - ol) / (TS - ol));\nnBands = size (I_MS_LR_D,3);\n\nDh = zeros (TS^2*resize_fact^2*nBands, nr*nc);\nDl = zeros (TS^2*nBands, nr*nc);\nytilde_k = zeros (TS^2*nBands, nr*nc);\n\n% Building the dictionaries (Dh and Dl)\nicount = 0;\nfor irow=1:nr\n for icol=1:nc\n icount = icount + 1;\n shiftr = 0; shiftc = 0;\n if irow == nr && mod(size(I_MS_LR_D,1)-ol, TS-ol) ~= 0\n shiftr = TS-ol - mod (size(I_MS_LR_D,1)-ol, TS-ol);\n end\n if icol == nc && mod(size(I_MS_LR_D,2)-ol, TS-ol) ~= 0\n shiftc = TS-ol - mod (size(I_MS_LR_D,2)-ol, TS-ol);\n end\n blockr = ((irow-1)*(TS-ol)*resize_fact+1 - shiftr*resize_fact) : ((irow*TS-(irow-1)*ol)*resize_fact - shiftr*resize_fact);\n blockc = ((icol-1)*(TS-ol)*resize_fact+1 - shiftc*resize_fact) : ((icol*TS-(icol-1)*ol)*resize_fact - shiftc*resize_fact);\n\n blockrl = ((irow-1)*(TS-ol)+1 - shiftr) : (irow*TS-(irow-1)*ol - shiftr);\n blockcl = ((icol-1)*(TS-ol)+1 - shiftc) : (icol*TS-(icol-1)*ol - shiftc);\n\n for iband = 1:nBands \n colmn = I_PAN_D(blockr,blockc,iband);\n colmnlr = I_PAN_LR_D(blockrl,blockcl,iband);\n colmny = I_MS_LR_D(blockrl,blockcl,iband);\n Dh((iband-1)*TS^2*resize_fact^2+1:(iband-1)*TS^2*resize_fact^2+length(colmn(:)),icount) = (colmn(:));\n Dl((iband-1)*TS^2+1:(iband-1)*TS^2+length(colmnlr(:)),icount) = (colmnlr(:));\n ytilde_k((iband-1)*TS^2+1:(iband-1)*TS^2+length(colmny(:)),icount) = (colmny(:));\n end\n end\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/SR-D/OMP.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% OMP is the Orthogonal matching Pursuit (OMP) modified to work with multispectral data.\n% \n% INPUTS\n% D: Dictionary (matrix);\n% y: Measurements (column vector);\n% delta: Maximum error allowed for the constraint y = D a;\n% nBands: Number of MS spectral bands;\n% iatom: Id of the actual atom under analysis.\n% n_atoms: max number of representation atoms\n%\n% OUTPUTS\n% a: Estimated alphas;\n% indx: Vector of the atom positions in the dictionary.\n% \n% REFERENCE\n% [Vicinanza15] M.R. Vicinanza, et al. \"A pansharpening method based on the sparse representation of injected details.\" \n% IEEE Geoscience and Remote Sensing Letters 12.1 (2015): 180-184.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [a, indx] = OMP(D, y, nBands, iatom, n_atoms)\n\n\nL_atom = size(D);\nn = round(L_atom / nBands);\ndelta = 0;\nres = y;\ncurr_delta = sum (res.^2);\nj = 0;\n\nwhile curr_delta > delta && j < n_atoms\n j = j+1;\n if j==1\n indx = iatom;\n else\n proj = D' * res;\n [~, imax] = max(abs(proj));\n imax = imax(1);\n indx = cat(2,indx,imax);\n end\n a = zeros (j, nBands);\n for iband = 1:nBands\n Di = D((iband-1)*n+1:iband*n,indx(1:j));\n yi = y((iband-1)*n+1:iband*n);\n DitDi = Di'*Di;\n if det (DitDi) > 1e-1\n a(:,iband) = ((DitDi)\\(Di')) * yi;\n end\n Da((iband-1)*n+1:iband*n) = Di * a(:,iband);\n end\n res = y - Da';\n curr_delta = sum(res.^2);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/SR-D/OMP_Rec_Detile.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% OMP_Rec_Detile performs:\n% 1) The estimation of the coefficients \\alpha at reduced resolution using an orthogonal matching pursuit (OMP) procedure for multispectral images;\n% 2) The reconstruction of the patches at full resolution using the hypothesis of invariance among scales of the \\alpha coefficients;\n% 3) The detiling step to get the final image details at full resolution for the approach proposed in [Vicinanza15].\n%\n% INPUTS\n% Dl: Low spatial resolution dictionary (Low resolution PAN details) built as in [Vicinanza15];\n% Dh: High spatial resolution dictionary (PAN details) built as in [Vicinanza15];\n% ytilde_k: Patches in column form of the details of the MS original image or the MS original image (depending on the flag \"do_detail\" in CSDetails);\n% H_PAN,L_PAN,C_PAN: PAN (row and column) dimensions and number of MS spectral bands;\n% resize_fact: Resize factor (ratio between PAN and MS images);\n% TS: Tiling (dimensions of the patches are TS x TS, e.g. 7 x 7);\n% ol: Overlap in pixels between contiguous tiles.\n% n_atoms: max number of representation atoms\n%\n% OUTPUT\n% I_Fus_CS: Reconstructed details (or fused image if do_detail flag is 0) using the CS approach in [Vicinanza15] for the final pansharpening product.\n%\n% REFERENCE\n% [Vicinanza15] M.R. Vicinanza, et al. \"A pansharpening method based on the sparse representation of injected details.\"\n% IEEE Geoscience and Remote Sensing Letters 12.1 (2015): 180-184.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Fus_CS = OMP_Rec_Detile(Dl, Dh, ytilde_k, H_PAN, L_PAN, C_MS, resize_fact, ol, TS, n_atoms)\n\nI_Fus_CS = zeros ([H_PAN L_PAN C_MS]);\ncountpx = zeros ([H_PAN L_PAN C_MS]);\nnr = ceil ((H_PAN/resize_fact - ol) / (TS - ol));\nnc = ceil ((L_PAN/resize_fact - ol) / (TS - ol));\nshiftr_glob = 0; shiftc_glob = 0;\n\nif mod(H_PAN/resize_fact-ol, TS-ol) ~= 0\n shiftr_glob = TS-ol - mod (H_PAN/resize_fact-ol, TS-ol);\nend\n\nif mod(L_PAN/resize_fact-ol, TS-ol) ~= 0\n shiftc_glob = TS-ol - mod (L_PAN/resize_fact-ol, TS-ol);\nend\n\nalpha_count = 0;\nLatom = size (Dl, 2);\nDict_Size = size (ytilde_k, 2);\niatom = 0;\nfor irow=1:nr\n for icol=1:nc\n iatom = iatom+1;\n if irow == nr\n shiftr = shiftr_glob;\n else\n shiftr = 0;\n end\n if icol == nc\n shiftc = shiftc_glob;\n else\n shiftc = 0;\n end\n blockr = ((irow-1)*(TS-ol)*resize_fact+1 - shiftr*resize_fact) : ((irow*TS-(irow-1)*ol)*resize_fact - shiftr*resize_fact);\n blockc = ((icol-1)*(TS-ol)*resize_fact+1 - shiftc*resize_fact) : ((icol*TS-(icol-1)*ol)*resize_fact - shiftc*resize_fact);\n Lr = length (blockr); Lc = length (blockc);\n y_cur = ytilde_k(:,iatom);\n \n % Sparse coding with OMP for MS data\n [alpha,inds] = OMP(Dl, y_cur, C_MS, iatom, n_atoms);\n\n % Patch reconstruction and detiling\n for iband = 1:C_MS\n reconstr_patch = Dh((iband-1)*TS^2*resize_fact^2+1:iband*TS^2*resize_fact^2,inds) * alpha(:,iband);\n I_Fus_CS(blockr,blockc,iband) = I_Fus_CS(blockr,blockc,iband) + reshape (reconstr_patch, Lr, Lc);\n countpx(blockr,blockc,iband) = countpx(blockr,blockc,iband) +1;\n end\n \n if mod(iatom,100)==1\n fprintf ('OMP band by band and detile: atom %i of %i\\n', iatom, Dict_Size);\n end\n alpha_count = alpha_count + sum( sum(alpha,2)~=0 );\n end\nend\n\n% Average overlapping patches\nI_Fus_CS = I_Fus_CS ./ countpx;\n\nfprintf ('Sparsity di alfa = %.2f: %.1f atoms on %i used for each patch on average\\n', (Dict_Size*Latom-alpha_count)/Dict_Size/Latom*100, alpha_count/Dict_Size, Dict_Size)\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/TV/TV_pansharpen.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% This function minimizes \n% J(x) = || y - M*x ||^2 + lambda*TV(x)\n% where\n% y = [yms^T, ypan^T]^T\n% x is the pansharpened ms image\n% M models the relationship between\n% y and x; see [Palsson07] for details\n% \n% Interface:\n% x = TV_pansharpen(yms,ypan,alpha,lambda,c,maxiter,w)\n%\n% Inputs:\n% yms: The observed MS image;\n% ypan: The PAN image;\n% alpha: convergence parameter 1, suggested value=0.75;\n% c: convergence parameter 2, suggested value=8;\n% maxiter: number of iterations;\n% w: We assume the pan image to be a linear \n% combination of the pansharpened ms image,\n% w contains the weights. \n% Output:\n% x: Pansharpened image.\n% \n% Reference:\n% [Palsson14] F. Palsson, J.R. Sveinsson, and M.O. Ulfarsson, A New Pansharpening Algorithm Based on Total Variation\n% IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 1, pp. 318 - 322, 2014.\n% [Vivone20] G. Vivone, M. Dalla Mura, A. Garzelli, R. Restaino, G. Scarpa, M.O. Ulfarsson, L. Alparone, and J. Chanussot, \"A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods\", \n% IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3019315.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction x = TV_pansharpen(yms,ypan,alpha,lambda,c,maxiter,w)\n \n z=zeros([size(ypan) size(yms,3)*2]);\n x=zeros([size(ypan) size(yms,3)]);\n\n for k=1:maxiter\n b=computeb(yms,ypan,x,alpha,w);\n z=znext(z,x,b,alpha,lambda,c);\n x=xnext(z,b,alpha);\n end\nend\n\nfunction b=computeb(yms,ypan,xk,alpha,w)\n [Hxms, Hxpan]=computeH(xk,w);\n b=alpha*xk+adjointH(yms-Hxms,ypan-Hxpan,w);\nend\n\nfunction [yms, ypan]=computeH(x,w)\n\n ypan=zeros([size(x,1) size(x,2)]);\n\n for i=1:size(x,3)\n yms(:,:,i)=decimate(x(:,:,i));\n ypan=ypan+w(i)*x(:,:,i);\n end\nend\n\nfunction y=decimate(x)\n % y = imfilter(x,fspecial('Gaussian',9,sigma),'replicate');\n % y = imfilter(y,fspecial('average',4),'replicate');\n % y = y(1:4:end,1:4:end);\n % h=0.25*[1 1 1 1];\n % x=imfilter(x,h'*h,'symmetric','same');\n % y=downsample(downsample(x,4,1)',4,1)';\n y=imresize(x,0.25,'bilinear');\n % y=MTF_downsample(x,'QB','none',4,1);\n % y=imresize(imresize(x,1/4,'bicubic'),4,'bicubic');\nend\n \nfunction x=adjointH(yms,ypan,w)\n for i=1:size(yms,3)\n x(:,:,i)=interpolate(yms(:,:,i))+w(i)*ypan;\n end\nend\n\nfunction y=interpolate(x)\n % y = upsample(upsample(x,4)',4)';\n y=imresize(x,4,'bilinear');\n % y = imfilter(y,fspecial('Gaussian',9,sigma),'replicate');\n % y = imfilter(y,fspecial('average',4),'replicate');\n % y=imresize(x,4,'bicubic');\n % y=MTF_upsample(x,'IKONOS','none',4,1);\n % y=interp23tap(x,4);\nend\n\nfunction z1=znext(z0,x0,b,alpha,lambda,c)\n for i=1:size(x0,3)\n W(:,:,i)= 2* alpha/lambda * sqrt(Dx(x0(:,:,i)).^2+Dy(x0(:,:,i)).^2)+c;\n W(:,:,i+size(x0,3))=2 * alpha/lambda * sqrt(Dx(x0(:,:,i)).^2+Dy(x0(:,:,i)).^2)+c;\n end\n z1=(computeDb(b)+cIDDTz(z0,c))./W;\nend\n\nfunction DX = Dx(v) \n DX=[diff(v,1,2) zeros(size(v,1),1)];\nend\n\nfunction DY = Dy(v) \n DY=[diff(v); zeros(1,size(v,2))];\nend\n\nfunction Db=computeDb(b)\n\n for i=1:size(b,3)\n Db(:,:,i)=Dx(b(:,:,i));\n end\n for i=size(b,3)+1:2*size(b,3)\n Db(:,:,i)=Dy(b(:,:,i-size(b,3)));\n end\nend\n\nfunction ddtz=cIDDTz(z,c)\n\n for i=1:size(z,3)/2\n dtz(:,:,i)=DxT(z(:,:,i))+DyT(z(:,:,i+4));\n end\n\n ddtz=computeDb(dtz);\n cIddtz=c*z-ddtz;\nend\n\nfunction DXT=DxT(v)\n DXT=DyT(v')';\nend\n\nfunction DYT = DyT(v)\n\n u0 = -v(1,:);\n u1 = -diff(v);\n u2 = v(end-1,:);\n DYT = [u0; u1(1:(end-1),:); u2];\n return\nend\n\nfunction x1=xnext(z1,b,alpha)\n x1=(b-DTz(z1))./alpha;\nend\n\nfunction dtz=DTz(z)\n for i=1:size(z,3)/2\n dtz(:,:,i)=DxT(z(:,:,i))+DyT(z(:,:,i+4));\n end\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/LPfilter.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% LPfilter filters the panchromatic (PAN) image using trous wavelet transform. \n% \n% Interface:\n% HRPanLP = LPfilter(HRPan,ratio)\n%\n% Inputs:\n% HRPan: PAN image;\n% ratio: Scale ratio between MS and PAN.\n%\n% Outputs:\n% HRPanLP: Output filtered MS image.\n% \n% References:\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction HRPanLP = LPfilter(HRPan,ratio)\n\nh=[1 4 6 4 1 ]/16;\ng=[0 0 1 0 0 ]-h;\nhtilde=[ 1 4 6 4 1]/16;\ngtilde=[ 0 0 1 0 0 ]+htilde;\nh=sqrt(2)*h;\ng=sqrt(2)*g;\nhtilde=sqrt(2)*htilde;\ngtilde=sqrt(2)*gtilde;\nWF={h,g,htilde,gtilde};\n\nLevels = ceil(log2(ratio));\n\nWT = ndwt2_working(HRPan,Levels,WF);\n\nfor ii = 2 : numel(WT.dec), WT.dec{ii} = zeros(size(WT.dec{ii})); end\n\nHRPanLP = indwt2_working(WT,'c');\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/LPfilterGauss.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% LPfilterGauss filters the panchromatic (PAN) image using a Gaussin filter with gain at Nyquist frequency 0.3. \n% \n% Interface:\n% I_PAN_LR = LPfilterGauss(I_PAN,ratio)\n%\n% Inputs:\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN.\n%\n% Outputs:\n% I_PAN_LR: Output filtered MS image.\n% \n% References:\n% [Aiazzi02] B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, Context-driven fusion of high spatial and spectral resolution images based on\n% oversampled multiresolution analysis, IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 10, pp. 23002312, October\n% 2002.\n% [Aiazzi06] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTF-tailored multiscale fusion of high-resolution MS and Pan imagery,\n% Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 591596, May 2006.\n% [Vivone14a] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening, IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930934, May 2014.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_PAN_LR = LPfilterGauss(I_PAN,ratio)\n GNyq = 0.3;\n N = 41;\n fcut = 1/ratio;\n\n alpha = sqrt((N*(fcut/2))^2/(-2*log(GNyq)));\n H = fspecial('gaussian', N, alpha);\n Hd = H./max(H(:));\n h = fwind1(Hd,kaiser(N));\n I_PAN_LR = imfilter(I_PAN,real(h),'replicate');\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/LPfilterPlusDec.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% LPfilterPlusDec filters and decimates the image I_PAN using a Starck and Murtagh (S&M) filter. \n% \n% Interface:\n% I_PAN_LR = LPfilterPlusDec(I_PAN,ratio)\n%\n% Inputs:\n% I_PAN: Image to be filtered and decimated;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Resize factors power of 2.\n%\n% Outputs:\n% I_PAN_LR: Filtered and decimated image.\n% \n% References:\n% [Starck07] J.-L. Starck, J. Fadili, and F. Murtagh, The undecimated wavelet decomposition and its reconstruction, IEEE Transactions on Image\n% Processing, vol. 16, no. 2, pp. 297309, February 2007.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nfunction I_PAN_LR = LPfilterPlusDec(I_PAN,ratio)\n\nh=[1 4 6 4 1 ]/16;\ng=[0 0 1 0 0 ]-h;\nhtilde=[ 1 4 6 4 1]/16;\ngtilde=[ 0 0 1 0 0 ]+htilde;\nh=sqrt(2)*h;\ng=sqrt(2)*g;\nhtilde=sqrt(2)*htilde;\ngtilde=sqrt(2)*gtilde;\nWF={h,g,htilde,gtilde};\n\nLevels = ceil(log2(ratio));\n\nWT = ndwt2_working(I_PAN,Levels,WF);\n\nfor ii = 2 : numel(WT.dec), WT.dec{ii} = zeros(size(WT.dec{ii})); end\n\nI_PAN_LR = indwt2_working(WT,'c');\n\nI_PAN_LR = imresize(I_PAN_LR,1/ratio,'nearest');\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/MTF.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% MTF filters the image I_MS using a Gaussin filter matched with the Modulation Transfer Function (MTF) of the MultiSpectral (MS) sensor. \n% \n% Interface:\n% I_Filtered = MTF(I_MS,sensor,ratio)\n%\n% Inputs:\n% I_MS: MS image;\n% sensor: String for type of sensor (e.g. 'WV2', 'IKONOS');\n% ratio: Scale ratio between MS and PAN.\n%\n% Outputs:\n% I_Filtered: Output filtered MS image.\n% \n% References:\n% [Aiazzi02] B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, Context-driven fusion of high spatial and spectral resolution images based on\n% oversampled multiresolution analysis, IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 10, pp. 23002312, October\n% 2002.\n% [Aiazzi06] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTF-tailored multiscale fusion of high-resolution MS and Pan imagery,\n% Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 591596, May 2006.\n% [Vivone14a] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening, IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930934, May 2014.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Filtered = MTF(I_MS,sensor,ratio)\n\nh = genMTF(ratio, sensor, size(I_MS,3));\n\nI_MS_LP = zeros(size(I_MS));\nfor ii = 1 : size(I_MS,3)\n I_MS_LP(:,:,ii) = imfilter(I_MS(:,:,ii),real(h(:,:,ii)),'replicate');\nend\n\nI_Filtered = double(I_MS_LP);\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/MTF_PAN.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% MTF filters the panchromatic (PAN) image using a Gaussin filter matched with the Modulation Transfer Function (MTF) of the PAN sensor. \n% \n% Interface:\n% I_Filtered = MTF_PAN(I_PAN,sensor,ratio)\n%\n% Inputs:\n% I_PAN: PAN image;\n% sensor: String for type of sensor (e.g. 'WV2', 'IKONOS');\n% ratio: Scale ratio between MS and PAN.\n%\n% Outputs:\n% I_Filtered: Output filtered PAN image.\n% \n% References:\n% [Aiazzi02] B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, Context-driven fusion of high spatial and spectral resolution images based on\n% oversampled multiresolution analysis, IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 10, pp. 23002312, October\n% 2002.\n% [Aiazzi06] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTF-tailored multiscale fusion of high-resolution MS and Pan imagery,\n% Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 591596, May 2006.\n% [Vivone14a] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening, IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930934, May 2014.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Filtered = MTF_PAN(I_PAN,sensor,ratio)\n\nswitch sensor\n case 'QB' \n GNyq = 0.15; \n case 'IKONOS'\n GNyq = 0.17;\n case {'GeoEye1','WV4'}\n GNyq = 0.16;\n case 'WV2'\n GNyq = 0.11;\n case 'WV3'\n GNyq = 0.14; \n case 'none'\n GNyq = 0.15;\nend\n\nN = 41;\nfcut = 1/ratio;\n \nalpha = sqrt(((N-1)*(fcut/2))^2/(-2*log(GNyq)));\nH = fspecial('gaussian', N, alpha);\nHd = H./max(H(:));\nh = fwind1(Hd,kaiser(N));\nI_PAN_LP = imfilter(I_PAN,real(h),'replicate');\n\nI_Filtered= double(I_PAN_LP);\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/estimation_alpha.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Estimation coefficients linear regression model. \n% \n% Interface:\n% alpha = estimation_alpha(I_MS,I_PAN,type_estimation)\n% \n% Inputs:\n% I_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% type_estimation: Type of estimation (i.e. local or global).\n%\n% Outputs:\n% alpha: Coefficients estimated by the linear regression model.\n% \n% References:\n% [Vivone14] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening, IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930934, May 2014.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction alpha = estimation_alpha(I_MS,I_PAN,type_estimation)\n\nif strcmp(type_estimation,'global')\n %%%%%%%% Global estimation\n IHc = reshape(I_PAN,[numel(I_PAN) 1]);\n ILRc = reshape(I_MS,[size(I_MS,1)*size(I_MS,2) size(I_MS,3)]);\n alpha = ILRc\\IHc;\nelse\n %%%%%%%% Local estimation\n block_win = 32;\n alphas = zeros(size(I_MS,3),1);\n cont_bl = 0;\n for ii = 1 : block_win : size(I_MS,1)\n for jj = 1 : block_win : size(I_MS,2)\n imHRbl = I_PAN(ii : min(size(I_MS,1),ii + block_win - 1), jj : min(size(I_MS,2),jj + block_win - 1));\n imageLRbl = I_MS(ii : min(size(I_MS,1),ii + block_win - 1), jj : min(size(I_MS,2),jj + block_win - 1),:);\n imageHRc = reshape(imHRbl,[numel(imHRbl) 1]);\n ILRc = reshape(imageLRbl,[size(imageLRbl,1).*size(imageLRbl,2) size(imageLRbl,3)]);\n alphah = ILRc\\imageHRc;\n alphas = alphas + alphah;\n cont_bl = cont_bl + 1;\n end\n end\n alpha = alphas/cont_bl;\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/genMTF.m", "content": "% Description: \n% Generate a bank of filters shaped on the MTF of the sensor. Each filter\n% corresponds to a band acquired by the sensor. \n% \n% Interface:\n% h = genMTF(ratio, sensor, nbands)\n%\n% Inputs:\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n% nbands: Number of spectral bands.\n%\n% Outputs:\n% h: Gaussian filter mimicking the MTF of the MS sensor\n% \n% References:\n% [Aiazzi02] B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, Context-driven fusion of high spatial and spectral resolution images based on\n% oversampled multiresolution analysis,? IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 10, pp. 2300?2312, October\n% 2002.\n% [Aiazzi06] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTF-tailored multiscale fusion of high-resolution MS and Pan imagery,?\n% Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 591?596, May 2006.\n% [Vivone14a] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening,? IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930?934, May 2014.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms?, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2565?2586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction h = genMTF(ratio, sensor, nbands)\n\nswitch sensor\n case 'QB'\n GNyq = [0.34 0.32 0.30 0.22]; % Band Order: B,G,R,NIR\n case 'IKONOS'\n GNyq = [0.26,0.28,0.29,0.28]; % Band Order: B,G,R,NIR\n case {'GeoEye1','WV4'}\n GNyq = [0.23,0.23,0.23,0.23]; % Band Order: B,G,R,NIR\n case 'WV2'\n GNyq = [0.35 .* ones(1,7), 0.27];\n case 'WV3'\n GNyq = [0.325 0.355 0.360 0.350 0.365 0.360 0.335 0.315];\n otherwise\n GNyq = 0.3 .* ones(1, nbands);\nend\n\n\n\n%%% MTF\nN = 41;\nnBands = length(GNyq);\nh = zeros(N, N, nBands);\nfcut = 1/ratio;\n\nfor ii = 1 : nBands\n alpha = sqrt(((N-1)*(fcut/2))^2/(-2*log(GNyq(ii))));\n H = fspecial('gaussian', N, alpha);\n Hd = H./max(H(:));\n h(:,:,ii) = fwind1(Hd,kaiser(N));\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/gen_LP_image.m", "content": "% Description:\n% gen_LP_image generates the Low Resolution version of the PAN image required for the calculation of the\n% segmentation-based version of the Gram-Schmidt algorithm, based on the segmentation S.\n%\n% Interface:\n% I_LR_input = gen_LP_image(Local_algorithm,I_MS,I_PAN,I_MS_LR,ratio,sensor,S)\n%\n% Inputs:\n% PS_algorithm: Employed segmentation-based algorithm\n% ('GSA','GS2GLP')\n% I_MS: MS image upsampled at PAN scale\n% I_PAN: PAN image\n% I_MS_LR: MS image\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value.\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n%\n% Outputs:\n% I_LR_input: Low Resolution version of the PAN image\n%\n% References:\n%\n% [Restaino17] R. Restaino, M. Dalla Mura, G. Vivone, J. Chanussot, Context-Adaptive Pansharpening Based on Image Segmentation,\n% IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 2, pp. 753766, February 2017.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms,\n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_LR_input = gen_LP_image(PS_algorithm,I_MS,I_PAN,I_MS_LR,ratio,sensor)\n\nswitch PS_algorithm\n \n case 'GSA'\n %%%%%%%%% Generation of LR PAN image\n PAN_LP = LPfilterGauss(I_PAN,ratio);\n %%%%%%%%%% Estimation of weights\n PAN_LP2 = imresize(PAN_LP,1/ratio,'nearest');\n alpha= estimation_alpha(cat(3,I_MS_LR,ones(size(I_MS_LR,1),size(I_MS_LR,2))),PAN_LP2,'global');\n [Height,Width,Bands] = size(I_MS);\n I_MS_col = reshape(double(I_MS), Height*Width, Bands);\n alpha = repmat(alpha', [size(I_MS_col,1),1]);\n I_LR_col = sum([I_MS_col, ones(size(I_MS_col,1),1)] .* alpha, 2);\n I_LR_input = reshape(I_LR_col, Height, Width);\n \n case 'GS2GLP'\n h = genMTF(ratio, sensor, size(I_MS,3));\n for ii=1:size(h, 3)\n PAN_LP(:,:,ii) = imfilter(I_PAN,real(h(:,:,ii)),'replicate');\n end\n PAN_LP2 = imresize(PAN_LP,1/ratio,'nearest');\n I_LR_input = interp23tap(PAN_LP2,ratio);\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/indexes_evaluation.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Reduced resolution quality indexes. \n% \n% Interface:\n% [Q_index, SAM_index, ERGAS_index, sCC, Q2n_index] = indexes_evaluation(I_F,I_GT,ratio,L,Q_blocks_size,flag_cut_bounds,dim_cut,th_values)\n%\n% Inputs:\n% I_F: Fused Image;\n% I_GT: Ground-Truth image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% L: Image radiometric resolution; \n% Q_blocks_size: Block size of the Q-index locally applied;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range.\n%\n% Outputs:\n% Q_index: Q index;\n% SAM_index: Spectral Angle Mapper (SAM) index;\n% ERGAS_index: Erreur Relative Globale Adimensionnelle de Synthse (ERGAS) index;\n% sCC: spatial Correlation Coefficient between fused and ground-truth images;\n% Q2n_index: Q2n index.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [Q_index, SAM_index, ERGAS_index, sCC, Q2n_index] = indexes_evaluation(I_F,I_GT,ratio,L,Q_blocks_size,flag_cut_bounds,dim_cut,th_values)\n\nif flag_cut_bounds\n I_GT = I_GT(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\n I_F = I_F(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\nend\n\nif th_values\n I_F(I_F > 2^L) = 2^L;\n I_F(I_F < 0) = 0;\nend\n\ncd Quality_Indices\n\nQ2n_index = q2n(I_GT,I_F,Q_blocks_size,Q_blocks_size);\nQ_index = Q(I_GT,I_F,2^L);\nSAM_index = SAM(I_GT,I_F);\nERGAS_index = ERGAS(I_GT,I_F,ratio);\nsCC = SCC(I_F,I_GT);\n\ncd ..\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/indexes_evaluation_FS.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Full resolution quality indexes. \n% \n% Interface:\n% [D_lambda,D_S,QNR_index,SAM_index,sCC] = indexes_evaluation_FS(I_F,I_MS_LR,I_PAN,L,th_values,I_MS,sensor,tag,ratio)\n%\n% Inputs:\n% I_F: Fused image;\n% I_MS_LR: MS image;\n% I_PAN: Panchromatic image;\n% L: Image radiometric resolution; \n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% I_MS: MS image upsampled to the PAN size;\n% sensor: String for type of sensor (e.g. 'WV2','IKONOS');\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% flagQNR: if flagQNR == 1, the software uses the QNR otherwise the HQNR is used.\n%\n% Outputs:\n% D_lambda: D_lambda index;\n% D_S: D_S index;\n% QNR_index: QNR index;\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [D_lambda,D_S,QNR_index] = indexes_evaluation_FS(I_F,I_MS_LR,I_PAN,L,th_values,I_MS,sensor,ratio,flagQNR)\n\nif th_values\n I_F(I_F > 2^L) = 2^L;\n I_F(I_F < 0) = 0;\nend\n\ncd Quality_Indices\n\nif flagQNR == 1\n [QNR_index,D_lambda,D_S]= QNR(I_F,I_MS,I_MS_LR,I_PAN,ratio);\nelse\n [QNR_index,D_lambda,D_S] = HQNR(I_F,I_MS_LR,I_MS,I_PAN,32,sensor,ratio);\nend\n\ncd ..\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/indwt2_working.m", "content": "function X = indwt2_working(W,varargin)\n%INDWT2 Inverse nondecimated 2-D wavelet transform.\n% INDWT2 will be removed in a future release of MATLAB. Use the\n% following function instead:\n% iswt2\n\n% Error in R2015a\n% error(message('Wavelet:warnobsolete:ErrorReplaceINDWT2'));\nnbIN = nargin-1;\nidxCFS = -1;\ncfsFLAG = false;\nif nbIN>0\n nbCELL = numel(W.dec);\n type = varargin{1};\n if ~ischar(type)\n error(message('Wavelet:FunctionArgVal:Invalid_ArgTyp'))\n end\n type = upper(type);\n cfsFLAG = isequal(upper(type(1)),'C');\n if cfsFLAG , type = type(2:end); end\n switch type\n case {'D','H'} , idxCFS = 0;\n case {'AA','LL','A','L'} , idxCFS = 1;\n case {'AD','LH'} , idxCFS = 2;\n case {'DA','HL'} , idxCFS = 3;\n case {'DD','HH'} , idxCFS = 4;\n end\n if nbIN>1 , levREC = varargin{2}; else levREC = W.level; end\n \n if idxCFS>1\n idxCFS = idxCFS + 3*(W.level-levREC);\n if ~cfsFLAG\n for j=1:nbCELL\n if ~isequal(j,idxCFS);\n W.dec{j} = zeros(size(W.dec{j}));\n end\n end\n else\n X = W.dec{idxCFS}; % Coefficients\n return\n end\n \n elseif idxCFS==1 % Approximations (AA or LL)\n if cfsFLAG && levREC==W.level \n X = W.dec{1}; \n return; % Coefficients of Approximation at level MAX\n end\n idxMinToKill = 1 + 3*(W.level-levREC)+1;\n for j=idxMinToKill:nbCELL\n W.dec{j} = zeros(size(W.dec{j}));\n end\n \n elseif idxCFS==0\n idxMaxToKill = 1 + 3*(W.level-levREC);\n for j=1:idxMaxToKill\n W.dec{j} = zeros(size(W.dec{j}));\n end\n \n else\n \n end\nend\n\n% Initialization.\nLo = W.filters.LoR;\nHi = W.filters.HiR;\ndwtEXTM = W.mode;\nperFLAG = isequal(dwtEXTM,'per');\ncfs = W.dec;\nsizes = W.sizes;\nlevel = W.level;\n\nmaxloop = level;\nif idxCFS==1 && cfsFLAG , maxloop = (level-levREC); end\n\nidxBeg = 1;\nfor k=1:maxloop\n idxEnd = idxBeg+3;\n dec = reshape(cfs(idxBeg:idxEnd),2,2);\n sizerec = sizes(k+1,:);\n X = recFUNC(dec,sizerec,Lo,Hi,perFLAG);\n cfs(1:idxEnd-1) = {[]};\n cfs{idxEnd} = X;\n idxBeg = idxEnd;\nend\n\nif abs(idxCFS)==1 && ~cfsFLAG && length(W.sizeINI)==3\n % X = uint8(X);\nend\n%-----------------------------------------------------------------------%\nfunction X = recFUNC(dec,sINI,Lo,Hi,perFLAG)\n\n% Reconstruction.\nperm = [2,1,3];\nW = cell(1,2);\nfor i = 1:2\n W{i} = wrec1D(dec{i,1},Lo{2},perm,perFLAG) + ...\n wrec1D(dec{i,2},Hi{2},perm,perFLAG);\nend\nX = (wrec1D(W{1},Lo{1},[],perFLAG) + wrec1D(W{2},Hi{1},[],perFLAG))/4;\n\n% Extraction of central part\nsREC = size(X);\nF = floor((sREC-sINI)/2);\nC = ceil((sREC-sINI)/2);\nX = X(1+F(1):end-C(1),1+F(2):end-C(2),:);\n%-----------------------------------------------------------------------%\nfunction X = wrec1D(X,F,perm,perFLAG)\n\nif ~isempty(perm) , X = permute(X,perm); end\nif perFLAG\n nb = length(F)-1;\n X = [X X(:,1:nb,:)];\nend\nX = convn(X,F);\nif ~isempty(perm) , X = permute(X,perm); end\n%-----------------------------------------------------------------------%\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/interp23tap.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% interp23tap interpolates the image I_Interpolated using a polynomial with 23 coefficients interpolator. \n% \n% Interface:\n% I_Interpolated = interp23tap(I_Interpolated,ratio)\n%\n% Inputs:\n% I_Interpolated: Image to interpolate;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Resize factors power of 2.\n%\n% Outputs:\n% I_Interpolated: Interpolated image.\n% \n% References:\n% [Aiazzi02] B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, Context-driven fusion of high spatial and spectral resolution images based on\n% oversampled multiresolution analysis,? IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 10, pp. 2300?2312, October\n% 2002.\n% [Aiazzi13] B. Aiazzi, S. Baronti, M. Selva, and L. Alparone, Bi-cubic interpolation for shift-free pan-sharpening,? ISPRS Journal of Photogrammetry\n% and Remote Sensing, vol. 86, no. 6, pp. 65?76, December 2013.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms?, \n% IEEE Transaction on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2565?2586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction I_Interpolated = interp23tap(I_Interpolated,ratio)\n\nif (2^round(log2(double(ratio))) ~= ratio)\n disp('Error: Only resize factors power of 2');\n return;\nend \n\n[r,c,b] = size(I_Interpolated);\n\nCDF23 = 2.*[0.5 0.305334091185 0 -0.072698593239 0 0.021809577942 0 -0.005192756653 0 0.000807762146 0 -0.000060081482];\nCDF23 = [fliplr(CDF23(2:end)) CDF23];\nBaseCoeff = CDF23;\nfirst = 1;\n\nfor z = 1 : ratio/2\n\n I1LRU = zeros((2^z) * r, (2^z) * c, b);\n \n if first\n I1LRU(2:2:end,2:2:end,:) = I_Interpolated;\n first = 0;\n else\n I1LRU(1:2:end,1:2:end,:) = I_Interpolated;\n end\n\n for ii = 1 : b\n t = I1LRU(:,:,ii); \n t = imfilter(t',BaseCoeff,'circular'); \n I1LRU(:,:,ii) = imfilter(t',BaseCoeff,'circular'); \n end\n \n I_Interpolated = I1LRU;\n \nend\n\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/k_means_clustering.m", "content": "% I_MS: Image to segment\n% n_segm: Number of segments\n% Output:\n% S: Segmentation map.\nfunction S = k_means_clustering(I_MS, n_segm)\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%+\n%%% k-means Segmentation of MS image\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%+\n\nF1 = zeros(size(I_MS,1)*size(I_MS,2),size(I_MS,3));\n\nfor ibands = 1 :size(I_MS,3)\n a = I_MS(:,:,ibands);\n F1(:,ibands) = a(:)/max(a(:));\nend\nIDX = kmeans(F1,n_segm);\nS = reshape(IDX,[size(I_MS,1) size(I_MS,2)]);\n\nend\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/matrix2latex.m", "content": "function matrix2latex(matrix, filename, varargin)\n\n% function: matrix2latex(...)\n% Author: M. Koehler\n% Contact: koehler@in.tum.de\n% Version: 1.1\n% Date: May 09, 2004\n\n% This software is published under the GNU GPL, by the free software\n% foundation. For further reading see: http://www.gnu.org/licenses/licenses.html#GPL\n\n% Usage:\n% matrix2late(matrix, filename, varargs)\n% where\n% - matrix is a 2 dimensional numerical or cell array\n% - filename is a valid filename, in which the resulting latex code will\n% be stored\n% - varargs is one ore more of the following (denominator, value) combinations\n% + 'rowLabels', array -> Can be used to label the rows of the\n% resulting latex table\n% + 'columnLabels', array -> Can be used to label the columns of the\n% resulting latex table\n% + 'alignment', 'value' -> Can be used to specify the alginment of\n% the table within the latex document. Valid arguments are: 'l', 'c',\n% and 'r' for left, center, and right, respectively\n% + 'format', 'value' -> Can be used to format the input data. 'value'\n% has to be a valid format string, similar to the ones used in\n% fprintf('format', value);\n% + 'size', 'value' -> One of latex' recognized font-sizes, e.g. tiny,\n% HUGE, Large, large, LARGE, etc.\n%\n% Example input:\n% matrix = [1.5 1.764; 3.523 0.2];\n% rowLabels = {'row 1', 'row 2'};\n% columnLabels = {'col 1', 'col 2'};\n% matrix2latex(matrix, 'out.tex', 'rowLabels', rowLabels, 'columnLabels', columnLabels, 'alignment', 'c', 'format', '%-6.2f', 'size', 'tiny');\n%\n% The resulting latex file can be included into any latex document by:\n% /input{out.tex}\n%\n% Enjoy life!!!\n\n rowLabels = [];\n colLabels = [];\n alignment = 'l';\n format = [];\n textsize = [];\n if (rem(nargin,2) == 1 || nargin < 2)\n error('matrix2latex: ', 'Incorrect number of arguments to %s.', mfilename);\n end\n\n okargs = {'rowlabels','columnlabels', 'alignment', 'format', 'size'};\n for j=1:2:(nargin-2)\n pname = varargin{j};\n pval = varargin{j+1};\n k = strmatch(lower(pname), okargs);\n if isempty(k)\n error('matrix2latex: ', 'Unknown parameter name: %s.', pname);\n elseif length(k)>1\n error('matrix2latex: ', 'Ambiguous parameter name: %s.', pname);\n else\n switch(k)\n case 1 % rowlabels\n rowLabels = pval;\n if isnumeric(rowLabels)\n rowLabels = cellstr(num2str(rowLabels(:)));\n end\n case 2 % column labels\n colLabels = pval;\n if isnumeric(colLabels)\n colLabels = cellstr(num2str(colLabels(:)));\n end\n case 3 % alignment\n alignment = lower(pval);\n if alignment == 'right'\n alignment = 'r';\n end\n if alignment == 'left'\n alignment = 'l';\n end\n if alignment == 'center'\n alignment = 'c';\n end\n if alignment ~= 'l' && alignment ~= 'c' && alignment ~= 'r'\n alignment = 'l';\n warning('matrix2latex: ', 'Unkown alignment. (Set it to \\''left\\''.)');\n end\n case 4 % format\n format = lower(pval);\n case 5 % format\n textsize = pval;\n end\n end\n end\n\n fid = fopen(filename, 'a');\n \n width = size(matrix, 2);\n height = size(matrix, 1);\n\n if isnumeric(matrix)\n matrix = num2cell(matrix);\n for h=1:height\n for w=1:width\n if(~isempty(format))\n matrix{h, w} = num2str(matrix{h, w}, format);\n else\n matrix{h, w} = num2str(matrix{h, w});\n end\n end\n end\n end\n \n if(~isempty(textsize))\n fprintf(fid, '\\\\begin{%s}', textsize);\n end\n\n fprintf(fid, '\\\\begin{tabular}{|');\n\n if(~isempty(rowLabels))\n fprintf(fid, 'l|');\n end\n for i=1:width\n fprintf(fid, '%c|', alignment);\n end\n fprintf(fid, '}\\r\\n');\n \n fprintf(fid, '\\\\hline\\r\\n');\n \n if(~isempty(colLabels))\n if(~isempty(rowLabels))\n fprintf(fid, '&');\n end\n for w=1:width-1\n fprintf(fid, '\\\\textbf{%s}&', colLabels{w});\n end\n fprintf(fid, '\\\\textbf{%s}\\\\\\\\\\\\hline\\r\\n', colLabels{width});\n end\n \n for h=1:height\n if(~isempty(rowLabels))\n fprintf(fid, '\\\\textbf{%s}&', rowLabels{h});\n end\n for w=1:width-1\n fprintf(fid, '%s&', matrix{h, w});\n end\n fprintf(fid, '%s\\\\\\\\\\\\hline\\r\\n', matrix{h, width});\n end\n\n fprintf(fid, '\\\\end{tabular}\\r\\n');\n \n if(~isempty(textsize))\n fprintf(fid, '\\\\end{%s}', textsize);\n end\n\n fclose(fid);" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/ndwt2_working.m", "content": "function varargout = ndwt2_working(X,level,varargin)\n%NDWT2 Nondecimated 2-D wavelet transform.\n% NDWT2 will be removed in a future release of MATLAB. Use the\n% following function instead:\n% swt2\n\n% Error in R2015a\n% error(message('Wavelet:warnobsolete:ErrorReplaceNDWT2'));\nnbIn = length(varargin);\nif nbIn < 1\n error(message('MATLAB:narginchk:notEnoughInputs'));\nelseif nbIn > 5\n error(message('MATLAB:narginchk:tooManyInputs'));\nend\n\nLoD = cell(1,2); HiD = cell(1,2); LoR = cell(1,2); HiR = cell(1,2);\nif ischar(varargin{1})\n [LD,HD,LR,HR] = wfilters(varargin{1}); \n for k = 1:2\n LoD{k} = LD; HiD{k} = HD; LoR{k} = LR; HiR{k} = HR;\n end\n\nelseif isstruct(varargin{1})\n if isfield(varargin{1},'w1') && isfield(varargin{1},'w2')\n for k = 1:2\n [LoD{k},HiD{k},LoR{k},HiR{k}] = ...\n wfilters(varargin{1}.(['w' int2str(k)]));\n end\n elseif isfield(varargin{1},'LoD') && isfield(varargin{1},'HiD') && ...\n isfield(varargin{1},'LoR') && isfield(varargin{1},'HiR')\n for k = 1:2\n LoD{k} = varargin{1}.LoD{k}; HiD{k} = varargin{1}.HiD{k};\n LoR{k} = varargin{1}.LoR{k}; HiR{k} = varargin{1}.HiR{k};\n end\n else\n error(message('Wavelet:FunctionArgVal:Invalid_ArgVal'));\n end\n \nelseif iscell(varargin{1})\n if ischar(varargin{1}{1})\n for k = 1:2\n [LoD{k},HiD{k},LoR{k},HiR{k}] = wfilters(varargin{1}{k});\n end\n else\n LoD(1:end) = varargin{1}(1); HiD(1:end) = varargin{1}(2);\n LoR(1:end) = varargin{1}(3); HiR(1:end) = varargin{1}(4);\n end\nelse\n \nend\nnextArg = 2;\n\ndwtEXTM = 'sym';\nwhile nbIn>=nextArg\n argName = varargin{nextArg};\n argVal = varargin{nextArg+1};\n nextArg = nextArg + 2;\n switch argName\n case 'mode' , dwtEXTM = argVal;\n end\nend\n\n% Initialization.\nif isempty(X) , varargout{1} = []; return; end\nsX = size(X);\nX = double(X);\nsizes = zeros(level+1,length(sX));\nsizes(level+1,:) = sX;\n\nfor k=1:level\n dec = decFUNC(X,LoD,HiD,dwtEXTM);\n X = dec{1,1,1};\n sizes(level+1-k,:) = size(X);\n dec = reshape(dec,4,1,1);\n if k>1\n cfs(1) = [];\n cfs = cat(1,dec,cfs);\n else\n cfs = dec;\n end\nend\n\nWT.sizeINI = sX;\nWT.level = level;\nWT.filters.LoD = LoD;\nWT.filters.HiD = HiD;\nWT.filters.LoR = LoR;\nWT.filters.HiR = HiR;\nWT.mode = dwtEXTM;\nWT.dec = cfs;\nWT.sizes = sizes;\nvarargout{1} = WT;\n\n%-------------------------------------------------------------------------%\nfunction dec = decFUNC(X,LoD,HiD,dwtEXTM)\n\ndec = cell(2,2);\npermVect = [];\n[a_Lo,d_Hi] = wdec1D(X,LoD{1},HiD{1},permVect,dwtEXTM);\npermVect = [2,1,3];\n[dec{1,1},dec{1,2}] = wdec1D(a_Lo,LoD{2},HiD{2},permVect,dwtEXTM);\n[dec{2,1},dec{2,2}] = wdec1D(d_Hi,LoD{2},HiD{2},permVect,dwtEXTM);\n%-------------------------------------------------------------------------%\nfunction [L,H] = wdec1D(X,Lo,Hi,perm,dwtEXTM)\n\nif ~isempty(perm) , X = permute(X,perm); end\nsX = size(X);\nif length(sX)<3 , sX(3) = 1; end\nlf = length(Lo);\nlx = sX(2);\nlc = lx+lf-1;\nswitch dwtEXTM\n case 'zpd' % Zero extension.\n \n case {'sym','symh'} % Symmetric extension (half-point).\n X = [X(:,lf-1:-1:1,:) , X , X(:,end:-1:end-lf+1,:)];\n \n case 'sp0' % Smooth extension of order 0.\n X = [X(:,ones(1,lf-1),:) , X , X(:,lx*ones(1,lf-1),:)];\n \n case {'sp1','spd'} % Smooth extension of order 1.\n Z = zeros(sX(1),sX(2)+ 2*lf-2,sX(3));\n Z(:,lf:lf+lx-1,:) = X;\n last = sX(2)+lf-1;\n for k = 1:lf-1\n Z(:,last+k,:) = 2*Z(:,last+k-1,:)- Z(:,last+k-2,:);\n Z(:,lf-k,:) = 2*Z(:,lf-k+1,:)- Z(:,lf-k+2,:);\n end\n X = Z; clear Z;\n \n case 'symw' % Symmetric extension (whole-point).\n X = [X(:,lf:-1:2,:) , X , X(:,end-1:-1:end-lf,:)];\n \n case {'asym','asymh'} % Antisymmetric extension (half-point).\n X = [-X(:,lf-1:-1:1,:) , X , -X(:,end:-1:end-lf+1,:)]; \n \n case 'asymw' % Antisymmetric extension (whole-point).\n X = [-X(:,lf:-1:2,:) , X , -X(:,end-1:-1:end-lf,:)];\n\n case 'rndu' % Uniformly randomized extension.\n X = [randn(sX(1),lf-1,sX(3)) , X , randn(sX(1),lf-1,sX(3))]; \n \n case 'rndn' % Normally randomized extension.\n X = [randn(sX(1),lf-1,sX(3)) , X , randn(sX(1),lf-1,sX(3))]; \n \n case 'ppd' % Periodized extension (1).\n X = [X(:,end-lf+2:end,:) , X , X(:,1:lf-1,:)];\n \n case 'per' % Periodized extension (2).\n if rem(lx,2) , X = [X , X(:,end,:)]; end\n X = [X(:,end-lf+2:end,:) , X , X(:,1:lf-1,:)]; \nend\nL = convn(X,Lo);\nH = convn(X,Hi);\nclear X\nswitch dwtEXTM\n case 'zpd'\n otherwise\n lenL = size(L,2);\n first = lf; last = lenL-lf+1;\n L = L(:,first:last,:); H = H(:,first:last,:);\n lenL = size(L,2);\n first = 1+floor((lenL-lc)/2); last = first+lc-1;\n L = L(:,first:last,:); H = H(:,first:last,:);\nend\nif isequal(dwtEXTM,'per')\n first = 1; last = lx;\n L = L(:,first:last,:);\n H = H(:,first:last,:);\nend\n\nif ~isempty(perm)\n L = permute(L,perm);\n H = permute(H,perm);\nend\n%-------------------------------------------------------------------------%\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/printAllImagesImWriteFR.m", "content": "MatrixPrint(:,:,:,1) = I_MS;\nMatrixPrint(:,:,:,2) = I_BT_H;\nMatrixPrint(:,:,:,3) = I_BDSD;\nMatrixPrint(:,:,:,4) = I_C_BDSD;\nMatrixPrint(:,:,:,5) = I_BDSD_PC;\nMatrixPrint(:,:,:,6) = I_GS;\nMatrixPrint(:,:,:,7) = I_GSA;\nMatrixPrint(:,:,:,8) = I_C_GSA;\nMatrixPrint(:,:,:,9) = I_PRACS;\nMatrixPrint(:,:,:,10) = I_AWLP;\nMatrixPrint(:,:,:,11) = I_MTF_GLP;\nMatrixPrint(:,:,:,12) = I_MTF_GLP_FS;\nMatrixPrint(:,:,:,13) = I_MTF_GLP_HPM;\nMatrixPrint(:,:,:,14) = I_MTF_GLP_HPM_H;\nMatrixPrint(:,:,:,15) = I_MTF_GLP_HPM_R;\nMatrixPrint(:,:,:,16) = I_MTF_GLP_CBD;\nMatrixPrint(:,:,:,17) = I_C_MTF_GLP_CBD;\nMatrixPrint(:,:,:,18) = I_MF;\nMatrixPrint(:,:,:,19) = I_FE_HPM;\nMatrixPrint(:,:,:,20) = I_SR_D;\nMatrixPrint(:,:,:,21) = I_PWMBF;\nMatrixPrint(:,:,:,22) = I_TV;\nMatrixPrint(:,:,:,23) = I_RR;\nMatrixPrint(:,:,:,24) = I_PNN;\nMatrixPrint(:,:,:,25) = I_PNN_IDX;\nMatrixPrint(:,:,:,26) = I_A_PNN;\nMatrixPrint(:,:,:,27) = I_A_PNN_FT;\n\nif size(I_MS,3) == 4\n vect_index_RGB = [3,2,1];\nelse\n vect_index_RGB = [5,3,2];\nend\n\ntitleImages = algorithms;\n\naddpath([pwd,'\\Tools']);\n\nfigure, MP = showImagesAll(MatrixPrint,titleImages,vect_index_RGB,flag_cut_bounds,dim_cut,0);\n\ncd 'Outputs'\nfor ii = 1 : size(MP,4)\n imwrite(MP(:,:,:,ii),sprintf('%s.png',algorithms{ii}));\nend\nimwrite(showPan(I_PAN,0,1,flag_cut_bounds,dim_cut),'PAN.png')\ncd .." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/printAllImagesImWriteRR.m", "content": "MatrixPrint(:,:,:,1) = I_GT;\nMatrixPrint(:,:,:,2) = I_MS;\nMatrixPrint(:,:,:,3) = I_BT_H;\nMatrixPrint(:,:,:,4) = I_BDSD;\nMatrixPrint(:,:,:,5) = I_C_BDSD;\nMatrixPrint(:,:,:,6) = I_BDSD_PC;\nMatrixPrint(:,:,:,7) = I_GS;\nMatrixPrint(:,:,:,8) = I_GSA;\nMatrixPrint(:,:,:,9) = I_C_GSA;\nMatrixPrint(:,:,:,10) = I_PRACS;\nMatrixPrint(:,:,:,11) = I_AWLP;\nMatrixPrint(:,:,:,12) = I_MTF_GLP;\nMatrixPrint(:,:,:,13) = I_MTF_GLP_FS;\nMatrixPrint(:,:,:,14) = I_MTF_GLP_HPM;\nMatrixPrint(:,:,:,15) = I_MTF_GLP_HPM_H;\nMatrixPrint(:,:,:,16) = I_MTF_GLP_HPM_R;\nMatrixPrint(:,:,:,17) = I_MTF_GLP_CBD;\nMatrixPrint(:,:,:,18) = I_C_MTF_GLP_CBD;\nMatrixPrint(:,:,:,19) = I_MF;\nMatrixPrint(:,:,:,20) = I_FE_HPM;\nMatrixPrint(:,:,:,21) = I_SR_D;\nMatrixPrint(:,:,:,22) = I_PWMBF;\nMatrixPrint(:,:,:,23) = I_TV;\nMatrixPrint(:,:,:,24) = I_RR;\nMatrixPrint(:,:,:,25) = I_PNN;\nMatrixPrint(:,:,:,26) = I_PNN_IDX;\nMatrixPrint(:,:,:,27) = I_A_PNN;\nMatrixPrint(:,:,:,28) = I_A_PNN_FT;\n\nif size(I_MS,3) == 4\n vect_index_RGB = [3,2,1];\nelse\n vect_index_RGB = [5,3,2];\nend\n\ntitleImages = algorithms;\n\naddpath([pwd,'\\Tools']);\n\nfigure, MP = showImagesAll(MatrixPrint,titleImages,vect_index_RGB,flag_cut_bounds,dim_cut,0);\n\ncd 'Outputs'\nfor ii = 1 : size(MP,4)\n imwrite(MP(:,:,:,ii),sprintf('%s.png',algorithms{ii}));\nend\nimwrite(showPan(I_PAN,0,1,flag_cut_bounds,dim_cut),'PAN.png')\ncd .." }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/printImage.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Print EPS image.\n% \n% Interface:\n% printImage(I_MS,title)\n%\n% Inputs:\n% I_MS: Image to print;\n% title: Filename.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction printImage(I_MS,title)\n\nfigure,imshow(I_MS,'Border','tight','InitialMagnification',100);\nprint(sprintf(title,'.eps'),'-depsc2','-r300');\n% print(sprintf(title,'.png'),'-dpng','-r400');\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/rectangleonimage.m", "content": "function ent=rectangleonimage(pic,sw,n, ch, c, scale, type)\n% sw: the location of the up-left, down-right\n% n: the width of the line\n% ch: ch = 1 (gray image); ch = 3 (color image) \n% c: the color of the line: c=1(red); c=2(green); c=3(blue);c=others\n% scale: the salce of zooming in for SR\n% type =1 (put to down-left); type =2 (put to down-right); \n% type =3 (put to up-right); type =4 (put to up-left); \n% Liang-Jian Deng (UESTC)\n% improved time: 2017-3-11\n%==============================%\n\nif nargin< 5\n scale = [];\nend\nx0=sw(1);x1=sw(2);y0=sw(3);y1=sw(4);\n[p q ch]=size(pic);\n\nmax_val = 1;\n\n%ch=1:gray image; ch=3: color image\nif ch==1\n if c==1\n pic(x0:x1,y0:y0+n)=max_val;\n pic(x0:x1,y1-n:y1)=max_val;\n pic(x0:x0+n,y0:y1)=max_val;\n pic(x1-n:x1,y0:y1)=max_val;\n elseif c==2\n pic(x0:x1,y0:y0+n)=0;\n pic(x0:x1,y1-n:y1)=0;\n pic(x0:x0+n,y0:y1)=0;\n pic(x1-n:x1,y0:y1)=0;\n else\n pic(x0:x1,y0:y0+n)=max_val-pic(x0:x1,y0:y0+n); %ȡ\n pic(x0:x1,y1-n:y1)=max_val- pic(x0:x1,y1-n:y1);\n pic(x0:x0+n,y0:y1)=max_val-pic(x0:x0+n,y0:y1);\n pic(x1-n:x1,y0:y1)=max_val-pic(x1-n:x1,y0:y1);\n end\nend\n\nif ch==3\n if c==1\n pic(x0:x1,y0:y0+n,1)=max_val; pic(x0:x1,y0:y0+n,2)=0; pic(x0:x1,y0:y0+n,3)=0;\n pic(x0:x1,y1-n:y1,1)=max_val; pic(x0:x1,y1-n:y1,2)=0; pic(x0:x1,y1-n:y1,3)=0;\n pic(x0:x0+n,y0:y1,1)=max_val; pic(x0:x0+n,y0:y1,2)=0; pic(x0:x0+n,y0:y1,3)=0;\n pic(x1-n:x1,y0:y1,1)=max_val; pic(x1-n:x1,y0:y1,2)=0; pic(x1-n:x1,y0:y1,3)=0;\n \n elseif c==2\n pic(x0:x1,y0:y0+n,1)=0;pic(x0:x1,y0:y0+n,2)=max_val;pic(x0:x1,y0:y0+n,3)=0;\n pic(x0:x1,y1-n:y1,1)=0;pic(x0:x1,y1-n:y1,2)=max_val;pic(x0:x1,y1-n:y1,3)=0;\n pic(x0:x0+n,y0:y1,1)=0;pic(x0:x0+n,y0:y1,2)=max_val;pic(x0:x0+n,y0:y1,3)=0;\n pic(x1-n:x1,y0:y1,1)=0;pic(x1-n:x1,y0:y1,2)=max_val;pic(x1-n:x1,y0:y1,3)=0;\n\n elseif c==3 \n pic(x0:x1,y0:y0+n,1)=0;pic(x0:x1,y0:y0+n,2)=0;pic(x0:x1,y0:y0+n,3)=max_val;\n pic(x0:x1,y1-n:y1,1)=0;pic(x0:x1,y1-n:y1,2)=0;pic(x0:x1,y1-n:y1,3)=max_val;\n pic(x0:x0+n,y0:y1,1)=0;pic(x0:x0+n,y0:y1,2)=0;pic(x0:x0+n,y0:y1,3)=max_val;\n pic(x1-n:x1,y0:y1,1)=0;pic(x1-n:x1,y0:y1,2)=0;pic(x1-n:x1,y0:y1,3)=max_val;\n\n else %inverse\n pic(x0:x1,y0:y0+n,1:3)=max_val-pic(x0:x1,y0:y0+n,1:3);\n pic(x0:x1,y1-n:y1,1:3)=max_val-pic(x0:x1,y1-n:y1,1:3);\n pic(x0:x0+n,y0:y1,1:3)=max_val-pic(x0:x0+n,y0:y1,1:3);\n pic(x1-n:x1,y0:y1,1:3)=max_val-pic(x1-n:x1,y0:y1,1:3);\n end\nend\n\nent=pic; \nsampIm = pic(x0:x1, y0:y1, :);\nSampIm = imresize(sampIm, scale,'nearest'); % nearest to zooming in the local part\nswitch type\n case 1 % put zoom in image on the down-left\n [a, b, third] = size(SampIm);\n ent((p-a+1):p,1:b, :) = SampIm;\n case 2 % put zoom in image on the down-left\n [a, b, third] = size(SampIm);\n ent((p-a+1):p,(q-b+1):q, :) = SampIm;\n \n case 3 % put zoom in image on the up-right\n [a, b, third] = size(SampIm);\n ent(1:a,(q-b+1):q, :) = SampIm;\n \n case 4 % put zoom in image on the up-right\n [a, b, third] = size(SampIm);\n ent(1:a,1:b, :) = SampIm; \nend\n\n\n\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/resize_images.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description:\n% Resize_images generates the low resolution panchromatic (PAN) and multispectral (MS) images according to Wald's protocol. \n% \n% Interface:\n% [I_MS_LR, I_PAN_LR] = resize_images(I_MS,I_PAN,ratio,sensor)\n% \n% Inputs:\n% \tI_MS: MS image upsampled at PAN scale;\n% I_PAN: PAN image;\n% ratio: Scale ratio between MS and PAN. Pre-condition: Integer value;\n% sensor: String for type of sensor (e.g. 'WV2', 'IKONOS').\n% \n% Outputs:\n% I_MS_LR: Low Resolution MS image;\n% I_PAN_LR: Low Resolution PAN image.\n% \n% References:\n% [Wald97] L. Wald, T. Ranchin, and M. Mangolini, Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,\n% Photogrammetric Engineering and Remote Sensing, vol. 63, no. 6, pp. 691699, June 1997.\n% [Aiazzi02] B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, Context-driven fusion of high spatial and spectral resolution images based on\n% oversampled multiresolution analysis, IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 10, pp. 23002312, October\n% 2002.\n% [Aiazzi06] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTF-tailored multiscale fusion of high-resolution MS and Pan imagery,\n% Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 591596, May 2006.\n% [Vivone14a] G. Vivone, R. Restaino, M. Dalla Mura, G. Licciardi, and J. Chanussot, Contrast and error-based fusion schemes for multispectral\n% image pansharpening, IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 5, pp. 930934, May 2014.\n% [Vivone15] G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. Licciardi, R. Restaino, and L. Wald, A Critical Comparison Among Pansharpening Algorithms, \n% IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 25652586, May 2015.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction [I_MS_LR, I_PAN_LR] = resize_images(I_MS,I_PAN,ratio,sensor)\n\nI_MS = double(I_MS);\nI_PAN = double(I_PAN);\n \nI_MS_LP = MTF(I_MS,sensor,ratio);\n\n%%% Decimation MS\nI_MS_LP_D = zeros(round(size(I_MS,1)/ratio),round(size(I_MS,2)/ratio),size(I_MS,3));\nfor idim = 1 : size(I_MS,3)\n I_MS_LP_D(:,:,idim) = imresize(I_MS_LP(:,:,idim),1/ratio,'nearest');\nend\n\nI_MS_LR = double(I_MS_LP_D);\n\nI_PAN_LR = imresize(I_PAN, 1/ratio);\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImage4.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print a four-band multispectral image.\n% \n% Interface:\n% showImage4(I_F,print,id,flag_cut_bounds,dim_cut,thvalues,L)\n%\n% Inputs:\n% I_MS: Four band multispectral image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% L: Radiomatric resolution of the input image.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction showImage4(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L)\n\nif flag_cut_bounds\n I_MS = I_MS(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\nend\n\nif th_values\n I_MS(I_MS > 2^L) = 2^L;\n I_MS(I_MS < 0) = 0;\nend\n\nIMN = viewimage(I_MS(:,:,1:3));\nIMN = IMN(:,:,3:-1:1);\n\nif print\n printImage(IMN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImage4LR.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print the original four-band multispectral image.\n% \n% Interface:\n% showImage4LR(I_F,print,id,flag_cut_bounds,dim_cut,thvalues,L,ratio)\n%\n% Inputs:\n% I_MS: Four band multispectral image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% L: Radiomatric resolution of the input image;\n% ratio: Resize factor.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction showImage4LR(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L,ratio)\n\nif flag_cut_bounds\n I_MS = I_MS(round(dim_cut/ratio):end-round(dim_cut/ratio),round(dim_cut/ratio):end-round(dim_cut/ratio),:);\nend\n\nif th_values\n I_MS(I_MS > 2^L) = 2^L;\n I_MS(I_MS < 0) = 0;\nend\n\nIMN = viewimage(I_MS(:,:,1:3));\nIMN = IMN(:,:,3:-1:1);\n\nif print\n printImage(IMN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImage4LR_zoomin.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print the original four-band multispectral image.\n% \n% Interface:\n% showImage4LR(I_F,print,id,flag_cut_bounds,dim_cut,thvalues,L,ratio)\n%\n% Inputs:\n% I_MS: Four band multispectral image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% L: Radiomatric resolution of the input image;\n% ratio: Resize factor.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction showImage4LR_zoomin(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L,ratio, location1, location2)\n\nif flag_cut_bounds\n I_MS = I_MS(round(dim_cut/ratio):end-round(dim_cut/ratio),round(dim_cut/ratio):end-round(dim_cut/ratio),:);\nend\n\nif th_values\n I_MS(I_MS > 2^L) = 2^L;\n I_MS(I_MS < 0) = 0;\nend\n\nIMN = viewimage(I_MS(:,:,1:3));\nIMN = IMN(:,:,3:-1:1);\n\nif isempty(location2)\n ent=rectangleonimage(IMN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n figure,imshow(ent,[])\nelse\n % type =1 (put to down-left); type =2 (put to down-right); \n % type =3 (put to up-right); type =4 (put to up-left); \n ent=rectangleonimage(IMN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n ent=rectangleonimage(ent,location2,1, 3, 2, 3, 2); % put close-up to down-right corner\n figure,imshow(ent,[])\nend\n\nif print\n printImage(IMN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImage4_zoomin.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print a four-band multispectral image.\n% \n% Interface:\n% showImage4(I_F,print,id,flag_cut_bounds,dim_cut,thvalues,L)\n%\n% Inputs:\n% I_MS: Four band multispectral image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% L: Radiomatric resolution of the input image.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction showImage4_zoomin(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L, location1, location2)\n\nif flag_cut_bounds\n I_MS = I_MS(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\nend\n\nif th_values\n I_MS(I_MS > 2^L) = 2^L;\n I_MS(I_MS < 0) = 0;\nend\n\nIMN = viewimage(I_MS(:,:,1:3));\nIMN = IMN(:,:,3:-1:1);\n\n\nif isempty(location2)\n ent=rectangleonimage(IMN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n figure,imshow(ent,[])\nelse\n % type =1 (put to down-left); type =2 (put to down-right); \n % type =3 (put to up-right); type =4 (put to up-left); \n ent=rectangleonimage(IMN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n ent=rectangleonimage(ent,location2,1, 3, 2, 3, 2); % put close-up to down-right corner\n figure,imshow(ent,[])\nend\n\n\nif print\n printImage(IMN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImage8.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print an eight-band multispectral image.\n% \n% Interface:\n% showImage8(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L)\n%\n% Inputs:\n% I_MS: Eight band multispectral image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% L: Radiomatric resolution of the input image.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction showImage8(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L)\n\nif flag_cut_bounds\n I_MS = I_MS(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\nend\n\nif th_values\n I_MS(I_MS > 2^L) = 2^L;\n I_MS(I_MS < 0) = 0;\nend\n\nif id == 1\n IMN = viewimage(I_MS(:,:,[1,3,5]));\n IMN = IMN(:,:,3:-1:1);\nelse\n IMN = viewimage(I_MS(:,:,[1,3,5]),[0.01 0.995]);\n IMN = IMN(:,:,3:-1:1);\nend\n\nif print\n printImage(IMN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImage8LR.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print an eight-band multispectral image.\n% \n% Interface:\n% showImage8LR(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L,ratio)\n%\n% Inputs:\n% I_MS: Eight band multispectral image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% L: Radiomatric resolution of the input image;\n% ratio: Resize factor.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction showImage8LR(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L,ratio)\n\nif flag_cut_bounds\n I_MS = I_MS(round(dim_cut/ratio):end-round(dim_cut/ratio),round(dim_cut/ratio):end-round(dim_cut/ratio),:);\nend\n\nif th_values\n I_MS(I_MS > 2^L) = 2^L;\n I_MS(I_MS < 0) = 0;\nend\n\nif id == 1\n \n IMN = viewimage(I_MS(:,:,[1,3,5]));\n IMN = IMN(:,:,3:-1:1);\nelse\n IMN = viewimage(I_MS(:,:,[1,3,5]),[0.01 0.995]);\n IMN = IMN(:,:,3:-1:1);\nend\n\nif print\n printImage(IMN,sprintf('Outputs/%d.eps',id));\nend\n\nend\n\n" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImage8LR_zoomin.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print an eight-band multispectral image.\n% \n% Interface:\n% showImage8LR(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L,ratio)\n%\n% Inputs:\n% I_MS: Eight band multispectral image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% L: Radiomatric resolution of the input image;\n% ratio: Resize factor.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction showImage8LR_zoomin(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L,ratio, location1, location2)\n\nif flag_cut_bounds\n I_MS = I_MS(round(dim_cut/ratio):end-round(dim_cut/ratio),round(dim_cut/ratio):end-round(dim_cut/ratio),:);\nend\n\nif th_values\n I_MS(I_MS > 2^L) = 2^L;\n I_MS(I_MS < 0) = 0;\nend\n\nif id == 1\n \n IMN = viewimage(I_MS(:,:,[1,3,5]));\n IMN = IMN(:,:,3:-1:1);\nelse\n IMN = viewimage(I_MS(:,:,[1,3,5]),[0.01 0.995]);\n IMN = IMN(:,:,3:-1:1);\nend\n\n\nif isempty(location2)\n ent=rectangleonimage(IMN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n figure,imshow(ent,[])\nelse\n % type =1 (put to down-left); type =2 (put to down-right); \n % type =3 (put to up-right); type =4 (put to up-left); \n ent=rectangleonimage(IMN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n ent=rectangleonimage(ent,location2,1, 3, 2, 3, 2); % put close-up to down-right corner\n figure,imshow(ent,[])\nend\n\n\nif print\n printImage(IMN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImage8_zoomin.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print an eight-band multispectral image.\n% \n% Interface:\n% showImage8(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L)\n%\n% Inputs:\n% I_MS: Eight band multispectral image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n% th_values: Flag. If th_values == 1, apply an hard threshold to the dynamic range;\n% L: Radiomatric resolution of the input image.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction showImage8_zoomin(I_MS,print,id,flag_cut_bounds,dim_cut,th_values,L, location1, location2)\n\nif flag_cut_bounds\n I_MS = I_MS(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\nend\n\nif th_values\n I_MS(I_MS > 2^L) = 2^L;\n I_MS(I_MS < 0) = 0;\nend\n\nif id == 1\n IMN = viewimage(I_MS(:,:,[1,3,5]));\n IMN = IMN(:,:,3:-1:1);\nelse\n IMN = viewimage(I_MS(:,:,[1,3,5]),[0.01 0.995]);\n IMN = IMN(:,:,3:-1:1);\nend\n\nif isempty(location2)\n ent=rectangleonimage(IMN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n figure,imshow(ent,[])\nelse\n % type =1 (put to down-left); type =2 (put to down-right); \n % type =3 (put to up-right); type =4 (put to up-left); \n ent=rectangleonimage(IMN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n ent=rectangleonimage(ent,location2,1, 3, 2, 3, 2); % put close-up to down-right corner\n figure,imshow(ent,[])\nend\n\nif print\n printImage(IMN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImagesAll.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize all the images applying the same stretching for visual comparison.\n% \n% Interface:\n% MatrixPrint = showImagesAll(MatrixImage,titleImages,vect_index_RGB,flag_cut_bounds,dim_cut,flagPAN)\n%\n% Inputs:\n% MatrixImage: Matrix that contains all the images to visualize; Size: [M x N x B x Z], where [M x N] is the\n% dimension of a single image band, B represents the number of bands for each image, and Z is the number of images to plot.\n% titleImages: Vector of strings that represents the titles for each image to plot; Size: [1 x Z].\n% vect_index_RGB: Identify the bands to plot to obtain an RGB representation of the multispectral data;\n% flag_cut_bounds: Cut the boundaries of the images to plot;\n% dim_cut: Define the dimension of the boundary cut;\n% flagPAN: Flag. If flagPAN == 1, the first image to plot is the panchromatic image otherwise it is the ground-truth.\n%\n% Outputs:\n% MatrixPrint: Matrix, with the same structure of MatrixImage, which contains the plotted images.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction MatrixPrint = showImagesAll(MatrixImage,titleImages,vect_index_RGB,flag_cut_bounds,dim_cut,flagPAN)\n\nif flag_cut_bounds\n MatrixImageCat = zeros(numel(dim_cut:size(MatrixImage,1)-dim_cut),numel(dim_cut:size(MatrixImage,2)-dim_cut),size(MatrixImage,3),size(MatrixImage,4));\n for ii = 1 : size(MatrixImageCat,4)\n t = MatrixImage(:,:,:,ii);\n MatrixImageCat(:,:,:,ii) = t(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\n end\nelse\n MatrixImageCat = MatrixImage;\nend\n\n[r,c,~] = size(MatrixImageCat(:,:,:,1));\n\nif flagPAN\n T = [];\n for ii = 2 : size(MatrixImageCat,4)\n T = cat(2,T,MatrixImageCat(:,:,vect_index_RGB,ii));\n end \nelse\n T = [];\n for ii = 1 : size(MatrixImageCat,4)\n T = cat(2,T,MatrixImageCat(:,:,vect_index_RGB,ii));\n end\nend\n\nIMN = viewimage2(T);\n\nif flagPAN\n MatrixPrint = zeros(size(MatrixImageCat(:,:,vect_index_RGB,:)));\n MatrixPrint(:,:,:,1) = viewimage2(MatrixImageCat(:,:,vect_index_RGB,1));\n ind_c = 1;\n for ii = 2 : size(MatrixImageCat,4) \n MatrixPrint(:,:,:,ii) = IMN(1 : r,ind_c : ind_c + c - 1,:);\n ind_c = ind_c + c;\n end \nelse\n MatrixPrint = zeros(size(MatrixImageCat(:,:,vect_index_RGB,:)));\n ind_c = 1;\n for ii = 1 : size(MatrixImageCat,4) \n MatrixPrint(:,:,:,ii) = IMN(1 : r,ind_c : ind_c + c - 1,:);\n ind_c = ind_c + c;\n end\nend\n\n% ha = tight_subplot(5,5,[.06 .03],[.01 .06],[.01 .01]);\nha = tight_subplot(4,7,[.02 0],[.01 .03],[.0 .0]);\nfor ii = 1 : size(MatrixImageCat,4)\n axes(ha(ii)); imshow(MatrixPrint(:,:,:,ii),[]);\n title(ha(ii),titleImages{ii});\nend\n \nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showImagesAllOld.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize all the images applying the same stretching for visual comparison.\n% \n% Interface:\n% MatrixPrint = showImagesAll(MatrixImage,titleImages,vect_index_RGB,flag_cut_bounds,dim_cut,flagPAN)\n%\n% Inputs:\n% MatrixImage: Matrix that contains all the images to visualize; Size: [M x N x B x Z], where [M x N] is the\n% dimension of a single image band, B represents the number of bands for each image, and Z is the number of images to plot.\n% titleImages: Vector of strings that represents the titles for each image to plot; Size: [1 x Z].\n% vect_index_RGB: Identify the bands to plot to obtain an RGB representation of the multispectral data;\n% flag_cut_bounds: Cut the boundaries of the images to plot;\n% dim_cut: Define the dimension of the boundary cut;\n% flagPAN: Flag. If flagPAN == 1, the first image to plot is the panchromatic image otherwise it is the ground-truth.\n%\n% Outputs:\n% MatrixPrint: Matrix, with the same structure of MatrixImage, which contains the plotted images.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction MatrixPrint = showImagesAll(MatrixImage,titleImages,vect_index_RGB,flag_cut_bounds,dim_cut,flagPAN)\n\nif flag_cut_bounds\n MatrixImageCat = zeros(numel(dim_cut:size(MatrixImage,1)-dim_cut),numel(dim_cut:size(MatrixImage,2)-dim_cut),size(MatrixImage,3),size(MatrixImage,4));\n for ii = 1 : size(MatrixImageCat,4)\n t = MatrixImage(:,:,:,ii);\n MatrixImageCat(:,:,:,ii) = t(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\n end\nelse\n MatrixImageCat = MatrixImage;\nend\n\n[r,c,~] = size(MatrixImageCat(:,:,:,1));\n\nif flagPAN\n T = [];\n for ii = 2 : size(MatrixImageCat,4)\n T = cat(2,T,MatrixImageCat(:,:,vect_index_RGB,ii));\n end \nelse\n T = [];\n for ii = 1 : size(MatrixImageCat,4)\n T = cat(2,T,MatrixImageCat(:,:,vect_index_RGB,ii));\n end\nend\n\nIMN = viewimage2(T);\n\nif flagPAN\n MatrixPrint = zeros(size(MatrixImageCat(:,:,vect_index_RGB,:)));\n MatrixPrint(:,:,:,1) = viewimage2(MatrixImageCat(:,:,vect_index_RGB,1));\n ind_c = 1;\n for ii = 2 : size(MatrixImageCat,4) \n MatrixPrint(:,:,:,ii) = IMN(1 : r,ind_c : ind_c + c - 1,:);\n ind_c = ind_c + c;\n end \nelse\n MatrixPrint = zeros(size(MatrixImageCat(:,:,vect_index_RGB,:)));\n ind_c = 1;\n for ii = 1 : size(MatrixImageCat,4) \n MatrixPrint(:,:,:,ii) = IMN(1 : r,ind_c : ind_c + c - 1,:);\n ind_c = ind_c + c;\n end\nend\n\nha = tight_subplot(5,5,[.06 .03],[.01 .06],[.01 .01]);\n% ha = tight_subplot(5,5,[.02 0],[.01 .03],[.0 .0]);\n\nfor ii = 1 : size(MatrixImageCat,4)\n axes(ha(ii)); imshow(MatrixPrint(:,:,:,ii),[]);\n title(ha(ii),titleImages{ii});\nend\n \nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showPan.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print the panchromatic image.\n% \n% Interface:\n% showPan(Pan,print,id,flag_cut_bounds,dim_cut)\n%\n% Inputs:\n% Pan: Panchromatic image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction IN = showPan(Pan,print,id,flag_cut_bounds,dim_cut)\n\nif flag_cut_bounds\n Pan = Pan(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\nend\n\nIN = viewimage(Pan);\n\nif print\n printImage(IN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/showPan_zoomin.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualize and print the panchromatic image.\n% \n% Interface:\n% showPan(Pan,print,id,flag_cut_bounds,dim_cut)\n%\n% Inputs:\n% Pan: Panchromatic image;\n% print: Flag. If print == 1, print EPS image;\n% id: Identifier (name) of the printed EPS image;\n% flag_cut_bounds: Cut the boundaries of the viewed Panchromatic image;\n% dim_cut: Define the dimension of the boundary cut;\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction IN = showPan_zoomin(Pan,print,id,flag_cut_bounds,dim_cut, location1, location2)\n\nratio = 4;\nif flag_cut_bounds\n %Pan = Pan(dim_cut:end-dim_cut,dim_cut:end-dim_cut,:);\n Pan = Pan(round(dim_cut/ratio):end-round(dim_cut/ratio),round(dim_cut/ratio):end-round(dim_cut/ratio),:);\n\nend\n\nIN = viewimage(Pan);\n\nif isempty(location2)\n ent=rectangleonimage(IN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n figure,imshow(ent,[])\nelse\n % type =1 (put to down-left); type =2 (put to down-right); \n % type =3 (put to up-right); type =4 (put to up-left); \n ent=rectangleonimage(IN,location1,1, 3, 3, 3, 1); % put close-up to up-right corner\n ent=rectangleonimage(ent,location2,1, 3, 2, 3, 2); % put close-up to down-right corner\n figure,imshow(ent,[])\nend\n\nif print\n printImage(IN,sprintf('Outputs/%d.eps',id));\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/tight_subplot.m", "content": "function ha = tight_subplot(Nh, Nw, gap, marg_h, marg_w)\n\n% tight_subplot creates \"subplot\" axes with adjustable gaps and margins\n%\n% ha = tight_subplot(Nh, Nw, gap, marg_h, marg_w)\n%\n% in: Nh number of axes in hight (vertical direction)\n% Nw number of axes in width (horizontaldirection)\n% gap gaps between the axes in normalized units (0...1)\n% or [gap_h gap_w] for different gaps in height and width \n% marg_h margins in height in normalized units (0...1)\n% or [lower upper] for different lower and upper margins \n% marg_w margins in width in normalized units (0...1)\n% or [left right] for different left and right margins \n%\n% out: ha array of handles of the axes objects\n% starting from upper left corner, going row-wise as in\n% going row-wise as in\n%\n% Example: ha = tight_subplot(3,2,[.01 .03],[.1 .01],[.01 .01])\n% for ii = 1:6; axes(ha(ii)); plot(randn(10,ii)); end\n% set(ha(1:4),'XTickLabel',''); set(ha,'YTickLabel','')\n\n% Pekka Kumpulainen 20.6.2010 @tut.fi\n% Tampere University of Technology / Automation Science and Engineering\n\n\nif nargin<3; gap = .02; end\nif nargin<4 || isempty(marg_h); marg_h = .05; end\nif nargin<5; marg_w = .05; end\n\nif numel(gap)==1; \n gap = [gap gap];\nend\nif numel(marg_w)==1; \n marg_w = [marg_w marg_w];\nend\nif numel(marg_h)==1; \n marg_h = [marg_h marg_h];\nend\n\naxh = (1-sum(marg_h)-(Nh-1)*gap(1))/Nh; \naxw = (1-sum(marg_w)-(Nw-1)*gap(2))/Nw;\n\npy = 1-marg_h(2)-axh; \n\nha = zeros(Nh*Nw,1);\nii = 0;\nfor ih = 1:Nh\n px = marg_w(1);\n \n for ix = 1:Nw\n ii = ii+1;\n ha(ii) = axes('Units','normalized', ...\n 'Position',[px py axw axh], ...\n 'XTickLabel','', ...\n 'YTickLabel','');\n px = px+axw+gap(2);\n end\n py = py-axh-gap(1);\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/viewimage.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualization [3-2-1] of images with 3 bands by exploiting linear stretching and fixing the saturation. \n% \n% Interface:\n% ImageToView = viewimage(ImageToView,tol)\n%\n% Inputs:\n% ImageToView: Image to view;\n% tol: Saturation; Default values: [0.01 0.99] equal for all the three bands.\n%\n% Outputs:\n% ImageToView: Image to view.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction ImageToView = viewimage(ImageToView,tol1,tol2,tol3)\n\niptsetpref('ImshowBorder', 'tight')\nImageToView = double(ImageToView);\nL=size(ImageToView,3);\nif (L<3)\n ImageToView=ImageToView(:,:,[1 1 1]);\nend\n\nif nargin == 1\n tol1 = [0.01 0.99];\nend\nif nargin <= 2\n tol = [tol1;tol1;tol1];\n ImageToView = linstretch(ImageToView,tol);\n figure,imshow(ImageToView(:,:,3:-1:1),[])\nelseif nargin == 4\n if sum(tol1(2)+tol2(2)+tol3(2)) <= 3\n tol = [tol1;tol2;tol3];\n ImageToView = linstretch(ImageToView,tol);\n figure,imshow(ImageToView(:,:,3:-1:1),[])\n else\n tol = [tol1;tol2;tol3];\n [N,M,~] = size(ImageToView);\n NM = N*M;\n for i=1:3\n b = reshape(double(uint16(ImageToView(:,:,i))),NM,1);\n b(btol(i,2))=tol(i,2);\n b = (b-tol(i,1))/(tol(i,2)-tol(i,1));\n ImageToView(:,:,i) = reshape(b,N,M);\n end\n figure,imshow(ImageToView(:,:,3:-1:1),[])\n end\nend\n\niptsetpref('ImshowBorder', 'loose')\n\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Linear Stretching. \n% \n% Interface:\n% ImageToView = linstretch(ImageToView,tol)\n%\n% Inputs:\n% ImageToView: Image to stretch;\n% tol: ;\n%\n% Outputs:\n% ImageToView: Stretched image.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction ImageToView = linstretch(ImageToView,tol)\n\n[N,M,~] = size(ImageToView);\nNM = N*M;\nfor i=1:3\n b = reshape(double(uint16(ImageToView(:,:,i))),NM,1);\n [hb,levelb] = hist(b,max(b)-min(b));\n chb = cumsum(hb);\n t(1)=ceil(levelb(find(chb>NM*tol(i,1), 1 )));\n t(2)=ceil(levelb(find(chbt(2))=t(2);\n b = (b-t(1))/(t(2)-t(1));\n ImageToView(:,:,i) = reshape(b,N,M);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/Tools/viewimage2.m", "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Visualization [3-2-1] of images with 3 bands by exploiting linear stretching and fixing the saturation. \n% \n% Interface:\n% ImageToView = viewimage2(ImageToView,tol)\n%\n% Inputs:\n% ImageToView: Image to view;\n% tol: Saturation; Default values: [0.01 0.99] equal for all the three bands.\n%\n% Outputs:\n% ImageToView: Image to view.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction ImageToView = viewimage2(ImageToView,tol1,tol2,tol3)\n\niptsetpref('ImshowBorder', 'tight')\nImageToView = double(ImageToView);\nL=size(ImageToView,3);\nif (L<3)\n ImageToView=ImageToView(:,:,[1 1 1]);\nend\n\nif nargin == 1\n tol1 = [0.01 0.99];\nend\nif nargin <= 2\n tol = [tol1;tol1;tol1];\n ImageToView = linstretch(ImageToView,tol);\nelseif nargin == 4\n if sum(tol1(2)+tol2(2)+tol3(2)) <= 3\n tol = [tol1;tol2;tol3];\n ImageToView = linstretch(ImageToView,tol);\n else\n tol = [tol1;tol2;tol3];\n [N,M,~] = size(ImageToView);\n NM = N*M;\n for i=1:3\n b = reshape(double(uint16(ImageToView(:,:,i))),NM,1);\n b(btol(i,2))=tol(i,2);\n b = (b-tol(i,1))/(tol(i,2)-tol(i,1));\n ImageToView(:,:,i) = reshape(b,N,M);\n end\n end\nend\n\niptsetpref('ImshowBorder', 'loose')\n\nend\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Description: \n% Linear Stretching. \n% \n% Interface:\n% ImageToView = linstretch(ImageToView,tol)\n%\n% Inputs:\n% ImageToView: Image to stretch;\n% tol: ;\n%\n% Outputs:\n% ImageToView: Stretched image.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nfunction ImageToView = linstretch(ImageToView,tol)\n\n[N,M,~] = size(ImageToView);\nNM = N*M;\nfor i=1:3\n b = reshape(double(uint16(ImageToView(:,:,i))),NM,1);\n [hb,levelb] = hist(b,max(b)-min(b));\n chb = cumsum(hb);\n t(1)=ceil(levelb(find(chb>NM*tol(i,1), 1 )));\n t(2)=ceil(levelb(find(chbt(2))=t(2);\n b = (b-t(1))/(t(2)-t(1));\n ImageToView(:,:,i) = reshape(b,N,M);\nend\n\nend" }, { "path": "02-Test-toolbox-for-traditional-and-DL(Matlab)/readme.md", "content": "# Test toolbox for traditional and DL\n\"Test toolbox for traditional and DL\" for simultaneously evaluating traditional and DL approaches, and finally output metrics and eps-format figures for your latex editing\n\n[English](https://github.com/.md) | [简体中文](https://github.com.md)\n\n\nThis repository is the official Matlab implementation of our IEEE GRSM paper “Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks”, 2022 ([paper](https://github.com/liangjiandeng/liangjiandeng.github.io/tree/master/papers/2022/review-grsm2022.pdf) | [homepage](https://github.com/liangjiandeng/DLPan-Toolbox)).\n\n## Features\n\n\n## Requirements\n* Matlab software\n\n## Quick Start\n\n### Full-resolution Evaluation\n\n* Directly run ``Demo_Full_Resolution.m`` which includes an WV3 example. After running this demo, readers can understand the whole procedure.\n\n* Note: the test dataset of full-resolution are too huge to upload to GitHub, thus we provide cloud links to readers to download them to\n successfully run this demo, including:\n - i) Download link for full-resolution WV3-NewYork example (named \"NY1_WV3_FR.mat\"): [[Link]](https://drive.google.com/file/d/1j1nyHuBxsNzIn-UEwZUgeziGCAFMLes9/view?usp=sharing) (put into the folder of \"1_TestData/Datasets Testing\")\n \n - ii) Download link of DL's results for full-resolution WV3-NewYork example: [[Link]](https://drive.google.com/file/d/16FSxdq6BY7STbmMzxcxJ5atNQ7ZV3mPT/view?usp=sharing) (put into the folder of \"'2_DL_Result/WV3\")\n \n* Once you have above datasets, you can run this demo successfully, then understand how this demo run!\n\n\n\n\n### Reduced-resolution Evaluation\n\n* Directly run ``Demo_Reduced_Resolution.m`` which includes an WV3 example. After running this demo, readers can understand the whole procedure.\n\n* Note: the test dataset of reduced-resolution are too huge to upload to GitHub, thus we provide cloud links to readers to download them to\n successfully run this demo, including:\n - i) Download link for reduced-resolution WV3-NewYork example (named \"NY1_WV3_RR.mat\"): same link as above i), then put into the folder of \"1_TestData/Datasets Testing\"\n \n - ii) Download link of DL's results for reduced-resolution WV3-NewYork example: same link as above ii), then put into the folder of \"2_DL_Result/WV3\"\n \n* Once you have above datasets, you can run this demo successfully, then understand how this demo run!\n\n\n### Others\n\n* You may find the quantitative results from Tex files such as ``FR_Assessment.tex``, ``RR_Assessment.tex`` and ``Avg_RR_Assessment.tex``, then copy for your Latex editing.\n* You may also find the generated high-resolution eps-format figures in the folder of \"3_EPS\" for your Latex editing. \n\n\n## Acknowledgement\n- We appreciate the great contribution of [Xiao Wu](https://xiaoxiao-woo.github.io/) who is a graduate student in [UESTC](https://www.uestc.edu.cn/) to this toolbox.\n\n\n## Citation\n* If you use this toolbox, please kindly cite our paper:\n\n```bibtex\n@ARTICLE{deng2022grsm,\nauthor={L.-J. Deng, G. Vivone, M. E. Paoletti, G. Scarpa, J. He, Y. Zhang, J. Chanussot, and A. Plaza},\nbooktitle={IEEE Geoscience and Remote Sensing Magazine},\ntitle={Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks},\nyear={2022},\npages={},\n}\n```\n\n* Also, the codes of traditional methods are from the \"pansharpening toolbox for distribution\", thus please cite the corresponding paper:\n```bibtex\n@ARTICLE{vivone2021grsm,\n author={Vivone, Gemine and Dalla Mura, Mauro and Garzelli, Andrea and Restaino, Rocco and Scarpa, Giuseppe and Ulfarsson, Magnus O. and Alparone, Luciano and Chanussot, Jocelyn},\n journal={IEEE Geoscience and Remote Sensing Magazine}, \n title={A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods}, \n year={2021},\n volume={9},\n number={1},\n pages={53-81},\n doi={10.1109/MGRS.2020.3019315}\n}\n```\n\n \n\n\n## License & Copyright\nThis project is open sourced under GNU General Public License v3.0.\n\n" }, { "path": "03-Data-Simulation(Matlab)/01-DataSimu/QB/readme.md.txt", "content": "" }, { "path": "03-Data-Simulation(Matlab)/Demo_DataSimu_qb.m", "content": "%% This is a demo to segment image into small patches (and big test imgs) \n% for the training=64x64x8 (and testing=256x256x8) pansharpening in remote sensing\n% L.-J. Deng(UESTC)\n% 2020-10-04\n%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nclear; close all;\n% please download the QB data from the website: \n% then put into the folder of \"Imgs_qb\"\n% at last run the demo directly to get patching examples\nfiles = dir('Imgs_qb/*.mat'); \nleng = length(files);\n\nPre_NumInd = 1;\nPre_NumInd_test = 1;\nscale = 4;\n\n%% ----------------------------------\nfor i = 1:leng \n % load inpainted images\n str = files(i).name;\n dir = strcat('load', 32, 'Imgs_qb/', str);\n eval(dir)\n \n PAN2 = I_PAN;\n LMS2 = I_MS;\n GT2 = I_GT;\n MS2 = I_MS_LR; \n \n maxval = max(PAN2(:));\n figure,\n subplot(2,2,1), imshow(PAN2/maxval); title('original PAN')\n subplot(2,2,2), imshow(MS2(:,:,[3 2 1])/maxval); title('LR MS')\n subplot(2,2,3), imshow(GT2(:,:,[3 2 1])/maxval); title('GT')\n subplot(2,2,4), imshow(LMS2(:,:,[3 2 1])/maxval); title('UP MS')\n \n %% leave one half of data 1 as the test data! \n if (i==1) % take Indianapolis to get test imags;\n cut_num = 512;\n [a, b, c] = size(GT2);\n GT = GT2(:, cut_num+1:end,:); % for training dataset\n GT_test = GT2(:, 1:cut_num, :); % for testing dataset\n \n PAN = PAN2(:, cut_num+1:end);\n PAN_test = PAN2(:, 1:cut_num);\n \n LMS = LMS2(:, cut_num+1:end, :);\n LMS_test = LMS2(:, 1:cut_num, :);\n \n MS = MS2(:, fix(cut_num/4)+1:end, :);\n MS_test = MS2(:, 1:fix(cut_num/4), :);\n \n %% 1) Big Test Imgs: segment pan into big Imgs 512x512x8 (testing Exm)\n size_l_test = 128; size_h_test = 512; overlap_test = 1; % (for testing data: 0<=overlaop<=64)\n \n tic\n [gt_Oneimg_test, pan_Oneimg_test, ms_Oneimg_test, lms_Oneimg_test] = segImg_new(PAN_test, LMS_test, GT_test, MS_test, size_l_test, size_h_test, scale, overlap_test);\n toc\n \n % save the Imgs into a tensor\n [NumInd_test, ~, ~, ~] = size(gt_Oneimg_test);\n Post_NumInd_test = Pre_NumInd_test + NumInd_test - 1;\n\n fprintf(['%d-th Img. (test): ', 'Pre_NumInd_test = %d; ', ' Post_NumInd_test = %d \\n'], i, Pre_NumInd_test, Post_NumInd_test)\n % save data\n gt_tmp_test(Pre_NumInd_test: Post_NumInd_test, :, :, :) = gt_Oneimg_test; % gt tensor: Nx512x512x8\n pan_tmp_test(Pre_NumInd_test: Post_NumInd_test, :, :) = pan_Oneimg_test; % pan tensor: Nx512x512\n ms_tmp_test(Pre_NumInd_test: Post_NumInd_test, :, :, :) = ms_Oneimg_test; % ms tensor: Nx128x128x8\n lms_tmp_test(Pre_NumInd_test: Post_NumInd_test, :, :, :)= lms_Oneimg_test; % lms tensor: Nx512x512x8\n\n Pre_NumInd_test = Post_NumInd_test + 1; \n \n else\n GT = GT2;\n PAN = PAN2;\n LMS = LMS2;\n MS = MS2;\n end\n \n %% 2) small training patches (training)\n size_l = 16; size_h = 64; overlap = 4; % (for traning data: 0<=overlaop<=16)\n \n tic\n [gt_Oneimg, pan_Oneimg, ms_Oneimg, lms_Oneimg] = segImg_new(PAN, LMS, GT, MS, size_l, size_h, scale, overlap);\n toc\n\n % save the patches into a tensor\n [NumInd, ~, ~, ~] = size(gt_Oneimg);\n Post_NumInd = Pre_NumInd + NumInd - 1;\n \n fprintf(['%d-th Img.(patching for training): ', 'Pre_NumInd = %d; ', ' Post_NumInd = %d \\n'], i, Pre_NumInd, Post_NumInd)\n % save data\n gt_tmp1(Pre_NumInd: Post_NumInd, :, :, :) = gt_Oneimg; % gt tensor: Nx64x64x8\n pan_tmp1(Pre_NumInd: Post_NumInd, :, :) = pan_Oneimg; % pan tensor: Nx64x64\n ms_tmp1(Pre_NumInd: Post_NumInd, :, :, :) = ms_Oneimg; % ms tensor: Nx16x16x8\n lms_tmp1(Pre_NumInd: Post_NumInd, :, :, :)= lms_Oneimg; % lms tensor: Nx64x64x8\n \n Pre_NumInd = Post_NumInd + 1;\n \nend\n\n%% ==========================================================\n%% ==== Increase samples to 10,000 (NxCxHxW's inverse = WxHxCxN)\n%% ==========================================================\n\nexp_num = size(gt_tmp1, 1); \n\nif exp_num < 10000\n\n % Step2: two flips (lr + ud) to add examples\n gt_tmp(1:exp_num, :, :, :) = gt_tmp1;\n gt_tmp(exp_num+1:2*exp_num, :, :, :) = flip(gt_tmp1, 2); % two flips (lr + ud) to add examples\n gt_tmp(2*exp_num+1:3*exp_num, :, :, :) = flip(gt_tmp1, 3);\n\n ms_tmp(1:exp_num, :, :, :) = ms_tmp1;\n ms_tmp(exp_num+1:2*exp_num, :, :, :) = flip(ms_tmp1, 2); % two flips (lr + ud) to add examples\n ms_tmp(2*exp_num+1:3*exp_num, :, :, :) = flip(ms_tmp1, 3);\n\n lms_tmp(1:exp_num, :, :, :) = lms_tmp1;\n lms_tmp(exp_num+1:2*exp_num, :, :, :) = flip(lms_tmp1, 2); % two flips (lr + ud) to add examples\n lms_tmp(2*exp_num+1:3*exp_num, :, :, :) = flip(lms_tmp1, 3);\n\n pan_tmp(1:exp_num, :, :) = pan_tmp1;\n pan_tmp(exp_num+1:2*exp_num, :, :) = flip(pan_tmp1, 2); % two flips (lr + ud) to add examples\n pan_tmp(2*exp_num+1:3*exp_num, :, :) = flip(pan_tmp1, 3);\n\n % Step3: only select first 10000 patches for training:\n num_cut = 10000;\n gt_tmp(num_cut+1:end, :, :, :) = []; \n ms_tmp(num_cut+1:end, :, :, :) = []; \n lms_tmp(num_cut+1:end, :, :, :) = []; \n pan_tmp(num_cut+1:end, :, :) = []; \n \nelse\n num_cut = exp_num;\n \n gt_tmp = gt_tmp1;\n ms_tmp = ms_tmp1;\n lms_tmp=lms_tmp1;\n pan_tmp=pan_tmp1;\nend\n\n%% ==========================================================\n%% (A) generate training: 1) training data (90%); 2) validation data (10%); \n%% ==========================================================\nPost_NumInd = num_cut;\n\nnz_idx = randperm(Post_NumInd);\nnum_train = fix(0.9*Post_NumInd); % # training samples\nnum_valid = Post_NumInd - num_train ; % # validation samples\n\n%% ==== save to H5 file (NxCxHxW's inverse = WxHxCxN) =====\n%==========================================================\n%% == generate training dataset:\ngt = gt_tmp(nz_idx(1:num_train), :, :, :); % NxHxWxC=1x2x3x4\npan = pan_tmp(nz_idx(1:num_train), :, :); % NxHxW = 1x2x3 (PAN)\nms = ms_tmp(nz_idx(1:num_train), :, :, :);\nlms = lms_tmp(nz_idx(1:num_train), :, :, :);\n\n%--- for training data:\nfilename_train = '01-DataSimu/QB/train_qb_10000.h5';\n\ngt = permute(gt,[3 2 4 1]); % beyond 2G, have to change dimension\npan_t(1,:,:,:) = pan; % CxNxHxW = 1x2x3x4 (PAN)\npan = permute(pan_t,[4 3 1 2]); % WxHxCxN\nms = permute(ms,[3 2 4 1]); \nlms = permute(lms,[3 2 4 1]); \n\ngtsz = size(gt);\nmssz = size(ms);\nlmssz = size(lms);\npansz =size(pan);\n\n\nh5create(filename_train, '/gt', gtsz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_train, '/ms', mssz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_train, '/lms', lmssz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_train, '/pan', pansz(1:end), 'Datatype', 'double'); % width, height, channels, number \n\nh5write(filename_train, '/gt', double(gt), [1,1,1,1], size(gt));\nh5write(filename_train, '/ms', double(ms), [1,1,1,1], size(ms));\nh5write(filename_train, '/lms', double(lms), [1,1,1,1], size(lms));\nh5write(filename_train, '/pan', double(pan), [1,1,1,1], size(pan));\n\nclear gt ms lms pan pan_t\n\n%% == generate validation dataset:\ngt = gt_tmp(nz_idx(num_train+1: num_train+num_valid), :, :, :);\npan = pan_tmp(nz_idx(num_train+1: num_train+num_valid), :, :);\nms = ms_tmp(nz_idx(num_train+1: num_train+num_valid), :, :, :);\nlms = lms_tmp(nz_idx(num_train+1: num_train+num_valid), :, :, :);\n\n%--- for valid data:\nfilename_valid = '01-DataSimu/QB/valid_qb_10000.h5';\n\ngt = permute(gt,[3 2 4 1]); % beyond 2G, have to change dimension\npan_t(1, :,:,:) = pan; % NxHxWx1 = 1x2x3x4 (PAN)\npan = permute(pan_t,[4 3 1 2]);\nms = permute(ms,[3 2 4 1]); \nlms = permute(lms,[3 2 4 1]); \n\ngtsz = size(gt);\nmssz = size(ms);\npansz =size(pan);\n\nh5create(filename_valid, '/gt', gtsz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_valid, '/ms', mssz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_valid, '/lms', gtsz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_valid, '/pan', pansz(1:end), 'Datatype', 'double'); % width, height, channels, number \n\nh5write(filename_valid, '/gt', double(gt), [1,1,1,1], size(gt));\nh5write(filename_valid, '/ms', double(ms), [1,1,1,1], size(ms));\nh5write(filename_valid, '/lms', double(lms), [1,1,1,1], size(lms));\nh5write(filename_valid, '/pan', double(pan), [1,1,1,1], size(pan));\n\nclear gt ms lms pan pan_t\n\n\n%% ==========================================================\n%% (B) generate Testing data:\n%% ==========================================================\n\nfilename_test = '01-DataSimu/QB/TestData_qb.h5';\n\ngt = permute(gt_tmp_test,[3 2 4 1]); % beyond 2G, have to change dimension\npan_t(1,:,:,:) = pan_tmp_test; % CxNxHxW = 1x2x3x4 (PAN)\npan = permute(pan_t,[4 3 1 2]); % WxHxCxN\nms = permute(ms_tmp_test,[3 2 4 1]); \nlms = permute(lms_tmp_test,[3 2 4 1]); \n\ngtsz = size(gt);\nmssz = size(ms);\nlmssz = size(lms);\npansz = size(pan);\n\n\nh5create(filename_test, '/gt', gtsz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_test, '/ms', mssz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_test, '/lms', lmssz(1:end), 'Datatype', 'double'); % width, height, channels, number \nh5create(filename_test, '/pan', pansz(1:end), 'Datatype', 'double'); % width, height, channels, number \n\nh5write(filename_test, '/gt', double(gt), [1,1,1,1], size(gt));\nh5write(filename_test, '/ms', double(ms), [1,1,1,1], size(ms));\nh5write(filename_test, '/lms', double(lms), [1,1,1,1], size(lms));\nh5write(filename_test, '/pan', double(pan), [1,1,1,1], size(pan));\n\nclear gt ms lms pan pan_t\n\n\n" }, { "path": "03-Data-Simulation(Matlab)/imgs/readme", "content": "You may download an original QB data to understand the usage of this toolbox from: https://www.dropbox.com/s/2ujmag14bkiw0mq/QB_Indianapolis_training_DL.mat?dl=0\n" }, { "path": "03-Data-Simulation(Matlab)/segImg_new.m", "content": "function [gt, pan, ms, lms] = segImg_new(PAN, LMS, GT, MS, size_l, size_high, scale, overlap)\n% This is a core function to segment big images into small patches\n% LJ Deng (UESTC); 2020-10-09\n\n% PAN: big PAN image\n% LMS: big upsampled MS image\n% GT: big original HRMS image\n% MS: big original LR MS image\n% size_l: the patch size LR patch\n% size_high: the patch size HR patch\n% scale: spatial ration of PAN and MS, here, scale = 4\n% overlap: the overlap among segmented patches\n% gt: segmented ground-truth (gt) or labeled data \n% pan: segmented pan data\n% ms: segmented ms data\n% lms: segmented lms data\n\n%% --------------------------\n[h, w, c] = size(MS); % size of LR: \nH = scale*h; W = scale*w;\n\nsize_low = size_l; % patch size of LR: 16x16\nsize_h = size_high; % patch size of LR: 64x64\noverlap_low = overlap; % overlap of LR\noverlap_h = scale*overlap; % overlap of HR\n\n% set patch indexs \n%---- LR indexs ---------\ngridy = 1:size_low - overlap_low : w;%-(mod(w,size_low-overlap_low)+1+size_low-overlap_low);\ngridy((gridy+size_low-1) > w) = []; % delet boudary points\ngridx = 1:size_low - overlap_low: h;%-(mod(h,size_low-overlap_low)+1+size_low-overlap_low);\ngridx((gridx+size_low-1) > h) = []; % delet boudary points\n\n%---- HR indexs ---------\nGridy = 1:size_h - overlap_h : W;%-(mod(W,size_h-overlap_h)+1+size_h-overlap_h); % is 2 or 8? ===>must be some problem here!\nGridy((Gridy+size_h-1) > W) = []; % delet boudary points\nGridx = 1:size_h - overlap_h : H;%-(mod(H,size_h-overlap_h)+1+size_h-overlap_h);\nGridx((Gridx+size_h-1) > H) = []; % delet boudary points\n\n%% -----Pre-define variables' sizes--------\npan = zeros(size(gridx,2)*size(gridy,2), size_h, size_h);\nlms = zeros(size(gridx,2)*size(gridy,2), size_h, size_h, c);\ngt = zeros(size(gridx,2)*size(gridy,2), size_h, size_h, c);\nms = zeros(size(gridx,2)*size(gridy,2), size_low, size_low, c);\n\n%% -----loops to segment--------\ncnt = 0;\nNum = 0;\nfor i = 1: length(gridx)\n for j = 1:length(gridy)\n cnt = cnt + 1;\n Num = Num + 1;\n xx = gridx(i);\n yy = gridy(j);\n XX = Gridx(i);\n YY = Gridy(j); \n \n % ---start to segment------\n pan_p = PAN(XX:XX+size_h-1, YY:YY+size_h-1);% 64x64: signle pan patch\n pan(Num, :, :) = pan_p; % save single to a \"pan\" tensor: Nx64x64\n\n lms_p = LMS(XX:XX+size_h-1, YY:YY+size_h-1, :); % 64x64x8: signle lms patch\n lms(Num, :, :, :) = lms_p; % save single to a \"lms\" tensor: Nx64x64x8\n\n gt_p = GT(XX:XX+size_h-1, YY:YY+size_h-1, :); % 64x64x8: signle gt patch\n gt(Num, :, :, :) = gt_p; % save single to a \"gt\" tensor: Nx64x64x8\n \n ms_p = MS(xx:xx+size_low-1, yy:yy+size_low-1, :); % 16x16x8: signle ms patch \n ms(Num, :, :, :) = ms_p; % save single to a \"ms\" tensor: Nx16x16x8\n \n if Num == 1 % to see if there needs registration!\n maxval = max(PAN(:));\n ww(:,:,1)=gt_p(:,:,3); % gt\n ww(:,:,2)=gt_p(:,:,2);\n ww(:,:,3)=gt_p(:,:,1);\n kk(:,:,1)=lms_p(:,:,3); % gt\n kk(:,:,2)=lms_p(:,:,2);\n kk(:,:,3)=lms_p(:,:,1);\n pp = pan_p; % pan \n figure,\n subplot(1,3,1), imshow(double(ww)/maxval + 0.3); title('gt')\n subplot(1,3,2), imshow(double(pp)/maxval + 0.3); title('pan')\n subplot(1,3,3), imshow(double(kk)/maxval + 0.3); title('lms')\n end\n \n end\nend\n%% -----End loops--------\n\nend\n\n" }, { "path": "LICENSE", "content": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free Software Foundation, Inc. \n Everyone is permitted to copy and distribute verbatim copies\n of this license document, but changing it is not allowed.\n\n Preamble\n\n The GNU General Public License is a free, copyleft license for\nsoftware and other kinds of works.\n\n The licenses for most software and other practical works are designed\nto take away your freedom to share and change the works. By contrast,\nthe GNU General Public License is intended to guarantee your freedom to\nshare and change all versions of a program--to make sure it remains free\nsoftware for all its users. We, the Free Software Foundation, use the\nGNU General Public License for most of our software; it applies also to\nany other work released this way by its authors. You can apply it to\nyour programs, too.\n\n When we speak of free software, we are referring to freedom, not\nprice. Our General Public Licenses are designed to make sure that you\nhave the freedom to distribute copies of free software (and charge for\nthem if you wish), that you receive source code or can get it if you\nwant it, that you can change the software or use pieces of it in new\nfree programs, and that you know you can do these things.\n\n To protect your rights, we need to prevent others from denying you\nthese rights or asking you to surrender the rights. Therefore, you have\ncertain responsibilities if you distribute copies of the software, or if\nyou modify it: responsibilities to respect the freedom of others.\n\n For example, if you distribute copies of such a program, whether\ngratis or for a fee, you must pass on to the recipients the same\nfreedoms that you received. You must make sure that they, too, receive\nor can get the source code. And you must show them these terms so they\nknow their rights.\n\n Developers that use the GNU GPL protect your rights with two steps:\n(1) assert copyright on the software, and (2) offer you this License\ngiving you legal permission to copy, distribute and/or modify it.\n\n For the developers' and authors' protection, the GPL clearly explains\nthat there is no warranty for this free software. For both users' and\nauthors' sake, the GPL requires that modified versions be marked as\nchanged, so that their problems will not be attributed erroneously to\nauthors of previous versions.\n\n Some devices are designed to deny users access to install or run\nmodified versions of the software inside them, although the manufacturer\ncan do so. This is fundamentally incompatible with the aim of\nprotecting users' freedom to change the software. The systematic\npattern of such abuse occurs in the area of products for individuals to\nuse, which is precisely where it is most unacceptable. Therefore, we\nhave designed this version of the GPL to prohibit the practice for those\nproducts. If such problems arise substantially in other domains, we\nstand ready to extend this provision to those domains in future versions\nof the GPL, as needed to protect the freedom of users.\n\n Finally, every program is threatened constantly by software patents.\nStates should not allow patents to restrict development and use of\nsoftware on general-purpose computers, but in those that do, we wish to\navoid the special danger that patents applied to a free program could\nmake it effectively proprietary. To prevent this, the GPL assures that\npatents cannot be used to render the program non-free.\n\n The precise terms and conditions for copying, distribution and\nmodification follow.\n\n TERMS AND CONDITIONS\n\n 0. Definitions.\n\n \"This License\" refers to version 3 of the GNU General Public License.\n\n \"Copyright\" also means copyright-like laws that apply to other kinds of\nworks, such as semiconductor masks.\n\n \"The Program\" refers to any copyrightable work licensed under this\nLicense. Each licensee is addressed as \"you\". \"Licensees\" and\n\"recipients\" may be individuals or organizations.\n\n To \"modify\" a work means to copy from or adapt all or part of the work\nin a fashion requiring copyright permission, other than the making of an\nexact copy. The resulting work is called a \"modified version\" of the\nearlier work or a work \"based on\" the earlier work.\n\n A \"covered work\" means either the unmodified Program or a work based\non the Program.\n\n To \"propagate\" a work means to do anything with it that, without\npermission, would make you directly or secondarily liable for\ninfringement under applicable copyright law, except executing it on a\ncomputer or modifying a private copy. Propagation includes copying,\ndistribution (with or without modification), making available to the\npublic, and in some countries other activities as well.\n\n To \"convey\" a work means any kind of propagation that enables other\nparties to make or receive copies. Mere interaction with a user through\na computer network, with no transfer of a copy, is not conveying.\n\n An interactive user interface displays \"Appropriate Legal Notices\"\nto the extent that it includes a convenient and prominently visible\nfeature that (1) displays an appropriate copyright notice, and (2)\ntells the user that there is no warranty for the work (except to the\nextent that warranties are provided), that licensees may convey the\nwork under this License, and how to view a copy of this License. If\nthe interface presents a list of user commands or options, such as a\nmenu, a prominent item in the list meets this criterion.\n\n 1. Source Code.\n\n The \"source code\" for a work means the preferred form of the work\nfor making modifications to it. \"Object code\" means any non-source\nform of a work.\n\n A \"Standard Interface\" means an interface that either is an official\nstandard defined by a recognized standards body, or, in the case of\ninterfaces specified for a particular programming language, one that\nis widely used among developers working in that language.\n\n The \"System Libraries\" of an executable work include anything, other\nthan the work as a whole, that (a) is included in the normal form of\npackaging a Major Component, but which is not part of that Major\nComponent, and (b) serves only to enable use of the work with that\nMajor Component, or to implement a Standard Interface for which an\nimplementation is available to the public in source code form. A\n\"Major Component\", in this context, means a major essential component\n(kernel, window system, and so on) of the specific operating system\n(if any) on which the executable work runs, or a compiler used to\nproduce the work, or an object code interpreter used to run it.\n\n The \"Corresponding Source\" for a work in object code form means all\nthe source code needed to generate, install, and (for an executable\nwork) run the object code and to modify the work, including scripts to\ncontrol those activities. However, it does not include the work's\nSystem Libraries, or general-purpose tools or generally available free\nprograms which are used unmodified in performing those activities but\nwhich are not part of the work. For example, Corresponding Source\nincludes interface definition files associated with source files for\nthe work, and the source code for shared libraries and dynamically\nlinked subprograms that the work is specifically designed to require,\nsuch as by intimate data communication or control flow between those\nsubprograms and other parts of the work.\n\n The Corresponding Source need not include anything that users\ncan regenerate automatically from other parts of the Corresponding\nSource.\n\n The Corresponding Source for a work in source code form is that\nsame work.\n\n 2. Basic Permissions.\n\n All rights granted under this License are granted for the term of\ncopyright on the Program, and are irrevocable provided the stated\nconditions are met. This License explicitly affirms your unlimited\npermission to run the unmodified Program. The output from running a\ncovered work is covered by this License only if the output, given its\ncontent, constitutes a covered work. This License acknowledges your\nrights of fair use or other equivalent, as provided by copyright law.\n\n You may make, run and propagate covered works that you do not\nconvey, without conditions so long as your license otherwise remains\nin force. You may convey covered works to others for the sole purpose\nof having them make modifications exclusively for you, or provide you\nwith facilities for running those works, provided that you comply with\nthe terms of this License in conveying all material for which you do\nnot control copyright. Those thus making or running the covered works\nfor you must do so exclusively on your behalf, under your direction\nand control, on terms that prohibit them from making any copies of\nyour copyrighted material outside their relationship with you.\n\n Conveying under any other circumstances is permitted solely under\nthe conditions stated below. Sublicensing is not allowed; section 10\nmakes it unnecessary.\n\n 3. Protecting Users' Legal Rights From Anti-Circumvention Law.\n\n No covered work shall be deemed part of an effective technological\nmeasure under any applicable law fulfilling obligations under article\n11 of the WIPO copyright treaty adopted on 20 December 1996, or\nsimilar laws prohibiting or restricting circumvention of such\nmeasures.\n\n When you convey a covered work, you waive any legal power to forbid\ncircumvention of technological measures to the extent such circumvention\nis effected by exercising rights under this License with respect to\nthe covered work, and you disclaim any intention to limit operation or\nmodification of the work as a means of enforcing, against the work's\nusers, your or third parties' legal rights to forbid circumvention of\ntechnological measures.\n\n 4. Conveying Verbatim Copies.\n\n You may convey verbatim copies of the Program's source code as you\nreceive it, in any medium, provided that you conspicuously and\nappropriately publish on each copy an appropriate copyright notice;\nkeep intact all notices stating that this License and any\nnon-permissive terms added in accord with section 7 apply to the code;\nkeep intact all notices of the absence of any warranty; and give all\nrecipients a copy of this License along with the Program.\n\n You may charge any price or no price for each copy that you convey,\nand you may offer support or warranty protection for a fee.\n\n 5. Conveying Modified Source Versions.\n\n You may convey a work based on the Program, or the modifications to\nproduce it from the Program, in the form of source code under the\nterms of section 4, provided that you also meet all of these conditions:\n\n a) The work must carry prominent notices stating that you modified\n it, and giving a relevant date.\n\n b) The work must carry prominent notices stating that it is\n released under this License and any conditions added under section\n 7. This requirement modifies the requirement in section 4 to\n \"keep intact all notices\".\n\n c) You must license the entire work, as a whole, under this\n License to anyone who comes into possession of a copy. This\n License will therefore apply, along with any applicable section 7\n additional terms, to the whole of the work, and all its parts,\n regardless of how they are packaged. This License gives no\n permission to license the work in any other way, but it does not\n invalidate such permission if you have separately received it.\n\n d) If the work has interactive user interfaces, each must display\n Appropriate Legal Notices; however, if the Program has interactive\n interfaces that do not display Appropriate Legal Notices, your\n work need not make them do so.\n\n A compilation of a covered work with other separate and independent\nworks, which are not by their nature extensions of the covered work,\nand which are not combined with it such as to form a larger program,\nin or on a volume of a storage or distribution medium, is called an\n\"aggregate\" if the compilation and its resulting copyright are not\nused to limit the access or legal rights of the compilation's users\nbeyond what the individual works permit. Inclusion of a covered work\nin an aggregate does not cause this License to apply to the other\nparts of the aggregate.\n\n 6. Conveying Non-Source Forms.\n\n You may convey a covered work in object code form under the terms\nof sections 4 and 5, provided that you also convey the\nmachine-readable Corresponding Source under the terms of this License,\nin one of these ways:\n\n a) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by the\n Corresponding Source fixed on a durable physical medium\n customarily used for software interchange.\n\n b) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by a\n written offer, valid for at least three years and valid for as\n long as you offer spare parts or customer support for that product\n model, to give anyone who possesses the object code either (1) a\n copy of the Corresponding Source for all the software in the\n product that is covered by this License, on a durable physical\n medium customarily used for software interchange, for a price no\n more than your reasonable cost of physically performing this\n conveying of source, or (2) access to copy the\n Corresponding Source from a network server at no charge.\n\n c) Convey individual copies of the object code with a copy of the\n written offer to provide the Corresponding Source. This\n alternative is allowed only occasionally and noncommercially, and\n only if you received the object code with such an offer, in accord\n with subsection 6b.\n\n d) Convey the object code by offering access from a designated\n place (gratis or for a charge), and offer equivalent access to the\n Corresponding Source in the same way through the same place at no\n further charge. You need not require recipients to copy the\n Corresponding Source along with the object code. If the place to\n copy the object code is a network server, the Corresponding Source\n may be on a different server (operated by you or a third party)\n that supports equivalent copying facilities, provided you maintain\n clear directions next to the object code saying where to find the\n Corresponding Source. Regardless of what server hosts the\n Corresponding Source, you remain obligated to ensure that it is\n available for as long as needed to satisfy these requirements.\n\n e) Convey the object code using peer-to-peer transmission, provided\n you inform other peers where the object code and Corresponding\n Source of the work are being offered to the general public at no\n charge under subsection 6d.\n\n A separable portion of the object code, whose source code is excluded\nfrom the Corresponding Source as a System Library, need not be\nincluded in conveying the object code work.\n\n A \"User Product\" is either (1) a \"consumer product\", which means any\ntangible personal property which is normally used for personal, family,\nor household purposes, or (2) anything designed or sold for incorporation\ninto a dwelling. In determining whether a product is a consumer product,\ndoubtful cases shall be resolved in favor of coverage. For a particular\nproduct received by a particular user, \"normally used\" refers to a\ntypical or common use of that class of product, regardless of the status\nof the particular user or of the way in which the particular user\nactually uses, or expects or is expected to use, the product. A product\nis a consumer product regardless of whether the product has substantial\ncommercial, industrial or non-consumer uses, unless such uses represent\nthe only significant mode of use of the product.\n\n \"Installation Information\" for a User Product means any methods,\nprocedures, authorization keys, or other information required to install\nand execute modified versions of a covered work in that User Product from\na modified version of its Corresponding Source. The information must\nsuffice to ensure that the continued functioning of the modified object\ncode is in no case prevented or interfered with solely because\nmodification has been made.\n\n If you convey an object code work under this section in, or with, or\nspecifically for use in, a User Product, and the conveying occurs as\npart of a transaction in which the right of possession and use of the\nUser Product is transferred to the recipient in perpetuity or for a\nfixed term (regardless of how the transaction is characterized), the\nCorresponding Source conveyed under this section must be accompanied\nby the Installation Information. But this requirement does not apply\nif neither you nor any third party retains the ability to install\nmodified object code on the User Product (for example, the work has\nbeen installed in ROM).\n\n The requirement to provide Installation Information does not include a\nrequirement to continue to provide support service, warranty, or updates\nfor a work that has been modified or installed by the recipient, or for\nthe User Product in which it has been modified or installed. Access to a\nnetwork may be denied when the modification itself materially and\nadversely affects the operation of the network or violates the rules and\nprotocols for communication across the network.\n\n Corresponding Source conveyed, and Installation Information provided,\nin accord with this section must be in a format that is publicly\ndocumented (and with an implementation available to the public in\nsource code form), and must require no special password or key for\nunpacking, reading or copying.\n\n 7. Additional Terms.\n\n \"Additional permissions\" are terms that supplement the terms of this\nLicense by making exceptions from one or more of its conditions.\nAdditional permissions that are applicable to the entire Program shall\nbe treated as though they were included in this License, to the extent\nthat they are valid under applicable law. If additional permissions\napply only to part of the Program, that part may be used separately\nunder those permissions, but the entire Program remains governed by\nthis License without regard to the additional permissions.\n\n When you convey a copy of a covered work, you may at your option\nremove any additional permissions from that copy, or from any part of\nit. (Additional permissions may be written to require their own\nremoval in certain cases when you modify the work.) You may place\nadditional permissions on material, added by you to a covered work,\nfor which you have or can give appropriate copyright permission.\n\n Notwithstanding any other provision of this License, for material you\nadd to a covered work, you may (if authorized by the copyright holders of\nthat material) supplement the terms of this License with terms:\n\n a) Disclaiming warranty or limiting liability differently from the\n terms of sections 15 and 16 of this License; or\n\n b) Requiring preservation of specified reasonable legal notices or\n author attributions in that material or in the Appropriate Legal\n Notices displayed by works containing it; or\n\n c) Prohibiting misrepresentation of the origin of that material, or\n requiring that modified versions of such material be marked in\n reasonable ways as different from the original version; or\n\n d) Limiting the use for publicity purposes of names of licensors or\n authors of the material; or\n\n e) Declining to grant rights under trademark law for use of some\n trade names, trademarks, or service marks; or\n\n f) Requiring indemnification of licensors and authors of that\n material by anyone who conveys the material (or modified versions of\n it) with contractual assumptions of liability to the recipient, for\n any liability that these contractual assumptions directly impose on\n those licensors and authors.\n\n All other non-permissive additional terms are considered \"further\nrestrictions\" within the meaning of section 10. If the Program as you\nreceived it, or any part of it, contains a notice stating that it is\ngoverned by this License along with a term that is a further\nrestriction, you may remove that term. If a license document contains\na further restriction but permits relicensing or conveying under this\nLicense, you may add to a covered work material governed by the terms\nof that license document, provided that the further restriction does\nnot survive such relicensing or conveying.\n\n If you add terms to a covered work in accord with this section, you\nmust place, in the relevant source files, a statement of the\nadditional terms that apply to those files, or a notice indicating\nwhere to find the applicable terms.\n\n Additional terms, permissive or non-permissive, may be stated in the\nform of a separately written license, or stated as exceptions;\nthe above requirements apply either way.\n\n 8. Termination.\n\n You may not propagate or modify a covered work except as expressly\nprovided under this License. Any attempt otherwise to propagate or\nmodify it is void, and will automatically terminate your rights under\nthis License (including any patent licenses granted under the third\nparagraph of section 11).\n\n However, if you cease all violation of this License, then your\nlicense from a particular copyright holder is reinstated (a)\nprovisionally, unless and until the copyright holder explicitly and\nfinally terminates your license, and (b) permanently, if the copyright\nholder fails to notify you of the violation by some reasonable means\nprior to 60 days after the cessation.\n\n Moreover, your license from a particular copyright holder is\nreinstated permanently if the copyright holder notifies you of the\nviolation by some reasonable means, this is the first time you have\nreceived notice of violation of this License (for any work) from that\ncopyright holder, and you cure the violation prior to 30 days after\nyour receipt of the notice.\n\n Termination of your rights under this section does not terminate the\nlicenses of parties who have received copies or rights from you under\nthis License. If your rights have been terminated and not permanently\nreinstated, you do not qualify to receive new licenses for the same\nmaterial under section 10.\n\n 9. Acceptance Not Required for Having Copies.\n\n You are not required to accept this License in order to receive or\nrun a copy of the Program. Ancillary propagation of a covered work\noccurring solely as a consequence of using peer-to-peer transmission\nto receive a copy likewise does not require acceptance. However,\nnothing other than this License grants you permission to propagate or\nmodify any covered work. These actions infringe copyright if you do\nnot accept this License. Therefore, by modifying or propagating a\ncovered work, you indicate your acceptance of this License to do so.\n\n 10. Automatic Licensing of Downstream Recipients.\n\n Each time you convey a covered work, the recipient automatically\nreceives a license from the original licensors, to run, modify and\npropagate that work, subject to this License. You are not responsible\nfor enforcing compliance by third parties with this License.\n\n An \"entity transaction\" is a transaction transferring control of an\norganization, or substantially all assets of one, or subdividing an\norganization, or merging organizations. If propagation of a covered\nwork results from an entity transaction, each party to that\ntransaction who receives a copy of the work also receives whatever\nlicenses to the work the party's predecessor in interest had or could\ngive under the previous paragraph, plus a right to possession of the\nCorresponding Source of the work from the predecessor in interest, if\nthe predecessor has it or can get it with reasonable efforts.\n\n You may not impose any further restrictions on the exercise of the\nrights granted or affirmed under this License. For example, you may\nnot impose a license fee, royalty, or other charge for exercise of\nrights granted under this License, and you may not initiate litigation\n(including a cross-claim or counterclaim in a lawsuit) alleging that\nany patent claim is infringed by making, using, selling, offering for\nsale, or importing the Program or any portion of it.\n\n 11. Patents.\n\n A \"contributor\" is a copyright holder who authorizes use under this\nLicense of the Program or a work on which the Program is based. The\nwork thus licensed is called the contributor's \"contributor version\".\n\n A contributor's \"essential patent claims\" are all patent claims\nowned or controlled by the contributor, whether already acquired or\nhereafter acquired, that would be infringed by some manner, permitted\nby this License, of making, using, or selling its contributor version,\nbut do not include claims that would be infringed only as a\nconsequence of further modification of the contributor version. For\npurposes of this definition, \"control\" includes the right to grant\npatent sublicenses in a manner consistent with the requirements of\nthis License.\n\n Each contributor grants you a non-exclusive, worldwide, royalty-free\npatent license under the contributor's essential patent claims, to\nmake, use, sell, offer for sale, import and otherwise run, modify and\npropagate the contents of its contributor version.\n\n In the following three paragraphs, a \"patent license\" is any express\nagreement or commitment, however denominated, not to enforce a patent\n(such as an express permission to practice a patent or covenant not to\nsue for patent infringement). To \"grant\" such a patent license to a\nparty means to make such an agreement or commitment not to enforce a\npatent against the party.\n\n If you convey a covered work, knowingly relying on a patent license,\nand the Corresponding Source of the work is not available for anyone\nto copy, free of charge and under the terms of this License, through a\npublicly available network server or other readily accessible means,\nthen you must either (1) cause the Corresponding Source to be so\navailable, or (2) arrange to deprive yourself of the benefit of the\npatent license for this particular work, or (3) arrange, in a manner\nconsistent with the requirements of this License, to extend the patent\nlicense to downstream recipients. \"Knowingly relying\" means you have\nactual knowledge that, but for the patent license, your conveying the\ncovered work in a country, or your recipient's use of the covered work\nin a country, would infringe one or more identifiable patents in that\ncountry that you have reason to believe are valid.\n\n If, pursuant to or in connection with a single transaction or\narrangement, you convey, or propagate by procuring conveyance of, a\ncovered work, and grant a patent license to some of the parties\nreceiving the covered work authorizing them to use, propagate, modify\nor convey a specific copy of the covered work, then the patent license\nyou grant is automatically extended to all recipients of the covered\nwork and works based on it.\n\n A patent license is \"discriminatory\" if it does not include within\nthe scope of its coverage, prohibits the exercise of, or is\nconditioned on the non-exercise of one or more of the rights that are\nspecifically granted under this License. You may not convey a covered\nwork if you are a party to an arrangement with a third party that is\nin the business of distributing software, under which you make payment\nto the third party based on the extent of your activity of conveying\nthe work, and under which the third party grants, to any of the\nparties who would receive the covered work from you, a discriminatory\npatent license (a) in connection with copies of the covered work\nconveyed by you (or copies made from those copies), or (b) primarily\nfor and in connection with specific products or compilations that\ncontain the covered work, unless you entered into that arrangement,\nor that patent license was granted, prior to 28 March 2007.\n\n Nothing in this License shall be construed as excluding or limiting\nany implied license or other defenses to infringement that may\notherwise be available to you under applicable patent law.\n\n 12. No Surrender of Others' Freedom.\n\n If conditions are imposed on you (whether by court order, agreement or\notherwise) that contradict the conditions of this License, they do not\nexcuse you from the conditions of this License. If you cannot convey a\ncovered work so as to satisfy simultaneously your obligations under this\nLicense and any other pertinent obligations, then as a consequence you may\nnot convey it at all. For example, if you agree to terms that obligate you\nto collect a royalty for further conveying from those to whom you convey\nthe Program, the only way you could satisfy both those terms and this\nLicense would be to refrain entirely from conveying the Program.\n\n 13. Use with the GNU Affero General Public License.\n\n Notwithstanding any other provision of this License, you have\npermission to link or combine any covered work with a work licensed\nunder version 3 of the GNU Affero General Public License into a single\ncombined work, and to convey the resulting work. The terms of this\nLicense will continue to apply to the part which is the covered work,\nbut the special requirements of the GNU Affero General Public License,\nsection 13, concerning interaction through a network will apply to the\ncombination as such.\n\n 14. Revised Versions of this License.\n\n The Free Software Foundation may publish revised and/or new versions of\nthe GNU General Public License from time to time. Such new versions will\nbe similar in spirit to the present version, but may differ in detail to\naddress new problems or concerns.\n\n Each version is given a distinguishing version number. If the\nProgram specifies that a certain numbered version of the GNU General\nPublic License \"or any later version\" applies to it, you have the\noption of following the terms and conditions either of that numbered\nversion or of any later version published by the Free Software\nFoundation. If the Program does not specify a version number of the\nGNU General Public License, you may choose any version ever published\nby the Free Software Foundation.\n\n If the Program specifies that a proxy can decide which future\nversions of the GNU General Public License can be used, that proxy's\npublic statement of acceptance of a version permanently authorizes you\nto choose that version for the Program.\n\n Later license versions may give you additional or different\npermissions. However, no additional obligations are imposed on any\nauthor or copyright holder as a result of your choosing to follow a\nlater version.\n\n 15. Disclaimer of Warranty.\n\n THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY\nAPPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT\nHOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM \"AS IS\" WITHOUT WARRANTY\nOF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,\nTHE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\nPURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM\nIS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF\nALL NECESSARY SERVICING, REPAIR OR CORRECTION.\n\n 16. Limitation of Liability.\n\n IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING\nWILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS\nTHE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY\nGENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE\nUSE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF\nDATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD\nPARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),\nEVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF\nSUCH DAMAGES.\n\n 17. Interpretation of Sections 15 and 16.\n\n If the disclaimer of warranty and limitation of liability provided\nabove cannot be given local legal effect according to their terms,\nreviewing courts shall apply local law that most closely approximates\nan absolute waiver of all civil liability in connection with the\nProgram, unless a warranty or assumption of liability accompanies a\ncopy of the Program in return for a fee.\n\n END OF TERMS AND CONDITIONS\n\n How to Apply These Terms to Your New Programs\n\n If you develop a new program, and you want it to be of the greatest\npossible use to the public, the best way to achieve this is to make it\nfree software which everyone can redistribute and change under these terms.\n\n To do so, attach the following notices to the program. It is safest\nto attach them to the start of each source file to most effectively\nstate the exclusion of warranty; and each file should have at least\nthe \"copyright\" line and a pointer to where the full notice is found.\n\n \n Copyright (C) \n\n This program is free software: you can redistribute it and/or modify\n it under the terms of the GNU General Public License as published by\n the Free Software Foundation, either version 3 of the License, or\n (at your option) any later version.\n\n This program is distributed in the hope that it will be useful,\n but WITHOUT ANY WARRANTY; without even the implied warranty of\n MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n GNU General Public License for more details.\n\n You should have received a copy of the GNU General Public License\n along with this program. If not, see .\n\nAlso add information on how to contact you by electronic and paper mail.\n\n If the program does terminal interaction, make it output a short\nnotice like this when it starts in an interactive mode:\n\n Copyright (C) \n This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.\n This is free software, and you are welcome to redistribute it\n under certain conditions; type `show c' for details.\n\nThe hypothetical commands `show w' and `show c' should show the appropriate\nparts of the General Public License. Of course, your program's commands\nmight be different; for a GUI interface, you would use an \"about box\".\n\n You should also get your employer (if you work as a programmer) or school,\nif any, to sign a \"copyright disclaimer\" for the program, if necessary.\nFor more information on this, and how to apply and follow the GNU GPL, see\n.\n\n The GNU General Public License does not permit incorporating your program\ninto proprietary programs. If your program is a subroutine library, you\nmay consider it more useful to permit linking proprietary applications with\nthe library. If this is what you want to do, use the GNU Lesser General\nPublic License instead of this License. But first, please read\n.\n" }, { "path": "README.md", "content": "# DLPan-Toolbox\n\n* This toolbox is related to the paper ``Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks, IEEE Geoscience and Remote Sensing Magazine, 2022`` (see the following reference [1]). Download: [[paper]](https://github.com/liangjiandeng/liangjiandeng.github.io/tree/master/papers/2022/review-grsm2022.pdf).\n\n* This is a deep learning (DL) toolbox for pansharpening, which can be used for training and testing getting the comparison between traditional and DL methods.\n \n\n## Introduction\nThis toolbox mainly contains two parts: one is the pytorch source codes for the eight DL-based methods presented in the paper (i.e., the folder \"01-DL toolbox (Pytorch)\"); the other is the Matlab source codes which can simultaneously evaluate the performance of traditional and DL approaches in a uniformed framework (\"02-Test toolbox for traditional and DL (Matlab)\"). Please see more details:\n\n- 01-DL-toolbox(Pytorch) contains source codes of DL methods, you may check the ``readme`` file for the usage.\n- 02-Test-toolbox-for-traditional-and-DL(Matlab) contains Matlab source codes (mainly from 'G. Vivone et al., A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods, IEEE GRSM, 2021', see the following reference [2]) for simultaneously evaluating traditional and DL approaches and outputing results, you may check the ``readme`` file for the usage. \n- 03-Data-Simulation(Matlab) contains Matlab source codes that are patching images to patches for training and validation. Also, you can simulate test examples by this toolbox.\n\nNote that, readers also could check the structure and relationship of these two folders in the following ``overview figure`` (also find it in the respository).\n\n\n\n\n\n## Dataset\nDue to the copyright issue, the datasets used in this GRSM paper are not available. Therefore, we recommend readers use the following dataset for pansharpening, both training and testing. The following dataset can be directly applied in our DLPan-Toolbox (put the data to the director for training: 01-DL-toolbox(Pytorch)/UDL/Data/pansharpening/training_data/).\n\n- [[PanCollection](https://github.com/liangjiandeng/PanCollection)] for multispectral pansharpening\n- [[HyperPanCollection](https://github.com/liangjiandeng/HyperPanCollection)] for hyperspectral pansharpening\n\n\n## Citation\n* [1] If you use this toolbox, please kindly cite our paper:\n\n```bibtex\n@ARTICLE{deng2022grsm,\nauthor={L.-J. Deng, G. Vivone, M. E. Paoletti, G. Scarpa, J. He, Y. Zhang, J. Chanussot, and A. Plaza},\nbooktitle={IEEE Geoscience and Remote Sensing Magazine},\ntitle={Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks},\nyear={2022},\npages={2-38},\ndoi={10.1109/MGRS.2020.3019315}\n}\n```\n\n\n* [2] Also, the codes of traditional methods are from the \"pansharpening toolbox for distribution\", thus please cite the corresponding paper:\n```bibtex\n@ARTICLE{vivone2021grsm,\n author={Vivone, Gemine and Dalla Mura, Mauro and Garzelli, Andrea and Restaino, Rocco and Scarpa, Giuseppe and Ulfarsson, Magnus O. and Alparone, Luciano and Chanussot, Jocelyn},\n journal={IEEE Geoscience and Remote Sensing Magazine}, \n title={A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods}, \n year={2021},\n volume={9},\n number={1},\n pages={53-81},\n doi={10.1109/MGRS.2020.3019315}\n}\n```\n\n## Acknowledgement\n\n- We appreciate the great contribution to this toolbox of [Xiao Wu](https://xiaoxiao-woo.github.io/) and Ran Ran, who are graduate students in [UESTC](https://www.uestc.edu.cn/).\n\n\n## License & Copyright\nThis project is open sourced under GNU General Public License v3.0.\n" }, { "path": "docs/en/DLPanToolbox/Evaluation.md", "content": "## Evaluation" }, { "path": "docs/en/DLPanToolbox/Example.md", "content": "" }, { "path": "docs/en/DLPanToolbox/PreProcess.md", "content": "## Data PreProcess\n\n### Image\n\nThis module provides some image processing methods, which requires `opencv` to be installed first.\n\n#### Read/Write/Show\n\nTo read or write images files, use `imread` or `imwrite`.\n\n```python\nimport mmcv\n\nimg = mmcv.imread('test.jpg')\nimg = mmcv.imread('test.jpg', flag='grayscale')\nimg_ = mmcv.imread(img) # nothing will happen, img_ = img\nmmcv.imwrite(img, 'out.jpg')\n```\n\nTo read images from bytes\n\n```python\nwith open('test.jpg', 'rb') as f:\n data = f.read()\nimg = mmcv.imfrombytes(data)\n```\n\nTo show an image file or a loaded image\n\n```python\nmmcv.imshow('tests/data/color.jpg')\n# this is equivalent to\n\nfor i in range(10):\n img = np.random.randint(256, size=(100, 100, 3), dtype=np.uint8)\n mmcv.imshow(img, win_name='test image', wait_time=200)\n```\n\n#### Color space conversion\n\nSupported conversion methods:\n\n- bgr2gray\n- gray2bgr\n- bgr2rgb\n- rgb2bgr\n- bgr2hsv\n- hsv2bgr\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\nimg1 = mmcv.bgr2rgb(img)\nimg2 = mmcv.rgb2gray(img1)\nimg3 = mmcv.bgr2hsv(img)\n```\n\n#### Resize\n\nThere are three resize methods. All `imresize_*` methods have an argument `return_scale`,\nif this argument is `False`, then the return value is merely the resized image, otherwise\nis a tuple `(resized_img, scale)`.\n\n```python\n# resize to a given size\nmmcv.imresize(img, (1000, 600), return_scale=True)\n\n# resize to the same size of another image\nmmcv.imresize_like(img, dst_img, return_scale=False)\n\n# resize by a ratio\nmmcv.imrescale(img, 0.5)\n\n# resize so that the max edge no longer than 1000, short edge no longer than 800\n# without changing the aspect ratio\nmmcv.imrescale(img, (1000, 800))\n```\n\n#### Rotate\n\nTo rotate an image by some angle, use `imrotate`. The center can be specified,\nwhich is the center of original image by default. There are two modes of rotating,\none is to keep the image size unchanged so that some parts of the image will be\ncropped after rotating, the other is to extend the image size to fit the rotated\nimage.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# rotate the image clockwise by 30 degrees.\nimg_ = mmcv.imrotate(img, 30)\n\n# rotate the image counterclockwise by 90 degrees.\nimg_ = mmcv.imrotate(img, -90)\n\n# rotate the image clockwise by 30 degrees, and rescale it by 1.5x at the same time.\nimg_ = mmcv.imrotate(img, 30, scale=1.5)\n\n# rotate the image clockwise by 30 degrees, with (100, 100) as the center.\nimg_ = mmcv.imrotate(img, 30, center=(100, 100))\n\n# rotate the image clockwise by 30 degrees, and extend the image size.\nimg_ = mmcv.imrotate(img, 30, auto_bound=True)\n```\n\n#### Flip\n\nTo flip an image, use `imflip`.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# flip the image horizontally\nmmcv.imflip(img)\n\n# flip the image vertically\nmmcv.imflip(img, direction='vertical')\n```\n\n#### Crop\n\n`imcrop` can crop the image with one or more regions. Each region is represented by the upper left and lower right coordinates as (x1, y1, x2, y2).\n\n```python\nimport mmcv\nimport numpy as np\n\nimg = mmcv.imread('tests/data/color.jpg')\n\n# crop the region (10, 10, 100, 120)\nbboxes = np.array([10, 10, 100, 120])\npatch = mmcv.imcrop(img, bboxes)\n\n# crop two regions (10, 10, 100, 120) and (0, 0, 50, 50)\nbboxes = np.array([[10, 10, 100, 120], [0, 0, 50, 50]])\npatches = mmcv.imcrop(img, bboxes)\n\n# crop two regions, and rescale the patches by 1.2x\npatches = mmcv.imcrop(img, bboxes, scale=1.2)\n```\n\n#### Padding\n\nThere are two methods, `impad` and `impad_to_multiple`, to pad an image to the\nspecific size with given values.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# pad the image to (1000, 1200) with all zeros\nimg_ = mmcv.impad(img, shape=(1000, 1200), pad_val=0)\n\n# pad the image to (1000, 1200) with different values for three channels.\nimg_ = mmcv.impad(img, shape=(1000, 1200), pad_val=(100, 50, 200))\n\n# pad the image on left, right, top, bottom borders with all zeros\nimg_ = mmcv.impad(img, padding=(10, 20, 30, 40), pad_val=0)\n\n# pad the image on left, right, top, bottom borders with different values\n# for three channels.\nimg_ = mmcv.impad(img, padding=(10, 20, 30, 40), pad_val=(100, 50, 200))\n\n# pad an image so that each edge is a multiple of some value.\nimg_ = mmcv.impad_to_multiple(img, 32)\n```" }, { "path": "docs/en/DLPanToolbox/Simulation.md", "content": "## Simulation" }, { "path": "docs/en/Makefile", "content": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line, and also\n# from the environment for the first two.\nSPHINXOPTS ?=\nSPHINXBUILD ?= sphinx-build\nSOURCEDIR = source\nBUILDDIR = build\n\n# Put it first so that \"make\" without argument is like \"make help\".\nhelp:\n\t@$(SPHINXBUILD) -M help \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n\n.PHONY: help Makefile\n\n# Catch-all target: route all unknown targets to Sphinx using the new\n# \"make mode\" option. $(O) is meant as a shortcut for $(SPHINXOPTS).\n%: Makefile\n\t@$(SPHINXBUILD) -M $@ \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)" }, { "path": "docs/en/_static/css/readthedocs.css", "content": ".header-logo {\n background-image: url(\"../image/logo-dlpan.png\");\n background-size: 160px 40px;\n height: 40px;\n width: 160px;\n}\n\ntable.colwidths-auto td {\n width: 50%\n}" }, { "path": "docs/en/_templates/classtemplate.rst", "content": ".. role:: hidden\n :class: hidden-section\n.. currentmodule:: {{ module }}\n\n\n{{ name | underline}}\n\n.. autoclass:: {{ name }}\n :members:\n\n\n..\n autogenerated from source/_templates/classtemplate.rst\n note it does not have :inherited-members:" }, { "path": "docs/en/citation.md", "content": "## Cite DLPan-Toolbox\nIf DLPan-Toolbox is helpful for you, you are encouraged to cite the following paper:\n```bibtex\n@ARTICLE{deng2022grsm,\nauthor={L.-J. Deng, G. Vivone, M. E. Paoletti, G. Scarpa, J. He, Y. Zhang, J. Chanussot, and A. Plaza},\nbooktitle={IEEE Geoscience and Remote Sensing Magazine},\ntitle={Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks},\nyear={2022},\npages={2-38},\ndoi={10.1109/MGRS.2020.3019315}\n}\n```\n\nAlso, the codes of traditional methods are from the \"pansharpening toolbox for distribution\", thus please cite the corresponding paper:\n```bibtex\n@ARTICLE{vivone2021grsm,\n author={Vivone, Gemine and Dalla Mura, Mauro and Garzelli, Andrea and Restaino, Rocco and Scarpa, Giuseppe and Ulfarsson, Magnus O. and Alparone, Luciano and Chanussot, Jocelyn},\n journal={IEEE Geoscience and Remote Sensing Magazine}, \n title={A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods}, \n year={2021},\n volume={9},\n number={1},\n pages={53-81},\n doi={10.1109/MGRS.2020.3019315}\n}\n```" }, { "path": "docs/en/conf.py", "content": "# GPL v3.0 License\n# Copyright (C) UESTC\n# All Rights Reserved\n# @Time : 2023/9/21\n# @Author : Xiao Wu\n# @reference:\n#\n\n#\n# Configuration file for the Sphinx documentation builder.\n#\n# This file does only contain a selection of the most common options. For a\n# full list see the documentation:\n# http://www.sphinx-doc.org/en/master/config\n\n# -- Path setup --------------------------------------------------------------\n\n# If extensions (or modules to document with autodoc) are in another directory,\n# add these directories to sys.path here. If the directory is relative to the\n# documentation root, use os.path.abspath to make it absolute, like shown here.\n#\nimport os\nimport sys\n\nimport pytorch_sphinx_theme\nfrom sphinx.builders.html import StandaloneHTMLBuilder\n\nsys.path.insert(0, os.path.abspath('../..'))\n\n# version_file = '../../mmcv/version.py'\n# with open(version_file) as f:\n# exec(compile(f.read(), version_file, 'exec'))\n# __version__ = locals()['__version__']\n__version__ = \"0.3.6\"\n\n# -- Project information -----------------------------------------------------\n\nproject = 'DLPan-Toolbox'\ncopyright = '2023, UESTC'\nauthor = 'Xiao Wu'\n\n# The short X.Y version\nversion = __version__\n# The full version, including alpha/beta/rc tags\nrelease = __version__\n\n# -- General configuration ---------------------------------------------------\n\n# If your documentation needs a minimal Sphinx version, state it here.\n#\n# needs_sphinx = '1.0'\n\n# Add any Sphinx extension module names here, as strings. They can be\n# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom\n# ones.\n\nextensions = [\n 'sphinx.ext.autodoc',\n 'sphinx.ext.autosummary',\n 'sphinx.ext.intersphinx',\n 'sphinx.ext.napoleon',\n 'sphinx.ext.viewcode',\n 'sphinx_markdown_tables',\n 'myst_parser',\n 'sphinx_copybutton',\n] # yapf: disable\n\nmyst_heading_anchors = 4\n\nmyst_enable_extensions = ['colon_fence']\n\n# Configuration for intersphinx\nintersphinx_mapping = {\n 'python': ('https://docs.python.org/3', None),\n 'numpy': ('https://numpy.org/doc/stable', None),\n 'torch': ('https://pytorch.org/docs/stable/', None)\n}\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = ['_templates']\n\n# The suffix(es) of source filenames.\n# You can specify multiple suffix as a list of string:\n#\nsource_suffix = {\n '.rst': 'restructuredtext',\n '.md': 'markdown',\n}\n\n# The master toctree document.\nmaster_doc = 'index'\n\n# The language for content autogenerated by Sphinx. Refer to documentation\n# for a list of supported languages.\n#\n# This is also used if you do content translation via gettext catalogs.\n# Usually you set \"language\" from the command line for these cases.\nlanguage = None\n\n# List of patterns, relative to source directory, that match files and\n# directories to ignore when looking for source files.\n# This pattern also affects html_static_path and html_extra_path.\nexclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']\n\n# The name of the Pygments (syntax highlighting) style to use.\npygments_style = 'sphinx'\n\n# -- Options for HTML output -------------------------------------------------\n\n# The theme to use for HTML and HTML Help pages. See the documentation for\n# a list of builtin themes.\n#\n# html_theme = 'sphinx_rtd_theme'\nhtml_theme = 'pytorch_sphinx_theme'\nhtml_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]\n\n# Theme options are theme-specific and customize the look and feel of a theme\n# further. For a list of options available for each theme, see the\n# documentation.\n#\nhtml_theme_options = {\n 'menu': [\n {\n 'name': 'GitHub',\n 'url': 'https://github.com/liangjiandeng/DLPan-Toolbox'\n },\n ],\n # Specify the language of shared menu\n 'menu_lang': 'en',\n}\n\n# Add any paths that contain custom static files (such as style sheets) here,\n# relative to this directory. They are copied after the builtin static files,\n# so a file named \"default.css\" will overwrite the builtin \"default.css\".\nhtml_static_path = ['_static']\nhtml_css_files = ['css/readthedocs.css']\n\n# Custom sidebar templates, must be a dictionary that maps document names\n# to template names.\n#\n# The default sidebars (for documents that don't match any pattern) are\n# defined by theme itself. Builtin themes are using these templates by\n# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',\n# 'searchbox.html']``.\n#\n# html_sidebars = {}\n\n# -- Options for HTMLHelp output ---------------------------------------------\n\n# Output file base name for HTML help builder.\nhtmlhelp_basename = 'dlpantoolboxndoc'\n\n# -- Options for LaTeX output ------------------------------------------------\n\nlatex_elements = {\n # The paper size ('letterpaper' or 'a4paper').\n #\n # 'papersize': 'letterpaper',\n\n # The font size ('10pt', '11pt' or '12pt').\n #\n # 'pointsize': '10pt',\n\n # Additional stuff for the LaTeX preamble.\n #\n # 'preamble': '',\n\n # Latex figure (float) alignment\n #\n # 'figure_align': 'htbp',\n}\n\n# Grouping the document tree into LaTeX files. List of tuples\n# (source start file, target name, title,\n# author, documentclass [howto, manual, or own class]).\nlatex_documents = [\n (master_doc, 'dlpantoolbox.tex', 'dlpantoolbox Documentation', 'DLPanToolbox Contributors',\n 'manual'),\n]\n\n# -- Options for manual page output ------------------------------------------\n\n# One entry per manual page. List of tuples\n# (source start file, name, description, authors, manual section).\nman_pages = [(master_doc, 'dlpantoolbox', 'dlpantoolbox Documentation', [author], 1)]\n\n# -- Options for Texinfo output ----------------------------------------------\n\n# Grouping the document tree into Texinfo files. List of tuples\n# (source start file, target name, title, author,\n# dir menu entry, description, category)\ntexinfo_documents = [\n (master_doc, 'dlpantoolbox', 'dlpantoolbox Documentation', author, 'dlpantoolbox',\n 'One line description of project.', 'Miscellaneous'),\n]\n\n# -- Options for Epub output -------------------------------------------------\n\n# Bibliographic Dublin Core info.\nepub_title = project\n\n# The unique identifier of the text. This can be a ISBN number\n# or the project homepage.\n#\n# epub_identifier = ''\n\n# A unique identification for the text.\n#\n# epub_uid = ''\n\n# A list of files that should not be packed into the epub file.\nepub_exclude_files = ['search.html']\n\n# set priority when building html\nStandaloneHTMLBuilder.supported_image_types = [\n 'image/svg+xml', 'image/gif', 'image/png', 'image/jpeg'\n]\n# -- Extension configuration -------------------------------------------------\n# Ignore >>> when copying code\ncopybutton_prompt_text = r'>>> |\\.\\.\\. '\ncopybutton_prompt_is_regexp = True" }, { "path": "docs/en/docutils.conf", "content": "[html writers]\ntable_style: colwidths-auto" }, { "path": "docs/en/faq.md", "content": "## Frequently Asked Questions\n\nWe list some common troubles faced by many users and their corresponding solutions here.\nFeel free to enrich the list if you find any frequent issues and have ways to help others to solve them.\n\n### Installation\n\n- KeyError: \"xxx: 'yyy is not in the zzz registry'\"\n\n The registry mechanism will be triggered only when the file of the module is imported.\n So you need to import that file somewhere. More details can be found at [KeyError: \"MaskRCNN: 'RefineRoIHead is not in the models registry'\"](https://github.com/open-mmlab/mmdetection/issues/5974).\n\n- \"No module named 'mmcv.ops'\"; \"No module named 'mmcv.\\_ext'\"\n\n 1. Uninstall existing mmcv in the environment using `pip uninstall mmcv`\n 2. Install mmcv-full following the [installation instruction](https://mmcv.readthedocs.io/en/latest/get_started/installation.html) or [Build MMCV from source](https://mmcv.readthedocs.io/en/latest/get_started/build.html)\n\n- \"invalid device function\" or \"no kernel image is available for execution\"\n\n 1. Check the CUDA compute capability of you GPU\n 2. Run `python mmdet/utils/collect_env.py` to check whether PyTorch, torchvision, and MMCV are built for the correct GPU architecture. You may need to set `TORCH_CUDA_ARCH_LIST` to reinstall MMCV. The compatibility issue could happen when using old GPUS, e.g., Tesla K80 (3.7) on colab.\n 3. Check whether the running environment is the same as that when mmcv/mmdet is compiled. For example, you may compile mmcv using CUDA 10.0 bug run it on CUDA9.0 environments\n\n- \"undefined symbol\" or \"cannot open xxx.so\"\n\n 1. If those symbols are CUDA/C++ symbols (e.g., libcudart.so or GLIBCXX), check\n whether the CUDA/GCC runtimes are the same as those used for compiling mmcv\n 2. If those symbols are Pytorch symbols (e.g., symbols containing caffe, aten, and TH), check whether the Pytorch version is the same as that used for compiling mmcv\n 3. Run `python mmdet/utils/collect_env.py` to check whether PyTorch, torchvision, and MMCV are built by and running on the same environment\n\n- \"RuntimeError: CUDA error: invalid configuration argument\"\n\n This error may be caused by the poor performance of GPU. Try to decrease the value of [THREADS_PER_BLOCK](https://github.com/open-mmlab/mmcv/blob/cac22f8cf5a904477e3b5461b1cc36856c2793da/mmcv/ops/csrc/common_cuda_helper.hpp#L10)\n and recompile mmcv.\n\n- \"RuntimeError: nms is not compiled with GPU support\"\n\n This error is because your CUDA environment is not installed correctly.\n You may try to re-install your CUDA environment and then delete the build/ folder before re-compile mmcv.\n\n- \"Segmentation fault\"\n\n 1. Check your GCC version and use GCC >= 5.4. This usually caused by the incompatibility between PyTorch and the environment (e.g., GCC \\< 4.9 for PyTorch). We also recommend the users to avoid using GCC 5.5 because many feedbacks report that GCC 5.5 will cause \"segmentation fault\" and simply changing it to GCC 5.4 could solve the problem\n 2. Check whether PyTorch is correctly installed and could use CUDA op, e.g. type the following command in your terminal and see whether they could correctly output results\n ```shell\n python -c 'import torch; print(torch.cuda.is_available())'\n ```\n 3. If PyTorch is correctly installed, check whether MMCV is correctly installed. If MMCV is correctly installed, then there will be no issue of the command\n ```shell\n python -c 'import mmcv; import mmcv.ops'\n ```\n 4. If MMCV and PyTorch are correctly installed, you can use `ipdb` to set breakpoints or directly add `print` to debug and see which part leads the `segmentation fault`\n\n- \"libtorch_cuda_cu.so: cannot open shared object file\"\n\n `mmcv-full` depends on the share object but it can not be found. We can check whether the object exists in `~/miniconda3/envs/{environment-name}/lib/python3.7/site-packages/torch/lib` or try to re-install the PyTorch.\n\n- \"fatal error C1189: #error: -- unsupported Microsoft Visual Studio version!\"\n\n If you are building mmcv-full on Windows and the version of CUDA is 9.2, you will probably encounter the error `\"C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v9.2\\include\\crt/host_config.h(133): fatal error C1189: #error: -- unsupported Microsoft Visual Studio version! Only the versions 2012, 2013, 2015 and 2017 are supported!\"`, in which case you can use a lower version of Microsoft Visual Studio like vs2017.\n\n- \"error: member \"torch::jit::detail::ModulePolicy::all_slots\" may not be initialized\"\n\n If your version of PyTorch is 1.5.0 and you are building mmcv-full on Windows, you will probably encounter the error `- torch/csrc/jit/api/module.h(474): error: member \"torch::jit::detail::ModulePolicy::all_slots\" may not be initialized`. The way to solve the error is to replace all the `static constexpr bool all_slots = false;` with `static bool all_slots = false;` at this file `https://github.com/pytorch/pytorch/blob/v1.5.0/torch/csrc/jit/api/module.h`. More details can be found at [member \"torch::jit::detail::AttributePolicy::all_slots\" may not be initialized](https://github.com/pytorch/pytorch/issues/39394).\n\n- \"error: a member with an in-class initializer must be const\"\n\n If your version of PyTorch is 1.6.0 and you are building mmcv-full on Windows, you will probably encounter the error `\"- torch/include\\torch/csrc/jit/api/module.h(483): error: a member with an in-class initializer must be const\"`. The way to solve the error is to replace all the `CONSTEXPR_EXCEPT_WIN_CUDA ` with `const` at `torch/include\\torch/csrc/jit/api/module.h`. More details can be found at [Ninja: build stopped: subcommand failed](https://github.com/open-mmlab/mmcv/issues/575).\n\n- \"error: member \"torch::jit::ProfileOptionalOp::Kind\" may not be initialized\"\n\n If your version of PyTorch is 1.7.0 and you are building mmcv-full on Windows, you will probably encounter the error `torch/include\\torch/csrc/jit/ir/ir.h(1347): error: member \"torch::jit::ProfileOptionalOp::Kind\" may not be initialized`. The way to solve the error needs to modify several local files of PyTorch:\n\n - delete `static constexpr Symbol Kind = ::c10::prim::profile;` and `tatic constexpr Symbol Kind = ::c10::prim::profile_optional;` at `torch/include\\torch/csrc/jit/ir/ir.h`\n - replace `explicit operator type&() { return *(this->value); }` with `explicit operator type&() { return *((type*)this->value); }` at `torch\\include\\pybind11\\cast.h`\n - replace all the `CONSTEXPR_EXCEPT_WIN_CUDA` with `const` at `torch/include\\torch/csrc/jit/api/module.h`\n\n More details can be found at [Ensure default extra_compile_args](https://github.com/pytorch/pytorch/pull/45956).\n\n- Compatibility issue between MMCV and MMDetection; \"ConvWS is already registered in conv layer\"\n\n Please install the correct version of MMCV for the version of your MMDetection following the [installation instruction](https://mmdetection.readthedocs.io/en/latest/get_started.html#installation).\n\n### Usage\n\n- \"RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one\"\n\n 1. This error indicates that your module has parameters that were not used in producing loss. This phenomenon may be caused by running different branches in your code in DDP mode. More datails at [Expected to have finished reduction in the prior iteration before starting a new one](https://github.com/pytorch/pytorch/issues/55582).\n 2. You can set ` find_unused_parameters = True` in the config to solve the above problems or find those unused parameters manually\n\n- \"RuntimeError: Trying to backward through the graph a second time\"\n\n `GradientCumulativeOptimizerHook` and `OptimizerHook` are both set which causes the `loss.backward()` to be called twice so `RuntimeError` was raised. We can only use one of these. More datails at [Trying to backward through the graph a second time](https://github.com/open-mmlab/mmcv/issues/1379)." }, { "path": "docs/en/get_started/Installation.md", "content": "## Installation\n\nThere are two versions of MMCV:\n\n- **mmcv**: comprehensive, with full features and various CUDA ops out of box. It takes longer time to build.\n- **mmcv-lite**: lite, without CUDA ops but all other features, similar to mmcv\\<1.0.0. It is useful when you do not need those CUDA ops.\n\n```{warning}\nDo not install both versions in the same environment, otherwise you may encounter errors like `ModuleNotFound`. You need to uninstall one before installing the other. `Installing the full version is highly recommended if CUDA is avaliable`.\n```\n\n### Install mmcv\n\nBefore installing mmcv, make sure that PyTorch has been successfully installed following the [PyTorch official installation guide](https://pytorch.org/get-started/locally/#start-locally). This can be verified using the following command\n\n```bash\npython -c 'import torch;print(torch.__version__)'\n```\n\nIf version information is output, then PyTorch is installed.\n\n#### Install with mim (recommended)\n\n[mim](https://github.com/open-mmlab/mim) is the package management tool for the OpenMMLab projects, which makes it easy to install mmcv\n\n```bash\npip install -U openmim\nmim install mmcv\n```\n\nIf you find that the above installation command does not use a pre-built package ending with `.whl` but a source package ending with `.tar.gz`, you may not have a pre-build package corresponding to the PyTorch or CUDA or mmcv version, in which case you can [build mmcv from source](build.md).\n\n
\nInstallation log using pre-built packages\n\nLooking in links: https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html
\nCollecting mmcv
\nDownloading https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/mmcv-2.0.0-cp38-cp38-manylinux1_x86_64.whl\n\n
\n\n
\nInstallation log using source packages\n\nLooking in links: https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html
\nCollecting mmcv==2.0.0
\nDownloading mmcv-2.0.0.tar.gz\n\n
\n\nTo install a specific version of mmcv, for example, mmcv version 2.0.0, you can use the following command\n\n```bash\nmim install mmcv==2.0.0\n```\n\n:::{note}\nIf you would like to use `opencv-python-headless` instead of `opencv-python`,\ne.g., in a minimum container environment or servers without GUI,\nyou can first install it before installing MMCV to skip the installation of `opencv-python`.\n\nAlternatively, if it takes too long to install a dependency library, you can specify the pypi source\n\n```bash\nmim install mmcv -i https://pypi.tuna.tsinghua.edu.cn/simple\n```\n\n:::\n\nYou can run [check_installation.py](https://github.com/open-mmlab/mmcv/blob/main/.dev_scripts/check_installation.py) to check the installation of mmcv-full after running the installation commands.\n\n#### Install with pip\n\nUse the following command to check the version of CUDA and PyTorch\n\n```bash\npython -c 'import torch;print(torch.__version__);print(torch.version.cuda)'\n```\n\nSelect the appropriate installation command depending on the type of system, CUDA version, PyTorch version, and MMCV version\n\n\n\n \n
\n \n \n \n \n \n \n \n \n
\n 
\n\n\n\n\nIf you do not find a corresponding version in the dropdown box above, you probably do not have a pre-built package corresponding to the PyTorch or CUDA or mmcv version, at which point you can [build mmcv from source](build.md).\n\n:::{note}\nmmcv is only compiled on PyTorch 1.x.0 because the compatibility\nusually holds between 1.x.0 and 1.x.1. If your PyTorch version is 1.x.1, you\ncan install mmcv compiled with PyTorch 1.x.0 and it usually works well.\nFor example, if your PyTorch version is 1.8.1, you can feel free to choose 1.8.x.\n:::\n\n:::{note}\nIf you would like to use `opencv-python-headless` instead of `opencv-python`,\ne.g., in a minimum container environment or servers without GUI,\nyou can first install it before installing MMCV to skip the installation of `opencv-python`.\n\nAlternatively, if it takes too long to install a dependency library, you can specify the pypi source\n\n```bash\nmim install mmcv -i https://pypi.tuna.tsinghua.edu.cn/simple\n```\n\n:::\n\nYou can run [check_installation.py](https://github.com/open-mmlab/mmcv/blob/main/.dev_scripts/check_installation.py) to check the installation of mmcv after running the installation commands.\n\n#### Using mmcv with Docker\n\nBuild with local repository\n\n```bash\ngit clone https://github.com/open-mmlab/mmcv.git && cd mmcv\ndocker build -t mmcv -f docker/release/Dockerfile .\n```\n\nOr build with remote repository\n\n```bash\ndocker build -t mmcv https://github.com/open-mmlab/mmcv.git#main:docker/release\n```\n\nThe [Dockerfile](release/Dockerfile) installs latest released version of mmcv-full by default, but you can specify mmcv versions to install expected versions.\n\n```bash\ndocker image build -t mmcv -f docker/release/Dockerfile --build-arg MMCV=2.0.0 .\n```\n\nIf you also want to use other versions of PyTorch and CUDA, you can also pass them when building docker images.\n\nAn example to build an image with PyTorch 1.11 and CUDA 11.3.\n\n```bash\ndocker build -t mmcv -f docker/release/Dockerfile \\\n    --build-arg PYTORCH=1.11.0 \\\n    --build-arg CUDA=11.3 \\\n    --build-arg CUDNN=8 \\\n    --build-arg MMCV=2.0.0 .\n```\n\nMore available versions of PyTorch and CUDA can be found at [dockerhub/pytorch](https://hub.docker.com/r/pytorch/pytorch/tags).\n\n### Install mmcv-lite\n\nIf you need to use PyTorch-related modules, make sure PyTorch has been successfully installed in your environment by referring to the [PyTorch official installation guide](https://github.com/pytorch/pytorch#installation).\n\n```python\npip install mmcv-lite\n```"
  },
  {
    "path": "docs/en/get_started/Introduction.md",
    "content": "## Introduction\n\nMMCV is a foundational library for computer vision research and provides the following functionalities.\n\n- [Image/Video processing](../understand_mmcv/data_process.md)\n- [Image and annotation visualization](../understand_mmcv/visualization.md)\n- [Image transformation](../understand_mmcv/data_transform.md)\n- [Various CNN architectures](../understand_mmcv/cnn.md)\n- [High-quality implementation of common CUDA ops](../understand_mmcv/ops.md)\n\nIt supports the following systems:\n\n- Linux\n- Windows\n- macOS\n\nIt supports many research projects as below:\n\n- [MMClassification](https://github.com/open-mmlab/mmclassification): OpenMMLab image classification toolbox and benchmark.\n- [MMDetection](https://github.com/open-mmlab/mmdetection): OpenMMLab detection toolbox and benchmark.\n- [MMDetection3D](https://github.com/open-mmlab/mmdetection3d): OpenMMLab's next-generation platform for general 3D object detection.\n- [MMRotate](https://github.com/open-mmlab/mmrotate): OpenMMLab rotated object detection toolbox and benchmark.\n- [MMYOLO](https://github.com/open-mmlab/mmyolo): OpenMMLab YOLO series toolbox and benchmark.\n- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation): OpenMMLab semantic segmentation toolbox and benchmark.\n- [MMOCR](https://github.com/open-mmlab/mmocr): OpenMMLab text detection, recognition, and understanding toolbox.\n- [MMPose](https://github.com/open-mmlab/mmpose): OpenMMLab pose estimation toolbox and benchmark.\n- [MMHuman3D](https://github.com/open-mmlab/mmhuman3d): OpenMMLab 3D human parametric model toolbox and benchmark.\n- [MMSelfSup](https://github.com/open-mmlab/mmselfsup): OpenMMLab self-supervised learning toolbox and benchmark.\n- [MMRazor](https://github.com/open-mmlab/mmrazor): OpenMMLab model compression toolbox and benchmark.\n- [MMFewShot](https://github.com/open-mmlab/mmfewshot): OpenMMLab fewshot learning toolbox and benchmark.\n- [MMAction2](https://github.com/open-mmlab/mmaction2): OpenMMLab's next-generation action understanding toolbox and benchmark.\n- [MMTracking](https://github.com/open-mmlab/mmtracking): OpenMMLab video perception toolbox and benchmark.\n- [MMFlow](https://github.com/open-mmlab/mmflow): OpenMMLab optical flow toolbox and benchmark.\n- [MMEditing](https://github.com/open-mmlab/mmediting): OpenMMLab image and video editing toolbox.\n- [MMGeneration](https://github.com/open-mmlab/mmgeneration): OpenMMLab image and video generative models toolbox.\n- [MMDeploy](https://github.com/open-mmlab/mmdeploy): OpenMMLab model deployment framework."
  },
  {
    "path": "docs/en/index.rst",
    "content": "Welcome to DLPanToolbox's documentation!\n================================\n\n\n\nYou can switch between Chinese and English documents in the lower-left corner of the layout.\n\n.. toctree::\n   :caption: Switch Language\n\n   switch_language.md\n\n.. toctree::\n   :glob:\n   :caption: Get Started\n\n   get_started/Introduction.md\n   get_started/Installation.md\n\n.. toctree::\n   :glob:\n   :caption: PanCollection\n\n   PanCollection/Simulation.md\n   PanCollection/PreProcess.md\n   PanCollection/Example.md\n   PanCollection/Evaluation.md\n\n.. toctree::\n   :caption: Utilization\n\n   citation.md\n   faq.md\n\n.. toctree::\n   :glob:\n   :maxdepth: 2\n   :caption: Python API\n\n\n\n\nIndices and tables\n==================\n\n* :ref:`genindex`\n* :ref:`search`"
  },
  {
    "path": "docs/en/make.bat",
    "content": "@ECHO OFF\n\npushd %~dp0\n\nREM Command file for Sphinx documentation\n\nif \"%SPHINXBUILD%\" == \"\" (\n\tset SPHINXBUILD=sphinx-build\n)\nset SOURCEDIR=.\nset BUILDDIR=_build\n\nif \"%1\" == \"\" goto help\n\n%SPHINXBUILD% >NUL 2>NUL\nif errorlevel 9009 (\n\techo.\n\techo.The 'sphinx-build' command was not found. Make sure you have Sphinx\n\techo.installed, then set the SPHINXBUILD environment variable to point\n\techo.to the full path of the 'sphinx-build' executable. Alternatively you\n\techo.may add the Sphinx directory to PATH.\n\techo.\n\techo.If you don't have Sphinx installed, grab it from\n\techo.http://sphinx-doc.org/\n\texit /b 1\n)\n\n%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\ngoto end\n\n:help\n%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\n\n:end\npopd"
  },
  {
    "path": "docs/en/switch_language.md",
    "content": "## Change to English\n\n## 切换到简体中文"
  },
  {
    "path": "docs/requirements.txt",
    "content": "-e git+https://github.com/XiaoXiao-Woo/pytorch_sphinx_theme.git#egg=pytorch_sphinx_theme\nsphinx==4.0.2\nsphinx-copybutton\nsphinx_markdown_tables>=0.0.16\nmyst-parser\nsphinx-autobuild"
  },
  {
    "path": "docs/run.sh",
    "content": "sphinx-autobuild en build/html"
  },
  {
    "path": "docs/zh-cn/DLPanToolbox/Evaluation.md",
    "content": "## Evaluation"
  },
  {
    "path": "docs/zh-cn/DLPanToolbox/Example.md",
    "content": ""
  },
  {
    "path": "docs/zh-cn/DLPanToolbox/PreProcess.md",
    "content": "## Data PreProcess\n\n### Image\n\nThis module provides some image processing methods, which requires `opencv` to be installed first.\n\n#### Read/Write/Show\n\nTo read or write images files, use `imread` or `imwrite`.\n\n```python\nimport mmcv\n\nimg = mmcv.imread('test.jpg')\nimg = mmcv.imread('test.jpg', flag='grayscale')\nimg_ = mmcv.imread(img)  # nothing will happen, img_ = img\nmmcv.imwrite(img, 'out.jpg')\n```\n\nTo read images from bytes\n\n```python\nwith open('test.jpg', 'rb') as f:\n    data = f.read()\nimg = mmcv.imfrombytes(data)\n```\n\nTo show an image file or a loaded image\n\n```python\nmmcv.imshow('tests/data/color.jpg')\n# this is equivalent to\n\nfor i in range(10):\n    img = np.random.randint(256, size=(100, 100, 3), dtype=np.uint8)\n    mmcv.imshow(img, win_name='test image', wait_time=200)\n```\n\n#### Color space conversion\n\nSupported conversion methods:\n\n- bgr2gray\n- gray2bgr\n- bgr2rgb\n- rgb2bgr\n- bgr2hsv\n- hsv2bgr\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\nimg1 = mmcv.bgr2rgb(img)\nimg2 = mmcv.rgb2gray(img1)\nimg3 = mmcv.bgr2hsv(img)\n```\n\n#### Resize\n\nThere are three resize methods. All `imresize_*` methods have an argument `return_scale`,\nif this argument is `False`, then the return value is merely the resized image, otherwise\nis a tuple `(resized_img, scale)`.\n\n```python\n# resize to a given size\nmmcv.imresize(img, (1000, 600), return_scale=True)\n\n# resize to the same size of another image\nmmcv.imresize_like(img, dst_img, return_scale=False)\n\n# resize by a ratio\nmmcv.imrescale(img, 0.5)\n\n# resize so that the max edge no longer than 1000, short edge no longer than 800\n# without changing the aspect ratio\nmmcv.imrescale(img, (1000, 800))\n```\n\n#### Rotate\n\nTo rotate an image by some angle, use `imrotate`. The center can be specified,\nwhich is the center of original image by default. There are two modes of rotating,\none is to keep the image size unchanged so that some parts of the image will be\ncropped after rotating, the other is to extend the image size to fit the rotated\nimage.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# rotate the image clockwise by 30 degrees.\nimg_ = mmcv.imrotate(img, 30)\n\n# rotate the image counterclockwise by 90 degrees.\nimg_ = mmcv.imrotate(img, -90)\n\n# rotate the image clockwise by 30 degrees, and rescale it by 1.5x at the same time.\nimg_ = mmcv.imrotate(img, 30, scale=1.5)\n\n# rotate the image clockwise by 30 degrees, with (100, 100) as the center.\nimg_ = mmcv.imrotate(img, 30, center=(100, 100))\n\n# rotate the image clockwise by 30 degrees, and extend the image size.\nimg_ = mmcv.imrotate(img, 30, auto_bound=True)\n```\n\n#### Flip\n\nTo flip an image, use `imflip`.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# flip the image horizontally\nmmcv.imflip(img)\n\n# flip the image vertically\nmmcv.imflip(img, direction='vertical')\n```\n\n#### Crop\n\n`imcrop` can crop the image with one or more regions. Each region is represented by the upper left and lower right coordinates as (x1, y1, x2, y2).\n\n```python\nimport mmcv\nimport numpy as np\n\nimg = mmcv.imread('tests/data/color.jpg')\n\n# crop the region (10, 10, 100, 120)\nbboxes = np.array([10, 10, 100, 120])\npatch = mmcv.imcrop(img, bboxes)\n\n# crop two regions (10, 10, 100, 120) and (0, 0, 50, 50)\nbboxes = np.array([[10, 10, 100, 120], [0, 0, 50, 50]])\npatches = mmcv.imcrop(img, bboxes)\n\n# crop two regions, and rescale the patches by 1.2x\npatches = mmcv.imcrop(img, bboxes, scale=1.2)\n```\n\n#### Padding\n\nThere are two methods, `impad` and `impad_to_multiple`, to pad an image to the\nspecific size with given values.\n\n```python\nimg = mmcv.imread('tests/data/color.jpg')\n\n# pad the image to (1000, 1200) with all zeros\nimg_ = mmcv.impad(img, shape=(1000, 1200), pad_val=0)\n\n# pad the image to (1000, 1200) with different values for three channels.\nimg_ = mmcv.impad(img, shape=(1000, 1200), pad_val=(100, 50, 200))\n\n# pad the image on left, right, top, bottom borders with all zeros\nimg_ = mmcv.impad(img, padding=(10, 20, 30, 40), pad_val=0)\n\n# pad the image on left, right, top, bottom borders with different values\n# for three channels.\nimg_ = mmcv.impad(img, padding=(10, 20, 30, 40), pad_val=(100, 50, 200))\n\n# pad an image so that each edge is a multiple of some value.\nimg_ = mmcv.impad_to_multiple(img, 32)\n```"
  },
  {
    "path": "docs/zh-cn/DLPanToolbox/Simulation.md",
    "content": "## Simulation"
  },
  {
    "path": "docs/zh-cn/Makefile",
    "content": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line, and also\n# from the environment for the first two.\nSPHINXOPTS    ?=\nSPHINXBUILD   ?= sphinx-build\nSOURCEDIR     = source\nBUILDDIR      = build\n\n# Put it first so that \"make\" without argument is like \"make help\".\nhelp:\n\t@$(SPHINXBUILD) -M help \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n\n.PHONY: help Makefile\n\n# Catch-all target: route all unknown targets to Sphinx using the new\n# \"make mode\" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).\n%: Makefile\n\t@$(SPHINXBUILD) -M $@ \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)"
  },
  {
    "path": "docs/zh-cn/_static/css/readthedocs.css",
    "content": ".header-logo {\n    background-image: url(\"../image/logo-dlpan.png\");\n    background-size: 85px 40px;\n    height: 40px;\n    width: 85px;\n}\n\ntable.colwidths-auto td {\n    width: 50%\n}"
  },
  {
    "path": "docs/zh-cn/_templates/classtemplate.rst",
    "content": ".. role:: hidden\n    :class: hidden-section\n.. currentmodule:: {{ module }}\n\n\n{{ name | underline}}\n\n.. autoclass:: {{ name }}\n    :members:\n\n\n..\n  autogenerated from source/_templates/classtemplate.rst\n  note it does not have :inherited-members:"
  },
  {
    "path": "docs/zh-cn/citation.md",
    "content": "## Cite PanCollecton\nIf PanCollection is helpful for you,  you are encouraged to cite the following paper:\n```bibtex\n@misc{PanCollection,\n    author = {Xiao Wu, Liang-Jian Deng and Ran Ran},\n    title = {\"PanCollection\" for Remote Sensing Pansharpening},\n    url = {https://github.com/XiaoXiao-Woo/PanCollection/},\n    year = {2022},\n}\n```\n```bibtex\n@ARTICLE{deng2022grsm,\nauthor={L.-J. Deng, G. Vivone, M. E. Paoletti, G. Scarpa, J. He, Y. Zhang, J. Chanussot, and A. Plaza},\nbooktitle={IEEE Geoscience and Remote Sensing Magazine},\ntitle={Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks},\nyear={2022},\npages={2-38},\ndoi={10.1109/MGRS.2020.3019315}\n}\n```\nFor Chinese Paper,\n```bibtex\n@ARTICLE{dengjig2022,\n\tauthor={邓良剑，冉燃，吴潇，张添敬},\n\tjournal={中国图象图形学报},\n\ttitle={遥感图像全色锐化的卷积神经网络方法研究进展},\n \tyear={2022},\n  \tvolume={},\n  \tnumber={9},\n  \tpages={},\n  \tdoi={10.11834/jig.220540}\n   }\n```\nAlso, the codes of traditional methods are from the \"pansharpening toolbox for distribution\", thus please cite the corresponding paper:\n```bibtex\n@ARTICLE{vivone2021grsm,\n  author={Vivone, Gemine and Dalla Mura, Mauro and Garzelli, Andrea and Restaino, Rocco and Scarpa, Giuseppe and Ulfarsson, Magnus O. and   Alparone, Luciano and Chanussot, Jocelyn},\n  journal={IEEE Geoscience and Remote Sensing Magazine}, \n  title={A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods}, \n  year={2021},\n  volume={9},\n  number={1},\n  pages={53-81},\n  doi={10.1109/MGRS.2020.3019315}\n}\n```"
  },
  {
    "path": "docs/zh-cn/conf.py",
    "content": "#  GPL v3.0 License\n#  Copyright (C) UESTC\n#  All Rights Reserved\n#  @Time    : 2023/9/21\n#  @Author  : Xiao Wu\n#  @reference:\n#\n\n#\n# Configuration file for the Sphinx documentation builder.\n#\n# This file does only contain a selection of the most common options. For a\n# full list see the documentation:\n# http://www.sphinx-doc.org/en/master/config\n\n# -- Path setup --------------------------------------------------------------\n\n# If extensions (or modules to document with autodoc) are in another directory,\n# add these directories to sys.path here. If the directory is relative to the\n# documentation root, use os.path.abspath to make it absolute, like shown here.\n#\nimport os\nimport sys\n\nimport pytorch_sphinx_theme\nfrom sphinx.builders.html import StandaloneHTMLBuilder\n\nsys.path.insert(0, os.path.abspath('../..'))\n\n# version_file = '../../mmcv/version.py'\n# with open(version_file) as f:\n#     exec(compile(f.read(), version_file, 'exec'))\n# __version__ = locals()['__version__']\n__version__ = \"0.3.6\"\n\n# -- Project information -----------------------------------------------------\n\nproject = 'pancollection'\ncopyright = '2023, UESTC'\nauthor = 'Xiao Wu'\n\n# The short X.Y version\nversion = __version__\n# The full version, including alpha/beta/rc tags\nrelease = __version__\n\n# -- General configuration ---------------------------------------------------\n\n# If your documentation needs a minimal Sphinx version, state it here.\n#\n# needs_sphinx = '1.0'\n\n# Add any Sphinx extension module names here, as strings. They can be\n# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom\n# ones.\n\nextensions = [\n    'sphinx.ext.autodoc',\n    'sphinx.ext.autosummary',\n    'sphinx.ext.intersphinx',\n    'sphinx.ext.napoleon',\n    'sphinx.ext.viewcode',\n    'sphinx_markdown_tables',\n    'myst_parser',\n    'sphinx_copybutton',\n]  # yapf: disable\n\nmyst_heading_anchors = 4\n\nmyst_enable_extensions = ['colon_fence']\n\n# Configuration for intersphinx\nintersphinx_mapping = {\n    'python': ('https://docs.python.org/3', None),\n    'numpy': ('https://numpy.org/doc/stable', None),\n    'torch': ('https://pytorch.org/docs/stable/', None)\n}\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = ['_templates']\n\n# The suffix(es) of source filenames.\n# You can specify multiple suffix as a list of string:\n#\nsource_suffix = {\n    '.rst': 'restructuredtext',\n    '.md': 'markdown',\n}\n\n# The master toctree document.\nmaster_doc = 'index'\n\n# The language for content autogenerated by Sphinx. Refer to documentation\n# for a list of supported languages.\n#\n# This is also used if you do content translation via gettext catalogs.\n# Usually you set \"language\" from the command line for these cases.\nlanguage = None\n\n# List of patterns, relative to source directory, that match files and\n# directories to ignore when looking for source files.\n# This pattern also affects html_static_path and html_extra_path.\nexclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']\n\n# The name of the Pygments (syntax highlighting) style to use.\npygments_style = 'sphinx'\n\n# -- Options for HTML output -------------------------------------------------\n\n# The theme to use for HTML and HTML Help pages.  See the documentation for\n# a list of builtin themes.\n#\n# html_theme = 'sphinx_rtd_theme'\nhtml_theme = 'pytorch_sphinx_theme'\nhtml_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]\n\n# Theme options are theme-specific and customize the look and feel of a theme\n# further.  For a list of options available for each theme, see the\n# documentation.\n#\nhtml_theme_options = {\n    'menu': [\n        {\n            'name': 'GitHub',\n            'url': 'https://github.com/XiaoXiao-Woo/PanCollection'\n        },\n    ],\n    # Specify the language of shared menu\n    'menu_lang': 'en',\n}\n\n# Add any paths that contain custom static files (such as style sheets) here,\n# relative to this directory. They are copied after the builtin static files,\n# so a file named \"default.css\" will overwrite the builtin \"default.css\".\nhtml_static_path = ['_static']\nhtml_css_files = ['css/readthedocs.css']\n\n# Custom sidebar templates, must be a dictionary that maps document names\n# to template names.\n#\n# The default sidebars (for documents that don't match any pattern) are\n# defined by theme itself.  Builtin themes are using these templates by\n# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',\n# 'searchbox.html']``.\n#\n# html_sidebars = {}\n\n# -- Options for HTMLHelp output ---------------------------------------------\n\n# Output file base name for HTML help builder.\nhtmlhelp_basename = 'pancollectiondoc'\n\n# -- Options for LaTeX output ------------------------------------------------\n\nlatex_elements = {\n    # The paper size ('letterpaper' or 'a4paper').\n    #\n    # 'papersize': 'letterpaper',\n\n    # The font size ('10pt', '11pt' or '12pt').\n    #\n    # 'pointsize': '10pt',\n\n    # Additional stuff for the LaTeX preamble.\n    #\n    # 'preamble': '',\n\n    # Latex figure (float) alignment\n    #\n    # 'figure_align': 'htbp',\n}\n\n# Grouping the document tree into LaTeX files. List of tuples\n# (source start file, target name, title,\n#  author, documentclass [howto, manual, or own class]).\nlatex_documents = [\n    (master_doc, 'pancollection.tex', 'pancollection Documentation', 'PanCollection Contributors',\n     'manual'),\n]\n\n# -- Options for manual page output ------------------------------------------\n\n# One entry per manual page. List of tuples\n# (source start file, name, description, authors, manual section).\nman_pages = [(master_doc, 'pancollection', 'pancollection Documentation', [author], 1)]\n\n# -- Options for Texinfo output ----------------------------------------------\n\n# Grouping the document tree into Texinfo files. List of tuples\n# (source start file, target name, title, author,\n#  dir menu entry, description, category)\ntexinfo_documents = [\n    (master_doc, 'pancollection', 'pancollection Documentation', author, 'pancollection',\n     'One line description of project.', 'Miscellaneous'),\n]\n\n# -- Options for Epub output -------------------------------------------------\n\n# Bibliographic Dublin Core info.\nepub_title = project\n\n# The unique identifier of the text. This can be a ISBN number\n# or the project homepage.\n#\n# epub_identifier = ''\n\n# A unique identification for the text.\n#\n# epub_uid = ''\n\n# A list of files that should not be packed into the epub file.\nepub_exclude_files = ['search.html']\n\n# set priority when building html\nStandaloneHTMLBuilder.supported_image_types = [\n    'image/svg+xml', 'image/gif', 'image/png', 'image/jpeg'\n]\n# -- Extension configuration -------------------------------------------------\n# Ignore >>> when copying code\ncopybutton_prompt_text = r'>>> |\\.\\.\\. '\ncopybutton_prompt_is_regexp = True"
  },
  {
    "path": "docs/zh-cn/docutils.conf",
    "content": "[html writers]\ntable_style: colwidths-auto"
  },
  {
    "path": "docs/zh-cn/faq.md",
    "content": "## Frequently Asked Questions\n\nWe list some common troubles faced by many users and their corresponding solutions here.\nFeel free to enrich the list if you find any frequent issues and have ways to help others to solve them.\n\n### Installation\n\n- KeyError: \"xxx: 'yyy is not in the zzz registry'\"\n\n  The registry mechanism will be triggered only when the file of the module is imported.\n  So you need to import that file somewhere. More details can be found at [KeyError: \"MaskRCNN: 'RefineRoIHead is not in the models registry'\"](https://github.com/open-mmlab/mmdetection/issues/5974).\n\n- \"No module named 'mmcv.ops'\"; \"No module named 'mmcv.\\_ext'\"\n\n  1. Uninstall existing mmcv in the environment using `pip uninstall mmcv`\n  2. Install mmcv-full following the [installation instruction](https://mmcv.readthedocs.io/en/latest/get_started/installation.html) or [Build MMCV from source](https://mmcv.readthedocs.io/en/latest/get_started/build.html)\n\n- \"invalid device function\" or \"no kernel image is available for execution\"\n\n  1. Check the CUDA compute capability of you GPU\n  2. Run `python mmdet/utils/collect_env.py` to check whether PyTorch, torchvision, and MMCV are built for the correct GPU architecture. You may need to set `TORCH_CUDA_ARCH_LIST` to reinstall MMCV. The compatibility issue could happen when  using old GPUS, e.g., Tesla K80 (3.7) on colab.\n  3. Check whether the running environment is the same as that when mmcv/mmdet is compiled. For example, you may compile mmcv using CUDA 10.0 bug run it on CUDA9.0 environments\n\n- \"undefined symbol\" or \"cannot open xxx.so\"\n\n  1. If those symbols are CUDA/C++ symbols (e.g., libcudart.so or GLIBCXX), check\n     whether the CUDA/GCC runtimes are the same as those used for compiling mmcv\n  2. If those symbols are Pytorch symbols (e.g., symbols containing caffe, aten, and TH), check whether the Pytorch version is the same as that used for compiling mmcv\n  3. Run `python mmdet/utils/collect_env.py` to check whether PyTorch, torchvision, and MMCV are built by and running on the same environment\n\n- \"RuntimeError: CUDA error: invalid configuration argument\"\n\n  This error may be caused by the poor performance of GPU. Try to decrease the value of [THREADS_PER_BLOCK](https://github.com/open-mmlab/mmcv/blob/cac22f8cf5a904477e3b5461b1cc36856c2793da/mmcv/ops/csrc/common_cuda_helper.hpp#L10)\n  and recompile mmcv.\n\n- \"RuntimeError: nms is not compiled with GPU support\"\n\n  This error is because your CUDA environment is not installed correctly.\n  You may try to re-install your CUDA environment and then delete the build/ folder before re-compile mmcv.\n\n- \"Segmentation fault\"\n\n  1. Check your GCC version and use GCC >= 5.4. This usually caused by the incompatibility between PyTorch and the environment (e.g., GCC \\< 4.9 for PyTorch). We also recommend the users to avoid using GCC 5.5 because many feedbacks report that GCC 5.5 will cause \"segmentation fault\" and simply changing it to GCC 5.4 could solve the problem\n  2. Check whether PyTorch is correctly installed and could use CUDA op, e.g. type the following command in your terminal and see whether they could correctly output results\n     ```shell\n     python -c 'import torch; print(torch.cuda.is_available())'\n     ```\n  3. If PyTorch is correctly installed, check whether MMCV is correctly installed. If MMCV is correctly installed, then there will be no issue of the command\n     ```shell\n     python -c 'import mmcv; import mmcv.ops'\n     ```\n  4. If MMCV and PyTorch are correctly installed, you can use `ipdb` to set breakpoints or directly add `print` to debug and see which part leads the `segmentation fault`\n\n- \"libtorch_cuda_cu.so: cannot open shared object file\"\n\n  `mmcv-full` depends on the share object but it can not be found. We can check whether the object exists in `~/miniconda3/envs/{environment-name}/lib/python3.7/site-packages/torch/lib` or try to re-install the PyTorch.\n\n- \"fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version!\"\n\n  If you are building mmcv-full on Windows and the version of CUDA is 9.2, you will probably encounter the error `\"C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v9.2\\include\\crt/host_config.h(133): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions 2012, 2013, 2015 and 2017 are supported!\"`, in which case you can use a lower version of Microsoft Visual Studio like vs2017.\n\n- \"error: member \"torch::jit::detail::ModulePolicy::all_slots\" may not be initialized\"\n\n  If your version of PyTorch is 1.5.0 and you are building mmcv-full on Windows, you will probably encounter the error `- torch/csrc/jit/api/module.h(474): error: member \"torch::jit::detail::ModulePolicy::all_slots\" may not be initialized`. The way to solve the error is to replace all the `static constexpr bool all_slots = false;` with `static bool all_slots = false;` at this file `https://github.com/pytorch/pytorch/blob/v1.5.0/torch/csrc/jit/api/module.h`. More details can be found at [member \"torch::jit::detail::AttributePolicy::all_slots\" may not be initialized](https://github.com/pytorch/pytorch/issues/39394).\n\n- \"error: a member with an in-class initializer must be const\"\n\n  If your version of PyTorch is 1.6.0 and you are building mmcv-full on Windows, you will probably encounter the error `\"- torch/include\\torch/csrc/jit/api/module.h(483): error: a member with an in-class initializer must be const\"`. The way to solve the error is to replace all the `CONSTEXPR_EXCEPT_WIN_CUDA ` with `const` at `torch/include\\torch/csrc/jit/api/module.h`. More details can be found at [Ninja: build stopped: subcommand failed](https://github.com/open-mmlab/mmcv/issues/575).\n\n- \"error: member \"torch::jit::ProfileOptionalOp::Kind\" may not be initialized\"\n\n  If your version of PyTorch is 1.7.0 and you are building mmcv-full on Windows, you will probably encounter the error `torch/include\\torch/csrc/jit/ir/ir.h(1347): error: member \"torch::jit::ProfileOptionalOp::Kind\" may not be initialized`. The way to solve the error needs to modify several local files of PyTorch:\n\n  - delete `static constexpr Symbol Kind = ::c10::prim::profile;` and `tatic constexpr Symbol Kind = ::c10::prim::profile_optional;` at `torch/include\\torch/csrc/jit/ir/ir.h`\n  - replace `explicit operator type&() { return *(this->value); }` with `explicit operator type&() { return *((type*)this->value); }` at `torch\\include\\pybind11\\cast.h`\n  - replace all the `CONSTEXPR_EXCEPT_WIN_CUDA` with `const` at `torch/include\\torch/csrc/jit/api/module.h`\n\n  More details can be found at [Ensure default extra_compile_args](https://github.com/pytorch/pytorch/pull/45956).\n\n- Compatibility issue between MMCV and MMDetection; \"ConvWS is already registered in conv layer\"\n\n  Please install the correct version of MMCV for the version of your MMDetection following the [installation instruction](https://mmdetection.readthedocs.io/en/latest/get_started.html#installation).\n\n### Usage\n\n- \"RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one\"\n\n  1. This error indicates that your module has parameters that were not used in producing loss. This phenomenon may be caused by running different branches in your code in DDP mode. More datails at [Expected to have finished reduction in the prior iteration before starting a new one](https://github.com/pytorch/pytorch/issues/55582).\n  2. You can set ` find_unused_parameters = True` in the config to solve the above problems or find those unused parameters manually\n\n- \"RuntimeError: Trying to backward through the graph a second time\"\n\n  `GradientCumulativeOptimizerHook` and `OptimizerHook` are both set which causes the `loss.backward()` to be called twice so `RuntimeError` was raised. We can only use one of these. More datails at [Trying to backward through the graph a second time](https://github.com/open-mmlab/mmcv/issues/1379)."
  },
  {
    "path": "docs/zh-cn/get_started/Installation.md",
    "content": "## Installation\n\nThere are two versions of MMCV:\n\n- **mmcv**: comprehensive, with full features and various CUDA ops out of box. It takes longer time to build.\n- **mmcv-lite**: lite, without CUDA ops but all other features, similar to mmcv\\<1.0.0. It is useful when you do not need those CUDA ops.\n\n```{warning}\nDo not install both versions in the same environment, otherwise you may encounter errors like `ModuleNotFound`. You need to uninstall one before installing the other. `Installing the full version is highly recommended if CUDA is avaliable`.\n```\n\n### Install mmcv\n\nBefore installing mmcv, make sure that PyTorch has been successfully installed following the [PyTorch official installation guide](https://pytorch.org/get-started/locally/#start-locally). This can be verified using the following command\n\n```bash\npython -c 'import torch;print(torch.__version__)'\n```\n\nIf version information is output, then PyTorch is installed.\n\n#### Install with mim (recommended)\n\n[mim](https://github.com/open-mmlab/mim) is the package management tool for the OpenMMLab projects, which makes it easy to install mmcv\n\n```bash\npip install -U openmim\nmim install mmcv\n```\n\nIf you find that the above installation command does not use a pre-built package ending with `.whl` but a source package ending with `.tar.gz`, you may not have a pre-build package corresponding to the PyTorch or CUDA or mmcv version, in which case you can [build mmcv from source](build.md).\n\n
\nInstallation log using pre-built packages\n\nLooking in links: https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html
\nCollecting mmcv
\nDownloading https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/mmcv-2.0.0-cp38-cp38-manylinux1_x86_64.whl\n\n
\n\n
\nInstallation log using source packages\n\nLooking in links: https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html
\nCollecting mmcv==2.0.0
\nDownloading mmcv-2.0.0.tar.gz\n\n
\n\nTo install a specific version of mmcv, for example, mmcv version 2.0.0, you can use the following command\n\n```bash\nmim install mmcv==2.0.0\n```\n\n:::{note}\nIf you would like to use `opencv-python-headless` instead of `opencv-python`,\ne.g., in a minimum container environment or servers without GUI,\nyou can first install it before installing MMCV to skip the installation of `opencv-python`.\n\nAlternatively, if it takes too long to install a dependency library, you can specify the pypi source\n\n```bash\nmim install mmcv -i https://pypi.tuna.tsinghua.edu.cn/simple\n```\n\n:::\n\nYou can run [check_installation.py](https://github.com/open-mmlab/mmcv/blob/main/.dev_scripts/check_installation.py) to check the installation of mmcv-full after running the installation commands.\n\n#### Install with pip\n\nUse the following command to check the version of CUDA and PyTorch\n\n```bash\npython -c 'import torch;print(torch.__version__);print(torch.version.cuda)'\n```\n\nSelect the appropriate installation command depending on the type of system, CUDA version, PyTorch version, and MMCV version\n\n\n\n    \n    
\n        \n        \n        \n        \n        \n        \n        \n        \n    
\n    
\n\n\n\n\nIf you do not find a corresponding version in the dropdown box above, you probably do not have a pre-built package corresponding to the PyTorch or CUDA or mmcv version, at which point you can [build mmcv from source](build.md).\n\n:::{note}\nmmcv is only compiled on PyTorch 1.x.0 because the compatibility\nusually holds between 1.x.0 and 1.x.1. If your PyTorch version is 1.x.1, you\ncan install mmcv compiled with PyTorch 1.x.0 and it usually works well.\nFor example, if your PyTorch version is 1.8.1, you can feel free to choose 1.8.x.\n:::\n\n:::{note}\nIf you would like to use `opencv-python-headless` instead of `opencv-python`,\ne.g., in a minimum container environment or servers without GUI,\nyou can first install it before installing MMCV to skip the installation of `opencv-python`.\n\nAlternatively, if it takes too long to install a dependency library, you can specify the pypi source\n\n```bash\nmim install mmcv -i https://pypi.tuna.tsinghua.edu.cn/simple\n```\n\n:::\n\nYou can run [check_installation.py](https://github.com/open-mmlab/mmcv/blob/main/.dev_scripts/check_installation.py) to check the installation of mmcv after running the installation commands.\n\n#### Using mmcv with Docker\n\nBuild with local repository\n\n```bash\ngit clone https://github.com/open-mmlab/mmcv.git && cd mmcv\ndocker build -t mmcv -f docker/release/Dockerfile .\n```\n\nOr build with remote repository\n\n```bash\ndocker build -t mmcv https://github.com/open-mmlab/mmcv.git#main:docker/release\n```\n\nThe [Dockerfile](release/Dockerfile) installs latest released version of mmcv-full by default, but you can specify mmcv versions to install expected versions.\n\n```bash\ndocker image build -t mmcv -f docker/release/Dockerfile --build-arg MMCV=2.0.0 .\n```\n\nIf you also want to use other versions of PyTorch and CUDA, you can also pass them when building docker images.\n\nAn example to build an image with PyTorch 1.11 and CUDA 11.3.\n\n```bash\ndocker build -t mmcv -f docker/release/Dockerfile \\\n    --build-arg PYTORCH=1.11.0 \\\n    --build-arg CUDA=11.3 \\\n    --build-arg CUDNN=8 \\\n    --build-arg MMCV=2.0.0 .\n```\n\nMore available versions of PyTorch and CUDA can be found at [dockerhub/pytorch](https://hub.docker.com/r/pytorch/pytorch/tags).\n\n### Install mmcv-lite\n\nIf you need to use PyTorch-related modules, make sure PyTorch has been successfully installed in your environment by referring to the [PyTorch official installation guide](https://github.com/pytorch/pytorch#installation).\n\n```python\npip install mmcv-lite\n```"
  },
  {
    "path": "docs/zh-cn/get_started/Introduction.md",
    "content": "## Introduction\n\nMMCV is a foundational library for computer vision research and provides the following functionalities.\n\n- [Image/Video processing](../understand_mmcv/data_process.md)\n- [Image and annotation visualization](../understand_mmcv/visualization.md)\n- [Image transformation](../understand_mmcv/data_transform.md)\n- [Various CNN architectures](../understand_mmcv/cnn.md)\n- [High-quality implementation of common CUDA ops](../understand_mmcv/ops.md)\n\nIt supports the following systems:\n\n- Linux\n- Windows\n- macOS\n\nIt supports many research projects as below:\n\n- [MMClassification](https://github.com/open-mmlab/mmclassification): OpenMMLab image classification toolbox and benchmark.\n- [MMDetection](https://github.com/open-mmlab/mmdetection): OpenMMLab detection toolbox and benchmark.\n- [MMDetection3D](https://github.com/open-mmlab/mmdetection3d): OpenMMLab's next-generation platform for general 3D object detection.\n- [MMRotate](https://github.com/open-mmlab/mmrotate): OpenMMLab rotated object detection toolbox and benchmark.\n- [MMYOLO](https://github.com/open-mmlab/mmyolo): OpenMMLab YOLO series toolbox and benchmark.\n- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation): OpenMMLab semantic segmentation toolbox and benchmark.\n- [MMOCR](https://github.com/open-mmlab/mmocr): OpenMMLab text detection, recognition, and understanding toolbox.\n- [MMPose](https://github.com/open-mmlab/mmpose): OpenMMLab pose estimation toolbox and benchmark.\n- [MMHuman3D](https://github.com/open-mmlab/mmhuman3d): OpenMMLab 3D human parametric model toolbox and benchmark.\n- [MMSelfSup](https://github.com/open-mmlab/mmselfsup): OpenMMLab self-supervised learning toolbox and benchmark.\n- [MMRazor](https://github.com/open-mmlab/mmrazor): OpenMMLab model compression toolbox and benchmark.\n- [MMFewShot](https://github.com/open-mmlab/mmfewshot): OpenMMLab fewshot learning toolbox and benchmark.\n- [MMAction2](https://github.com/open-mmlab/mmaction2): OpenMMLab's next-generation action understanding toolbox and benchmark.\n- [MMTracking](https://github.com/open-mmlab/mmtracking): OpenMMLab video perception toolbox and benchmark.\n- [MMFlow](https://github.com/open-mmlab/mmflow): OpenMMLab optical flow toolbox and benchmark.\n- [MMEditing](https://github.com/open-mmlab/mmediting): OpenMMLab image and video editing toolbox.\n- [MMGeneration](https://github.com/open-mmlab/mmgeneration): OpenMMLab image and video generative models toolbox.\n- [MMDeploy](https://github.com/open-mmlab/mmdeploy): OpenMMLab model deployment framework."
  },
  {
    "path": "docs/zh-cn/index.rst",
    "content": "Welcome to PanCollection's documentation!\n================================\n\nYou can switch between Chinese and English documents in the lower-left corner of the layout.\n\n.. toctree::\n   :maxdepth: 2\n   :caption: Get Started\n\n   get_started/introduction.md\n   get_started/installation.md\n\n.. toctree::\n   :maxdepth: 2\n   :caption: PanCollection\n\n   PanCollection/PreProcess.md\n   PanCollection/Evaluation.md\n\n.. toctree::\n   :caption: Switch Language\n\n   switch_language.md\n\n.. toctree::\n   :maxdepth: 2\n   :caption: Related Toolbox\n\n   related.md\n\n.. toctree::\n\n   faq.md\n\nIndices and tables\n==================\n\n* :ref:`genindex`\n* :ref:`search`"
  },
  {
    "path": "docs/zh-cn/make.bat",
    "content": "@ECHO OFF\n\npushd %~dp0\n\nREM Command file for Sphinx documentation\n\nif \"%SPHINXBUILD%\" == \"\" (\n\tset SPHINXBUILD=sphinx-build\n)\nset SOURCEDIR=.\nset BUILDDIR=_build\n\nif \"%1\" == \"\" goto help\n\n%SPHINXBUILD% >NUL 2>NUL\nif errorlevel 9009 (\n\techo.\n\techo.The 'sphinx-build' command was not found. Make sure you have Sphinx\n\techo.installed, then set the SPHINXBUILD environment variable to point\n\techo.to the full path of the 'sphinx-build' executable. Alternatively you\n\techo.may add the Sphinx directory to PATH.\n\techo.\n\techo.If you don't have Sphinx installed, grab it from\n\techo.http://sphinx-doc.org/\n\texit /b 1\n)\n\n%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\ngoto end\n\n:help\n%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%\n\n:end\npopd"
  },
  {
    "path": "docs/zh-cn/related.md",
    "content": ""
  },
  {
    "path": "docs/zh-cn/switch_language.md",
    "content": "## English\n\n## 简体中文"
  }
]